I don't believe it's a fad, but it will take time and companies/institutes who pursue it need to exploit the best technologies possible, i.e. active range-sensing/LIDAR rather than just passive (stereo) methods, while also furthering the computer vision techniques needed. It's tempting for companies like Tesla to decide that LIDAR is too expensive and that they can get by with software and a pair of cameras, but the laser impinging on the scene obviates difficult/slow correspondence problems that are onerous for computers to solve.
Meanwhile, deep learning has to emphasize object recognition based on 3D point clouds (PointNet-style) much more as opposed to just 2D image processing, to which the bulk of research on DL applies currently.
Finally, I also think that these companies and institutes need to accelerate testing of their autonomous navigation algorithms based on traffic simulation (SUMO-style), which makes it possible to validate autonomous nav algos faster than real time as well as on a massively parallel scale.
No, whilst it may take a few decades more to mature before they even start to be more prevailent, once they do, it will be an exponential rise from there in my opinion. I think we will first see them in places inner city driving with companies like Uber / Lyft using them to reduce costs and increase efficiencies. Maybe even previous Uber drivers will buy one and lease them to such companies to continue making a massive income like they did in the past, the only difference being they no longer have to sit in the car all day long to do that. Win win. Once this becomes more accepted other areas will start to emerge too such as long distance trucking which once again is way more efficient to do this with automation. Someone like Tesla is primed to take over this market and is already researching into it from what I have seen in their public announcements. This will all lead to a generation who grew up with self driving cars who will laugh at the fact humans had to manually do things in the past to get from A to B. Much like how kids these days dont understand how I grew up without a smartphone to ask a question and get an answer immediately from Google Assistant. This may take a generation or two to mature, but when it does, life as we know it will be changed and it will be here to stay.