By Dr. Lance Eliot, the AI Trends Insider I lived in Germany for a year and “fondly” recall one of the most terrifying driving experiences in my life, which involved being on an autobahn in the middle of a snow storm. When I left my apartment in my rented car to head out to work […] 
By Dr. Lance Eliot, the AI Trends Insider
I lived in Germany for a year and “fondly” recall one of the most terrifying driving experiences in my life, which involved being on an autobahn in the middle of a snow storm.
When I left my apartment in my rented car to head out to work one day, the snow was heavy on the ground but relatively well-plowed and the roads were open. I knew to be generally cautious in driving in the snow, but didn’t anticipate any special situations or predicaments that I’d need to be worried about. The skies looked ugly with the potential for some lite snow showers. This was pretty much a typical winter looking day in Frankfurt. It was certainly a drivable day and the locals all acted like the weather was not a concern at all.
After driving onto the autobahn, snow started to come down from the skies. At first, it was those cute little flakes of fluffy snow that are fun to see and that melt right away. Then, the snow started to come down like it was angry and wanted to blanket the earth in a dense and thick layer of snow. I figured that maybe if I drove fast enough, I could make it to my exit and get to work before things became really dicey. The windshield wipers on the car could not keep up with the snow and gradually I couldn’t see out the windshield. I rolled down the driver side window and tried to stick my head out, hoping to see the road ahead of me.
Meanwhile, there were other determined drivers on the autobahn that were trying to pass me. Other cars were going very slowly. The snow on the road itself was piling up. A car near me began to slide and I just barely avoided getting hit. My face was freezing as I continued to use a strained neck and my head out the window to see where I was going. The whole situation had rapidly deteriorated and I was unsure of what to do. A few cars were pulling over to the side of the autobahn and parking their cars. This seemed quite dangerous as they were still on the autobahn and now parked in the so-called slow lane.
Believe it or not, the snow became so heavy and had fallen so fast that some cars were now stopped in the middle of the autobahn. I tried to drive around them. My car though was now becoming impossible to drive. I had only regular tires on the car and had no chains or snow tires. The windshield was completely obscured by iced-up snow. Trying to get out of my lane was questionable and other than going straight ahead, I probably would have spun the car if I tried to now head toward the side of the autobahn. The nearest exit was a few miles up ahead, and might as well have been a thousand miles away. Yikes!
I decided to stop where I was, and sit there, in the middle of the autobahn. Some drivers nearby had gotten out of their cars to survey the situation. I decided that I might as well get out of my car too, and thought that if some idiot driver came along and rammed my now snow-bound car, I’d be better off hiding over near the autobahn railings. I had hopes that I might be able to somehow magically drive out of there, and even got out of the trunk my windshield scraper and used it to try and get some of the snow and ice off my windshield. The situation was quite beguiling and one of the worst snow driving moments I’ve ever had.
What does this have to do with AI self-driving cars?
At the Cybernetic Self-Driving Car Institute, we are working on developing AI that can drive while in the snow.
This is a very hard problem.
Indeed, for most of the auto makers and tech firms that are developing self-driving cars, the capability of driving in the snow is a stretch goal that everyone knows is going to be really hard to achieve.
Let me clarify though that there are situations of driving in the snow that are easy and there are situations that are hard. I mention this aspect because there are daily exclamations of one self-driving car vendor or another that says they have solved driving in the snow. I’d suggest you give some scrutiny to those claims.
It could be that the snow driving consisted of a nicely plowed road that was a straight away and that the AI had been conveniently provided with detailed 3D maps indicating the roads and surrounding aspects. And, there wasn’t any snow actually coming down from the sky. And, the self-driving car happened to have snow tires. And so on. In other words, things are kind of rigged-up to be able to make life really easy for the self-driving car. For me, this is not true snow driving. This is a constrained and somewhat contrived version of snow driving.
That being said, I am certainly an advocate of walking before we run. Incremental improvements in driving in the snow are welcomed. All I’m saying is that before you believe someone that says their self-driving car drives in the snow and that this hard problem is solved, find out what kind of arrangements were made to aid the AI and the self-driving car. To achieve true snow driving, I assert that the AI and the self-driving car need to be able to drive in the same kinds of circumstances that human drivers face, including roads they don’t know, snow that is falling from the sky, roads that are half-plowed, and so on.
At the same time, I’ll emphasize that I am not expecting a self-driving car to be able to drive in places and ways that humans cannot. This is an important point. Some pundits seem to suggest that the AI of a self-driving car is going to be so grandiose that it will be able to drive a car in ways and places that humans cannot do so in the snow. No matter how good the AI is, you cannot get around the physics of snow. A car, whether conventional or the best of self-driving cars, cannot drive in snow if the tires can’t get traction and there’s three feet of snow all around the car. No AI can overturn that.
I would say that once we get really good AI, the ability of the AI to drive a self-driving car in the snow – in comparison to humans that aren’t good at snow driving – will be a nice positive about AI self-driving cars. There are human drivers that aren’t used to snow driving and they are dangerous when they find themselves in the snow. Here in Southern California, during the winter we all drive a couple of hours up into the local mountains to go see the snow, go skiing, and frolic in the stuff. Many of the drivers only do this maybe once or twice a year. They have no idea how to drive in snow.
For those of you that drive in snow all winter long, you might get a good laugh if you came and watched as the locals here struggle to drive in the snow while in the local mountains. Tons of cars end-up sliding off the mountain roads and end-up in banks of snow. In the small towns that up in the mountains, drivers can be seen spinning their tires and spinning their cars. Cars get frozen in place. Drivers don’t have chains for their cars. It’s all an annual spectacle and ritual as sunny SoCal drivers try to contend with the harshness of nature and the dynamics of snow driving.
Let’s consider what makes snow driving particularly hard for AI self-driving cars.
SNOW SENSORY DEPRIVATION
Snow can potentially cover the sensors that are crucial to the navigation and situational awareness of the self-driving car. Similar to my story about the snow covering my windshield when I was driving on the autobahn, snow can easily cover-up the cameras that are on the self-driving car. This can blind the self-driving car. No vision means driving is dicey. There are other sensors such as LIDAR, radar, and ultrasonic that can help to compensate for snow-covered cameras, but those can also be impacted by snow and ice on the car.
Some vendors are starting to provide add-ons to the sensors of self-driving cars that seek to melt snow and ice that’s obscuring a lenses, or they provide mini-windshield wipers, or chemical sprays to get rid of the snow and ice. We’ll likely see these types of technologies come into the marketplace once we have a prevalence of self-driving cars on our roadways.
SNOW SENSORY OBSFUCATION
Besides snow that’s actually sitting on the car, there’s also snow that falls from the skies. The falling snow can play tricks with the sensors. Active sensors that send out signals can have those signals blocked by the snow, or worse still tricked by the snow into believing that something is there that maybe isn’t really there. Passive sensors that receive aspects such as cameras that take pictures will find the pictures to be cluttered with images of snow particles.
The sensor processing needs to be able to figure out how to deal with data that has been distorted or obscured by the falling snow. This involves often dealing with uncertainties and probabilities. Maybe that partially obscured image is a pedestrian stepping into the roadway. Or, maybe it’s just a statue at the side of the road that appears to look like a pedestrian when the sensor can only sense a fraction of what’s actually there. Machine learning techniques and improvements in image analysis are helping to improve on dealing with snow sensory obfuscation.
SNOW DETECTION
Is that an inch of snow on the road, or three inches, or a foot? Is that a snow bank over to the right? Is the road ahead passable or does the snow get deeper up ahead? A significant aspect to snow driving is being able to figure out where the snow is, how much of it there is, whether it is passable or not, and so on.
This is not easy to figure out via remote means. Even humans have a hard time figuring this out. But, it is crucial that there be sensors that can help to detect the snow. The AI of the self-driving car needs to get an accurate indication of what the snow conditions are. Without this, the chances of getting the car stuck in the snow or getting into a snow-related car accident is heightened.
COLD TEMPERATURES IMPACT TECH
Some of the sensory devices are vulnerable to the cold. Those devices might not work reliably in really cold weather. This could mean that the sensors are intermittently working, which could falsely lead the self-driving car to get into a driving situation that might become quite dangerous. It’s similar to the day I drove onto the autobahn and step-by-step got myself into an increasingly bad predicament.
Besides the sensors, the cold could also impact the on-board computer processors and memory of the self-driving car. Once again this could cause intermittent operations or could otherwise reduce the reliability of what the AI is doing.
GLARE FROM SNOW AND ICE
The cameras are especially vulnerable to capturing images that are perhaps filled with glare and reflections due to the snow and ice. This requires some special image processing and the AI needs to decide the validity of what the sensors think is out there.
ROAD SIGNS AND STRUCTURES OBSCURED BY SNOW
I’m sure we’ve all driven on roads that had a fresh layer of snow and you could not read the road signs, and maybe not see the road markers. Where’s the side of the road? Am I about to drive off a cliff? Is there a sign that maybe is trying to warn me that a bridge is out up ahead? This makes things tough for the AI driving the self-driving car in the snow.
Some are suggesting that we just need to have really good 3D maps and when combined with GPS, the AI can figure out where it is. This presumably suggests that obscured road signs and structures won’t be a problem. I’d say this is rather optimistic thinking. I certainly agree that having the detailed 3D maps will help, and it allows then for the AI to piece together clues such as a partially viewable road sign and the top of a road marker off to the left. But, this also assumes that the detailed 3D maps exist, and that they are current and the road hasn’t changed recently, and that the GPS is working properly and precisely, etc. A lot of important assumptions.
SELF-DRIVING CAR IS STILL A CAR
Suppose the cold weather prevents the battery of the car from working correctly. Suppose the ignition system won’t start the car because of the cold and ice. Suppose the brakes aren’t up-to-par and so the car can’t brake well in the snowy roads. Suppose the tires are conventional tires and they will slide in the snow and ice. In other words, a self-driving car is still a car.
As mentioned, the AI cannot overcome the physics of snow and ice. Somehow, the self-driving car, as a car, needs to be ready to drive in the snow and ice. Generally, for now, we should expect that the human needs to make sure that the self-driving car is in proper physical condition to drive in the snow and ice. I suppose we might eventually have more sensors on the self-driving car that it can detect whether the tires have the needed tread, and whether the brakes are in proper shape, etc.
KNOWING WHERE AND WHAT OF THE SNOW
Your AI self-driving car is parked in the parking lot of the ski resort that you’ve been at for three days. You are ready to leave after a wonderful skiing vacation. You go out to your AI self-driving car. You get into it and instruct it to take you home. Turns out there’s a few inches of snow on the ground that completely surrounds the vehicle. A human would likely notice this, and might try to clear a path by slowing driving the car forward a few feet, and then backward a few feet, and do this until a path is created.
For the AI to do this, it must first realize there is snow on the ground and the amount and where it is. In addition, it needs to devise a method to drive out of the parking lot. Besides trying snow driving tricks like the one I just mentioned, it might reach a point that it cannot move and needs the human to go out and shovel snow out of the way. This then requires interacting with the human and explaining the situation.
Indeed, this points out that the interaction between the AI and the human occupants can be crucial in snow-related conditions. Suppose the AI has looked up the weather forecast and detected that heavy snow is forecasted to fall in an hour, likely occurring while it is driving down an already dangerous mountainous road. Seems like it would be prudent to alert the human occupants and see what they want to do. Do they really want to try having the AI drive out, or do they want to stay where they are?
TIRE TRACTION AND DRIVING THE CAR
Let’s talk about the coefficient of friction when driving. On a dry asphalt road, you’ll likely get a nearly 1.0 coefficient of friction from a good tire. On a rainy slick road, it’s maybe 0.7. On snow, it drops to a paltry 0.15, and with ice it goes to a downright scary 0.08.
The AI needs to know the nature of the tires on the self-driving car. It might even advise the human occupants to put on snow chains for the tires. The AI and the human might need to work together to get the self-driving car in a ready-shape for driving in the snow.
This raises an ethical issue too. If the AI self-driving car believes that it is too dangerous to drive the car due to the roadway conditions, should it refuse to do so? Even if the humans insist they want it to drive? You might say that of course the AI should do whatever the humans command it to do. But suppose they are drunk? You might then say that of course the AI should not drive the car if it has calculated that the risks are too high. But suppose the human is near death and needs to urgently get to a hospital and is willing to take a chance on driving there in the lousy weather conditions?
SNOW AND ICE DRIVING TECHNIQUES
The AI needs to know the snow and ice driving techniques that any savvy human driver would know. For example, once underway, you try to keep in motion, if feasible, since coming to a stop and then trying to go in motion again is one of the most difficult aspects in such conditions. You also need to go slower than normal, maybe a lot less than whatever the speed limit is. The AI needs to moderate acceleration, avoiding hitting the gas that might make the tires spin.
Going up a hill requires the AI to do some careful driving, and likewise coming down a hill is also fraught with danger and requires careful driving. Making lane changes must be done with greater care, such as in my story about being on the autobahn where I realize that trying to steer into another lane might have spun my car.
The AI needs to decide which lane is best to be in. If the roads have been plowed, often the plows will only do one lane rather than say both lanes. The AI also needs to be scanning ahead and deciding which lane to be in for an upcoming left turn or right turn. It’s not sufficient to be just focusing on the driving at the moment, but also must be creating action plans of what is up ahead and what driving actions will be needed up ahead.
Have you ever helped to push someone’s car to get it out of a snow bank? I have, many times. You usually have a human driver that is at the controls, being very cautious since there are humans outside the car trying to shove the car to get it out of the snow. In this circumstance, the AI needs to be aware of the group effort of both the AI driving and the humans that are trying to help get the car underway. The humans might be putting sand or gravel under the tires. The AI could harm these helping humans if it suddenly opts to try and move the car.
The intricate and delicate dance between humans and the AI can make the snow driving situation very difficult to achieve. This is why I mentioned earlier that auto makers that claim they’ve solved the snow driving problem are maybe only considering a fraction of the total problem involved.
EMERGENCY ACTIONS
The AI of the self-driving car might be driving the car and all of a sudden the car starts to skid off the road. As such, the AI needs to determine what action to take in an emergency circumstance. There are rear-wheel skids which require one approach, and front-wheel skids that require a different approach.
It’s not so simple though, because suppose there are pedestrians standing near to the car. The action taken to try and correct the skid could end-up ramming into the pedestrians. The whole situation needs to be assessed and the course of action must take into account what the car’s situation is, what the road conditions are, what’s around the car, and so on.
ROAD CLOSURES AND ALTERNATIVE PATHS
The GPS and maps might say that the road ahead is the right way to go. Meanwhile, suppose the snow has come down so heavily that the authorities have put up a sign that says road closed ahead. The AI needs to deal with rerouting the car and possibly taking paths other than what it thought would be best to take.
Of course, once again we have the risk factors and the ethics questions. If the AI decides that taking a side road would seem prudent, should it let the human occupants know that it is taking the alternative route or should it just proceed? Should it change its path if the human occupants object? Maybe they know that the alternative path is worse and shouldn’t be taken. More of the ethical dilemmas involved in snow driving conditions.
CONTENDING WITH OTHER CARS ON THE ROAD
I had mentioned that while on the autobahn, I had some crazy human drivers that tried to speed past me, and I had others that slowed down, and some that stopped entirely. The AI needs to be watching out for other traffic during snow and ice conditions, even more so than with sunny weather, since other drivers can be more likely reckless and cause an accident during adverse weather.
Is that other car up ahead going too fast and maybe it will go into a skid, which means that the self-driving car might need to maneuver out of the way of the soon to be skidding car. The AI has to be not only detecting other cars, but also predict what those other cars might do. This gets added into the calculations being made by the AI about how to deal with the snowy conditions.
CONCLUSION
As I’ve indicated, driving in the snow is not so easy. Auto makers and tech firms are tackling this rather hard problem since they know that an AI self-driving car that cannot drive in the snow will be of very limited use. Not many people would be willing to spend the big bucks to buy an AI self-driving car and then have to park it for the winter.
Waymo has been using Chrysler Pacific hybrids in snowy Detroit to try and give their AI more time in snow driving conditions. VTT Technical Research Centre of Finland has produced a prototype AI system called Marti that is driving an Aurora E8 in snowy Muonio. Aptiv-owned nuTonomy is testing with Lyft in snowy Boston. Uber is testing in cold winter weather Pittsburgh. There are even Department of Transportation (DOT) proving grounds for AI self-driving cars that are located in wintery and snowy climates. Etc.
Don’t be misled by potentially over-the-top claims that the snow driving problem has been solved. It is a multi-faceted problem with lots of moving parts, and includes aspects that are “easy” and aspects that are definitely hard. Real-world snow driving is difficult and stymies even humans.
From an AI perspective, snow driving is a fascinating problem to be solved. If AI can contend with all of the variables involved in snow driving, which includes a lot of generalized intelligence aspects, it would help to illustrate further progress of AI overall.
Snow driving is also a very practical problem that will determine whether AI self-driving cars will have widespread adoption and success.
All due to a little bit of fluffy white snowflakes.
This content is originally posted on AI Trends.
You must be logged in to post a comment.