Nighttime Driving and AI Autonomous Cars

By Dr. Lance Eliot, AI Trends Insider

Are you a nighttime driver?

Some people like to drive at night and enjoy pretending they are in their Batmobile and seeking to fight crime.

There are some people that reluctantly drive at night — they drive at night because they need to get home from work or carry out other errands, but otherwise believe that nighttime driving has inherent hazards.

There are those that entirely avoid driving at night because they either have difficulty driving at night, or they know that the stats about night driving are scary, or because they believe that vampires come out at night.

Speaking of stats, according to various governmental reports, it is said that there is about 60% less traffic at night but 40% of all fatal car accidents happen at nighttime, furthermore the fatality rate for car occupants is about three times higher at night than during the daytime, and the fatal crash rate for 16-year-old drivers is twice what it is at night versus during the day (a good reason to keep your teenager from driving at night!).

What makes nighttime driving so terrible?

One aspect is the impact on visibility.

Human drivers depend almost entirely on their vision to undertake the driving task. Without sufficient lighting being available, vision can be impaired. I’m sure you’ve driven on country roads that had no street lighting, and likely suddenly realized how much light is added to the streets when there are street lamps available. Even when there are street lights, the type of lighting varies, the number of street lights in a given stretch of road varies, and there can be roadway obstructions that block the lighting too. You’ve got shadows, you’ve got patches of area that have no direct street lighting, you’ve got sometimes other bright lights from billboards that maybe add light but can create dancing light and alter colors of lighting, and so on.

I suppose if it was only the lighting that made a difference, perhaps we could all accommodate it as a nighttime issue.

But, as you know, there is also driver behavior that comes to play.

Drunk driving is much more likely at nighttime than daytime. This means that a driver is likely not able to properly navigate the roads and can either get into an accident or cause someone else to get into an accident. There are also drowsy drivers at nighttime. These drivers are maybe worn out from a long day’s work, or maybe they need to get up at 3 a.m. to drive to work in the early morning darkness. Being drowsy causes them to be less aware of the traffic, more likely to swerve or take adverse actions, and be more prone to either getting into an accident or causing one.

Additional Difficulties Of Nighttime Driving

I’ll punch things up with even more factors about nighttime driving.

Since there tends to be less overall traffic at night, it also tends to allow drivers to go faster and perhaps cut corners more so.

In addition, I know some drivers that believe they are less likely to get a traffic ticket at nighttime, apparently they believe they are less likely to be caught under the cloak of darkness, so they tell me that they aren’t as worried about being full legal in their driving at night.

Kind of makes you scared of driving at nighttime with that kind of logic among some drivers.

Oddly enough, it seems like pedestrians and bike riders also become riskier at night.

They are already at risk because they cannot be as well seen as they typically can in daylight. You’ve probably encountered high-risk pedestrians that are willing to jaywalk and dart across the street at nighttime, doing so in front of oncoming headlights, for which those same jaywalkers during the day might not have taken the same chances. Not sure if they do this because they misjudge the cars in the darkness, or whether they somehow believe that the darkness is protecting them from potential harm.

Bike riders are also often crazy at nighttime and will either have no lights on their bikes, or if they do they suddenly think that a tiny bike light can generate a beam as bright as the headlights of a car.

AI Autonomous Cars And Nighttime Driving

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI for self-driving cars and undertaking special attention to the nighttime driving capabilities.

There are some automakers and tech firms developing self-driving cars that consider nighttime driving to be nothing more than daytime driving. In other words, they make no special distinction in their AI system as to whether it is daytime driving or nighttime driving. To them, if the AI can handle daytime driving it can ergo handle nighttime driving.

Driving is driving, they say.

We respectfully disagree with that somewhat gross oversimplification of the matter.

Our viewpoint is that nighttime driving requires added capabilities.

That being said, there are some that consider these added nighttime capabilities as though they are a so-called “edge” problem and will concede that yes, there are specialties involved, but it’s just an edge matter. An edge problem is one that is not at the core of a problem. Edges are considered something that you get to when you get to it, but otherwise it is not the mainstay of what is needing to be solved. We eschew this notion and instead assert that a good daytime driving AI is not necessarily a good nighttime driving AI. Furthermore, we would even argue that it is unlikely that a good daytime driving AI is a good nighttime driving AI, all other things being equal and if the AI has not been tweaked (as it were) for proper nighttime driving.

In this discussion, let’s first establish that there are true AI self-driving cars to be sought, which are considered at a Level 5. The Level 5 self-driving car is one that can be driven without any human intervention needed. The self-driving car is driven by the AI. The AI can drive the car in the same manner that a human can. For self-driving cars less than a Level 5, those such self-driving cars are dependent upon having a human driver in the self-driving car. Indeed, the human driver in those less than a Level 5 self-driving cars is ultimately considered the true driver of the car. In some sense, you could say there is a co-shared driving task responsibility in those self-driving cars less than a Level 5.

See my article about the levels of AI self-driving cars: https://aitrends.com/selfdrivingcars/richter-scale-levels-self-driving-cars/

For a self-driving car less than a Level 5, it’s kind of a ticking time bomb about the issues associated with nighttime driving.

If the AI of the self-driving car has difficulties driving the car, and it occurs at nighttime, there is a danger that if the AI suddenly hands control back to the human driver that’s in the car, the human driver might be caught unawares. And, the human driver might have limited visibility due to the darkness, and therefore be impeded in terms of taking corrective action or even being aware of what has caused the AI to toss the football to the human driver. It could be that the nighttime has caused the AI system to become perplexed and so it has handed control back to the human driver, or it could be something not having to do at all with it being nighttime, but for which the human driver is now having to contend with the matter and simultaneously deal with the nighttime aspects too.

See my article about human back-up drivers and AI self-driving cars: https://aitrends.com/selfdrivingcars/human-back-up-drivers-for-ai-self-driving-cars/

For a Level 5 self-driving car, in theory it is assumed that the AI will be able to handle the nighttime aspects of driving. Let’s explore ways in which the AI can be impacted by the nighttime driving task, and also how the AI should be enhanced or made to handle the nighttime requirements.

First, perhaps the most obvious aspect is the detriment to visibility.

The cameras on the self-driving car will likely not be able to catch as sharp of images and the video will also be less illuminated than were it daytime. The headlights of the self-driving car become particularly vital. If the headlights are in poor shape or occluded due to dust or dirt, it could have an even worse impact on the image processing. The cameras at the rear of the self-driving car are without any direct lighting such as headlights, and so those are obviously severely degraded in terms of the images captured.

The AI self-driving car should be checking to make sure that the headlights seem to be working sufficiently to capture apt images, as best possible, which can be tested beforehand in terms of doing testing at the start of a journey. The AI can then alert someone, such as a human occupant, and might allow the human occupant to possibly clean-off the headlights or maybe even decide not to start the journey. As a side note, years ago I went to rent a car, drove the rental car off the lot during daytime, and then when it reached night time I turned on the headlights – found out to my shock and surprise that the headlamps were burnt out. If I had checked during the daytime, I likely would have switched to another car, or only used the car during daylight, or taken some other action. Keep this in mind the next time you rent a car!

The image processing of nighttime images is different than the image processing for daytime images.

Images that are well lit can be more readily processed. Images that have dark shadows and other darkened portions is harder to process. This often involves doing various image processing clean-up. It can also change the probabilities associated with object detection. An object that during daytime had say a 100% probability of being a wheelchair at the curb might at nighttime have a 60% chance of being a wheelchair.

There is also typically much more light reflections and distortions.

Street lights and the headlights can bounce off the car ahead of you, perhaps doing so off the chrome bumper that acts like a kind of mirror, and thus other objects either become more well lit or become overly lit. The bouncing light can actually cause a sunburst effect and make the object recognition harder. Again, the image processing software has to be ready for this impact.

Machine Learning Aspects

For machine learning and the use of artificial neural networks, if those neural networks were trained with pictures of those objects during the daytime, it could be that those objects are no longer detected by the neural network. It’s important the training of the neural network include objects as seen in darker conditions. Some prefer to do both daytime image and nighttime image training together, as a smorgasbord, while others believe that the neural network will be better off if one instance is trained to daytime images and the other to nighttime images.

One aspect that visually can become more problematic is the detection and interpretation of street signs.

During daylight, it is usually relatively easy to spot a street sign, such as a stop sign or a caution sign. At nighttime, it can be much harder to spot the street sign. The darkness might obscure it, or the brightness of the headlights might create a sunburst effect. As mentioned, if a machine learning approach was used to train on street sign detection, it is important to be mindful of whether it can handle nighttime versions.

See my article about road sign spotting and interpretation by AI self-driving cars: https://aitrends.com/selfdrivingcars/making-ai-sense-of-road-signs/

Many self-driving cars use various relatively simple road-following techniques when the AI is driving the car.

For example, the roadway markings are examined and used to decide where the boundaries of a lane are. Some roads use flat paint to mark the lanes, while some use reflective Botts’ dots. Nighttime driving can be tricky in trying to figure out where the lane markings are. It can also be easy to be fooled by the road markings in terms of falsely believing that they are correct when in fact they might be something left over from prior markings that have since been changed. The AI system portions dealing with this need to be prepared for nighttime driving.

See my article about the use of lane following and AI self-driving cars: https://aitrends.com/selfdrivingcars/pied-piper-approach-car-following-self-driving-cars/

So far, I’ve focused mainly on the sensory aspects of nighttime driving. Consider that there are five key stages involved in AI self-driving cars during the driving task:

Sensors Data collection and interpretation
Sensor Fusion
Virtual World Model updating
AI Action Plan preparation
Car Controls Command issuance

See my article about the overall framework for AI self-driving cars: https://aitrends.com/selfdrivingcars/framework-ai-self-driving-driverless-cars-big-picture/

Sensor Fusion Aspects And More

During sensor fusion, the sensor results are compared and combined in various ways.

If the cameras are not getting good images, it is usually the case that the AI system is devised to give less weight to those sensors. Thus, if the camera is suggesting that there doesn’t seem to be a pedestrian at the side of the road (perhaps the image is so darkened that there is no ready way to detect a figure standing in the shadows), but the radar is detecting that something might be there, the sensor fusion has to determine what to do. Should it rely upon the radar and claim that there is a pedestrian there? During daylight, often times the sensor fusion opts to wait until the cameras are able to agree with the radar results. This though can be dicey depending upon how much time is available before a decision about the matter needs to be made.

Overall, at nighttime, the cameras are likely to be much less reliable and thus the sensor fusion should be designed to cope with this aspect.

It’s almost the same as a human driver. If a human driver cannot readily see at night, they are at heightened risk of properly performing the driving task. Purists for AI self-driving cars point out that the aspect of multiple sensory devices on an AI self-driving car presumably implies that therefore the AI self-driving car will be better able to do nighttime driving than a human can (most humans don’t have radar or LIDAR built into their bodies). This is a somewhat arguable point in that yes the self-driving car has other sensory devices that a human does not have, but it then belies the aspect that humans are (for now) generally better at the “intelligence” aspects of driving than the AI systems of this era are so far.

The virtual world model that is used by the AI must also be updated and be apprised of the nighttime aspects. This can include that the associated probabilities with detected objects are going to be lower than the normal daytime driving, as mentioned about whether a wheelchair is at the curb or not. And, the AI action plans need to take into account the now somewhat degraded virtual world model, along with what kinds of actions can be taken in nighttime situations.

Suppose the AI decides that a hard braking action is needed.

At nighttime, the driver behind you might not be as ready to stop, versus during daytime they might have more readily been aware of you ahead of them. Too, keep in mind that the human drivers on the roads at nighttime are potentially drunken or drowsy, or cannot see the road and traffic as well as they could during daytime. The AI needs to consider how other drivers will be acting and reacting at nighttime. This might also dictate that the AI needs to be going slower or taking other defensive maneuvers that would not necessarily be needed during daylight hours or used as frequently.

See my article about defensive driving for AI self-driving cars: https://aitrends.com/selfdrivingcars/art-defensive-driving-key-self-driving-car-success/

Additional Nighttime Driving Concerns

I’m sure that some proponents for AI self-driving cars will instantly say that there shouldn’t be any human drivers on the roads, and that we need to get to the point of having only AI self-driving cars out there. Thus, there would be no need for the AI to worry about human drivers at nighttime. No more drunken human drivers that leave the bar after midnight and then crash into other cars. No more drowsy human drivers that wake-up before sunrise and are so sleepy when they drive that they ram into a pedestrian.

Instead, nighttime driving will be a breeze.

The AI self-driving car would communicate via V2V (vehicle to vehicle communications) with other self-driving cars, and they would all help each other in nighttime driving conditions. The AI self-driving car ahead of you might have figured out that there’s a pedestrian standing at the crosswalk, which perhaps your AI self-driving car cannot quite determine due to the darkness and angle, and shared this aspect with your AI of your self-driving car. Cross sharing would aid the AI’s in doing enhanced nighttime driving on a collective and collaborative basis.

Sure, this might well happen, but it’s going to be far off in the future. There are about 250 million conventional cars in the United States today. Those are not going to magically become AI self-driving cars right away. For many decades to come, we will have a mix of human driven cars and AI self-driving cars. You cannot be designing AI self-driving cars for only the day in which we have exclusively AI self-driving cars, since we are putting AI self-driving cars on our streets today. The streets today and for the foreseeable future will have human driven cars. That’s a fact.

This brings up another aspect about AI self-driving cars that is worthy of important consideration about nighttime driving.

There are predictions that we will likely be using AI self-driving cars on a round-the-clock basis. And, it makes sense to do so. You have a car that will go wherever you want it to, with a built-in driver that doesn’t need to rest. If you own an AI self-driving car, you are likely to turn it into a revenue generating ride sharing vehicle. Most AI self-driving cars will probably be driving around on a 24×7 basis.

See my article about the non-stop aspects of AI self-driving cars: https://aitrends.com/selfdrivingcars/non-stop-ai-self-driving-cars-truths-and-consequences/

How does this tie into nighttime driving?

It means that as AI self-driving cars become more popular and prevalent, there will be more and more of them on our roadways at nighttime. This means a higher likelihood of getting entangled with a drunken human driver or a drowsy human driver. It would seem a reasonable bet that the first rash of AI self-driving car accidents will most likely arise at nighttime, once AI self-driving cars become prevalent.

What might hopefully aid in offsetting the encounters with human drunk drivers will be that those humans that were going to drive drunk will instead use a ride sharing service, of which, maybe there will be an abundance due to the prevalence of AI self-driving cars.

This does though bring up the aspect that some ask me about at conferences, namely, how will an AI self-driving car handle a drunken occupant? If you have a human driver for say Uber or Lyft, the human driver sometimes will talk with the human drunken passenger or even physically aid them to get into and out of the vehicle. The AI is not going to be able to do the same, for now. I’ve mentioned that most AI self-driving cars will be outfitted with cameras pointing into the car, and audio capabilities, such that the human owner of the AI self-driving car can at least see and talk with the human occupants.

There are more twists and turns about nighttime driving for AI self-driving cars.

Imagine that it is nighttime and there are heavy rains coming down. When you combine the weather with the nighttime aspects, it can be a potent danger for any driver, human or AI. Another twist is roadwork. Generally, most of the road work that we have done on our infrastructure is done at nighttime, which is done to reduce the adverse impact to daytime traffic. For the AI to contend with roadwork can be difficult, and especially so at nighttime. Trying to figure out that the street is coned off and you can’t make a right turn up ahead, it’s not easy to do at nighttime.

Conclusion

As they say, do not go gentle into that good night, but instead rage, rage against the dying of the light.

We need to have AI systems for self-driving cars that realize there is a difference between daylight driving and nighttime driving.

Treating daylight as the same as nighttime, or assuming that nighttime driving is equivalent to daytime, it’s a dangerous trap that could cause a grave and overbearing darkness to descend upon the future of AI self-driving cars.

This content is originally posted on AI Trends.

[Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column: https://forbes.com/sites/lanceeliot/]