Proving Grounds for AI Self-Driving Cars
AI Research AI Trends Insider Artificial Intelligence autonomous cars Robotics Self Driving Cars
Proving Grounds for AI Self-Driving Cars
When I was in my teens, a good friend of mine offered to teach me how to ride a motorcycle. We started out by riding in his backyard, which offered seclusion and allowed me to make mistakes without anyone else knowing. His backyard was relatively small and so it precluded being able to do any […]
AI Research AI Trends Insider Artificial Intelligence autonomous cars Robotics Self Driving Cars

Proving Grounds for AI Self-Driving Cars

When I was in my teens, a good friend of mine offered to teach me how to ride a motorcycle. We started out by riding in his backyard, which offered seclusion and allowed me to make mistakes without anyone else knowing. His backyard was relatively small and so it precluded being able to do any substantive riding.

After I had learned how to do the basics on the motorcycle, we opted to go over to a nearby empty dirt lot and continue the lessons there. The abandoned lot was handy because I was ultimately aiming to go dirt biking with my friend in the desert and the nearby empty lot was mainly flat dirt with some small mounds here and there. It was a handy place to learn and practice, but it had the downside that we could be readily seen by any passerby’s. I am pretty sure we were also violating someone’s private property rights, though there weren’t any signs posted and the lot itself was not fenced off.

Here’s some salient points about my learning to ride a motorcycle in this manner:

  • I needed a place or places that I could start small and ultimately get ready for the real riding in the desert
  • I wanted to have some privacy while learning how to ride
  • The aspects of where I was learning to ride were somewhat similar to what I’d be doing in the desert and so it was handy to practice in a somewhat similar manner
  • I was on private property rather than on the public roads, which helped due to my not yet being legally able to ride on public roads
  • This initial foray was easier than being in a real desert location and it also was close to home in case I needed anything during my lessons

You might wonder what does this have to do with AI self-driving cars?

At the Cybernetic Self-Driving Car Institute, we are doing testing of AI self-driving cars and similar to the auto makers and tech firms that are doing likewise, it is advantageous to have someplace to try out these newly being formed experimental vehicles.

In essence, there is a move afoot to provide proving grounds for AI self-driving cars.  And, similar to my indications about learning to ride a motorcycle, there are some salient aspects about these proving grounds that the AI self-driving car developers are looking for.

First, it would be handy if a newly being tested self-driving car could be tested in a place offering a certain amount of privacy. This is key to many of the developers since they don’t want the general public to needlessly get worried if a self-driving car during testing hits an obstacle or otherwise gets into an accident. It’s similar to rocket makers that know that when they test fire a rocket that it might or might not work correctly, and if others see it falter or even blow-up they might think that the rockets are unduly dangerous – but, if it was only a test then the developers were already aware that such a result could occur. Better to catch such issues during testing, rather than once it is put into actual use.

The privacy aspects cover various elements when referring to self-driving cars. Can the self-driving car been seen by someone off-site of the proving grounds, and if so, would they be able to pry and catch a glimpse of the self-driving car during the testing process? Suppose too that a competitor wanted to spy on them, could this spying readily occur by simply watching from across the street? Those are the privacy aspects of being seen. Another aspect would be the privacy aspects of the results of the testing effort. Namely, is the location private to the degree that the results of the tests are known to just anyone, or only to those doing the tests.

Secondly, it would be handy if the providing grounds were on private property, rather than public lands, which then would allow for the testing to occur without having to abide necessarily by the public roadways laws and regulations. During testing, it might be the case that the AI self-driving car could inadvertently violate some legal requirement for driving. Getting into trouble on such an aspect while on a public road would be thorny since it was intended only as a test. Of course, being on public roads would have another even more serious issue, namely the potential for doing harm to others while on a publicly available roadway.

Thirdly, the proving grounds need to be sufficiently alike to whatever roadway conditions that the AI self-driving car is going to be expected to operate on during actual use. Suppose the proving grounds are in an always sunny environment, and yet the self-driving car is supposed to be able to operate in the rain and snow. It would be important to test the self-driving car in the same kinds of conditions as the intended operations of the self-driving car, otherwise the testing would be incomplete and insufficient.

Now, when you think about the roadway conditions, there are a lot of different kinds of road circumstances that a self-driving car should ultimately be able to handle. Thus, a particular proving ground is bound to only touch upon a subset of the myriad of conditions, since trying to replicate in the small all possible conditions is a rather tough situation to recreate.

Let’s take a look at some of those roadway aspects:

Location

  • Is it a large location or a small location?
  • Does it have small-scale tracks?
  • Does it have large-scale tracks?
  • Is it nearby or faraway from public roads?
  • Is it nearby or faraway from city/populous?
  • Does it allow for privacy while on the grounds?
  • Is their space for the facilities and other aspects?
  • Is it a secured area?
  • Is the layout conducive to testing?
  • Can the space/area be altered as needed?
  • Are there any needed permits for use?

Lighting

  • Does the proving ground allow for daylight time lighting tests?
  • Does the proving ground allow for variations of daylight such as brightly lit, dimly lit, etc.?
  • Does the proving ground allow for nighttime tests?
  • Does the proving ground allow for mixtures of both light and darkness?

Roadway Surfaces

  • Does the proving ground have conventional smooth asphalt streets for testing?
  • Does the proving ground have everyday cracked and potholed streets for testing?
  • Does the proving ground have dirt roads and unpaved conditions for testing?
  • Does the proving ground allow for slick roads or other surfaces?

Roadway Fixtures and Maneuvers

  • Are there straightaways?
  • Are there curves (and of different degrees)?
  • Are there turns (sharp, soft, etc.)?
  • Are there intersections?
  • Are there underpasses?
  • Are there overpasses?
  • Are there roundabouts?
  • Are there tunnels?
  • Are there mixtures of these (involving segue ways between them)?
  • Etc.

Weather Conditions

  • Can there be sunny weather?
  • Can there be rainy weather?
  • Can there be flooding weather?
  • Can there be foggy weather?
  • Can there be snowy weather?
  • Can there be icy weather?
  • Etc.

Roadway Signs

  • Are conventional roadway signs posted?
  • Can special roadway signs be posted?
  • Can the roadway signs be aged and obscured like in real-world conditions?
  • Can roadway signs be altered, shifted in size and angle, and so on?

Roadway Markings

  • Are there roadway markings?
  • Are there different kinds of roadway markings?
  • Can the roadway markings be aged and obscured like in real-world conditions?
  • Are there roadway barriers and side markers?
  • Etc.

Road Driving Obstructions

  • Can debris be placed into the paths on the roadways?
  • Can obstacles be placed into the paths?
  • Are there pedestrians (faked or stunts) able to be used?
  • Can bicyclists be used (faked or stunts)?
  • Can motorcyclists be used (faked or stunts)?
  • Can animals be used (faked or stunts)?
  • Etc.

Traffic Conditions

  • Can multiple cars be used during the testing?
  • Can a mix of human-driven and self-driving cars be used?
  • Can various driving scenarios be undertaken?
  • Can traffic conditions be undertaken (clogged, open, etc.)?
  • Etc.?

Facilities

  • Are there car repair/maintenance capabilities?
  • Is there storage for the self-driving cars?
  • Is there storage for needed parts and other equipment?
  • Are there conference rooms?
  • Is there a computer center or data center?
  • Are there hardware/software facilities and specialists?
  • Is there housing for visiting personnel?
  • Is the facility available 7×24 or are there limits of days/times?
  • Is there food available on-site?
  • Is there a visitor’s center?
  • Is there a security access/badging capability?
  • Is insurance available, required, provided?
  • Is there a fire department/capability?
  • Is there a first aid/hospital/ambulance capability?
  • Etc.

It is unlikely that any one particular proving ground could provide the complete range of all possible kinds of conditions and circumstances. As such, most of the self-driving car makers will be needing to use various of the proving grounds for specific aspects being tested. For example, a proving ground that is based in sunny Southern California is unlikely to provide for testing of snowy and icy driving conditions, and so a proving ground in a more inclement weather environment such as the Midwest or East Coast might be used for those tests.

This does bring up one potential difficulty of doing testing on proving grounds, namely that the proving ground is able to provide testing year-round and whether it can control the conditions at the proving ground. In other words, let’s suppose its winter in the Midwest and so assume that the proving grounds there have snow and ice, not by choice but because that’s what nature has done. It would mean that if the self-driving car maker wanted to test dry and sunny conditions, they wouldn’t be able to do so at that time of the year at that particular proving ground.

Some proving grounds are trying to setup an ability to control the conditions on the proving grounds, such as making artificial snow or ice when otherwise nature has not provided it. This can be quite expensive and not easy to logistically undertake.

The physical location of the proving ground is another factor to be considered. Besides the now obvious point that the location might be in a geography that allows for varying kinds of weather, or might not, there is also the aspect of the size of the location itself. Is it big enough to allow for large-scale testing involving miles of driving, or is it small and thus you can’t really do any large-scale driving tests? How many acres does it provide?

Another aspect involves the facilities at the proving ground. Are there sufficient facilities to keep your self-driving cars there for days or weeks at a time? Are there car repair capabilities to be able to maintain and change-up the self-driving cars hardware? Are there spare parts stocked there? Is there housing for the self-driving car mechanics, developers, and testers that would be needed at the facility? And so on.

In some cases, the location might be at some outpost far beyond normal cities, and this could be good since it provides privacy, but it could also be bad because it takes a lot of effort to get the self-driving cars and the teams out to the remote location. Also, in some instances, it might be handy to have not only the private land, but also be close to public areas that might also be used with the testing (if so appropriately arranged for).

For testing, the odds are that the auto maker or tech firm will want to video the tests and then replay the videos to closely study them. This is akin to a football team practicing and then watching the videos to improve on their game. Thus, the facility should have either already setup extensive video capabilities or provide the infrastructure for it.

Whatever testing is going to be done at the proving ground, there is a crucial need for safety measures and considerations. The facility should have various safety precautions in place, and also double-check the auto maker or tech firm that is coming to do tests at the facility. There are numerous federal, state, and local regulations and authorizations that are emerging about the testing and safety aspects of self-driving cars, all of which needs to be included into the efforts at the proving ground.

In some cases, the auto maker or tech firm will intentionally want to test how their self-driving car handles accidents. As such, there needs to be sufficient capabilities to handle intentional accidents, whether it be the availability of fire trucks and ambulances, which are also needed just-in-case an unintentional accident happens. This also brings up the insurance and liability aspects involved when at a proving ground.

The United States Department of Transportation (DOT) realized that self-driving car makers would want to have proving grounds to do their tests, and so in 2016 they issued a solicitation of proposals for becoming a government designated Automated Vehicle Proving Ground. The intent was to form a network of multiple proving grounds, and to foster a Community of Practice involving the sharing of best practices regarding the testing, demonstration, and deployment of automated vehicles.

The US DOT was seeking closed tracks, campuses, and limited roads that could be used for testing of self-driving cars and trucks, and each such approved proving ground would have a Designated Safety Officer, providing for the safety management needed to operate at the proving grounds. Proving grounds that became part of the designated set would also need to agree to share safety data generated during the testing and operations at the facility (data of a non-proprietary and non-confidential nature).

There were sixty applicants and the ten that the DOT selected are:

  • City of Pittsburgh and the Thomas D. Larson Pennsylvania Transportation Institute
  • Texas AV Proving Grounds Partnership
  • U.S. Army Aberdeen Test Center
  • American Center for Mobility (ACM) at Willow Run
  • Contra Costa Transportation Authority (CCTA) & GoMentum Station
  • San Diego Association of Governments
  • Iowa City Area Development Group
  • University of Wisconsin-Madison
  • Central Florida Automated Vehicle Partners
  • North Carolina Turnpike Authority

Some critics of the choices have argued that the geographic dispersion of the chosen proving grounds is insufficient and believe that there should be more proving grounds approved.  We’ll need to see how that comes along.

The data sharing requirements present an issue for some auto makers and tech firms, and so it is unclear as to how many of them will be willing to participate at one of the official proving grounds. Instead, many of the big hitters in the self-driving car realm will use their own facilities or go to unofficial and more private facilities instead.  Or, they might first prove out their vehicles more privately, and then bring them to the proving grounds once they believe the vehicles to be more finely tuned.

Recently, the proving grounds in Willow Run by the American Center for Mobility (ACM) was able to do testing once a major snowstorm hit at the start of the winter, allowing for snow and icy driving conditions to be tested on their closed track. The Toyota Research Institute (TRI) and Visteon Corporation are making use of the facility, partially due to aspect that Visteon is headquartered at the nearby Van Buren Township. Ford is also making use of the ACM facility. Ironically so, since the grounds were previously used by Ford to build B-24 bombers in World War II. The Michigan economic development entity had been trying to find a means to make use of the 335-acre property and doing so for the modern act of testing of self-driving vehicles seems quite fitting.

At the ACM facility, there is a 2.5 mile long highway loop, two double-decker overpasses, intersections, roundabouts, and a 700-foot curved tunnel.  There is also a public highway that goes along an edge of the property and can be used as part of the overall highway loop. This year and into next year, the facility will be expanding its capabilities including adding a city-like driving capability and a technology park. Private investors for the facility include AT&T, Hyundai, Visteon, TRI, Ford, and others.

One aspect of proving grounds involves whether they are focused on the early-stage of self-driving cars or the later stages of self-driving cars. For example, the Mcity facility at the University of Michigan is setup more so for early-stage testing, partially due to the more limited facilities in terms of shorter test tracks. The Mcity facility is only about a dozen miles from the ACM facility, though it was the ACM that got the nod to become one of the official ten locations approved by the US DOT.

In a less conventional approach, some self-driving car makers or tech firms are using unusual proving grounds. For example, a start-up firm called Voyage (a spin-out of Udacity) is using Ford Fusion’s outfitted with self-driving car gear and trying their vehicles out in a gated community of mainly retirees. The Villages Golf and Country Club in San Jose, California, provides a testing ground for Voyage. The speed limit there is just 25 miles per hour, and the roads are relatively tame. Residents can summon a Voyage self-driving car via a smartphone app and use it for door-to-door transportation within the community. It’s an intriguing approach and perhaps provides more of a shuttle-level kind of self-driving testing than the more elaborate aspects available at true proving grounds.

In considering the official proving grounds, we might also begin to ponder whether we will want self-driving cars to ultimately be certified in some fashion. Should AI self-driving cars need to get a government approved certification to be allowed to drive on our public roadways? If so, what would the test for certification consist of? Would the tests be conducted at the official proving grounds or elsewhere?

Will there be a kind of Consumer Reports or third-party entity that will try to review and assess self-driving cars? Will the public come to expect that an independent assessment of self-driving cars is needed?

This also brings up whether the public will be able to get access to the data collected at the proving grounds. One might make the case that the unofficial private proving grounds would not need to provide to the public the testing data, on the other hand there are bound to be some that believe they should. In the case of the official proving grounds, we have yet to see how much of the data will be shared with the public, versus only being shared by those that use the proving grounds. Will competitors that don’t use the official proving grounds be able to get access to the testing data of their competition that is using the official proving grounds?

The good news overall is that most of the auto makers and tech firms have so far realized that they need to be somewhat thoughtful about where and when they are testing their self-driving cars. Those that jump the gun and move right away to do testing in the public, risk not only their own efforts, but also put into jeopardy the entire field of AI self-driving cars.

It won’t take too many of the bad apples, so to speak, which cause or get involved in public accidents, for there to be a potential backlash against the self-driving car industry. This could cause heavy regulations to appear, and also create doubt that would dry-up the funding for self-driving cars.

Making use of proving grounds is the better and safest way to proceed, though the proving grounds have to be similar enough to the real-world that the developers aren’t falsely lulled into thinking that their self-driving cars will work in the real-world.  Getting an AI self-driving car to work on a small-scale track under controlled conditions is not a satisfactory form of testing. Likewise, it is possible that the self-driving car AI would learn how to navigate the proving grounds, but then be overfitting to an artificial layout and not be able to handle the real-world open-ended conditions. It’s important that testing be done of AI self-driving cars, and that it be done in the right ways.

This content is originally posted in AI Trends.