Researcher aims to help autonomous vehicles make faster, safer decisions in real time

Imagine a child jumping out in front of your car as you drive down a residential street. Now count to three.

That’s how long it could take a self-driving car with insufficient data — confused over whether to brake or swerve — to turn control over to the driver. By then it’s too late to react.

In dynamic urban environments, three seconds is an eternity, says  Ehsan Hashemi, an expert on autonomous navigation in the University of Alberta’s Faculty of Engineering.

“It’s called a handover situation, when the autonomous driving system realizes it’s not capable of driving the vehicle,” he says. “The current standard transition of between three and 10 seconds is huge, and really scary.”

The ability for an autonomous vehicle to quickly perceive and understand the scene around it remains a big hurdle to safe commercialization of the technology, says Hashemi, mainly because of computer processing constraints and uncertainties in the environment.

He and his team are aiming to dramatically improve that processing speed by limiting the information a vehicle receives to what is most relevant, allowing them to set the processing bar (including decision making) at about 20 milliseconds for vehicles to refresh available data and make faster, safer decisions in real time.

‘Shared perception’ provides a wider view

Hashemi’s “shared perception” system works by combining on-board sensors — which read the vehicle’s immediate surroundings and track moving objects — with remote camera and  LIDAR sensors mounted on nearby fixed objects such as lampposts and buildings at intersections, providing a much wider view.

Together, they have a more comprehensive view of the vehicle’s environment, catching obstructed or unexpected moving objects in their sights and reducing the need to hand over control to the driver.

“There might be some limited field of view if, for example, a pedestrian crosses in front of a bus,” or a cyclist suddenly emerges from around a corner, says Hashemi.

If a child chases a ball across a residential street, a remote sensor could detect that movement before the car enters the scene from as far away as 100 metres. 

“This would allow for proactive measures, braking or steering ahead of time rather than waiting for a driver to see a moving object and understand the circumstances.”

A key innovation in Hashemi’s work is the filtering of information using artificial intelligence. Rather than accounting for everything in a vehicle’s field of view, as would a human driver, the system’s algorithm targets only relevant data — such as objects moving in its path and important features like traffic lights and signs.

“The co-operative perception system is also scalable for various urban environments considering higher probabilities of occlusion or faster dynamic objects,” he says.

Testing technology for Canadian weather

Hashemi hopes to launch a small pilot project using perhaps 10 remote sensors in downtown Edmonton as part of the city’s  Vision Zero initiative, aimed at reducing the number of collisions involving pedestrians and cyclists.

“With the harsh weather conditions we have in Edmonton, snow and ice can occlude many of the features autonomous vehicles rely on,” even lines separating lanes, he says, adding that much of the technology so far has been tested in milder climates such as California and Arizona.

Hashemi and his team of three doctoral candidates, two master’s students and one post-doctoral fellow have received funding from the Canada Foundation for Innovation and the Natural Sciences and Engineering Research Council of Canada to test their system on the U of A’s North Campus. Four LIDAR sensing units installed on light poles will analyze traffic in the area.

He has met with TELUS to explore a partnership to take advantage of the high speed and low latency of 5G networks to enable safe and fast shared perception for autonomous driving.

The technology also has potential for warehouses where robots navigate through environments in close proximity or in co-operation with humans, or for networked mobile robots performing maintenance/repairs in dynamic environments, says Hashemi.

“ Our lab is developing technology to enhance the safety of pedestrians, cyclists and vehicles. As a result, our technology is well poised for immediate commercialization in various communities in Alberta and Canada.”