Driving a car can be an experience in informational and sensory overload. Drivers are distracted, stressed, and more obsessed with multitasking than ever before, making every moment behind the wheel an opportunity for human error.
So, what happens if we take humans out of the equation and let the cars do the driving instead?
Welcome to the world of mobility, where researchers at Michigan State University are asking that question in ways that capitalize on the strengths of the university’s research enterprise, namely in the nitty-gritty details of sensors, sociology, and safety as they apply to the present and future of autonomous and connected vehicles.
Taking sensor technology to school
Autonomous vehicles cannot be expected to do everything we need them to do right away. But they can be taught, and that’s where sensors—and the CANVAS (Connected and Autonomous Networked Vehicles for Active Safety) research group—come in.
Imagine a sedan, outfitted with a variety of sensors, including radar, lidar, and multispectral cameras, as well as a powerful Graphics Processing Unit based computer. It’s a lot of technology for one car, but it’s the most essential piece of the puzzle.
“One of the challenges with these sensors is that they each have weaknesses, particular conditions in which they don’t work as well. So you have to be able to correlate data from multiple sensors,” says John Verboncoeur. Verboncoeur is associate dean for research and graduate studies, and a faculty member in the College of Engineering. He oversees CANVAS, and acts as an evangelist, a motivator, and a coordinator of mobility efforts at MSU.
He explains further, “Think of it like having two or three different viewpoints of the same scene, and from each of those viewpoints, you see a different perspective that you couldn’t see from the other position. Data from each sensor serves as a backup and augmentation for the other sensors, and combining multiple streams of information adds more detail.
“For example, multiple visual perspectives can provide depth perception, such as human stereoscopic vision. Multiple spectral components let us see details that one spectral component alone may not reveal, for example infrared providing detail in situations of low-contrast visible light, or microwaves providing information when a visible camera image is washed out by direct sunlight.”
CANVAS researchers are working on ways to automate that sedan’s ability to fuse all of that information together into one very detailed, real-time picture of its surroundings. Perhaps the sensors will pick up on a steady drizzle that has made the pavement wet, a deep pothole a few meters ahead and a set of orange barrels blocking off a curb repair project in the opposite lane. The sensors will also see the cyclist in the adjacent bike lane, a maintenance worker slowly driving a mower through the crosswalk and a student on a skateboard rolling down the sidewalk nearby.
Then, researchers will teach the sedan to classify these objects and conditions, and make informed decisions about how to proceed: move left to give the cyclist space, go slow over the pothole, allow plenty of stopping distance on the wet road to avoid hitting the mower in the crosswalk. The sedan will also be enabled to react, not just to these definite, visible obstacles, but the potential of an obstacle, such as if the student on the skateboard loses her balance and topples into the street, or an oncoming vehicle appears and has to swerve into the sedan’s lane to avoid the orange barrels at the construction site.
Currently, this type of active safety technology only focuses on the individual vehicle it is supporting. The sedan is learning about and proceeding through its environment in a vacuum—not communicating any of the vital information it collects to the vehicles around it. But another team in CANVAS is hoping to change that, too, and move the norm toward sharing and exploiting the data the sedan collects. So when the sedan recognizes the unexpected construction barrels and the student on the skateboard, it can transmit that information to the other connected vehicles in the area: truck in the next lane, the oncoming delivery van, the campus bus preparing to turn at the stoplight, so they, too, are equipped with the knowledge to react accordingly to obstacles and potential obstacles that their own sensors may not be able to see.
In a world where vehicles are, essentially, being trained to drive, collaborate, and communicate, the infrastructure should be able to communicate too. MSU is a leader in pavement sensor technology, and, just this year, helped the Michigan Department of Transportation deploy 2,000 tiny sensors on the Mackinac Bridge. The coin-sized sensors, built in partnership with researchers at Washington University in Missouri, are powered solely by vibrations from traffic. They are inexpensive to produce, but provide critical data about the condition of the bridge’s structural components.
Installing the sensors on roadways—a concept that, according to Verboncoeur, is unique to MSU—will also be a major step towards “smart” infrastructure, which is friendly to connected and autonomous vehicles. They can provide lane-edge guidance in all types of conditions, including snow-covered roads or other low-visibility conditions, and on all types of roads.
Imagining the ‘what ifs’ of an autonomous future
Working parallel to the researchers at CANVAS are a group of researchers from other areas of strength at MSU, including urban planning, media and information, economics, business, law, and public policy, who are working on what the industry calls sociomobility.
Mark Wilson, professor of urban and regional planning in the College of Social Science, is one of those researchers, who compares the dawn of autonomous vehicles to the dawn of another disruptive innovation: the internet.
“When you look at the internet,” Wilson says, “it seemed to appear from out of nowhere in the early nineties, and at the time, it was seen as an engineering challenge.” The initial deployment of the internet considered all the necessary technological components—hardware and software, modems, circuits.
“Today on the internet, some of the biggest issues aren’t in the details of the engineering or the technology at all. They’re social. They’re related to the internet’s impact on interaction and engagement, retailing, freedom of speech and expression,” Wilson says.
There was no infrastructure or precedent for dealing with the social issues that came with the internet, because nobody was thinking that far into the future.
The goal of the sociomobility research group is to do that kind of thinking about autonomous vehicles, so that when the technology is ready, nobody is left scratching their heads about its effect on jobs, cities, traffic laws, the insurance industry, the supply chain or the security of information. Their ever-evolving body of research includes:
- Looking at how the general public is being educated about connected and autonomous vehicle technology, what their perceptions are, and what can be done to address skepticism or doubt.
- Looking at autonomous public transit, how transit passengers interact with one another and how a bus or a taxi with no driver present to enforce rules and decorum might adapt.
- Considering how the prospect of “having the car do all the work” affects the decisions people make before they get into a car. Are they more likely to consume drugs or alcohol, or be excessively tired or impaired in some other way that would affect their ability to take the wheel in an emergency?
- Understanding what type of workforce will be needed—and what kind of education they’ll need—to build and service the complex devices and technologies associated with the vehicles and the infrastructure.
- Additionally, considering how those complex technologies will be made accessible for persons with disabilities, the elderly and underserved populations, and people who have never driven a vehicle.
“This is more than just our research. This is about MSU providing a service to the public. We need to be mindful that public citizens will be the ones to make the decisions, and we want them to be informed,” Wilson says of the potential impact of these questions and answers he and the Sociomobility research group are seeking. “Our engineers want to say, ‘Here’s the technology,’ and we want to say ‘Here’s how we talk to communities and work through scenarios to make decisions about how that technology is used.’”
Putting it all together—to solve a real problem, today
At any given time on Michigan State’s 5,300-acre campus, people on foot, on bikes and on small electric- or gas-powered scooters are attempting to share the road, the crosswalks and the parking lots with people operating much larger, heavier, faster-moving vehicles. There are thousands of variables and opportunities for human error around every corner on a day-to-day basis.
But instead of trying to completely eliminate humans from the equation in every vehicle on every road in the country, MSU wants to specialize in low-speed pedestrian-dense multi-modal environments.
“Think of these areas as ‘first mile, last mile,’” Verboncoeur says. “If you’re commuting from somewhere, your first mile is the streets you travel to get to the highway or the public transit station, and your last mile is the streets you travel to get to your destination—your office, your corporate campus, your factory.”
First mile last mile streets, like those on campus, are frequently congested and make finding parking a challenge. But what would happen if, instead of driving their heavy passenger vehicles that “last mile” onto campus, people parked in lots on the periphery and utilized a fleet of much smaller, lightweight electric vehicles to get to and from their destinations on campus?
Alongside the ubiquitous scooters and bikes, Verboncoeur suggests connected, automated golf-cart-sized vehicles that are electric and made for short-distance movement. They will take much less of a toll on the infrastructure and the environment, and greatly increase the safety of pedestrians.
This is an area of mobility that, Verboncoeur suggests, isn’t being explored as deeply at other research institutions, which are focused more on automation in high speed and intermediate environments, rather than first mile, last mile environments.
“Right now, we think we can be the center of the universe for this particular niche,” Verboncoeur says, “developing technologies for low speed, last mile, multimodal transportation beyond autonomous cars.”
MSU researchers in engineering and sociomobility are also heavily focused on making campus a friendly environment for autonomous vehicles. In fact, MSU’s diverse landscape is the perfect place to test, teach and refine the technology as it continues to evolve.
Researchers are already working to incorporate sensors into vehicles that already spend a lot of time on campus. Imagine a city bus. It has a driver, and it goes about its route like normal, but it is outfitted with a suite of sensors that track obstacles and events the bus encounters during the day.
Then, with the addition of what Verboncoeur calls a “decision engine,” the bus will be able to take in the information from the sensors and make immediate judgments on what it might do in a particular situation. The decision engine won’t change the behavior of the bus, it is merely a simulation, but it will allow researchers to grade the engine’s decisions and determine whether it made a better or quicker decision.
Once the vehicles reach a certain level of proficiency—one in which they’re consistently equal to or better than a human driver—researchers will know that the technology is ready for use in an actual, autonomous car, which they will start testing in the former Spartan Village apartment complex south of campus, renamed Spartan Mobility Village.
There, a loop of road is being outfitted with pavement sensors and configured for testing of autonomous vehicles. The brick buildings will provide more realistic challenges for the radar, lidar, and cameras which respond differently to actual buildings than they do to, say, plywood facades of buildings. Real pedestrians and obstacles can be brought in to simulate an environment very similar to the first mile last mile landscapes where MSU hopes autonomous and connected cars and other vehicles will make the most difference. Ultimately, these technologies will demonstrate proficiency levels surpassing human drivers, and graduate to the full campus environment, initially under close researcher supervision.
If CANVAS and the Sociomobility group can demonstrate pedestrian safety, less traffic congestion, and a smaller ecological and infrastructural footprint to the public, they are confident that they can help people see autonomous cars the same way.
“We don’t want to get rid of walking and biking and driving, and change the layout of our cities just because we have autonomous vehicles,” Mark Wilson says. “We don’t want to get so caught up by the shiny new object and the novelty of it that we lose all sight of what this is really about, which is ‘I need to get from point A to point B. What is the best way to do that?’”