In my previous post (https://www.pikcharitra.com/post/knight-industries-two-thousand) I talked briefly about how self-driving levels are categorized and the basic structure of car communications. Let's now walk our way through the basics of car sensors, driving models, how AI in cars work, and see if that can help us understand why "self driving" is difficult and why it's so cool when it works well. In this two-part post we will break down the details of how this is done at a conceptual level.
And oh by the way, it's not always about cars. This video shows an autonomous food tray collection robot at a conference. For this robot to move around without running into people it needs to sense the environment, make adjustments (speed, direction), and be able to recover from tight corners (literally).
Sensors
Cars have had sensors for a very long time now. Sensors in a car measure temperature (air, oil, coolant, etc.), voltage (battery, alternator, oxygen, etc.), movement (rpm, position, etc.), light (day/night, oncoming headlights), vibration/acceleration/impact (airbags, etc.), liquid levels (coolant, oil, fuel). All sensors measure voltage or a current that is impacted by the parameter that is being measured, e.g., a change in light levels changes a resistance and therefore the current flowing through the resistance. Most sensors are digital, and their outputs are binary digits that represent the output values. These values are then sent on the CANbus (the communication network inside the car) to the computers that can then take action on these values. This is in the form of messages that are sent on the CANbus.
The actions that the computers take based on the values of the sensors are also messages that will contain the type of action to be taken, and sometimes additional numbers to indicate "how much of action" is to be taken. As an example, when the cruise control is activated, and the speed sensor detects that the speed has fallen below the set speed, the cruise control computer monitors this and calculates the amount by which the throttle needs to be opened or fuel needs to be provided. A similar case is when the antilock braking system (ABS) is activated, the ABS computer will calculate the amount of braking force to be sent to each wheel and send out signals to that effect (several 100 times a second in this case!).
Actuators
Actuators can be thought of as the "opposite" of sensors. While sensors convert a measured attribute into a numerical value or signal, actuators can take a value or signal as input and do some action, which is usually mechanical in nature. A very simple example of an actuator in a car is the door lock relay that opens the latch when you press a door, or stop the car. Other actuators in a car can open/close valves or make other kind of movements.
An example of an actuator in a car is an electronically controlled steering. As you turn the steering wheel, a motor can provide assistance that makes it easier to turn the wheel based on the speed of the car. This amount of assistance can be varied to provide the feel of stiffer turning (e.g. "sports mode") or easier turning (e.g. "comfort mode"). But if there is a motor attached to the steering wheel, can the motor then turn the wheel on its own without the need for a human? Let's hold that thought as we temporarily take a diversion.
If you are interested in getting a little more technical and want to look for more reading a great topic to start with is PID Control. Interesting side note, if you have a hobby-level 3d printer at home and have been playing with modifying the electronics, switching around the boards, etc. you probably had to a "PID calibration" or "PID test" at some point. Hopefully the link will help you understand why you had to do it.
Car See...
For a car to exhibit any level of autonomy when moving on the road it is necessary to be able to sense the physical environment outside the car, e.g., distance to cars and objects in front. For additional capabilities such as steering, it is necessary to not just look ahead but also get some side glances in. Cameras are clearly in play here. A car needs to have some kind of cameras to be able to "see" around.
But the cameras don't always have to be similar to the ones we use to take pictures or what we use in our daily lives. There are other options. Basic ultrasonic sensors use sound waves to detect the presence of nearby objects. The sensor has an emitter that sends out high frequency sound waves (beyond human nearing) and a receiver listens for reflections/echos. A lot of cars have them in the front and rear bumpers and they are used for parking assist. They are cheap, inaccurate and work best at short ranges. The picture shows an example of a hobbyist ultrasonic sensor that can be used for projects and experiments.
Another technology is to use radar sensors that work similar to radars that track planes. Here the emitter sends out radio waves, instead of sound in the case of the ultrasonic sensors, and listens for reflections. The time gap is then used to compute the distance. The radar modules are usually positioned low in the front bumper of the car. They are more expensive than ultrasonic sensors but provide faster and more accurate measurements. There may also be other radar sensors to detect cross-traffic or pedestrians and these sensors are often in the back of the car. See here for a detailed explanation of how radar sensors work in a car.
And then there's LIDAR that uses lasers instead of radio waves or sound waves (the purists will wince here because light is also a kind of radio wave but we shall let that slide). A LIDAR unit uses light in visible/nonvisible wavelengths and rotating mirrors and detectors to detect reflections from a sweeping beam of light. LIDAR is extremely accurate and can provide very detailed visualization of the objects around. It's also used for imaging internal environments such as rooms, workshops, etc. for 3D rendering, virtual reality applications and so on. It's also more expensive than the other types of sensors we have talked about.
...Car do
It looks like we are getting some place now (pardon the pun). We have the ability to visualize the environment around so the car can now "see" what's ahead and around. We have seen how cars already use sensors and actuators to respond to environmental attributes. We can form a mental conceptual construct of how a radar sensor can be used to detect the distance between a car in front of us and then use that to adjust the speed and/or brakes to maintain that distance, aka adaptive or smart cruise control. A little while ago we left a thought behind on whether a motor can turn the steering wheel without a human. Clearly it appears it can. All we need is to know how much of a steering correction to make and then instruct the motor to turn the wheel that amount. And therein lies the rub. Steering also requires us to visually understand the view. A curve in the road is not easibly visible to radar or lidar. Imagine an empty parking lot with lane markings. There are no physical objects to reflect ultrasonic or radio waves back that will provide any hints of the layout. Since the parkings lots and roads are designed for humans we now have colors, lines, shapes, etc. painted on them that provide us with those cues. And that's why cars also need regular visual cameras to get a complete picture. You see an example of a car being steered using visual input from a camera in the video below.
In the next part of this post (edited for readability) we shall look at how to work with images and respond to components of an image.
Comments