One of the most popular reasons people buy Tesla, aside from performance and range, is the company’s suite of self-driving features. Elon Musk previously said that Tesla is as much a software company as it is a car manufacturer.
So what sets Tesla’s Autopilot and Full-Self Driving features apart from every other automaker on the planet? How do Tesla vehicles see what is happening in the world around them?
The basics of an autonomous driving system
Before we get into Tesla’s Autopilot, let’s take a look at the basics of a self-driving system. For a vehicle to successfully navigate a city road, it must be careful about the physical objects around it and any signs, traffic lights, and other visual cues it encounters.
To capture this data, you need two types of inputs in your car:
- Distance measurement sensors: These are usually ultrasonic or Lidar sensors placed around the car to generate a virtual map of what is happening around the car.
- cameras: To detect stop signs, traffic lights, lane change signs on the highway, and any other visual cues the system needs to make a safe decision.
A basic navigation system would use both types of input to accurately judge the car’s trajectory. However, more advanced systems, like those used in a Tesla, are slowly decreasing the reliance on distance-measuring sensors to make important decisions.
Instead, they rely on a suite of 360-degree cameras that cover the car to get a full field of view, map other cars, objects, pedestrians, and relevant signage in the vicinity and work accordingly.
That’s not to say that distance measurement sensors are going to be obsolete. They will still be present in your Tesla and will play a crucial role. It’s just that as computers get faster, smaller, and more capable, computer vision is also progressing at a much faster rate. It is better to process only one data stream instead of handling multiple inputs.
How do Tesla systems work?
As of April 2019, every Tesla comes with eight cameras that provide a full 360-degree view of everything around the car at a range of up to 250 meters. The video streams coming from each of these cameras are processed to detect objects, signage, and anything else that might be relevant to the car’s movement and navigation.
The actual processing itself is handled by the on-board computer. Each source is processed by the algorithm and points of interest such as traffic lights, signs, any moving traffic and pedestrians in the vicinity are marked.
Detecting objects in a video stream is not too difficult a task either. Using an image processing library like OpenCV, you can easily write a small script yourself that will detect anything from moving traffic to the actual speed at which traffic is moving.
This is where Tesla’s neural networks come into force. If you were to build a simplified version of Tesla’s Autopilot software or Tesla’s Full Self-Driving software, you would need a neural network that can identify things.
As with any machine learning application, the larger the training data set, the more accurate the algorithm will be at detecting things, whether it’s cars, pedestrians, or street markings. With hundreds of thousands of cars driving millions of miles every day, Tesla has created a training data set.
This data set forms the backbone of Tesla’s neural network. Getting better at detecting or ‘seeing’ things every day as more and more cars on the road enter information into the database.
This neural network, combined with the aforementioned array of eight cameras, forms what Tesla calls Tesla Vision, which offers Autopilot and related features. Please note that while Autopilot comes standard on all Tesla (as of April 2019), additional features must be purchased through your Tesla account.
Once all the decisions are made, it’s just a matter of translating that logic into the movement of the car safely and gradually. The algorithm communicates with the car’s on-board motor controller and tells it what to do. Then the car does the same thing and you get what appears to be self driving.
What autonomous driving capabilities do Tesla cars offer?
At the moment, the autopilot packages available from Tesla are as follows.
Autopilot
This is the standard set of self-driving features that come in every Tesla. Available features include:
- Traffic Aware Cruise Control: matches the speed of your car with that of the surrounding traffic.
- auto direction: it helps the car stay within a “clearly marked lane”. It works in conjunction with Traffic-Aware Cruise Control.
Improved autopilot
This is probably the best value package when it comes to Tesla driving assists, coming in at $6,000. Available features include:
- Automatic lane change: It moves the car into an adjacent lane on the road when the driver activates the turn signal.
- Parking lot: Help in parallel or perpendicular parking.
- Navigate on autopilot: It suggests lane changes, helps navigate junctions and assists the automatic lane change feature by providing guidance for on-ramps and off-ramps on highways. Currently in beta.
- Convene: Automatically ‘calls’ your car to or from a tight space. It is only designed to be used over short distances.
- Smart Summon: Browse more complex environments and parking spaces to find you in a parking lot near you.
Full autonomous driving capability
As the name suggests, this is the most advanced level of autonomous driving a Tesla can get for a while. It currently has a starting price of $15,000, following an increase of $3,000 in October 2022 from the original price, or a subscription of $199 per month. Available features include:
- All features included in Autopilot and Enhanced Autopilot.
- Control of Traffic and Stop Signs: Actively look for stop signs and traffic lights to stop the car on time. The feature is currently in beta and requires active driver supervision.
- Automatic steering on city streets: This is an upcoming feature with more details yet to be announced.
Is Tesla’s fully autonomous driving really autonomous?
That really depends on what you expect from this system. If you’re just looking for convenience on your daily commute or road trips, these features are sure to help. However, if you wait for a ride, the systems are not ready yet.
As mentioned above, Enhanced Autopilot is Tesla’s most comprehensive set of driver assistance features at this time. The name Full Self-Driving is a bit confusing, as Tesla itself clearly states that driver supervision is required at all times.
Despite the company moving towards full autonomous driving, it still has opposition and regulations to fight. The Full Self-Driving system is still being tested, which in itself is a whole different dilemma to tackle.