A Cruise somehow ran into a very large bus
Incident
On March 23rd, 2023 at 2:46pm a Cruise vehicle named “Souffle” was traveling west on Haight St in San Francisco and ran into the rear of a stationary Muni articulated public-transportation bus. The Cruise vehicle was in autonomous mode and carried no passengers. There was significant damage to the front of the Cruise and it was towed away from the scene. A software update was later made to allegedly fix a problem in determining the speed of an articulated vehicle. The Muni bus was not significantly damaged but was out of service for approximately 40 minutes.
Cruise claimed that the problem was due to poor speed prediction of the articulated bus and that they fixed the problem via a software recall but this analysis shows that they were not being truthful and the danger likely persists.
Social media and additional info
News reports
- Forbes 3/24/23 –
Cruise Cars Crash Into San Francisco Muni Bus And Tangle In Fallen Trolley Wires - GM Authority 3/28/23 –
Cruise AV Rear-Ends City Bus In San Francisco - Forbes 3/31/23 –
Cruise DMV Crash Report Suggests Their Car At Fault In Hitting Bus - Forbes 4/7/23 –
GM’s Cruise Robotaxi vs Bus Crash Caused By Confusion Over Articulated Bus; They Say It’s Fixed - GM Authority 4/10/23 –
GM’s Cruise Recalls 300 AVs After Crash With City Bus
Details provided by Cruise
Kyle Vogt, then the CEO of Cruise, published a blog post on 4/7/2023 describing the incident, analysis, and the actions Cruise took in order to address the problem. It does not address the root cause, which was that the vehicle should have stopped in time based on other sensor data.
One of our Cruise AVs was recently involved in a minor collision after a city bus slowed and the AV was late to brake behind it. It resulted in minor damage to the front fender of the AV and caused no injuries.
…What Happened
Less than an hour after the collision, we had fully assembled a team to investigate what happened. We also moved quickly to brief our state and federal regulators on the incident and made our team available to answer any questions they had.
We quickly determined the bus’s behavior was reasonable and predictable. It pulled out into a lane of traffic from a bus stop and then came to a stop. Although our car did brake in response, it applied the brakes too late and rear-ended the bus at about 10 mph. We identified the root cause, which was a unique error related to predicting the movement of articulated vehicles (i.e. vehicles with two sections connected by a flexible joint, allowing them to bend in the middle) like the bus involved in this incident.
In this case, the AV’s view of the bus’s front section became fully blocked as the bus pulled out in front of the AV. Since the AV had previously seen the front section and recognized that the bus could bend, it predicted that the bus would move as connected sections with the rear section following the predicted path of the front section. This caused an error where the AV reacted based on the predicted actions of the front end of the bus (which it could no longer see), rather than the actual actions of the rear section of the bus. That is why the AV was slow to brake.
What We Did
Once we understood the root cause, our engineering teams immediately started creating a software update that would significantly improve performance near articulated vehicles. Once that work was completed, tested, and validated, our operations team rolled the change out to the fleet. This work was completed within two days of the incident occurring. The results from our testing indicated that this specific issue would not recur after the update.
Although we resolved the root cause in this particular incident, our teams continued to investigate the full extent to which this kind of issue occurred in the past, might occur under a variety of conditions in the future, and might be identified sooner. Our vehicles encounter buses like this one every day, but we’d never caused this kind of collision before. We needed to understand if it was more widespread or isolated to a very unique and rare set of initial conditions.
Our data and simulations showed that it was exceptionally rare. At the time of the incident, our AVs had driven over 1 million miles in fully driverless mode. We had no other collisions related to this issue, and extensive simulation showed that similar incidents were extremely unlikely to occur at all, even under very similar conditions. The collision occurred due to a unique combination of specific parameters such as the specific position of the vehicles when the AV approached the bus (with both sections of the bus visible initially, and then only one section), the AV’s speed, and the timing of the bus’s deceleration (within only a few seconds of the front section becoming occluded).
Although we determined that the issue was rare, we felt the performance of this version of software in this situation was not good enough. We took the proactive step of notifying NHTSA that we would be filing a voluntary recall of previous versions of our software that were impacted by the issue.
Why We Do AV Software Recalls – Kyle Vogt, Cruise CEO, 4/7/2023
Report to the DMV
Cruise submitted a report to the California DMV, though it did not provide any actual useful details of the incident. There was no way for the DMV to understand the problem based on this paucity of information.
A Cruise autonomous vehicle (“Cruise AV”), operating in driverless autonomous mode, was traveling on westbound Haight Street between Ashbury Street and Masonic Avenue. At the same time, a San Francisco Municipal Railway (“MUNI”) Bus stopped ahead of the Cruise AV. Shortly thereafter, the Cruise AV made contact with the rear bumper of the MUNI Bus, damaging the front fascia and front fender of the Cruise AV. The parties exchanged information. There were no reported injuries and the Cruise AV was towed from the scene.
https://www.dmv.ca.gov/portal/file/cruise_032323-pdf/
It is noteworthy though that while Kyle Vogt reported in his blog post that “It resulted in minor damage to the front fender of the AV”, the information submitted to the DMV actually indicated that there was moderate damage, damage that corresponds to photos taken at the scene.
NHTSA Safety Recall Notice
Cruise filed a software Safety Recall Notice with the NHTSA. The description is notable because it avoids the root cause, which is that the Cruise Automated Driving System (ADS) did override early enough the faulty prediction system with clear information from their sensors, including their LiDAR sensors, that an object was immediately ahead and that the vehicle should brake in time to avoid a collision. Cruise’s description of the problem that they submitted to the NHTSA is:
Description of the Defect: This issue could occur when (a) the ADS perceived both the front section and rear section of an articulated vehicle initially; (b) the articulated vehicle then maneuvered in such a manner that the rear section of the vehicle fully obstructed the front section of the vehicle; and (c) the articulated vehicle then decelerated close to the AV within a few seconds of the front section becoming obstructed. In such a circumstance, the ADS could inaccurately determine that the obstructed front section of the vehicle was continuing to move forward, and that the rear section of the vehicle would continue to move forward with the front section, even if the vehicle was decelerating.
This issue resulted in a single collision on March 23, 2023, in which a Cruise AV inaccurately predicted the movement of an articulated San Francisco Municipal Transit Authority (“MUNI”) bus. In this incident, the ADS initially perceived both sections of the bus as the bus was pulling out of a bus stop in front of the AV. As the bus proceeded forward into the AV’s lane of travel, the rear section of the bus obstructed the front section. Shortly thereafter, the bus began decelerating. The ADS inaccurately determined that the bus was continuing to move forward in traffic based on the anticipated behavior of the front section of the bus, which was by then obstructed, and the ADS commanded the AV to begin decelerating too late to avoid a rear-end collision with the bus.
The aspect of the Subject ADS Software that caused this issue was implemented with a software release on January 12, 2023. Cruise has determined that this scenario would not recur after a software update was installed on all affected vehicles on March 25th 2023.
No other collisions have occurred as a result of this issue.
Part 573 Safety Recall Report 23E-029
Cruise software release notes
In their subsequent software release notes Cruise indicates that the issue with articulated vehicles was addressed. But it is very disconcerting to see their statement that behavior was only improved by 20 to 30%. That certainly does not appear to be a reliable fix to that particular problem.
April 2023 Release Notes – posted on 5.11.2023
- Shipped improvements to our ability to predict trajectories for articulated vehicles improving behavior around these vehicles by 20-30%.
Analysis by Brad Templeton in Forbes
Brad Templeton, who covers robocar technology for Forbes and previously worked on Google’s car team, published an in depth analysis of the incident in Forbes on April 7th titled GM’s Cruise Robotaxi vs Bus Crash Caused By Confusion Over Articulated Bus; They Say It’s Fixed. Brad accepts the possibility that Cruise might have found and fixed a problem with how their AI system predicts the speed of an articulated bus. But he strongly emphasizes that it was a failure for Cruise ADS to not have a reliable and simple system to brake when the AI system appeared to be obviously incorrect.
Cruise soon fixed their mistake about predicting the bus only from its invisible front. But it’s unclear if they have fixed the larger reason for the crash, which was not correcting that in real time when the sanity checks failed.
Brad Templeton in Forbes
Brad also points out that this issue should have been found and fixed before this crash:
There’s been so much concern about this crash because encountering a slowing bus in the middle of a lane is definitely not what we call a “corner case.” While self-driving vehicles will make mistakes and have incidents, we mostly expect them to be in more unusual situations. Normal situations should be very well tested in daily driving and in simulation.
Brad Templeton in Forbes
And Brad explicitly states how Cruise could have done better:
Brad Templeton in Forbes
- Slow to acknowledge fault and to state public was not at risk due to fault and why
- Not detecting a problem of this nature in testing
- Inadequate sanity checks to prevent crash, though some checks reduced the severity of the crash
- No transparency yet on sanity checks to prevent different problems of this magnitude
- While the immediate cause is fixed, it’s not yet clear if the broader problem is fixed. Ideally, their car should be updated so that even if it still made the mistake with the bus, it would not hit it any more.
Brad had also published an earlier article Cruise DMV Crash Report Suggests Their Car At Fault In Hitting Bus with interesting details on the many sensors that the Cruise vehicles have, and how they absolutely should have detected the bus in time such that the car would have been able to brake in time.
Why it shouldn’t happen
The trouble is there is essentially little that’s more obvious to the sensors of a self-driving car than a bus slowing to a stop in front of it. The Cruise Bolt is bristling with sensors, all of which easily detected that bus, but for some reason the vehicle did not act correctly. The Bolt has 5 LIDARS which would see the bus extremely clearly — even the tilted LIDARs meant to see around the sides of the vehicle would spot it. It has multiple long range radars and 10 short range radars, many of which would have solidly sensed this bus, particularly while it was moving. It has 14 cameras, and while computer vision is still imperfect, again this is an obvious target, and the cameras would see the bus via other vision methods that are not machine learning based, such as motion parallax and possibly stereo vision, if Cruise uses that.
… it’s very out of place for a car to run into something as obvious as a bus. There should be many systems in place to make sure that never happens, not just to protect buses but to stop it happening with anything, including vulnerable road users.
Brad Templeton – Cruise DMV Crash Report Suggests Their Car At Fault In Hitting Bus
Analysis on what actually went wrong
The Cruise narrative alone is simply not plausible. They claim that the bus pulled out of a bus stop, that the ADS first detected the front and rear sections of the bus since the articulated bus was bent, that the front of the bus was then occluded because the bus was straight, and then the bus stopped within a few seconds before the AV could properly detect it. While the problem of not properly handling articulated buses is reasonable, the Cruise vehicle followed the bus for a significant distance and time before crashing into it and should have had a safety system in place that would have easily stopped the robotaxi in time.
Software layers
To understand how the crash should have been prevented one needs to look at the various software layers involved.
- Routing Layer – Highest level. How the vehicle gets to its destination. Doesn’t involve vehicle sensors and instead uses traffic and other centralized information so likely done in the Cloud. Similar to Google Maps trip planner. Not really involved with safety, though the routing layer likely avoids dangerous routes, such as where an AV might make an unprotected left turn.
- Immediate Path Planning Layer – Middle level. Fuses vehicle sensor data to determine what to do next. Predicts what other vehicles and pedestrians are going to do. Everything is in terms of percentages, such as 90% chance vehicle will turn right and 10% it will continue forward. Very complicated software. Can use AI and Machine Learning. Needs to handle all situations, even ones not foreseen. Cannot be perfect. Computation done on vehicle.
- Safety Layer – Low level. Similar to driver assist technology such as cruise control with braking. Critical since the path planning layer cannot be perfect and handle all situations. Likely deterministic and doesn’t use neural nets (though can definitely use ML). Uses multiple sensors that are also used for Immediate Path Planning Layer, especially radar. Needs to override the other layers since safety is key. Safety layer is of utmost importance and is critical for preventing crashes.
Key point is that the Safety Layer is critical because the Immediate Path Planning Layer is inherently imperfect. The Safety Layer must always take precedence.
Root cause of incident
Cruise claimed that the root cause of the crash was a problem in the Immediate Path Planning Layer that improperly handled an articulated vehicle and was incorrect in determining the speed and thereby the location of the back of the bus. But this was not the “root cause” of the crash since the Immediate Path Planning Layer is inherently vague. It is buggy because it is complicated and can’t foresee all situations. Addressing this one situation still leaves many other problems, and therefore will not prevent future crashes sufficiently. The articulated vehicle problem was the trigger of the crash but not the root cause, an important distinction.
The true root cause of the crash was that the Safety Layer did not cause the vehicle to brake in time, even though the situation was extremely simple. The sensors must have detected the very large bus stopped directly ahead. There would have been no ambiguity. Yet the Safety Layer failed to override the inherently less dependable information from the much more complicated Immediate Path Planning Layer. This failure of the Safety Layer is the root cause of the crash, and it is what needed to be fixed. Yet Cruise wrongly told the public, the DMV, the CPUC, and the NHTSA that they “fixed” the particular problem with a software recall to address the issue in the Immediate Path Planning Layer. But for other situations, and there are an infinite number of other situations, the Immediate Path Planning Layer can still provide incorrect information, and the Cruise vehicle could still crash due to the root cause, the Safety Layer being broken.
The conclusions stated here correspond well to the analysis by Brad Templeton described above, providing even greater credence to them.
Important detail on where crash occurred
While it is clear that the low-level Safety Layer failed, it is still useful to understand the exact geographical location where the failure occurred. This way one can see how egregious the failure really was since the Cruise vehicle had a long time to accurately locate the stopped bus and initiate braking. This is also relevant because it shows that Cruise providing a very misleading narrative.
Precise determination of crash location
The exact location of the crash can be easily determined from available photos along with Google StreetView. And from the impact location we can determine how far the rear of the bus traveled in a straight path after pulling away from the bus stop.
Then used the Google Maps distance measuring tool to determine how far the rear of the bus traveled between when it was first straight in line with the road, until it stopped and was hit by the Cruise vehicle.
Confirmation using GPS data for the bus
To confirm that the bus was hit a significant distance away from where it pulled away from the stop, we created a 1 minute video playing back the GPS data for the bus. The video below clearly shows where the impact occurred, and that it was a significant distance (approximately 120 feet ) from the bus stop. Note: the GPS data is for the front of the bus, which is almost 60 feet from the rear of the bus, where the collision occurred. The GPS data also show that the bus indeed first stopped at 2:47pm, and that it didn’t continue until 3:19pm, upon at which time it ended up doing a short-turn and going back downtown since it was so far behind schedule. Other buses in both directions were able to pass through the crash scene during that time, minimizing the resulting service disruption.
Haight Street has slow speeds
Duration between sighting bus and crash
The bus traveled in a straight path for approximately 120 ft. It pulled out of a bus stop and then stopped again, on a slow street with a speed limit of 20mph. With the starting and stopping It is reasonable to expect the average speed of the bus for the 120 ft it traveled in a straight line to have been approximately 10 mph, which is 15 feet/sec. Covering 120 ft at 15 feet/sec would take approximately 8 seconds.
Distance away from bus when braking started
According to Cruise, the vehicle was moving at 10mph when it hit the bus. Given the 20 mph speed limit and the good adherence of Cruise vehicles to follow the limit, the car was traveling at most 20 mph when it finally detected the stopped bus and braked. One can simply subtract the stopping distance at 10 mph from the stopping distance at 20 mph to determine how far away the vehicle was when braking was initiated. Using a stopping distance calculator, with zero for reaction time, the distance from the bus at which the Cruise initiated braking was under 14 feet, quite a short distance.
Conclusion of analysis
The conclusion of the analysis is that the Cruise vehicle followed the bus as it traveled in a straight line for approximately 8 seconds. 8 seconds is a very large amount of time for the Cruise ADS to properly determine the speed of the bus. Not properly determining the behavior of the bus within 8 seconds is more than just a software bug. It is definitely an architectural problem that should have been resolved as part of the software recall.
It is noteworthy that in their submittal to the NHTSA cruise stated “the articulated vehicle then decelerated close to the AV within a few seconds of the front section becoming obstructed”. But 8 seconds is certainly longer than just a few seconds.
The Untruths by Cruise
There were several important statements made by Cruise that are simply not honest:
- In original blog post they state the crash “resulted in minor damage to the front fender of the AV”. The “minor damage” narrative was picked up by the news. But the pictures from the scene of the crushed front end show a very different situation. When they submitted a report to the DMV they upgraded the damage description to “moderate”. But they didn’t report any damage to the NHTSA.
- Not clarifying that the crash occurred 120 feet after the articular bus fully straightened misattributes the cause of the problem to the regulators, which include the DMV, CPUC, and the NHTSA.
- Not explaining that the low-level safety system was the root cause of the crash is very misleading. It also means that the regulators never were given the opportunity to understand if the problem that caused the crash was actually addressed.
- Conducting a software recall just for the trigger of the crash instead of for the root cause of the crash provided the public and regulators a false sense that the true problem was corrected.
Conclusions
There is no sugarcoating the situation. The Cruise robotaxi ran into the back of a very large and obvious stationary bus. It did so because of buggy software and the lack of a functioning safety system, one that should have easily been able to notice the bus as it traveled in a straight line for approximately 8 seconds. Testing clearly problematic software on 2-ton vehicles on public roads without safety drivers is not reasonable. And obfuscating the problems to both the public and to regulators is not acceptable.
