This jogger incident makes me think even more strongly that self driving software should never rely on a database of features for navigation. Because those database are inevitably brittle and become outdated, and then the software ends up with a “false sense of confidence” about what it is doing. Forcing the software to always rely on live sensor fusion data from the real world seems much more reliable, even if it makes it much harder to get something that basically always works.
> The backup human driver then “slammed the brakes at the last moment” and the car “stopped within a few feet of the pedestrian.” Had the human not intervened, Apple’s tests indicated that the car “would have almost certainly hit the jogger.”
Sounds like this contributed to Goodfellow’s departure - timing wise at least. If I were the VO overseeing the project and you are an AI expert claiming me sophisticated technology but somehow your models could not identify a regular person, I’d be very pissed.
> Apple reportedly hopes to “gain exemptions” from the National Highway Traffic Safety Administration to remove the steering wheel and brakes, relying fully on self-driving technology.
That sounds crazy to me given the state of self-driving technology. But it sounds like Apple.
I don't think Apple releasing a self driving car is good fit for the Apple brand or product development culture. Especially their value for secrecy and consumer privacy. We've already learnt from Waymo, Cruise and Tesla the importance of getting millions of miles of driving data. This "data first" approach has been challenging for each company. Apple is trying to go design first and delaying collecting data to refine their self driving capabilities - this will most likely delay their ability to be competitive from a safety perspective which is the most important thing for a customer. Pretty phones can't kill you but pretty cars can.
I think there are enough of us around who have seen really badly written software to know that there really is an easy-to-read piece of code somewhere in a high-frequency loop missing that simply says avoid object in front of me as a sanity check.
I wouldn't doubt for a second that their self-driving software is a nightmare to read.
I would wager that it's more likely the model that they feed sensor data to is unknowable, in the sense that most ML models and Neural Nets can't adequately be understood. Why does model x produce output y for input z? No one can say, it's based on thousands or tens of thousands of hours of training.
What is that big thing that I'm failing to see, that prevents all big auto manufacturers from implementing such a specific and straightforward piece of logic?
In the incident in question, it sounds like the vehicle classified the object as "stationary," before realizing it was a moving person. So the jogger was probably not in the way of the vehicle initially, and by the time it realized it was going to be in its way, it wouldn't have been able to brake in time.
If it had classified the person correctly, it would have been able to anticipate the danger and slowed down / stopped.
I have a willow tree over my driveway that grows some branches down every year. I don’t always trim them right away because I like the shade. When reversing out, the car often warns about imminent collision; I ignore the warning and let the branches slide over the top of the car. If the car refused then it would be stuck in situations like this.
This sounds dangerous. You’ll eventually get used to the warnings when backing out and wont even check if they’re actually false alarms. Little Timmy becoming pancake mix as a result.
Collision warnings are a backup system. The driver is also expected to look where they're going. Little Timmy will be just fine, unless he made little Sally cry, in which case he's fair game.
I've wondered the same thing. The only thing I can think of is some kind of god awful false positive rate. Which doesn't really make sense either considering the technology we have today.
