Self-driving cars and the externalisation of danger (to other road users) - Thanks Mercedes Benz!

There's surely very few situations in which it would be certain for someone to die, in or out of the car.

Yes.

But these companies are tasked with creating software that will work in all scenarios including these super rare edge cases. How those systems are trained and programmed right now will shape how those systems perform in the wild in the future.

The question is about the super rare edge case... once a computer has enough data to determine that a severe accident is unavoidable and that the occupant is extremely likely to die, given the data it has about it's surroundings... what should it do in the milliseconds that remain in which it may be able to influence the future events?

Always save the occupant or minimise harm to the occupant? A fairly easy thing to write code for. Just disregard all external factors, be willing to kill or harm others.

Always minimise the external impact? A hard thing to do, and also a hard sales line... if you find yourself in one of these rare moments, this car may kill you.

But still... a self-driving car should have fewer of these moments anyway, so the real question should be how should a self-driving vehicle perform in a built-up area in which externalities are lives? In my opinion the vehicle needs to come to as much of a controlled stop as soon as possible, avoiding external impact where possible. In other words, it shouldn't prioritise or seek to protect the occupant beyond all of the existing safety measures for the occupant.

Meaning, if Mercedes cannot produce software that spares the occupant without also increasing the danger to those in the immediate vicinity, they need to increase the internal safety of the vehicle rather than externalise the danger.