LINGO-1: Autonomous Driving Explains Itself
One of the most requested features in the robotaxi rides I took with around 300 people was that you should be able to talk to the car after all. Instead of changing the route or selecting music on the app, why not instruct the car to do so through a voice command?
Thanks to ChatGPT and other generative AIs, the wish seems to be coming true quickly. Engineers at London-based startup Wayve Technologies demonstrate how an autonomous vehicle uses text output to explain to passengers its individual maneuvers and the reasons for their choices. Thanks to the in-house development LINGO-1, the system comments on the current driving situation, as shown in the following video.
The technology is based on a so-called Vision-Language-Action Models (VLAMs), which, starting from large language models (LLMs), can be used not only to explain, but also to train robots – such as an autonomous car. Some of the maneuvers that the model then describes are as follows:
slowing down for a lead vehicle or a change in traffic lights,
changing lanes to follow a route,
accelerating to the speed limit,
noticing other cars coming onto the road or stopped at an intersection
approaching hazards such as roundabouts and Give Way signs,
parked cars, traffic lights or schools,
actions other road users are taking, such as changing lanes or overtaking parked vehicles,
cyclists and pedestrians waiting at zebra crossings or coming up from behind the car in a cycle lane.
With the training driving data, the data obtained from the sensors and linked to these instructions, new enriched data sets can be generated with which the VLAM can be trained.
This approach is still in its infancy, but developers expect it to have many benefits. For example, thanks to LLMs and VLAMs, robots of all kinds should be able to be trained much more quickly and automatically, and thus taught movement patterns. And most importantly, the AI can explain its actions, and that is another approach to creating explainable AI.
Wayve got even more examples on the VLAM in action in this blog post.
But beware. What you want – being able to talk to the robotaxi – may not always be want you wished for. As seen in this funny commercial….
In November my book (in German) on artificial intelligence and large language models will be released, where I go into more detail about generative AI and show many examples. It can already be pre-ordered. Here is more information:
KREATIVE INTELLIGENZ
Über ChatGPT hat man viel gelesen in der letzten Zeit: die künstliche Intelligenz, die ganze Bücher schreiben kann und der bereits jetzt unterstellt wird, Legionen von Autoren, Textern und Übersetzern arbeitslos zu machen. Und ChatGPT ist nicht allein, die KI-Familie wächst beständig. So malt DALL-E Bilder, Face Generator simuliert Gesichter und MusicLM komponiert Musik. Was erleben wir da? Das Ende der Zivilisation oder den Beginn von etwas völlig Neuem? Zukunftsforscher Dr. Mario Herger ordnet die neuesten Entwicklungen aus dem Silicon Valley ein und zeigt auf, welche teils bahnbrechenden Veränderungen unmittelbar vor der Tür stehen.
This article was also published in German.
