Experts Doubt Ghost, Now Supported by OpenAI, Can Overcome Self-Driving Setbacks with LLMs

    Ghost Autonomy, a startup in the self-driving car industry, claims to have the answer to safer autonomous driving technology. The company plans to explore the use of multimodal large language models (LLMs), which are AI models that can understand both text and images, in self-driving cars. To achieve this, Ghost has partnered with OpenAI and received a $5 million investment. The idea is to use LLMs to enhance scene interpretation and make higher complexity road decisions based on images from car-mounted cameras.

    However, some experts are skeptical about Ghost’s approach. They argue that LLMs are not specifically designed or trained for self-driving purposes and may not be the most efficient solution to the challenges in autonomous driving. They compare the use of LLMs in self-driving to using a fancy solution for a simple problem.

    Despite the skepticism, Ghost and OpenAI remain convinced of the potential of multimodal models. They believe that these models can expand the applicability of LLMs to new use cases, including autonomous driving. By combining video, images, and sounds, multimodal models may provide a new way to understand and navigate complex and unusual driving environments.

    Ghost CEO John Hayes acknowledges that there is still work to be done to improve the reliability and performance of LLMs for autonomous driving. However, he believes that specific application-focused companies like Ghost, with their training data and deep understanding of the domain, can significantly enhance the existing general models. Ultimately, autonomous driving will require a complete system with multiple model types and functions, and multimodal models are just one tool to help achieve that.

    While Ghost’s vision sounds promising, the reality is that self-driving technology is still facing many challenges. Even well-financed companies like Cruise and Waymo have experienced setbacks after years of testing. Thus, it remains to be seen whether Ghost can deliver on its promises with unproven technology.

    In summary, Ghost Autonomy believes that multimodal large language models can improve the reliability and performance of self-driving cars. However, experts are skeptical, and the success of this approach remains uncertain in an industry facing notable challenges.

    Latest articles

    Related articles