The development of autonomous vehicles relies heavily on machine learning models that need extensive training to understand and navigate the complexities of the real world. To achieve this, companies like Lyft utilize sophisticated data curation platforms, often incorporating web-based tools to streamline the process and enhance efficiency.
Lyft’s journey into autonomous vehicle technology involved building a robust data curation platform from the ground up. In 2018, a dedicated team at Lyft embarked on a mission to revolutionize their data handling process. Self-driving test vehicles are equipped with a suite of sensors – radar, lidar, cameras, IMUs, and GPS – to gather comprehensive environmental data. Professional drivers operate these vehicles along predetermined routes, capturing vast amounts of data. This raw data then requires meticulous processing by software engineers to enable the hardware to interpret it effectively. Crucially, human curators play a vital role in annotating this information manually, essentially creating the datasets from which machine learning algorithms learn. This manual annotation process, using traditional industry tools, was often identified as a bottleneck, being both time-consuming and expensive.
Recognizing the inefficiencies, Lyft aimed to create in-house, web-based software tools with a focus on exceptional user experience. The goal was ambitious: to drastically improve data curation, saving significant time and costs without compromising the quality of the annotated data that is crucial for training self-driving car AI.
Initial research involved qualitative surveys, lab studies, and competitor analysis to deeply understand the data needs of their models and the intricacies of the curator annotation workflow. Observing data curators using existing industry-standard tools revealed significant pain points and areas ripe for improvement. Lyft identified opportunities to enhance both the user interface and the back-end data modeling capabilities of these tools, specifically within a web-based environment.
The challenge was significant as the market for lidar annotation tools, a critical component for self-driving car perception, was nascent. Inspiration was drawn from other complex data-driven industries that prioritize user-friendly interfaces, such as filmmaking, music production, software development environments (IDEs), 3D modeling software, and even sophisticated video games like Fortnite and SimCity. These diverse fields offered valuable insights into designing intuitive web-based tools for managing complex datasets.
Through brainstorming and iterative sketching, Lyft developed a clear understanding of the functionalities required for their web-based tools and how to build a scalable platform to manage tasks across large teams of data curators. The unique nature of each annotation task necessitated a high degree of user customization. Features like customizable tool layouts, multi-view editing, and precise camera angle controls within a 3D scene were implemented. User studies highlighted the importance of workflow customization, allowing curators to save presets and tailor the interface to their individual visual, cognitive, and functional specializations, ultimately boosting efficiency.
Furthermore, research strongly suggested that a keyboard-centric workflow would dramatically accelerate data curation. The aim was to minimize the need for curators to switch between keyboard and mouse, a frequent interruption observed with existing tools. This focus led to the development of a proprietary, highly efficient web-based workflow enabling rapid annotation of vast datasets. Lyft has since secured patents for four innovative concepts arising from this work, including US Patent 10909392 and US Patent 11151788, showcasing the novelty of their web-based solutions.
To effectively communicate design concepts and interaction flows, a mid-fidelity, end-to-end prototype of the lidar annotation workflow was built using Framer. This interactive web-based prototype allowed for rapid feedback from curators and facilitated on-the-spot iteration, significantly accelerating the design process.
This prototyping phase was crucial in solidifying key concepts and demonstrating the potential of their web-based data curation tools to engineering and product partners. Ultimately, this web-based prototype became the blueprint for all data curation tools within Lyft’s self-driving car division. It paved the way for a significantly faster and more efficient development cycle for data tools compared to previous approaches.
The suite of web-based tools developed based on these designs had a transformative impact, slashing average data curation time from 6 hours per dataset to a mere 45 minutes. This dramatic improvement translated into millions of dollars in annual savings for the company, highlighting the profound impact of well-designed web-based tools in the autonomous vehicle space.
Web-Based Autonomous Vehicle System Visualizations
Beyond data curation, ensuring the safety and accuracy of self-driving systems requires robust visualization tools. Lyft developed CarViz, a proprietary web-based tool, to provide engineers with a clear view of what their autonomous vehicles are sensing and how they are interpreting the environment. While an initial MVP of CarViz existed, it suffered from complexity and lacked a unified design vision.
Efforts were focused on simplifying the CarViz interface based on core principles developed in collaboration with the engineering team. These principles emphasized:
- Visual Fidelity: Representing real-world elements like roads, cars, and traffic lights in a recognizable manner within the web-based visualization.
- Machine Learning Integration: Leveraging machine learning insights to highlight crucial information dynamically for both engineers and vehicle operators.
- Contextual Communication: Utilizing color and visual cues to convey critical information effectively, such as potential hazards or sudden lane changes, within the web interface.
These design principles guided the engineering team in creating a more consistent and reliable web-based visualization platform. The improved clarity and intuitiveness of CarViz directly contributed to a safer and more performant autonomous vehicle system.
Developing CarViz involved significant learning in designing within 3D space and utilizing spatial and temporal cues effectively within a web environment. One notable innovation was a system for alerting vehicle operators to potential hazards outside their immediate field of view, drawing inspiration from off-screen indicators in games like Fortnite. This “vulnerable indicator,” visualized within the web tool, enhances safety by providing operators with crucial awareness of potential threats.
Web-Based Lyft Ride Experience
Expanding beyond internal tools, Lyft also recognized the need for a modern web-based experience for their core ride-sharing service. Historically, Lyft was primarily a mobile app, with a limited and outdated web presence.
A small team was formed to develop a minimum viable product for a revamped web experience, including responsive design. The goal was to test market response and explore the potential of web-based ride-sharing. Initial hypotheses centered on the idea that a modern web experience could significantly boost account creation and ride bookings. Furthermore, a flexible web platform could facilitate integration with third-party services like Yelp and Facebook Messenger.
Market research included auditing existing web-based ride-sharing applications and similar booking platforms across travel and hospitality industries. This analysis helped define user expectations and best practices for web-based interfaces.
Wireframing and prototyping were central to the development process, with close collaboration between designers and engineers. The initial focus was on creating a web-based “fare estimator” that users could access without needing a Lyft account. Account functionality and ride booking features were planned for subsequent phases.
Launched just two months after project inception, the initial web-based Lyft experience proved successful. User adoption was strong, and qualitative feedback confirmed a significantly improved user experience compared to the previous web tool. This success validated the importance of web-based access and paved the way for future web-based tools at Lyft.
Today, Lyft offers a comprehensive ride-booking experience on the web, a testament to the impact of this initiative and the power of web-based tools in expanding service accessibility and user engagement.
