Waabi's Implementation of Neptune for Enhanced Experimentation Workflow and Resource Monitoring
- Analytics & Modeling - Machine Learning
- Sensors - Autonomous Driving Sensors
- Education
- Equipment & Machinery
- Product Research & Development
- Experimentation Automation
- Virtual Training
- Cloud Planning, Design & Implementation Services
- Training
Waabi, a company focused on developing the next generation of self-driving truck technology, faced a significant challenge in managing their large-scale experimentation workflow. Their Machine Learning teams, organized around different technical pillars, constantly launched experiments for different tasks, seeking model improvements by iteratively fine-tuning them and regularly comparing results against established benchmarks. The data involved in these experiments was diverse, including maps, LiDAR, camera, radar, inertial, and other sensor data. Keeping track of the data collected from these experiments and exporting it in an organized and shareable way became a challenge. The company also identified a lack of tooling for planning and building consistent benchmark datasets. They needed a solution that would allow them to share benchmark results in a constant place and format and retain data for later comparison after the end of a project.
Waabi is a company founded by AI pioneer and visionary Raquel Urtasun, with the goal of building the next generation of self-driving truck technology. They aim to develop a solution for self-driving trucks that can be used on a large scale. To achieve this, they use deep learning, probabilistic inference, and complex optimization to create software that is end-to-end trainable, interpretable, and capable of very complex reasoning. Their Machine Learning teams are organized around different technical pillars, each responsible for delivering technology for a different functional area. These teams are usually self-sufficient and able to deliver features and product capabilities from start to finish.
Waabi evaluated open-source solutions and well-known vendor products, looking for feature-rich experiment tracking, collaboration capabilities, and high-quality documentation. They chose Neptune, a tool that provided a single place to organize all the results, allowing everyone in the organization to access and share experiments within and across different teams. During the evaluation phase, Waabi worked closely with Neptune’s Sales and Product teams to find the best way to leverage the tool for their use case and collaborate on the roadmap. Neptune also provided resource monitoring capabilities, which were crucial for Waabi's data-heavy models. The tool allowed them to track utilization and improve it, monitor memory increases over time, and remotely stop cloud training jobs from the Neptune UI without navigating cloud infrastructure-oriented dashboards.