Download PDF
Pachyderm > Case Studies > How Pachyderm Is Used to Support Adarga in Analyzing Huge Volumes of Information
Pachyderm Logo

How Pachyderm Is Used to Support Adarga in Analyzing Huge Volumes of Information

Technology Category
  • Analytics & Modeling - Machine Learning
  • Analytics & Modeling - Natural Language Processing (NLP)
  • Platform as a Service (PaaS) - Data Management Platforms
Applicable Industries
  • National Security & Defense
  • Software
Applicable Functions
  • Product Research & Development
  • Quality Assurance
Services
  • Data Science Services
  • Software Design & Engineering Services
The Challenge
Adarga is an AI software development company that provides organizations with the capability to build and maintain a dynamic intelligence picture. Its AI analytics platform processes huge volumes of unstructured data, such as reports, global news feeds, presentations, videos, audio files, etc., at a speed unachievable by humans alone. The software extracts the essential facts in context and presents them in a comprehensible manner to unlock actionable insights at speed and enable more confident decision-making. However, the company faced challenges in developing, training, productionalizing, and scaling the necessary data models. They needed a solution that could drive data consistency, understand lineage, and enable model scaling.
About The Customer
Adarga is one of the UK’s leading developers of artificial intelligence software for Defense and National Security. Their powerful Knowledge Platform processes huge volumes of information—global news feeds, internal powerpoints and PDFs etc—at a speed simply unachievable by humans. Using state-of-the-art AI, the essential facts are extracted from the information, in context, and presented to the user in a variety of comprehensible formats, unlocking relevant, insightful and actionable intelligence at scale so organizations can mitigate risk, act at speed, and gain a competitive edge.
The Solution
Adarga chose Pachyderm as a key aspect of its MLOps to drive data consistency, understand lineage, and enable model scaling. Pachyderm provides clear understanding of data lineage during model experimentation, giving Adarga’s data scientists the insight needed for traceability and reproducibility. This effectively creates a controlled environment for Adarga, allowing the team to quickly assess and understand model development. Pachyderm also offers several key advantages for data processing. It only processes new data as it’s added rather than rerunning an entire data set, significantly decreasing overall processing times. It also allows teams to switch and scale data sets without impacting the underlying architecture. Pachyderm also speeds development by allowing the team to take advantage of parallel processing and GPU resource sharing.
Operational Impact
  • Adarga was able to use Pachyderm to split up pre-processing across multiple parallel pipelines, providing a 10-12x reduction in processing time.
  • Pachyderm has allowed Adarga to significantly narrow the gap between data science research and product development.
  • Pachyderm facilitates MLOps best practice by providing audit trails and traceability from production all the way back to training.
Quantitative Benefit
  • 10-12x Improvement in Processing Speed
  • Significant reduction in overall processing times due to Pachyderm only processing new data as it’s added rather than rerunning an entire data set.
  • Increased confidence within product development due to improved exposure of data science across the organization.

Related Case Studies.

Contact us

Let's talk!

* Required
* Required
* Required
* Invalid email address
By submitting this form, you agree that IoT ONE may contact you with insights and marketing messaging.
No thanks, I don't want to receive any marketing emails from IoT ONE.
Submit

Thank you for your message!
We will contact you soon.