Download PDF
Databricks > Case Studies > Real-Time Analytics at Scale: Akamai's Transformation with Delta Lake
Databricks Logo

Real-Time Analytics at Scale: Akamai's Transformation with Delta Lake

Technology Category
  • Cybersecurity & Privacy - Cloud Security
  • Robots - Parallel Robots
Applicable Industries
  • National Security & Defense
  • Retail
Applicable Functions
  • Sales & Marketing
Use Cases
  • Real-Time Location System (RTLS)
  • Tamper Detection
Services
  • Cloud Planning, Design & Implementation Services
  • Cybersecurity Services
The Challenge
Akamai, a global content delivery network (CDN) provider, manages approximately 30% of the internet’s traffic through its 345,000 servers spread across more than 135 countries. In 2018, Akamai launched a web security analytics tool to provide its customers with a unified interface for assessing a wide range of streaming security events and perform real-time analysis. This tool ingests approximately 10GB of data related to security events per second, with data volumes increasing significantly during peak retail periods. The tool initially relied on an on-premises architecture running Apache Spark™ on Hadoop. However, Akamai faced challenges in meeting its strict service level agreements (SLAs) of 5 to 7 minutes from when an attack occurs until it is displayed in the tool. The company sought to improve ingestion and query speed to meet these SLAs and provide real-time data to its customers.
About The Customer
Akamai is a global content delivery network (CDN) provider that operates approximately 345,000 servers in more than 135 countries and over 1,300 networks worldwide. The company routes internet traffic for some of the largest enterprises in media, commerce, finance, retail, and many other industries. In addition to its CDN services, Akamai also provides cloud security solutions. In 2018, the company launched a web security analytics tool that offers its customers a single, unified interface for assessing a wide range of streaming security events and performing real-time analysis of those events. The tool helps Akamai's customers take informed actions in relation to security events in real time.
The Solution
After conducting proofs of concept with several companies, Akamai chose to base its streaming analytics architecture on Spark and the Databricks Lakehouse Platform. The web security analytics tool now ingests and transforms data, stores it in cloud storage, and sends the location of the file via Kafka. It then uses a Databricks Job as the ingest application. Delta Lake, the open source storage format at the base of the Databricks Lakehouse Platform, supports real-time querying on the web security analytics data and enables Akamai to scale quickly. Akamai also uses Databricks SQL (DBSQL) and Photon, which provide extremely fast query performance. The combination of Databricks’ streaming architecture, DBSQL, and Photon enables Akamai to achieve real-time analytics, translating to real-time business benefits.
Operational Impact
  • The transition to the Databricks Lakehouse Platform has significantly improved Akamai's ability to meet its strict SLAs and provide real-time data to its customers. The use of Delta Lake, an open-source storage format, has not only improved query performance but also enabled Akamai to scale quickly in response to an 80% increase in traffic and data over the past year. The use of Databricks SQL and Photon has further enhanced query performance, enabling Akamai to provide real-time analytics. The move to Databricks has also improved customer experience, with over 85% of queries now completing in under 7 seconds. This real-time data provision helps Akamai's customers stay vigilant and maintain optimal security configurations.
Quantitative Benefit
  • Min ingestion time reduced from 15 minutes to under 1 minute
  • Over 85% of queries have a response time of 7 seconds or less
  • 70% of security event data has been moved from on-prem architecture to Databricks

Related Case Studies.

Contact us

Let's talk!

* Required
* Required
* Required
* Invalid email address
By submitting this form, you agree that IoT ONE may contact you with insights and marketing messaging.
No thanks, I don't want to receive any marketing emails from IoT ONE.
Submit

Thank you for your message!
We will contact you soon.