Game Server Monitoring

• Analytics & Modeling - Real Time Analytics
• Application Infrastructure & Middleware - Data Exchange & Integration

• Software

• Discrete Manufacturing
• Product Research & Development

• Predictive Quality Analytics
• Real-Time Location System (RTLS)

• Cloud Planning, Design & Implementation Services
• Data Science Services

EA DICE was preparing for the scale of traffic they expected following Battlefield V’s beta launch. They wanted to plan the launch to ensure stability and low latency and be confident that their customers could enjoy the new game with no interruptions. The game server team was continually on the lookout for a central log management solution to complement their infrastructure monitoring with Datadog, and they had evaluated a number of logging solutions. From the outset, they were only interested in a solution that could provide insight into their logs without their team needing to run and maintain the logging system or incur any other overhead. A second requirement was finding a cost-effective logging solution for game server monitoring due to the large volume of logs. Finally, they wanted a logging solution that integrated with everything in their tech stack.

EA Digital Illusions CE AB (EA DICE) is a video game development company based in Stockholm, Sweden. The company was founded in 1992 and has been a subsidiary of Electronic Arts since 2006. Its releases include the Battlefield, Mirror’s Edge, and Star Wars Battlefront series. The Battlefield series alone has been played by over 50 million people around the world. The latest PlayStation 4 version of Battlefield V sold 110,653 copies in Japan during its first week of release. DICE’s infrastructure is 100% on the cloud and completely Linux-based. They run everything using orchestration services such as Mesos and Apache Aurora. The game server architecture uses Amazon EC2 instances with custom dynamic scaling extensions to AWS. It depends on myriad backend services and microservices, including connections to MySQL, Cassandra, Redis, Amazon RDS, and Amazon S3 storage services.

DICE has long used Datadog to collect and monitor custom metrics. To prepare for this launch event, they adopted Datadog Log Management so they could seamlessly correlate infrastructure health with logs across their tech stack. Datadog’s scalable Log Management solution allowed DICE to send millions of logs per minute while prioritizing high-value ones. This was done cost-effectively, without any additional overhead for running and maintaining a separate logging system. Datadog’s solution decouples log ingestion and log indexing, thereby enabling DICE to collect all their logs and bring them into Datadog, without any fear of incurring an expensive log management bill. DICE’s engineers did not have to filter their logs upfront or remove any of the log content since neither the size of the log files nor the daily peaking ingested volume affected their indexing costs with Datadog. This incentivized DICE’s engineers to add more context or metadata to their logs that could help them when they were troubleshooting incidents.

• DICE is able to cost-effectively ingest and index high volumes of logs each day with Datadog.
• Battlefield has a global fan base that eagerly waits for every release, so DICE must be able to handle traffic peaks in the first 48 hours post-launch.
• Datadog’s easy-to-use UI, coupled with its intuitive search syntax and short learning curve, has helped increase user adoption among DICE’s engineers.

• During the beta, DICE anticipated that log volumes would exceed 12 billion log events per day.
• Traffic peaks in the first 48 hours post-launch, often with upwards of 10 million users.