Simon Data's Journey to Fast, Flexible, and Stable Embedded Dashboards

• Application Infrastructure & Middleware - Data Visualization
• Application Infrastructure & Middleware - Event-Driven Application

• Buildings
• Cement

• Product Research & Development
• Warehouse & Inventory Management

• Picking, Sorting & Positioning
• Time Sensitive Networking

• System Integration
• Testing & Certification

Simon Data, a SaaS company based in New York, operates a data platform that allows clients to manage their marketing data. The platform is powered by a complex, multi-tenant environment with datasets of varying lineages, schemas, and business purposes. Some datasets are common across client accounts, while others are client-specific. This setup, while beneficial for customers, posed a significant challenge for engineers trying to drive analytical insights across the system. The platform had several analytics products built into it, but the company wanted to consolidate them on a standard foundation and streamline the development and deployment processes. They aimed to build a framework that could be used for future analytics product development on any part of their core platform. The goals for the system included a seamless development experience, support for querying arbitrary data, fluency in managing various schemas, and an ability to rapidly prototype and develop the user experience. They also wanted to transparently present the queries and transformations used to produce results to facilitate QA by multiple stakeholders.

Simon Data is a SaaS company based in New York, with a team of 101-250 employees. The company operates a data platform that enables clients to access, operationalize, and centralize their marketing data. The platform is designed to provide marketers with an intuitive, non-technical platform to manage sophisticated data operations required to drive great customer marketing outcomes. The platform is powered by a complex, multi-tenant environment comprising datasets of varying lineages, schemas, and business purposes. Some datasets are common across client accounts, such as data produced or imported through marketing channel integrations, while other datasets are client-specific. This setup allows their customers to orchestrate bespoke end-user impressions.

Simon Data started by reviewing the analytics products they already had to gather requirements for the new reporting suite. They also reviewed technologies they could use to build a new reporting platform. After considering several commercial products and open-source projects, they chose Cube. Cube was designed for embedding analytics into an existing application and fit nicely into Simon Data's existing technology stack. It offered first-class support for the tools they use, pluggable authentication and schema configuration hooks for integration with the rest of their platform, and out-of-the-box support for deploying as a serverless application on AWS Lambda with API Gateway, ElastiCache, and Aurora/MySQL. Cube also provided a developer sandbox app, multiple layers of caching for performance, and an integrated framework for building frontend data components. Simon Data had to invest in integrating Cube with their infrastructure to provide credentials and schema configuration for each client's data warehouse. They also had to create a common application bootstrap and configuration for Cube that could be used in dev, staging, and production environments. They made schemas immutable to safely and securely deploy configurations to production without impacting what was already running.

• The integration of Cube into Simon Data's platform delivered three major wins: speed, flexibility, and stability. The company can now develop and ship analytics products without dealing with any new infrastructure development. The Cube Playground integration enables rapid schema and query building and implementation in the front end. The new framework allows seamless querying of any data in Snowflake and presenting it in the web application. The company can now deploy new product features with confidence that they will not introduce regression on existing functionality because their schemas are immutable. They can also roll out new features to client accounts in a controlled manner, one by one or in batches, and even keep the changes hidden to perform production validation before they are released to end users. The company has already built new analytics products on this platform that solve fundamental client pain points, proving that the platform works. They are also starting to realize the velocity gains when iterating on these products to add functionality and refine the user experience.