Download PDF
Contentsquare's Successful Migration from Elasticsearch to ClickHouse: A Case Study
Technology Category
- Cybersecurity & Privacy - Endpoint Security
- Cybersecurity & Privacy - Identity & Authentication Management
Applicable Industries
- Buildings
- Cement
Applicable Functions
- Procurement
- Quality Assurance
Use Cases
- Building Automation & Control
- Time Sensitive Networking
Services
- System Integration
- Testing & Certification
The Challenge
Contentsquare, a SaaS company, was facing significant challenges with its existing Elasticsearch setup. The company had 14 Elasticsearch clusters in production, each with 30 nodes. However, they were struggling with horizontal scalability, as they were unable to assemble larger clusters and maintain their stability for their workload. This limitation in cluster size meant that they could not handle any tenant that would not fit into a single cluster, severely restricting their ability to grow. The upper bound on the amount of traffic they could handle was slowing down the company's growth for technical reasons, which was unacceptable. They were left with two options: either find a way to host each tenant efficiently in a multi-cluster setup or migrate to a more scalable technology.
About The Customer
Contentsquare is a SaaS company that provides digital experience analytics. The company helps businesses understand how and why users are interacting with their app, mobile, and web sites. They compute billions of touch and mouse movements and transform this knowledge into profitable actions that increase engagement, reduce operational costs, and maximize conversion rates. Using behavioral data, artificial intelligence, and big data to provide automatic recommendations, Contentsquare empowers every member of the digital team to easily measure the impact of their actions and make fast and productive data-driven decisions to optimize the customer journey.
The Solution
Contentsquare decided to migrate to a more scalable technology and began looking into OLAP database engines that would meet their requirements of minimal latency for queries, a rich query language, efficiency with spinning disks, and simplicity in deployment and operation. After extensive engineering studies, they found that ClickHouse met all their requirements and began planning the migration. The migration process was divided into three phases: getting familiar with ClickHouse and building a new product with it, mirroring all the existing features with custom tooling to ensure no regression, and migrating their clients one by one. They built a new product on top of ClickHouse to familiarize themselves with the technology, then migrated their main product iteratively, rewriting each API endpoint one by one to use ClickHouse instead of Elasticsearch. Finally, they migrated their customers to the new infrastructure, taking care not to move everyone at once to identify potential issues.
Operational Impact
Quantitative Benefit
Related Case Studies.
Case Study
Energy Saving & Power Monitoring System
Recently a university in Taiwan was experiencing dramatic power usage increases due to its growing number of campus buildings and students. Aiming to analyze their power consumption and increase their power efficiency across 52 buildings, the university wanted to build a power management system utilizing web-based hardware and software. With these goals in mind, they contacted Advantech to help them develop their system and provide them with the means to save energy in the years to come.
Case Study
System 800xA at Indian Cement Plants
Chettinad Cement recognized that further efficiencies could be achieved in its cement manufacturing process. It looked to investing in comprehensive operational and control technologies to manage and derive productivity and energy efficiency gains from the assets on Line 2, their second plant in India.
Case Study
Intelligent Building Automation System and Energy Saving Solution
One of the most difficult problems facing the world is conserving energy in buildings. However, it is not easy to have a cost-effective solution to reduce energy usage in a building. One solution for saving energy is to implement an intelligent building automation system (BAS) which can be controlled according to its schedule. In Indonesia a large university with a five floor building and 22 classrooms wanted to save the amount of energy being used.
Case Study
Powering Smart Home Automation solutions with IoT for Energy conservation
Many industry leaders that offer Smart Energy Management products & solutions face challenges including:How to build a scalable platform that can automatically scale-up to on-board ‘n’ number of Smart home devicesData security, solution availability, and reliability are the other critical factors to deal withHow to create a robust common IoT platform that handles any kind of smart devicesHow to enable data management capabilities that would help in intelligent decision-making
Case Study
Commercial Building Automation Boosts Energy Efficiency
One of the challenges to building automation is the multitude of non-interoperable communications protocols that have evolved over the years. Buildings have several islands of automation. Bridging the islands of different automation without losing the considerable investment in each specialized control network is the main focus in this solution.
