下载PDF
Leveraging ClickHouse Kafka Engine for Enhanced Data Collection and Analysis: A Case Study of Superology
技术
- 分析与建模 - 机器学习
- 应用基础设施与中间件 - 事件驱动型应用
适用行业
- 建筑物
- 建筑与基础设施
适用功能
- 质量保证
用例
- 实验自动化
- 时间敏感网络
服务
- 系统集成
- 测试与认证
挑战
Superology 是体育博彩行业领先的产品技术公司,面临着有效收集和分析定量数据以改善客户体验和业务运营的挑战。该公司需要收集应用程序或网站访问量、特定页面上的客户点击量、社交部分中的评论和关注者数量以及各种转化事件和跳出率等指标。收集的数据结构各异,需要采用动态方法进行数据收集和分析。 Superology 使用 Google Protocol Buffers (Protobuf) 来收集这些数据,但需要更高效和可扩展的解决方案来处理大量数据及其动态特性。
关于客户
Superology是一家经验丰富的产品科技公司,自2012年以来一直在体育博彩行业进行创新。2017年被Superbet集团收购,它已成为该行业的主导力量,其平台有数十万人使用,处理数以百万计的数据每日交易量。 Superology 在各个工作层面都使用基于数据的方法来满足用户需求并实现业务目标。公司与公司发展一样重视个人成长,并授权员工充分发挥自己的才能并掌控自己的工作。
解决方案
Superology 采用 ClickHouse Kafka Engine 和 Protocol Buffers 来增强其数据收集和分析过程。 ClickHouse 凭借其内置的 Kafka 连接器和 Protobuf 输入类型,提供了快速可靠的解决方案。该公司能够轻松水平和垂直扩展其 ClickHouse 实施。收集到的数据被提取到一个“大”原始表中,ClickHouse 柱状结构提供了很好的可扩展性。 Superology 还实现了一个系统,允许更改其原型方案,确保向后兼容性。然后使用物化视图对数据进行过滤和转换,从而能够对客户行为的特定方面进行集中分析。 ClickHouse 还凭借其广泛的统计功能,促进了高效的 AB 测试和其他实验。 Superology 计划通过与 MindsDB 耦合来进一步丰富其 ClickHouse 架构,以创建数据库级别的机器学习架构。
运营影响
数量效益
相关案例.
Case Study
Energy Saving & Power Monitoring System
Recently a university in Taiwan was experiencing dramatic power usage increases due to its growing number of campus buildings and students. Aiming to analyze their power consumption and increase their power efficiency across 52 buildings, the university wanted to build a power management system utilizing web-based hardware and software. With these goals in mind, they contacted Advantech to help them develop their system and provide them with the means to save energy in the years to come.
Case Study
IoT System for Tunnel Construction
The Zenitaka Corporation ('Zenitaka') has two major business areas: its architectural business focuses on structures such as government buildings, office buildings, and commercial facilities, while its civil engineering business is targeted at structures such as tunnels, bridges and dams. Within these areas, there presented two issues that have always persisted in regard to the construction of mountain tunnels. These issues are 'improving safety" and "reducing energy consumption". Mountain tunnels construction requires a massive amount of electricity. This is because there are many kinds of electrical equipment being used day and night, including construction machinery, construction lighting, and ventilating fan. Despite this, the amount of power consumption is generally not tightly managed. In many cases, the exact amount of power consumption is only ascertained when the bill from the power company becomes available. Sometimes, corporations install demand-monitoring equipment to help curb the maximum power demanded. However, even in these cases, the devices only allow the total volume of power consumption to be ascertained, or they may issue warnings to prevent the contracted volume of power from being exceeded. In order to tackle the issue of reducing power consumption, it was first necessary to obtain an accurate breakdown of how much power was being used in each particular area. In other words, we needed to be able to visualize the amount of power being consumed. Safety, was also not being managed very rigorously. Even now, tunnel construction sites often use a 'name label' system for managing entry into the work site. Specifically, red labels with white reverse sides that bear the workers' names on both sides are displayed at the tunnel work site entrance. The workers themselves then flip the name label to the appropriate side when entering or exiting from the work site to indicate whether or not they are working inside the tunnel at any given time. If a worker forgets to flip his or her name label when entering or exiting from the tunnel, management cannot be performed effectively. In order to tackle the challenges mentioned above, Zenitaka decided to build a system that could improve the safety of tunnel construction as well as reduce the amount of power consumed. In other words, this new system would facilitate a clear picture of which workers were working in each location at the mountain tunnel construction site, as well as which processes were being carried out at those respective locations at any given time. The system would maintain the safety of all workers while also carefully controlling the electrical equipment to reduce unnecessary power consumption. Having decided on the concept, our next concern was whether there existed any kind of robust hardware that would not break down at the construction work site, that could move freely in response to changes in the working environment, and that could accurately detect workers and vehicles using radio frequency identification (RFID). Given that this system would involve many components that were new to Zenitaka, we decided to enlist the cooperation of E.I.Sol Co., Ltd. ('E.I.Sol') as our joint development partner, as they had provided us with a highly practical proposal.
Case Study
Intelligent Building Automation System and Energy Saving Solution
One of the most difficult problems facing the world is conserving energy in buildings. However, it is not easy to have a cost-effective solution to reduce energy usage in a building. One solution for saving energy is to implement an intelligent building automation system (BAS) which can be controlled according to its schedule. In Indonesia a large university with a five floor building and 22 classrooms wanted to save the amount of energy being used.
Case Study
Powering Smart Home Automation solutions with IoT for Energy conservation
Many industry leaders that offer Smart Energy Management products & solutions face challenges including:How to build a scalable platform that can automatically scale-up to on-board ‘n’ number of Smart home devicesData security, solution availability, and reliability are the other critical factors to deal withHow to create a robust common IoT platform that handles any kind of smart devicesHow to enable data management capabilities that would help in intelligent decision-making
Case Study
Splunk Partnership Ties Together Big Data & IoT Services
Splunk was faced with the need to meet emerging customer demands for interfacing IoT projects to its suite of services. The company required an IoT partner that would be able to easily and quickly integrate with its Splunk Enterprise platform, rather than allocating development resources and time to building out an IoT interface and application platform.