Download PDF
Case Studies > How we built BI on Clickhouse with row-level security in Deutsche Bank Technology Centre

How we built BI on Clickhouse with row-level security in Deutsche Bank Technology Centre

Technology Category
  • Analytics & Modeling - Big Data Analytics
  • Analytics & Modeling - Predictive Analytics
  • Application Infrastructure & Middleware - Data Exchange & Integration
  • Application Infrastructure & Middleware - Database Management & Storage
Applicable Industries
  • Finance & Insurance
Applicable Functions
  • Business Operation
  • Quality Assurance
Services
  • Data Science Services
  • Software Design & Engineering Services
  • System Integration
The Challenge
Deutsche Bank Technology Centre faced significant challenges in managing and analyzing data within its Investment Bank division. The natural data silos and the need for a robust Business Intelligence (BI) system were evident. The existing data warehouse solutions were either too expensive or too slow, such as Vertica and Hive, respectively. Additionally, the bank required a data-driven access control mechanism that could provide record-level granularity and full access to SQL, while reusing existing bank-wide access rules. The challenge was to find a solution that could handle heterogeneous data from 72 different data sources and systems, manage over 100 ETL jobs, and provide a seamless user experience across various UIs.
About The Customer
Deutsche Bank Technology Centre is a part of Deutsche Bank, one of the world's leading financial service providers. The Technology Centre focuses on developing and implementing cutting-edge technology solutions to support the bank's operations and services. With a global presence and a diverse range of financial products, Deutsche Bank serves millions of customers worldwide. The Technology Centre plays a crucial role in ensuring the bank's technological infrastructure is robust, secure, and capable of handling the complex demands of the financial industry. The centre is responsible for managing data, developing software solutions, and ensuring compliance with regulatory requirements.
The Solution
To address these challenges, Deutsche Bank Technology Centre implemented a combination of Spark, ClickHouse, and Tableau/RShiny for their BI needs. In 2017, they adopted Spark for its powerful data processing capabilities and ClickHouse for its high-performance columnar storage. Tableau and RShiny were used for data visualization and reporting. By 2018, the solution evolved to include Alpakka for data integration, Kafka for real-time data streaming, and a Web UI for user interaction. The Access-Based Access Control (ABAC) system was integrated to provide data-driven access control with record-level granularity. This comprehensive solution allowed the bank to track changes, manage investment planning, and improve client interaction quality. The use of ClickHouse enabled fast query performance, while Spark and Kafka ensured efficient data processing and real-time analytics.
Operational Impact
  • The implementation of ClickHouse and Spark significantly improved the bank's ability to manage and analyze data from multiple sources.
  • The integration of ABAC provided a robust security mechanism, ensuring data-driven access control with record-level granularity.
  • The use of Tableau and RShiny for data visualization enhanced the reporting capabilities, making it easier for users to interact with the data.
Quantitative Benefit
  • Managed data from 72 different data sources and systems.
  • Handled over 100 ETL jobs efficiently.
  • Utilized 4 servers with a total storage capacity of 500GB/26TB.

Related Case Studies.

Contact us

Let's talk!

* Required
* Required
* Required
* Invalid email address
By submitting this form, you agree that IoT ONE may contact you with insights and marketing messaging.
No thanks, I don't want to receive any marketing emails from IoT ONE.
Submit

Thank you for your message!
We will contact you soon.