About the Role

We are looking for a Data Engineer to build and maintain scalable batch and real-time data pipelines. You will work with high-volume data using Python, Kafka, CDC, ClickHouse, Spark, and modern data engineering tools.

Responsibilities:

• Design, develop, and maintain batch and real-time data pipelines
• Build event-driven pipelines using Kafka and CDC/Debezium
• Develop data processing services and APIs using Python and FastAPI
• Design and optimize ClickHouse databases and clusters
• Build streaming applications using Faust or similar frameworks
• Write and optimize complex SQL queries
• Design Fact/Dimension models, Star Schema, and data warehouse structures
• Monitor, troubleshoot, and optimize data pipelines and data systems
• Ensure data quality, reliability, and consistency
• Collaborate with backend, BI, and product teams to deliver data solutions

Requirements:

• 3+ years of experience in Data Engineering or a related field
• Strong Python programming skills
• Strong SQL skills, including complex queries, CTEs, window functions, joins, and query optimization
• Strong understanding of Apache Kafka, including topics, partitions, consumer groups, offsets, and Schema Registry
• Hands-on experience with CDC, preferably Debezium
• Understanding of ClickHouse, including MergeTree, ReplicatedMergeTree, Distributed tables, replication, sharding, and clustering
• Experience with Apache Spark, Apache Airflow, and FastAPI
• Experience with Faust or other stream-processing technologies
• Good understanding of Fact/Dimension modeling and Star Schema
• Experience with Git and Git-based development workflows