دیجی پی
دیجی پی

Data Engineer

Tehran/Vanak
Full Time
شنبه تا چهارشنبه
-
-

این فرصت شغلی چقدر برای من مناسب است؟

201 - 500 employees
Finance / Investment
Iranian company dealing only with Iranian entities
1397
Privately held
توضیحات بیشتر

key Requirements

3 years experience in similar position
Microsoft SQL Server - Advanced
PostgreSQL - Intermediate
Python - Advanced
Apache Kafka - Intermediate
Docker - Intermediate
Grafana - Basic
Apache Airflow - Intermediate

Job Description

Job Summary

We are looking for a skilled Data Engineer to join our team at Digify. In this role, you will design, develop, and optimize scalable data infrastructure and pipelines, ensuring high performance, reliability, and data quality.

You will work with large-scale datasets, real-time streaming systems, and distributed data processing technologies while collaborating with cross-functional teams.

Key Responsibilities

  • Design, develop, and maintain scalable ETL/ELT pipelines using Python.

  • Build and optimize large-scale data processing workflows using Apache Spark and PySpark.

  • Develop and manage real-time streaming pipelines using Apache Kafka and Debezium.

  • Manage, optimize, and troubleshoot PostgreSQL and ClickHouse databases.

  • Orchestrate data workflows using Apache Airflow or Dagster.

  • Maintain and improve data infrastructure using Linux, Docker, and/or Kubernetes.

  • Monitor system performance and troubleshoot production issues using Grafana or Zabbix.

  • Ensure data quality, consistency, reliability, and pipeline fault tolerance.

  • Contribute to data architecture design and distributed system optimization.

Requirements

  • 3+ years of professional experience in Data Engineering or related fields.

  • Strong proficiency in Python, OOP, and clean coding practices.

  • Advanced SQL skills and hands-on experience with PostgreSQL and ClickHouse.

  • Practical experience with Apache Spark/PySpark and familiarity with Apache Flink.

  • Experience with Apache Kafka, Debezium, and CDC concepts.

  • Experience with workflow orchestration tools such as Apache Airflow or Dagster.

  • Strong Linux administration and troubleshooting skills (LPIC-1 level or equivalent).

  • Hands-on experience with Docker or Kubernetes.

  • Familiarity with monitoring tools such as Grafana or Zabbix.

  • Solid understanding of batch processing, stream processing, and distributed systems.

  • Strong problem-solving skills, ownership, and ability to work independently and collaboratively.

Nice to Have

  • Experience designing end-to-end data platforms.

  • Knowledge of Data Warehousing, Data Lakes, and Lakehouse architectures.

  • Experience optimizing ClickHouse and implementing CDC with Debezium.

  • Familiarity with CI/CD, Git, and Infrastructure as Code.

  • Understanding of exactly-once processing, idempotency, schema evolution, and data contracts.

Job Requirements

Gender
Men / Women
Software
Python| Advanced Microsoft SQL Server| Advanced PostgreSQL| Intermediate Apache Kafka| Intermediate Apache Airflow| Intermediate Docker| Intermediate Grafana| Basic

ارسال رزومه برای دیجی پی

گزارش مشکل یا تخلف آگهی