Key Responsibilities
• Design, implement, and optimize data ingestion pipelines using Apache NiFi to handle sources like CSV and RDBMS, converting data to formats such as Parquet.
• Configure and manage a Spark standalone cluster for efficient data processing.
• Set up and maintain MinIO for object storage, including raw and processed buckets.
• Orchestrate end-to-end data workflows using Apache Airflow, integrating with Postgres for metadata storage and triggering NiFi flows and Spark jobs.
• Monitor system performance, logs, and health across nodes using built-in tools and optional monitoring services; ensure high availability and quick issue resolution.
• Work cross-functionally with other stakeholders to align infrastructure with business needs, including documentation and knowledge sharing.

Required Qualifications
• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field (Master's preferred).
• 3+ years of experience in data engineering, infrastructure, or operations roles, with a focus on building and maintaining data pipelines and systems.
• Proven hands-on experience with Apache NiFi for data ingestion and ETL processes.
• Strong expertise in Apache Spark (standalone or clustered) for distributed data processing.
• Proficiency with object storage solutions like MinIO (or S3-compatible systems) and database management using Postgres and Oracle.
• Experience with workflow orchestration tools such as Apache Airflow.
• Solid understanding of data formats (e.g., Parquet, CSV), data flows, and optimization techniques for performance and scalability.
• Knowledge of monitoring, logging, and troubleshooting in data environments.
• Excellent problem-solving skills, with the ability to handle complex, multi-node architectures.

• Working Hours: Saturday to Wednesday: 07:30AM to 04:14PM
• Flexible Hours: 1 hour of flexibility available
• Benefits: Seasonal gift packages and rewards
• Supplementary insurance
• Groceries
• Performance-based bonuses
• Workplace Location: Jordan, Soltani (Sayeh)