Data Engineer (Big Data MSO)

Job Description:

We are looking for a skilled Data Engineer to join our team and help us build and maintain our data pipelines and infrastructure. The ideal candidate should have a strong background in data engineering and be familiar with modern data tools and technologies.

Responsibilities:

• Design and build data pipelines and workflows to ingest, transform, and store large volumes of data.
• Develop and maintain data warehouses and data lakes to support data analysis and reporting.
• Collaborate with data analysts and scientists to understand their data needs and provide them with the necessary data sets and tools.
• Monitor and optimize the performance of our data systems to ensure efficient and reliable data processing.
• Develop and tune data pipelines throughout NiFi and be able to work with HBase, Hive and HDFS.
• Stay up-to-date with the latest developments in data engineering and bring new ideas and techniques to our team.

Qualifications:

• Bachelor's degree in computer science, Engineering, or a related field.
• Strong programming skills in at least one language (e.g., Python, Java, Scala).
• Experience with big data processing frameworks (e.g., Hadoop, Spark).
• Experience to work with NiFi, HBase, Hive and other big data relevant tools is an advantage.
• Familiarity with SQL and relational/non-relational databases.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.