The mission of a data engineer is to design, develop, and implement the infrastructure and processes that enable organizations to collect, store, manage, analyze, and utilize their data effectively. They play a crucial role in transforming raw data into valuable insights that drive business decisions.

Responsibilities:

• Designing and building systems for collecting, storing, and analyzing data at scale.
• Acquire datasets that align with business needs.
• Collecting data from different data sources, including XML files, JSON files, REST APIs, NBI, etc.
• Develop algorithms to transform data into useful, actionable information.
• Build, test, and maintain ETL/ELT pipeline architectures.
• Collaborate with management to understand company objectives.
• Create new data validation methods and data analysis tools.
• Ensure compliance with data governance and security policies.