Data Engineer

Job Description

A. JOB PURPOSE:

• The Data Engineer is responsible for orchestrating data transformations and integrations using Microsoft Azure services to develop a comprehensive Data Lake.

• This role emphasizes advanced data engineering practices, including the utilization of Databricks and Azure Synapse Link, to support our operations across Vietnam, Cambodia, Indonesia, Myanmar, and other business units in Asia and other regions.

• This role enables advanced data analytics, reporting, and dashboard functionalities.

B. ACCOUNTABILITIES:

1. System Development and Maintenance:

• Design, develop, and maintain Azure Cloud data lakes, data warehouses, and both SQL and NoSQL databases.

• Implement and manage batch processing, streaming, and integration solutions using Databricks and Azure Synapse Link for data ingestion from various applications to data repositories.

• Optimize SQL queries and Databricks notebooks to reduce processing times across data lakes, data warehouses, and cloud data cubes.

• Automate data pipeline processes using Azure Data Factory, ensuring secure source data access and efficient data policy management.

• Collaborate with data visualization specialists, analysts, and IT teams to support backend data solutions that meet business reporting needs.

2. Performance and Quality Assurance:

• Ensure system stability and performance without impacting user experiences.

• Monitor databases and data processing activities to meet specific business scenarios.

• Ensure data accuracy and availability for ongoing data analytics projects.

• Manage and optimize costs related to data storage and processing services through efficient resource management and a pay-as-you-go approach.

• Maintain comprehensive documentation for data architecture, ingestion, processing, and operations.

• Manage source data access securely.

• Configure data access authentication and authorization.

• Manage and enforce data policies and procedure standard.

• Collaborate with Data Visualization and Analyst specialist, Data champion and IT team to ensure about data back-end can support building dashboard as business requested.

C. RESULTS:

• Ensure current systems can work smoothly and do not impact to performance.

• Data architect from design to real life operation.

• Ensure data availability and accuracy for further data analytics project.

• Optimize the cost of use for data storage and data processing services (impact to the amount of payment) on monthly basis by pay-as-you-go approach.

• Data catalogue documentation.

• Scripting and documentation for data ingestion, data processing, data serving.

• Data architect and operation can be extended to acquire other data in other BU, other division, new business…

• Data available timely and in good quality.

• Updated authorization matrix in quarterly basis.

Job Requirement

QUALIFICATIONS:

• Bachelor’s or Engineering degree in IT, Data Science

• Related field.MS Azure Data Engineer (MS Certified) is preferred

EXPERIENCES:

• Minimum 3 to 5 years experience in successfully role of Data Engineer

• Minimum 1 years experience in AI Generative

• Azure Synapse Link hands on eperience is an advantage

COMPETENCIES:

• Very strong in SQL-PL SQL-Spark SQL.

• Interaction with No SQL Databases

• Interaction with Event Hub, IoT Hub Databases

• Files Processing XML, CSV, Excel, Jason, etc. formats

• Structured and Non-Structured Data processing

• Data ETL Capabilities.

• Knowledge of Distributed Big Data Processing

• Data Warehouse and Data Lake Implementation experience in past.

• Interaction with REST APIs, Web Services.

• Proficiency in data programming languages such as Python, R, Java, C#, and tools like Jupyter, VS Studio Code.

LANGUAGES:

• Proficiency in English

OTHER REQUIREMENTS:

• Customer-focused with excellent verbal and written communication skills

• Ability to collaborate with globally diverse teams and work flexibly

• Strong analytical thinking and problem-solving skills

• A team player with a willingness to learn and adapt in a fast-paced environment