Lead Data Engineer

Design, create, and maintain scalable data infrastructure, which includes data lakes, pipelines, and metadata repositories, to ensure accurate and timely delivery of data to stakeholders.

Collaborate with data scientists to develop and maintain data models, integrate data sources, and facilitate machine learning workflows and experimentation environments.

Build and enhance large-scale, batch, and real-time data processing systems to improve operational efficiency and align with business goals.

Use Python, Apache Airflow, and AWS services to automate data workflows and processes, ensuring efficient scheduling and monitoring.

Utilize AWS services like S3, Glue, EC2, and Lambda to manage data storage and compute resources, striving for high performance, scalability, and cost-effectiveness.

Implement comprehensive testing and validation methods to guarantee the reliability, accuracy, and security of data processing workflows.

Keep updated on the latest industry best practices and emerging technologies in data engineering and data science to suggest innovative solutions and enhancements.

Core Expertise: Proficiency in Python for data processing and scripting (pandas, pyspark), workflow automation (Apache Airflow), and experience with cloud computing services (Glue, S3, EC2, Lambda).

Containerization & Orchestration: Experience working with Kubernetes and Docker for managing containerized environments in the cloud.

Data Engineering Tools: Hands-on experience with columnar and big data databases (Athena, Redshift, Vertica, Hive/Hadoop), along with version control systems like Git.

Cloud Services: Strong familiarity with cloud services for data processing and management.

CI/CD Pipeline: Experience with CI/CD tools for continuous integration and deployment.

Data Engineering Focus (75%): Expertise in building and managing robust data architectures and pipelines for large-scale data operations.

Data Science Support (25%): Ability to support data science workflows, including collaboration on data preparation, feature engineering, and enabling experimentation environments.

Langchain Experience: Familiarity with Langchain for building data applications involving natural language processing or conversational AI frameworks.

Advanced Data Science Tools: Experience with machine learning environments.

Big Data & Analytics: Familiarity with both RDBMS and NoSQL databases.

BI Tools: Experience with enterprise BI tools.

Messaging & Event Streaming: Familiarity with distributed messaging systems for event streaming.

Monitoring & Logging: Experience with monitoring and log management tools.

Data Privacy and Security: Knowledge of best practices for ensuring data privacy and security, particularly in large data infrastructures.

Offers competitive salary

Salary bands are reviewed annually

Provides 13th-month salary pro rata based on employee's length of service

Monthly lunch allowance of 700,000 VND per employee

Covers monthly parking fee for employee motorbikes

Conducts performance evaluation once a year for performance bonuses and salary increments

Provides private health insurance including accident, outpatient, in-patient, maternity, and dental for permanent employees after 2-month probation

Offers expense claim for eyewear and annual health check-ups

Provides a maximum of 18-day vacation leave per year, with the option to carry over 5 days until the following year

Grants an additional annual leave day for every two-year anniversary

Allocates annual fund for fitness activities based on team's vote

Offers a range of healthy snacks, tea, coffee, milk, and beer on tap

Conducts company townhall meetings every 6 weeks

Engages in CSR activities as per company's CSR guideline

Offers onsite tour/training courses at other offices and client's destination overseas where applicable.

PEGASI – IT Recruitment Consultancy | Email: recruit@pegasi.com.vn | Tel: +84 28 3622 8666