Big Data Engineer

JOB DESCRIPTION

Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.

Implementing ETL process to transform data from OLTP databases to OLAP DB and Data Lake using event streaming platforms such as Kafka.

Develop, transform large datasets and maintain robust data pipelines that can support various use cases with high performance.

Monitoring performance and advising any necessary infrastructure changes.

Defining data retention, data governance policies and framework

JOB REQUIREMENT

At least 5 years experience in Java programming languages.

At least 5 years experience with Big Data, Java Spring, Kafka Streams, Spark Streams frameworks.

Experience in large scale deployment and performance tuning.

Experience with schema design and dimensional data modeling

Experience with non-relational and relational databases (MySQL, MongoDB)

Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

Experience with data pipeline and workflow management tool

Fluent written and spoken English.

Strong analytical and problem-solving skills.

Bonus Points if You

Have experience with Delta Lake technology.

Good Docker/Kubernetes knowledge is a plus.

Good Kibana, Elasticsearch ELK stack knowledge is a plus.

WHAT'S ON OFFER

We have a track record of success and a vision and a plan for a promising future. Our company has closed to 100% market share for player location regulatory compliance in the US gaming space. And we have fuelled that momentum with the expansion into new markets - media & entertainment and fintech.

We are proud of our values and we live them in all of our actions, conversations, and work: There’s always a way; Together we can do more; Aim higher. Then higher; Act with integrity; For the greater good.

We are proud to be part of a global team that develops award-winning solutions for some of the world’s largest and most innovative companies.

We will support you on your learning journey. We invest in employee career growth and development. Our learning & development commitment includes leadership and technical development, a substantial budget for education and training, as well as dedicated work hours for self-study.

We care about our team. Our team is talented, has a bias for action, and is known for their positive attitude and energy. Team members are generously rewarded with competitive salaries, incentives, and a comprehensive benefits package.

We care about giving back to the communities in which we live and work. We supports a broad range of community initiatives through donations and employee volunteer activities.

We know that work can be fun. We take the time to create employee events and experiences where everyone can connect and celebrate.

CONTACT

PEGASI – IT Recruitment Consultancy | Email: recruit@pegasi.com.vn | Tel: +84 28 3622 8666

We are PEGASI – IT Recruitment Consultancy in Vietnam. If you are looking for new opportunity for your career path, kindly visit our website www.pegasi.com.vn for your reference. Thank you!

Job Summary

Company Type:

Product

Technical Skills:

Data Engineering, Java

Location:

Ho Chi Minh - Viet Nam

Working Policy:

Job ID:

J01078

Status:

Related Job:

Senior Engineer Manager - Technology Risk

Ho Chi Minh - Viet Nam

Product

Management
Backend
Data Engineering
Devops

Develop and implement a cohesive roadmap for strategic execution based on the company's strategic ambitions Coordinate and oversee multiple engineering teams within the organization to achieve desired outcomes Collaborate with various internal teams, including technology, risk, business, and product teams, to ensure requirements are met before and after project delivery Take responsibility for both the development and ongoing operational support of applications in production Manage cyber security updates, ensuring software versions are up to date and infrastructure is patched regularly Monitor and control investment delivery to maintain alignment between different domains and optimize spending Facilitate collaboration between engineering and risk domain experts to achieve strategic technology goals

Negotiation

View details

Data Engineer

Ho Chi Minh - Viet Nam

Outsource

Data Engineering

Develop, test, and manage batch and streaming data pipelines on various cloud platforms like Snowflake, AWS Redshift, and Google Cloud Platform (BigQuery, Dataflow, Pub/Sub, Cloud Composer). Import data from various sources such as client APIs, SaaS platforms, event streams, and databases into secure cloud storage and warehouse layers. Create transformation logic using SQL and Python and deploy tested, well-documented models in dbt. Establish datasets and dashboards in Sigma for easy querying by business users and AI systems. Prepare and deliver data for AI applications, including feature tables, embeddings, and retrieval datasets, and provide access to data through controlled APIs and MCP servers. Utilize AI programming tools and automated workflows to enhance development efficiency, testing, and documentation, ensuring accuracy in the output. Set up data quality assessments, monitoring, and alerts, and develop automated procedures for diagnosing pipeline failures with AI support. Record details about pipelines, schemas, and data structures for the benefit of other teams and AI systems. Engage in code assessments and sprint planning within an agile, globally distributed team.

Negotiation

View details

Senior/ Lead Data Engineer

Ho Chi Minh - Viet Nam

Outsource

Data Engineering

Develop pipelines on various cloud platforms such as Google Cloud Platform, AWS, and Snowflake for large enterprises. Create data models using dbt and make them accessible through layers that can be easily queried by analysts and applications. Construct data products that support automated systems, including layers for retrieval, pipelines for embedding, and interfaces for tools that agents use. Create workflows where agents oversee pipelines, address issues, conduct tests, and suggest improvements. Collaborate with teams from different backgrounds to hasten pipeline development, testing, and migration from older systems, ensuring that all changes are thoroughly checked before implementation. Establish and uphold data engineering standards for automation, continuous integration/continuous deployment (CI/CD), infrastructure as code, testing, and monitoring through code reviews. Take charge of data quality, data lineage, security, and cost management on various platforms, ensuring proper access controls and audit trails. Provide guidance to Data Engineers, evaluate their work, and serve as the representative for the data engineering team in technical conversations. Transform client business needs into technical designs and estimates for new projects.

Negotiation

View details