Senior Machine Learning Engineer

Creating the V1 Evaluation Platform: You will be responsible for designing and building the core backend systems for our new LLM Evaluation Platform, using Arize Phoenix as the basis for traces, evaluations, and experiments.

Implementing Production Observability: You will need to architect and implement the observability backbone for our AI services by integrating Phoenix with OpenTelemetry to establish a centralized system for logging, tracing, and evaluating LLM behavior in production.

Standardizing LLM Deployment Pipeline: You will be in charge of designing and implementing the CI/CD framework for versioning, testing, and deploying prompt-based logic and LLM configurations, ensuring reproducible and auditable deployments across all AI features.

Providing Practical Solutions: Your role will involve making pragmatic technical decisions that prioritize business value and speed of delivery, in line with our early-stage startup environment.

Collaborating with Other Teams: You will work closely with the Data Science team to understand their workflow and ensure that the platform you build meets their core needs for experiment tracking and validation.

Establishing Core Patterns: You will also help in establishing and documenting the initial technical patterns for MLOps and model evaluation that will serve as the foundation for future development.

At least 5 years of software engineering experience, particularly in backend or platform systems.

Proficiency in Python and a track record of constructing reliable, testable, and maintainable production systems.

Substantial hands-on experience in productionizing and assessing modern ML/LLM systems, including familiarity with evaluation frameworks such as Arize Phoenix, LangSmith, or similar platforms.

Practical knowledge of modern observability frameworks, particularly OpenTelemetry.

Ability to apply practical and effective solutions to problems, emphasizing robustness while avoiding over-engineering.

Hands-on expertise with Arize Phoenix for LLM observability, including the setup of custom evaluators, dataset management, and implementation of trace-based debugging.

Familiarity with core AWS services in a platform context (Kubernetes/EKS, RDS, S3, IAM).

Previous experience in a startup environment and willingness to work in a fast-paced and ambiguous setting.

We offer a professional and enjoyable working atmosphere.

We prioritize your long-term development.

We are dedicated to creating a future-ready digital bank platform.

Competitive salary

13th-month salary guarantee

Performance bonus

Access to professional English courses

Premium health insurance

Generous annual leave allowance

PEGASI – IT Recruitment Consultancy | Email: recruit@pegasi.com.vn | Tel: +84 28 3622 8666