Principal Engineer, System Software Platform Engineering

ABOUT CLIENT

Our client is a leading technology company specializing in graphics processing units (GPUs) and artificial intelligence (AI).

JOB DESCRIPTION

Create and manage a platform for AI that provides services for multiple users, handles identity and policy management, configures quotas, and controls costs. Additionally, this platform should offer easy paths for teams to work on AI projects.
Oversee the deployment of AI models at scale, including routing, autoscaling, and implementing safety measures to ensure reliability and observability.
Manage GPU resources in a Kubernetes environment, including device plugins, feature discovery, and scheduling strategies, among other responsibilities.
Take charge of the entire lifecycle of GPUs, ensuring that driver, firmware, and runtime updates are implemented safely and consistently.
Implement virtualization strategies for GPU resources, such as vGPU and PCIe passthrough, while defining policies for resource placement, isolation, and preemptive actions.
Establish secure traffic and networking protocols, including gateways, service mesh, and authentication/authorization measures.
Enhance observability and operational efficiency through monitoring tools for GPUs, response protocols for incidents, and optimization of costs.
Develop reusable templates, integrate SDKs and CLIs, and implement infrastructure-as-code standards for the platform.
Influence the platform's direction by creating design documents, mentoring engineers, and aligning platform development with the needs of AI products.

JOB REQUIREMENT

Requires a minimum of 15 years' experience in building and operating large-scale distributed systems or platform infrastructure, with a strong track record of shipping production services.
Proficiency in one or more of Python, Go, Java, C++, with a deep understanding of concurrency, networking, and systems design.
Expertise in containers, orchestration, Kubernetes, cloud networking, storage, IAM, and infrastructure-as-code.
Practical experience with GPU platforms, including Kubernetes GPU operations, scheduling, isolation, preemption, and utilization tuning.
Background in virtualization, including deploying and operating vGPU, PCIe pass-through, and mediated devices in production.
Experience in Site Reliability Engineering (SRE) or equivalent, including SLOs, incident management, performance tuning, resource management, and financial oversight.
Strong security mindset, with experience in TLS/mTLS, RBAC, secrets, policy-as-code, and secure multi-tenant architectures.
In-depth experience in GPU operations, including MIG partitioning, MPS sharing, NUMA/topology awareness, DCGM telemetry, GPUDirect RDMA/Storage.
Exposure to inference platforms, including serving runtimes, caching/batching, autoscaling patterns, and continuous delivery.
Exposure to agentic platforms, including workflow engines, tool orchestration, policy/guardrails for tool access, and data boundaries.
Experience in traffic/data plane technologies, such as gRPC, HTTP, Protobuf, service mesh, API gateways, CDN/caching, and global traffic management.
Proficiency with tools such as Terraform, Helm, GitOps, Prometheus, Grafana, OpenTelemetry, and policy engines; bare-metal provisioning experience is a plus.

WHAT'S ON OFFER

CONTACT

PEGASI – IT Recruitment Consultancy | Email: recruit@pegasi.com.vn | Tel: +84 28 3622 8666
We are PEGASI – IT Recruitment Consultancy in Vietnam. If you are looking for new opportunity for your career path, kindly visit our website www.pegasi.com.vn for your reference. Thank you!

Job Summary

Company Type:

Product

Technical Skills:

Devops, Backend, AI

Location:

Ho Chi Minh, Ha Noi - Viet Nam

Working Policy:

Onsite

Salary:

Negotiation

Job ID:

J01969

Status:

Close

Related Job:

Senior Data Engineer

Ho Chi Minh - Viet Nam

Product

We're seeking a Staff Data Engineer to own and evolve our data infrastructure as we scale globally. You'll design and build the data systems that power our platform - from real-time pipelines and analytics infrastructure to the AI/ML foundations enabling intelligent insurance products.#Data Architecture & Engineering Design and implement scalable, future-proof data architectures aligned with business objectives across multiple regions and regulatory environments Build and maintain data pipelines for ingestion, transformation, and delivery using modern orchestration tools (Airflow, Spark, Kafka) Architect data solutions spanning data warehousing, data lakes, and real-time analytics Create and maintain data models (conceptual, logical, physical) using recognized modeling approaches Develop and document the enterprise data landscape, mapping data stores and flows across our microservices architecture#AI/ML Infrastructure Build and maintain data infrastructure supporting ML model training, deployment, and monitoring (MLOps) Design and implement vector database solutions for AI-powered features (e.g., MongoDB Atlas Vector Search, Pinecone, Weaviate) Develop data pipelines feeding recommendation engines, claims processing automation, fraud detection, and other AI-driven capabilities Ensure AI infrastructure scales globally while meeting data residency and compliance requirements#Data Operations & Quality Implement DataOps practices ensuring data quality, lineage, and governance across the platform Define and enforce data strategy and architectural principles across engineering teams Build monitoring and alerting for pipeline health, data quality, and SLA compliance Optimize query performance and cost efficiency across data systems#Technical Leadership Collaborate with product and engineering teams to translate business requirements into data solutions Act as a change agent, driving adoption of modern data practices across the organization Contribute to architectural reviews and technical decision-making Own data problems through to resolution

Negotiation

View details

Android Engineer - Hanoi

Ha Noi - Viet Nam

Product

  • Android

Creating and managing Android applications using Kotlin Constructing Android services for production use and contributing to live service operations Utilizing Jetpack Compose to design modern Android UI Incorporating asynchronous programming through Coroutines and Flow Developing scalable Android app architecture with modularization and dependency injection Collaborating with cross-functional teams through effective communication

Negotiation

View details

iOS Engineer - Hanoi

Ha Noi - Viet Nam

Product

  • iOS

Create and update iOS applications with Swift Utilize UIKit and SwiftUI for building user interfaces Integrate and design APIs for effective data processing Employ reactive and asynchronous programming for strong app architecture Communicate effectively with cross-functional teams Enhance code quality, performance, and maintainability of iOS applications

Negotiation

View details