Principal Engineer, System Software Platform Engineering

ABOUT CLIENT

Our client is a leading technology company specializing in graphics processing units (GPUs) and artificial intelligence (AI).

JOB DESCRIPTION

Create and manage a platform for AI that provides services for multiple users, handles identity and policy management, configures quotas, and controls costs. Additionally, this platform should offer easy paths for teams to work on AI projects.
Oversee the deployment of AI models at scale, including routing, autoscaling, and implementing safety measures to ensure reliability and observability.
Manage GPU resources in a Kubernetes environment, including device plugins, feature discovery, and scheduling strategies, among other responsibilities.
Take charge of the entire lifecycle of GPUs, ensuring that driver, firmware, and runtime updates are implemented safely and consistently.
Implement virtualization strategies for GPU resources, such as vGPU and PCIe passthrough, while defining policies for resource placement, isolation, and preemptive actions.
Establish secure traffic and networking protocols, including gateways, service mesh, and authentication/authorization measures.
Enhance observability and operational efficiency through monitoring tools for GPUs, response protocols for incidents, and optimization of costs.
Develop reusable templates, integrate SDKs and CLIs, and implement infrastructure-as-code standards for the platform.
Influence the platform's direction by creating design documents, mentoring engineers, and aligning platform development with the needs of AI products.

JOB REQUIREMENT

Requires a minimum of 15 years' experience in building and operating large-scale distributed systems or platform infrastructure, with a strong track record of shipping production services.
Proficiency in one or more of Python, Go, Java, C++, with a deep understanding of concurrency, networking, and systems design.
Expertise in containers, orchestration, Kubernetes, cloud networking, storage, IAM, and infrastructure-as-code.
Practical experience with GPU platforms, including Kubernetes GPU operations, scheduling, isolation, preemption, and utilization tuning.
Background in virtualization, including deploying and operating vGPU, PCIe pass-through, and mediated devices in production.
Experience in Site Reliability Engineering (SRE) or equivalent, including SLOs, incident management, performance tuning, resource management, and financial oversight.
Strong security mindset, with experience in TLS/mTLS, RBAC, secrets, policy-as-code, and secure multi-tenant architectures.
In-depth experience in GPU operations, including MIG partitioning, MPS sharing, NUMA/topology awareness, DCGM telemetry, GPUDirect RDMA/Storage.
Exposure to inference platforms, including serving runtimes, caching/batching, autoscaling patterns, and continuous delivery.
Exposure to agentic platforms, including workflow engines, tool orchestration, policy/guardrails for tool access, and data boundaries.
Experience in traffic/data plane technologies, such as gRPC, HTTP, Protobuf, service mesh, API gateways, CDN/caching, and global traffic management.
Proficiency with tools such as Terraform, Helm, GitOps, Prometheus, Grafana, OpenTelemetry, and policy engines; bare-metal provisioning experience is a plus.

WHAT'S ON OFFER

CONTACT

PEGASI – IT Recruitment Consultancy | Email: recruit@pegasi.com.vn | Tel: +84 28 3622 8666
We are PEGASI – IT Recruitment Consultancy in Vietnam. If you are looking for new opportunity for your career path, kindly visit our website www.pegasi.com.vn for your reference. Thank you!

Job Summary

Company Type:

Product

Technical Skills:

Devops, Backend, AI

Location:

Ho Chi Minh, Ha Noi - Viet Nam

Working Policy:

Onsite

Salary:

Negotiation

Job ID:

J01969

Status:

Close

Related Job:

PreSales Solutions Engineer

Ho Chi Minh - Viet Nam

Product

  • Presale
  • System
  • Google Cloud

PreSales Support: Collaborating with the Sales team to understand client needs and develop tailored solutions using Google Maps and Google Cloud services. This involves conducting technical presentations, product demonstrations, and creating proof of concepts (POCs) for prospective clients, as well as contributing to proposals and RFP responses with detailed technical information. Post-Sales Support: Leading the technical implementation of Google Maps and Google Cloud services, ensuring smooth deployment and integration. Providing ongoing technical support and troubleshooting for clients after implementation, working closely with cross-functional teams to ensure client satisfaction and build long-term relationships. Technical Expertise: Staying up-to-date with the latest Google Maps and Google Cloud technologies, serving as a subject matter expert (SME) for both internal teams and clients. Integrating new features and services into client solutions and providing guidance on best practices. Collaboration: Working closely with Sales, Product, Infrastructure, Data, and Engineering teams to align solutions with client needs and company goals. Mentoring junior team members and contributing to training initiatives.

Negotiation

View details

Technical Lead

Ho Chi Minh - Viet Nam

Product

  • NodeJS
  • Python

Leading the backend development team, providing technical direction, mentorship, and best practices. Designing and implementing scalable, secure, and high-performance microservices-based architectures. Architecting and implementing agentic AI workflows and RAG (Retrieval-Augmented Generation) systems for personalized user interactions and automated coaching features. Overseeing data pipelines and infrastructure required for real-time AI model inference within a microservices-based environment. Collaborating with stakeholders to align on requirements and delivery timelines. Optimizing application performance, monitoring system reliability, and proactively troubleshooting issues. Advocating for CI/CD pipelines, automated testing, and robust version control strategies. Documenting key architectural decisions, APIs, and processes for internal use.

Negotiation

View details

Featured Job

Chief Technology Officer

Ha Noi - Viet Nam

Product

  • Cloud
  • Backend

Planning & designing overall system architecture: Creating a Technology Roadmap for a Game Server system with high concurrency and low latency for global players. Cost optimization: Deciding on the strategy for using Cloud infrastructure (AWS, GCP, Azure) or Hybrid Cloud to balance performance and operational expenses. High-level consultation: Participating in the Executive Board to address the relationship between speed-to-market of features and system stability. Tech-stack selection: Evaluating and finalizing programming languages (Go, C++, Java, Node.js) and processing models (Microservices vs Monolith) suitable for the complex logic of the game. Scalability solution: Directing the development of Auto-scaling, Load Balancing mechanisms, and managing Player State on large clusters. Data management: Designing Database structure (SQL/NoSQL) and Cache system (Redis, Memcached) to handle billions of queries daily without congestion. Ensuring Uptime: Building real-time monitoring and alerting systems to maintain 99.99% Availability. Network security: Implementing solutions to combat DDoS attacks, game fraud (Anti-cheat), and comprehensive user data security. Infrastructure & CI/CD: Standardizing automatic deployment processes to ensure game updates (Hotfix/Update) do not disrupt players. Deployment strategy & Optimization: Developing plans to optimize Cloud Services costs (AWS/GCP/Azure), evaluating the use of Spot Instances, Reserved Instances, or Private Cloud solutions to save operational budget. Meanwhile, establishing 24/7 monitoring and incident response systems.

Negotiation

View details