Backend Engineer

ABOUT CLIENT

Our client is a leading research company specializing in technology innovation

JOB DESCRIPTION

Develop a unified, OpenAI-compatible API for various providers such as OpenAI, Anthropic, Google, and self-hosted and OSS models.
Manage provider routing, load balancing, automatic failover, and cost and latency-aware routing for improved reliability.
Establish precise billing and metering methods including per-request token accounting, usage ledgers, cost attribution per team, user, and key, as well as budgets and spend limits.
Introduce organization controls such as API key management, per-team and per-user quotas, rate limiting, and RBAC.
Address streaming and performance with low-overhead proxying, streaming responses, connection handling, and caching optimization.
Collaborate in integrating the Jan Agent with the router, ensuring that the model and tool calls are routed through the gateway and prioritize agent traffic in metering, controls, and observability.
Make well-considered speed-versus-correctness decisions, maintaining agility in iterations while refraining from cutting corners that could result in erroneous charges or key leaks, and taking initiative to address technical debt.

JOB REQUIREMENT

Work independently as your default, running multiple tasks concurrently, optimizing token throughput and implementing review and guardrail discipline, using open harnesses like Pi, Hermes Agent, and OpenCode on various models.
Familiar with LLM API systems including providers, OpenAI-compatible endpoints, streaming, tool calling, and token accounting.
Experience with LLM infrastructure such as inference proxies, provider SDKs, token counting, or existing gateways and routing services as reference points.
Understanding of core AI concepts like context windows, inference, prompting, evals, and differences between open models and hosted providers in practice.
Proven ability to take projects from inception to production, including infrastructure, deployment, alerting, and CI/CD, regardless of the technology stack.
Open-minded and pragmatic approach, balancing between speed and correctness, adapting to evidence, and comfortable with full ownership and ambiguity.
Track record developing production backend services handling real traffic, including APIs, authentication, data modeling, deployment, and monitoring.
Experience with payments, billing, metering, or usage-based systems, with emphasis on building them correctly.
Comfort with high-throughput proxying, gateways, and streaming, and consideration of associated performance concerns.
Strong knowledge of data stores like PostgreSQL, Redis, and queue systems, and robust schema design for usage and billing data.
Proficiency in at least one of TypeScript or Python.
Familiarity with Docker, Kubernetes, OAuth and OIDC, API keys, RBAC, tenant isolation, and secure secrets handling.
Knowledge of the Jan.ai ecosystem or other OSS LLM tooling.
MCP and tool-calling knowledge, relevant for assisting in carrying the traffic for Jan Agent's agent and tools.
Experience in Go or Rust for high-performance proxying, not mandatory.
Contributions to open agent tooling or harnesses within the open-models ecosystem.

WHAT'S ON OFFER

Join a renowned research team to work on impactful projects
Take ownership of the core training code infrastructure used by the team
Engage with real models, data, and scale, rather than small-scale problems
Contribute to bridging the gap between research velocity and engineering quality
Enjoy a flexible work environment with a culture that values depth, clarity, and curiosity

CONTACT

PEGASI – IT Recruitment Consultancy | Email: recruit@pegasi.com.vn | Tel: +84 28 3622 8666
We are PEGASI – IT Recruitment Consultancy in Vietnam. If you are looking for new opportunity for your career path, kindly visit our website www.pegasi.com.vn for your reference. Thank you!

Job Summary

Company Type:

Product

Technical Skills:

Typescript, NodeJS, Python, AI

Location:

Ho Chi Minh - Viet Nam

Working Policy:

Hybrid

Job ID:

J02226

Status:

Active

Related Job:

DevOps Engineer - FinOps

Ho Chi Minh - Viet Nam


Product

  • Devops

Analyze and optimize cloud costs through usage, performance, and architecture insights Identify, prioritize, and implement cost-saving opportunities with engineering teams Leverage IaC, automation, and CI/CD to drive efficient, scalable optimizations Track savings, monitor impact, and maintain cost visibility and reporting Promote cost-aware practices across engineering and operations teams

Negotiation

View details

DevSecOps Engineer

Ho Chi Minh - Viet Nam


Product

  • Devops
  • Security

Design and automate secure cloud infrastructure using IaC Embed security controls and scanning into CI/CD pipelines Manage secure Kubernetes and cloud environments Implement monitoring, logging, and incident response solutions Drive security best practices, compliance, and cost optimization Collaborate with teams to deliver secure, scalable systems

Negotiation

View details

[EMN] Senior IP Design Verification Engineer

Ho Chi Minh - Viet Nam


Outsource

  • Chip Verification

Lead pre-silicon Silicon IP verification from planning to sign-off Develop verification environments, testbenches, and coverage models Run simulations, debug issues, and drive root-cause analysis Track verification progress, risks, and quality metrics Ensure compliance with verification standards and best practices Mentor junior engineers and support technical reviews

Negotiation

View details