Backend Engineer

Build the core gateway: a unified, OpenAI-compatible API in front of multiple providers (OpenAI, Anthropic, Google, plus self-hosted and OSS models).

Own provider routing and reliability: load balancing, automatic failover, and cost and latency-aware routing.

Build billing and metering that is correct, not approximate: per-request token accounting, usage ledgers, cost attribution per team, user, and key, budgets, and spend limits.

Ship org controls: API key management, per-team and per-user quotas, rate limiting, and RBAC.

Handle streaming and performance: low-overhead proxying, streaming responses, connection handling, and caching where it helps.

Contribute to the Jan Agent and connect it to the router: route its model and tool calls through the gateway, and make agent traffic first-class in metering, controls, and observability.

Make deliberate speed-versus-correctness calls: move fast where iteration is cheap, refuse to cut corners where a bug means a bad charge or a leaked key, and pay down debt on your own initiative.

Work agent-natively as your default, running multiple agents in parallel, pushing token throughput hard, with your own review and guardrail discipline, using open harnesses like Pi, Hermes Agent, and OpenCode as your daily drivers, on open and frontier models alike.

Acquainted with LLM API systems: providers, OpenAI-compatible endpoints, streaming, tool calling, and token accounting.

Experience with LLM infra: inference proxies, provider SDKs, token counting, or existing gateways and routing services (LiteLLM, cliproxyapi, 9router, Omnirouter, and similar) as reference points.

Acquainted with core AI concepts: context windows, inference, prompting, evals, and how open models differ from hosted providers in practice.

Proven ability to ship from zero to production and own the result, including infra, deploy, alerting, and CI/CD, regardless of which stack you did it in.

Open-minded and pragmatic: you weigh speed against correctness case by case, hold strong opinions loosely, and change course when the evidence says so, all while staying fast-moving and comfortable with full ownership and ambiguity.

 

A track record building production backend services that handle real traffic: APIs, auth, data modeling, deploy, and monitoring.

Experience with payments, billing, metering, or usage-based systems, or the rigor to build them correctly (idempotency, reconciliation, no dropped or double charges).

Comfort with high-throughput proxying, gateways, and streaming, and the performance concerns that come with them.

Solid datastore skills: PostgreSQL, Redis, queues where needed, and sound schema design for usage and billing data.

Strong in at least one of TypeScript or Python.

Comfort with Docker, Kubernetes, OAuth and OIDC, API keys, RBAC, tenant isolation, and secure secrets handling.

Familiarity with the Jan.ai ecosystem or other OSS LLM tooling.

MCP and tool-calling knowledge, directly relevant since you will help the router carry the Jan Agent's agent and tool traffic.

Go or Rust for high-performance proxying, not required.

Contributions to open agent tooling or harnesses: a Pi extension, a Hermes skill, an OpenCode plugin, or anything in the open-models ecosystem we can look at.

Collaborate with a world-class research team on meaningful, high-impact projects

Own and shape the core training code infrastructure used daily by the team

Work on real models, real data, and real scale - not toy problems

Help bridge the gap between research velocity and engineering quality

Flexible work environment with a culture that values depth, clarity, and curiosity

PEGASI – IT Recruitment Consultancy | Email: recruit@pegasi.com.vn | Tel: +84 28 3622 8666