Backend Engineer

Develop a unified, OpenAI-compatible API for various providers such as OpenAI, Anthropic, Google, and self-hosted and OSS models.

Manage provider routing, load balancing, automatic failover, and cost and latency-aware routing for improved reliability.

Establish precise billing and metering methods including per-request token accounting, usage ledgers, cost attribution per team, user, and key, as well as budgets and spend limits.

Introduce organization controls such as API key management, per-team and per-user quotas, rate limiting, and RBAC.

Address streaming and performance with low-overhead proxying, streaming responses, connection handling, and caching optimization.

Collaborate in integrating the Jan Agent with the router, ensuring that the model and tool calls are routed through the gateway and prioritize agent traffic in metering, controls, and observability.

Make well-considered speed-versus-correctness decisions, maintaining agility in iterations while refraining from cutting corners that could result in erroneous charges or key leaks, and taking initiative to address technical debt.

Work independently as your default, running multiple tasks concurrently, optimizing token throughput and implementing review and guardrail discipline, using open harnesses like Pi, Hermes Agent, and OpenCode on various models.

Familiar with LLM API systems including providers, OpenAI-compatible endpoints, streaming, tool calling, and token accounting.

Experience with LLM infrastructure such as inference proxies, provider SDKs, token counting, or existing gateways and routing services as reference points.

Understanding of core AI concepts like context windows, inference, prompting, evals, and differences between open models and hosted providers in practice.

Proven ability to take projects from inception to production, including infrastructure, deployment, alerting, and CI/CD, regardless of the technology stack.

Open-minded and pragmatic approach, balancing between speed and correctness, adapting to evidence, and comfortable with full ownership and ambiguity.

Track record developing production backend services handling real traffic, including APIs, authentication, data modeling, deployment, and monitoring.

Experience with payments, billing, metering, or usage-based systems, with emphasis on building them correctly.

Comfort with high-throughput proxying, gateways, and streaming, and consideration of associated performance concerns.

Strong knowledge of data stores like PostgreSQL, Redis, and queue systems, and robust schema design for usage and billing data.

Proficiency in at least one of TypeScript or Python.

Familiarity with Docker, Kubernetes, OAuth and OIDC, API keys, RBAC, tenant isolation, and secure secrets handling.

Knowledge of the Jan.ai ecosystem or other OSS LLM tooling.

MCP and tool-calling knowledge, relevant for assisting in carrying the traffic for Jan Agent's agent and tools.

Experience in Go or Rust for high-performance proxying, not mandatory.

Contributions to open agent tooling or harnesses within the open-models ecosystem.

Join a renowned research team to work on impactful projects

Take ownership of the core training code infrastructure used by the team

Engage with real models, data, and scale, rather than small-scale problems

Contribute to bridging the gap between research velocity and engineering quality

Enjoy a flexible work environment with a culture that values depth, clarity, and curiosity

PEGASI – IT Recruitment Consultancy | Email: recruit@pegasi.com.vn | Tel: +84 28 3622 8666