Job Specifications
Organizations everywhere struggle under the crushing costs and complexities of “solutions” that promise to simplify their lives. To create a better experience for their customers and employees. To help them grow. Software is a choice that can make or break a business. Create better or worse experiences. Propel or throttle growth. Business software has become a blocker instead of ways to get work done.
There’s another option. Freshworks. With a fresh vision for how the world works.
At Freshworks, we build uncomplicated service software that delivers exceptional customer and employee experiences. Our enterprise-grade solutions are powerful, yet easy to use, and quick to deliver results. Our people-first approach to AI eliminates friction, making employees more effective and organizations more productive. Over 72,000 companies, including Bridgestone, New Balance, Nucor, S&P Global, and Sony Music, trust Freshworks’ customer experience (CX) and employee experience (EX) software to fuel customer loyalty and service efficiency. And, over 4,500 Freshworks employees make this possible, all around the world.
Fresh vision. Real impact. Come build it with us.
Job Description
We are seeking a Backend AI Platform Engineer to design and build core backend services that power our Agentic AI Platform, enabling reasoning-driven agents, multi-agent orchestration, dialog management, and outcome-based workflows.
You will work closely with the Principal AI Agentic Architect to implement high-performance APIs, orchestration runtimes, agent lifecycle management, and integration with frameworks like LangChain, LangGraph, and LangSmith. This role is hands-on and ideal for someone passionate about building robust, scalable backend systems in a fast-paced, startup-like environment.
Key Responsibilities:
Platform Backend Development
Experience building agent runtime systems, orchestration layers, or reasoning engines.
Implement agent runtime orchestration services for reasoning, planning, and tool invocation (integrating LangChain/LangGraph).
Develop APIs and SDKs for building, deploying, and managing agent workflows and tools.
Build stateful dialog management and memory services to support multi-turn, context-aware agents.
Implement multi-agent communication (A2A) protocols and shared context for collaborative agent behavior.
System Architecture & Scalability
Work with the Architect to build cloud-native, event-driven backend services using microservices or service mesh architecture.
Design task scheduling and workflow orchestration (e.g., Temporal/Airflow) for agent planning and long-running operations.
Develop real-time message buses (Kafka/Pub-Sub) for agent communication and event propagation.
Optimize LLM orchestration performance and cost efficiency with caching, batching, and token usage monitoring.
Integrations & Data Services
Implement integrations with enterprise systems, APIs, and knowledge sources (REST, GraphQL, gRPC).
Work with vector DBs (e.g., ElasticSearch, Pinecone, Weaviate, FAISS) and RAG pipelines for knowledge-augmented reasoning agents.
Support metadata and telemetry pipelines for LangSmith-based evaluation and observability.
Security, Compliance & Governance
Build secure multi-tenant backend services with RBAC, API authentication, and tenant isolation.
Ensure agent action auditability and reproducibility for compliance and governance.
Collaborate on implementing platform guardrails (rate limiting, content filtering, safe tool usage).
Testing, Observability & Reliability
Write automated test suites (unit, integration, load testing) for backend services.
Implement logging, tracing, and metrics (OpenTelemetry, Prometheus, Grafana) for full observability.
Participate in on-call rotations and improve platform resiliency and fault tolerance.
Please note: This is a hybrid role that will be based in San Mateo, CA or Bellevue, WA and requires an in-office presence three days per week (Tuesday - Thursday).
Qualifications
Required Qualifications:
7+ years of backend engineering experience, with at least 2+ years in AI/ML platform development.
Strong programming skills in Java and Python
Hands-on experience with LangChain, LangGraph, LangSmith LLM/agent frameworks
Expertise in distributed systems, event-driven architectures, and microservices
Solid experience with cloud platforms (AWS), Kubernetes, and IaC (Terraform/Helm)
Familiarity with databases (PostgreSQL, Redis, MongoDB) and vector databases (ElasticSearch, Pinecone, FAISS, Weaviate)
Understanding of workflow orchestration (Temporal/Airflow) and messaging systems (Kafka, RabbitMQ)
Knowledge of AI/LLM (OpenAI, Anthropic, Azure OpenAI, etc.) and prompt/tool orchestration
Preferred Qualifications:
Exposure to conversation orchestration, dialog state tracking, and AI-driven planning systems
Familiarity with enterprise security standards (RBAC, SSO/OAuth2, data encryption)
Prior experience in startup or high-growth environments building SaaS platf
About the Company
Freshworks Inc. (NASDAQ: FRSH) builds uncomplicated service software that delivers exceptional customer and employee experiences. Our enterprise-grade solutions are powerful, yet easy to use, and quick to deliver results. Our people-first approach to AI eliminates friction, making employees more effective and organizations more productive.
Over 72,000 companies, including Bridgestone, New Balance, Nucor, S&P Global, and Sony Music, trust Freshworks' customer experience (CX) and employee experience (EX) software to fuel cust...
Know more