Skip to content

    Avaneesh Kumar Yadav · Engineering Manager

    Building AI.
    Scaling Systems.
    Leading Engineering.

    Engineering lessons from 12+ years in enterprise AI and cloud systems: how production AI is designed, run and led. Written as articles, courses and small tools you can use.

    Articles

    Architecture, trade-offs and what goes wrong, written for engineers who have to run these systems.

    All articles

    Architecture lab

    Diagrams and decision frameworks

    Reference shapes for production AI systems, and the questions that decide between the common options.

    From request to response

    Illustrative architecture
    Request path for an LLM-backed featureA client calls an API gateway that handles authentication and rate limits. An orchestrator builds the prompt and routes the request. It calls a retriever for context and a model provider for the answer. The answer passes through output checks for schema and policy before returning to the client. A band along the bottom shows that traces, cost and evaluation samples are recorded for every step.ClientAPI gatewayauth · rate limitsOrchestratorprompt · tools · routingRetrieverindex · access filtersModel providerprimary · fallbackOutput checksschema · policyObservability across every step: traces · prompts and outputs · cost per request · evaluation samples
    Illustrative architecture: a common reference shape, not a description of any specific system.

    Failure scenarios to design for: the retriever returns nothing relevant; the primary model is slow or unavailable; the output fails the schema or policy check; a prompt change quietly lowers quality. Each needs a defined behaviour, not a hope.

    An agent you can stop

    Illustrative architecture
    Agent loop with limits and a human approval gateA goal feeds a planning step. Before a risky action runs, an approval gate asks a person to confirm. The approved action runs as a tool call, and the result is observed. A decision step either finishes or sends the agent back to planning. A band along the bottom shows that every loop is bounded by a maximum number of steps, a budget and a timeout.GoalPlannext stepApproval gaterisky actions onlyTool callleast privilegeObservevalidate the resultDecidedone, or replanEvery loop is bounded: maximum steps · token and cost budget · timeout · full trace of each decision
    Illustrative architecture: the control points an agent needs, not a description of any specific system.

    Failure scenarios to design for: the loop never terminates; a tool call has more privilege than the task needs; an irreversible action runs unreviewed; a wrong observation is trusted and compounds.

    Retrieval, long context or fine-tuning?

    A way to ask the right questions, not a benchmark. Your data, volume and constraints decide.

    The knowledge changes often, or is private to you.
    Retrieve it at request time. Keep the index easy to update, filter by who is asking, and show the sources.
    The whole body of knowledge is small, stable and fits comfortably in the context window.
    Try putting it in the prompt first. Add retrieval only when it stops fitting or the cost becomes a problem.
    You need a consistent format, tone or behaviour that prompting does not deliver.
    Consider fine-tuning, but only after an evaluation set shows that prompting and retrieval fall short.
    You cannot yet say how you would know it works.
    Build the evaluation set first. Every other choice here depends on it.

    Tools

    Small tools that do one job

    Free calculators and advisors for AI engineering decisions. They run in your browser or call a small API.

    All tools
    Live · Powered by Llama 3.1

    Try the AI demo

    Ask an enterprise-AI question and see a live, RAG-style answer — a small demo of the patterns I write about.

    Click an example question or type your own above.

    Production-grade answers in seconds.

    Powered by Groq · Llama 3.1-8B-Instant · A live demo, not professional advice

    Learning instead? Free courses and Claude certification practice exams.

    About

    Avaneesh Kumar Yadav

    Engineering Manager

    12+ years in enterprise AI and cloud systems. I write about how production AI is designed, run and led, and turn it into free courses and small tools.

    Newsletter

    Scaled AI Weekly

    A short weekly email on building, running and leading production AI. New articles and what changed in the courses, nothing else.

    Free. We email a confirmation link first, and you can unsubscribe from any email.

    Abuildingai.in

    Enterprise AI architecture, engineering leadership, and production patterns — from 12+ years of shipping at scale.

    Enterprise AI Readiness Assessment — Featured on Product Hunt

    Scaled AI Weekly

    A short weekly email on building production AI.

    © 2026 Avaneesh Kumar Yadav · Engineered with intent.PrivacyTerms