Public case (sourced). Based on Uber Engineering's published post "QueryGPT - Natural Language to SQL Using Generative AI" (2024-09-19); every fact below is attributed to it.
This case belongs to Agents, Tools and MCP. The post is the only source. Where it says nothing, this page says "the post does not say", and the diagrams draw only the elements the post names.
Context
Uber's post says that SQL is used daily by engineers, operations managers and data scientists, and that writing queries takes SQL syntax and knowledge of Uber's internal data models. It says the data platform handles approximately 1.2 million interactive queries each month, and that the Operations organisation, one of the largest user cohorts, contributes about 36% of them.
The post says QueryGPT began as a proposal during Uber's Generative AI Hackdays in May 2023, and that there were about 20+ iterations of the algorithm. It describes two of them: a first version and the current design.
Problem statement
The post says authoring involves searching for relevant datasets in the data dictionary and then writing the query, and that each query can take around 10 minutes. It says QueryGPT can generate reliable queries in about 3 minutes. The post does not say how either figure was measured, and it states no other numeric target.
Constraints
Everything below is stated in the post.
- Schema size. Some tier 1 tables have schemas spanning over 200 columns, which could use 40-60K tokens in the request. The post says that having 3 or more of these tables would break the LLM call, since the largest available model at the time supported only 32K tokens.
- Coverage. Uber has hundreds of thousands of datasets with varying levels of documentation, so no evaluation set covers every business question.
- Non-determinism. Identical runs can differ; Uber does not over-index decisions on about 5% run-to-run changes in most metrics.
- Expectations. Users expect queries that just work; "The bar is high!"
- Not stated. The post does not say how access control or data privacy are handled, and it sets no latency or cost target.
Why the traditional approach fell short
Two approaches fell short according to the post: manual authoring (roughly 10 minutes per query, and it needs SQL and data-model knowledge) and the team's own first version, a simple retrieval-augmented design. The post says that version vectorised the user's prompt and ran a k-nearest-neighbour search over SQL samples and schemas to fetch 3 relevant tables and 7 relevant SQL samples, used 7 tier 1 tables and 20 SQL queries as its sample data set, and worked well for a small set of schemas. Accuracy declined as more tables and samples were onboarded.
The post names three problems: similarity search of a raw prompt against schema samples and SQL queries did not return relevant results; going from a natural language prompt to the right schemas was very difficult; and having 3 or more of the large schemas would break the LLM call. The post does not discuss other tools such as BI products or semantic layers.
Options considered
| Option | What the post says | Outcome |
|---|---|---|
| A. Write the SQL by hand (non-AI, the status quo) | About 10 minutes per query and needs knowledge of SQL and Uber's data models | The problem QueryGPT addresses |
| B. First version: embed the prompt, retrieve tables and samples, few-shot prompt | Fetches 3 tables and 7 samples from 7 tier 1 tables and 20 queries; good on a small set of schemas | Accuracy declined as more tables were onboarded; replaced |
| C. Current design: workspaces and a chain of narrow LLM agents | Intent agent, Table agent, Column Prune agent, described below | The design the post describes as current |
The post does not describe other options it considered.
Solution architecture
Figure 1. The current design as the post describes it
How to read this: the post states that every prompt runs through the intent agent first; it presents the table and column prune agents after it but does not state their order in so many words.
Figure 2. One question through the agents
How to read this: in this flow the user confirms the table list (the post also mentions a chat style mode where users can iterate on the generated query), and the last note marks the failure the post says it has not solved.
How AI solved it, step by step
Per the post, one request goes like this.
- The user writes a prompt, for example "Write a query to find the number of trips that were completed by Teslas in Seattle yesterday".
- The intent agent uses an LLM call to map the question to one or more business domains or workspaces, and through them to SQL samples and tables. The post says picking a business domain drastically narrowed the search radius for retrieval.
- A workspace is a curated collection of SQL samples and tables for a domain (the post lists Mobility, Ads and Core Services). Uber shipped 11 other system workspaces besides Mobility, and users can create custom ones.
- The table agent, another LLM agent, picks tables. The post says it was added after some users reported that the tables QueryGPT picked were not correct, and that users can confirm the list or edit it.
- The column prune agent uses an LLM call to prune irrelevant columns from the schemas.
- The output keeps the structure of the first version: a SQL query and an explanation of how it was generated.
The post does not say which model runs each agent or how long the chain takes.
Under the hood
The mechanism the post spells out is the token budget. Some tier 1 schemas span over 200 columns and could use 40-60K tokens, and the post says that 3 or more such tables would break the call because the largest available model at the time supported only 32K tokens. It then says Uber used GPT-4 Turbo with a 128K limit (1106) and still saw intermittent token limit issues for some requests, which led to the column prune agent. The post says pruning improved the token size and, by extension, the cost of each call, and reduced latency because inputs were smaller. It gives no figures.
An arithmetic check, which is this page's and not a figure from the post: three tables at the post's 40-60K range would use 3 x 40K = 120K to 3 x 60K = 180K tokens of schema, which can exceed a 128K limit before samples, instructions and the answer are counted.
In agent loops the context grows across calls; the lesson's section on context growth and compaction covers that case. The post describes something simpler: schemas are pruned once, before generation, and it does not call this compaction.
What went wrong / failure modes
The post reports these problems.
- Accuracy fell as tables were added, addressed by workspaces and the intent agent; wrong tables, which some users reported, addressed by the table agent with a confirm-or-edit step; token-size failures, addressed by the column prune agent.
- Hallucinated tables and columns. The post says the LLM sometimes generates queries with tables that do not exist, and that this is not completely solved. It mentions prompt changes and a chat mode, and says Uber is looking to add a "Validation" agent that recursively tries to fix hallucinations.
- Queries that run but return nothing. The post's example is a hallucinated filter such as status = "Finished" instead of "Completed", which is why its metrics count whether a run has output.
- Vague prompts. The post says a "prompt enhancer" or "prompt expander" was needed; it does not say whether one shipped.
- Similarity is not correctness. The post's example is a question that regularly fails because it omits a partition filter, although the LLM similarity score rates the query as similar to the golden SQL.
How the post evaluates. It builds a golden set from real questions in QueryGPT logs, with intent, schemas and SQL verified manually. A vanilla flow measures baseline performance; a decoupled flow supplies the correct intent and datasets, which the post describes as measuring performance with the human-in-the-loop experience and which lets each component be evaluated separately. Per question it records intent accuracy, table overlap (a 0 to 1 score; in the post's example two tables were required, one of the two was identified, and the score is 0.5), whether the query ran, whether it returned more than 0 records, and a similarity to the golden SQL that an LLM scores between 0 and 1.
Figure 3. The two evaluation flows
How to read this: the decoupled flow removes the dependency on earlier steps so one component can be measured on its own.
Results
Only the post's own numbers are used here. In a limited release to some Operations and Support teams, the post says QueryGPT is averaging about 300 daily active users, with about 78% saying the generated queries reduced the time they would otherwise spend writing queries from scratch. The post also estimates about 10 minutes to author a query by hand against about 3 minutes with QueryGPT.
The post does not publish accuracy figures (intent accuracy, table overlap averages, run success rates), latency, cost, or the size of the golden set. The 78% is user-reported, not a measured accuracy.
Tradeoffs and when NOT to use this approach
This section is this page's reading of the post, not statements by Uber.
- Each agent is another LLM call. The post says pruning cut latency and cost per call but gives no end-to-end figures.
- The first version worked well for a small set of schemas, so simple retrieval can be enough when there are few.
- The confirm-or-edit step keeps a human in the loop but asks the user to judge a table list.
- Silent errors, such as a query on the wrong table, matter most for users who cannot read SQL; the post advises targeting the right personas first.
- The post does not say how access control is handled, so a team with strict data permissions needs its own answer.
Design rules extracted
- Narrow the search space before retrieving. The post's workspaces and intent agent cut the retrieval radius.
- Give each LLM step one narrow job. The post says the LLMs worked well when given a small unit of specialised work.
- Add a human confirmation where users report errors, such as the table list.
- Cut the context before the expensive call; the post's column prune agent did this for schemas.
- Evaluate components separately on a golden set from real logs, check that queries run and return rows, and do not over-react to about 5% run-to-run noise.
Sources
- Jeffrey Johnson, Callie Busch and Abhi Khune, "QueryGPT - Natural Language to SQL Using Generative AI", Uber Engineering, 2024-09-19: https://www.uber.com/blog/query-gpt/ (accessed 2026-10-21, re-checked 2026-10-22). All facts above are from this post.
Last verified: 2026-10-22