Is RAG feasible for querying large databases?

19 October 2026 · Toni Martir, arquitecto de software

To answer questions about a database, the AI needs to know its tables: it uses them to write the SQL statement (the technical instruction that fetches the data). But every model has a context, the most text it can read at once, and company databases are big: those in Spider 2.0, one of the public benchmarks for these systems, often have more than a thousand columns. The whole schema does not fit.

One way out is RAG (retrieval-augmented generation): before each question, a search engine picks the tables that look most like what was asked, and only those reach the model. It often works. The question is what happens when it picks wrong, and whether there is something better.

Three ways to give the AI the schema

The whole database Every table at once. In a large database it does not fit in the context

Does not fit

RAG A search engine picks a few tables for each question, by how closely they match the words you use

Depends on the match

A domain sub-schema The tables of one area, such as sales or finance, with every field described. Whoever asks picks the area

Complete and fits

If the database is small, the whole schema fits and there is nothing to choose.

Why the search cannot find ROE

Someone in management asks: «what is our ROE this year?». ROE is return on equity, which the European Central Bank defines as «the ratio of total profit (loss) for the year to total equity». In the database, no table or field is called anything like it: profit and equity come out of the journal entries and the chart of accounts, which talk about entries, accounts, debits and credits.

The search compares the question with the names and descriptions of the tables, and «ROE» matches none of them. It will pick the closest ones, which are not the ones needed, and the model will write the statement with whatever it gets. Researchers call this table selection schema linking, and they warn that when it is imperfect it «can often exclude required columns needed for accurate query generation». And the model does not know what it is missing.

The domain sub-schema

The alternative is not to search at all. You prepare one sub-schema per domain (sales, purchasing, finance), each with its tables, every field described and the relationships between them. The person asking knows which domain their question belongs to and what they can ask about.

With the finance domain, ROE stops being a problem: the chart of accounts and the journal entries are already there, described, and the model knows what ROE is. It has the schema it needs without looking for it.

The vendors recommend the same. Databricks says «Aim for five or fewer tables» and that each assistant «should answer questions for a particular topic and audience, not general questions across various domains». Snowflake advises starting with «5–10 tables, one business domain» and calls descriptions «the single most important element for accuracy». And the authors of the schema linking paper do not select tables when the schema fits.

How much domain fits?

It depends on the model. Snowflake recommends staying under about 100,000 tokens (the chunks of text models count in) and allows larger schemas with large-context models. With a powerful model a broad domain fits; with one installed on a machine of your own, the limit is usually the context. I hit the same boundary asking local models for a whole report: how much you ask of them at once.

So is it feasible?

Yes: RAG can work for many questions. But it fails far more often than a sub-schema of the domain the question is about, because when the search misses a table, the model never finds out. With the domain set out in advance, the model has every table it needs and only has to understand the question.

At Reportman AI we work with predefined schemas by domain (sales, purchasing, finance), because the whole database does not fit in the context: each one brings its tables with every field described and their relationships, and the AI receives the one you pick in full, with no search in between. With a powerful model you can use a broader domain. With a local model, which can run on your own machine with no subscription (you supply the machine), Reportman AI lets you set the context size.

Sources

Fangyu Lei et al., Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows (arXiv, March 2025 version): «The databases in Spider 2.0 are sourced from real data applications, often containing over 1,000 columns».
European Central Bank, ECB publishes consolidated banking data for end-June 2021, note to chart 3: «Return on equity is defined as the ratio of total profit (loss) for the year to total equity.»
Karime Maamari et al., The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models (arXiv, August 2024): «imperfect schema linking can often exclude required columns needed for accurate query generation»; their method «entirely forgoes schema linking in cases where the schema fits within the model's context window».
Databricks, Curate an effective Genie Agent: «Aim for five or fewer tables.»; «An agent should answer questions for a particular topic and audience, not general questions across various domains.»
Snowflake, Best practices for modeling semantic views: «Start with 5–10 tables, one business domain, and a specific use case.»; «Descriptions are the single most important element for accuracy.»; «Keep the semantic view under about 100,000 tokens.»; and it prefers a larger view when «you’re using large context window models where pruning enables larger semantic views».
Reportman AI: home page and local AI.

Toni Martir, arquitecto de software · 19 October 2026 · sources checked on 19 October 2026