Governed agentic RAG over annual reports on Databricks

The scenario

LedgerLens is a compliance-grade Q&A copilot for financial filings. It indexes the latest annual reports of 25 listed companies, retrieves the relevant passages for each question, and writes an answer where every sentence carries a numbered citation. When the evidence isn’t there it declines rather than guessing. The agent runs as a LangGraph graph on Databricks, with Unity Catalog governing the data, Vector Search handling retrieval, an AI gateway in front of the model, and Databricks Apps serving the UI. Deployment runs from GitHub Actions using workload identity — no long-lived cloud keys anywhere.

System architecture

High-level flow of LedgerLens, from input to output.

What you will learn

• Designing a retrieve → draft → verify graph that refuses when evidence is missing
• Governing documents, embeddings and access with Unity Catalog
• Putting a gateway in front of the LLM for rate limits, logging and model swaps
• Citation-level evaluation: groundedness, faithfulness and refusal accuracy
• Keyless CI/CD to Databricks with OIDC federation

What you will build

• Filing ingestion pipeline that chunks reports by section and tags company, year and page
• LangGraph agent with a citation verifier that strips unsupported claims
• Databricks App with an answer view that links every footnote back to the source page
• Evaluation suite with a golden set of answerable and unanswerable questions

Build plan

1. Ingest and section 25 annual reports into Delta
2. Index with Vector Search and build the retrieval node
3. Add the verifier and refusal path to the graph
4. Ship the Databricks App with clickable citations
5. Wire keyless deployment and the evaluation gate

Tech stack