AI-Driven Financial Insight Pipeline
Identified a critical gap — payment data locked in unstructured PDFs — and built an AI pipeline using Claude and Gemini APIs for extraction, Airflow-orchestrated dbt transforms in BigQuery, and automated variance analysis with narrative insight generation.
Problem
Critical payment data was locked in unstructured PDFs — invoices, bank statements, transaction records — that no existing pipeline could read. Finance teams spent 10+ hours per week manually extracting and reconciling this data, with no systematic forecasting or anomaly detection in place.
Solution
Built an LLM-powered extraction layer using Claude and Gemini APIs to parse unstructured financial PDFs into structured records. Airflow DAGs orchestrate the full pipeline: extract → validate → load to BigQuery → dbt transforms → automated variance analysis with narrative insight generation.
Architecture
Claude and Gemini APIs parse raw PDF content — invoices, statements, transaction records — into structured JSON payloads with merchant, amount, date, and category fields.
Apache Airflow DAGs manage the full pipeline: extraction, validation, BigQuery loading, and dbt model execution. Retries and alerting handle transient API failures.
dbt models in BigQuery define staging, intermediate, and mart layers. Dimensional modeling separates raw financial records from aggregated KPI and forecast tables.
Time-series models trained on historical payment data generate revenue forecasts with confidence intervals. Variance analysis highlights deviations from plan automatically.
LLM-generated narrative summaries explain key variances, flag anomalies, and surface revenue opportunities — replacing manual analyst commentary with automated reports.
Highlights
- +20% improvement in revenue forecast accuracy over previous manual process.
- +15% revenue identified through systematic analysis of previously unread PDF data.
- 10 hours per week recovered for the finance team — zero manual extraction.
- Claude and Gemini API extraction layer handles diverse PDF formats without schema changes.
- Airflow-orchestrated dbt pipeline with full data lineage and quality validation.
- Automated variance analysis with LLM-generated narrative commentary.