The scenario
ScholarSwarm is a production-grade research platform. Four agents turn any topic into a sourced report, every request passes through managed guardrails and an LLM gateway with fallback, and a three-tier memory (session, long-term and semantic cache) keeps it fast and consistent. Reports are scored automatically by an LLM judge, and a red-team dashboard runs jailbreak and injection attacks against the system to prove the defences hold. Infrastructure is Terraform; deployment is GitHub Actions.
System architecture
01 Topic User request
02 Guardrails Input/output checks
03 Search → Summarise Sourced notes
04 Write → Verify Report + fact check
05 Memory tiers Redis · pgvector · cache
06 Judge + red team Quality and safety
High-level flow of ScholarSwarm, from input to output.
What you will learn
- Defence-in-depth for agent systems
- Gateway-based model fallback and cost control
- Short-term, long-term and semantic-cache memory
- Automated red-teaming and LLM-as-judge scoring
- Infrastructure as code for AI workloads
What you will build
- Four-agent research pipeline
- Guardrail + gateway layer
- Three-tier memory
- Red-team dashboard and evaluation reports
Build plan
- Research pipeline with sources
- Guardrails and gateway
- Memory tiers
- Judge scoring and red-team runs
- Terraform + CI/CD