TergeuPro 07 / 25
AI legal assistant (RAG over the Criminal Code)
A self-hosted assistant that answers investigators’ questions with citations to articles of the Criminal Code of Kazakhstan.
Problem
General chatbots hallucinate laws and can’t be used with sensitive case data in the cloud.
What it does
- Answers in Russian or Kazakh with references to specific articles
- Exact match when the question names an article number
- Health checks of LLM inference and RAG search
How it works
- Question → bge-m3 embedding (via Ollama) + BM25 keyword search over the knowledge base
- Candidates re-ranked by bge-reranker-v2-m3 (CrossEncoder on GPU)
- Top chunks + question → Qwen 2.5 72B via an OpenAI-compatible FastAPI proxy
- Answer streamed back with article references
Stack & models
PythonFastAPIOllamaQwen 2.5 72Bbge-m3bge-reranker-v2-m3BM25Yii2 API