LegalMind: A Fine-Tuned Gemma-2-Based Legal Assistant for Indian Judiciary with RAG and Embedding Integration

Abstract

Legal research and case analysis remain labor-intensive tasks requiring domain expertise and significant time investment. LegalMind is an AI-driven legal assistant built to streamline and automate legal understanding through fine-tuned large language models (LLMs) and Retrieval-Augmented Generation (RAG) techniques. The system integrates Google’s text-embedding-004 for document-level embeddings and a fine-tuned Gemma 2 (9B) model for legal summarization and question answering. LegalMind is enhanced through a situation-based RAG pipeline that retrieves contextually relevant content from "The Bharatiya Nyaya Sanhita, 2023", ensuring precise legal responses grounded in statute. The model's performance is rigorously evaluated using standardized natural language generation metrics, including ROUGE-L, BLEU, METEOR, BERTScore, and F1-score. Evaluation is conducted against the pseudo-reference output of GPT-4o, enabling a reliable benchmark for quality comparison with other LLMs such as Gemini Pro 1.5, LLaMA 3, and Mistral. Empirical results highlight significant improvements post fine-tuning, with LegalMind outperforming several baseline and open-source models on key semantic and syntactic metrics. The system offers a scalable, cost-effective legal NLP pipeline suited for real-world use cases in law firms, research institutions, and legal consultancies, reinforcing the potential of domain-specific LLMs in transforming legal technology.