A Multi-Agent RAG Framework for Automated Source Code Vulnerability Detection and Repair

Research Paper Accepted at ICCCT 2025 (International Conference on Computer and Communication Technology)

Overview

This research presents an innovative automated framework for detecting and repairing security vulnerabilities in source code using a multi-agent architecture powered by Large Language Models (LLMs) and Retrieval Augmented Generation (RAG). The system addresses critical challenges in modern software security by providing both detection and automated remediation in a unified pipeline.

Research Achievement

Published Research: “A Multi-Agent RAG Framework for Automated Source Code Vulnerability Detection and Repair”

Conference: ICCCT 2025 (International Conference on Computer and Communication Technology)

Authors: Samridh Anand, Sakshi S, Samritha R, Samarth S I, Shruthi L (PES University, Bengaluru)

Problem Statement

In modern software development, security threats are pervasive and continuously evolving. Effective security risk management requires consistent vulnerability assessment throughout the development lifecycle. Traditional approaches to vulnerability detection face several critical challenges:

Key Challenges Addressed

• Time-Intensive Manual Review: Traditional code analysis requires significant human resources and expertise
• High False Positive Rates: Static analysis tools generate numerous irrelevant alerts, reducing developer efficiency
• Limited Dataset Quality: Shortage of real-world vulnerability datasets that accurately reflect production code complexity
• Separation of Detection and Repair: Most tools only identify vulnerabilities, leaving remediation to manual developer intervention
• Language and Framework Diversity: Difficulty in maintaining consistent security analysis across multiple programming environments
• Scalability Issues: Challenges in analyzing large, complex, and interconnected modern codebases

Novel Solution Approach

Our framework introduces a multi-agent system that integrates detection and repair into a cohesive, automated pipeline:

1. Multi-Agent Architecture

The system employs specialized LLM agents, each with distinct roles:

• Attack Vector Identification Agent: Analyzes code to identify potential security weaknesses
• Vulnerability Detection Agent: Confirms presence of specific CWE (Common Weakness Enumeration) vulnerabilities
• Remediation Agent: Generates secure code fixes based on detected vulnerabilities

2. Retrieval Augmented Generation (RAG)

To minimize hallucinations and improve accuracy, the system implements RAG with:

• Vulnerability Knowledge Base: Comprehensive database of MITRE’s Top 25 CWEs including descriptions, consequences, mitigation strategies, and demonstrative examples
• Vector Database: ChromaDB for efficient storage and retrieval of vulnerability information
• Semantic Search: Context-aware retrieval using all-MiniLM-L6-v2 sentence transformers for embedding

3. Iterative Detection and Repair Cycle

The framework operates through intelligent iteration:

1. Source code is analyzed for potential attack vectors
2. Relevant vulnerability information is retrieved from the knowledge base
3. Vulnerabilities are detected with specific CWE identification
4. Automated repair is attempted with contextual understanding
5. Process repeats until no vulnerabilities remain or attempt limit is reached

Technical Implementation

Technology Stack

• LLM: Google Gemini-1.5-flash (chosen for capability and cost-effectiveness)
• Vector Database: ChromaDB for semantic search and retrieval
• Embedding Model: all-MiniLM-L6-v2 sentence transformer
• Knowledge Base: MITRE Top 25 CWE dataset with comprehensive vulnerability information
• Target Languages: C and C++ (extensible to other languages)

System Architecture

The framework consists of three primary components:

1. Vulnerability Knowledge Construction: Automated scraping and vectorization of CWE information from MITRE database
2. Context-Aware Retrieval: Intelligent querying of vulnerability knowledge based on code analysis
3. Coordinated Detection and Repair: Multi-agent collaboration for comprehensive vulnerability resolution

Structured Output and Validation

• JSON-formatted responses for consistent parsing
• Explicit validation to prevent hallucinations
• Confidence-based CWE identification
• Iterative verification with configurable attempt limits

Research Validation and Results

Testing Methodology

• Dataset: 92 samples from CVEFixes dataset (real-world vulnerability data)
• Sample Distribution: 2 vulnerable and 2 non-vulnerable samples per CWE from MITRE Top 25
• Languages Tested: C and C++
• Baseline Comparison: Single LLM approach without RAG or multi-agent architecture

Performance Results

Our multi-agent RAG framework demonstrated significant improvements over traditional single LLM approaches:

Key Findings

• Enhanced Accuracy: 60.71% improvement in correct vulnerability detection and repair
• Reduced Errors: 46.15% reduction in system errors compared to single LLM approach
• Minimized False Negatives: 13.16% reduction in incorrect predictions
• Processing Time: 3-10 seconds per function analysis and repair
• Hallucination Reduction: RAG significantly improved response reliability

Limitations Identified

1. Authentication-Related Vulnerabilities: System shows reduced effectiveness for authorization and privilege escalation vulnerabilities requiring broader project context
2. JSON Formatting: Occasional issues with special character escaping in structured output
3. Scope Limitation: Currently focused on MITRE Top 25 CWEs

Research Contributions

This work makes several significant contributions to the field of automated software security:

1. Unified Detection-Repair Pipeline: First comprehensive framework integrating both vulnerability identification and automated remediation
2. Multi-Agent Collaboration: Novel application of cooperative LLM agents for software security
3. RAG for Security Analysis: Innovative use of retrieval augmented generation to improve accuracy and reduce hallucinations
4. Cost-Effective Approach: Demonstrates effective vulnerability detection without expensive fine-tuning
5. Real-World Validation: Tested on actual CVE data rather than synthetic datasets

Future Research Directions

Proposed Enhancements

1. GraphRAG Integration: Implementing graph-based knowledge representation for more sophisticated semantic relationships between vulnerabilities
2. Project Context Awareness: Incorporating project-specific requirements and architectural patterns to improve authentication vulnerability detection
3. Fine-Tuned LLM Specialization: Exploring domain-specific model fine-tuning for enhanced vulnerability understanding
4. Extended Language Support: Expanding beyond C/C++ to modern languages like Python, JavaScript, and Go
5. Continuous Learning: Implementing feedback mechanisms for agent improvement based on developer corrections

Impact and Significance

This research addresses critical gaps in automated software security:

• Accelerated Security Workflows: Reduces time from vulnerability detection to remediation
• Reduced Security Debt: Enables proactive vulnerability management during development
• Democratized Security Analysis: Provides enterprise-grade security capabilities without extensive infrastructure
• Scalable Solution: Cost-effective approach suitable for organizations of all sizes
• Developer Productivity: Minimizes manual security review burden on development teams

Academic Recognition

Conference Presentation: This research has been accepted for presentation at ICCCT 2025, demonstrating its contribution to the academic understanding of AI-powered software security analysis.

Keywords

vulnerability detection • vulnerability repair • AI agents • static analysis • large language models • retrieval augmented generation • software security • CWE detection • automated remediation • multi-agent systems

Resources

• Full paper available upon publication at ICCCT 2025
• Research conducted at PES University, Bengaluru, India
• Collaborators: Samridh Anand, Sakshi S, Samritha R, Samarth S I, Shruthi L