A Multimodal Deep Learning Framework for Software Vulnerability Detection

Comparative study and implementation of three deep learning architectures — Cross-Model Learning, Hypergraph Neural Networks (HGNN), and Kolmogorov–Arnold Networks (KAN) — for detecting vulnerabilities in C/C++ source code, evaluated across three benchmark datasets (BigVul, ReVeal, Devign), resulting in 9 model–dataset combinations.

Information Security Awareness and Education (ISEA) Project Phase – III Ministry of Electronics and Information Technology, Government of India Department of Computer Engineering, National Institute of Technology Kurukshetra Author: Komal (Intern, B.Tech CSE, Central University of Haryana) · Guide: Dr. Kuldeep Kumar, Assistant Professor, NIT Kurukshetra March – April 2026

---

📌 Problem Statement

Manually finding vulnerabilities in source code is slow and error-prone. Automating this with deep learning is hard because:

1. Source code is not plain text — detecting vulnerabilities requires understanding syntax structure, control flow, and data dependencies, not just token sequences.
2. Severe class imbalance — vulnerable functions are a small minority compared to safe code in real-world datasets.
3. Subtle patterns — vulnerable and safe code often look syntactically almost identical.

🧠 Approach

This project implements and compares three distinct architectures, each combining structural and/or semantic understanding of code differently:

1. Cross-Model (Graph + Transformer + Contrastive Learning)

• Builds an Abstract Syntax Tree (AST) from source code (via tree-sitter) and encodes it with a GNN (GCNConv / GATConv)
• Extracts contextual semantic embeddings using GraphCodeBERT / CodeBERT
• Aligns structural and semantic embeddings with NT-Xent contrastive learning
• Fuses both representations (concatenation / gated fusion) → classification head

2. Hypergraph Neural Network (HGNN)

• Constructs a hypergraph combining AST + Control Flow Graph (CFG) + Data Flow Graph (DFG) to capture higher-order relationships beyond pairwise graph edges
• HypergraphConv layers → BiGRU → attention-based pooling
• Trained with contrastive regularization + Focal Loss for imbalance

3. Kolmogorov–Arnold Network (KAN)

• Uses GraphCodeBERT CLS embeddings (768-dim) as input features
• Replaces fixed activation functions with learnable B-spline-based activations
• Progressive dimensional reduction through stacked KAN layers → binary classification

📊 Datasets

Dataset	Vulnerable	Non-Vulnerable	Class Ratio	Source
BigVul	10,900	177,736	1:16	Kaggle (MSR Dataset)
ReVeal	1,664	16,505	1:10	Google Drive (Chromium & Debian projects)
Devign	13,244	14,074	Nearly balanced	HuggingFace (claudios/code_x_glue_devign)

All datasets contain real-world C/C++ functions labeled vulnerable or non-vulnerable. Preprocessing includes null/empty removal, line-count filtering (3–500 lines), and dataset-specific class balancing (random undersampling for BigVul/ReVeal; Devign is used near-natively).

📈 Results

F1-Score & Accuracy Summary

Dataset	Model	Accuracy (%)	Precision (%)	Recall (%)	F1-score (%)	MCC (%)	AUC-ROC (%)
BigVul	Cross-Model	86	79	78	79	68	92
BigVul	HGNN	68	61	62	62	34	72
BigVul	KAN	89	73	70	72	65	92
ReVeal	Cross-Model	75	71	84	77	52	84
ReVeal	HGNN	72	56	68	61	40	79
ReVeal	KAN	70	67	77	72	41	78
Devign	Cross-Model	63	57	78	66	29	72
Devign	HGNN	64	58	80	66	31	73
Devign	KAN	64	59	74	66	31	72

AUC-ROC Comparison

Key takeaways:

• Cross-Model architecture achieves the best F1-score and AUC-ROC on BigVul and ReVeal — combining structural graph learning with transformer semantics clearly helps.
• KAN delivers competitive results with a simpler design, achieving the highest accuracy on BigVul (89%) using only GraphCodeBERT embeddings + spline-based layers.
• HGNN captures higher-order structural relationships but is more sensitive to noisy graph structure under class imbalance, resulting in lower scores.
• All models perform noticeably worse on Devign, indicating that diverse, structurally complex code makes vulnerability detection harder — a clear direction for future work.

🛠️ Tech Stack

• Python 3.x, PyTorch, PyTorch Geometric
• HuggingFace Transformers (CodeBERT, GraphCodeBERT)
• Tree-sitter (AST parsing for C/C++)
• Scikit-learn, Imbalanced-learn, Pandas, NumPy
• Matplotlib, Seaborn
• Google Colab (NVIDIA Tesla T4 GPU)

📂 Repository Structure

.
├── notebooks/
│   ├── bigvul_cross_model_final_submission.ipynb
│   ├── bigvul_hgnn_model_final_submission.ipynb
│   ├── BigVul_KAN_model_final_submission.ipynb
│   ├── ReVeal_Cross_model_final_submission.ipynb
│   ├── reveal_hgnn_model_final_submission.ipynb
│   ├── ReVeal_KAN_model_final_submission.ipynb
│   ├── devign_cross_model_final_submission.ipynb
│   ├── devign_hgnn_model_final_submission.ipynb
│   └── devign_kan_model_final_submission.ipynb
├── report/
│   └── Vulnerability_Detection_Report.pdf
├── assets/
│   ├── f1_score_comparison.png
│   └── auc_roc_comparison.png
├── requirements.txt
├── .gitignore
└── README.md

🚀 Getting Started

git clone https://github.com/<your-username>/software-vulnerability-detection.git
cd software-vulnerability-detection
pip install -r requirements.txt

Each notebook in notebooks/ is self-contained (dataset download → preprocessing → training → evaluation) and designed to run on Google Colab. Some notebooks additionally require:

pip install torch-geometric tree-sitter==0.20.1 tree-sitter-languages transformers accelerate

Dataset access: BigVul via kagglehub, ReVeal via gdown (Google Drive), Devign via HuggingFace datasets.

⚠️ Limitations & Future Work

• Graph construction (AST + CFG + DFG) adds significant preprocessing overhead
• Transformer-based branches (GraphCodeBERT) are GPU/memory-intensive
• Class imbalance in BigVul/ReVeal still biases learning despite undersampling
• Hand-crafted node features may not capture all relevant program semantics

Planned directions: dynamic/temporal graph learning, instruction-tuned code LLMs for semantic understanding, automated graph construction, and real-time large-scale vulnerability scanning.

📄 Full Report

The complete ISEA internship report with literature review, methodology, hyperparameter tables, and full evaluation is in report/Vulnerability_Detection_Report.pdf.

📜 License

This project is released for academic and educational purposes.