Case study
CVE LLM Pipeline: separating verifiable facts from generated content
Personal MLOps and LLM engineering learning project
2026
Personal CVE/LLM project: experiments exposed evaluation errors and RAG limitations, leading to a design that separates verifiable facts from generated content.
- 24,292unique indexed CVEs
- 50held-out test CVEs
- 184pytest tests passed
- Python
- Mistral 7B / LoRA
- PyTorch / ROCm
- ChromaDB / RAG
- MLflow
- DVC
- FastAPI
- Docker
- pytest
The problem: how far should the LLM be trusted?
I built this personal project to learn LoRA fine-tuning, RAG, and MLOps experimentation with real vulnerability data. The pipeline collects CVEs from the NVD, fine-tunes Mistral 7B on a local AMD/ROCm GPU, indexes the CVEs in ChromaDB, and compares answers with and without additional context.
The reference LoRA adapter was trained on 3,000 examples for one epoch. The 24,292 CVEs shown above are the records indexed in ChromaDB, not the number of examples used for training.
The main lesson was not a better model score. The experiments showed that good-looking results could hide evaluation errors, and that RAG or fine-tuning alone could not make factual information reliable. An LLM can explain a vulnerability, but should it be responsible for providing precise facts such as the CVSS score or CWE?
The final design separates these two roles: verifiable facts come from the reference data, while the LLM is used to generate the explanation.
How the experiments changed the design
The main evaluation set contains 50 explicitly identified CVEs that are present in the index. In the current version, these 50 CVEs are excluded from training and validation. However, this does not make it possible to confidently reconstruct the datasets used by some older LoRA adapters. The results below therefore describe this specific test set rather than general model performance.
-
Exp1 : compare answers with and without RAG. Adding context improves the answers, but similar CVEs returned by semantic search can also introduce incorrect values. Without context, the historical LoRA model repeated a Tenda vulnerability pattern in 50/50 answers.
- Fix the evaluation before changing the model. The evaluation code sometimes interpreted “CVSS 3.1” as a score of 3.1, or confused a product-related year with the NVD publication year. I recalculated the saved results without rerunning the model. This showed that evaluation code needs to be checked as carefully as the model itself.
-
Exp2 : use only the exact CVE when it is known. Removing similar CVEs from the context when an exact identifier is found eliminated 6/6 CVSS contradictions linked to that additional context, although some omissions increased. Semantic search is still useful when no exact CVE is found.
-
Exp3 : provide better context. A structured record combined with a fuller description greatly improved coverage and the proportion of correct facts on this test set. Because both changes were introduced together, the gain cannot be attributed to only one of them. Incorrect CWE values still appeared even when the correct value was present in the context.
-
Exp4 : change what the model learns. Training was adjusted to focus on the expected answer. The Tenda pattern disappeared in this comparison (44/50 → 0/50), but other generic answers appeared instead. Fine-tuning therefore changed the type of error without making generated facts reliable enough to trust.
The 50/50 result from Exp1 and the 44/50 control used in Exp4 come from different experimental runs; they are not a direct before/after comparison.
The architecture that emerged
Normalized NVD records are indexed in ChromaDB, with one CVE per document. The API then follows one of two paths depending on whether an exact identifier is found.
Exact ID found
The API retrieves the CVE reference data directly. The LLM uses this information as context to explain the vulnerability, but it does not generate the factual fields itself.
No exact match
Similar CVEs provide context for the LLM to generate an explanation. Because no exact CVE was identified, facts remains empty.
The reference data includes CVE ID, official NVD CVSS when available, severity, CWE, publication date, and attack vector. facts contains this verifiable information. answer contains the explanation generated by the LLM. For precise values such as CVSS or CWE, clients should therefore use facts instead of extracting them from generated text.
After Exp4, more training would not solve the main problem. The real question was which information should not be generated by the LLM at all. I therefore stopped the GPU experiments and separated factual data from the generated explanation. This does not mean the hallucination problem is solved; the architecture simply avoids relying on the model for fields that must remain verifiable.
Traceability and engineering practice
The first version of the data preparation split generated examples rather than complete CVEs. The same CVE could therefore appear in more than one dataset, making the early comparisons unreliable. The current pipeline reserves the 50 test CVEs first, then splits CVEs between training and validation before generating instruction variants. It also checks that no CVE is shared between training, validation, and test sets.
DVC tracks the data used and MLflow records the experiments. Saved contexts and responses also make it possible to recalculate some evaluations without rerunning the model on the GPU. A 184-test pytest suite checks areas such as data isolation, index behavior, and the separation between facts and answer.
Historical traceability remains partial: for some older artifacts, it is not possible to reconstruct the exact training dataset that was used.
Coding agents substantially assisted with implementation and review. I defined their scope and remained responsible for the architecture, experiment design, acceptance criteria, targeted reviews, tests, troubleshooting, and conclusions. This project complements my SRE / Platform Engineering experience; it is not intended to be a production security platform or an academic research project.
What I learned
- A flawed evaluation can make a model look better than it really is.
- Finding the correct CVE does not guarantee that the LLM will reproduce its values correctly.
- RAG and fine-tuning can change the type of errors, but they do not automatically make generated text reliable.
- Verifiable facts should come from the reference source; the LLM is mainly useful for explaining them.
- Knowing when to stop training and change the architecture is also part of the engineering work.