6K-plus AI fashions could also be affected by important RCE vulnerability – Go Well being Professional

A important vulnerability in a preferred Python bundle for large-language fashions (LLMs) might have an effect on greater than 6,000 fashions and will result in provide chain assaults.

Click on for extra particular protection

The open-source llama-cpp-python bundle was discovered to be weak to server-side template injection, which might result in distant code execution (RCE). The flaw, tracked as CVE-2024-34359, was found by Patrick Peng, a safety researcher and developer who goes by retro0reg on-line.

The llama-cpp-python bundle gives Python bindings for the broadly well-liked llama.cpp library; llama.cpp is a C++ library to run LLMs like Meta’s LLaMA and fashions from Mitral AI on one’s personal private pc. The llama-cpp-python bundle additional allows builders to combine these open-source fashions into Python.

CVE-2024-34359, which has a important CVSS rating of 9.7, dangers RCE resulting from an improper implementation of the Jinja2 template engine. The flaw permits chat templates saved in metadata to be parsed by Jinja2 with out sanitization or sandboxing, creating a gap for attackers to inject malicious templates, Peng defined in a weblog submit.

Peng uploaded a proof-of-concept exploit for the vulnerability on Hugging Face, demonstrating how a mannequin compromised with a malicious template might execute arbitrary code upon loading or initiating a chat session. Peng’s weblog submit additionally describes how the malicious code might be injected into mannequin downloaded as a .gguf file, a standard file format for sharing pure language processing (NLP) fashions on open-source hubs equivalent to Hugging Face.

Greater than 6,000 fashions on Hugging Face use llama_cpp_python, Jinja2 and the gguf format, in response to Checkmarx. A menace actor might doubtlessly obtain a weak mannequin, inject the .gguf metadata with their very own malicious template, and redistribute the mannequin for provide chain assaults on unsuspecting AI builders.

A repair for CVE-2024-34359 was added in model 0.2.72 of llama_cpp_python final week. This model provides enter validation and sandboxing measures whereas rendering templates.

“The invention of CVE-2024-34359 servers as a stark reminder of the vulnerabilities that may come up on the confluence of AI and provide chain safety. It highlights the necessity for vigilant safety practices all through the lifecycle of AI techniques and their elements,” the Checkmarx weblog submit concludes. “As AI expertise turns into extra embedded in important purposes, making certain these techniques are constructed and maintained with a security-first strategy is significant to safeguard in opposition to potential threats that might undermine the expertise’s advantages.”

Leave a Comment