Hugging Face and JFrog partner to make AI Security more transparent
Enhancing AI Security with Hugging Face and JFrog Partnership
In a significant move to bolster the security of artificial intelligence (AI) models, Hugging Face has partnered with JFrog, the creators of the JFrog Software Supply Chain Platform. This collaboration aims to provide a safer and more reliable platform for the machine learning (ML) community.
The Challenge of Model Security
To share models, developers serialize weights, configurations, and other data structures used to interact with the models. However, some serialization formats are vulnerable to nasty exploits, such as arbitrary code execution. For instance, the use of pickle can lead to shared models that use those formats potentially being dangerous.
As Hugging Face has become a popular platform for model sharing, the company has developed tools like picklescan to help protect the community from these vulnerabilities. The integration of JFrog's scanner suite is a significant step forward in enhancing model security.
JFrog's Scanner Suite
JFrog's scanner suite is designed to detect potential malicious usage in model weights. Unlike Hugging Face's picklescan scanner, which only performs pattern matching on module names, JFrog's scanner parses and analyzes code found in model weights to identify potential security threats.
Interested in joining the security partnership or providing scanning information on the Hub? Please get in touch with us at security@huggingface.co.
Integration with Hugging Face Hub
There's nothing you need to do to benefit from this integration. All public model repositories will be scanned by JFrog automatically as soon as you push your files to the Hub. For example, you can check out the mcpotato/42-eicar-street repository to see the feature in action.
Community Engagement
In total, we have already scanned hundreds of millions of files, because we believe that empowering the community to share models in a safe and frictionless manner will lead to growth for the whole field.
Conclusion
The partnership between Hugging Face and JFrog is a significant step forward in enhancing AI security. By integrating JFrog's scanner suite, Hugging Face is providing a safer and more reliable platform for the ML community. We encourage developers to join the security partnership and provide scanning information on the Hub.
Model Security Refresher
To share models, we serialize weights, configs, and other data structures we use to interact with the models, in order to facilitate storage and transport. Some serialization formats are vulnerable to nasty exploits, such as arbitrary code execution (looking at you pickle), making shared models that use those formats potentially dangerous.
As Hugging Face has become a popular platform for model sharing, we’d like to help protect the community from this, hence why we have developed tools like picklescan and why we are integrating JFrog in our scanner suite.
Pickle is not the only exploitable format out there, see for reference how one can exploit Keras Lambda layers to achieve arbitrary code execution. The good news is that JFrog catches both of these exploits and more in additional file formats – see their Model Threats page for up to date scanner information.
Read all our documentation on security here: https://huggingface.co/docs/hub/security 🔥
Code Example
import pickle
# Load the model weights from a file
with open('model_weights.pkl', 'rb') as f:
weights = pickle.load(f)
# Use the loaded weights to make predictions
predictions = model.predict(weights)
In this example, we load the model weights from a file using pickle. However, this code is vulnerable to arbitrary code execution attacks. To mitigate this, we can use JFrog's scanner suite to detect potential security threats in the model weights.
Future Development
We plan to continue enhancing the security of our platform by integrating more advanced security features and tools. We encourage developers to join the security partnership and provide feedback on our security efforts.
Join the Conversation
Sign up or log in to comment on this article and join the conversation.
Edit Preview
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
