HuggingFaceEval

non-profit
Activity Feed

AI & ML interests

None defined yet.

Recent Activity

HuggingFaceEvalInternal's activity

albertvillanovaย 
posted an update 2 days ago
view post
Post
3130
๐Ÿš€ New smolagents update: Safer Local Python Execution! ๐Ÿฆพ๐Ÿ

With the latest release, we've added security checks to the local Python interpreter: every evaluation is now analyzed for dangerous builtins, modules, and functions. ๐Ÿ”’

Here's why this matters & what you need to know! ๐Ÿงต๐Ÿ‘‡

1๏ธโƒฃ Why is local execution risky? โš ๏ธ
AI agents that run arbitrary Python code can unintentionally (or maliciously) access system files, run unsafe commands, or exfiltrate data.

2๏ธโƒฃ New Safety Layer in smolagents ๐Ÿ›ก๏ธ
We now inspect every return value during execution:
โœ… Allowed: Safe built-in types (e.g., numbers, strings, lists)
โ›” Blocked: Dangerous functions/modules (e.g., os.system, subprocess, exec, shutil)

3๏ธโƒฃ Immediate Benefits ๐Ÿ’ก
- Prevent agents from accessing unsafe builtins
- Block unauthorized file or network access
- Reduce accidental security vulnerabilities

4๏ธโƒฃ Security Disclaimer โš ๏ธ
๐Ÿšจ Despite these improvements, local Python execution is NEVER 100% safe. ๐Ÿšจ
If you need true isolation, use a remote sandboxed executor like Docker or E2B.

5๏ธโƒฃ The Best Practice: Use Sandboxed Execution ๐Ÿ”
For production-grade AI agents, we strongly recommend running code in a Docker or E2B sandbox to ensure complete isolation.

6๏ธโƒฃ Upgrade Now & Stay Safe! ๐Ÿš€
Check out the latest smolagents release and start building safer AI agents today.

๐Ÿ”— https://github.com/huggingface/smolagents

What security measures do you take when running AI-generated code? Letโ€™s discuss! ๐Ÿ‘‡

#AI #smolagents #Python #Security
  • 2 replies
ยท
albertvillanovaย 
posted an update 3 days ago
view post
Post
3592
๐Ÿš€ Big news for AI agents! With the latest release of smolagents, you can now securely execute Python code in sandboxed Docker or E2B environments. ๐Ÿฆพ๐Ÿ”’

Here's why this is a game-changer for agent-based systems: ๐Ÿงต๐Ÿ‘‡

1๏ธโƒฃ Security First ๐Ÿ”
Running AI agents in unrestricted Python environments is risky! With sandboxing, your agents are isolated, preventing unintended file access, network abuse, or system modifications.

2๏ธโƒฃ Deterministic & Reproducible Runs ๐Ÿ“ฆ
By running agents in containerized environments, you ensure that every execution happens in a controlled and predictable settingโ€”no more environment mismatches or dependency issues!

3๏ธโƒฃ Resource Control & Limits ๐Ÿšฆ
Docker and E2B allow you to enforce CPU, memory, and execution time limits, so rogue or inefficient agents donโ€™t spiral out of control.

4๏ธโƒฃ Safer Code Execution in Production ๐Ÿญ
Deploy AI agents confidently, knowing that any generated code runs in an ephemeral, isolated environment, protecting your host machine and infrastructure.

5๏ธโƒฃ Easy to Integrate ๐Ÿ› ๏ธ
With smolagents, you can simply configure your agent to use Docker or E2B as its execution backendโ€”no need for complex security setups!

6๏ธโƒฃ Perfect for Autonomous AI Agents ๐Ÿค–
If your AI agents generate and execute code dynamically, this is a must-have to avoid security pitfalls while enabling advanced automation.

โšก Get started now: https://github.com/huggingface/smolagents

What will you build with smolagents? Let us know! ๐Ÿš€๐Ÿ’ก
albertvillanovaย 
posted an update about 1 month ago
view post
Post
3717
๐Ÿš€ Introducing @huggingface Open Deep-Research๐Ÿ’ฅ

In just 24 hours, we built an open-source agent that:
โœ… Autonomously browse the web
โœ… Search, scroll & extract info
โœ… Download & manipulate files
โœ… Run calculations on data

55% on GAIA validation set! Help us improve it!๐Ÿ’ก
https://huggingface.co./blog/open-deep-research
  • 3 replies
ยท
albertvillanovaย 
posted an update 2 months ago
thomwolfย 
posted an update 3 months ago
view post
Post
6009
We are proud to announce HuggingFaceFW/fineweb-2: A sparkling update to HuggingFaceFW/fineweb with 1000s of ๐Ÿ—ฃ๏ธlanguages.

We applied the same data-driven approach that led to SOTA English performance in๐Ÿท FineWeb to thousands of languages.

๐Ÿฅ‚ FineWeb2 has 8TB of compressed text data and outperforms other multilingual datasets in our experiments.

The dataset is released under the permissive ๐Ÿ“œ ODC-By 1.0 license, and the ๐Ÿ’ป code to reproduce it and our evaluations is public.

We will very soon announce a big community project, and are working on a ๐Ÿ“ blogpost walking you through the entire dataset creation process. Stay tuned!

In the mean time come ask us question on our chat place: HuggingFaceFW/discussion

H/t @guipenedo @hynky @lvwerra as well as @vsabolcec Bettina Messmer @negar-foroutan and @mjaggi
  • 2 replies
ยท
thomwolfย 
posted an update 3 months ago
thomwolfย 
posted an update 3 months ago
thomwolfย 
posted an update 4 months ago
SaylorTwiftย 
posted an update 4 months ago
albertvillanovaย 
posted an update 4 months ago
view post
Post
1807
๐Ÿšจ How green is your model? ๐ŸŒฑ Introducing a new feature in the Comparator tool: Environmental Impact for responsible #LLM research!
๐Ÿ‘‰ open-llm-leaderboard/comparator
Now, you can not only compare models by performance, but also by their environmental footprint!

๐ŸŒ The Comparator calculates COโ‚‚ emissions during evaluation and shows key model characteristics: evaluation score, number of parameters, architecture, precision, type... ๐Ÿ› ๏ธ
Make informed decisions about your model's impact on the planet and join the movement towards greener AI!
thomwolfย 
posted an update 4 months ago
albertvillanovaย 
posted an update 4 months ago
view post
Post
1602
๐Ÿš€ New feature of the Comparator of the ๐Ÿค— Open LLM Leaderboard: now compare models with their base versions & derivatives (finetunes, adapters, etc.). Perfect for tracking how adjustments affect performance & seeing innovations in action. Dive deeper into the leaderboard!

๐Ÿ› ๏ธ Here's how to use it:
1. Select your model from the leaderboard.
2. Load its model tree.
3. Choose any base & derived models (adapters, finetunes, merges, quantizations) for comparison.
4. Press Load.
See side-by-side performance metrics instantly!

Ready to dive in? ๐Ÿ† Try the ๐Ÿค— Open LLM Leaderboard Comparator now! See how models stack up against their base versions and derivatives to understand fine-tuning and other adjustments. Easier model analysis for better insights! Check it out here: open-llm-leaderboard/comparator ๐ŸŒ
albertvillanovaย 
posted an update 4 months ago
view post
Post
3186
๐Ÿš€ Exciting update! You can now compare multiple models side-by-side with the Hugging Face Open LLM Comparator! ๐Ÿ“Š

open-llm-leaderboard/comparator

Dive into multi-model evaluations, pinpoint the best model for your needs, and explore insights across top open LLMs all in one place. Ready to level up your model comparison game?
thomwolfย 
posted an update 5 months ago
view post
Post
4219
Parents in the 1990: Teach the kids to code
Parents now: Teach the kids to fix the code when it starts walking around ๐Ÿค–โœจ
  • 2 replies
ยท