⬢github Python · 3.8K ★ +2.5K since we first saw it · pushed 6 h ago · Apache-2.0
NVIDIA/SkillEvaluator
Multi-tier framework for evaluating AI agent skills and plugins with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.
SkillEvaluator is a Python CLI framework from NVIDIA for evaluating AI agent skills—folders of instructions following the Agent Skills spec. It runs tiered checks: static quality gates with no API key, semantic overlap detection, synthetic dataset generation, and live agent runs comparing behavior with and without a skill. It can run directly, in CI via a setup command, or as a skill inside agents like Claude Code.
Why now: Recently spotlighted via an NVIDIA Developer blog post, arXiv paper, and 'Ask the Experts' livestream, plus active releases under the NVIDIA Verified Skills pipeline
Who it is for: Developers and teams building, publishing, or securing agent skills for Claude Code, Codex, and other skill-aware agents
agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkclaude-codecodexevaluateevaluation
Stars over our 344 snapshots: 1.3K to 3.8K, since 3 d ago.
Where people talked about it
- ⬢github NVIDIA/SkillEvaluator 11 min ago
API: https://socialmediatrends-api.osmike.com/v1/repos/NVIDIA/SkillEvaluator