AI Tools Radar › Categories › Evaluation & observability
LLM evaluation, benchmarks & observability tools
Benchmarks, evals, arenas, guardrails, tracing, observability and token or cost tracking for LLM apps. 22 items in today's radar (data fetched 21 Oct 2026, 14:13 (Berlin time)). Categories are assigned by fixed keyword rules on GitHub topics, repo names, descriptions and Show HN titles; an item can be in several categories. How categories work.
All 393Agents & automation 196Coding assistants & dev tools 168Chat apps & assistants 28LLM inference & serving 36RAG & search 28Training & fine-tuning 11Models & research 36Image, video & audio generation 52Voice & speech 21Computer vision 8MCP, skills & plugins 146Evaluation & observability 22Data & datasets 30Local / on-device AI 45Security & sandboxing 33Learning & resources 48Other 16
Trending on GitHub today (4)
| Repository | Stars today | Total stars | Language | License | Created |
|---|---|---|---|---|---|
| 1. MadsLorentzen/ai-job-search The job search that runs on your machine. AI job application framework built on Claude Code: evaluate postings, tailor CVs, write cover letters, prep interviews. Fork it and own it. | 3,861 | 62,160 | Python | MIT | 2026-03-18 |
| 2. Asymptote-Labs/agent-beacon The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu | 447 | 2,296 | Go | MIT | 2026-05-12 |
| 3. usestrix/strix Open-source AI penetration testing tool to find and fix your app’s vulnerabilities. | 393 | 70,627 | Python | Apache-2.0 | 2025-08-05 |
| 4. trycua/cua Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation. | 208 | 30,514 | Rust | MIT | 2025-01-31 |
Trending on GitHub this week (1)
| Repository | Stars this week | Total stars | Language | License | Created |
|---|---|---|---|---|---|
| 1. MadsLorentzen/ai-job-search The job search that runs on your machine. AI job application framework built on Claude Code: evaluate postings, tailor CVs, write cover letters, prep interviews. Fork it and own it. | 12,848 | 62,160 | Python | MIT | 2026-03-18 |
New repos of the last 30 days (11)
| Repository | Age | Total stars | Language | License | Created |
|---|---|---|---|---|---|
| 1. Folkbench/folkbench AI API 中转站评测排行与切换工具：实测可用率、延迟与价格 · Open-source AI API relay benchmark, rankings & switcher for Claude Code / Codex | 19 d | 2,223 | Rust | Other | 2026-10-02 |
| 2. rociiu/talorys Your personal AI agent in your own Cloudflare account. Chat, memory, tasks, notes and scheduled reminders, deployed with one command: npx create-talorys@latest. Free-tier friendly, single-user, no telemetry. | 11 d | 1,362 | TypeScript | MIT | 2026-10-10 |
| 3. mikehasa/golive-skill Take your agent-built product live: hosting, database, domain, email, payments — on your own accounts. Open-source Agent Skill + zero-dependency Node CLI: detect → plan → approve → apply → verify. No GoLive account, backend or telemetry. | 27 d | 1,307 | TypeScript | MIT | 2026-09-23 |
| 4. jwuthri-cloud/Tracely-ai Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable. | 9 d | 1,160 | Python | MIT | 2026-10-11 |
| 5. Jakeschincariol/arena-skill When Claude keeps giving you bad answers, make 100 versions of it fight to the death. Same task, 100 different strategies, a bracket, one answer left. Free Claude Code skill. | 23 d | 835 | Python | MIT | 2026-09-27 |
| 6. scaleapi/agentenv-framework Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now. | 26 d | 350 | Python | Apache-2.0 | 2026-09-24 |
| 7. Chengjun023/agent-smith Token 超支？快给史密斯打电话！ Better Call Smith. Adaptive model routing, a glassy Codex usage monitor, and reproducible benchmarks with receipts. | 20 d | 301 | Python | MIT | 2026-09-30 |
| 8. zgcagi/Harness-RSI Harness self-evolution: update targets, theory, evaluation, and reported evidence. Bilingual reading list and project website. | 19 d | 293 | TeX | CC-BY-4.0 | 2026-10-01 |
| 9. Entelligence-AI/agentplay Replay your coding agent sessions as a map and a timeline | 7 d | 278 | Python | Apache-2.0 | 2026-10-14 |
| 10. heaven999b/hello-agent-system 👋 Hello Agent System — learn enterprise AI agent system design by building every production layer from scratch. 17 bilingual lessons (EN/中文) with tested exercises: tools, architectures, reliability, security, evals, distributed execution, cost, RAG, release ops. 企业级 Agent 系统设计训练营 | 24 d | 276 | Python | MIT | 2026-09-27 |
| 11. ZCodium-project/ZCodium ZCodium，Independent audit fork of ZCode (ZCode → ZCodium). All monitoring and telemetry removed; upstream commits reviewed one by one. ZCode 的独立审计分支（ZCode → ZCodium）：已移除监控与遥测，上游改动逐提交审阅。 | 27 d | 239 | TypeScript | MIT | 2026-09-23 |
AI launches on Show HN (7)
| Show HN post | Points | Comments | Posted |
|---|---|---|---|
| 1. Show HN: Sphinx – A hook that quizzes you on your AI's code before shipping HN discussion · by critiquebrowser | 252 | 89 | 2026-10-18 |
| 2. Show HN: Parsimony – an open-source benchmark for how much code AI models add HN discussion · by lasu | 77 | 19 | 2026-10-13 |
| 3. Show HN: OpenAPPA – open-source deterministic guardrails that don't break agents HN discussion · by motakuk | 25 | 12 | 2026-09-28 |
| 4. Show HN: AI SRE Arena, an Open Benchmark for AI SRE Agents on Kubernetes HN discussion · by emrahsamdan | 24 | 10 | 2026-10-08 |
| 5. Show HN: LLMxRay an open-source observability tool for LLMs HN discussion · by lognebudo | 11 | 0 | 2026-10-11 |
| 6. Show HN: Premortem – AI agents that red-team your startup idea HN discussion · by ahoskins | 11 | 12 | 2026-10-01 |
| 7. Show HN: Pacer – will your AI coding subscription last until the reset? HN discussion · by dkremsa | 10 | 8 | 2026-10-08 |