Best AI agent evaluation courses
How do I evaluate AI agents?
Evaluate the full trajectory: tool calls, source use, intermediate decisions, final answer, and stopping behavior. Agent evals need traces and scenario datasets, not just final-...
Learnetto AI directory
Find the best AI educators, resources, and learning paths.
A compact directory for people who want practical AI skills and up-to-date official docs: prompting, automation, coding agents, MCP, model selection, AI product work, RAG, evals, model internals, and production AI engineering.
183 educators
12 learning paths
10 agent tools
41 AI answers
625 resources
New guide · Agent evaluation
Backtest an AI agent without future-data leakage
Recreate what the agent could know at each cutoff, replay fixed cases in time order, and compare decisions, evidence, cost, and failure rates against a simple baseline.
AI Q&A
Answers to common AI learning questions
Short, crawlable answers connected to fuller guides, educators, docs, courses, and videos.
Best AI agent evaluation courses
What course teaches agent evals?
Evaluating AI Agents is the clearest course-style starting point. Follow it with OpenAI agent eval docs, Phoenix, Promptfoo, or Hamel Husain's eval material for practical implem...
Best AI agent evaluation courses
What failures should agent evals include?
Include wrong tool choice, bad retrieval, stale data, unsafe actions, loops, missing clarification, and cases where the agent should stop. These are the failures that polished d...
Best coding agent courses
What is the best course for learning coding agents?
Building Coding Agents with Tool Execution is a strong first course if you want to understand tool use and iteration. Then study Claude Code, Codex, and real workflow examples f...
Best AI resources for model release notes and migrations
Which AI release notes should I actually follow?
Follow the official changelog or release-note page for every provider you actively use, plus its deprecations or shutdown page if available. OpenAI, Anthropic, Gemini, DeepSeek,...
Best AI resources for model release notes and migrations
Where can I track model deprecations and migrations across providers?
Use OpenAI deprecations, Claude model deprecations, Gemini deprecations, DeepSeek updates and pricing, Mistral changelog, and xAI release notes. Pair those with the provider's m...
Recently added
Daily new AI learning picks
New directory additions across educator videos, newsletters, social feeds, official model docs, agents, coding tools, and model-selection references. Today's selection: Oct 20, 2026.
Choose how to scan the same daily picks: keep the current layout, mix formats, or read it like a feed.
Technical article
FS Data Lab
How to Cost Your AI-Powered Filters
FS Data Lab published this guide on October 1, 2026. Use it when you want a rigorous way to estimate GPU latency and inference cost for AI-powered SQL filters, compare filter orderings, and understand how compute, memory bandwidth, selectivity, batching, and KV-cache reuse affect a query plan.
ai-sql
inference cost
filter ordering
Connected-finance safety guide
OpenAI
ChatGPT Finances privacy and accuracy guide
OpenAI announced on October 2, 2026 that ChatGPT Finances is rolling out to Free and Go users in the United States. Use this guide to understand the connected-account workflow, check the source data behind answers, and review privacy controls before linking an account.
chatgpt
personal finance
connected accounts
Desktop agent safety guide
GitHub
GitHub Copilot computer use explained
GitHub released Copilot computer use in public preview on October 1, 2026. Use this guide to decide when visual desktop control is appropriate, enable it deliberately, and test its permission and stop controls before real work.
github copilot
computer use
desktop agents
Agent security architecture guide
Meta AI
How Meta built safety into Muse
You want Meta's September 8, 2026 technical account of defense-in-depth for a long-running personal agent, including isolated runtime cells, credential surrogates, a separate permission authority, tainted-egress tracking, scoped approvals, browser controls, red teaming, and prompt-injection evals.
meta
muse
agent security
Coding agent workflow release notes
Qwen
Qwen Code September 10 weekly update
You want Qwen Code's September 10, 2026 update on workflow run history, agent and token observability, context-usage inspection, combining Plan with YOLO mode, and named parallel channel tasks in isolated workspaces.
qwen code
coding agents
workflow observability
Model launch and selection guide
Google AI
Gemini 3.8 Flash launch
You want Google's September 2, 2026 launch details for Gemini 3.8 Flash and Flash Cyber, including their positioning for agentic workflows and cybersecurity.
google
gemini
frontier models
Agent security architecture guide
Anthropic
Enterprise Frontier Safeguards
You want Anthropic's September 1, 2026 architecture for combining customer-controlled data storage with automated misuse monitoring for sensitive frontier-model workloads.
anthropic
ai safety
enterprise
Tabular foundation model explainer
NVIDIA
NVIDIA Kumo Tabular explained
NVIDIA released Kumo Tabular on September 29, 2026. Use this guide to understand its in-context prediction workflow, reproduce a baseline, and test its vendor-reported results on your own tables.
kumo tabular
tabular foundation models
classification
Model migration explainer
Anthropic
Claude Sonnet 5.5 explained
Anthropic released Claude Sonnet 5.5 on September 28, 2026. Use this guide to understand the efficiency claims, breaking API changes, and a safe migration test from Sonnet 5.
claude sonnet 5.5
claude code
coding agents
►
Video · AI Engineer
Insurance Claims Are AI's Messiest Data Problem — Denys Linkov, Wisedocs
Use this October 18 talk when an AI feature depends on messy documents such as scans, faxes, handwriting, and large PDFs, and you need a cleaning and evaluation plan.
Technical article · FS Data Lab
How to Cost Your AI-Powered Filters
FS Data Lab published this guide on October 1, 2026. Use it when you want a rigorous way to estimate GPU latency and inference cost for AI-powered SQL filters, compare filter orderings, and understand how compute, memory bandwidth, selectivity, batching, and KV-cache reuse affect a query plan.
ai-sql
inference cost
filter ordering
roofline model
Connected-finance safety guide · OpenAI
ChatGPT Finances privacy and accuracy guide
OpenAI announced on October 2, 2026 that ChatGPT Finances is rolling out to Free and Go users in the United States. Use this guide to understand the connected-account workflow, check the source data behind answers, and review privacy controls before linking an account.
chatgpt
personal finance
connected accounts
plaid
►
Video · AI Engineer
Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse
Use this October 18 workshop when an error is expensive and you need tracing, datasets, evaluators, and a CI gate before a prompt change can ship.
Desktop agent safety guide · GitHub
GitHub Copilot computer use explained
GitHub released Copilot computer use in public preview on October 1, 2026. Use this guide to decide when visual desktop control is appropriate, enable it deliberately, and test its permission and stop controls before real work.
github copilot
computer use
desktop agents
copilot cli
Agent security architecture guide · Meta AI
How Meta built safety into Muse
You want Meta's September 8, 2026 technical account of defense-in-depth for a long-running personal agent, including isolated runtime cells, credential surrogates, a separate permission authority, tainted-egress tracking, scoped approvals, browser controls, red teaming, and prompt-injection evals.
meta
muse
agent security
prompt injection
►
Video · AI Engineer
Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Use this October 18 workshop when a coding agent declares success and you want it to check its own work with tests, health checks, and snapshots before a person looks.
Coding agent workflow release notes · Qwen
Qwen Code September 10 weekly update
You want Qwen Code's September 10, 2026 update on workflow run history, agent and token observability, context-usage inspection, combining Plan with YOLO mode, and named parallel channel tasks in isolated workspaces.
qwen code
coding agents
workflow observability
context management
Model launch and selection guide · Google AI
Gemini 3.8 Flash launch
You want Google's September 2, 2026 launch details for Gemini 3.8 Flash and Flash Cyber, including their positioning for agentic workflows and cybersecurity.
google
gemini
frontier models
agentic workflows
Agent security architecture guide · Anthropic
Enterprise Frontier Safeguards
You want Anthropic's September 1, 2026 architecture for combining customer-controlled data storage with automated misuse monitoring for sensitive frontier-model workloads.
anthropic
ai safety
enterprise
privacy
Tabular foundation model explainer · NVIDIA
NVIDIA Kumo Tabular explained
NVIDIA released Kumo Tabular on September 29, 2026. Use this guide to understand its in-context prediction workflow, reproduce a baseline, and test its vendor-reported results on your own tables.
kumo tabular
tabular foundation models
classification
regression
VID
AI Engineer posted a video pick
2026 · unstructured data · insurance
Insurance Claims Are AI's Messiest Data Problem — Denys Linkov, Wisedocs
Use this October 18 talk when an AI feature depends on messy documents such as scans, faxes, handwriting, and large PDFs, and you need a cleaning and evaluation plan.
F
FS Data Lab added technical article
How to Cost Your AI-Powered Filters
FS Data Lab published this guide on October 1, 2026. Use it when you want a rigorous way to estimate GPU latency and inference cost for AI-powered SQL filters, compare filter orderings, and understand how compute, memory bandwidth, selectivity, batching, and KV-cache reuse affect a query plan.
ai-sql
inference cost
filter ordering
roofline model
O
OpenAI added connected-finance safety guide
ChatGPT Finances privacy and accuracy guide
OpenAI announced on October 2, 2026 that ChatGPT Finances is rolling out to Free and Go users in the United States. Use this guide to understand the connected-account workflow, check the source data behind answers, and review privacy controls before linking an account.
chatgpt
personal finance
connected accounts
plaid
VID
AI Engineer posted a video pick
2026 · evals · LLM-as-judge
Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse
Use this October 18 workshop when an error is expensive and you need tracing, datasets, evaluators, and a CI gate before a prompt change can ship.
G
GitHub added desktop agent safety guide
GitHub Copilot computer use explained
GitHub released Copilot computer use in public preview on October 1, 2026. Use this guide to decide when visual desktop control is appropriate, enable it deliberately, and test its permission and stop controls before real work.
github copilot
computer use
desktop agents
copilot cli
M
Meta AI added agent security architecture guide
How Meta built safety into Muse
You want Meta's September 8, 2026 technical account of defense-in-depth for a long-running personal agent, including isolated runtime cells, credential surrogates, a separate permission authority, tainted-egress tracking, scoped approvals, browser controls, red teaming, and prompt-injection evals.
meta
muse
agent security
prompt injection
VID
AI Engineer posted a video pick
2026 · coding agents · verification
Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
Use this October 18 workshop when a coding agent declares success and you want it to check its own work with tests, health checks, and snapshots before a person looks.
Q
Qwen added coding agent workflow release notes
Qwen Code September 10 weekly update
You want Qwen Code's September 10, 2026 update on workflow run history, agent and token observability, context-usage inspection, combining Plan with YOLO mode, and named parallel channel tasks in isolated workspaces.
qwen code
coding agents
workflow observability
context management
G
Google AI added model launch and selection guide
Gemini 3.8 Flash launch
You want Google's September 2, 2026 launch details for Gemini 3.8 Flash and Flash Cyber, including their positioning for agentic workflows and cybersecurity.
google
gemini
frontier models
agentic workflows
A
Anthropic added agent security architecture guide
Enterprise Frontier Safeguards
You want Anthropic's September 1, 2026 architecture for combining customer-controlled data storage with automated misuse monitoring for sensitive frontier-model workloads.
anthropic
ai safety
enterprise
privacy
N
NVIDIA added tabular foundation model explainer
NVIDIA Kumo Tabular explained
NVIDIA released Kumo Tabular on September 29, 2026. Use this guide to understand its in-context prediction workflow, reproduce a baseline, and test its vendor-reported results on your own tables.
kumo tabular
tabular foundation models
classification
regression
AI video library
Watch first
Current picks across coding agents, MCP, model selection, RAG, evals, and production AI.
►
Smarter Models Killed the Harness. Here's What's Left — Jediah Katz, SpaceXAI
AI Engineer · 2026, agent harnesses, coding agents
►
Why AI Agents Still Won't Pay for Anything — Jan Curn, Apify
AI Engineer · 2026, agent payments, x402
►
Stop Telling Agents How — Dan Barrett, Raymond Weitekamp & Jose Montes de Oca, OpenProse
AI Engineer · 2026, multi-agent, declarative workflows
►
Insurance Claims Are AI's Messiest Data Problem — Denys Linkov, Wisedocs
AI Engineer · 2026, unstructured data, insurance
►
Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse
AI Engineer · 2026, evals, LLM-as-judge
►
Teach Your Agent to Prove It's Done — Nick Nisi & Zack Proser, WorkOS
AI Engineer · 2026, coding agents, verification
New directory
AI skills, agent harnesses, and worker tools
Find reusable skill registries, local agent harnesses, and AI worker products before choosing what to learn or buy.
Self-improving agent
Hermes Agent
Nous Research · Open-source agent
Use this when persistent memory, skill creation from completed tasks, and remote agent operation matter more than a simple chat UI.
AI coworker
Viktor
Viktor · Slack and Teams AI employee
Use this when the adoption surface should be chat-native and team-facing rather than a separate automation builder.
AI assistant builder
Lindy
Lindy · AI executive assistant
Use this when the job is business admin across apps rather than custom coding or local agent infrastructure.
Skill marketplace
ClawHub
OpenClaw · Registry
Use this when you need a public registry for OpenClaw skills and plugins rather than hand-copying skill folders.
Agent orchestration
Mission Control
Builderz Labs · Open-source dashboard
Use this when the hard part is seeing, assigning, and coordinating agent work rather than writing another prompt.
AI worker platform
Mission Control AI
Mission Control AI · Preconfigured AI workers
Use this when you want role-specific AI workers with SOPs, integrations, and governance policies already built in.
Educators to start with
Sorted for range: founders, product teams, developers, evals, model internals, and production systems.
| Educator | Best for | Skills | Start with |
|---|---|---|---|
| Engineers, ML practitioners | AI engineering, Systems, Production ML | Use the book page and related essays as a production engineering path. | |
| Beginners, educators, teams | Prompting, AI literacy, Security | Start at the fundamentals section, then move to prompt hacking/security. | |
| Developers, researchers | Prompting, RAG, Reasoning, Agents | Use the prompting techniques and RAG sections as a reference. | |
| ML engineers, data teams | Applied ML, Recommenders, LLM systems | Read the applied ML and LLM systems posts. | |
| ML engineers, researchers | RLHF, Post-training, Alignment | Read the RLHF book chapters in order. | |
| Programmers learning LLM internals | LLM internals, PyTorch, Model training | Clone the repo and work through the notebooks with the book. | |
| Visual learners, developers | Transformers, Embeddings, LLM concepts | Read The Illustrated Transformer, then The Illustrated GPT-2. | |
| Engineers, researchers | Agents, RAG, ML research | Read the posts on LLM-powered autonomous agents and prompt engineering. | |
| Developers, engineering leaders | AI coding, Engineering workflows, Frontend | Look for AI coding and engineering workflow posts. | |
| Developers and self-directed learners building with AI coding agents | AI coding, Claude Skills, Agentic workflows, AI SDK, MCP, LLM fundamentals, Personalized learning | Use LLM Fundamentals or the AI Engineer Roadmap if you need concepts, the Vercel AI SDK Tutorial or AI SDK v6 Crash Course if you want to build apps, and the AI Skills catalog if you want practical agent workflows like /teach, /grill-me, /tdd, and /triage. | |
| AI engineers, founders, researchers | AI engineering, Industry context, Model ecosystem | Pick interviews with engineers building tools you already use. | |
| Researchers, advanced builders | NLP research, Open models, Multilingual AI | Browse research posts and community programs. |
Choose by skill
Use these as quick routes into the directory.
Automation and agents
Useful for: Founders, operations teams, developers
Learn: Trigger-based workflows; Tool calling; Agent failure modes; Human review points
Resources: Brian Casel , Josh Pigford , AI Engineer
AI product work
Useful for: PMs, designers, founders
Learn: AI UX patterns; Use-case selection; Prototyping; Measuring quality
Resources: Peter Yang , Lenny Rachitsky , Shreya Shankar
Coding with AI
Useful for: Software developers
Learn: Codebase navigation; Test generation; Refactoring; Reviewing AI output
Resources: Simon Willison , Addy Osmani , Matt Pocock , AI Engineer
Context engineering and MCP
Useful for: Developers using coding agents and tool-connected workflows
Learn: How to structure instructions, memories, and skills; When to use MCP servers versus retrieval or file search; Subagents, hooks, and tool boundaries; How to keep agent context small, relevant, and testable
Resources: Anthropic , Hugging Face , OpenAI , Matt Pocock , Simon Willison
RAG and knowledge systems
Useful for: Developers building AI apps
Learn: Chunking and retrieval; Structured extraction; Reranking; Grounded answers
Resources: Jason Liu , Hamel Husain , Eugene Yan
Evals and reliability
Useful for: AI product teams
Learn: Test sets; Human review; Regression checks; Quality metrics
Resources: Hamel Husain , Shreya Shankar , Chip Huyen
Model internals
Useful for: Engineers who want deeper understanding
Learn: Neural nets from scratch; Transformers; Training loops; Fine-tuning basics
Resources: Andrej Karpathy , Sebastian Raschka , Jay Alammar
Production AI engineering
Useful for: Engineering teams
Learn: Observability; Cost and latency; Deployment; Data pipelines
Resources: Full Stack Deep Learning , Chip Huyen , Eugene Yan
Resources worth opening
A rotating mix of official docs, hands-on guides, model catalogs, and practical AI engineering resources.
| Resource | Type | Level | Use when |
|---|---|---|---|
|
How to Cost Your AI-Powered Filters
FS Data Lab
|
Technical article | Advanced | FS Data Lab published this guide on October 1, 2026. Use it when you want a rigorous way to estimate GPU latency and inference cost for AI-powered SQL filters, compare filter orderings, and understand how compute, memory bandwidth, selectivity, batching, and KV-cache reuse affect a query plan. |
|
OpenAI model guide
OpenAI
|
Model docs | Beginner to advanced | You need to choose between current GPT-5.6 variants, smaller GPT-5.4 variants, reasoning levels, tool support, and cost-sensitive API paths, including the current default starting points of GPT-5.6 Sol, Terra, and Luna. |
|
Claude Code skills
Anthropic
|
Guide | Intermediate | You want Anthropic's official guide for packaging reusable Claude Code skills instead of leaving long instructions inline in every session. |
|
Union Alpha revealed as Pareto 26.9
Unbiased
|
Multi-model routing launch explainer | Intermediate to advanced | Union Alpha was revealed as Unbiased's Pareto 26.9 on September 17, 2026: a hosted system that routes work across several models and returns one checked answer. |
|
OpenRouter Ori Harness
OpenRouter
|
Coding-agent guide | Intermediate | You want to run an existing Codex, Claude Code, Grok Build, OpenCode, Pi, Prime Agent, Hermes, or DeepSeek Harness workflow against OpenRouter models without replacing the agent CLI you already use. |
|
Qwen Code September 3 weekly update
Qwen
|
Coding agent release notes | Beginner to advanced | You want Qwen's September 3, 2026 practical guide to its latest coding-agent workflows, including learning-oriented output styles, scheduled tasks, a browser terminal, safer workspace writes, skills, MCP stability, and current vision-model support. |
|
Muse Spark 1.3
Meta AI
|
Model launch and selection guide | Intermediate | You are comparing frontier models for long-horizon coding or knowledge work and need Meta's September 2, 2026 release notes on Muse Spark 1.3's tool use, instruction following, user steering, multitasking, model self-awareness, and availability through Muse Code and Meta Model API. |
|
Project HydraFusion
GitHub
|
Model orchestration research preview | Intermediate to advanced | You want GitHub's September 4, 2026 technical explanation of runtime model orchestration for coding tasks, including plan decomposition, draft-critique-revise patterns, model cascading, evaluation design, and quality-versus-cost tradeoffs. |