ALL INTELLIGENCE
Hacker News AIT2·Tools

Interactive Notes on RLHF, Attention, LoRA and LLM Evals

Original title: Show HN: Interactive notes on RLHF, attention, LoRA and LLM evals
42AI Score

A developer released interactive technical notes covering core concepts like RLHF, attention mechanisms, LoRA, and LLM evaluations. The resource allows users to modify inputs to intuitively understand engineering principles, serving as a supplementary learning tool for AI practitioners.

Full Report

Contents3 sections

Engineering ideas you understand by changing the inputs.

Evals

The map of evalsStart here

  1. Is 82% better than 80%?
  2. Exact match and F1
  3. Precision, recall and reciprocal rank
  4. nDCG@k
  5. pass@k
  6. LLM as judge
  7. Pairwise preference and Elo
  8. Human agreement and kappa
  9. RAG evals
  10. Agent evals
  11. Online evals and A/B tests
  12. Contamination and Goodhart

Machine learning

Search and ranking

Notes I keep for myself, added as I learn. Found a mistake? · Source · Analytics choice

Allow Google Analytics to count page views? It sets cookies. Nothing else on this site tracks you.