API Reference¶
Auto-generated reference from source code docstrings.
What are you looking for?¶
| I want to... | Go to |
|---|---|
| Look up YAML task config fields (Jinja API) | TaskConfig |
| Look up YAML group config fields | GroupConfig |
| Run an evaluation from Python | Entry Points |
| Load and discover tasks programmatically | TaskManager |
| Configure an evaluation run | Evaluation Config |
| See available model backends and their args | Available Backends |
| Create a new model backend | LM Base Class |
| Understand the Task class for authoring tasks | Task Class |
| Organize tasks into groups | Groups |
| See available metrics and aggregation functions | Metrics |
| Understand the scoring pipeline | Scorers |
| Apply post-processing filters to model outputs | Filters |