docs: add concepts, guides, configuration, and troubleshooting pages - #48
Merged
Conversation
- Add Concepts page explaining datasets, experiments, tasks, evaluators, and the upload/run/compare lifecycle - Add Writing Custom Evaluators guide covering the evaluator contract, expected dict structure, float vs bool, file naming, composing multiple evaluators, and error handling - Add Configuration Reference page documenting all evalwire.toml keys, on_exist modes, and a full CLI flag reference for both commands - Add Troubleshooting page for common errors: event loop closed, Phoenix connection issues, missing CSV path, dataset not found, optional extras, CSV formatting, and empty evaluator list - Expand uploader.md and runner.md API pages with prose introductions, usage examples, pitfalls, and cross-links - Add cross-link to custom evaluators guide from evaluators.md - Add Changelog page (copy of CHANGELOG.md) to docs site - Update mkdocs.yml nav with all new pages and a Guides section - Update index.md navigation links
Add mkdocs-include-markdown-plugin as a dev dependency and replace the static docs/changelog.md copy with a one-line include directive that reads directly from the root CHANGELOG.md. This means the changelog page is always in sync with the file release-please maintains.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements all high and medium priority proposals from
enhancements/03-documentation.md.expectedis structuredexpecteddict parsing, float vs bool return types, file/name conventions, composing multiple evaluators per experiment, and error handling best practicesevalwire.tomlkeys with types, defaults, and descriptions; includes a full CLI flag reference for bothuploadandrunwith their config-file equivalentsuploader.mdandrunner.mdnow have prose introductions, usage examples, pitfall sections, and cross-links;evaluators.mdlinks to the custom evaluators guideCHANGELOG.mdcopied intodocs/and added to the navmkdocs.ymladds Concepts, Guides, Configuration, Troubleshooting, and Changelog to the nav