🔬 ERM: Experiment Results Manager
Light-weight alternative to mlflow
experiment tracking that doesn't require kubernetes. Useful tool to compare metrics between training attempts in your model training workflow
✨ Features
- 📈 Track plots, metrics, & other data
- 👀 Side-by-side comparison
- 💾 Experiment registry
- ⛅️ Supports S3, GCS, Azure and others (via
fsspec
)
🚀 Examples & Demos
- Quick and easy: serialize_and_deserialize.ipynb
- Practical but more involved: compare_runs.ipynb
- Browse the registry: browse_registry.ipynb
✅ Get Started
Installation
pip install experiment-results-manager \
gcsfs \
s3fs
# install s3fs if you plan to store data in s3
# install gcsfs if you plan to store data in google cloud storage
Basic Usage
import experiment_results_manager as erm
# Create an experiment run
er = erm.ExperimentRun(
experiment_id="my_experiment",
variant_id="main"
)
# Log relevant data
er.log_param("objective", "rmse")
er.log_metric("rmse", "0.9")
er.log_figure(mpl_fig, "ROC Curve")
er.log_text("lorem ipsum...", "text")
# Display the report (if you are in a notebook)
html = erm.compare_runs(er)
display(HTML(html))
# Save to registry
saved_path = erm.save_run_to_registry(er, "s3://erm-registry")
Made with ❤️ in Berlin