Maximus
Maximus is an LLM harness for contest ML, with a human on the train gate.
The problem
Contest work with a coding agent usually degrades into the same mess. You paste logs into chat and the model suggests a tweak, and then either you babysit every train yourself or you let it fire runs unsupervised. I have done both. Babysitting wastes my attention, and unsupervised runs burn compute I meant to approve. I wanted a fixed loop where the agent does the grunt work and I still own when training happens.
Why I built it
Day to day, I wanted the agent to iterate on contest models without me pasting logs by hand and without burning compute unsupervised.
The shape is fixed on purpose, and it runs propose → (you approve train) → eval → log → improve. The agent may request a train, but I am the one who runs it.
What I built
Maximus is a Python harness around competition folders. It has a CLI for train, eval, submit, status, log, and new, plus an MCP server that lets a Cursor agent check status, read logs, make scoped edits, run eval, prep a local submission, and call request_train. Training stays on the human path, so the agent writes a proposal and I run maximus train. MCP train is refused unless MAXIMUS_ALLOW_TRAIN=1, and I prefer to keep that flag off.
Each successful eval appends to an experiment log (experiments/log.md and history.jsonl) under a strict metrics.json contract, which requires a finite float metric and fails soft or partial parses. A built-in toy_binary competition proves the loop on CPU (logistic regression, optional random forest, ROC-AUC), and maximus new scaffolds another contest from a template. Actual contests land later as new folders. The MVP does not upload to DrivenData or Kaggle, and it is not a GEMS solution or an AutoML system that trains in a loop on its own.
This is the primary MVP, and future changes toward the final shape are already in the works. It has enough of the loop to use and trust today, and I am not pretending the product is finished.
Three decisions
These are the three choices I would defend in a design review.
1. You approve train, and the agent only asks.
The agent inspects, makes one change, and calls request_train, which writes a pending proposal. I review it and then run maximus train myself, so the human owns train and any cloud spend. I was counting on that gate holding when the agent gets eager, because the failure I care about is unsupervised compute I meant to approve. The default path has no autonomous multi-train loops.
2. MCP is for grunt work, not for burning GPUs.
Status, logs, scoped file I/O, eval, submit prep, and diagnose are agent tools. Train is gated, and per competition, eval can be locked down too if it is expensive. The agent edits only allowed_edit_globs, one change per cycle. The bet was that scoped tools let the agent improve things without owning the spend. If train rode the same MCP path as status, I could no longer trust the harness with a GPU bill.
3. Eval follows a hard metrics contract and writes an actual log.
Eval has to leave artifacts/metrics.json with a numeric metric, and soft or partial parses fail. Successful runs land in the experiment log, so the next propose step has evidence to work from and I am not copying chat scrollback by hand. The log has to be something a peer can read later, because if the metric is fuzzy, the improve step is guessing.
Proof
You can follow the same loop the harness enforces, from propose through approve train, eval, and log to improve.
request_train, eval), and train stays on the human CLI unless MAXIMUS_ALLOW_TRAIN=1. The diagram shows the MVP shape, and more is planned toward the final product.
The public repo (README, docs/QUICK-START.md, docs/OPERATOR-GUIDE.md) tells the same story in writing. It documents the fixed propose → approve-train → eval → log → improve loop and the rule that MCP train is refused unless MAXIMUS_ALLOW_TRAIN=1. It also covers the toy_binary CPU path for train, eval, and status, the experiment log plus strict metrics.json (metric as a finite float), and the guardrails (one change per cycle, allowed edit globs, no contest-site upload in the MVP).
Why I built it this way
I hold the same boundaries elsewhere. I want clear ownership and real evidence, and I want a design a technical peer can judge quickly.
If an agent is going to help on contest ML, it should not get unsupervised compute. It can propose, edit within the globs, eval, and log, while I keep the train gate. That boundary is the product, and this MVP is where the product starts, with more shape still coming.
The code lives at github.com/rdegraci/maximus, and you can go back home from here.