Connection check
verified live · 27h ago
xfms
Pick the right LLM for any task. Ranked shortlist with rationale across 8 evaluators.
Tools
5
GitHub stars
—
Installs / wk
—
Licence
—
Transport
streamable-http
Last checked
27h ago
Tools & capabilities
5 toolsRead from the running server on 27h ago.
benchmark
read-only
purpose*
Run a live A/B test against the engine's TOP 3 PICKS for a stated purpose — the engine chooses the candidates from the full catalog. Generates 5 representative test queries (auto-e… Run a live A/B test against the engine's TOP 3 PICKS for a stated purpose — the engine chooses the candidates from the full catalog. Generates 5 representative test queries (auto-expands to 10 or 15 if results are too close to call), runs them through the picked models in parallel, and returns real cost, latency, and plain-English commentary on who won what. Use AFTER `pick` or `rank` when the user wants the engine's own picks stress-tested with live data. DO NOT use this when the user has already named specific candidate models — the engine will ignore the names and test its own picks. Use `compare` instead in that case. Costs more than `rank` (15+ live LLM calls).
compare
read-only
primarypurpose*model_ids*
Run a live A/B test between 2–5 user-specified models for a stated purpose. NO ranking step — the supplied model_ids ARE the candidate set. Generates 5 representative test queries… Run a live A/B test between 2–5 user-specified models for a stated purpose. NO ranking step — the supplied model_ids ARE the candidate set. Generates 5 representative test queries from the purpose, runs them through every named model in parallel, and returns real cost, latency, and plain-English commentary on who won what. Unknown IDs are dropped with a note; if fewer than 2 IDs resolve, the call refuses. Use this whenever the user names specific models to compare (e.g. 'A/B test X and Y'). For engine-chosen candidates, use `benchmark` instead. Costs more than `rank` (10+ live LLM calls). Free-tier note: when any candidate ends in ':free', the probe is capped at 3 queries (no adaptive expansion) because free-tier rate limits often push longer probes past the deploy's 5-minute ceiling — evidence will be shallower. The commentary surfaces this when it happens.
discover
read-only
purpose*
Show which quality dimensions matter for a stated purpose, WITHOUT ranking any models. Returns the inferred weights and the discovery-walk trace. Useful for understanding how XFMS… Show which quality dimensions matter for a stated purpose, WITHOUT ranking any models. Returns the inferred weights and the discovery-walk trace. Useful for understanding how XFMS interprets the purpose before committing to a pick.
pick
read-only
purpose*
Return the single best LLM for a stated purpose. Concise output, no list. Use when the user has settled on the criteria and just wants one answer. Return the single best LLM for a stated purpose. Concise output, no list. Use when the user has settled on the criteria and just wants one answer.
rank
read-only
top_nprimarypurpose*capabilities