Connection check
verified live · 28h ago
rubrkit
MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.
Tools
41
GitHub stars
—
Installs / wk
—
Licence
—
Transport
streamable-http
Last checked
28h ago
Tools & capabilities
41 toolsRead from the running server on 28h ago.
rubrkit_add_golden_case
tagsinput*originweightcriteriaoriginRef
+2
Add a golden case (a real edge-case input with a confirmed-correct output) to an artifact bundle via POST /api/v1/artifact-bundles/{artifactBundleId}/golden-cases. Requires artifac… Add a golden case (a real edge-case input with a confirmed-correct output) to an artifact bundle via POST /api/v1/artifact-bundles/{artifactBundleId}/golden-cases. Requires artifact_bundles:write for API keys.
rubrkit_apply_audit_rewrite
auditRunId*artifactBundleId*
Apply a completed audit's AI rewrite to the bundle files (a new version per changed file). Requires audits:run for API keys. Apply a completed audit's AI rewrite to the bundle files (a new version per changed file). Requires audits:run for API keys.
rubrkit_convert_to_rubr_flow
modelTieroutputPathtargetFileIdreasoningEffortartifactBundleId*targetVersionNumber
+1
Start an async rubr_flow conversion job. Requires rubr_flow:convert for API keys. Start an async rubr_flow conversion job. Requires rubr_flow:convert for API keys.
rubrkit_create_artifact_bundle
name*settingsdescriptioncustomRubric
Create an artifact bundle through /api/v1/artifact-bundles. Requires artifact_bundles:write for API keys. Create an artifact bundle through /api/v1/artifact-bundles. Requires artifact_bundles:write for API keys.
rubrkit_create_drift_monitor
cadencerepeatspinnedVersionId*artifactBundleId*
Create a drift monitor pinned to a file version via POST /api/v1/artifact-bundles/{artifactBundleId}/drift-monitors. Requires artifact_bundles:write for API keys. Create a drift monitor pinned to a file version via POST /api/v1/artifact-bundles/{artifactBundleId}/drift-monitors. Requires artifact_bundles:write for API keys.
rubrkit_delete_artifact_bundle
artifactBundleId*
Archive an artifact bundle (soft delete) through DELETE /api/v1/artifact-bundles/{artifactBundleId}. The bundle moves to the archived state and can be restored or permanently delet… Archive an artifact bundle (soft delete) through DELETE /api/v1/artifact-bundles/{artifactBundleId}. The bundle moves to the archived state and can be restored or permanently deleted later. Requires artifact_bundles:write for API keys.
rubrkit_export_proof_report
evalRunIdauditRunIdsourceFileIdincludeExportrubricVersioncandidateFileId
+5
Start an async proof report job with export storage enabled by default. Requires evals:run for API keys. Start an async proof report job with export storage enabled by default. Requires evals:run for API keys.
rubrkit_hard_delete_artifact_bundle
artifactBundleId*
Permanently delete an archived artifact bundle and all of its files, versions, and run history through DELETE /api/v1/artifact-bundles/{artifactBundleId}/permanent. This is irrever… Permanently delete an archived artifact bundle and all of its files, versions, and run history through DELETE /api/v1/artifact-bundles/{artifactBundleId}/permanent. This is irreversible and only works on bundles that are already archived. Requires artifact_bundles:write for API keys.
rubrkit_list_artifact_bundles
limitstatus
List artifact bundles through /api/v1/artifact-bundles. Requires artifact_bundles:read for API keys. List artifact bundles through /api/v1/artifact-bundles. Requires artifact_bundles:read for API keys.
rubrkit_list_artifact_bundle_versions
limitartifactBundleId*
List bundle-level version events. Requires artifact_bundles:read for API keys. List bundle-level version events. Requires artifact_bundles:read for API keys.
rubrkit_list_artifact_files
limitartifactBundleId*
List files in an artifact bundle. Requires files:read for API keys. List files in an artifact bundle. Requires files:read for API keys.
rubrkit_list_audits
limitartifactBundleId*
List persisted audit history. Requires artifact_bundles:read for API keys. List persisted audit history. Requires artifact_bundles:read for API keys.
rubrkit_list_drift_monitors
artifactBundleId*
List the drift monitors for an artifact bundle via /api/v1/artifact-bundles/{artifactBundleId}/drift-monitors. Requires artifact_bundles:read (or artifacts:pull) for API keys. List the drift monitors for an artifact bundle via /api/v1/artifact-bundles/{artifactBundleId}/drift-monitors. Requires artifact_bundles:read (or artifacts:pull) for API keys.
rubrkit_list_evals
limitartifactBundleId*
List eval history. Requires artifact_bundles:read for API keys. List eval history. Requires artifact_bundles:read for API keys.
rubrkit_list_file_versions
limitfileId*artifactBundleId*
List immutable versions for a file. Requires files:read for API keys. List immutable versions for a file. Requires files:read for API keys.
rubrkit_list_golden_cases
statusartifactBundleId*
List the golden cases for an artifact bundle via /api/v1/artifact-bundles/{artifactBundleId}/golden-cases. Requires artifact_bundles:read (or artifacts:pull) for API keys. List the golden cases for an artifact bundle via /api/v1/artifact-bundles/{artifactBundleId}/golden-cases. Requires artifact_bundles:read (or artifacts:pull) for API keys.
rubrkit_list_job_events
jobId*limit
Read append-only job progress events. Requires jobs:read for API keys. Read append-only job progress events. Requires jobs:read for API keys.
rubrkit_list_rubr_flow_conversions
limitartifactBundleId*
List conversion history. Requires artifact_bundles:read for API keys. List conversion history. Requires artifact_bundles:read for API keys.
rubrkit_poll_job
jobId*
Read async job progress. Requires jobs:read for API keys. Read async job progress. Requires jobs:read for API keys.
rubrkit_read_api_doc
slugOrPath*
Read a docs/api Markdown guide or OpenAPI source by slug, file name, or docs/api path. Read a docs/api Markdown guide or OpenAPI source by slug, file name, or docs/api path.
rubrkit_read_artifact_bundle
artifactBundleId*
Read artifact bundle details, files, and file tree. Requires artifact_bundles:read for API keys. Read artifact bundle details, files, and file tree. Requires artifact_bundles:read for API keys.
rubrkit_read_artifact_file
fileId*artifactBundleId*
Read the latest file metadata and content. Requires files:read for API keys. Read the latest file metadata and content. Requires files:read for API keys.
rubrkit_read_audit
auditRunId*artifactBundleId*
Read a single audit run. Requires artifact_bundles:read for API keys. Read a single audit run. Requires artifact_bundles:read for API keys.
rubrkit_read_drift_observations
driftMonitorId*
List the recorded observations for a drift monitor via /api/v1/drift-monitors/{driftMonitorId}/observations. Requires artifact_bundles:read (or artifacts:pull) for API keys. List the recorded observations for a drift monitor via /api/v1/drift-monitors/{driftMonitorId}/observations. Requires artifact_bundles:read (or artifacts:pull) for API keys.
rubrkit_read_eval
evalRunId*artifactBundleId*
Read one eval run, including the `stats` statistical verdict on advanced-mode runs (empty for standard runs). Requires artifact_bundles:read for API keys. Read one eval run, including the `stats` statistical verdict on advanced-mode runs (empty for standard runs). Requires artifact_bundles:read for API keys.
rubrkit_read_input_drift
binCountwindowDaysartifactBundleId*
Compare recent production inputs against a prior reference window (population stability index) for an artifact bundle via GET /api/v1/artifact-bundles/{artifactBundleId}/drift/inpu… Compare recent production inputs against a prior reference window (population stability index) for an artifact bundle via GET /api/v1/artifact-bundles/{artifactBundleId}/drift/input. Requires artifact_bundles:read (or artifacts:pull) for API keys.
rubrkit_read_openapi_contract
Read the canonical docs/api/openapi.yaml contract. Read the canonical docs/api/openapi.yaml contract.
rubrkit_read_proof_report
proofReportId*artifactBundleId*
Read one proof report. Requires artifact_bundles:read for API keys. Read one proof report. Requires artifact_bundles:read for API keys.
rubrkit_read_rubr_flow_conversion
conversionId*artifactBundleId*
Read one conversion report. Requires artifact_bundles:read for API keys. Read one conversion report. Requires artifact_bundles:read for API keys.
rubrkit_read_validity_drift
artifactBundleId*
Correlate rubric scores against recorded outcome metrics for an artifact bundle to check whether the rubric is still predictive via GET /api/v1/artifact-bundles/{artifactBundleId}/… Correlate rubric scores against recorded outcome metrics for an artifact bundle to check whether the rubric is still predictive via GET /api/v1/artifact-bundles/{artifactBundleId}/drift/validity. Requires artifact_bundles:read (or artifacts:pull) for API keys.
rubrkit_record_outcome
inputHashoccurredAtinputFeatureoutcomeLabeloutcomeMetricartifactBundleId*
+1
Record a production outcome event (input feature, metric, and/or label) for an artifact bundle via POST /api/v1/artifact-bundles/{artifactBundleId}/outcomes. Requires artifact_bund… Record a production outcome event (input feature, metric, and/or label) for an artifact bundle via POST /api/v1/artifact-bundles/{artifactBundleId}/outcomes. Requires artifact_bundles:write for API keys.
rubrkit_restore_artifact_bundle_version
messageversionNumber*artifactBundleId*
Restore files from a prior artifact-bundle version by creating new file versions. Requires artifact_bundles:write and files:write for API keys. Restore files from a prior artifact-bundle version by creating new file versions. Requires artifact_bundles:write and files:write for API keys.
rubrkit_restore_file_version
fileId*messageversionNumber*artifactBundleId*
Restore a prior file version by creating a new version. Requires files:write for API keys. Restore a prior file version by creating a new version. Requires files:write for API keys.
rubrkit_retire_golden_case
goldenCaseId*
Retire a golden case via DELETE /api/v1/golden-cases/{goldenCaseId}. Requires artifact_bundles:write for API keys. Retire a golden case via DELETE /api/v1/golden-cases/{goldenCaseId}. Requires artifact_bundles:write for API keys.
rubrkit_run_drift_check
driftMonitorId*
Queue a drift check immediately instead of waiting for the monitor's schedule, via POST /api/v1/drift-monitors/{driftMonitorId}/check. The first check on a new monitor calibrates i… Queue a drift check immediately instead of waiting for the monitor's schedule, via POST /api/v1/drift-monitors/{driftMonitorId}/check. The first check on a new monitor calibrates its baseline. Costs one audit per repeat. Requires artifact_bundles:write for API keys.
rubrkit_run_evals
moderepeatsmaxCasestestCasesauditRunIdsourceFileId
+9
Start an async eval run. Requires evals:run for API keys. Set mode: "advanced" with baselineVersionId and candidateVersionId to run a statistically-backed A/B (repeats defaults to… Start an async eval run. Requires evals:run for API keys. Set mode: "advanced" with baselineVersionId and candidateVersionId to run a statistically-backed A/B (repeats defaults to 3, max 5).
rubrkit_search_api_docs
limitquery*
Search the canonical docs/api Markdown guides and OpenAPI operation summaries. Search the canonical docs/api Markdown guides and OpenAPI operation summaries.
rubrkit_set_drift_monitor_status
status*driftMonitorId*
Pause or resume a drift monitor via PATCH /api/v1/drift-monitors/{driftMonitorId}. Requires artifact_bundles:write for API keys. Pause or resume a drift monitor via PATCH /api/v1/drift-monitors/{driftMonitorId}. Requires artifact_bundles:write for API keys.
rubrkit_start_audit
forcemodelTierrubricKeyartifactTypetargetFileIdreasoningEffort
+4
Start an async audit job. Audit results are cached per account by a content hash of their inputs; pass force: true to bypass the cache and run a fresh audit (the same as the CLI's… Start an async audit job. Audit results are cached per account by a content hash of their inputs; pass force: true to bypass the cache and run a fresh audit (the same as the CLI's --no-cache flag). Requires audits:run for API keys.
rubrkit_update_artifact_file
fileId*content*messagemediaTypeartifactBundleId*
Write a new editable version for a file. Requires files:write for API keys. Write a new editable version for a file. Requires files:write for API keys.
rubrkit_upload_artifact_files
files*artifactBundleId*