EvalPal
Eval tracking for AI features Free · tamper-evident · shareable

Show the denominator.

Every AI team says its feature is getting better. EvalPal shows how many test cases exist and how many each version passes, on one chart that reads the same for every team.

Start free → Open the example

Your test content never leaves your machine. You send case IDs, fingerprints and pass or fail.

Example · live from the APIFull dashboard →
Loading example…
Evals in test set Latest version Previous Older
The problem

Every AI chart goes up and to the right.

A pass rate can rise because the product improved, or because the test changed. Most reports don't let you tell which. These are the three ways numbers drift, and what EvalPal does about each.

Drop the hard cases

Cases can be added, never quietly removed. A removal needs a written reason, is marked on the chart in red, and stays in the ledger for good.

Report only the percentage

The denominator is always on screen. Bars show the size of the test set every day, so a dip after 50 new hard cases reads as honest work, not a regression.

Retire the old version

Old versions keep being scored. Each version is a line on the same set, so progress shows as a gap you can see, not a claim.

One view

Learn to read it once. Read it everywhere.

Bars: the test set

How many cases existed that day. It grows as your team finds new failures in production.

Lines: each version

How many of that day's cases a pipeline version passed. Skipped cases count as failed.

Set v4 +56

Notes: what changed

New set versions, removals and edits are labelled on the day they happened.

✓ No cases removed

Integrity line

Under every chart: removals, edits, skipped cases and backfilled history, spelled out.

Trust, both ways

You want to trust your team. They want to be trusted.

For executives

One picture for every feature, every team.

  • No briefing needed. The chart and its integrity line say what happened.
  • Open the link any morning. It's the same view your team sees.
  • Check it yourself. The verify page rebuilds every number in your browser.
For teams

Show your work without a slide deck.

  • When a score dips because you added hard cases, the chart says so.
  • One HTTP request from the eval harness you already run.
  • Send the link, not a screenshot. It stays current.
Tamper-evident by default

Every change is on the record.

Each upload, new case, edit and removal becomes an entry in an append-only ledger. Every entry carries the SHA-256 hash of the one before it, so changing history breaks the chain. EvalPal records each chain's head every hour.

Anyone with the dashboard link can open the verify page. It downloads the ledger, recomputes every hash and rebuilds the dashboard numbers in their own browser.

Verify the example →
What entries look like (illustrative)
#212
cases_added · allergen-checkset v4 · 56 cases · ids + fingerprints onlyprev 4c1e…9a07 → 7f30…e2b1
#213
run_recorded · v3.0passed 212 of 221 · ran 221 · results sha256 b8d2…41c0prev 7f30…e2b1 → 0a9c…57de
#214
cases_removed · 2 casesreason: “duplicates of sl-0012 and sl-0019”prev 0a9c…57de → c44b…1f83
#215
run_recorded · v2.4passed 176 of 228 · ran 228prev c44b…1f83 → 91e6…d0aa
How it works

Three steps. About ten minutes.

1

Register an app

Sign up with an email and password. Name the product you evaluate and you get an upload key.

2

Post each run

After your eval harness runs, send one JSON document per feature: version, case IDs, fingerprints, pass or fail.

3

Share the link

Turn on sharing and send the dashboard to whoever needs it. They can view and verify. Nothing else.

# after each eval run
curl https://evalpal.io/v1/runs \
  -H "Authorization: Bearer $EVALPAL_KEY" \
  -H "Content-Type: application/json" \
  -d '{"feature": "allergen-check", "pipeline_version": "v3.0",
       "cases": [{"id": "ac-0001", "fingerprint": "sha256:9f2c…", "pass": true}, …]}'
Read the docs →
Pricing

Tracking is free. Verification is a service.

Ledger
Free
  • Unlimited runs, up to 25 apps
  • Tamper-evident ledger with hourly anchors
  • Shareable dashboards anyone can verify
  • No test content stored, ever
Start free
Verified
On request
  • We run your test set against your endpoint ourselves
  • Independent grading, with the method written up
  • A Verified result you can put in front of buyers and boards
Request verification