OpenJev scores three local AI models against a hosted baseline

OpenJev loads Qwen3 0.6B, MiniCPM5 2B or Qwen3.5 4B into a browser over WebGPU and runs the same decision two ways at once.

That's a live way to check whether a small local model comes close to a hosted score. The comparison matters for anyone deciding whether to run inference locally or pay for an API call. On a 102-row test set, the three models scored 40.7%, 63.7% and 84.5%, against 88.3% for the hosted Jev reference.

OpenJev gets there two different ways. One method reads the model's raw choice logits and normalizes them across only the options offered. The other prompts the model to write out the same probability distribution as JSON, token by token.

The gap to the hosted score shrinks fast as the models get bigger. It's 47.6 points for the 0.6B model, 24.6 points for the 2B model, and 3.8 points for the 4B model.

The published subset gives one score per model, not a breakdown by method. There's no way to tell how much of the gap comes from parameter count and how much comes from which scoring approach produced it.