Company positioning vs measured evidence.
Artificial Index now uses two clean perspectives. Company Positioning scores what the vendor publicly documents about the product. Measured Evidence uses independent benchmark results and Artificial Index operational measurements only.
Company Positioning
Capability 35%, Access 15%, Deployment 15%, Privacy/Security 15%, Openness 10%, Cost Efficiency 10%. At least four dimensions and 60% Evidence Confidence are required.
Measured Evidence
Independent or Artificial Index measured observations only. R17's first cohort requires independent intelligence, output throughput and TTFT measurements; scores use 70% intelligence, 15% throughput and 15% latency.
Current score coverage
| System | Company position | Confidence | Measured | Confidence | Measured observations |
|---|---|---|---|---|---|
| Mistral Vibe | 81.5 | 85.6% | — | 0.0% | 0 |
| Grok | 66.7 | 84.1% | — | 0.0% | 0 |
| ChatGPT | 65.2 | 83.9% | — | 0.0% | 0 |
| Gemini | 63.7 | 87.8% | — | 25% | 1 |
| ElevenLabs | 63.6 | 78.1% | — | 0.0% | 0 |
| DeepSeek | 62.1 | 83.7% | — | 0.0% | 0 |
| Claude | 60.5 | 87.8% | — | 30% | 2 |
| Runway | 60.5 | 77.5% | — | 0.0% | 0 |
| Cohere Command | 59.9 | 88.9% | — | 0.0% | 0 |
| Amazon Nova | 53.9 | 79.8% | — | 0.0% | 0 |
| DeepL | 52.0 | 68.8% | — | 0.0% | 0 |
| GitHub Copilot | 42.3 | 68.5% | — | 0.0% | 0 |
| Pinecone Assistant | 41.6 | 74.4% | — | 0.0% | 0 |
| Cursor | 39.2 | 74.6% | — | 0.0% | 0 |
R17 pilot cohort
Measured evidence policy
Independent first
Independent benchmark observations can contribute when exact model mapping and comparison conditions are verified.
Artificial Index measurements
TTFT, throughput, uptime, error rate, structured-output success, tool-use success and task economics can contribute under R14 protocols.
Vendor benchmarks excluded
Company-published benchmark claims may be displayed, but never increase the Measured Evidence Score.
Pending ≠ zero
If there are not enough comparable measurements, the measured score remains Pending.