Code AI evaluations
Rigorous evaluation of code-generation models by senior developers. RLHF preferences, hallucination detection, debugging of coding agents.
Cohorte AI
Evaluations, RLHF, and red-teaming by French-Canadian human experts.
Quality data for demanding AI labs.
Based in Montreal — Canada
Expertise
SaaS & business AI
Evaluation of response relevance for enterprise use cases and SaaS products. Contextual validation by operators of platforms in production.
Agents & agentic workflows
Evaluation of autonomous agent behavior, red-teaming of agentic pipelines, validation of multi-step reasoning.
Approach
The value of human evaluation comes from expert human judgment, not from the absence of tools. Our experts use modern AI tools (Claude, Cursor, smart IDEs) to amplify their productivity — the same way a doctor uses diagnostic tools. But every final judgment is strictly human: timestamped, signed, traceable, tested against fraud.
- Inter-annotator agreement measured on every batch
- Golden datasets for continuous calibration
- Full traceability: who annotated what, when, on what basis
- Transparent workflow, declared to the client
Transparency pledge
AI has become a daily tool in every technical job in 2026. We acknowledge this openly. Here is precisely what we do and don't do:
- Yes: our experts use Claude, Cursor & similar tools to fact-check, read dense code, brainstorm edge cases, or structure their notes.
- No: zero AI output is ever pasted into your deliverables. The final judgment is human.
- Anti-fraud audits: we regularly test our annotators with prompts designed to detect undisclosed AI use.
- Traceability: every annotation carries the human's ID, reflection time, and exact version of guidelines applied.
Our conviction: what you're buying isn't the absence of AI — it's the expert human judgment, irreducible, that decides in the end.
Target clients
AI labs and startups training or evaluating models touching French, code, or business use cases. Priority focus on the Canadian ecosystem (Cohere, Borealis, Mila spinouts), with openness to demanding international clients.
Contact
Cohorte AI launches in May 2026. Accepting 2–3 pilot clients.