For customers in the US
Checking your tests for adverse impact
If a test helps decide who gets hired, you should be able to check whether it passes some groups at a much lower rate than others. Here is how to get the data out, and an honest account of the part we deliberately do not do.
Who holds what, and why
A four-fifths analysis needs two things joined together: pass or fail for each candidate, and each candidate's demographic group.
We hold the outcomes. You hold the demographics. The join happens on your side.
That is not us dodging the work. You already hold applicant demographic data lawfully, normally from a voluntary self-identification form that is kept separate from the hiring file precisely so it cannot influence a decision. You are the party with a lawful basis for holding it and the party who has to run the analysis. Our job is to hand you outcomes that are clean and easy to join.
Getting the export
Signed in as the account that owns the tests, call:
GET /api/compliance/pass-rates?format=csvAdd &code=YOURTESTCODE to limit it to one test. Drop format=csv for JSON, which also carries a per-test summary block.
One row per submitted attempt, with these columns:
| Column | What it is |
|---|---|
| candidate_key | The email address you invited. This is your join key. |
| identity_status | Whether the account that took it matched the address you invited, or whether it arrived through a forwarded link. |
| score / total | Questions right, out of questions asked. |
| score_percent | The score as a percentage. |
| pass_mark_percent | The pass mark in force for that test, as typed by a person on your team. |
| passed | score_percent is greater than or equal to pass_mark_percent. Empty if no pass mark was set. |
| auto_submitted | Whether the attempt was submitted automatically when the clock ran out. |
| submitted_at | When it was submitted. |
Aggregate pass rates are suppressed below five attempts. With four results, a published rate plus one thing you already know identifies a person.
Running the four-fifths check
- Pull the export for the test and the period you care about.
- Join it to your own self-identification records on the email address.
- For each group, work out the pass rate: candidates who passed, divided by candidates who took it.
- Find the group with the highest pass rate. Divide every other group's rate by that one.
- Any ratio below 0.8 is the flag that something needs looking at.
A ratio below 0.8 is a prompt to investigate, not a verdict. Small numbers swing wildly, and the fix is often the pass mark or the question mix rather than the test as a whole. This is also the point where you want your employment counsel, not a vendor page.
What we chose not to build, and why
The obvious alternative is for this product to collect voluntary self-reported demographics itself, keep them apart from the recruiter's view of a candidate, and produce the analysis directly. We decided not to build that without legal advice, and we would rather say so than ship it badly.
- Race, ethnicity and health data are special-category data under GDPR. The lawful bases for holding them are much narrower than for ordinary personal data, and this product has EU candidates.
- The US rules on how self-identification is separated from the hiring file are specific, and getting the separation wrong turns a compliance feature into evidence against the customer.
- Most employers already run a self-identification form. A second one, from a vendor, mostly adds a place for the data to leak.
If you want it built into the product, tell us, and it goes to a lawyer before it goes to a keyboard.
Nothing is inferred, ever
No part of this product guesses a candidate's gender, ethnicity, age, disability or nationality from a name, an email address, a photograph, a writing sample, or anything else. There is no such code, and there will not be.
Inferred demographics are wrong often, and wrong unevenly. An inference that misfires more for one group than another manufactures exactly the bias the analysis is supposed to find, and the person never agreed to be labelled in the first place.
One thing to note about how scoring works
The pass or fail in this export is a comparison of a score against a number a person on your team typed. There is no model, no weighting, no ranking, and no learned scoring anywhere in a recruiter test. That is a deliberate design constraint, not a current state of the roadmap. The full description is in the product summary.