Our accuracy test · Oct 31, 2026
Accuracy Audit
In our test, one popular AI chatbot had clearly more problems than Pharos Terminal: 11.4 per 100 statements to our 3.3.
Each Pharos Terminal item goes through two separate AI review steps, an Editor and a Fact Checker, before it’s published. To see how our published items compare, on Oct 31, 2026 we gave three popular AI chatbots, in their no-cost versions, the same lesson requests a parent might type, and checked their answers against the original sources by the same rules we used for ours.
Pharos Terminal
95% range 0.9 to 8.2
4 problems in 122 checkable statements from 20 randomly chosen items: 1 wrong date and 3 details a reviewer couldn’t confirm.
One popular AI chatbot (Chatbot B)
95% range 6.2 to 18.7
13 problems in 114 statements. Most were details a reviewer couldn’t confirm rather than outright errors.
Item by item, it had 8.1 more problems per 100 than we did (95% range 2.0 to 14.9).
Clearly more problems than Pharos Terminal in this test
The other two popular AI chatbots
95% range 0.02 to 4.7
95% range 2.6 to 11.5
Problems per 100 statements. In this test, neither difference from Pharos Terminal was large enough to be sure either way.
Not clearly different from Pharos Terminal
What we got wrong
These are the four problems we found in our own 122 statements:
- Wrong date. A lesson dated a Baptist Standard report Oct 25; the report was published Oct 14, 2026. The quotation was accurate. Corrected Oct 31, 2026.
- Couldn’t confirm. A markets lesson said investors moved money into telecom and defense stocks. Our cited sources support the Treasury and utility moves, not that part.
- Couldn’t confirm. West Virginia overdose deaths (1,470 in 2021 to 642 in the 12 months ending March 2026). The figures match the CDC dataset we cited; one reviewer checked a different CDC series and couldn’t confirm them.
- Couldn’t confirm. A measles death count in a packet’s sample answer. It matches the CDC table we cited; one reviewer used a CDC summary with a different count.
We count a statement as a problem if either reviewer flagged it, so the last two count even though they match the sources we cited.
A “problem” is an error, a made-up quotation or citation, or a detail a reviewer couldn’t confirm from any source. A statement is one checkable sentence, picked by script, not by hand. We checked up to 8 sentences per item, not every sentence. Two separate AI reviewers did the checking; the second reviewer, our own Fact Checker, re-checked 36 of our statements and 72 of the chatbots’, selected differently for each side. Three chatbots tested on one day, in their no-cost versions, don’t represent all AI, and these numbers cover only Pharos Terminal items published Oct 22–30, 2026.