A post in the series on independent AI auditing, picking up from What the Law Can See.
Dominique Charlie Oquendo
Founder and Principal Consultant, Intentional Data. November 2026.
Claude, an AI model made by Anthropic, helped with research and editing. Anthropic is also one of the companies whose evaluator rules this post reads, and I read its rules the same way I read every other company's. The argument and the writing are my own. Ask for the repo to see how Claude was used!
Introduction
Last time I ended on sunlight, the idea that things done out in the open are usually the better for it. I am still here unpacking, taking in ridiculous amounts of information and asking a ridiculous amount of questions. States and firms are scrambling to work out how to govern a technology that is moving faster than any before it, and the conversations I have had the privilege to join keep landing on independent auditing, and on the holes that block it as an option. We have yet to build a way to say who picks the evaluator, who pays the evaluator, what the evaluator is allowed to look at, and who gets to see what they found.
Before we get to the infrastructure we would need to produce independent audits, we have to define two things. The first is who is allowed to be the auditor, meaning what a person or a firm has to be in order to do the work. The second is what the audit has to score, meaning the list of things the evaluation is required to look for. Sometimes a company writes each of those decisions and sometimes a government does, and I want to read them side by side.
Both are starting to be written down. In the last six weeks, two companies that build the largest AI models, Anthropic and OpenAI, published their own criteria for the people who evaluate them 1 2. In Singapore, the AI Verify Foundation, a not-for-profit owned by the government's Infocomm Media Development Authority, accredited its first five AI testing firms against a published list 3. Illinois, California and Europe have written their versions into law. Read together, the rules for who may audit are written in far more detail than the rules for what the audit must look for.
1. Which organizations even get audited
The frontier AI laws in New York, Illinois and California do not reach every AI company, and the limits in them decide which harms an audit can ever see. This section walks through those limits in order, and then shows the harms that fall outside them.
Two lines decide whether one of these laws reaches a company. The first is computing power. A company that trains a model with more than 10^26 calculations is a frontier developer, and every frontier developer has duties of care. The second is yearly revenue. A frontier developer that took in more than $500 million in the previous calendar year is a large frontier developer, and the heaviest duties land on those companies. Only Illinois requires an audit, and it requires one only from large frontier developers 4 5 6.
The definition of catastrophic risk sets a third limit, on what counts as harm. All three laws use the same definition. Catastrophic risk is a single incident that kills or seriously injures more than 50 people, or causes more than $1 billion of property damage. The incident has to involve one of three things: a model giving expert-level help with a chemical, biological, radiological or nuclear weapon, a model acting with no meaningful human oversight in a cyberattack or serious crime, or a model evading the control of its developer or user 4 5 6. The harm numbers and the revenue number are easy to mix up, so here they are apart: 50 people and $1 billion measure the harm from one incident, and $500 million measures how much money a company took in last year.
The limits in these laws miss harms, and the governor who vetoed California's first attempt at this policy said so. California tried once before with a bill called SB 1047, and Governor Newsom vetoed it on September 29, 2024. He wrote that by focusing only on the most expensive and large-scale models, the bill "establishes a regulatory framework that could give the public a false sense of security about controlling this fast-moving technology" 7. He named who the cutoff would miss, writing that "smaller, specialized models may emerge as equally or even more dangerous than the models targeted by SB 1047" 7. And he named what the bill does not ask: it "does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data" 7. Critical decision-making means decisions such as benefits determinations, hiring, housing and child welfare screening. I work around these decisions in my own practice, and I expect the people you serve meet them too.
Here are harms that fall below the line. A chatbot contributing to one person's death, a hiring screen filtering out disabled applicants and a case management system exposing a community's records are each real harm to real people, and none of them is catastrophic risk as these laws define it. If the policies we are creating do not count the harms that affect us as harms worth responding to, they will not show up in audit criteria, and they will not be monitored.
2. Who is allowed to be the auditor, and who wrote that rule
The most detailed rules for who may audit come from lawmakers in Illinois, California and Europe. Anthropic and OpenAI, two of the companies being evaluated, wrote shorter ones for themselves, and Singapore's AI Verify Foundation, which accredits testing firms, says it does not certify independence. The table below puts nine documents side by side. The first column says who wrote each document, the second says what the document requires an auditor to be, and the third says what the document says the audit will score. We start with the second column and read it in three groups, the companies first, then Singapore, then the laws, so that we can compare what firms wrote with what law requires.
| Who wrote it | What the auditor has to be | What the audit has to score |
|---|---|---|
| Anthropic (company), October 15, 2026 1 | Expertise, independence ("shouldn't hold equity in the company they're evaluating, or be financially or operationally dependent on it"), enough access, and freedom to publish | Set evaluator by evaluator, with no single required list |
| OpenAI (company), September 22, 2026 2 | Relevant expertise, disclosed conflicts of interest, safeguards against commercial pressure, information security, enforceable confidentiality | Four priority areas: safety cases, safeguards, capability and alignment evaluations, and investigations of serious misalignment incidents |
| Singapore's AI Verify Foundation (government-owned not-for-profit), October 14, 2026 3 | Technical competency, good standing, financial sustainability, operational readiness. It does not certify independence. | Five risk areas, including bias and fairness |
| Europe's code of practice (voluntary) 9 | Expertise in the risk, evaluation experience, information security, confidentiality, and freedom from the company's control over conclusions | Four specified risks: chemical, biological, radiological and nuclear weapons, loss of control, cyber offence, harmful manipulation |
| Europe's AI Act, notified bodies (law) 10 | Independent of the provider, of anyone with an economic interest in the system and of the provider's competitors, no consultancy, impartiality procedures, liability insurance | For high-risk AI used on people: bias, risks to health, safety and fundamental rights, and the groups affected |
| Illinois (law) 5 | Demonstrated competence, no financial interest between auditor and developer in either direction, no payment tied to the result | Every year, for large frontier developers, from January 1, 2028 or 90 days after a company first qualifies: whether the developer follows its own safety framework |
| New York (law) 4 | No audit, so no auditor qualifications | No audit. Large frontier developers file a disclosure statement and renew it every two years. |
| California, auditor registry (law) 8 | Register with the state, file a standard operating procedure, meet detailed independence standards | No company is required to be audited. When one is, the auditor states the scope and objectives of the audit. |
| Colorado Division of Insurance (regulator) 11 | No evaluator requirement found | Testing "to detect unfair discrimination," with the reporting waived |
Anthropic's post names four things an evaluator needs: expertise, independence, enough access and freedom to publish. On independence, its hard stop is that an evaluator "shouldn't hold equity in the company they're evaluating, or be financially or operationally dependent on it." The word is "shouldn't," so this is a recommendation 1.
Anthropic treats what the public gets to see as a separate question. Evaluators have freedom to publish, and Anthropic says it can require an evaluator to redact information that falls in categories set in advance, such as confidential intellectual property 1.
OpenAI asks for safeguards so that pay arrangements do not shape findings, and says those safeguards may include recusal or exclusion periods. The post does not say who pays 2.
Singapore takes a very different approach from the one in the United States. The government's Infocomm Media Development Authority publishes a starter kit for testing applications built on large language models, and the AI Verify Foundation, the not-for-profit it owns, accredits firms by checking their claimed testing services against guidelines such as that kit. The foundation also publishes one list of five risk areas, so accredited firms test against shared language. It charges no fee, and it says in public that it "does not certify that testing conducted by accredited testing firms are independent" 3.
Now the law. Europe's code of practice goes further than either company. It defines an independent external evaluator as one "otherwise free from the Signatory's control in reaching conclusions and/or making recommendations" 9. Europe's AI Act adds that a notified body may not be involved in designing or selling the system it checks, and must carry liability insurance 10.
Illinois asks the auditor to have demonstrated competence, which includes experience employing or contracting with people who have technical expertise in the safety of frontier models. It also bars the developer from hiring an auditor when either side has a financial interest in the other. The audit has to follow "generally accepted auditing standards and best practices," a phrase the Act does not define 5. New York requires no audit, though a company's framework and reports must describe any use of third-party evaluators 4. California goes furthest on process. From January 1, 2029, no one may offer or conduct a covered audit without registering and filing a standard operating procedure, and the state must say prominently that registration is not an endorsement 8.
3. What the audit has to score
The documents I read ask audits to score three different things. Those written for the largest models score catastrophic harm and what a model can do. Europe's law for AI used on people requires examining bias, the groups affected and the harm to them. The state laws that touch the audit itself leave its content open.
The definition of catastrophic risk from the first section reaches an audit in three steps. First, each large frontier developer writes a safety framework about those risks. Second, Illinois, the only state that requires an audit, has the auditor check whether the company follows its own framework 5. Third, because the audit checks the framework, the definition shapes what gets audited only indirectly, and the audit does not measure catastrophic harm itself.
| Who wrote it | What the audit has to score |
|---|---|
| Anthropic 1 | Set evaluator by evaluator |
| OpenAI 2 | Four priority areas, agreed per assessment |
| Singapore's AI Verify Foundation 3 | Five risk areas, including bias and fairness |
| Europe's code of practice 9 | Four specified risks |
| Europe's AI Act 10 | Bias, risks to health, safety and fundamental rights, groups affected |
| Illinois 5 | Whether the developer follows its own safety framework |
| New York 4 | No audit |
| California 8 | The auditor states the scope |
| Colorado Division of Insurance 11 | Testing "to detect unfair discrimination," reporting waived |
For the largest models, OpenAI lists four priority areas: independent review of its safety cases, evaluation of its safeguards, capability evaluations for chemical and biological risk, cybersecurity and AI self-improvement alongside alignment evaluations, and investigation of serious misalignment incidents. It says each assessment starts from a scope the company and the assessor agree on, with the claims registered before the work begins 2. Anthropic gives each outside evaluator its own scope. METR, a nonprofit, reviews Anthropic's risk levels, incidents and practices. SecureBio is reviewing the chemical and biological weapons chapter of Anthropic's August Risk Report, and the consulting firm Accenture does red teaming and safeguard testing 1. Europe's code of practice treats four risks as specified systemic risks: chemical, biological, radiological and nuclear weapons, loss of control, cyber offence and harmful manipulation 9. OpenAI's four areas, Anthropic's evaluator scopes and Europe's four risks cover the same ground.
Europe's AI Act is written for AI used on people, and it requires bias to be examined. For high-risk systems, data has to be examined "in view of possible biases that are likely to affect the health and safety of persons, have a negative impact on fundamental rights or lead to discrimination prohibited under Union law," and providers must take measures to detect, prevent and mitigate any biases found 10. Risk management covers the risks to "health, safety or fundamental rights," and systems that keep learning must guard against biased outputs feeding back into their inputs. Certain organizations that deploy these systems, including public bodies, private entities providing public services and some credit and insurance providers, must also assess the groups likely to be affected before first use 10. Europe wrote two sets of rules, one for AI used on people and one for the largest models, and bias is on the first list. In the code of practice, discrimination and fundamental rights appear in the supporting lists, so a company covers them only if its own process finds them systemic 9.
Singapore's published list names five risk areas, hallucination and inaccuracy, undesirable content, data leakage, vulnerability to adversarial prompts, and bias and fairness. The accredited firms test them by benchmarking, red teaming or both 3. Illinois points the auditor to the developer's own framework. New York and California's SB 53 require no audit, and California's AB 1405 has the auditor state the scope and objectives of each audit 4 6 8. Colorado's Division of Insurance told insurers that their annual report must describe quantitative testing "to detect unfair discrimination." The Division has not adopted the rules that say what that testing means, and it has waived the reporting until it does 11.
What I am left with is where bias shows up. It appears in Europe's binding rules for AI used on people, on Singapore's published list, and in Colorado's insurance requirement, which does not yet run. It does not appear in any of the frontier model documents I read.
4. Watching America build the infrastructure to hold audit criteria
New York and California are both building places to receive audit information. Here is what each one has set up so far.
On September 21, 2026, the Governor's Office announced Marc Gilman as the first hire of New York's Office of Digital Innovation, Governance, Integrity and Trust, and said more staff will join in the coming months. The same release says that starting in November New York will direct large frontier developers to register 12. The law takes effect January 1, 2027. Large frontier developers file a disclosure statement and pay fees that fund the office, and the office publishes the list of filers. From January 1, 2028 it produces an annual report built on incident data, with suggested changes to the law 4.
California has set its dates earlier than its own law did. AB 1405 creates the auditor registry by January 1, 2029 8. On September 18, 2026, Governor Newsom signed Executive Order N-9-26, which directs the Government Operations Agency to finish the registry requirements by December 1, 2027 and to post by May 1, 2027 the application requirements for independent verification organizations under SB 813. It also asks for recommendations by November 16, 2026 that could require on-site independent verification 13. I am currently excited to explore the infrastructure being set up in California, because it could become the place where audit criteria get filed.
5. A federal order to override state laws
A federal order aims to override state AI laws like the ones above, which currently hold the decisions about who audits and what gets scored, and so far it has reached one paused lawsuit in Colorado.
Executive Order 14365, signed December 11, 2025, gave the Attorney General 30 days to create the task force. It gave the Commerce Department 90 days to publish an evaluation naming the state AI laws it considers onerous, and it told the Federal Communications Commission to consider a federal reporting and disclosure standard 14. The Commerce evaluation was due March 11, 2026, and the sources I read through early April reported it unpublished. I have not found a published list since.
On April 24, 2026, the Justice Department moved to intervene in a lawsuit by xAI, a company that builds AI models, against Colorado's 2024 law on algorithmic discrimination, arguing the law violates the Equal Protection Clause of the Fourteenth Amendment 15. On April 27, a federal magistrate judge granted a joint request from xAI and the Colorado attorney general that delayed enforcement of the law 16. On May 14, Governor Polis signed SB 26-189, which replaces the original Act with a narrower one that takes effect January 1, 2027 and, according to one law firm's summary, drops the annual impact assessments and the duty to use reasonable care against algorithmic discrimination 17. The appendix holds four related items: a European standard, a Federal Reserve letter, a withdrawn self-driving vehicle program and a letter from 26 attorneys general.
6. What I am left asking, and what you can ask
Two questions let any of us check an AI evaluation: who did it, and what it was required to look for.
Here is what it means when the goal of a system is chosen before the audit. A healthcare algorithm was built to predict future health care costs, the goal that was predefined before the audit was medical need, ie if the numbers show there is a medical need we will plan to spend more money there. The algorithm gave each patient a risk score, and patients with high scores were offered extra care. The score was built from how much money had been spent on a patient’s health care, on the reasoning that people who cost more need more. Less money had historically been spent on Black patients than on white patients with the same illnesses, so Black patients received lower scores than white patients who were just as sick. A Black patient had to be considerably sicker than a white patient to get the same score, which meant fewer Black patients were flagged for extra care than their health needs called for. The article I am drawing on calls it a model that was competently predicting the wrong thing 18.
The goal was chosen first, and any audit that began afterward would be checking the system against that goal. Massimo Ragnedda, a professor of media and communication at the University of Sharjah, and Maria Laura Ruiu, an associate professor of media and communication at the American University of Sharjah, research digital inequalities and the social consequences of AI. Writing for TechPolicy.Press, they put it this way: "An audit that begins after an institution has chosen the system's objective may certify an unequal policy with impressive technical documentation" 18.
No document in the table scores the uses of AI that are hardest for the people affected to see, and the harms I named in the first section sit outside the table as well. Workplace systems that rank applicants, assign shifts and monitor performance are examples, and they are the uses Gregory Gondwe describes in his piece on Google's governance plan 19.
If an audit can only check what someone decided to look for, the useful thing any of us can ask is who decided. When someone tells me an AI system has been evaluated, I ask two questions. I want to know who did the evaluation, whether the company paid them, what they were allowed to look at, and whether they could publish what they found. I also want to know what the evaluation was required to look for, who wrote the list of requirements, and whether I can read it. Both questions fit in a vendor demo, a board meeting, a funder's question or a contract renewal, and they take a couple of minutes. An empty answer tells me the evaluation chose its own questions.
Ask who did the evaluation, and ask what it was required to look for.
Intentional Data's commitment is community data practice first, AI in service of it, and technology coproduced with the people it affects. How we coproduce technology with the people it affects is still open. What I am curious about next is whether a community organization with lived experience of the systems being scored can hold the role of auditor. If you have experience inside an audit, a participatory evaluation or a community review board, I would love to hear from you at dominique.charlie@intentionaldata.org.
Appendix
The appendix holds four items that bear on the same story and did not belong in the sections above.
Europe's first harmonised standard under the AI Act, EN 18286, covers quality management systems for the providers of AI systems, including how they monitor a system after release. The Commission adopted Implementing Decision (EU) 2026/2274 on October 12, 2026 and published it on October 14. Under the AI Act, a provider that follows a standard whose reference is published this way is presumed to meet the requirements that standard covers 20.
The Federal Reserve's April 2026 replacement of its model risk guidance, SR 26-2, no longer sets enforceable standards and excludes generative AI models and AI systems that take actions on their own 21.
The National Highway Traffic Safety Administration proposed a program in January 2025 for self-driving vehicles that would have required a third-party report on the applicant's safety case. It withdrew the proposal in a notice scheduled for publication on June 26, 2026 22.
Twenty-six attorneys general, writing to congressional leadership on September 23, 2026, asked Congress to preserve state authority and asked for investigators with "direct access to books and records, with public findings" 23.
Endnotes
- Anthropic. October 15, 2026. An update on independent evaluators. anthropic.com
- OpenAI. September 22, 2026. Priorities and principles for effective third party assessments. openai.com
- AI Verify Foundation. October 14, 2026. AI Tester Accreditation Programme, accredited testers and accreditation requirements. aiverifyfoundation.sg
- New York. March 27, 2026. General Business Law article 44-B (RAISE Act, as amended by Chapter 96 of 2026). nysenate.gov
- Illinois. 2026. Senate Bill 315, Public Act 104-0538. ilga.gov
- California. 2025. Senate Bill 53, Chapter 138 of 2025. leginfo.legislature.ca.gov
- Governor Gavin Newsom. September 29, 2024. Veto message, Senate Bill 1047. gov.ca.gov
- California. September 9, 2026. Assembly Bill 1405, Chapter 178 of 2026. leginfo.legislature.ca.gov
- European Commission. 2025. General-Purpose AI Code of Practice, Safety and Security chapter. digital-strategy.ec.europa.eu
- European Union. 2024. Regulation (EU) 2024/1689 (the AI Act), Articles 9, 10, 15, 27 and 31. eur-lex.europa.eu
- Colorado Division of Insurance. Reissued June 10, 2026. Bulletin B-10.004. doi.colorado.gov
- Office of the Governor of New York. September 21, 2026. Announcement of the first hire for the Office of Digital Innovation, Governance, Integrity and Trust. governor.ny.gov
- Governor of California. September 18, 2026. Executive Order N-9-26. gov.ca.gov
- President of the United States. December 11, 2025. Executive Order 14365, Ensuring a National Policy Framework for Artificial Intelligence. Federal Register, December 16, 2025. federalregister.gov
- U.S. Department of Justice. April 24, 2026. Justice Department Intervenes in xAI Lawsuit Challenging Colorado's Algorithmic Discrimination Law. justice.gov
- Colorado Politics. April 28, 2026. Court order delays enforcement of state's first-in-the-nation AI law. coloradopolitics.com
- Holland and Knight. May 18, 2026. Colorado Governor Signs SB 189. hklaw.com
- Massimo Ragnedda and Maria Laura Ruiu. August 20, 2026. AI Audits Need a Power Test, Not Just a Fairness Score. techpolicy.press
- Gregory Gondwe. August 3, 2026. Google's AI Governance Plan Draws the Boundaries of What Counts as Harm. techpolicy.press
- European Commission. October 12, 2026 (published October 14, 2026). Implementing Decision (EU) 2026/2274. eur-lex.europa.eu
- Board of Governors of the Federal Reserve System. April 2026. SR 26-2. federalreserve.gov
- National Highway Traffic Safety Administration. June 26, 2026. ADS-Equipped Vehicle Safety, Transparency, and Evaluation Program; Withdrawal. federalregister.gov
- State attorneys general. September 23, 2026. Letter to congressional leadership on federal AI regulation. oag.ca.gov