What Should an AI Governance Contract Cover?

Businesses adopt AI faster than their contracts, policies and approval processes can keep up. A company may already be using AI through a vendor, an employee’s browser, a customer-facing feature or a model behind an API — without ever having adopted an AI strategy.

That creates practical questions. What data may be submitted to an AI system? May the provider use prompts, files or outputs to train its models? Who owns or may use AI-assisted output? What has the vendor actually promised about performance and security? What must the business tell customers or employees? And who bears the cost when the system produces an inaccurate, biased, infringing or insecure result?

Key takeaways: AI governance is not only a policy exercise. For most businesses it begins with contracts — what data may enter a system, what the provider may do with it, who owns the output, and who carries the risk when something goes wrong. The right structure depends on the system, the data, the people affected and the role the business plays in the AI supply chain. This is an overview; each section links to a deeper treatment.

What Is AI Governance?

AI governance is the set of decisions, policies, contracts and controls a business uses to manage how AI is selected, purchased, built, deployed, monitored and retired. No single contract or policy completes the work. The right structure depends on the system, the data involved, the people affected, the jurisdictions in which it is used, and the company’s role in the supply chain.

The voluntary NIST AI Risk Management Framework (currently under revision) organises AI risk management around four functions: govern, map, measure and manage. It offers a useful operational vocabulary, but using the framework does not by itself establish legal compliance.

For contract-focused counsel the task is narrower: identify the business decision, work out which party controls each material risk, and document the allocation. Allocate the risk by contract. Do not represent that a contract review is an audit of the training corpus.

Why Start With the Contract?

Policies govern employees. Technical controls restrict access. But the contract usually determines what a business can require from an outside provider — and what recourse exists when the provider falls short.

A vendor’s marketing describes the product. The agreement should address the commitments that matter to the specific deployment: permitted and prohibited uses; treatment of prompts, inputs, files, outputs and metadata; model-training and product-improvement rights; confidentiality and security; subprocessors and model-provider dependencies; IP ownership and licence rights; testing and change management; incident notification; warranties, indemnities, liability limits and insurance; suspension, termination, data return and deletion; and assistance with customer inquiries or investigations.

The contract should match the actual use case. A low-risk internal drafting aid, an employment-screening tool and a customer-facing automated decision system do not present the same facts and do not need the same controls.

What Should an AI Vendor Agreement Address?

Data use and model training. The agreement should identify what the provider receives and what it may do with it. A defined term like “Customer Data” may not clearly cover prompts, feedback, generated output, telemetry, embeddings or files passed to a third-party model. Worth asking: does the provider train on customer content; is there an opt-out and does it reach every model and subprocessor; how long is content retained; may it be used for benchmarking, abuse monitoring or human review; what deletion commitments survive termination; and does the provider disclose material changes to its models or data practices?

These questions matter most when personal information, confidential information, regulated data, source code or third-party content may enter the system. For a fuller list, see our seven questions to ask before connecting an AI vendor to your systems.

Output rights and intellectual property. “Ownership” language alone may not answer whether an output is protectable, non-infringing, exclusive or fit for the intended use. The U.S. Copyright Office’s current position is that copyright protects human authored expression, not material generated entirely by artificial intelligence, while works created with AI assistance may qualify to the extent they contain sufficient human authorship. See the Office’s Copyright and Artificial Intelligence, Part 2: Copyrightability.

A contract review should therefore separate ownership as between the parties, the existence and scope of copyright protection, licence rights in provider materials or model components, restrictions affecting open-source components, third-party infringement risk, and the evidence needed to document meaningful human authorship.

Performance claims and validation. AI performance claims should be defined and supported rather than repeated as marketing shorthand. In its final action involving Workado, the Federal Trade Commission challenged unsupported claims that an AI-content detector was highly accurate; the FTC’s final-order announcement illustrates why a business should look at the evidence, test conditions, error rates and limitations behind a measurable AI claim. Before relying on an automated detector or a decision threshold, work out how performance was measured for the intended population and use.

If you are the one making the claims, see what your customer contract must say when your product uses AI.

What About Internal AI Use Policies?

A policy that says “use AI responsibly” does not tell anyone which tools are approved or what may be entered into them. The questions a workable policy has to answer — approved systems and account tiers, prohibited data, human review before output reaches a customer, disclosure, and how the policy squares with the handbook and confidentiality agreements already in place — are covered in our guide to workplace AI-use policies.

What Should Employers Review Before Using AI?

Employers may use automated tools to source candidates, screen applications, assess interviews, monitor workers or recommend employment decisions. The analysis depends on the tool, the decision, the people affected and the jurisdiction.

The Americans with Disabilities Act restricts employers from using employment tests or other selection criteria that screen out or tend to screen out individuals with disabilities unless the criteria are job-related and consistent with business necessity, and it separately requires reasonable accommodations absent undue hardship. See 42 U.S.C. § 12112(b)(5)–(6).

Before deployment, consider what decision the system makes or influences; what data and proxy variables it uses; whether applicants and employees receive appropriate notice; how a person requests an accommodation or human review; whether the vendor provides validation and testing information; whether results are monitored for unintended effects; and how the arrangement allocates responsibility between employer and vendor.

A multistate employer should not assume one national workflow resolves every state and local requirement. Tie the review to the locations and roles involved. See our employment practice for the multistate version of this problem.

How Do Privacy Terms and DPAs Fit In?

An existing data processing addendum may not address AI-specific data flows. Confirm whether personal information is used for training, evaluation, abuse monitoring, human review or model improvement, and whether it moves to additional model providers.

An AI-focused privacy review may examine the parties’ roles and processing instructions, purpose limitations, categories of data and affected individuals, sensitive-data restrictions, retention and deletion, subprocessors and cross-border transfers, security and incident response, assistance with individual-rights requests, limits on combining or monetising data, and the representations already made in privacy notices and customer contracts.

The goal is consistency: a business should not promise customers one treatment of their information while granting a provider broader rights elsewhere. See enterprise SaaS agreements, MSAs and DPAs and our privacy policy and terms practice.

What About Open-Source Models and Components?

“Open source” is not a complete licence analysis. AI products combine code, model weights, datasets, APIs, libraries and generated components under different terms. A review may need to identify which components are in use, the licence attached to each, any attribution, notice, source-availability or use restrictions, restrictions imposed by model-specific licences or acceptable-use policies, whether the planned distribution or hosted use triggers additional duties, and whether customer commitments exceed the rights available upstream.

This contract-and-licence review is different from a training-data provenance audit. A firm may advise on contractual rights and disclosed licences without representing that it has traced or cleared every item in a model’s training corpus.

How Does AI Governance Improve Diligence Readiness?

A buyer, investor, enterprise customer or insurer may ask a company to explain how it uses AI and what rights it holds. A company that waits for diligence to begin often struggles to reconstruct basic facts.

A practical readiness file may include an inventory of approved systems and use cases, the relevant vendor and model-provider agreements, data-flow and subprocessor information, AI policies and approval records, testing and monitoring documentation, privacy and security assessments, records supporting product-performance claims, IP and open-source analyses, customer-facing disclosures, and a list of identified exceptions with owners and remediation plans.

Readiness does not mean claiming every risk has been eliminated. It means knowing what the organisation uses, what its contracts say, what evidence supports its claims, and who owns the unresolved decisions. If data itself is the asset, see licensing your data for AI.

When Should a Business Seek AI Governance Counsel?

A focused review is often useful when a business is purchasing or renewing an AI-enabled product; adding an AI feature to a SaaS or platform offering; changing how customer or employee data is used; preparing customer-facing AI terms or disclosures; adopting AI for hiring or employment decisions; licensing data, model access or model components; responding to an enterprise diligence questionnaire; preparing for financing, acquisition or sale; responding to a complaint about an AI-assisted decision; or discovering that employees adopted tools outside the approved process.

Early review preserves negotiating leverage. Once a system is integrated, customer commitments are made and sensitive data has been submitted, the available options narrow.

What Is Outside a Contract-Focused Engagement?

Scope should be explicit. Accord & Shield Legal’s AI governance work is contract- and policy-focused. It may include AI use policies, vendor and model-provider agreements, customer-facing terms, AI-output ownership, privacy terms and DPAs, employment-use review, open-source and model-component licensing, data-licensing agreements, and diligence readiness.

It does not include a technical or forensic audit of a model’s training corpus, a training-data provenance review, a dataset-licence audit, or remediation of a potentially tainted training corpus. Where an engagement identifies a need for that work, the matter may require referral to or coordination with appropriately qualified technical professionals or separate counsel.

The distinction protects the client and describes the work accurately: contract counsel can allocate identified risk without claiming to have audited the corpus. The engagement letter should confirm the actual scope, assumptions, client responsibilities, excluded services, and any referral or co-counsel arrangement.

A Practical Starting Point

A business does not need to solve every AI question at once. Five will do to begin:

1. What AI systems are we using?
2. What data enters those systems, and where does it go?
3. What decisions or customer features depend on the output?
4. What do our contracts, policies and public statements promise?
5. Which material risks remain unassigned or unsupported?

The answers help prioritise contract amendments, policy updates, diligence, testing and escalation.

How Accord & Shield Legal Helps

Accord & Shield Legal reviews and negotiates commercial contracts and technology agreements for businesses in Arizona, California and Texas, and advises on AI governance as a contract and policy matter.

Nadine Deeb is admitted in Arizona, California and Texas. Before private practice, she served as the sole in-house legal resource at a California technology company, supporting a workforce operating across nineteen states.

Sources

• NIST AI Risk Management Framework — official National Institute of Standards and Technology page
• U.S. Copyright Office, Copyright and Artificial Intelligence, Part 2: Copyrightability (January 2025) — official report
• FTC Approves Final Order against Workado, LLC (August 28, 2025) — official Federal Trade Commission press release
• 42 U.S.C. § 12112 — official United States Code, Office of the Law Revision Counsel

This article is general information from Accord & Shield Legal, PLLC and is not legal advice. Reading it does not create an attorney-client relationship. For guidance on your specific situation, please consult a qualified attorney.