When a Model Changes, What Do You Reassess?

Providers ship updates continuously. Management systems assume a component that sits still, gets changed deliberately, and leaves a record when it does. The gap between those two facts is where most AI governance programmes quietly stop being accurate — usually not because a team decided to skip reassessment, but because nobody defined what would have triggered one.

The rule this page argues for. Reassess when a change plausibly moves capability, behaviour, data handling or an approved workflow. Log everything else and move on. A process that reassesses every release buries the one that mattered. A process that reassesses nothing cannot tell you it did not.

Not every model release warrants reassessment, and treating them all as material is a failure mode in its own right. The test is whether the change plausibly affects one of four things: what the system is capable of, how it behaves on your inputs, how data is handled, or the workflow the approval was granted for. A change that touches none of them is logged and classified, not reassessed. A change that touches any of them goes through a gate before it reaches production. That sentence is easy to write and awkward to operate, for a reason worth stating plainly at the top: the change is made by somebody else, on their schedule, to a component sitting inside your assessed boundary, and you may not be told. What follows is the material-change test in full, the detection problem and the three levers that address it, a regression model that scales past a handful of systems, what an AI management system actually has to update, deprecation as a continuity risk, and the same analysis applied to prompt, tool and workflow changes — which are usually more consequential than the model updates everyone watches.

A provider-side model update fits none of that shape. You did not authorise it, you cannot test it before it exists, and in the ordinary case you learn about it from a changelog you have to remember to read. The criterion that actually carries the weight is a different one: CC3.4, under which the entity identifies and assesses changes that could significantly impact the system of internal control. That is a detection-and-judgement obligation rather than an authorisation one, and it is the correct home for this problem. Alongside it, CC9.2 covers vendor and business partner risk, and CC7.1 speaks to detection and monitoring procedures that identify configuration changes introducing new vulnerabilities. Together they describe the actual control set: notice, classify, decide.

On the ISO/IEC 42001 side the same work sits mostly in Annex A rather than in the clauses. A.6.2.5 addresses AI system deployment, A.6.2.6 addresses AI system operation and monitoring, A.6.2.4 addresses verification and validation, and A.10 covers third-party and customer relationships, which is where the supplier half of a model dependency belongs. A.5 covers assessing impacts of AI systems, and risk and impact assessment are different exercises, which matters here because a model change more often moves impact than it moves the risk profile. ISO/IEC 42005:2025 gives dedicated guidance on AI system impact assessment, and ISO/IEC 5338:2023 sets out AI system life cycle processes. Both are useful references when you are deciding what a revision should contain.

One consequence is worth being blunt about. Nothing in any of these documents tells you which releases are material. That judgement is yours to make, to write down, and to apply consistently — which is precisely why the written test below matters more than the individual decisions it produces.

Two words in the test are doing deliberate work. Plausibly sets the standard below certainty: you are not required to prove that behaviour moved before you look, only to judge whether it could have. And approved anchors the fourth axis to a specific prior decision — the use-case approval you actually granted, with its stated purpose and boundaries, rather than a general sense that the system is allowed.

You are not notified. Notification practices differ sharply between consumer products, standard business plans and enterprise agreements, and that difference is structural rather than incidental: enterprise arrangements are where notice periods, version-availability commitments and change communication tend to be contractible at all. Whether your particular agreement carries any of that is a question to answer from the contract in front of you, not from a vendor comparison page.

The version label is opaque. Model identifiers are product decisions. Some providers publish dated or otherwise immutable identifiers alongside floating aliases. Some expose a family name that resolves to whatever is current. Some return version metadata on the response and some do not. The governance question is not what any one provider does today — that changes — but which of these you are relying on, and whether you would be able to prove, six months later, which version served a given request. Verify in current provider documentation what pinning is available on the endpoint and plan you actually use, and re-verify at your supplier review cadence.

Behaviour moves without a version bump. This is the hard one. Safety and refusal policy, system prompts applied at the platform layer, serving-stack changes, routing between capacity pools, and inference optimisations can all shift observable behaviour while the identifier you queried stays exactly the same. No amount of contractual notice covers this reliably, because from the provider’s perspective nothing was released.

Three consumption patterns behave differently, and confusing them is why one governance policy rarely fits a whole estate. Direct API consumption — from a model provider, or through a cloud platform such as Azure OpenAI Service or Google Cloud Vertex AI — is where explicit model identifiers, version selection and deprecation schedules are most likely to be exposed to you, and therefore where the population question is most likely to be answerable later. Packaged assistants such as Microsoft 365 Copilot, Gemini for Workspace and ChatGPT Enterprise are governed at the tenant and admin-console level rather than at the model level. The model underneath is generally a vendor implementation detail you consume rather than choose, which shifts your controls towards permissions, connectors and data boundaries. Developer tools — GitHub Copilot, Cursor, Claude Code and the agent frameworks around them — often expose model selection to the individual engineer, which turns the governance question into who chose it, whether policy constrains that choice, and whether the choice is recorded anywhere. None of these characterisations is permanent. Verify the current position for your own plan tier and endpoint in the vendor’s documentation, because this is precisely the layer that changes.

Pin where pinning is offered. Consuming a dated or immutable identifier rather than a floating alias converts an undetectable event into a scheduled one: the version stops moving under you, and the change happens when you decide it does. It also makes the population question answerable months later, which is the single most common evidence request on this topic. Where a platform offers no pinning at all — common in packaged assistants and embedded product features — record that fact in the inventory, because it changes what assurance you can honestly give about that system.

Subscribe to change communications and route them somewhere accountable. Provider changelogs, deprecation pages, status feeds and enterprise notification channels are worth having, and worth being realistic about: they land in an inbox nobody owns unless you give them an owner, a triage cadence and a place to be recorded. The failure here is almost never that the notice did not arrive. It is that it arrived and nothing happened, which is the harder thing to explain afterwards.

Run your own regression tests on a schedule. This is the only detector you fully control, and the scheduling detail is what makes it a detector rather than a gate. A regression suite that runs only when a provider announces something can, by construction, never catch an unannounced change. The same suite running nightly or weekly against a stable golden set becomes a canary: when results move and nothing on your side changed, something on theirs did.

A fourth measure is worth adopting even though it detects nothing on its own. Store the model identifier, and whatever version metadata the response carries, alongside the output itself. It costs a column. It answers the population question permanently, it lets you correlate a quality complaint with a version boundary, and it is the difference between investigating an incident and speculating about one — which is also what makes an AI incident classifiable after the fact.

There is a SOC 2 corollary worth knowing if you are also under examination. The description criteria — DC section 200, which apply to SOC 2 and not to SOC 1 — call for the system description to include relevant details of changes to the system during the period. A model change that altered how the service operates is a candidate for that disclosure, and it is far easier to write the sentence from a change log than to reconstruct it in December. The related question of what happens when the system moves underneath a running observation window is covered in mid-period changes.

Treat a retirement notice as a business-continuity event with a named owner rather than a ticket in the engineering backlog. Under SOC 2 this reads naturally against CC9.1, which addresses risk mitigation activities for risks arising from potential business disruptions, and against CC9.2 for the vendor relationship behind it. Under ISO/IEC 42001 it belongs with A.10, third-party and customer relationships, and with the resourcing and life-cycle controls that assume the AI system remains operable. If Availability is in your SOC 2 scope, the ordinary availability criteria apply to the service the model sits inside, which is worth remembering before you promise a customer an uptime figure that depends on a component with a published end date.

Four practical measures, roughly in order of value. Keep a second qualified model that has passed the regression set at least once, so migration is a cutover rather than an evaluation project begun under a deadline. Write prompts to the task rather than to a model’s idiosyncrasies wherever you can, accepting that some tuning will always be model-specific. Set the cutover target well ahead of the retirement date, because the last fortnight before a forced migration is exactly where the regression gate gets waived. And check what your contract actually says about notice periods and version-availability windows, since that is a negotiable term at enterprise tiers and an unknown one everywhere else.

The failure mode this section exists to prevent is specific and common: an organisation with a mature change process runs it faithfully for a year, then skips it entirely on the forced migration, because the deadline is external and the alternative is an outage. That is the one change most likely to move all four axes at once, and it is the one most likely to go through ungated.

Tool changes are more consequential still, because they move the blast radius rather than the behaviour. Adding one connector can take an agent from reading tickets to writing to a production system, and the model did not change at all. Excessive Agency sits at LLM03 in the 2026 edition of the OWASP Top 10 for LLM Applications, up from LLM06 in 2025 — the largest upward move in the list, driven by exactly this class of production incident. Where tools are reached over the Model Context Protocol (MCP), the tool surface is a moving object by design and carries its own change semantics, which makes it a second detection problem rather than a variant of the first. The approval mechanics, and the specific case of a server that offers something it did not offer on the day you reviewed it, are covered in approving MCP tools and handling capability change.

Workflow changes are the quietest of the three. Removing a confirmation step, widening what an agent may do without asking, raising a spending threshold, or extending an agent from one department to five are all reclassification events: the system is no longer the one that was assessed. A change of this kind should re-enter approval at the use-case level rather than being handled as a configuration tweak, and it is the change most likely to be made under delivery pressure with no artefact at all. Where autonomy widens, the stop mechanisms and transaction limits have to widen with it or they stop bounding anything.

The practical instruction is short. Put all four change types — model, prompt, tool, workflow — in one change log with one classification test, and resist the instinct to treat the model as the interesting one. In our readiness work the change that turns out to have mattered is a prompt or a tool considerably more often than it is a model.

Two vocabulary notes make the requests easier to anticipate. In an attestation engagement, anything you export is information produced by the entity, and the practitioner has to establish that it is complete and accurate before relying on it — which is why a change-log export arrives with its source, its filter, its date range and a record count rather than as a tidy summary. And in an ISO/IEC 42001 assessment, an undetected or unlogged change is not a documentation gap: it is a candidate nonconformity against your own procedure, handled through the improvement clause with a corrective action and a stated root cause. Broadly, an unlogged change is worse than a logged change you classified as immaterial and were wrong about, because the second one shows a control operating imperfectly and the first shows no control at all.

Frameworks converge more than they diverge here. NIST’s AI Risk Management Framework, with its Generative AI Profile (NIST AI 600-1, July 2024), places post-deployment monitoring and the tracking of identified risks over time in its MEASURE and MANAGE functions. ISO/IEC 42001 puts the same expectation in A.6.2.6, operation and monitoring. SOC 2 reaches it through CC3.4 and CC7.1. Different vocabularies, one obligation: know when the thing you assessed stopped being the thing you are running.

A boundary note, since these engagements involve several parties. ISO/IEC 42001 certificates are issued by accredited certification bodies, and SOC 2 reports by licensed CPA firms. Tranquility Cybersecurity designs the change procedure, builds the regression and evidence structure, and coordinates the assessment or examination — it does not certify, attest, audit or sign. ISO 42001 pricing is all-inclusive and indicative: from $4,000 (₹3–3.5 Lakh) for small organisations, $5,000–6,000 (₹4.5–5 Lakh) for growing companies and $8,000 (₹7–10 Lakh) for large enterprises. It includes the accredited certification body’s audit fee. The certificate is issued by that certification body, not by Tranquility Cybersecurity, and the final figure depends on the certification body you choose.

Related reading: the AI agent governance framework, the AI system inventory, MCP tool approval and capability change, risk versus impact assessment, AI incident classification, SOC 2 mid-period changes, and the ISO 42001 hub.