AI Decision Engine Readiness Framework

The decision being automated must be clearly described before any build work begins. Vague decisions — "help sales prioritise better" or "automate compliance review" — are not decisions. They are problem statements that still need to be translated into a specific, recurring judgment with defined inputs, explicit criteria, and a measurable outcome.

Score 2 if all four conditions are true:

• The decision type is recurring — the same judgment is made repeatedly across a defined workflow.
• The criteria are explicit — you can describe the logic as "if X and Y then Z" without relying on unstated institutional knowledge.
• A named KPI owner exists — one person is accountable for the business metric this decision affects post-launch.
• Success is measurable — there is a specific operational metric (conversion rate, resolution time, escalation rate, error rate) that will move if the decision engine performs correctly.

Score 1 if:

• The decision is described but the criteria have not been made fully explicit, or the KPI exists but has no named owner.

Score 0 if:

• The decision type is still described as a problem area rather than a specific recurring judgment, or no one can articulate what operational metric will improve.

The decision engine needs a defined set of data fields to reason about each decision. Those fields must be present, consistent, and current when the decision run is triggered — not available in theory from a data warehouse query, but retrievable in practice from the operational system at the moment the trigger fires.

Score 2 if all four conditions are true:

• The required data fields are defined — you have a canonical entity schema listing every field the engine needs.
• Population rate is sufficient — for each required field, the actual population rate in the operational system meets a defined minimum threshold (typically 85%+ for core fields).
• Consistency is acceptable — the same field means the same thing across different teams, regions, and entry points that feed the system.
• A fallback is defined — there is a specified behaviour for every field that may be missing at inference time (use default, skip, escalate, or hold).

Score 1 if:

• Core fields are available but some secondary fields have inconsistent population, or the fallback for missing fields has not been defined.

Score 0 if:

• The data required for the decision has not been audited, or known quality issues exist in core fields with no remediation plan.

An AI decision engine reasons against company-specific knowledge — policy documents, eligibility rules, product guidelines, regulatory constraints, approval authority matrices. That knowledge must be available in a form the engine can retrieve at inference time. Documents that exist on a SharePoint drive but have not been indexed, validated, and structured for retrieval are not usable knowledge.

Score 2 if all four conditions are true:

• The knowledge sources required for the decision are identified — you have a list of every policy document, rule set, or guideline the engine needs to reason correctly.
• Sources are current — each source has a confirmed last-updated date and an owner responsible for keeping it accurate.
• Sources are structured for retrieval — documents have been cleaned, chunked, and are ready for indexing into a vector store, or this work is scoped with a defined completion date.
• An update process is defined — there is a mechanism for propagating policy changes into the knowledge base when they occur, not on an ad-hoc basis.

Score 1 if:

• Sources are identified and reasonably current but have not been structured for retrieval, or no update process exists yet.

Score 0 if:

• The relevant knowledge sources have not been identified, or known sources are significantly out of date with no owner.

A production AI decision engine is not a model plus an API. It is a system with controls: confidence gating, external policy checks, structured output, escalation routing, and audit logging. These cannot be added as an afterthought once the model layer is built. They must be part of the architecture specification from the start, because the model layer is designed around them.

Score 2 if all five conditions are true:

• A confidence threshold mechanism is defined — there is a specified threshold per decision type that gates automated action versus human escalation.
• Policy checks are externalised — the compliance rules that must be enforced regardless of model output are identified and will be implemented as hardcoded checks outside the model.
• The output schema is defined — the structure of every decision run output (decision type, confidence score, rationale, policy checks, action triggered) is specified before build begins.
• An escalation path is designed — low-confidence and policy-blocked decisions have a defined route to a human reviewer, with specified context delivered to the reviewer.
• An audit log schema is defined — the fields that every decision run record must contain are specified and will be implemented from day one.

Score 1 if:

• Some control layers are defined but others (typically audit log or escalation path) are deferred to a later phase.

Score 0 if:

• The architecture plan does not yet include a confidence evaluation layer, policy checks, or audit logging — the current plan is model input/output without production controls.

A decision engine that delivers output to a human who then manually acts on it is not a decision engine — it is a recommendation tool. For the system to deliver operational leverage, the decision output must trigger a workflow action in an operational system automatically. This requires API access, schema mapping, write-back logic, and error handling. None of these are trivial, and all of them must be scoped before build begins.

Score 2 if all four conditions are true:

• The receiving system is identified — the specific operational system (CRM, case management tool, ERP, task queue) that will receive the decision output is named.
• API or integration access is confirmed — the technical mechanism for the write-back is confirmed with IT or the system owner, not assumed.
• The triggered action is defined — the specific workflow state change or task that the decision output creates is described (e.g., "CRM record status updated to Qualified, task assigned to rep X, priority set to High").
• Error handling is scoped — the behaviour when the write-back fails is defined (retry logic, fallback notification, decision record preserved).

Score 1 if:

• The receiving system is identified and access is likely, but integration has not been confirmed with the system owner, or error handling has not been scoped.

Score 0 if:

• The integration path has not been defined — the plan is to deliver output to a dashboard or email and have humans act on it, with no live workflow write-back.

Most AI projects are scoped to launch. Governance — the ongoing monitoring, escalation review, knowledge maintenance, and performance accountability that keeps the system performing after launch — is not scoped or funded. This is the single most common reason a system that works at launch degrades within six months. Governance must be defined before go-live, not after the first quality problem is discovered.

Score 2 if all four conditions are true:

• Business outcome monitoring is defined — the KPI metric from Gate 1 has a review cadence and a named reviewer who will assess whether it is moving in the right direction.
• Output quality monitoring is defined — decision type distribution and escalation rate are tracked and have alert thresholds that trigger investigation.
• Knowledge update ownership is assigned — every knowledge source in Gate 3 has a named owner with a defined update schedule and a process for propagating changes into the system.
• Escalation review is operational — the process for a human reviewer to receive an escalated decision, assess it, take action, and provide structured feedback to the system is defined and assigned.

Score 1 if:

• Monitoring intent exists but specific metrics, cadences, or owners are not yet assigned — it is on the plan but not confirmed.

Score 0 if:

• Post-launch governance has not been scoped. The project plan ends at go-live with no monitoring, escalation review process, or ownership defined for ongoing performance.