A model without a decision
The project optimises a technical score without explaining who uses the output, what action follows, or how mistakes affect people and operations.
04 · AI capability
Machine Learning & Data Intelligence
Develop analytical and predictive capabilities from governed data, explicit questions, representative evaluation, and an owned operating workflow.
The context
Begin with the work, evidence, and consequence.
Machine learning becomes useful when a statistical result changes a real decision or workflow. The work begins by defining that decision, the available evidence, acceptable errors, and how model output will be interpreted, challenged, and maintained over time.
Historical data is treated as neutral
Missingness, changing definitions, selection effects, leakage, and past operating choices are not examined before training and evaluation.
The model stops at a notebook
There is no dependable feature flow, application integration, monitoring, feedback, retraining decision, or accountable production owner.
Areas of attention
Connect AI behaviour to a dependable product and operation.
The appropriate techniques, controls, and delivery depth follow the use case. These areas keep the technical work connected to the people and decisions around it.
01
Decision and metric framing
Define the intended use, unit of prediction, decision timing, baseline, error costs, segments, and evaluation criteria.
02
Data assessment and preparation
Review provenance, access, quality, definitions, coverage, leakage, representativeness, and reproducible transformation.
03
Model development and validation
Compare an appropriate baseline and candidate approaches using held-out, segmented, and scenario-based evaluation.
04
Production and model operations
Integrate inference, versioning, monitoring, feedback, access, documentation, and review into the wider product workflow.
Potential outputs
Create evidence and artefacts the team can use.
Outputs are shaped around the decision and engagement stage. Each should have a clear audience, purpose, owner, review criteria, and stated limitation.
- 01Decision, data, and evaluation definition
- 02Data-quality and suitability findings
- 03Reproducible baseline and candidate model analysis
- 04Validated model or bounded data-intelligence product increment
- 05Deployment, monitoring, review, and model-change guidance
Evaluation and responsibility
Make inputs, useful evidence, and boundaries explicit.
AI quality cannot be separated from its context of use. Representative inputs, subject expertise, realistic scenarios, and accountable owners are part of the engineering work.
What we need to understand
- The decision, workflow, audience, and action the output should support
- Historical or representative data with definitions and permitted access
- Baseline behaviour, error costs, segments, timing, and operational constraints
- Domain experts who can review assumptions, labels, scenarios, and unexpected results
Evidence of a useful direction
- The model is compared with a meaningful non-model or existing baseline
- Evaluation represents relevant time periods, groups, scenarios, and error types
- Data provenance, transformations, assumptions, and limitations are documented
- The live workflow has monitoring, feedback, review, and an accountable owner
Important boundary
Historical performance does not guarantee future performance or causal impact. Data changes, operating behaviour, feedback loops, and shifts in the environment can alter model usefulness after release.
Working path
Reduce uncertainty before expanding commitment.
The exact sequence depends on the use case and current evidence. Each stage should leave a reviewable result and an informed choice about the next step.
Define
Connect the analytical question to a user, decision, action, timing, baseline, and cost of error.
Assess
Examine data provenance, permissions, coverage, quality, bias, leakage, and suitability.
Develop and validate
Build reproducible baselines and candidate approaches, then evaluate representative outcomes and limitations.
Integrate and observe
Embed the chosen capability in the workflow with versioning, monitoring, feedback, and review ownership.
AI capability questions
Important details to clarify.
Early questions should expose the task, evidence, uncertainty, and responsibilities that shape a safe and useful implementation.
How much data is needed?+
There is no universal amount. Suitability depends on the question, signal, variability, labels, coverage, quality, expected error, model approach, and how the output will be used.
Do we always need a custom model?+
No. Rules, analytics, an existing service, or a pre-trained model may be sufficient. Custom modelling should be justified by the task, evidence, constraints, differentiation, and ownership cost.
How do you evaluate a model fairly?+
Evaluation should include relevant segments, time periods, edge cases, error types, baselines, and operational consequences. Fairness questions depend on the people affected and the decision context.
What is model drift?+
It describes changes in inputs, relationships, or outcomes that can make previous evaluation less representative. Monitoring should connect detected change to a defined review response.
Start a conversation
Let's discuss machine learning and data intelligence.
Share the workflow, evidence, people affected, and the outcome you want. We can help frame a responsible first question and bounded next step.