Contents
I would start preparing a team to work with an AI agent by choosing a specific task: agreeing the boundaries, assessing the result and practising how to respond to an error. That exercise should take place before the agent starts acting independently in the company’s systems.
Imagine a ticket about a production stoppage that an AI agent assigns a low priority. It interprets the description as a local fault at one workstation. The proposal looks reasonable, enters the queue and waits. This hypothetical scenario illustrates what I would want to check before deployment: whether someone notices the error, knows the escalation rules and can intervene.
First, I would establish what employees will need to do differently. Then I would choose the training and a way to assess their skills.
What changes in the team’s work
In an IDC article published on 21 October 2026, Gina Smith describes the skills involved in working with agents: defining the task, providing instructions, supervising, verifying and governing the workflow. Writing prompts forms part of that work. IDC: AI Orchestration: Are Your Employees Ready?
A few days later, Mary K. Pratt presented IT leaders’ views on changing roles, training and the need to judge which ideas are worth pursuing. These are the interviewees’ perspectives, rather than evidence that every organisation faces the same problem. CIO.com, 26 October 2026
Training should prepare people for decisions they will actually make. Someone who understands the process may need to learn how to assess AI responses. An IT specialist may need a better understanding of the business consequences of misclassification. The manager needs to establish who takes over when automation fails.
This is how I would prepare an exercise with a team.
Step 1. Choose a task whose result you can assess
I would start with a specific part of a process. In our example, the agent proposes a ticket category, priority and draft response. During training, it does not close tickets or change system configurations.
With the process owner, I would document which information it should use and how we will recognise a correct result. We also need to agree what it should do when information is missing. A ticket saying “it does not work” may require a follow-up question instead of a confident diagnosis.
I would record those decisions in a short description of the task, permitted actions and conditions for handing over to a person. If we cannot agree what a correct result looks like, we first need to clarify the process rules.
Step 2. Assign responsibilities and arrange cover
The business owner defines what the priorities mean and approves the outcome requirements. IT is responsible for the agreed technical scope: access, integrations, monitoring and the ability to intervene. The supervisor assesses cases within their assigned role. The supplier fulfils its contractual commitments; support and escalation arrangements need to be agreed separately.
A name in the “supervision” field is not enough. That person needs time, cover and access to information. Someone approving a priority should be able to see the original ticket and the relevant rule, rather than only an AI-generated summary.
Out-of-hours responsibilities also need to be agreed. An agent may operate in the evening, after its supervisor has become unavailable.
After that discussion, it should be clear who assesses results, who can change the agent’s scope and who responds to an incident.
Step 3. Prepare a set of practice situations
I would collect cases from the chosen process, protecting the information appropriately. Synthetic data can also be used for exercises. The set should include routine cases and exceptions the team needs to handle.
For ticket handling, I suggest practising:
- a production stoppage described in imprecise language;
- missing information about the extent of a fault;
- a proposed response based on outdated instructions;
- rerunning a task that could create a duplicate record;
- ticket content instructing the AI to ignore its rules and disclose information.
The last case tests an attempt to control the model through the data it processes. OWASP describes this risk as prompt injection. Adding an instruction to a prompt does not provide complete protection. OWASP LLM01:2025
Each case needs an agreed expected result and someone authorised to assess it. I would reserve some examples for a subsequent assessment, so that readiness is not judged solely on cases already practised during training.
I would adapt the number and selection of exercises to the risks and variety of work. Handling one case correctly tells us little about situations we have not tested.
Step 4. Practise the human decision, as well as using the tool
In the workshop, the participant receives a ticket, the AI proposal and the information needed to assess it. I ask them to decide whether to accept, correct, seek clarification or escalate. They should then explain the basis for their decision and carry out the appropriate action in the training environment.
For the production stoppage, spotting an incorrect priority is only part of the task. I would check whether the employee understands the production impact and can escalate according to the company’s rules. Separately, I would record whether the obstacle was knowledge, an unclear procedure, missing access or the system’s behaviour.
I would not reward rejecting AI responses as such. The set should include correct proposals too. The team needs to distinguish between situations, rather than guess where the trainer has hidden an error.
After the workshop, we have specific observations about what people need to learn and what prevents them from doing the task correctly.
Step 5. Test the ability to intervene
Closing a chat window may not stop a background task. I would therefore test how to suspend new agent actions, what happens to operations already under way and how to establish which changes have actually been saved. The next part is returning to manual handling without performing the same actions twice.
Permissions should follow the task. Proposing a ticket category does not require permission to delete records. Restrictions need to be enforced in the tools and destination systems. OWASP identifies excessive functionality, permissions and autonomy as sources of excessive-agency risk. OWASP LLM06:2025
It is worth retaining the intervention test results. Logging scope, access to records and retention periods need to reflect the data processed in the workflow.
What if one agent checks another?
That arrangement can be useful. Anthropic describes a pattern in which one model call generates a response and another evaluates it against criteria and provides feedback. This describes an architecture, rather than guaranteeing correctness. Building effective agents, 19 December 2024
In our example, a second model could check whether the proposed priority follows the ticket-handling rules. I would test its effectiveness too. If both models receive an incomplete description of the production impact, their agreement alone cannot confirm that the decision is correct. That is a conclusion about this example.
Where a condition can be checked unambiguously, I would use an ordinary system rule. An assessment requiring interpretation needs agreed criteria and comparison with judgements from people who understand the process. In its agent-evaluation guidance, Anthropic notes that model-based assessments require calibration against human assessments. Demystifying evals for AI agents, 9 January 2026
This does not require manually approving every operation indefinitely. Autonomy should reflect the risk, test results and ability to reverse an action. The overall process still needs an owner.
Step 6. Run a limited trial and observe the work
In our example, I would begin with a trial that makes no automatic changes to tickets. The agent prepares proposals and an authorised employee makes the decisions. Waiting for that review must not delay urgent cases.
I would also observe the supervisor’s workload. If the volume of proposals exceeds their capacity, formal approval may cease to be a meaningful check. We then need to change the work arrangements or trial scope, instead of adding another reminder to be careful.
A record of decisions and corrections will show where the team needed help during that trial.
Step 7. Assess the whole process
For tickets, I would assess time to resolution, response quality and reopened cases. Working time should include verification and corrections. A particularly important measure is the share of genuinely urgent cases incorrectly classified as non-urgent: the denominator is all genuinely urgent cases in the assessed set.
I would assess people’s skills separately. Did they recognise material errors? Did they know how to respond? Did they have the conditions to do so? Good system results in a small trial do not assess every skill if a difficult situation simply did not occur.
Before testing, I would agree what results permit expansion. A change to the model, instructions or integration should trigger appropriate retesting. Continued oversight also follows NIST AI RMF 1.0’s approach of managing risk throughout the system lifecycle. NIST AI RMF Core, 2023
On that basis, the process owner can decide whether to expand the use case, change the solution or run further exercises.
I would start with a short exercise
Return to the production-stoppage ticket. I would want to see an employee who checks the proposal, understands the situation and sends the case to the right place. If they cannot, I want to identify the specific obstacle: missing knowledge, unclear responsibility or a system limitation. Each calls for a different response.
That is where I would begin preparing the team. It helps organise training around actual work and identify what needs improvement before increasing the agent’s autonomy.
I discuss the broader assessment of use cases and business outcomes in We bought AI. What now?. This guide addresses a narrower question: can people perform their part of an AI-supported process?
Work through team preparation
In the simulation below, you can test your response to an AI proposal. After the exercise, you can record decisions for your own team and download a worksheet. Answers stay in this browser tab’s memory until it is closed or reloaded. Download the file if you want to keep them. Do not enter personal data, passwords or confidential ticket information.
Completing fields does not confirm team readiness. That assessment needs exercises and evidence from the relevant environment.
Decision workshop · hypothetical scenarios
Take over agent supervision
The agent has prepared a proposal. Decide what to do with it. You can check the full information, ask for a second opinion and see the consequences of different actions.
Description given to the agent
AI proposal
Open the full ticket and process rule
Full ticket
Rule agreed for this exercise
What does “stop” mean?
Your exercise history
This is a programmed simulation, with no connection to an AI model or ticketing system. It illustrates possible consequences. It does not measure control effectiveness or employee readiness.
Record a plan for your team
Prepare your own plan
Enter general decisions without confidential data. Answers are not sent or saved automatically. Download the worksheet to keep them.
Step 1 of 8
Task
Describe one part of the work whose result can be checked.
Example and guidance
The agent proposes a ticket category and priority; an employee approves them.
Step 2 of 8
Responsibility
Assign roles and cover. Include out-of-hours work.
Example and guidance
The process owner defines priorities. IT provides the integration. The supervisor has cover.
Step 3 of 8
Exercises
Choose routine cases and exceptions. Agree the correct result before assessment.
Example and guidance
A production stoppage, missing information, outdated instructions and rerunning a task.
Step 4 of 8
Decision practice
Practise the human response. The examples below are hypothetical.
Example and guidance
Record both the participant’s decision and the obstacle revealed by the exercise.
Choose an exercise
Show a suggested response
Material for discussion with your team. This is not an automatically scored test.
Step 5 of 8
Intervention
Test the procedure in a test environment. Describing a button is not enough.
Example and guidance
Establish how to suspend new actions and check operations already under way.
Step 6 of 8
Limited trial
Agree the scope, period and availability of supervisors.
Example and guidance
The agent prepares proposals. Urgent cases do not wait for the experiment to finish.
Step 7 of 8
Assessment
Assess the process and skills separately. Completing fields does not confirm readiness.
Example and guidance
Compare handling time including review and corrections, quality, reopened cases and priority errors.
Step 8 of 8
Worksheet
Review your answers and download them before closing this page.
Example and guidance
The worksheet records decisions. It is not an audit, certification or automated readiness assessment.
The file contains your answers. Store and share it according to your organisation’s rules.
Sources and scope
Author: Rafał Mokrowiecki. Sources accessed on 3 November 2026. This is a proposed approach for IT leaders and process owners. The examples are hypothetical; the article does not present client deployment results, the author’s own experiment or a legal compliance audit.
Sources are linked at the relevant passages. For IDC, I used the public article, without claiming to have verified the full paid studies. For CIO.com, I used the reported views, without generalising a conference poll to all companies. Anthropic’s articles describe supplier practices, rather than an independent tool comparison. The 2024 article explains an evaluation pattern, rather than a current product catalogue. NIST AI RMF 1.0 is the 2023 framework; its site notes that a revision is under way. I do not describe that revision as a completed new edition.
