Halo Auto Triage · Post-implementation results

OUR SHORTCUT TO SERVICE

How we bypassed our biggest bottleneck and moved from automation to measurable impact

MVP LIVE IN PROD 27 SERVICE CATEGORIES IN SCOPE DUO FOLLOW-UP AUTOMATED TICKET-LEVEL RESULTS VALIDATED

Ticket submitted / forwarded to askIT

Assigned to Auto Triage

Analyzed by Auto Triage

In scope and confidently classified?

YESAuto route

YES + DUO MISSING INFOAuto route + request info

NOManual triage

Service delivered by assigned team

02

Before Auto Triage / Manual Process

Service waited on triage

Before the right team could help, a Service Desk agent had to read, classify, and route the request

01Request emailed

The requester emailed, or was forwarded to, askIT@yorku.ca

02Ticket created

Halo recorded the new request

03Agent read

A triage agent identified the intent

04Agent routed

The agent selected the category and team

05Service began

The assigned team could then start resolving the request

Valuable time was spent routing instead of resolving

03

The problem we solved

Delayed service

Every minute spent moving a ticket was a minute before diagnosis, support, or resolution began

Peak times magnified all four risks

01

Queue delay

Requests waited for a routing decision before reaching the people who could act

02

Variation

Manual classification could vary from agent to agent.

03

Back-and-forth

Incomplete access requests created repeat contact loops and avoidable waiting

04

Capacity drain

Skilled agents spent scarce capacity moving routine work between queues

04

The new value delivered

Faster service today

Every supported ticket classified with high confidence bypasses manual triage and moves closer to service delivery

Faster routing · fewer follow-up loops · more capacity for service

01

Direct routing

Supported, high-confidence tickets move directly to the team that can act

02

Consistent decisions

A consistent classification approach can now be applied to every eligible request.

03

Automated follow-up

Incomplete DUO requests receive the approved request for missing information earlier in the process

04

Capacity returned

Confidently routed in-scope tickets no longer require manual Service Desk classification

05

Our new process

Automated efficiency

Every incoming askIT ticket is assessed. Only in-scope, high-confidence tickets are auto-routed

01Ticket created in Halo

02Assigned to Auto Triage

03Intent matched to catalogue

04Scope + confidence checked

Direct to resolvers

ClassifyApply the supported service category

AssignSelect the responsible processing team

DeliverThe team reviews, responds, and resolves

No Service Desk handoff required

06

New controls

Mitigating risk

Many of the core risks existed in both manual and automated triage. We added specific controls for the new risks introduced by automation.

Mitigation

Auto route only when confident

Launch scope stayed limited to the supported categories with the strongest training data
Receiving teams follow current Departmental Triage practices for misrouted tickets
Recurring patterns are reported and are being analyzed to improve the model

07

THE ROLLOUT PLAN

From pilot to production

We tested in QA and production, set readiness gates, and moved only when the evidence supported it

2026

Build, test, prepare

Completed technical validation, UAT, guidance, training, communications, and operational readiness assessments

Aug 14–28

Monitoring phase began

Analyzed live tickets and recorded decisions without updating, assigning, or routing them

Aug 30

Readiness gate

Reviewed accuracy, exceptions, operations and people readiness before active routing

Aug 31

Active routing began

Eligible, high-confidence tickets began moving automatically. All others followed manual triage

Now

Hypercare and refinement

Continue monitoring results, recovering exceptions and tuning based on evidence.

08

IT agent communication and resources

Prepared our Service Desk to...

01 · UNDERSTAND

Understand the process

Recognize when Otto Excellence is actively processing and avoid interrupting live triage

02 · WATCH

Watch for issues

Monitor the queue and action stalled tickets that need attention

03 · ROUTE

Route stalled tickets

Confirm the stall, apply the correct category and team, then route the ticket appropriately.

04 · IMPROVE

Turn patterns into improvement

Report recurring errors, unusual trends and repeated stalls to Operational Excellence.

09

Why during peak?

Highest volume = highest value

Started with a small scope

Limited automation to the 27 most-requested supported service categories

Required high confidence

Kept uncertain tickets with the Service Desk

Observed before acting

Used production monitoring to confirm readiness before go-live

Kept humans at the heart

Kept people responsible for exceptions, judgement and service quality

10

SCALING THE PEAK

How we did in September

4,413

Tickets processed

Model coverage

53.9%2,380 / 4,413

2,380

Classified with confidence

Model accuracy

86.3%2,053 / 2,380

1,255

DUO 2FA tickets

DUO responses sent

30.9%388 / 1,255

Reporting periodTickets received and resolved during September 2026.

How accuracy is measuredAgent-assigned categories are treated as ground truth, assuming the resolving agent selects the correct category before closing each ticket.

11

Our success

We moved the needle... RIGHT

Ticket-level results showed Duo requests moved through the process MUCH faster after automation

Mean

i

The mathematical average. It shows the overall process change, but can be pulled upward by unusually long-running tickets.

51.6% lower

Average total process time · all ticket durations averaged together

Before

54.1 h

After

26.2 h

Median

i

The middle ticket in the dataset. Half of tickets were faster and half were slower, so this is closer to the typical experience.

74.1% lower

Median total process time · the middle ticket in the dataset

Before

39.4 h

After

10.2 h

p < .001 · statistically significant

12

What it means for york

The story behind the data

29h

Faster for our community

Saving 29 hours from the median ticket means less time waiting for help and less frustration when DUO issues block our community's ability to work or study

↑

More capacity for real support work

When follow-up happens automatically, agents spend less time chasing the same missing details and more time handling exceptions and delivering higher-value service work

13

Applying what we've learned

Automation
VS
Ambiguity

The MVP worked best where service patterns were clear. Working through our ambiguities is our next step toward improving auto triage

01

Clearer patterns

Auto Triage is trained against existing human patterns. We expected, and proved, that it inherited some misrouting habits from our original data set. Giving our staff the tools and resources to perform more accurately today, will improve our future data for even more accurate performance

02

Stronger service boundaries

When multiple IT service desks offer similar support, classification and model training need more context supported by clearer service definitions

03

Factoring entity info

Our next step is in improving how we consider the requestor's entity information. Improving matching to the requestor's local service desk when appropriate

04

Scaling thoughtfully

The next step is tightening accuracy of the current in-scope categories and then slowly expanding out to our next batch of high priority services