azyware
Technology

How to measure whether application maintenance and support services is working

EZ
Eazyware
· 7 min read
Quick answer

How do you measure application maintenance and support services?

Measure application maintenance and support services on four layers: availability against an agreed target, response and restoration against the SLA clock, change safety shown by failed releases, and debt reduction shown by patch currency and reopen rate. Ticket volume is activity, not outcome.

Measure application maintenance and support services on four layers: availability against an agreed target, response and resolution against the SLA clock, change safety shown by failed releases and rollbacks, and debt reduction shown by patch currency and reopen rate. Ticket counts and hours consumed describe activity, not whether the service is working.

What follows is the scoreboard we put in front of clients every month, the metrics we refuse to report because they reward the wrong behaviour, the instrumentation you need before any of it is meaningful, and the point at which measuring more stops helping.

Why ticket counts are the wrong headline

Almost every support report opens with tickets raised and tickets closed. Both numbers can move in either direction for good or bad reasons. Tickets fall because the system got better, or because users gave up reporting. Tickets close because defects were fixed, or because someone bulk-closed a stale queue on the last day of the month.

A support scoreboard has to answer four questions that a business actually cares about. Is the system up when people need it? When something breaks, how fast does a human engage and how fast is service restored? Are our own changes making things worse? And is the underlying debt shrinking or growing?

The reason the layers matter is that a single composite score hides trade-offs. A team can hold availability at four nines by refusing to deploy, and a team can ship daily while the patch backlog grows for a year. Kept separate, the four layers argue with each other in public, which is exactly what you want in a review.

Those four questions map to four measurement layers, and each layer needs its own owner. Availability belongs to whoever runs the infrastructure. Response belongs to the support contract. Change safety belongs to whoever ships. Debt belongs to the engineering lead, and it is the layer that quietly decides how expensive the next two years are.

Which application maintenance and support services metrics actually matter?

Nine metrics carry almost all the signal. Each row below states what the number proves and how a vendor or an internal team can make it look good without doing the work, because every metric is gamed eventually.

MetricLayerWhat it provesHow it gets gamed
Availability against targetAvailabilityThe system was usable during the hours that matterMeasuring the load balancer rather than a real user journey
Time to first human responseResponseSomeone engaged inside the contracted clockAuto-acknowledgement bots that reply in nine seconds
Time to service restoredResponseUsers could work again, even on a workaroundClosing the incident at workaround and opening a quiet defect
Reopen rateResponseFixes actually heldRaising a new ticket instead of reopening the old one
Change failure rateChange safetyReleases are safe to deployShipping less, which hides the problem rather than solving it
Rollback frequency and durationChange safetyYou can undo a bad release quicklyFixing forward under pressure and never recording the rollback
Patch currencyDebtDependencies and platforms are supported versionsCounting patches applied rather than versions behind
Open critical vulnerabilities, agedDebtSecurity findings are being closed, not cataloguedRe-classifying severity downwards at triage
Enhancement hours deliveredThroughputThe hours you pay for turned into visible changeLogging incident work as enhancement to fill the allowance

The metrics that mislead

Average resolution time is the worst offender. One ten-day defect buried in three hundred five-minute password resets disappears into the mean. Report percentiles instead: the ninetieth and ninety-fifth are where your users live. Even better, report the three longest incidents by name each month.

Customer satisfaction scores on support tickets measure politeness more than engineering. They are worth collecting, but they will not tell you that a scheduled job has been silently failing since March.

Hours consumed is a budget control, not a quality signal. A month in which the vendor used four of ten hours because nothing broke is a good month, and a scoreboard that treats unused hours as waste will produce busywork by the second quarter. The distinction between the promises worth measuring and the ones vendors pad is covered in SLAs that mean something.

What has to exist before any of this is measurable

A synthetic journey, not a ping

Availability measured by pinging a health endpoint tells you the process is alive. Measure instead a scripted journey that logs in, loads the screen people actually use and reads one record, run every minute from outside your network. That is the number you should put in a contract.

A severity scale everyone agrees on in advance

Severity has to be defined by business impact and written down before the first incident, otherwise every triage becomes a negotiation. The difference between engaging and fixing is defined in the glossary entry on response versus resolution, and both clocks need a start event that a system records rather than a person remembers.

Deployment and incident records that join up

Change failure rate requires you to know which deploys preceded which incidents. That means every release is tagged, timestamped and attributable. Without it, change safety is an opinion.

A dependency inventory that updates itself

Patch currency means measuring how many major versions behind you are, per component, every week. A generated inventory makes the number honest; a spreadsheet makes it aspirational. Cadence, windows and rollback expectations are set out in our note on security patching cadence.

How to run the monthly review

Thirty minutes, the same agenda every time, with the same six artefacts on screen. Anything that does not fit this list belongs in a quarterly instead.

  • The four headline numbers, each against its target and against last month, with no commentary yet
  • The three longest incidents, named, with a one-paragraph cause and what changed as a result
  • Every failed release or rollback, with the gap in testing that let it through
  • Patch currency and open vulnerabilities, as versions behind and days aged, not as a count of work done
  • Hours used against hours available, split between incident work and enhancement work
  • One item of debt retired, because a support relationship that never reduces debt is only renting you time

Run this with the same two people on each side. Reviews that rotate attendees lose the thread and turn into status theatre.

What does the measurement itself cost?

Monitoring, synthetic journeys, deployment tagging and a dependency inventory are a two to three week piece of engineering on a typical mid-size application, and they are part of the onboarding audit rather than an extra. Reporting against them is included in every Eazyware Software Maintenance and Support tier: Essential at $1,000 or ₹68,000 per month, Standard at $2,500 or ₹1,60,000 and Enterprise at $5,250 or ₹3,40,000 with 24x7 cover, a one hour response and a named engineer. Systems with AI components add $750 or ₹40,000 per month, which buys evaluation runs, prompt regression checks, cost monitoring and re-indexing, because a retrieval system degrades in ways that uptime never shows. The tiers are listed on the pricing page.

If a vendor treats monitoring as a chargeable extra, read that as a statement about how they intend to be measured.

Budget a little internal time as well. Someone on your side has to define severity in business terms, confirm which user journey represents the system, and own the target for availability. That is a few hours in the first month and about an hour a month afterwards, and no vendor can do it for you because only you know what an outage costs on the last working day of a quarter.

When more measurement is the wrong move

Two cases. The first is a small internal application with a handful of users and no revenue exposure. Building synthetic journeys and change failure tracking for a tool that ten people use twice a week is engineering effort that would be better spent retiring the tool.

The second is a system where everyone already knows what is wrong. If the nightly reconciliation job has failed most weeks for a year, a dashboard confirming it is procrastination dressed as rigour. Fix the job. Measurement earns its place when the argument is about what is true, not about who will do the work.

There is also a limit on user-facing metrics. Adoption and satisfaction matter for new features, and our note on measuring adoption after go-live covers them properly, but they are the wrong lens for judging whether a maintenance contract is holding up a stable system.

A worked example

A last-mile logistics operator we work with runs a dispatch platform and an offline-first driver app. Their first scoreboard was ticket volume, which looked healthy while drivers quietly restarted the app to force a sync. We replaced it with four numbers: a synthetic booking journey, ninety-fifth percentile time to restore, change failure rate per release, and crash-free sessions on the driver app, a metric explained in our piece on crash-free sessions. The platform is described in the dispatch platform case study. Within two quarters the arguments in the monthly review moved from whether there was a problem to which fix came first.

Our guide to what belongs in an application maintenance contract sets out the clauses these metrics enforce, and the reopen rate glossary entry explains why a fix that does not hold is worse than no fix. Google's SRE guidance on service level objectives makes the case for choosing a small number of indicators that reflect user experience rather than instrumenting everything, which is the discipline most support scoreboards lack.

Pick four numbers your users would recognise as describing their week, publish them every month, and resist the temptation to add a fifth.

Frequently asked questions

What are the most important KPIs for application maintenance and support?

▾

Availability measured on a real user journey, ninety-fifth percentile time to restore service, change failure rate per release, and patch currency expressed as versions behind. Reopen rate is a strong fifth. Ticket volume, average resolution time and hours consumed describe activity rather than whether the service is working.

How often should you review a support vendor's performance?

▾

Monthly, in a thirty-minute review with a fixed agenda: the four headline numbers against target, the three longest incidents by name, every failed release, patch currency and aged vulnerabilities, hours used against hours available, and one piece of technical debt retired. Quarterly reviews arrive too late to change anything.

Is response time or resolution time the better SLA metric?

▾

Response time is the promise a vendor can genuinely keep, because it measures engagement rather than the size of an unknown defect. Resolution should be tracked as time to service restored, reported at the ninetieth and ninety-fifth percentile, and paired with reopen rate so workarounds recorded as fixes become visible.