Problem Management: Killing Recurring Incidents at the Root

The difference between restoring service and fixing the cause

Every IT team has an incident process: something breaks, someone gets it working again, the ticket closes. Far fewer have a problem process, and the gap shows up as the same incidents recurring month after month. The printer fault resolved two hundred times a year. The overnight batch job that fails every few weeks and gets restarted. The application that "just needs a reboot" on a schedule nobody questions anymore.

Incident management restores service for one user, now. It is reactive by design and it is essential. Problem management removes the underlying cause so the incident never happens again. The two are complementary, but they are not the same work, and the most common operational failure is doing only the first. A team that treats every recurrence as a fresh incident is running on a treadmill — always busy, never getting ahead. Problem management is how you step off it.

Separate the two queues, and set a trigger

Problem management fails when it has no dedicated place to live. If root-cause work competes with live incidents for the same attention, live incidents always win, because they are urgent and problems are merely important. The fix is structural: problems get their own record type, their own owner, and their own queue, distinct from the incident stream.

Then you need a trigger that decides when an incident becomes a problem. Reacting to gut feel means the loudest incidents get investigated while the quietly expensive recurring ones never do. Useful triggers:

• Recurrence threshold — the same issue crosses a defined count (say, the third occurrence in 30 days) and a problem record opens automatically.
• High-impact single incidents — any major outage gets a problem record for root-cause analysis regardless of whether it has recurred, because you do not want the second occurrence to be the trigger.
• Cost to serve — a category that consumes disproportionate labor even in small increments earns investigation.

Automating the recurrence trigger is what turns problem management from good intentions into a running practice. The system, not someone's memory, decides what deserves a root-cause look.

Root-cause analysis techniques that actually find the cause

The core of problem management is disciplined root-cause analysis. The goal is to get past the symptom and the convenient explanation to the condition that, if removed, prevents recurrence. A few proven techniques, matched to the situation:

• The Five Whys — repeatedly ask why until you reach a cause you can act on. Fast and effective for linear, single-thread failures. "The service crashed — why? Out of memory — why? A leak in the new build — why? An unclosed connection — why? No pooling limit — why? The default config shipped unbounded." Now you have something to fix.
• Ishikawa (fishbone) diagrams — map candidate causes across categories such as people, process, technology, and environment. Best when a problem could stem from several contributing factors and you need to structure the search.
• Kepner-Tregoe problem analysis — a rigorous method that contrasts what is affected against what is not to isolate the true cause. Worth the overhead for high-stakes or genuinely puzzling problems where guessing is expensive.

The technique matters less than the discipline. Two rules keep analysis honest: distinguish root cause from contributing cause, and never stop at a cause you cannot actually change. "Human error" is almost never a root cause — the real cause is the process or system that allowed a normal human mistake to reach production.

Workarounds and the known-error database

Root-cause fixes take time. Meanwhile the incident keeps happening, and users keep suffering. The bridge is a workaround: a documented, reliable way to restore service quickly while the permanent fix is in progress. A good workaround turns a painful firefight into a routine, two-minute action for tier-one.

Workarounds and confirmed root causes live in a known-error database (KEDB) — a record of problems whose cause is understood, paired with the current workaround and the status of the permanent fix. The KEDB is one of the highest- value artifacts in all of IT operations because it compounds:

• Tier-one resolves faster — a matching known error means the agent applies a proven workaround instead of re-diagnosing from scratch.
• Repeat diagnosis disappears — the organization stops paying to solve the same puzzle twice.
• Fix priority becomes visible — the KEDB shows which unresolved problems are generating the most pain, so permanent fixes get sequenced by impact.

Feed the KEDB into your help desk and IT support knowledge base so front-line agents and self-service both draw on the same confirmed answers.

Reactive versus proactive problem management

Everything so far is reactive problem management: incidents happen, you find and remove their causes. The mature practice adds a proactive dimension — finding problems before they cause a major incident at all.

Proactive problem management mines the data you already have:

• Trend analysis across incidents to spot a category quietly growing before it becomes a crisis — the disk filling a few percent a week, the error rate ticking up.
• Monitoring and alert patterns from your infrastructure that reveal degrading components — rising latency, climbing retry counts, memory that never quite returns to baseline.
• Weak-signal review of near-misses and minor incidents that share a cause, treating them as free warnings rather than noise.

Continuous infrastructure monitoring is what makes proactive problem management possible: it surfaces the slow degradations that never trip a hard alarm until they cause an outage. The payoff is the highest form of operational maturity — fixing the thing that was going to page you at 3 a.m. next month, during business hours this week.

Measure whether problems actually die

Problem management is easy to fake — you can open records, hold meetings, and still fix nothing. The metrics keep it honest by measuring elimination, not activity:

• Repeat-incident rate — the share of incidents that are recurrences of a known issue. This is the primary scorecard. If it is falling, problem management is working; if it is flat, you are documenting problems, not solving them.
• Problems resolved versus opened — whether the backlog of unfixed root causes is shrinking or growing.
• Time-to-permanent-fix — how long problems sit at "workaround in place" before the real fix lands. Workarounds are meant to be temporary; a KEDB full of permanent workarounds is a warning sign.
• Incident volume by category — the ultimate proof. A category that shrinks and stays shrunk after a problem fix is a recurring cost you removed for good.

The bottom line

The teams that escape permanent firefighting are the ones that stop treating every recurrence as a new emergency and start hunting the causes behind them. Give problems their own queue and owner, trigger investigation on recurrence and impact, run real root-cause analysis, capture workarounds in a known-error database, and add proactive trend hunting once the reactive engine runs. Then watch repeat-incident rate fall.

intSignal builds problem management into managed IT support so recurring tickets get retired at the root instead of resolved on repeat. If the same incidents keep coming back, talk to our team about eliminating them for good.