How to Monitor a Production App

The goal is not a wall of dashboards. It is knowing something is broken before a customer tells you, and knowing where to look.

How to Monitor a Production App — Troiana insight cover

In short

Effective monitoring answers two questions: is the system working for users, and if not, where is it failing. For most teams that means four signals — error rate, latency, traffic and saturation — plus real-user performance and uptime from outside your own infrastructure. Alert only on things a human must act on now; everything else belongs on a dashboard you check deliberately.

The two questions

Everything worth monitoring answers one of these.

Is it working for users? Are requests succeeding, fast enough, from where people actually are?

If not, where is it failing? Which service, which query, which dependency.

Monitoring that answers neither — CPU graphs nobody reads, dashboards nobody opens — is cost without benefit. Start from the questions and work backwards.

The four signals

A long-standing and still-correct default:

Errors. The rate of failing requests, ideally split by endpoint. The single most useful number you can have.

Latency. How long requests take — measured at percentiles, never as an average. An average hides the tail entirely; p95 and p99 are where the unhappy users live. A 200ms average with a 4s p99 means one request in a hundred is unusable.

Traffic. Request volume. Mostly useful as context: errors doubling matters differently if traffic tripled.

Saturation. How close a resource is to its limit — connection pool, memory, disk, queue depth. This is what predicts a failure rather than reporting one.

Add two more for anything user-facing: uptime checked from outside your infrastructure, because a monitor inside the network cannot tell you the network is unreachable; and real-user performance, because synthetic tests from a fast data centre do not reflect a phone on mobile data.

Metrics, logs, traces

Three different tools, routinely confused.

Metrics are numbers over time — cheap to store, fast to query, good for alerting and trends. They tell you that something changed.

Logs are discrete events with detail. Expensive at volume, essential for understanding a specific failure. They tell you what happened in one case.

Traces follow a single request across services, showing where the time went. Invaluable once you have more than one service, unnecessary before that.

A small team on one application needs metrics and logs. Add tracing when "which service is slow" stops being obvious.

Alerts people do not ignore

Alert fatigue is the real failure mode. A team that gets twenty alerts a week stops reading them, and the twenty-first is the outage.

Alert on symptoms, not causes. "Error rate above 2% for five minutes" is worth waking someone. "CPU above 80%" is not — high CPU with healthy latency and errors is a machine doing its job.

Every alert must be actionable. If the response is "watch it", it is not an alert; it belongs on a dashboard.

Require duration. Almost nothing warrants alerting on a single data point. Sustained over several minutes removes most false positives.

Distinguish urgent from important. Page someone for user-facing breakage. Everything else can be a ticket in the morning.

Delete alerts that never fire usefully. An alert firing weekly and being dismissed each time is training your team to ignore alerts.

A good test: if an alert fired at 3am, would you want to be woken? If not, it should not page.

Errors specifically

Aggregate them. A thousand instances of one bug is one problem. Error tracking that groups by fingerprint turns noise into a prioritised list.

Attach context — user, request, release, input. An error without context is a puzzle; with context it is usually a five-minute fix.

Track the release. Errors appearing right after a deploy have an obvious first suspect. Tagging errors with the version makes that immediate.

Watch new errors specifically. A brand-new error type is more urgent than a familiar one at normal volume.

Logging without regret

Never log secrets or personal data. Passwords, tokens, card numbers, health information. Logs are copied, shipped to third parties, and retained — everything in them becomes a liability.

Log structured, not prose. Machine-readable fields can be queried; sentences cannot.

Include a request identifier on every line and pass it between services, so you can reconstruct one journey.

Set retention deliberately. Logs are usually the largest and least examined data you store.

Before you buy anything

A surprising amount comes free.

Your host already reports uptime, error rates and response times. Search Console reports Core Web Vitals from real visitors. Your database reports slow queries. Most frameworks emit request timing with one line of configuration.

Set those up, watch for a fortnight, and you will know which gaps a paid tool needs to fill. Buying first tends to produce an expensive dashboard nobody reads.

A reasonable starting point

Uptime monitoring from outside. Error tracking with grouping and release tagging. Request latency at p95 and p99 by endpoint. Real-user performance data. Structured logs with request identifiers and a retention policy. Three or four alerts, each of which would genuinely justify waking someone.

That is a day of setup and covers the large majority of what a small team needs to know.

If you want a second opinion on whether your monitoring would actually catch an outage, book a call.

Common questions

What should a small team monitor?

Four signals — error rate, latency at p95 and p99, traffic, and saturation of limited resources — plus uptime checked from outside your own infrastructure and real-user performance data. That covers whether the system is working for users and gives you somewhere to look when it is not.

What is the difference between metrics, logs and traces?

Metrics are numbers over time — cheap, fast to query, good for alerting and trends. Logs are discrete events with detail, expensive at volume but essential for understanding a specific failure. Traces follow one request across services. A small team on one application needs metrics and logs; add tracing when 'which service is slow' stops being obvious.

Why should you alert on symptoms rather than causes?

Because causes are often benign. High CPU alongside healthy latency and error rates is a machine doing its job, and alerting on it trains people to dismiss alerts. Alert on what users experience — error rate above a threshold, sustained latency — and use dashboards for the underlying resource numbers.

Why should latency be measured at percentiles rather than averages?

An average hides the tail completely. A 200ms average with a 4-second p99 means one request in a hundred is effectively unusable, and those are the users who complain and churn. Track p95 and p99 by endpoint so you can see the experience of the worst-served requests.

What should never be written to logs?

Secrets and personal data — passwords, tokens, card numbers, health information. Logs get copied, shipped to third-party services, and retained for long periods, so anything in them becomes a security and compliance liability. Log structured fields rather than prose, and set retention deliberately.

Have something worth building right?