Proactive Maintenance: a Senior Platform Engineer in a Daily Cron Job

A good senior platform engineer starts the day the same way: pull up the dashboards, read the overnight signals across every system they look after, and form a short, opinionated view of what needs attention now, what should be scheduled, and what is just noise. The skill is not reading any single chart. It is doing that across a wide estate, every day, without missing the thing that matters and without raising alarms that do not.

That is exactly the kind of work that is too valuable to skip and too repetitive to do by hand at scale. So we automated the analysis, not the judgement. This issue is the full version of the maintenance preview from our Q2 2026 report: the pipeline that does it, why it stays cheap and safe, and where it still needs a person.

To be blunt about the framing: this is not a product, and it is not generic advisor-style insight of the ‘have you considered enabling autoscaling’ variety. It is opinionated analysis grounded in best practice and in what we already know good looks like for the systems we run. The value is in the opinion, and the opinion comes from running these systems, not from a model reading a metric in isolation.

What runs every morning?

The pipeline has four stages. Each one is reproducible by another team; none of it depends on anything you could not build on your own stack.

Stage 1: collect the telemetry

A scheduled workflow runs early each morning, around 06:00, and pulls monitoring data for every client into our Azure data lake. The point is breadth: more than anyone would sit and read by hand, every day, across every system.

The sources include:

• Security scanning and certificate status.
• Uptime and response-time monitoring, the external view of availability.
• Azure Application Insights: exceptions, request volumes, availability, failures, slow operations, and data ingestion as a cost signal.
• Azure SQL: processor, worker, and storage trends against the provisioned tier.
• Error tracking from the application layer.
• Kubernetes, via kubectl, Helm, and in-cluster Prometheus: NotReady nodes, crash-looping or high-restart pods, warning events, failed Helm releases, autoscalers stuck at their ceiling, persistent volume claim (PVC) capacity, out-of-memory (OOM) kills, and per-workload right-sizing against vertical pod autoscaler (VPA) targets.

Collecting it is the easy half. The hard half is turning that volume into a few things worth a human’s time.

Stage 2: have the model write the brief

A condensing step takes the latest snapshot, reduces it to compact JSON, and calls the Anthropic API with structured tool use. We use Claude Haiku 4.5 (claude-haiku-4-5-20251001), prompted to behave as a senior cloud platform engineer. The output schema is strict:

• Three urgent items, each needing action within a fortnight.
• Three planned items, each to be scheduled within the quarter.
• Up to four Kubernetes observations, graded attention, watch, or healthy.
• Short narrative decks for the Uptime and Performance sections of the report.

The system prompt is where the quality lives. Every statement must cite a specific number and name the affected service, and any sizing recommendation must be based on the cluster’s own VPA target rather than a generic rule of thumb. It carries domain guardrails so the model does not cry wolf: known SSL false positives are ignored, a ‘Pending’ ticket in the service desk is not treated as an outage, and specific HTTP and exception codes that we know to be noise are filtered out. Each item also produces an evidence field, so an engineer can act on it without leaving the work item to go hunting for context.

The result is the opposite of a wall of alerts. It is the short, reasoned read that a good engineer would give you if they had somehow looked at everything.

Stage 3: turn an insight into tracked work

A finding that lives only in a report is a finding that gets forgotten. So any insight can become an Azure DevOps maintenance user story in one click, or be linked to an existing one if it is already being tracked.

A few deliberate choices make this safe:

• The work item is created on behalf of the signed-in user, using their own personal access token, which never reaches the browser.
• It is pre-filled with the model’s evidence, the affected service, the snapshot date, and a stable reference back to the originating insight.
• A query against the work-item system, by tag, prevents the same insight being raised twice.

The person stays in the loop. The system removes the friction between noticing something and tracking it, which is usually where good intentions die.

Stage 4: let the register reconcile itself

The final stage is the one that keeps the whole thing honest over time. Each insight has a durable lifecycle: open, tracked, resolved, reopened. On each build, the current set of insights is reconciled against the committed snapshot of the work-item system, and the register updates itself:

• An insight is promoted to resolved when its linked work item is marked Done.
• An untracked insight that has been absent for three consecutive builds is resolved automatically, so transient noise clears itself without anyone tidying up.
• Anything that resurfaces is reopened rather than silently duplicated.
• Where an insight looks related to a pre-existing manual work item, it is flagged as possibly related, using a conservative word-overlap check rather than an over-confident match.

The net effect is a maintenance register that surfaces problems and capacity or cost risks before clients feel them, and keeps a durable record of whether each was actioned, in sync automatically. It is the analysis layer behind our managed application support service.

How is it kept cheap and safe to run?

A job that runs every day across every client has to be both robust and inexpensive, or it will either break a report or quietly run up a bill. Two design principles handle that.

It is non-blocking. The analysis is an enhancement to the maintenance report, never a dependency of it. If the API key is missing, if the monitoring data is absent, or if the model call fails, the pipeline falls back to static copy and the report still renders. If the model returns a field in the wrong shape, the code coerces it defensively rather than crashing. Nothing about a bad model day can take down the thing it is meant to support.

It is cost-controlled. Results are cached on a hash of the monitoring context, so a day where nothing material has changed skips the model call entirely. The system prompt and the tool schema, which are large and stable, use prompt caching. A job that could have been expensive at this cadence is, in practice, cheap, because most days do not need a fresh opinion and the pipeline knows it.

Neither of these is exotic. They are the same instincts you would apply to any scheduled integration: fail open, and do not pay for work you have already done.

What this does not do, and where it needs a human

This is the part a product brochure would skip.

• It does not act on production. The pipeline reads and recommends. It does not restart pods, change autoscaler limits, resize databases, or touch infrastructure. A human decides what becomes tracked work, and that work follows the normal review and deployment path.
• It is only as good as the telemetry. A signal that is not collected cannot be reasoned about. The breadth of sources is doing real work here, and gaps in instrumentation are gaps in the analysis. The model will not tell you about a failure mode you are not measuring.
• It is tuned to be conservative. The guardrails that suppress known noise are also capable of suppressing a genuine signal that resembles noise. We would rather miss occasionally than flood an engineer with false positives, because a tool that cries wolf gets ignored, but that is a deliberate trade, not a free lunch.
• Novel failure modes still belong to people. The model is strong on the patterns it has seen across well-run systems. A genuinely new failure, or one specific to an unusual architecture, is exactly where a senior engineer’s judgement still earns its keep. The pipeline buys that engineer time and context; it does not replace them.
• The opinion is borrowed, not innate. The analysis is good because it is grounded in how we run these systems and what we have decided good looks like. Point the same mechanism at an estate with no encoded standards and you get back generic advice, which is the thing we built this specifically to avoid.

Why this is the interesting development

Most AI operations tooling promises to watch everything and alert you to anything. The failure mode of that promise is well known: more alerts, not more clarity. What we wanted was the opposite, a smaller number of better-reasoned findings, each one backed by a real number and a named service, each one one click away from being tracked, and each one cleaning up after itself.

The reproducible idea, for any team that wants to build their own version, is this. Put a senior platform engineer in a daily cron job, but keep the human as the actor and the model as the analyst. Make every finding cite its evidence. Make it non-blocking and cache it hard so it is cheap. And accept that the quality of the output is the quality of the standards you feed it, not a property of the model on its own.

Our next report returns to the quarterly cadence. Before then, the other Between Reports issue goes deep on the harness, the governed scaffolding that makes all of this, maintenance included, consistent enough to trust. Our AI-augmented application support guide covers the wider practice; for everything else, browse the Talk Think Do guides hub.