Skip to content

Kubernetes Watches vs Polling: Why Live Cluster Views Feel Different

Updated BiebieKubernetes, Monitoring, Kubernetes API

You have a Kubernetes dashboard open when a Pod's displayed status changes from Running to CrashLoopBackOff. Some interfaces show the change almost immediately. Others only notice it after the next refresh.

That difference often comes down to how the client talks to the Kubernetes API: polling or watching. Polling asks for another snapshot on a schedule. Watching receives updates as matching resources change, reducing the delay without repeatedly downloading the whole list.

Neither approach guarantees an instant, accurate screen on its own. A watch reduces waiting between updates, but it also creates recovery work for the client. Understanding that trade-off helps when choosing a tool or building a live resource interface.

Two ways to notice a Pod changed

With polling, the client requests the current Pods, displays them, waits, and asks again. This sketch uses a five-second interval; /pods is shorthand for the actual API endpoint.

Client                  Kubernetes API
  |                           |
  |------ GET /pods --------->|
  |<----- current state ------|
  |                           |
  |       wait 5 seconds      |
  |                           |
  |------ GET /pods --------->|
  |<----- current state ------|

With a watch, the client subscribes to changes. The WATCH label below describes the operation; on the wire it is an HTTP GET with watch=true.

Client                  Kubernetes API
  |                           |
  |------ WATCH /pods ------->|
  |                           |
  |<------- ADDED ------------|
  |<------ MODIFIED ----------|
  |<------- DELETED ----------|
  |                           |

Polling repeatedly asks “what is the state now?”

Watching asks Kubernetes to tell the client when the state changes.

If a container starts failing just after a poll, the interface waits almost a full interval before requesting the next snapshot. A watch has no polling interval to wait out, although API processing, network delivery, client work and rendering still take time.

One Kubernetes detail matters here: CrashLoopBackOff is a container waiting reason commonly shown as a Pod's status, not a Pod phase. A Pod can still have phase Running while one of its containers repeatedly crashes. A useful interface interprets container status as well as the top-level phase.

What repeated snapshots cost

A client listing Pods might send this request every few seconds:

GET /api/v1/namespaces/default/pods

Each successful full-list response carries the matching objects again, including those that have not changed. If a namespace contains 400 Pods, an ordinary unpaginated list returns those 400 objects each time. More clients and shorter intervals increase response traffic and client processing.

That does not mean every request reads every object directly from etcd. Kubernetes can serve list requests from its watch cache, depending on version and request semantics. Caching reduces storage work; it does not eliminate the cost of serialising and transferring repeated results.

For an ideal five-second polling schedule, detection waits between approximately zero and five seconds before request and rendering time. An average of 2.5 seconds only follows if changes arrive uniformly between polls. Slow requests, retries and background-tab scheduling can make the delay longer.

A watch generally saves repeated full-list transfers when changes are sparse. Busy resources still generate traffic, and streams, server bookkeeping and local caches all consume resources. The right comparison includes object count, change rate, number of viewers and recovery behaviour.

Why production clients use List + Watch

A stream of future changes cannot, by itself, tell a new client which objects already exist. The conventional solution is to obtain a snapshot and then continue from its position in the API's history.

1. LIST
   ↓
Get current objects + list resourceVersion

2. WATCH
   ↓
Start watching from that resourceVersion

3. RECEIVE EVENTS
   ↓
ADDED / MODIFIED / DELETED update the local view

4. CONNECTION BREAKS
   ↓
Reconnect or relist when necessary

The list response includes metadata.resourceVersion. Treat it as an opaque token: pass it back to Kubernetes, rather than parsing it as a timestamp or incrementing it. Use the list's version for this handoff, not the version of an arbitrary object in the list.

The next request looks like this, with an illustrative version:

GET /api/v1/namespaces/default/pods?watch=true&resourceVersion=41827

Using the snapshot's version lets the client receive subsequent changes, including those made between completion of the list and opening the watch, while that history remains available. Keep the resource scope and selectors consistent between the two requests.

ADDED introduces an object into the watched collection. MODIFIED updates it. DELETED removes it. For ordinary full-object watches, notifications contain an object representation rather than a JSON patch; clients replace or update the corresponding cached entry.

For example, this is an abbreviated notification, with unrelated Pod fields omitted:

{
  "type": "MODIFIED",
  "object": {
    "kind": "Pod",
    "metadata": {
      "namespace": "default",
      "name": "api-7d9f",
      "resourceVersion": "41831"
    }
  }
}

Record progress as events are processed. Do not treat a version merely received into a network buffer as proof that every corresponding cache update succeeded.

Kubernetes also supports streaming initial objects through watch requests on supported versions and configurations. That is another way to initialise state, not permission to skip initialisation. The conventional List + Watch model remains useful for understanding client behaviour. The Kubernetes API concepts documentation describes the protocol and its version-specific options.

Watches break: recovery is part of the design

An HTTP stream is not a permanent connection. API servers impose timeouts, proxies close connections, laptops sleep, VPNs disconnect and credentials expire. A client must distinguish a recoverable interruption from an empty cluster.

After an ordinary interruption, it can reconnect from its last processed resourceVersion, if Kubernetes still retains the necessary history. Retries need backoff so an unavailable cluster does not trigger a tight request loop. Authentication and authorisation failures also need useful errors; retrying cannot restore permission that was revoked.

When the requested history is no longer available, Kubernetes can return 410 Gone, including as an error notification on a watch. Retrying the same expired version will not recover the missing changes.

Expired resourceVersion / 410 Gone
              ↓
Discard the stale snapshot as authoritative
              ↓
LIST again and replace the local collection
              ↓
WATCH from the new list resourceVersion

Replacing the collection matters. Simply adding the newly listed objects to an old cache leaves behind objects deleted during the gap. A UI may keep old rows visible while reconnecting, but it should identify them as stale rather than imply that the cluster has confirmed them.

A relist recovers current state, not the sequence of everything that happened while disconnected. A short-lived Pod could have appeared and disappeared during that gap. Watches support current resource views; they are not a durable audit log or a substitute for historical monitoring.

Clients can request BOOKMARK events with allowWatchBookmarks=true. A bookmark carries a progress version without a full resource object. Its delivery and timing are not guaranteed, so it helps maintain progress but cannot replace timeout handling or relisting.

What an informer adds

Engineers normally use an established client library instead of building all this recovery logic into each screen. In Go, client-go informers combine resource observation with a shared local cache and event handlers.

That cache lets filtering, sorting and repeated reads happen locally. It avoids an API round trip for each keystroke or row lookup, though local reads still cost CPU and memory. The cache is eventually consistent: it reflects changes after they have reached and been processed by the client.

An interface must also wait for initial cache synchronisation before treating an empty cache as “no resources.” Otherwise, opening a namespace can briefly look like every Pod disappeared while the initial list is still loading.

Informer resync and API relist are different operations. A configured resync can re-deliver cached objects to handlers; it does not inherently mean downloading a fresh list on that timer. Recovery of the underlying watch is managed separately by the client machinery.

Memory grows with the objects retained, so share informers where appropriate and limit their scope. Opening every resource kind in every namespace across every cluster is a different workload from observing the few collections an engineer is investigating.

Is polling wrong?

No. A script checking a small deployment once a minute may need nothing more than a periodic request. Infrequently changing data, tiny tools, low-frequency status checks and APIs without watch semantics are all reasonable uses of polling.

The simpler implementation can be easier to operate. Each successful poll gives a new snapshot, and a failed request can be retried on the next cycle. It still needs sensible timeouts and a clear indication when the last request failed.

For live resource interfaces, watches are often worth the additional complexity because users care about changes between refreshes. For periodic reporting, the recovery and cache machinery may offer little practical benefit.

PollingWatch
Update latencyDepends on intervalNear real-time
API requestsRepeated requestsPersistent stream + recovery
ImplementationSimplerMore complex
Failure recoveryStraightforwardRequires careful handling
Good forPeriodic state checksLive resource interfaces

“Near real-time” describes expected behaviour under working conditions, not a latency guarantee. A broken watch with no visible warning can be less trustworthy than a poller that clearly reports its last successful check.

Practical consequences for cluster interfaces

During a rollout, a watch can expose short-lived startup states that polling steps over. That helps an engineer connect a failing container with the preceding scheduling or image-pull activity. It still does not guarantee every intermediate state will appear on screen: clients may batch updates, and the API itself reports observations rather than every runtime event.

Deletion also has stages. A deletion request may first set deletionTimestamp, producing an update while finalizers or graceful shutdown keep the object present. The row should disappear when the object is actually removed, not merely because the user clicked Delete.

Across several clusters, each connection needs independent cache and recovery state. A customer's VPN failure must not freeze unrelated cluster views, and inactive watches need releasing. The broader multi-customer workflow includes the network and identity boundaries behind those failures.

A current view answers what the API reports now. Understanding whether it matches the intended configuration is a separate step, covered in investigating Git and live state together.

How Biebie Kube handles live resources

Biebie Kube uses Kubernetes watches through client-go informers to keep resource views current without repeatedly requiring manual refresh. Its resource cache feeds the interface as changes arrive.

Pod changes state
        ↓
Kubernetes API watch event
        ↓
Biebie Kube resource model updates
        ↓
UI reflects the new state

The implementation batches change notifications over a short window to avoid repainting for every event in a busy rollout. It also limits retained watches and expires idle ones. Those are deliberate compromises between responsiveness, rendering work and memory use, rather than a promise of instantaneous updates.

For engineers working across clusters, the Biebie Kube Kubernetes desktop client puts this approach into a resource interface alongside the operational tools used to investigate changes.