[{"data":1,"prerenderedAt":453},["ShallowReactive",2],{"blog-\u002Fblog\u002Fkubernetes-watches-versus-polling":3,"blog-surround-\u002Fblog\u002Fkubernetes-watches-versus-polling":448},{"id":4,"title":5,"author":6,"body":7,"description":433,"extension":434,"image":435,"meta":436,"navigation":437,"path":438,"publishedAt":439,"relatedProducts":440,"seo":442,"stem":443,"tags":444,"updatedAt":435,"__hash__":447},"blog\u002Fblog\u002Fkubernetes-watches-versus-polling.md","Watches vs Polling: Why Kubernetes Tools Feel Slow","Biebie",{"type":8,"value":9,"toc":425},"minimark",[10,14,17,22,25,36,39,42,45,49,52,55,61,68,71,77,80,263,266,269,272,278,282,285,291,301,304,310,320,334,338,346,349,352,355,361,364,368,371,378,389,395,399,402,405,408,421],[11,12,13],"p",{},"There is a particular feeling you get from a slow Kubernetes interface. You delete a pod, the row stays. You hit refresh, it is still there. You hit refresh again and now there are two pods, one terminating and one starting, and you are not sure which of those states is current and which is the interface catching up.",[11,15,16],{},"That feeling almost always has the same cause: the tool is polling. Understanding what it should be doing instead explains a lot about why some Kubernetes tools feel immediate and others feel like they are describing the recent past.",[18,19,21],"h2",{"id":20},"what-polling-actually-costs","What polling actually costs",[11,23,24],{},"A polling client asks the same question on a timer:",[26,27,32],"pre",{"className":28,"code":30,"language":31},[29],"language-text","GET \u002Fapi\u002Fv1\u002Fnamespaces\u002Fdefault\u002Fpods\nGET \u002Fapi\u002Fv1\u002Fnamespaces\u002Fdefault\u002Fpods\nGET \u002Fapi\u002Fv1\u002Fnamespaces\u002Fdefault\u002Fpods\n","text",[33,34,30],"code",{"__ignoreMap":35},"",[11,37,38],{},"Each request makes the API server read every matching object from etcd, serialise all of them, and send the complete set over the wire. The client then throws away the previous copy and renders the new one.",[11,40,41],{},"The cost scales with the number of objects and the number of clients, not with how much actually changed. A namespace with 400 pods that have not moved in an hour still transfers 400 pods every poll. Ten engineers with the dashboard open is ten times that. This is why cluster administrators have opinions about dashboards.",[11,43,44],{},"The latency is also structurally bad. With a five-second interval, the average change is displayed 2.5 seconds after it happened, and the worst case is five. Shortening the interval improves the latency and makes the load problem worse in exact proportion. There is no interval that is both fast and cheap, because the mechanism is wrong.",[18,46,48],{"id":47},"the-list-and-watch-protocol","The list-and-watch protocol",[11,50,51],{},"Kubernetes provides a better primitive, and it is the same one every controller in the system uses.",[11,53,54],{},"The client starts with one list:",[26,56,59],{"className":57,"code":58,"language":31},[29],"GET \u002Fapi\u002Fv1\u002Fnamespaces\u002Fdefault\u002Fpods\n",[33,60,58],{"__ignoreMap":35},[11,62,63,64,67],{},"The response includes a ",[33,65,66],{},"resourceVersion"," on the list itself. That value is a position in the cluster's change stream — a bookmark saying \"this is the state of the world as of here\".",[11,69,70],{},"The client then opens a watch starting from that position:",[26,72,75],{"className":73,"code":74,"language":31},[29],"GET \u002Fapi\u002Fv1\u002Fnamespaces\u002Fdefault\u002Fpods?watch=true&resourceVersion=41827\n",[33,76,74],{"__ignoreMap":35},[11,78,79],{},"This request does not return. It stays open and the API server writes an event into it every time a matching object changes:",[26,81,85],{"className":82,"code":83,"language":84,"meta":35,"style":35},"language-json shiki shiki-themes github-dark","{\"type\":\"MODIFIED\",\"object\":{\"kind\":\"Pod\",\"metadata\":{\"name\":\"api-7d9f\",\"resourceVersion\":\"41831\"},...}}\n{\"type\":\"DELETED\",\"object\":{\"kind\":\"Pod\",\"metadata\":{\"name\":\"api-6c2a\",\"resourceVersion\":\"41832\"},...}}\n{\"type\":\"ADDED\",\"object\":{\"kind\":\"Pod\",\"metadata\":{\"name\":\"api-9f1b\",\"resourceVersion\":\"41833\"},...}}\n","json",[33,86,87,159,211],{"__ignoreMap":35},[88,89,92,96,100,103,107,110,113,116,119,121,124,126,129,131,134,136,139,141,144,146,149,152,156],"span",{"class":90,"line":91},"line",1,[88,93,95],{"class":94},"s95oV","{",[88,97,99],{"class":98},"sDLfK","\"type\"",[88,101,102],{"class":94},":",[88,104,106],{"class":105},"sU2Wk","\"MODIFIED\"",[88,108,109],{"class":94},",",[88,111,112],{"class":98},"\"object\"",[88,114,115],{"class":94},":{",[88,117,118],{"class":98},"\"kind\"",[88,120,102],{"class":94},[88,122,123],{"class":105},"\"Pod\"",[88,125,109],{"class":94},[88,127,128],{"class":98},"\"metadata\"",[88,130,115],{"class":94},[88,132,133],{"class":98},"\"name\"",[88,135,102],{"class":94},[88,137,138],{"class":105},"\"api-7d9f\"",[88,140,109],{"class":94},[88,142,143],{"class":98},"\"resourceVersion\"",[88,145,102],{"class":94},[88,147,148],{"class":105},"\"41831\"",[88,150,151],{"class":94},"},",[88,153,155],{"class":154},"s6RL2","...",[88,157,158],{"class":94},"}}\n",[88,160,162,164,166,168,171,173,175,177,179,181,183,185,187,189,191,193,196,198,200,202,205,207,209],{"class":90,"line":161},2,[88,163,95],{"class":94},[88,165,99],{"class":98},[88,167,102],{"class":94},[88,169,170],{"class":105},"\"DELETED\"",[88,172,109],{"class":94},[88,174,112],{"class":98},[88,176,115],{"class":94},[88,178,118],{"class":98},[88,180,102],{"class":94},[88,182,123],{"class":105},[88,184,109],{"class":94},[88,186,128],{"class":98},[88,188,115],{"class":94},[88,190,133],{"class":98},[88,192,102],{"class":94},[88,194,195],{"class":105},"\"api-6c2a\"",[88,197,109],{"class":94},[88,199,143],{"class":98},[88,201,102],{"class":94},[88,203,204],{"class":105},"\"41832\"",[88,206,151],{"class":94},[88,208,155],{"class":154},[88,210,158],{"class":94},[88,212,214,216,218,220,223,225,227,229,231,233,235,237,239,241,243,245,248,250,252,254,257,259,261],{"class":90,"line":213},3,[88,215,95],{"class":94},[88,217,99],{"class":98},[88,219,102],{"class":94},[88,221,222],{"class":105},"\"ADDED\"",[88,224,109],{"class":94},[88,226,112],{"class":98},[88,228,115],{"class":94},[88,230,118],{"class":98},[88,232,102],{"class":94},[88,234,123],{"class":105},[88,236,109],{"class":94},[88,238,128],{"class":98},[88,240,115],{"class":94},[88,242,133],{"class":98},[88,244,102],{"class":94},[88,246,247],{"class":105},"\"api-9f1b\"",[88,249,109],{"class":94},[88,251,143],{"class":98},[88,253,102],{"class":94},[88,255,256],{"class":105},"\"41833\"",[88,258,151],{"class":94},[88,260,155],{"class":154},[88,262,158],{"class":94},[11,264,265],{},"Three properties follow from this, and they are the entire difference.",[11,267,268],{},"Traffic is proportional to change. A quiet namespace costs nothing after the initial list, no matter how many objects it contains or how long you leave the window open.",[11,270,271],{},"Latency is proportional to nothing. The event is written when the change is committed. There is no interval to wait out.",[11,273,274,275,277],{},"Nothing is missed. Because every event carries a ",[33,276,66],{}," and the stream is ordered, the client always knows exactly where it is.",[18,279,281],{"id":280},"why-watches-expire-and-what-to-do-about-it","Why watches expire, and what to do about it",[11,283,284],{},"A watch is not permanent. The API server will close it, and there are two distinct reasons that need different handling.",[11,286,287,288,290],{},"The mundane one is that the server closes idle or long-lived connections deliberately, to spread load when API server instances come and go. The client reconnects from the last ",[33,289,66],{}," it saw and continues without a gap.",[11,292,293,294,297,298,300],{},"The interesting one is ",[33,295,296],{},"410 Gone",". The API server keeps a limited window of change history — etcd compacts old revisions, and the watch cache holds a bounded number of recent events. If your client is offline long enough that its ",[33,299,66],{}," falls out of that window, the server can no longer tell you what you missed. It cannot send you the events, because they are gone.",[11,302,303],{},"The only correct response is to list again from scratch and resynchronise:",[26,305,308],{"className":306,"code":307,"language":31},[29],"410 Gone  ->  GET \u002Fapi\u002Fv1\u002F...\u002Fpods  ->  watch from the new resourceVersion\n",[33,309,307],{"__ignoreMap":35},[11,311,312,313,316,317,319],{},"A client that handles ",[33,314,315],{},"410"," by simply reconnecting from the same stale version gets ",[33,318,315],{}," again, forever, and quietly stops updating while continuing to look like it is working. This is a real and common bug, and it produces exactly the symptom of a view that was correct when you opened it and has been drifting ever since.",[11,321,322,323,326,327,329,330,333],{},"There is also a ",[33,324,325],{},"BOOKMARK"," event type, which exists specifically for this problem. The server periodically sends an event with no object payload, just a current ",[33,328,66],{},", so that a client watching a quiet resource keeps its position fresh instead of falling behind the compaction window while nothing happens. A client that requests bookmarks with ",[33,331,332],{},"allowWatchBookmarks=true"," is much harder to strand.",[18,335,337],{"id":336},"what-an-informer-adds","What an informer adds",[11,339,340,341,345],{},"In practice, clients do not implement list-and-watch by hand. The Kubernetes client libraries provide an ",[342,343,344],"strong",{},"informer",", which wraps the protocol and adds a local cache.",[11,347,348],{},"The informer holds every object of its resource type in memory, updated by the watch stream. That cache is what your interface reads from. It has three consequences worth naming.",[11,350,351],{},"Reads become free. Filtering, sorting and searching happen against local memory. Typing in a filter box does not generate API traffic, so it can be genuinely instant rather than debounced-and-hopeful.",[11,353,354],{},"The full object is available. A watch event carries the entire object, not a diff, so the cache always has complete objects and any view can be rendered without going back to the API server for detail.",[11,356,357,358,360],{},"Reconnection is handled once. Expiry, ",[33,359,315],{},", resync and backoff live in the informer rather than being reimplemented, differently and incompletely, in each part of the application.",[11,362,363],{},"The cost is memory, and it is proportional to the number of objects being watched. This is a real constraint at scale, which is why watching every resource kind in every namespace at once is not a sensible default. Watching what is currently on screen is.",[18,365,367],{"id":366},"where-this-shows-up-in-the-interface","Where this shows up in the interface",[11,369,370],{},"The architecture is visible from the outside once you know what to look for.",[11,372,373,374,377],{},"A watch-backed view has no refresh button, because a refresh button would do nothing that is not already happening. When you delete a pod, the row disappears when the deletion is committed. A pod that enters ",[33,375,376],{},"CrashLoopBackOff"," at 14:02 changes colour at 14:02, not on the next tick.",[11,379,380,381,384,385,388],{},"A watch-backed view can also show transient states at all. Pods pass through ",[33,382,383],{},"Pending"," and ",[33,386,387],{},"ContainerCreating"," in a few seconds. A five-second poll will frequently step straight over them, so failures during startup appear as an object that was fine and is now broken, with the intermediate state that would have explained why never displayed.",[11,390,391,392,394],{},"And a watch-backed view stays honest when you leave it open. The reason ",[33,393,315],{}," handling matters is that the failure it causes is invisible: nothing errors, the view simply stops changing. If you have ever left a dashboard open over lunch and come back to a cluster state that turned out to be an hour old, you have probably seen it.",[18,396,398],{"id":397},"why-this-is-harder-in-a-multi-cluster-tool","Why this is harder in a multi-cluster tool",[11,400,401],{},"Everything above describes one client watching one cluster. A tool that shows several clusters at once has to do all of it per cluster, in parallel, and the failure modes stop being independent.",[11,403,404],{},"Each cluster needs its own informers and its own reconnection state. A cluster that becomes unreachable — a VPN dropping, most often — must not stall the views for the others. Credentials expire at different times, because they came from different identity providers. And a cluster you switch away from should have its watches released rather than left running, or the memory cost grows with every cluster you have visited rather than every cluster you are using.",[11,406,407],{},"None of that is conceptually difficult, but it is a meaningful amount of state to get right, and it is the reason that many otherwise good single-cluster tools feel fragile the moment you point them at a dozen clusters belonging to a dozen organisations.",[11,409,410,415,416,420],{},[411,412,414],"a",{"href":413},"\u002Fbiebie-kube","Biebie Kube"," is built on watch-backed resource views across every cluster you have configured, which is what makes its tables update in place rather than on a timer. If you want the operational patterns that go with it, the article on ",[411,417,419],{"href":418},"\u002Fblog\u002Fmanaging-kubernetes-clusters-for-multiple-customers","managing Kubernetes clusters for multiple customers"," covers kubeconfig layout, context naming and production safety.",[422,423,424],"style",{},"html pre.shiki code .s95oV, html code.shiki .s95oV{--shiki-default:#E1E4E8}html pre.shiki code .sDLfK, html code.shiki .sDLfK{--shiki-default:#79B8FF}html pre.shiki code .sU2Wk, html code.shiki .sU2Wk{--shiki-default:#9ECBFF}html pre.shiki code .s6RL2, html code.shiki .s6RL2{--shiki-default:#FDAEB7;--shiki-default-font-style:italic}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":35,"searchDepth":213,"depth":213,"links":426},[427,428,429,430,431,432],{"id":20,"depth":161,"text":21},{"id":47,"depth":161,"text":48},{"id":280,"depth":161,"text":281},{"id":336,"depth":161,"text":337},{"id":366,"depth":161,"text":367},{"id":397,"depth":161,"text":398},"How the Kubernetes list-and-watch protocol works, what informers actually cache, and why a tool that polls the API server will always feel a step behind.","md",null,{},true,"\u002Fblog\u002Fkubernetes-watches-versus-polling","2026-08-11",[441],"biebie-kube",{"title":5,"description":433},"blog\u002Fkubernetes-watches-versus-polling",[445,446],"Kubernetes","Architecture","lzqyHUkWGNvVpL4Xm6IGGBJc9fMoYkGVqckZa8NAL7Y",[435,449],{"title":450,"path":418,"stem":451,"description":452,"children":-1},"How to Manage Kubernetes Clusters for Multiple Customers","blog\u002Fmanaging-kubernetes-clusters-for-multiple-customers","Practical patterns for kubeconfig layout, context naming and production safety when the clusters you work with belong to different organisations.",1788424373700]