Skip to content

How to Manage Kubernetes Clusters Across Multiple Customers

Updated BiebieKubernetes, DevOps, Multi-Cluster

Managing 20 Kubernetes clusters is one problem.

Managing 20 clusters belonging to eight different customers is another.

The Kubernetes commands are still the same. The difficult part is knowing which customer, network, credential, environment and production boundary you are currently working with.

To manage multiple Kubernetes clusters safely, make those boundaries visible before acting: group by customer, name contexts predictably, scope access and record the network path. A faster context switcher helps, but it cannot supply information that was never recorded.

More than a context switcher

The inventory may look straightforward:

Customer A
├── dev
├── staging
└── production

Customer B
├── dev
└── production

Customer C
├── staging
└── production

But each cluster sits at the end of a chain of choices:

Customer
   ↓
VPN / Network
   ↓
Identity
   ↓
Kubeconfig
   ↓
Context
   ↓
Cluster
   ↓
Namespace

A context selects a cluster, a user entry and optionally a namespace. It does not establish the customer's VPN or prove that your cloud CLI is logged into the intended account. For consultants, MSPs and DevOps teams, those surrounding decisions are often where a routine operation goes wrong.

Start with this chain when diagnosing access failures, too. A timeout, an expired login and an RBAC denial all prevent work, but each points to a different layer.

1. Context confusion

Before running commands, checking the active context helps:

kubectl config current-context

The answer is only useful if the name means something. Compare production, default or a provider-generated identifier with:

customer-a-dev
customer-a-prod
customer-b-dev
customer-b-prod

Use the customer and environment first, adding region or purpose when they disambiguate clusters. A name such as customer-a-prod-eu-payments is longer, but it answers the questions that matter before a write.

Rename contexts in the intended file explicitly:

kubectl --kubeconfig="$HOME/.kube/customers/customer-a.yaml" \
  config rename-context old-prod-name customer-a-prod

Renaming changes local configuration, not the cluster or its permissions. A shell prompt that displays the current context reduces repeated checking, but it remains a reminder rather than an access control.

Make the namespace explicit too

A correct cluster with the wrong namespace can still produce a bad outcome. Set useful namespace defaults where appropriate:

kubectl --kubeconfig="$HOME/.kube/customers/customer-a.yaml" \
  config set-context customer-a-prod --namespace=payments

For a runbook or a command copied between terminals, explicit flags make the target easier to review:

kubectl --kubeconfig="$HOME/.kube/customers/customer-a.yaml" \
  --context=customer-a-prod --namespace=payments get deployments

A namespace default does not restrict permissions, and cluster-scoped resources are outside that namespace boundary. Keep that distinction clear when turning a useful convention into a safety practice.

2. Multiple kubeconfig files

Separate files can mirror customer boundaries:

~/.kube/
  config
  customers/
    customer-a.yaml
    customer-b.yaml
    customer-c.yaml

This makes onboarding, rotation and removing an inactive customer's configuration easier to reason about. It also reduces accidental sharing of unrelated entries. Separate files provide logical organisation; they do not isolate credentials from another process running as the same operating-system user.

You can choose one file with --kubeconfig, or use KUBECONFIG to combine a selected set at runtime. On macOS and Linux:

export KUBECONFIG="$HOME/.kube/customers/customer-a.yaml:$HOME/.kube/customers/customer-b.yaml"
kubectl config get-contexts

Windows uses a semicolon-separated file list. An explicit --kubeconfig selects one file instead of using the environment's merged list.

Runtime merging is convenient for cross-customer visibility without creating another stored copy. A persistent merged file can also be reasonable for a small, well-maintained inventory, but it couples credential rotation and cleanup into one artefact.

The merge rules deserve attention: the first file to set a value or map key wins. Duplicate context, cluster or user names are not automatically renamed into safe unique entries. A collision in a user or cluster name can matter even when context names differ. Check all three namespaces before relying on a merged view. The Kubernetes kubeconfig guide documents this precedence.

Choose the arrangement your team can maintain consistently. Separate customer files are a useful default for consulting work, but a controlled merged setup is not inherently wrong.

Keep credentials scoped and renewable

A kubeconfig may contain embedded tokens, client keys or certificates. Treat those files as sensitive, including any flattened copies. Do not paste full configuration output into tickets simply because the filename sounds like settings.

Where the customer's identity system supports it, an exec credential plugin can obtain short-lived credentials instead of storing a long-lived token. This example user entry asks the AWS CLI for an EKS token:

users:
  - name: customer-a-prod
    user:
      exec:
        apiVersion: client.authentication.k8s.io/v1
        interactiveMode: Never
        command: aws
        args:
          - eks
          - get-token
          - --cluster-name
          - prod-main
          - --region
          - ap-southeast-1
          - --profile
          - customer-a-prod

The example assumes that profile is already configured and authenticated. interactiveMode is required for this exec API version; here authentication happens through a separate login workflow when needed.

The token's lifetime comes from the identity provider, not from the fact that an exec plugin is used. The kubeconfig can also contain other sensitive material. Review customer-supplied configurations before using them because an exec entry can run a local command.

3. Production mistakes

Consider this command as an example of a destructive action, not an investigation step:

kubectl delete deployment api

The command itself may be correct. The dangerous question is: Which cluster is your terminal currently pointing at? The next question is which namespace it will use.

Clearly label production in names and prompts. Use separate production identities, read-only access for ordinary inspection where practical, and restricted RBAC for changes. Avoid using a broadly privileged account for every customer task just because it saves a login.

Confirmation adds useful friction, especially when it includes the customer, cluster, namespace and resource name. It cannot replace RBAC: a confirmation dialog only protects operations that pass through that interface.

For runbooks, prefer explicit targets over assuming the reader's current-context. Recheck after interruptions. An engineer returning from a call may remember the intended operation accurately while forgetting that another terminal task changed the context in the meantime.

Preserve concurrency checks when editing live objects

There is another production boundary: time. A controller or colleague can update an object after you read it and before you save your edit.

For an approved live edit, a read followed by replacement with the original resourceVersion provides optimistic concurrency checking:

kubectl --context=customer-a-prod -n payments \
  get deployment api -o yaml > api.yaml
# Review and edit api.yaml, preserving metadata.resourceVersion.
kubectl --context=customer-a-prod -n payments replace -f api.yaml

If the object changed, Kubernetes rejects a stale replacement with a conflict. Read the new object and reconsider the edit; do not bypass the conflict just to make the command succeed.

This is not a universal description of kubectl apply. Apply uses merge and field-management semantics; server-side apply has field ownership conflicts, which are different from requiring that the whole object still has the version you read. In GitOps environments, check ownership before any live edit because reconciliation may revert it.

4. VPN and network context

Two customers can have perfectly valid kubeconfigs while only one network is reachable:

Customer A cluster
        ↑
Customer A VPN

Customer B cluster
        ↑
Customer B VPN

Switching with kubectl config use-context does not switch VPNs. The selected context references a user entry, but it does not report current VPN state, routing, customer login identity or external authentication state. There is no standard kubectl context command that bundles those layers together.

This becomes especially awkward with overlapping private address ranges or conflicting DNS settings. A cluster endpoint may resolve differently after a VPN connects, or traffic may take an unexpected route while two clients are active.

Record the expected VPN profile, identity, endpoint and relevant routing or DNS dependencies alongside the cluster. During a failure, check network reachability first, then TLS, authentication and authorisation. A connection timeout alone does not prove the API server is down.

Keep VPN credentials and secrets in the approved credential store, rather than copying them into the inventory. The inventory should tell an engineer which access path to use and where to obtain authorisation.

5. Operational context switching

An investigation rarely stops at selecting a cluster. You may move through:

Customer → Environment → VPN → Credential
                              ↓
Cluster → Namespace → Git repository → Argo CD application

Customer A deploys from a production overlay; Customer B uses a Helm values repository; Customer C requires an incident approval before syncing. Remembering those differences consumes attention that should be available for diagnosing the problem.

This is human context scalability. The API servers may comfortably handle their workloads while engineers struggle to carry eight organisations' conventions in their heads.

Keep repository and Argo CD Application references with the cluster and namespace they target. When an image or replica count looks unexpected, follow that relationship before changing anything. The guide to investigating Git versus live configuration explains that next step.

A practical workflow across customers

These steps work with ordinary files, a team inventory and existing Kubernetes tools:

  1. Group clusters by customer. Record the owning team and escalation contact so ambiguous names have an accountable owner.
  2. Use predictable context names. Include environment and distinguish regions or purposes where needed. Check cluster and user names for merge collisions too.
  3. Separate production visibly. Show the target in prompts and runbooks, including namespace. Use explicit command flags for reviewed operations.
  4. Keep credentials scoped. Prefer the customer's supported short-lived login flow and least-privilege roles; document how to renew access.
  5. Record network dependencies. Associate the VPN profile, identity and endpoint with the cluster, including known routing constraints.
  6. Associate GitOps sources. Keep repository, revision or branch, environment path and owning Application discoverable together.
  7. Add safeguards around destructive actions. Combine target confirmation with RBAC and the customer's change process; neither naming nor a dialog is sufficient alone.

At handover, ask a colleague to reach the correct read-only view using those records. If they still need several undocumented facts from you, the workflow is incomplete.

Close out access when an engagement ends

Remove inactive entries from the working inventory and follow the customer's revocation and retention requirements. Preserve permitted, non-secret operational notes when they will help future work.

Archiving useful context is different from retaining working credentials indefinitely. If configuration must be retained, store it according to the customer's policy and outside the active KUBECONFIG set. Restarting an engagement should require a deliberate access check.

How Biebie Kube keeps operational context together

Biebie Kube groups clusters by customer, reads existing kubeconfigs in place and supports production marking. Its destructive-action confirmations can require the resource name, giving the engineer a deliberate checkpoint alongside the customer and environment context.

The same application provides resource views, logs, exec and port forwarding. Its watch-backed resource views help an investigation follow cluster changes without repeated manual refreshes.

For GitOps work, it exposes Argo CD Application sync and health information and can compare a supported source manifest with the live object. Source lookup and comparison have limits, particularly when rendering sits between a repository file and the deployed manifest. Keeping that context nearby helps investigation without changing who owns deployment decisions.

The network side: Biebie Access

Biebie Access organises VPN connections and login identities around customer access. VPN handling depends on the supported client: some integrations connect directly, while others launch the vendor application for the remaining steps.

Biebie Access
      ↓
VPN / network access
      ↓
Biebie Kube
      ↓
Kubernetes cluster

Biebie Access addresses access and network context; Biebie Kube addresses Kubernetes operational context. Neither requires the other: an existing VPN workflow can serve Biebie Kube, and Biebie Access can organise connections used by other tools.

For a desktop workflow across customer clusters, explore Biebie Kube. The separate Biebie Access overview covers the network side and supported VPN integrations.