Skip to main content

Kustomize secret/configMap generator per namespace/environment

The Problem

You have set up Kustomize for your deployment. Directory structure is something like:

- base
- overlays
  - test
  - staging
  - prod

For your secret(s), you want to use Kustomize’s secret generators, and you have a kustomization.yaml in the base directory similar to this:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: dev
commonLabels:
  app: my-app
secretGenerator:
  - name: my-app
    envs:
      - envfile
resources:
  - deployment.yaml
  - service.yaml

Kustomize uses the dev.env to generate a secret you can then reference from your deployment.yaml:

envFrom:
  - secretRef:
      name: my-app

It works fine. But how do you write the test/staging/prod/etc. environment overlay? Do you leave it as is, and just provide an envfile in that directory? Do you list it under resources? Do you declare a secretGenerator again?

Well, none of those will work, at least not without some coaxing.

If you leave it as is, a secret will get created, but it will use the envfile from the base dir.

If you list it under resources, Kustomize has no idea what to do with that and you’ll get an error.

And most significantly, if you re-declare the secretGenerator? It will conflict with the one from base (and all others, when you try to create them), even though you’ve declared a different namespace and all the other resources created respect that. You’ll get an error ending in "behavior must be merge or replace". Therein lies the hint to the solution.

The Solution

You need to specify a behavior for the secretGenerator:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: test
bases:
  - ../../base
secretGenerator:
  - name: my-app
    behavior: replace
    envs:
      - test.envfile

It should be mentioned that the filename doesn’t matter, it can be envfile for every environment, or you can prefix it with the namespace (e.g. dev.envfile) etc.

There are three options: the default is create, while the other two are replace and merge, as per the docs.

Naturally, you could have avoided the whole thing by naming each configMap/secret differently, but that seems to clash with the point of using Kustomize in the first place.

Of course, everything said about secretGenerator applies to configMapGenerator as well.

Here’s an example of an overlay deployment showing how to include a generated configMap:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      containers:
      - image: my-image:tag
        name: my-app
        envFrom:
          - configMapRef:
             name: check