[Python] Python Kubernetes MutatingWebhook Not Adding Labels

I have created a Kubernetes MutatingWebhook using a Docker image I built from Python. The purpose of the webhook is to add labels to any Pod that is launched. All Kubernetes resources are running fine, however, when new Pods are launched they do not get the expected labels from the MutatingWebhook.

Here is the Python webhook code:

import json
from flask import Flask, request, jsonify
import base64

app = Flask(__name__)

@app.route('/mutate', methods=['POST'])
def mutate():
# Read the admission review request
admission_review = request.get_json()
print("AdmissionReview Request:", json.dumps(admission_review))

# Define the patch to add labels
patch = [
{
"op": "add",
"path": "/metadata/labels/live",
"value": "true"
},
{
"op": "add",
"path": "/metadata/labels/environment",
"value": "production"
},
{
"op": "add",
"path": "/metadata/labels/service",
"value": "my-service"
},
{
"op": "add",
"path": "/metadata/labels/version",
"value": "7.55.2"
}
]

# Encode the patch to base64
patch_base64 = base64.b64encode(json.dumps(patch).encode()).decode()

# Prepare the admission review response
admission_response = {
"uid": admission_review['request']['uid'],
"allowed": True,
"patchType": "JSONPatch",
"patch": patch_base64
}

# Return the admission review response
response = {
"apiVersion": "admission.k8s.io/v1",
"kind": "AdmissionReview",
"response": admission_response
}

return jsonify(response)

if __name__ == '__main__':
app.run(host='0.0.0.0', port=443, ssl_context=('tls.crt', 'tls.key'), debug=True)


Here are the various K8s manifest files that I used for the deployment, services, etc.

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: add-labels-webhook
namespace: mutatingwh
spec:
replicas: 1
selector:
matchLabels:
app: add-labels-webhook
template:
metadata:
labels:
app: add-labels-webhook
spec:
containers:
- name: webhook
image: 805960120419.dkr.ecr.us-east-1.amazonaws.com/noc/add-labels-webhook:latest
ports:
- containerPort: 443
imagePullSecrets:
- name: ecr-secret


service.yaml

apiVersion: v1
kind: Service
metadata:
name: add-labels-webhook-svc
namespace: mutatingwh
spec:
ports:
- port: 443
targetPort: 443
selector:
app: add-labels-webhook


mutatingwebhookconfiguration.yaml

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: add-labels-webhook
webhooks:
- name: add-labels.k8s.io
clientConfig:
service:
name: add-labels-webhook-svc
namespace: mutatingwh
path: "/mutate"
caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURNekNDQWh1Z0F3SUJBZ0lVTEpJVlFkcXBiR05yb3N3Rk9SRDA0SzREbnpjd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0tURW5NQ1VHQTFVRUF3d2VZV1JrTFd4aFltVnNjeTEzWldKb2IyOXJMbVJsWm1GMWJIUXVjM1pqTUI0WApEVEkwTVRBeU5UQTRORGd3T0ZvWERUSTFNVEF5TlRBNE5EZ3dPRm93S1RFbk1DVUdBMVVFQXd3ZVlXUmtMV3hoClltVnNjeTEzWldKb2IyOXJMbVJsWm1GMWJIUXVjM1pqTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEEKTUlJQkNnS0NBUUVBc05CWG0yWE1TZVNTUUxsZ2ZvR01wRk0xeHRtdmxxN1JrK2RoTUw0b211VVN1NDJaMk9VcAp1S3RJdkdtRnorYUpBU3FONmFTY0xJbFFTTElSdmNqNnJ3aXVwcGZVQldSNkdDeEhGYW52R0VxT0YxSkR6OEptCko4Zkp3b09aMFJqQmtYdWFLSFdtNC9NSHV6T240SUdnV3dRRnN2Tk1ZUitFMkRsRjZnOWE4c0cwbjVQTVVydHkKNGdiVDduY2pFZnc2NFc2SmZqRzFNTDNLTXI3aUZ1NXBxM0hJMXh2cSt5T2lDNGQ4WEVJL3FibU9vbDdOYUZDRwpyNVZRd0hLMWx4aHBZY2VWVTNlQWIxS0tHL3FDSVdURVptbTlFbGZuMWJRbzZtOEVxenVES2QyTHE1VmRqUWVnCmFJRkNMYmd2YWx0S1FWaGIwUEhNRjlqUE5yN0xLenRsclFJREFRQUJvMU13VVRBZEJnTlZIUTRFRmdRVUpYcDEKdnVmcXVFbW9VTDhmL3BUZXEzQ25vME13SHdZRFZSMGpCQmd3Rm9BVUpYcDF2dWZxdUVtb1VMOGYvcFRlcTNDbgpvME13RHdZRFZSMFRBUUgvQkFVd0F3RUIvekFOQmdrcWhraUc5dzBCQVFzRkFBT0NBUUVBSTNUczh0UE1zR29kCjZvU2ZxanQ0Rzk0SWR4SjVRZkZMaG5hTnBmS0tBRldDZVI2cVEzYWtWWFNnVzVFSlBud0dtdjBYRkh2YWtXa1EKc3A5QjhvaVI1OWlWRHpFME5QdytHQmkzMWkrTW9aaDAzYXYyWFg0K1FKOTFkRENldkFZeEQ4RkxackZ3UTJEMwpjUVdmQk1kSS9JSys5V1dWNXRtZmdtN2d3bi9XNy9nTC9ScVNkaVQwWllmRjZSVjhNeHFaano4bGFWUXhCU2FYCi9kNVROZi9XcldIUjlIek1xRU91UXBIL3NqRVlUM2xkbTA2aTdKMFVsOFJDQTkyL3c2L3dLRk82SHFYVWR1RVoKS25KTHdZV1A4YnJkMHhTcFhYUFRrbHl0T3ZLYkt3N2ljeDRaYzkxbXJOMjRGZ2k5OWN3L2tIZng4cnQ2RVJqawp0S3BLSCt2OVpRPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
admissionReviewVersions: ["v1"]
sideEffects: None
rules:
- operations: ["CREATE"]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
failurePolicy: Ignore


My question is why is everything running fine, but when new Pods are created they do not get the labels from the Python webhook?

Continue reading...