Retry configurations are not working with proxy_next_upstream as I tried scaling the pods to 0 using the command kubectl scale --replicas=0. Checked the logs of the ingress-nginx-controller pod, I am getting 503 Service Temporary Unavailable.

Here are my deployment, service and ingress.

Service
----------

apiVersion: v1
kind: Service
metadata:
name: prometheus-flask-app-prometheus-flask-app
labels:
app: prometheus-flask-app
chart: sbd-prometheus-flask-app-1.1
release: prometheus-flask-app
heritage: Helm
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 5000
protocol: TCP
name: sbd-prometheus-flask-app
selector:
app: prometheus-flask-app
release: prometheus-flask-app

Deployment
----------------

apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus-flask-app-prometheus-flask-app
labels:
app: prometheus-flask-app
chart: sbd-prometheus-flask-app-1.1
release: prometheus-flask-app
heritage: Helm
spec:
replicas: 1
selector:
matchLabels:
app: prometheus-flask-app
release: prometheus-flask-app
template:
metadata:
labels:
app: prometheus-flask-app
release: prometheus-flask-app
spec:
containers:
- name: sbd-prometheus-flask-app
image: flaskapp:latest
imagePullPolicy: Always
ports:
- containerPort: 5000
livenessProbe:
httpGet:
path: /healthz
port: 5000
initialDelaySeconds: 600
readinessProbe:
httpGet:
path: /healthz
port: 5000
initialDelaySeconds: 30
resources:
requests:
cpu: 20m
memory: 100Mi

Ingress
----------

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: prometheus-flask-app-prometheus-knative-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
# nginx.ingress.kubernetes.io/configuration-snippet: |
# proxy_next_upstream error timeout http_502 http_503 http_504 non_idempotent;
nginx.ingress.kubernetes.io/proxy-next-upstream: "error non_idempotent http_502 http_503 http_504"
nginx.ingress.kubernetes.io/retry-non-idempotent: "true"
spec:
rules:
- host: flaskapp.eu.staging.mcdpricing.com
http:
paths:
- path: /?(.*)
pathType: Prefix
backend:
service:
name: prometheus-flask-app-prometheus-flask-app
port:
number: 80