We have discussed how to deploy nginx-gateway in part 2 of this blog series. As the nginx-gateway is deployed and ready now, we can start setting up HTTP routes, so that it is ready to replace the ingress-nginx (retired) routes already working in the AKS cluster. The example in this blog uses, routes to AKS hosted elastic search, so there are two routes, the kibana dashboard route and the elastic search route used by the apps. The currrent routes use a private DNS, and a private IP within Azure vNET. We have used a new private IP for nginx gateway in deploy nginx-gateway in part 2 , so that current retired ingress-nginx internal loadbalancer is not affected, while we setup nginx-gateway (uses another private IP for loadbalancer of gateway), making them both available in AKS cluster. Therefore, setting up same host names for gateway routes will not cause issues in existing ingress-nginx setup. After this step of setting up routes, we will have both ingress-nginx (retired) and nginx-gateway with routes, working in same cluster, but live traffic will be still using ingress-nginx.
The expectation is to have routes using nginx-gateway is setup for ealstic search and kibana as shown below.
The current route setup with ingress-nginx (retired)---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: es-search-ingress
namespace: elastic-stack
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/proxy-body-size: 4g
nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
nginx.ingress.kubernetes.io/proxy-send-timeout: "30"
nginx.ingress.kubernetes.io/proxy-read-timeout: "30"
nginx.ingress.kubernetes.io/proxy-max-temp-file-size: "0"
spec:
ingressClassName: nginx
rules:
- host: es-search${sys_sh_svc_deploy_dns_zone}$
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: es-search-es-http
port:
number: 9200
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: es-kibana-ingress
namespace: elastic-stack
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/proxy-body-size: 4g
nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
nginx.ingress.kubernetes.io/proxy-send-timeout: "30"
nginx.ingress.kubernetes.io/proxy-read-timeout: "30"
nginx.ingress.kubernetes.io/proxy-max-temp-file-size: "0"
spec:
ingressClassName: nginx
rules:
- host: es-kibana${sys_sh_svc_deploy_dns_zone}$
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: es-kibana-kb-http
port:
number: 5601
Compare ingress-nginx and nginx-gateway
Below chart shows, for each annotations etc, we use in ingress-nginx, what would be the related setup in nginx-gateway setup.
The HTTP route setup with nginx-gateway
Below is how we can setup HTTP routes in nginx-gateway, as a replacement to above mentioned ingress-nginx routing setup for elastic search and kibana.
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: es-search-http-route
namespace: elastic-stack
spec:
parentRefs:
- name: selfhost-apps-gateway
namespace: nginx-gateway
hostnames:
- "es-search${sys_sh_svc_deploy_dns_zone}$"
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: es-search-es-http
port: 9200
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: es-kibana-http-route
namespace: elastic-stack
spec:
parentRefs:
- name: selfhost-apps-gateway
namespace: nginx-gateway
hostnames:
- "es-kibana${sys_sh_svc_deploy_dns_zone}$"
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: es-kibana-kb-http
port: 5601
Nginx-gateway policies instead of ingress-nginx annotations
We specifically have used below annotation in ingress nginx for the kibana and elastic search routes.
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/proxy-body-size: 4g
nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
nginx.ingress.kubernetes.io/proxy-send-timeout: "30"
nginx.ingress.kubernetes.io/proxy-read-timeout: "30"
nginx.ingress.kubernetes.io/proxy-max-temp-file-size: "0"
In the nginx-gateway as long as we have not specified https/SSL based listener, the ssl-redirect-false in ingress nginx, is already setup there in nginx-gateway.
The Gateway listener:
No HTTPS listener, no redirect configured.
➡️ Equivalent to:
✔️ Nothing else needed.
Now we configure the remaining annotations
These apply to both Kibana and Elasticsearch routes:
Best practice in nginx-gateway would be to
👉 Create TWO policies
1️⃣ Client policy (body size)
Attach to HTTPRoutes or Gateway. Since in the case of elastic search and kibana route uses same policy, we have already setu[ it for gateeway as shown below (see how to deploy nginx-gateway in part 2 of this blog series).
---
apiVersion: gateway.nginx.org/v1alpha1
kind: ClientSettingsPolicy
metadata:
name: selfhost-apps-gateway-client-settings
namespace: nginx-gateway
spec:
targetRef:
group: gateway.networking.k8s.io
kind: Gateway
name: selfhost-apps-gateway
body:
maxSize: "4g"
2️⃣ Upstream settings policy (timeouts + buffering)
Attach to BOTH Services:
- es-kibana-kb-http
- es-search-es-http
Explanation of above settings is below.
Equivalent of:
Below is additional information on how Kubernetes API Groups Work
A resource is uniquely identified by:
Omce we have get all above deployed, now we have both ingress-nginx and niginx gateway working routes for elastic seach and kibana hosted in AKS. The traffic ciurrently routes via ingress nginx as the private DNS zone still uses, private IP of internal load balancer of the ingress-nginx, for the host name specified. In the next post of this series, let's look at how to switch to use nginx-gateway roputes, by changing private DNS zone IP. Note that, we must get the nginx gateway with route setup deployed with release 1 to all production environments, before we start switch to use nginx-gateway in the next release 2 (see
part 1 of this blog series for detailed plan).
No comments:
Post a Comment