--- name: debugging-k8s-networking description: Debugs Kubernetes networking issues including Service connectivity, DNS resolution, Ingress routing, Endpoints, and NetworkPolicy. Use when services are unreachable, DNS fails, ingress not routing, or network connectivity problems. allowed-tools: Bash --- # Debugging Kubernetes Networking Investigates Service, DNS, Ingress, and connectivity issues. ## Common Network Issues | Symptom | Likely Cause | First Check | |---------|-------------|-------------| | Service unreachable | No endpoints, selector mismatch | Endpoints exist | | DNS not resolving | CoreDNS issue, wrong service name | DNS from inside pod | | Ingress not routing | Missing backend, TLS issue | Ingress + Service config | | Connection refused | Pod not listening, wrong port | Target port matches | | Connection timeout | NetworkPolicy blocking | NetworkPolicy rules | ## Investigation Workflow ### Step 1: Verify Service and Endpoints ```bash # Check service exists kubectl get svc -n # Check endpoints (should list pod IPs) kubectl get endpoints -n # Detailed service info kubectl describe svc -n ``` **No endpoints?** Check: - Service selector matches pod labels - Pods are Running and Ready ```bash # Compare service selector with pod labels kubectl get svc -n -o jsonpath='{.spec.selector}' kubectl get pods -n --show-labels ``` ### Step 2: Test DNS Resolution ```bash # From inside a pod (use any running pod) kubectl exec -it -n -- nslookup kubectl exec -it -n -- nslookup ..svc.cluster.local # Check CoreDNS is running kubectl get pods -n kube-system -l k8s-app=kube-dns ``` DNS format: `..svc.cluster.local` ### Step 3: Test Connectivity ```bash # From inside a pod, test connection kubectl exec -it -n -- wget -qO- --timeout=5 http:\/\/:\/ kubectl exec -it -n -- nc -zv # Or using curl if available kubectl exec -it -n -- curl -s --max-time 5 http:\/\/:\/ ``` ### Step 4: Check NetworkPolicy ```bash # List NetworkPolicies kubectl get networkpolicy -n # Check policy details kubectl describe networkpolicy -n ``` NetworkPolicy can block: - Ingress (incoming to pod) - Egress (outgoing from pod) ## Specific Issues ### Service Has No Endpoints ```bash # Check if pods match selector kubectl get svc -n -o jsonpath='{.spec.selector}' # Find pods with those labels kubectl get pods -n -l = # Check if pods are Ready kubectl get pods -n -o wide ``` ### Ingress Not Working ```bash # Check ingress config kubectl get ingress -n kubectl describe ingress -n # Check ingress controller logs kubectl logs -n -l app.kubernetes.io\/name=ingress-nginx --tail=50 ``` Common ingress issues: - Backend service doesn't exist - Service port mismatch - TLS secret missing - Ingress class not specified ```bash # Check ingress class kubectl get ingressclass ``` ### Port Mismatch ```bash # Service targetPort must match container port kubectl get svc -n -o jsonpath='{.spec.ports[*].targetPort}' kubectl get pod -n -o jsonpath='{.spec.containers[*].ports[*].containerPort}' ``` ### Pod Not Listening ```bash # Check what ports the container exposes kubectl get pod -n -o jsonpath='{.spec.containers[*].ports}' # Check if process is listening inside pod kubectl exec -it -n -- netstat -tlnp 2>\/dev\/null || kubectl exec -it -n -- ss -tlnp ``` ## Quick Debug Commands ```bash # Full service + endpoints overview kubectl get svc,ep -n # Check all ingresses kubectl get ingress -A # DNS debugging pod (if needed) kubectl run -it --rm debug --image=busybox --restart=Never -- nslookup ..svc.cluster.local ``` ## Notes - Load `analyzing-k8s-events` to check for network-related events - Load `debugging-k8s-pods` if the target pods are not healthy