Kubernetes DNS Troubleshooting Guide: A Step-by-Step Workflow for Service Resolution Failures
KubernetesCoreDNSCloud OperationsTroubleshootingNetworking

Kubernetes DNS Troubleshooting Guide: A Step-by-Step Workflow for Service Resolution Failures

CChallenges.pro Editorial Team
2026-11-14
6 min read

A reusable Kubernetes DNS troubleshooting checklist for CoreDNS, Services, namespaces, NetworkPolicies, upstream resolvers, and pod configuration.

Kubernetes DNS failures can look like application bugs, network outages, or intermittent cloud problems. This guide provides a reusable, evidence-first workflow for determining whether the fault is in the pod configuration, Service records, CoreDNS, network policy, kube-proxy behavior, or an upstream resolver.

Overview

A Kubernetes pod may need to resolve two broad types of names: cluster services such as payments.api.svc.cluster.local, and external names such as example.com. These paths overlap but are not identical. A service lookup normally depends on the Kubernetes Service object, the DNS records served by CoreDNS, and pod-to-DNS network connectivity. An external lookup also depends on CoreDNS forwarding configuration and the resolver reachable from the cluster.

Start by defining the exact failure. Record the name being queried, the namespace of the calling pod, whether the problem affects one pod or many, and whether the result is a timeout, NXDOMAIN, SERVFAIL, or an unexpected address. A timeout often points toward connectivity, policy, or an unhealthy resolver. NXDOMAIN can indicate a wrong name, namespace, or missing Service. SERVFAIL requires closer inspection of CoreDNS and its upstream path, but these signals are clues rather than proof.

Use a temporary diagnostic pod in the same namespace and, where possible, with the same network policies as the affected workload. Avoid testing only from an administrator workstation: a name that resolves outside the cluster may still fail from the application pod.

Checklist by scenario

1. A pod cannot resolve a Kubernetes Service

  1. Confirm that the Service exists in the expected namespace: kubectl get svc -n <namespace>.
  2. Check its selector and endpoints: kubectl describe svc <service> -n <namespace> and kubectl get endpointslice -n <namespace> -l kubernetes.io/service-name=<service>.
  3. Test the fully qualified name, for example <service>.<namespace>.svc.cluster.local. This avoids ambiguity caused by search domains.
  4. Test the short name from the same namespace. If the fully qualified name works but the short name fails, inspect the pod's search list and resolver configuration.
  5. Verify that the application is using the intended port. DNS resolution can succeed even when the Service target port or application listener is wrong.

A Service without ready endpoints may still have a DNS record, depending on its configuration, so do not treat successful resolution as proof that traffic will reach a healthy workload.

2. A pod cannot resolve an external hostname

  1. Compare an external lookup with a cluster Service lookup. This separates general pod-to-DNS problems from upstream forwarding problems.
  2. Inspect CoreDNS logs for forwarding errors, timeouts, or repeated failures: kubectl logs -n kube-system -l k8s-app=kube-dns. The label can vary by installation, so identify the actual CoreDNS pods first.
  3. Review the CoreDNS configuration: kubectl -n kube-system get configmap coredns -o yaml. Pay particular attention to the forward or proxy behavior and any custom zones.
  4. Check whether the configured upstream resolver is reachable from the node and pod networks. Cloud environments may provide a platform-specific resolver address or require a particular VPC or virtual network configuration.
  5. Repeat the lookup several times if the issue is intermittent. Capture timestamps and the queried names rather than relying on a single successful attempt.

3. CoreDNS appears unhealthy

  1. Check pod state, restarts, readiness, and scheduling: kubectl get pods -n kube-system -l k8s-app=kube-dns -o wide.
  2. Inspect events and recent logs: kubectl describe pod <coredns-pod> -n kube-system.
  3. Verify that the DNS Service has endpoints: kubectl get svc,endpointslice -n kube-system.
  4. Check resource pressure and node placement. CPU starvation, memory pressure, eviction, or a scheduling constraint can produce symptoms that resemble a DNS configuration error.
  5. Review recent changes to the CoreDNS ConfigMap, cluster version, CNI, node resolver settings, and admission or security controls before changing configuration.

4. DNS works in one namespace or pod but not another

Compare the working and failing pods directly. Inspect dnsPolicy, dnsConfig, host networking, namespace labels, and NetworkPolicies. A policy may block UDP or TCP traffic to the cluster DNS Service. Some environments also apply namespace-specific egress controls or sidecar behavior that changes DNS handling.

5. DNS succeeds but the connection still fails

Move beyond DNS. Confirm the returned address, Service endpoints, routing, kube-proxy or the configured service-proxy implementation, CNI health, and NetworkPolicy rules. A successful lookup only proves that a name was mapped to an address; it does not prove that packets can reach the destination or that the destination accepts the connection. For policy examples, see the Kubernetes NetworkPolicy guide.

What to double-check

Pod-level resolver settings

From the affected pod, inspect /etc/resolv.conf. Record the nameserver, search domains, and resolver options. Kubernetes commonly supplies a cluster DNS nameserver, but hostNetwork, an explicit dnsPolicy, or custom dnsConfig can alter the result. Long search lists can also cause unexpected queries or delays. Test the complete name first, then the short name.

Diagnostic tools and commands

If the application image lacks troubleshooting tools, use an approved temporary image in the same namespace. A minimal test sequence is:

kubectl run dns-debug --rm -it --restart=Never \
  --image=<approved-diagnostic-image> -- sh

cat /etc/resolv.conf
nslookup <service>.<namespace>.svc.cluster.local
nslookup example.com
dig <service>.<namespace>.svc.cluster.local
dig example.com

Diagnostic image names and package availability vary by organization and base image. Use an image permitted by your supply-chain and runtime policies rather than copying an unapproved image into production.

Evidence to capture

  • Cluster version, node pool or platform, CNI, and the time window of the failure.
  • Calling namespace, pod name, node name, and pod IP.
  • Exact query name, query type, response code, returned address, and whether the result was intermittent.
  • Pod /etc/resolv.conf, dnsPolicy, and relevant dnsConfig.
  • CoreDNS pod status, restarts, logs, Service endpoints, and current configuration.
  • Recent changes to NetworkPolicies, routes, node images, kube-proxy, CNI, CoreDNS, or cloud networking.

Common mistakes

  • Testing from the wrong location: A laptop or node may use a different resolver path from the failing pod.
  • Assuming every failure is CoreDNS: Check names, namespaces, search domains, policies, and endpoint data before restarting DNS.
  • Using an incomplete Service name: A short name is namespace-relative. Use the fully qualified name when comparing results.
  • Checking only UDP: DNS commonly uses UDP, but TCP can also be involved. A policy that permits one protocol but blocks the other can create confusing failures.
  • Changing configuration before collecting evidence: Restarts may hide the original symptom and make intermittent problems harder to correlate.
  • Confusing DNS with application reachability: Once a name resolves, test the relevant port and protocol separately.
  • Ignoring version and platform differences: CoreDNS deployment labels, kube-proxy behavior, CNI capabilities, and cloud-provider resolver details vary. Verify assumptions against the cluster's current implementation.

When to revisit

Keep this checklist with the cluster's operational runbooks and revisit it before planned upgrade cycles, after changing CoreDNS or CNI settings, when NetworkPolicies are introduced, and when node images or cloud networking are modified. It is also useful during recurring cluster health reviews and whenever a deployment reports intermittent name-resolution errors.

After an upgrade, revalidate the diagnostic image, CoreDNS labels and configuration, kube-proxy or service-proxy behavior, and cloud-provider DNS assumptions. If the cluster uses custom forwarding rules, document the expected upstream resolvers and ownership so future responders know where to look. Record a known-good lookup from a representative namespace and workload type, then repeat it after material changes.

For a practical incident workflow, preserve the evidence above before remediation, state the narrowest confirmed failure, and change one layer at a time. If the DNS issue is part of a broader deployment problem, pair this guide with the Kubernetes troubleshooting decision tree and review relevant SLO signals in the SLO burn-rate alert checklist. The goal is not simply to restart CoreDNS; it is to identify the failing path and leave behind enough evidence to prevent a repeat investigation.

Related Topics

#Kubernetes#CoreDNS#Cloud Operations#Troubleshooting#Networking
C

Challenges.pro Editorial Team

DevOps and Platform Engineering Editors

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.