Diagnosing Host-Level DNS and TCP Connectivity Failures

Hello. Last time, you traced a failed service start to a dependency, a pre-start command, or the main process using systemd state and logs from the same attempt. A service can also start successfully and then fail because it cannot resolve a name or establish a connection. This lesson applies the same evidence-first method to that different failure.

In about 40–45 minutes, you’ll isolate where a host-level connection attempt breaks: name resolution, local addressing or routing, a listening socket, or the network path. You’ll use ip, ss, dig, traceroute, and a short packet capture, while keeping each tool’s conclusions within its limits.

---

Define the failed attempt before testing the network

“Cannot connect” is a symptom, not yet a network diagnosis. Start with one source host, destination name, intended IP address, protocol, port, and time. Record the application’s exact error if available. A name-resolution error, Connection refused, and a connection timeout call for different investigations.

Keep the source and destination roles clear. On the client host, you can inspect its DNS configuration, selected route, outbound socket, and transmitted packets. On the server host, if you have access, you can check whether the service listens on the intended address and whether connection attempts arrive. A listening socket on the client does not tell you whether the remote server is listening.

For example, suppose an application on a Linux host must reach api.internal.example on TCP port 443. Treat the hostname and addresses in the commands below as illustrative placeholders, not destinations to test. Preserve the time of a failing application attempt so you can compare it with your command output and capture.

A useful investigation asks, in order:

1. Does the name produce the expected address through the resolver the host uses?
2. Does the host have a plausible interface, source address, and route to that address?
3. Is the relevant server listening, if you can inspect it?
4. During an actual attempt, which packets leave and which responses return?

The order prevents a common mistake: troubleshooting a TCP path to an IP that the application never obtained.

---

Inspect the host’s address and chosen route with ip

ip reads the host’s interface and routing state without changing it. Begin on the client:

ip -br link
ip -br addr
ip route show

Look for an interface that is up, an appropriate address on it, and routes relevant to the destination. A default route may carry traffic outside directly connected networks, but its presence does not prove the chosen gateway or the rest of the path works.

This short segment of How to Use the ip Command in Linux: A Beginner’s Guide, from Learn Linux TV, shows how to read a routing table and recognize a default gateway. Focus on interpreting the output; this investigation does not require adding or deleting routes.

For a particular IP, ip route get is more focused than scanning the whole table:

ip route get 192.0.2.80

A result might name an outgoing interface (dev), a gateway (via), and a selected source address (src). Compare those fields with what this connection should use. An error such as Network is unreachable is strong evidence of a local route-selection problem. A plausible result means only that the kernel selected a route; it says nothing yet about successful delivery. On a multi-interface host, run this check for the DNS server’s IP as well as the service’s IP. The route to one may be correct while the route to the other is not.

Red Hat’s section “26.3. Routing DNS traffic through a certain interface” gives a concrete multi-interface example: a DNS server becomes unreachable because traffic takes the wrong default gateway. Read the scenario and its ip route get verification, not the configuration-change procedure.

In production, collect this evidence before changing an interface or route. Disabling an interface or replacing a default route during diagnosis can disconnect your own session and erase the original failure condition.

---

Separate DNS answers from DNS transport failures

First inspect which resolver the host is configured to use:

cat /etc/resolv.conf
resolvectl status

resolvectl is useful on hosts using systemd-resolved; it may not be installed or manage DNS on every host. A loopback nameserver such as 127.0.0.53 usually points to a local resolver stub, not directly to the upstream DNS server. Check that resolver’s configuration before concluding that the loopback address is wrong.

Query the relevant name and record the full dig output:

dig +time=2 +tries=1 api.internal.example A
dig +time=2 +tries=1 api.internal.example AAAA

The answer, response status, responding SERVER, and any timeout matter. An A query tests IPv4 addresses; an AAAA query tests IPv6 addresses. If the application may select IPv6, testing only A can miss its failing path.

To distinguish a resolver-specific problem from a general inability to query DNS, compare the configured resolver with another resolver authorized to answer that name:

dig @192.0.2.53 +time=2 +tries=1 api.internal.example A

Here 192.0.2.53 stands for a resolver you have identified, not an address to use literally. A public resolver is useful for an appropriate public name, but it generally cannot answer a private internal name. Cloudflare’s Troubleshooting DNS Resolver guide demonstrates comparing explicit resolvers for a public-domain query.

Interpret the result precisely:

• A timeout means this query did not receive a usable reply in time. It does not, alone, tell you whether the request was never sent, the resolver was unreachable, or the reply was lost.
• NXDOMAIN is a DNS response saying the queried name does not exist from that resolver’s perspective. It is not a TCP timeout.
• SERVFAIL means a resolver returned a failure to complete the lookup; investigate that resolver or its dependencies rather than treating the answer as an absent name.
• NOERROR with no address answer is not the same as NXDOMAIN. Check the requested record type and the rest of the response.

An explicit dig @resolver also bypasses the host’s normal resolver selection. It is an isolation test, not proof that the application uses that resolver. Likewise, dig does not fully reproduce an application’s name-resolution behavior: applications may use /etc/hosts, search domains, caches, or a different resolver path. If the configured lookup fails but an explicit query succeeds, inspect that difference rather than declaring DNS healthy.

Where the host uses a local DNS stub, check for its socket:

ss -lunp | grep ':53'
ss -ltnp | grep ':53'

These look for UDP and TCP port 53 respectively. A stub configured at 127.0.0.53 should have a relevant local socket; a missing socket is a lead to investigate the resolver service and its logs. Process details from -p may require elevated privileges. A present socket proves that something is bound, not that it can answer or reach its upstream resolver.

---

Test the intended TCP port and inspect socket state

Once you have an intended destination IP, test that IP and port, separately from the hostname. For an authorized destination, a short connect probe can make the distinction concrete:

nc -vz -w 3 192.0.2.80 443

Substitute the real destination. A successful TCP connect establishes that the handshake completed; it does not prove that TLS, HTTP, authentication, or the application request will succeed. If the probe succeeds by IP but the application fails by name, revisit DNS answers and which address family the application selected.

If you control the destination host, inspect its listener there:

ss -ltnp

Find the expected port and read the local address as well as the state. A service listening only on 127.0.0.1:443 accepts local connections, not connections addressed to the server’s network interface. A LISTEN entry on the intended address is useful evidence, but it still does not prove that a remote client can reach it.

On the client, inspect sockets while reproducing the failure:

ss -tanp

A connection remaining in SYN-SENT shows that the client has begun a TCP handshake without completing it at that instant. It does not locate the loss by itself. No matching socket in a single snapshot is also inconclusive: the attempt may have ended between observations, or the application may not have reached the TCP stage.

Red Hat Enterprise Linux’s How to Use SS to Monitor Listening Ports on Your System demonstrates the numeric and process flags. Numeric output is particularly helpful when DNS itself is suspect: the display will not depend on resolving socket addresses.

The error wording offers a hypothesis, not a verdict. Connection refused commonly corresponds to an active rejection, such as a TCP reset when no suitable listener exists; a firewall can also reject. A timeout suggests no handshake completion, but could involve filtering, a lost response, or another path failure. Packet evidence can narrow these possibilities.

---

Use traceroute for path clues, then capture the actual attempt

traceroute sends probes with increasing hop limits and displays routers that answer them. Use a numeric destination so name resolution does not obscure the result:

traceroute -n 192.0.2.80

The default probe protocol is not necessarily the application’s TCP traffic. If supported and permitted in your environment, a TCP-mode trace to the intended port can be more relevant:

traceroute -n -T -p 443 192.0.2.80

Neither trace proves that the application connection succeeds. Routers may decline to answer traceroute probes while forwarding ordinary traffic. Compare the trace with ip route get and with a real connection attempt.

In the image, hop 2 has only * entries, yet later hops respond. That alone rules out interpreting hop 2’s silence as proof that forwarding stopped there. A single unusually high time at hop 3 is also insufficient to establish sustained latency. Near the bottom, repeated * entries leave the destination’s status unresolved in this excerpt.

When routing and socket checks still leave uncertainty, capture a brief, targeted reproduction. On the client, start a capture before triggering one connection attempt:

sudo tcpdump -ni any 'host 192.0.2.80 and tcp port 443'

Stop it with Ctrl-C after the attempt. -n avoids name lookups in the display; any helps avoid choosing the wrong interface for an initial host-level capture. Replace the placeholder address and port, and keep captures short: even targeted network data can contain sensitive metadata. If you need to prove which interface carried traffic, repeat on the interface selected by ip route get.

Read the TCP flags as an exchange, not as isolated lines:

• Repeated outbound SYN, no visible response: the client sent connection attempts, but this capture did not see a reply. Check the selected route and the path; you cannot yet assign blame to a particular firewall, router, or server.
• Outbound SYN, inbound RST: something actively rejected the attempt. Identify the packet’s apparent source and investigate the destination listener or rejection policy; a source address alone may not identify every intermediate device.
• SYN, SYN-ACK, then ACK: the TCP handshake completed at the observed host. If the application still fails, examine what happens after connection establishment rather than calling it a TCP reachability failure.

For a DNS timeout, capture the DNS attempt instead:

sudo tcpdump -ni any 'port 53'

Compare the observed destination and reply with the resolver configuration and your dig result. A query leaving for an unexpected resolver suggests a resolver-selection issue. A query leaving for the expected resolver with no visible reply narrows the question to delivery or response, but does not pinpoint where it failed. If the host answers from a cache, there may be no upstream query to capture. Encrypted DNS or an application using another resolution mechanism may not appear as port-53 traffic.

A capture’s absence needs care: confirm that the attempt occurred during the capture, that you observed the correct host or interface, and that the filter matched the actual IP and protocol. If you have access to both endpoints, captures from both sides are especially useful: a client’s outbound SYN that never appears at the server narrows the problem to the intervening path; a SYN arriving at the server changes what you investigate next.

---

Turn observations into a bounded diagnosis

Suppose the application reports a timeout to api.internal.example:443. A defensible report would preserve the chain of evidence:

At the time of failure, the host’s configured lookup returned the expected IPv4 address. ip route get selected the expected source address and interface. The client entered SYN-SENT, and a capture on that host showed repeated SYN packets to the resolved IP and port with no response visible. The failure is past DNS resolution and TCP initiation, but this host-only evidence does not distinguish a downstream drop from an absent or lost server reply.

That conclusion is more useful than “the network is down”: it identifies what worked, where evidence stops, and what to collect next. A different record—NXDOMAIN from the configured resolver but a valid answer from an appropriate alternate resolver—would focus the investigation on resolver choice or resolver behavior before TCP. If a server-side ss check shows the service bound only to loopback, and its capture shows the connection attempt arriving for a non-loopback address, the listener configuration becomes a much stronger lead.

Keep proposed changes proportional to the evidence. A route that looks unusual is not a reason to replace it during an incident; a traceroute * is not a reason to declare a firewall broken. Record the observation, the inference it supports, and the next check that could disprove that inference.

---

Key takeaways

Use dig to establish what answer and resolver are involved; ip to inspect local addressing and route selection; ss to inspect listeners and live TCP state; traceroute for limited path clues; and tcpdump to see whether the relevant DNS query or TCP packets actually traverse the observed host. No single tool proves an end-to-end cause. Correlate them around the same destination and attempt, and state where your evidence ends.

Next, the course turns from host connectivity to Linux namespaces, cgroups, and capabilities—the host mechanisms that provide container isolation.