Networking help

[bscott at ntisys.com]

On Mon, 25 Nov 2002, at 3:29pm, pll at lanminds.com wrote:
> However, by ssh'ing to systemB, and from there to systemC, I run 'tcpdump
> -i eth1 icmp' and I can see that systemC *is* in fact receiving the "icmp
> echo request" packets.  systemC just isn't replying to them!

  That is significant.

  There are two possibilities.  One: The system is receiving those packets,
but thinks it should not reply to them.  Two: The system is replying, but
you are not seeing the replies.

  You assert systemC is not replying to them, based on tcpdump.  That's
normally a pretty good indication, but you've already demonstrated this is
not a normal problem.

  Just to satisfy paranoia, use another system to sniff the Ethernet that
"system C" is plugged into.  If "system C" is plugged into a switch, add a
repeater for "system C" and the sniffer.

  Is the system multi-homed?  If so, is there any chance it is sending the
packets out the wrong interface?

  You said these systems are running Linux, right?  Create firewall rules
that log ICMP packets but don't specify a jump target.  That will show you
what the kernel router *thinks* is going on.  This has the added benefit of
not watching a particular interface.

  Assuming you find no evidence of replies going into a blackhole, that
means the system must not be replying for some reason.  Why?

  If "system C" thinks the packet is for a different host, the packet will
be dropped.  Since it sometimes *does* reply, I cannot see how this could be
the cause.

  Firewall rules could do this, but you've already checked that.  What about
ICMP rate limiting?  Does the kernel do anything like that outside of
IPTABLES?  I don't think it does, but maybe?

  If the packet's checksum is bad, the packet will be silently dropped.  If
it has a header somewhere that is somehow bad, it may be silently dropped.  
Is something somewhere (router) corrupting the packets?  Try dumping the
full packet structure.  Ethereal (or tethereal) is great for this.  Look for
any reason the packet might be considered invalid.

  If the system thinks the packet is part of an incomplete fragment, it will
hold it in memory for reassembly.  According to your tcpdump output, the DF
(Don't Fragment) bit is set... is a router somewhere ignoring that bit?

-- 
Ben Scott <bscott at ntisys.com>
| The opinions expressed in this message are those of the author and do not |
| necessarily represent the views or policy of any other person, entity or  |
| organization.  All information is provided without warranty of any kind.  |