NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
Solved
DHCP Response not traversing stacked switches
Hi, hopefully only another user error, but at the moment I am out of ideas again. I have 2 S3300's (FW 6.6.17) [A1,A2] connected via stacking in 2 different rooms. On one of the 10G Ports another ...
Alright, that is good! Thanks for clarifying.
I think we will focus on one direction for now (not the music band :smileylol:). Since your Windows server will be the DHCP server in the end, we can use that one and leave the Linux one turned off/disconnected for now.
I am not sure where the Windows server is connected, but let's just assume that it is connected to A1 in the stack and your client is connected to switch B. Connection flow is like this then:
DHCP server --> A1-stack --> A2-stack --> Switch B --> DHCP client.
You have already done some good troubleshooting with Wireshark. From my understanding, you have port mirrored and Wiresharked the link between A2-stack and switch B?
- You see the DHCP discover send from the client?
- You also see the DHCP offer from the server?
- Hereafter, you see nothing further?
Just to clarify how DHCP works:
1. Client sends a DHCP discover
2. Server sends a DHCP offer, offering an IP address to the client
3. Client checks that the offered address is not in use by anyone else (gratuitous ARP)
a. If the offered address is in use by someone else, the client won't use the offered address and sends a DHCP decline message to the server. Hereafter the client starts the DHCP discover process all over again.b. If the offered address is not use then the client will accept the address, by sending a DHCP request to the server. A request to obtain the offered address.
4. The server will acknowledge to the client, with a DHCP ACK package, and then the server will register the DHCP entry in its DHCP binding table.
So, we gotta find out exactly where this does wrong. That will be important.
Let's try this:
- Port mirror the uplink between A2-stack and switch B, as you have done already. Run Wireshark to capture the traffic on that uplink.
- Packet capture the server as well, at the same time.
- Packet capture the client as well, at the same time.
So, we have 3 packet captures running now.
- Now, connect the DHCP client to switch B and see what happens. Filter the Wireshark captures by: bootp. This is so that you only see DHCP traffic.
What happens exactly?
The client sends the discover. The discover reaches the server?
The server replies with and offer? If so, does the offer reach the client?
etc.
As you have 3 Wiresharks running, you can see exactly where the problem lies. For example:
If the server sees the DHCP discover and sends the offer and the client sees the offer but never sends a request --> then we need to look into why the client didn't send the request.
or
If the server sees the DHCP discover and sends the offer, and you see the offer being forwarded on the link between A2-stack and switch B, but the client never sees the offer --> then we have to investigate why the offer was not forwarded to the client, by switch B.
etc.
I hope that makes sense? It is a bit of work, but this will give you a much better picture, because until we have the full picture we can only make educated guesses :)
Any questions, give me a shout!
Cheers
The usual mode of operation is only the AD DHCP to be active, the linux one is for troubleshooting only:)
But of course the two are not active at the same time:)
Also, this is all on VLAN 1 i.e. native/untagged traffic
Alright, that is good! Thanks for clarifying.
I think we will focus on one direction for now (not the music band :smileylol:). Since your Windows server will be the DHCP server in the end, we can use that one and leave the Linux one turned off/disconnected for now.
I am not sure where the Windows server is connected, but let's just assume that it is connected to A1 in the stack and your client is connected to switch B. Connection flow is like this then:
DHCP server --> A1-stack --> A2-stack --> Switch B --> DHCP client.
You have already done some good troubleshooting with Wireshark. From my understanding, you have port mirrored and Wiresharked the link between A2-stack and switch B?
- You see the DHCP discover send from the client?
- You also see the DHCP offer from the server?
- Hereafter, you see nothing further?
Just to clarify how DHCP works:
1. Client sends a DHCP discover
2. Server sends a DHCP offer, offering an IP address to the client
3. Client checks that the offered address is not in use by anyone else (gratuitous ARP)
a. If the offered address is in use by someone else, the client won't use the offered address and sends a DHCP decline message to the server. Hereafter the client starts the DHCP discover process all over again.b. If the offered address is not use then the client will accept the address, by sending a DHCP request to the server. A request to obtain the offered address.
4. The server will acknowledge to the client, with a DHCP ACK package, and then the server will register the DHCP entry in its DHCP binding table.
So, we gotta find out exactly where this does wrong. That will be important.
Let's try this:
- Port mirror the uplink between A2-stack and switch B, as you have done already. Run Wireshark to capture the traffic on that uplink.
- Packet capture the server as well, at the same time.
- Packet capture the client as well, at the same time.
So, we have 3 packet captures running now.
- Now, connect the DHCP client to switch B and see what happens. Filter the Wireshark captures by: bootp. This is so that you only see DHCP traffic.
What happens exactly?
The client sends the discover. The discover reaches the server?
The server replies with and offer? If so, does the offer reach the client?
etc.
As you have 3 Wiresharks running, you can see exactly where the problem lies. For example:
If the server sees the DHCP discover and sends the offer and the client sees the offer but never sends a request --> then we need to look into why the client didn't send the request.
or
If the server sees the DHCP discover and sends the offer, and you see the offer being forwarded on the link between A2-stack and switch B, but the client never sees the offer --> then we have to investigate why the offer was not forwarded to the client, by switch B.
etc.
I hope that makes sense? It is a bit of work, but this will give you a much better picture, because until we have the full picture we can only make educated guesses :)
Any questions, give me a shout!
Cheers
So it looks like I was mistaken - at the moment I receive no OFFERs on the Netgear switchport/uplink to switch B.
I thought this was the case yesterday when I opened this but maybe I confused it with packages the other way round.
I have been testing Client->A1->A2->B->Server now. Still see incoming/outgoing requests on Server, but mirror port on A2->B uplink dows not see offfers.
So looks like Switch B is not passing on (or A2 dropping) the packages. I checked obvious things like DHCP Snooping and discarded packages to see whether I could find out if its A2 or B but current indications point to B (as I can't find anything on A2).
B is a similar-to-Cisco switch without support so I'll have to dig around there.
Happy to get pointers but o/c not your job :)
Thanks for your help, I assume we can close this for now (or keep open for now in case I have to follow up, whatever you prefer)
Hi,
We, the community a whole, are always happy to give a helping wherever we can :) I understand that switch B is not a Netgear, but I think we can still double-check things to be sure of where the issue lies.
So, you are connected in this way now: Client--> A1-stack--> A2-stack--> switch B --> Server. You are saying that, on the server, you see the DHCP discover coming in and you also see the DHCP offer going out. However, on the uplink between A2-stack and switch B, you don't see the offer traverse back towards the client.
It could be a problem on switch B, but let's check a few things to be sure:
1. Please ensure that your port mirror of the link between A2-stack and switch B, is set to mirror traffic is both directions (RX and TX).
2. Is the port mirror working correctly? The DHCP discover is a broadcast so it will be seen by all, whereas the DHCP offer is a unicast. So, if the port mirror is not working correctly then you will still see the discover (all devices on the network sees those), but you won't see the offer as that is a unicast. To confirm that the port mirror is OK, send a ping or other unicast traffic across. Does the probe see those on the uplink? If yes, then the port mirror is OK.
3. You said that if the client and server are both on switch B, then it works. It is odd right? Why would it not work across the uplink, but indeed work if client and server are both on switch B? I am wondering if the address table is not being populated correctly on switch B, for some reason. When the DHCP discover has been sent from the client, switch B should pick up the client's mac address and that address should be linked to the uplink port that connects to A2-stack. How does the address table look, on switch B?
4. Switch B, is it in production? Any chance you can reset it? Is there a lot of config on it? Just to see if the same occurs on a factory defaulted unit, with no additional settings. Since you are running on VLAN 1, out of the box this should work on the switch as all switches use VLAN 1 per default.. Maybe even try a different switch, as a test, if you have another one.
One more thing. If a client does not get the offer message, it just send more discover messages. Can you confirm that you see this? i.e. client sends discover --> server sends offer (offer never reaches client) --> client re-sends the discover. This cycle should happen over and over until the client give up. Can you see that cycle in the pcap?
Cheers
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
