I have a 2 datacenters composed of a Load balancer and a switch. Each location has it own internet connection/ gateway.
I have a LANX connection between both location to stack my S3300.
Servers on each side have their location's gateway.
We have an asymetric routing issue, eg:
If server 192.168.3.10 tries to talk to 192.168.6.20, it won't work since the path will be 3.10 > LB01 > 6.20 > LB02 > 3.10
We are trying to find a solution to allow this architecture to work. I was thinking about configuring static route on the S3300, and add route for each servers so it won't go up to the LB to do the routing.
eg, on SW1: route 192.168.6.20 192.168.3.2, so if 3.10 tries to talk to 6.20, it won't go up to its default gateway 3.1
But I don't know how to do that, i've tried adding a VLAN, enabling VLAN routing but no luck. Is that possible ? What are your recommendation ?
The problem is that you can create all the static routes you want on the switch-stack, but it won't matter as the servers will never send traffic to the switches, for routing purposes. The servers will use their default gateway - the LBs.
You have two choices here:
1. Change the routing design and let the switch-stack do the routing internally (servers will have the switch-stack as default gateway). That would fix this, but you would have an issue with the two LB gateways after the switch-stack. This model switch can't do more than one default gateway in its routing table. You would need something like policy based routing on the switch-stack. So, this option is probably not good for you.
2. Don't add any routes on the switch-stack, but instead simply add IP addresses to the VLAN interfaces on your switch-stack (I assume you have two VLANs based on the diagram?). Then do static routes on the servers instead.
Example
VLAN 3 Overview: Subnet: 192.168.3.x Server 1: 192.168.3.10 Server 2: 192.168.3.20 VLAN 3 IP address on the switch-stack: 192.168.3.254 (could be any IP in the subnet not, already in use) Gateway: 192.168.3.1/192.168.3.2 (The LBs)
VLAN 6 Overview: Subnet: 192.168.6.x Server 1: 192.168.6.10 Server 2: 192.168.6.20 VLAN 6 IP address on the switch-stack: 192.168.6.254 (could be any IP in the subnet, not already in use) Gateway: 192.168.6.1/192.168.6.2 (The LBs)
On the servers in VLAN 3 you simply add a static routes to the subnet of VLAN 6. Destination Network: 192.168.6.0 Subnet mask: 255.255.255.0 Gateway/next hop: 192.168.3.254 (switch-stack VLAN 3 IP address)
Same story for the servers in VLAN 6. On those servers you just add a static route to the VLAN 3 subnet. Destination Network: 192.168.3.0 Subnet mask: 255.255.255.0 Gateway/next hop: 192.168.6.254 (switch-stack VLAN 6 IP address)
This way the switch-stack would do routing for the servers, when they want to communicate internally. And for everything else (Internet, etc.), the servers will go to they default gateway (the LBs). That will work fine. Remember, you would need to make sure routing is also enabled on the switch-stack: "Routing" --> "IP" --> Enable "Routing Mode".
I hope that clarifies it for you. Else let me know.
Hi Hopchen, thx for the reply. I do understand the solution where my server will have the switch are their gateway for internal routing.
The switch-stack is seen as only 1 equipement, so just 1 default gateway. Do I configure gateway in Route > Routing Table > Configure Route > DefaultRoute ?
How would the switch know that traffic from servers 3.10 will go to 3.1, or 3.20 will go to 3.2 ?
I assume you are referring to my suggestion number 2? In that case, I have answered your questions below.
"Do I configure gateway in Route > Routing Table > Configure Route > DefaultRoute" No need to do any additional routes on the switch-stack essentially. The switch-stack will have two routing duties. Route traffic from the 192.168.3.x network to the 192.168.6.x network and vice versa. You will create VLAN interfaces in each VLAN (3 and 6) to accomplish this. You don't need to make routes to tell the switch-stack where networks 192.168.3.x and 192.168.6.x are. The switch-stack knows already. It has interfaces in those networks 🙂
"How would the switch-stack know that traffic from servers 3.10 will go to 3.1, or 3.20 will go to 3.2?" That is not how it would work :). The end solution would work like this: - If server in VLAN/network 3 or 6 wants to communicate with a server in the other network, then the request is sent to the switch-stack (defined by the static routes you created on those servers). The packet is routed to the other VLAN/network solely by the switch-stack. Your LBs will neversee that traffic. - If a server in VLAN/network 3 or 6 wants to communicate with anything else (Internet, etc.) the request is sent to the server's default gateway, which will be the LBs. The switch-stack is not involved here at all, from a routing point of view.
Note: I can't see your pictures/screenshots yet, so I reply based on your written text.
As you had two subsets, I (as I wrote) assumed that you used VLANs already. It seems you weren't? In that case, what was your setup? I guess you used just default VLAN 1 and had 2 IP networks?
In any case, I wish you had asked before messing with the VLANs as we could have advise you better. The config you describe shows that you aren't used to VLANs and that is fine, but please ask first then. You are running on smart switches so there is no console ports here. If you locked yourself out, there is nothing you can do but reset the stack!
Now, that being said - you won't loose access to a switch just because you untag multiple VLANs on a port. Did you ever change the PVIDs? Did you remove the VLAN 1 association on the ports? If no, then you can access the switch-stack through the VLAN 1 interface (the original IP the switch-stack had), by simply plugging a PC into any port. Ensure your PC has an IP address in the same range as the switch-stack.
As a side note. I understand why you don't want to mess with the LBs so you aren't a fan of the VLANs. However, I don't see a way to do this without the VLANs though - if you want the switch to do some internal routing. But, if you weren't using VLANs in the first place then why have two IP networks? Those networks are on the same layer 2 so there is no effective separation anyway. Why not just run one IP network then and be done with it?
We have multiple subnet because it was easier to segregate our whole network, webserver, databases, etc and have iptables based on subnet. Anyway, everything is like that now, it's too late the change this.
What I remember doing is select VLAN 1, then select Untagged all and apply, and do the same for the VLAN 3 and 4. Only VLAN 4 was working, I could ping 4.254 from a server in the same subnet. What I did then was a "remove all" on VLAN 4, that's when I lost the connectivity. I don't even know what is PVID.
My load balancer is connected to my switch with a 10G link, and the interface is set to untagged.
From a VLAN 4 point of view you killed the access since you removed VLAN 4 from all ports. This is why the server in that network can't communicate to the switch-stack anymore. Still that should leave you able to access the switch via VLAN 1 as all ports are untagged for VLAN 1 still and have a PVID of 1.
The PVID and untag combination allows for non VLAN devices (like PCs) to be associated with a VLAN. So, as your ports on the switch-stack are all untagged for VLAN 1 (since you didn't change that) and all ports have PVID 1 (since you didn't change that either) - this means that if you plug a laptop into any free port on the switch-stack you should be able to access it. You need to access it via the switch's VLAN 1 IP address then and your PC must have an IP in the same subnet. From there, get access and reverse whatever config you did so you are back to where you were.
Do you know what the VLAN 1 IP address of the switch-stack is? What IP did you use to access it on before?
I don't mean to give you a lecture. It is your network and I understand that changing it might not be what you or the company wants. Maybe you just inherited the network as it is, who knows. The only thing I would point out is that segregation of a network happens on layer 2 (VLANs), and not on layer 3 (IP scemes). The IP schemes later help to route between VLANs, etc. but still the segregation is on layer 2.
Anywho, here is what I think you should do: 1. Get access to the switch-stack and revert all the config back to how it was before. 2. Take a think about how you wanna do this. If you want the switch to be involved in the internal routing then you are probably looking at doing VLANs. Research before implementing. 3. If you don't want VLANs, then you need to figure how to get this working using the LBs.
