Additional/alias IP for NIC team

Request:

I need a way to specify an additional IP address for a NIC team. For example, if a two NIC team is created and a base IP address of 10.1.1.10 is assigned to it, I need a way to add a second IP address to that team such as 10.1.1.11. Basically a GUI/persistent way of doing "ifconfig bond0 add 10.1.1.12 netmask 255.255.255.0". This would result in better utilization of networking when using NFS.

Details:

I have been trying to figure out the best way to configure networking for my ReadyNAS as an NFS datastore for a VMware host cluster. From everything I have read, I am seeing that on the ESXi hosts, I should create a single vSwitch with 2 physical NICs in an IP Hash team (etherchannel). I will then create a VMkernel port and give it an IP address in the NFS storage network. On the ReadyNAS, I should create an 802.3ad/LACP team. Upon close inspection though I see that this is not an optimal configuration.

Scenario:

Let's say I have two ESXi hosts which will use a ReadyNAS 2100 as a shared NFS datastore. Each host has two physical 1GB NICs assigned to a vSwitch using IP Hash for the teaming with a VMkernel port dedicated to NFS storage. The NFS storage network is 10.1.1.0/24. Host1 is 10.1.1.1 and Host2 is 10.1.1.2. On the ReadyNAS, I have created an 802.3ad team with the two onboard NICs and assigned the team an IP address of 10.1.1.11. They all connect to the same switch (or in real life a Cisco Catalyst 3750 stack that allows cross stack etherchannel) and all teams are working appropriately with IP based load balancing.

Now, on Host1 I add my NFS datastore from the ReadyNAS 2100. I'll call it VMStore1. On Host2 I add the same datastore. Each can see the datastore, work with it, and things seem fine.

If I look more closely though, I would see that Host1 and Host2 are only using one of their two NICs dedicated to NFS. This is because NFS is not MPIO aware. The IP Hash mechanism sees Host1's IP address of 10.1.1.1 and the NFS target address of 10.1.1.11 and makes that traffic go over one of the NICs. The same thing happens on Host2. This essentially means that either host would only ever be able to utilize 1GB of througput to the ReadyNAS.

If we look at the ReadyNAS in this configuration, however, it would have traffic for Host1 going out one NIC in the team, and traffic for Host2 going out the other NIC in the team. This is pretty much expected, but still not ideal. While it does balance the load, it too would be limited to 1GB of throughput to each host. If we only had Host1, then 1 interface on both the host and the ReadyNAS would be wasted.

One way to work around this would be to break the team on the ReadyNAS and assign an IP address to each interface. This is not an ideal option however because it causes a loss of redundancy.

The best solution I have come up with is based on recommendations by NetApp, EMC, and other storage vendors. They allow "alias" IP addresses to be assigned to the teamed interface. In our case, we would put both 10.1.1.11 and 10.1.1.12 on the ReadyNAS's team interface. The next step is to create a second NFS store (VMStore2) and map it to the ESXi hosts. When using these datastores on the hosts, the administrator can manually load balance VM's across the two datastores. This should allow potentially 1GB of throughput to VMStore1 and 1GB of throughput to VMStore2.

I have confirmed that this works by going in through SSH to the ReadyNAS and issuing "ifconfig bond0 add 10.1.1.12 netmask 255.255.255.0".

Here are some references I looked through while researching this (most are from NetApp but principles still apply for NIC teams and NFS):

http://communities.netapp.com/blogs/eth ... ival-guide

http://blogs.netapp.com/virtualstorageg ... mware.html
http://blog.scottlowe.org/2008/07/16/un ... mware-esx/

Thank you!

John Yarborough