Firewalls are not Routers

I am no network super-genius but I do enough with networking to be able to get by. Two common mistakes I find many times are flat networks and firewalls as the default gateway. A flat network is when generally switches are connected to one another without any configuration. There is one broadcast domain which means every packet that the switch does not have an entry in the MAC address table, is sent out all the ports but the originating port. This repeats across all of the switches until the layer 2 destination is found. Now, this means your expensive Cisco switches are barely better than hubs. You don’t have collisions like you would on a hub and once the switch learns where the MAC address lives it keeps that information for a certain amount of time. Then again in this network setup the logs are most likely not monitored so if there where collisions and other errors it goes unnoticed.
That is not the title of this post though. Although related to a flat network using the firewall is a different issue. Using the firewall as the router works just fine when you have a flat network. You may never notice the problem in a small network, but as your network grew you noticed how problems can come up when there is just one big network. So someone smart said use vlans to segment the network, create smaller broadcast domains. Then when you try to fix or change the flat network with subnets and vlans can you find out the new vlans can not reach the rest of the original network.

media_1272596360227.png

The current flat network with switches and the firewall used as the default gateway or router.

media_1272597099867.png

The problem comes when you add subnets that are different than the interface ip of the firewall. Firewalls in general have issue with redirecting traffic bound for other networks back out of the same interface. So in the picture above traffic from vlan 1 that is using the firewall as the default gateway trying to reach the subnet on vlan 10. Since the host on vlan 1 does not know where that network lives it sends the traffic to the default gateway. Even if you added a static route to the firewall the traffic will often fail. That is because firewalls are not meant to route but rather send traffic between trusted and untrusted networks and vice-versa. So the question becomes how do you actually fix your flat network that has the firewall as the router. There is of course more complicated solutions to provide high availability using VRRP or HSRP.
First get a real layer 3 device. That is a router or a switch capable of routing between multiple vlans. The good news is many of your newer switches are capable of layer 3, it is included in many Dell and HP switches, it may still be an add-on with Cisco. I haven’t used a new switch in the last year that did not have layer 3.
Next important step is use the layer 3 device (switch or router) to route everything. Set a default route in the layer 3 device to send only outbound traffic to the firewall and bam everything works. Why is this so hard. Many times there is hundreds of servers and desktops already configured to use the firewall as their router. We will do a lot of work to avoid having to do a bunch of manual work.

media_1272597858840.png

Now you are using a router to route and the firewall to block bad things and maybe even do NAT. (note: If you are doing NAT be sure to add your new VLANs to your NAT rules so the new networks can reach the outside of your firewall.)

Using Network Load Balancing with View

If you have a smaller View deployment but still want to have redundant connection servers look no further than Microsoft NLB. Solve this problem without the need for an expensive hardware loadbalancer. Will it have all of the bells and whistles? No. If you have less than a 1000 users you probably would not see the benefit of the advanced features in a hardware load balancer. Make sure to read the whitepaper from VMware about NLB in Virtual Machines.

I am making the assumption you are like me and want everything to be as virtual as possible. So the View Connection Manager servers will be VM’s

Setup the primary and replica View Servers

I won’t go over installing View. Just be sure to setup the initial manager server. Then go ahead and setup the replica VM.

Configure NLB

media_1272294630960.png

Go the the Administrative tools and open the Network Load Balancing Manager. Right click the top node in the tree menu on the left and select New Cluster.
Set the IP and other information you will used for the Load Balanced cluster. This is a new IP not used by your View Manager servers.
In the VMware document referenced above VMware recommends setting the Cluster operation mode to Multicast.
Click Next then next again. When asked to configure port rules I leave it on the default and click next. You can chose limit this to certain ports.

media_1272295397190.png

Click Next again and enter localhost in the wizard to configure the local interfaces for NLB. Click next and make sure to note the priority. When setting up the replica server this number needs to be different. Finally click finish and wait for the configuration to finish. You should now be able to ping your new cluster IP address.

Setup the Replica Server in the Load Balancer

media_1272295861268.png

Righ Click the node in the tree menu for the NLB Cluster you just created and select Add new host to cluster. Enter the IP for the Replica Server and click connect. Select the interface that will be used for the Load Balancing and click next. Make sure the Priority is unique from the first server. If it gives you any grief after this point close and re-open the Network Load balancing Manager. The working cluster should look like this:

media_1272296379739.png

Test the Failover

media_1272296907908.png

Start a continual ping to the cluster IP. Now use the vSphere Client to disconnect the network from one of the servers. Watch the pings continue to come back.

Finally, create a DNS A record (something like desktop.yourdomain.com) and point it to the cluster IP. You now have some decent failover in case of a VM failure and even a host failure (suggestion would be to use seperate hosts for the VM’s).

Note – You may need to add static ARP entries into your switching depending on your network topology. Be sure to test this fully and consult your network manufacturer’s documention for help with static ARP.

Adaptive Queuing in ESX

While troubleshooting another issue a week or two ago I came across this VMware knowledge base article. Having spent most of the time with other brand arrays in the past, I thought this was a pretty cool solution verses just increasing the queue length of the HBA. I would recommend setting this on your 3par BEFORE you get QFULL problems. Additionally, Netapp has an implementation of this as well.

Be sure to read the note at the bottom especially:

If hosts running operating systems other than ESX are connected to array ports that are being accessed by ESX hosts, while the latter are configured to use the adaptive algorithm, make sure those operating systems use an adaptive queue depth algorithm as well or isolate them on different ports on the storage array.

I do need to dig deeper how this affects performance as the queue begins to fill, not sure if one method is better than another. Is this the new direction that many Storage Vendors will follow?

Until then, the best advice is to do what your storage vendor recommends, especially if they say it is critical.

Here is a quick run through for you.

In the vSphere Client

wpid348-media_1272214293023.png

Select the ESX host and go to the configuration tab and click on the Advanced Settings under Software.

In the Advanced Settings

wpid349-media_1272214590686.png

Select the option for Disk and scroll down to the QFullSampleSize and QFullThreshold.
Change the values to the 3par recommended values:
QFullSampleSize = 32
QFullThreshold = 4

My Fun with the VMware Enterprise Administration and Design Exams

Sorry I have been missing for a few weeks. I know many were quite worried why I hadn’t blogged for a couple weeks (not really).

Back in February I sat for the Enterprise Administration Exam at PEX in Las Vegas. It was scheduled the day after the Super Bowl, what a bunch of distractions. Thankfully I passed and I want to give my experience so as to not violate any rules or anything I agreed to. This was a technical test. A lot of settings and configurations and information like that. Still multiple choice so at least you know the right answer is on the screen (hopefully, I did have one I thought none of these are right). The lab section was actually as fun as test taking could be. I wish there was more lab practical type things when it comes to these kinds of tests. Overall there is more intricate settings and config questions then you will find on the VCP exam.

At the end of April I took the Design Exam. This was a much different experience. I had a extremely hard time finding a study list of things that would help. Know the Exam Blueprint is all I would say. Also, this I think is where VMware can start finding out who does Architecture work and who may be an Administrator. I could say you could read every PDF on VMware.com and still not know how to pass this test unless you work with the solutions multiple times. The design drawing was a challenge, I wasted too much time reading the requirements document and ran of time, but I feel I was able to get a good portion of what I needed up on the page. Technically the interface was kind of quirky.

I felt both exams were challenging and but were fair to the Exam Blueprints. Nothing on there made me scream, “they didn’t say they would test on THAT!” The design exam needs some technical improvement (matching questions were buggy).

Now begins the harder and more involved process. The Design submission and hopefully an invitation to a defense.

Ask Good Questions

This happened a long time ago. I arrived at a customer site to install View Desktop Manager (may have been version 2). This was before any cool VDI sizing tools like Liquidware Labs. I am installing ESX and VDM I casually ask, “What apps will you be running on this install?” The answer was, “Oh, web apps like youtube, flash and some shockwave stuff.” I thought “ah dang” in my best Mater voice. This was a case of two different organizations thinking someone else had gathered the proper information. Important details sometimes fall through the cracks. Since that day, I try to at least uncover most of this stuff before I show up on site.

Even though we have great assessment tools now, remember to ask some questions and get to know what is your customers end goal.

Things I learned that day. As related to VDI.

1. Know what your client is doing, “What apps are you going to use?”

2. Know where your client wants to do that thing from, “So, what kind of connection do you have to that remote office with 100+ users?”

This is not the full list of questions I would ask, just some I learned along the way.