Fibre or Ethernet Saturation – Which comes First?

I was thinking about how far I can scale a VI3 Enviroment yesterday. I started to think, and that can be dangerous.
What will saturate first? The Fibre network or the Ethernet network?
So in my envisioned setup it would have dual quad core processors so if I can still do math that is 8 cores. If I might fit 4 x 1vCPU virtuals per core, I could theorize 32 VM’s per host? Now lets say I bought 2 of those quad port NICs for each host, so to be simple there is 8 network ports per host. Finally, lets say I have 2 single port HBA’s connecting to the fibre and I am lucking enough for it to be 4gb all the way to the SAN.
We have 2 Cisco 48 port 3560 GigE switches for the ESX hosts to access and 2 24 port Brocade Fibre Switches. So I scale my ESX hosts to fill the Cisco switches and it tops out at 11. I will use 11 ports on each Brocade, the Storage Processors use 4 more ports.
So what fills up first?
– Fibre
– Ethernet
– Disk IO

Practically my bet would be on Memory, but lets say memory can go as high as we need.

Another snag is CPU resources, to generate enough network traffic to kill that many GigE Nics I would think the CPU’s would pin out first?

I really wish I had a good lab with lots of vendor equipment I could test and try to break. That would be fun for me.

6 Replies to “Fibre or Ethernet Saturation – Which comes First?”

  1. Isn’t it going to depend on the applications that you are running?
    Some apps are disk IO intensive, say a database server, others are network intensive, like webservers. So as always in IT, it depends 🙂

    Memory has often been the first bottle neck hit, with CPU possible second. But both of those and networking can easily and cheaply resolved.
    However this is not the case with disk IO, it tends to be a much more expensive problem to resolve. Whether the solution is a faster HBA, switches, storage processors or disks, your boss is going to cringe at the price.

  2. Isn’t it going to depend on the applications that you are running?Some apps are disk IO intensive, say a database server, others are network intensive, like webservers. So as always in IT, it depends :)Memory has often been the first bottle neck hit, with CPU possible second. But both of those and networking can easily and cheaply resolved. However this is not the case with disk IO, it tends to be a much more expensive problem to resolve. Whether the solution is a faster HBA, switches, storage processors or disks, your boss is going to cringe at the price.

  3. It will always depend on the application.

    If all else was equal though, what multiplies faster? Fibre traffic or ethernet?
    If I am running say exchange on all the vm’s. Since it is something I think that is heavy on disk IO like a database and can get hammered via the network.
    I guess if I had all the resources of Cisco. What switch keels over first the fibre or the ethernet?

    Just using Cisco as an example since they make both kinds of switches.

  4. It will always depend on the application.If all else was equal though, what multiplies faster? Fibre traffic or ethernet? If I am running say exchange on all the vm’s. Since it is something I think that is heavy on disk IO like a database and can get hammered via the network.I guess if I had all the resources of Cisco. What switch keels over first the fibre or the ethernet?Just using Cisco as an example since they make both kinds of switches.

Leave a Reply

Your email address will not be published. Required fields are marked *