Just realised that I have too many stupid disks that are just too large, i.e. 100 GB. I need to change the disk to thin provision.
http://virtualbox-provenpractice.blogspot.com/2009/04/thick-disk-to-thin-disk.html
Sort of like slimming down all my ESXI hard disks. Sample code to do conversion of thick disks to thin disks:
vmkfstools -i "DaVinci – Dynamic IP_1.vmdk" newfile3.vmdk -d thin
Hmm.. this looks pretty good for learning about Open Solaris…
Was wondering how I could ssh into my VMWare instance until I hit upon this video:
Hmmm.. this guy face the exact bandwidth limit!
I was wondering if 75MB/s means I did something wrong.. because if you multiply 75 by 8.. you still fall far short of a 1000.
Hmmm.. maybe the infiniband network is something I should look at too… looks like u can connect direct between 2 PCs without a switch! That is real cool…
Imagine I can connect it with ZFS – ESXI. That will be so cool!
Hardware Supported List:
http://communities.vmware.com/cshwsw.jspa?sortField=1&sortOrder=1&start=25
Seems to suggest we need a switch with subnet manager.
Also explains how we can download the right drivers for the mellanox Connect-X HBA cards
EBay Options
Okay… looks.. not too easy
http://www.nexentastor.org/boards/1/topics/1113
Needs cards with memory
is that Mellanox also has working drivers which is good. Also important to note, Solaris needs cards with onboard memory as you cannot use the memory free ones which utilize system memory instead. Finally, FreeBSD and OpenSolaris were ROUGH to set up
This article looks a little complicated.. but seems to be what I am trying to do:
Nice Instructions on Setting up ISCI and VMWare
http://www.vmadmin.co.uk/vmware/35-esxserver/231-esxvmkernelrrmpiodvs
ISCI
I think I finally got the concept. Link Aggregation is not for ISCI. It seems like we got to have dedicated NICs for Link Aggregation for ISCI.
ISCI Part 2
Pretty interesting read.. on how this guy got it working with his Thecus N7700PRO which is .. a NAS with a 10Gbe interface. 
http://www.gavinadams.org/blog/2010/07/19/esxi-41-and-the-9000-byte-mtu-on-vmk0
Interesting Terms
Interesting new terms I would like to know.
Interesting Quotes
If performance is important, have you thought about how many workloads (guests) are you running? Both individually and in aggregate are they typically random, or streaming? Random I/O workloads put very little throughput stress on the SAN network. Conversely, sequential, large block I/O workloads place a heavier load.
Found an interesting article that allows us to run commands to the ESXI host.
http://virtualfuture.info/2009/01/run-esxtop-on-esxi-from-vima/
One of the main problems with ESXI is that… we are always running out of RAM. Yeah.. too many Virtual Machines…
So I am thinking of getting this :
RAM Monster
Well it seems like it has 32 slots for RAM.. that will work out to over 128GB if we use cheap 4GB DDR3 rams!
ESXI Compatible
And yeap! It is VMWare compatible. So glad that VMWare actually started maintaining their Hardware Compatibility List for server boards…
http://www.vmware.com/resources/compatibility/detail.php?device_cat=server&device_id=18842
Documentation
Well, my Link Aggregation is definitely not working.. which is kind of expected… ‘cos I am currently using a cheapo 35 USD 8 port switch… I mean.. if that works… it will be kind of amazing.
I tried running my benchmarks simultaneously on both VMS, i.e. Picasso and Davinci which are both on a LUN on my ZFS computer. Their bandwidth is like halfed…
(Compared to the blazing fast response at http://james.com.sg/2011/04/10/esxidisk-reads-benchmarking/)
You might be thinking.. maybe its your harddisk that is slow? Well, I actually did a
show performance network
on the nexenta system … and guess what… most of the traffic is still routed through e1000g0. Almost nothing at all goes to e1000g1. 
Guess will have to work on it tonight and try to upgrade it to a much nicer switch… this will have to wait till at night.. when nobody is at the office.. .and yeah! I got the whole office to myself again!
TRENDnet TEG-240WS
Okay. Just swapped in my TRENDnet…
After Trunk Settings
Link Aggregation Results
Hmmm… not sure if this means it is partially working… somehow for ZFS system, it magically dedicates one port to Tx and the other to Rx. So I guess the operation for ZFS is sort of working on the ZFS server.
However, for ESXI, it is definitely not working. Both VMs are still hitting at the same PortID 4.
I really need a switch to handle Link Aggregation. Otherwise, there is really no point in doing all those ZFS stuff when the choke point is at a gigabit internet. So here are some options:
Do not see to find any article on the internet on successful implementation of Link Aggregation for the above switch.
Seems that the above switch supports both static and dynamic link aggregation:
But ESXI only seem to support Static Link Aggregation
TRENDnet TEG-240WS
Well, I do have a WebSmart switch currently… and maybe it might work. I mean.. there is some kind of Trunk Setting in the configuration…
This looks promising:
I like the concept of:
Compression however has been in the system for a bit and works well. The idea is that because you have to read/write less to the IO subsystem you get better response times. The decompression is done in CPU and we’re all happy with much better throughput at the cost of a bit more CPU (which seems ubiquitous in this day and age). The default compression algorithm LZJB is unnoticeable on most systems and gets a decent amount of compression (~40% on my system).
Yes… my CPU is always like grinding at 20%. Will like to make it just a little harder… if it can give me better performance… 
