In an earlier article VMDirectPath I/O was discussed and how it can be used to virtualize a storage lab. Based on some of the comments in that article and more information received from Sunny LiYu Zhang, I wanted to add the following points:
- The VT-d specification states that all conventional PCI devices behind a PCIe-to-PCI bridge have to be assigned to the same domain.
- PCIe devices do not have this restriction.
Notes from Sunny:
Please see the VMDirect IO configed on my ESXi 5.0 server.
My mainboard contain “5520 I/O HUB PCI Express root port”. ASIC PI7C21P100 as a translator, bridge PCI-E resource to PCI-X. Each of this ASIC has two channel. It means with one “PCI Express root port”, only two PCI-X device can be used.
VMDirectPath I/O has a little bit limitation. Example, If you assign some device in the red box selected (09:xx:xx) to one VM:
1: at least one sub-device can be assign to one VM.
2: if one sub-device has been assign to one VM, other can’t be used by other VM. Because of root-tree ASIC (PI7C21P100) has been used by VM DirectI/O.
What this means is that with PCI-e cards you will have much more flexibility but no doubt at a higher cost. It is important that if you are going to use PCI-X cards, that you take into consideration the limitations that PCI bridges place on VMDirectPath I/O at this time.
