In my current role at Nutanix as a Nutanix Services Consultant, me and my colleagues are often tasked with assisting our customers in large 100+ host deployments of new Nutanix Nodes. Since Nutanix hosts are shipped from factory with Nutanix AHV Hypervisor, it is required to use a tool called Nutanix Foundation to reinstall the nodes with either ESXi, Hyper-V or XenServer Hypervisor depending on the customer’s requirements. Nutanix Foundation is available in two separate downloadable packages. The first one is known as the “Foundation Applet” which is a Java Applet you download and run on your laptop. The foundation applet discovers Nutanix Nodes using IPV6 multi-cast and then calls the foundation process running on the Nutanix CVM. A webbrowser can then be launched and you configure the nodes with the target HyperVisor and AOS Package for installation. The foundation applet first installs one node with the target hypervisor and AOS package, and then transfers the imaging process to the rest of the nodes. One of the requirements for the foundation applet is that your laptop must be on the same subnet as the target Nutanix Cluster.
The second method is known as the “Foundation VM” and is available to Nutanix Staff and Partners. The Foundation VM run’s as a Virtual Machine on either VMware Workstation/Fusion, ESXi, Virtual Box, or Nutanix AHV and deploys multiple nodes sequentially and also allows you to configure multi-homing which is used if initially deploying nodes on a flat switch for later connectivity to the customers target network. Using Multi-Homing, you can configure the foundation VM with multiple IP addresses to match the target nodes IP address range for Hypervisor/CVM and IPMI.
Typically, I have been deploying Nutanix Nodes using the foundation VM, but in some larger deployments, customers have different requirements such as different IP ranges for some clusters, different HyperVisors, or different AOS versions. Whilst I could deploy these one-by-one by running a single foundation instance at a time, I have found it to be more efficient and faster to run multiple foundation instances at the same time. To achieve this, I have turned an Intel NUC into a mobile Nutanix Foundation deployment server running Nutanix Community Edition. Nutanix Community edition runs Nutanix AHV and can be downloaded free of charge from here https://www.nutanix.com/products/community-edition/
Community Edition allows you to configure either a single, three or four node Nutanix Cluster on your own hardware. In this instance I have configured a single node nutanix cluster
The specs and BOM I am using for the Intel NUC are below,
- Intel Skulltrail NUC with Quad Core Core i7 NUC – NUC6i7KYK https://www.amazon.com/Intel-NUC-Kit-NUC6i7KYK-Mini/dp/B01DJ9XS52
- Memory – Crucial 32GB Kit (CT2K16G4SFD8213) https://www.amazon.com/Crucial-16GBx2-PC4-17000-SODIMM-260-Pin/dp/B015YPB8ME/ref=sr_1_1? s=pc&ie=UTF8&qid=1477954733&sr=1-1&keywords=CT2K16G4SFD8213
- SSD – 2x Crucial MX300 525GB SATA M.2 SSD (CT525MX300SSD4) https://www.amazon.com/Crucial-MX300-525GB-Internal-CT525MX300SSD4/dp/B01L80DH4G/ref=sr_1_1?s=pc&ie=UTF8&qid=1477954775&sr=1-1&keywords=CT525MX300SSD4
- Boot Device SanDisk Cruzer Fit 16GB USB 3.0 flash drive https://www.amazon.com/SanDisk-Cruzer-Low-Profile-Drive-SDCZ33-016G-B35/dp/B005FYNSZA/ref=sr_1_1?ie=UTF8&qid=1477954852&sr=8-1&keywords=SanDisk+Cruzer+Fit+16GB
So quite a powerful, yet compact device that I can easily take with me to customers sites for deployments.
To allow me to run four different foundation instances on the Intel NUC, I have configured each foundation VM with 2x vCPU and 3GB of RAM. The Nutanix Controller VM is configured with 2x vCPU and 16GB of RAM. Whilst you could potentially lower the memory requirements of the CVM, four Foundation VM’s was more than enough for my requirements.
The following diagram shows the IP address and physical configuration of the Intel NUC (AHV + CVM), Laptop (I’m using a Macbook Pro), and corresponding Foundation VM Instances along with the clusters each foundation instance will be building. Since Nutanix Community Edition requires internet access to start, I also have a router running a linux distro called “Zentyal” running as a VMware Fusion VM on my Macbook. Zentyal is configured with the default gateway address for the NUC (192.168.25.1) and a secondary interface tethered to Wifi. Zentyl also provides NTP services for new clusters, so the time is set correctly when the cluster is built.
Each Foundation VM is configured with three IP addresses as follows.
- IP address on the same subnet as my macbook, using the 192.168.25.x/24 range
- IP address on the same subnet as the Hypervisor Hosts/CVM
- IP address on the same subnet as the IPMI interfaces.
I have also duplicated the thunderbolt adapter on my macbook and set an IP address in the same range as each target cluster, so each cluster can be configured directly from my macbook once it is built using the nutanix prism interface.
Here is an overview of how to setup your own NUC to run Nutanix Community Edition as a Nutanix Foundation Server.
- Install Nutanix Community Edition on the Intel NUC. I will not go into details on how this is done, as my colleague Thomas Findelkind already has a great guide how to do this on his blog here http://tfindelkind.com/2016/06/27/intel-nuc-nuc6i7kyk-installation-nutanix-community-edition-ce-part2/
- Download Nutanix Foundation from https://portal.nutanix.com/#/page/foundation/list Make sure you download the qcow2 image for Nutanix AHV. As of this post, the latest version is Foundation_VM-3.9.1-disk1.qcow2
- Upload the foundation qcow image to Nutanix CE using the image upload service
- Create a new three new VM’s, use whatever naming standard you want. Set the following hardware specifications
- vCPU = 2vCPU, cores per vCPU = 1
- RAM = 3GB
- Add a hardisk to the VM, ensure “Clone from image service” is selected and select the qcow image uploaded earlier.
- Start each foundation VM, when it boots up run the “Set Foundation IP address” shortcut on the desktop, and set a static IP address for the foundation VM on the same subnet you have configured for Nutanix AHV and the CVM.
- Once each foundation VM is configured with a Static IP address, you can launch foundation straight from a webbrowser on your laptop over port 8000
- After the foundation VM is configured, you can begin cabling all the nodes to your deployment switches. In this example, 58 nodes have been cabled simultaneously to x3 24 port switches and can be discovered from a single foundation instance. If you are having trouble determining what nodes should be foundationed from one instance, you can temporarily uncable nodes you don’t want to be discovered.
- Once you get to the node imaging page, you can upload your target hypervisor and AOS packages directly to the foundation VM by clicking the “Manage” button
- Browse to the image on your local laptop, and click upload.
- Next, start foundation and repeat on each foundation VM for any nodes which require a different AOS version, Subnet, or Hypervisor. The below screenshot shows a successful run of 38 nodes on one single foundation instance running on the NUC.
Using an Intel NUC running Nutanix Community Edtion for large Nutanix Deployments has definitely made things easier for larger Nutanix Installs, and it will be accompanying me when time is a constraint for getting nodes deployed. I have been impressed with the performance of the Intel NUC and Nutanix community edition, in the below screenshot you can see one of the foundation VM’s pulling 1195 read IOP’s during the image validation phase that happens at the start of the foundation imaging process.