PXE-Less Windows Server Bare Metal Installations on UCS

[Vallard Benincosa]

Last year I wrote about automating Red Hat Linux installations using VMedia policies.  Today we'll talk about a brand new way to install Windows Operating Systems using the same method.  This method has not been well documented in the past and presents a brilliant way for us to automate windows installations on UCS. While what we present in this document will appear complex we have automated the entire process using KUBAM.  KUBAM is an open source tool for deploying bare metal operating systems on UCS with minimal hassle.  Here we present how we do it.  Much of this work is based on a method we developed 10+ years ago on the open source xCAT project that was never well documented. With UCS that process is now simplified greater.

The process is comprised of the following steps:

1. Create a WinPE image using Microsoft Assessment and Deployment Kit (ADK) 8.1.  Yes, you can use this to deploy both Windows Server 2012 R2 and Server 2016.  We use this for both.
2. Create Windows autounattend.xml file and put in hard drive image.
3. Serve Windows ISO image, WinPE image, autounattend image, and UCS drivers via SAMBA share.

Daniel Brlekovic and myself have written a detailed guide on this process on the KUBAM site that goes into further details of how this is to happen.  In Daniel's guide he talks of how this can be done without SAMBA with just pure VMedia policies.  This is a great more traditional approach.  The method of using SAMBA is a more scalable situation when dozens of servers need to be deployed at the same time.  The reason for the SAMBA is that it is more scalable for the media to be downloaded over the UCS 40Gb links rather than the 1Gb Management interface.

In this article we will discuss the SAMBA method.

1. Creating the WinPE ISO image

We need what Microsoft calls a "Technician Computer" or a windows machine (virtual or physical, doesn't have to be UCS) where we can build our WinPE image.  The WinPE image is a small version of Windows that can be used to boot strap our operating system install.  There are a few tricks we are going to do to the generic WinPE image:

1. We're going to add the UCS drivers into the WinPE image.  This is necessary because we want the WinPE image to start up and grab the rest of the installation media over the 40Gb links and mount the remote drive using SAMBA.
2. We're going to hack the startnet.cmd file.  This is the file that Windows uses to do a default install.

Here's how we do it:

1.1 Make WinPE image

1.1.1 Download the ADK

Downloading the ADK has been well documented.  Install it using the default settings.  Whether you are installing Windows 2016 or Windows 2012R2 it doesn't matter which operating system you use to stage.  We have deployed Windows ADK 8.1 on both Windows 2012R2 and 2016 and installed both operating systems from the same image. There is a newer version of ADK but 8.1 works for both so we use it as its well tested and supported.

1.1.2 Download the UCS Windows Drivers

This is usually the most difficult part trying to find Server drivers.  Going to Cisco's home page, then support, then searching for UCS is usally the easiest route but links change from time to time.  This link works at the time of this writing.  You'll have to download a ridiculous 500MB image and then just grab the enic drivers.  In this case you'll need the 2012R2 drivers even if you are installing Windows 2016.  This is because ADK 8.1 bases the micro windows OS off the 2012R2 kernel. For our process to work put these drivers into a new directory called C:\Drivers on the Technician Computer. 

1.1.3 Download and run KUBAM WinPE builder scripts

To make creating the WinPE image easier we offer a simple batch script to make this automatic.  You can go to the version 2 branch of KUBAM and grab two files.  Download these files onto the Windows Desktop, the same machine where you have installed ADK.

Looking at this winkubam.bat script you'll see that it does a few important things:

1. It does the basic WinPE build process and adds addon packs for things like powershell etc.
2. It copies a new startnet.cmd into the WinPE image, overwriting the default one.
3. It adds the UCS drivers that are in the c:\drivers directory (and any other drivers that are there) into the image.
4. It then takes the WinPE image and builds an ISO image that can be used by the VMedia policy.

Run the command and you'll have a nice C:\WinPE_KUBAM.iso ready to be used by your install server.

1.1.4 Examining startnet.cmd

It's worth understanding what our startnet.cmd replacement does.  Opening the file you'll see that we run wpeinit which is the thing that the default startnet.cmd does. From here we read in a file that we will create on our autoinstall.img command.  The file looks like this:

<ip>
<netmask>
<gateway>
<SAMBA IP>
<OS>

An example would be:

192.168.1.11
255.255.255.0
192.168.1.1
10.0.0.3
win2016

Notice that since we are given a router we can actually install this operating system across different subnets, something that is very difficult to do with PXE.

With these variables in place the startnet.cmd mounts the SAMBA share, then starts the installation process.

2. Creating the Windows Unattend.xml file

Next up for the automation is to create the small files that we wish to serve out on the samba share. This requires a few steps:

1. Create an Autounattend.xml file
2. Create the network.txt file
3. Create a hard drive image with these files on it.
4. Add UCS Drivers for desired operating system into network share directory

2.1 Autounattend.xml file

In Daniels KUBAM documentation above he does a great job in showing how to customize an unattend.xml file for Windows.  As a starting point we have two samples you can use.  These samples will actually install remote desktop and other nice features, including setting up static IP addresses.  We've gone through lots of iterations to get it perfect how we like it but you may want to update it and add more. Notice that you'll have to create a separate file for each OS that you want to install. This is part of the magic that KUBAM does is to figure that part out for you.

2.2 network.txt file

As we mentioned above in section 1.1.4 the network.txt file is just a simple file that has 5 lines.  Refer above to how that should be.  A unique file should be made for every server you wish to deploy.

2.3 Hard Drive Image

To create an image that can be mounted by the VMedia policy we use a Linux server.  There are probably ways you can do this on Windows but we're more Linux savvy so we used that.  Plus, in our setup with KUBAM we stage all these images on a Linux server since we're also installing other operating systems like CentOS and ESXi. To create a hard drive image run the following:

dd if=/dev/zero of=kubam01.img bs=1M count=1  
mkfs -t fat kubam01.img  
mkdir tmpmnt  
mount -o loop kubam01.img tmpmnt  
cp autoinstall.xml tmpmnt/  
cp network.txt tmpmnt/  
umount tmpmnt  
rmdir tmpmnt 

Be sure to name the hard drive image the same name of the service profile.  In the above case the service profile created in UCS is called kubam01.

2.4 Network Drivers

Finally on the boot server you'll need to download those drivers that we downloaded earlier.  This time the drivers should match the operating system you are attempting to deploy.  In this way the new OS will come up with the installed drivers properly.

The way we make this work is by referencing these drivers in the autounattend.xml file.  These couple of lines in particular are worth calling out:

<settings pass="offlineServicing">  
<component name="Microsoft-Windows-PnpCustomizationsNonWinPE" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">  
<DriverPaths>  
<PathAndCredentials wcm:action="add" wcm:keyValue="1">  
<Path>i:\windows\drivers\win2016</Path>  
</PathAndCredentials>  
</DriverPaths>  
</component>  
</settings>  

Notice that we grab the windows drivers from the i: share directory.  This is the samba mounted directory.  We can put any other drivers we want in here to install it as part of the process.

3. Serve the Windows Media to the Service Profile

In the last step we need to serve out the media to the service profile.  We do this in a few ways using KUBAM.  You'll need two services:

1. Web Service for the VMedia Policy
2. SAMBA service for the Windows boot service

In KUBAM we create these automatic for you. It may be that you already know how to do this on a Windows Server and can make this possible.  The other possibility would be using Linux.  We use an nginx container to serve out a directory by mounting that directory.  For the Samba service, we were pleasantly surprised how easy it was to do by starting a samba docker container.  For us we ran:

docker run -d -v ~/kubam:/kubam \  
  -p 139:139 -p 445:445 \  
  dperson/samba \  
  -s "kubam;/kubam"  

Here you can see we put all our images into a ~/kubam directory.  The contents of the directory with relevant files are:

kubam01.img # 1MB image with autounattend.xml and network.txt  
win2016   # exploded ISO image of Windows 2016 6GB DVD  
windows/drivers/win2016/enic6x64.cat # driver files for OS that will be installed  
windows/drivers/win2016/enic6x64.inf  
windows/drivers/win2016/enic6x64.sys  
WinPE_KUBAM.iso  

In sections 2 and 3 in the article we did on Red Hat VMedia policies you can see how to create the Vmedia policy so it servers out the two images.

4. Conclusion

At this point we hope that you have a good idea of how this process can work for you!  The advantages of this method with UCS are the following:

1. The process is completely programmable.  We can script the entire process end to end.  By using the UCSM SDK referenced in several of the links of this page we can create the UCS boot policies and the boot media.
2. The process is scalable.  By downloading only the initial media (360MB) over the 1Gb management interface instead of the entire 5-6GB installation image we can improve the time it takes to deploy the entire process.
3. The process is secure.  No DHCP is required to configure with fancy TFTP flags.  Here we can deploy over production networks without need to configure intermediate processes nor mess with existing network settings.  We've seen many customers that install a completely separate VNIC template to accomplish this same action. In this way we keep the configuration simple.

We can now scale to any amount of simultaneous UCS server installations .  We would love for your feedback as you attempt to role this deployment method out in your own data centers.  We also recommend that you look into KUBAM to automate this OS deployment for you as well as other operating systems like CentOS, RedHat, and ESXi.