PXE (Preboot eXecution Environment) Server allows unattended or automated OS installation over the Network. The main benefit of pxe is that we don’t need any bootable drive to boot OS(Operating system) and we do not need to to burn any ISO file into DVD or usb device.
Once the PXE server is configured we can install hundreds of System at the same time over the network. As it works on Client-Server architecture, to get the OS installation on clients, boot the clients via PXE option.
In this article, I would be setting up PXE server on RHEL 7.9.
Following configuration is required in Cisco Intersight for PXE boot for Cisco UCSX 210c.
Modify the boot policy for PXE boot as below:
eth0 is the interface that will get the IP address via DHCP.
You can capture the interface information in Cisco Intersight in server profiles in inventory tab. Also note down the interface MAC address for configuring IP reservation in dhcp configuration file which will be covered later in this article.
Configure the Boot order in intersight as shown below:
Install Required Packages
# yum install dhcp tftp tftp-server syslinux vsftpd xinetd
Assumptions
This article assumed the following:
- PXE setup requirements such as DHCP, TTFP, HTTP is hosted in a single server, although it is not mandatory.
- The PXE server can reach the internet.
- The PXE server is setup on Red Hat Enterprise Linux (RHEL) 7.9
Configure DHCP Server
Environment:
PXE server IP: 10.10.1.5
DNS: 10.10.1.4
Gateway: 10.10.1.4
PXE Server OS: RHEL 7.9
Firewall: disabled
For configuring DHCP, specify the subnet and the range used for offering IP address via DHCP. You can also specify lease time.
In this configuration, we have specified IP reservation for nodes with the MAC address of the interface configured for PXE boot.
Below are the steps to set up the DHCP to support PXE boot for UEFI servers
Note: This DHCP configuration will not work for BIOS or legacy mode. For this to work, you need to specify filename “pxelinux.0”. To work for both BIOS and UEFI mode, you can also specify conditional checks on vendor-class-identifier for specifying filename. More details on this can be found at https://www.redhat.com/sysadmin/pxe-boot-uefi
Configure dhcp using the following conf file. Change according to your environment
[root@e26-linuxjb images]# cat /etc/dhcp/dhcpd.conf
#
# DHCP Server Configuration file.
# see /usr/share/doc/dhcp*/dhcpd.conf.example
# see dhcpd.conf(5) man page
#
ddns-update-style interim;
ignore client-updates;
authoritative;
allow booting;
allow bootp;
allow unknown-clients;
# internal subnet for my DHCP Server
subnet 10.10.1.0 netmask 255.255.255.0 {
range 10.10.1.45 10.10.1.100;
option domain-name-servers 10.10.1.4;
option domain-name "sjc02-cdip.cisco.local";
option broadcast-address 10.10.1.255;
option routers 10.10.1.4;
default-lease-time 600;
max-lease-time 7200;
next-server 10.10.1.5;
filename "grubx64.efi";
host boootstrap {
hardware ethernet 00:25:b5:00:26:06;
fixed-address 10.10.1.80;
}
host master0 {
hardware ethernet 00:25:B5:00:26:00;
fixed-address 10.10.1.50;
}
host master1 {
hardware ethernet 00:25:B5:00:26:01;
fixed-address 10.10.1.51;
}
host master2 {
hardware ethernet 00:25:B5:00:26:02;
fixed-address 10.10.1.52;
}
host worker0 {
hardware ethernet 00:25:B5:00:26:03;
fixed-address 10.10.1.53;
}
host worker1 {
hardware ethernet 00:25:B5:00:26:04;
fixed-address 10.10.1.54;
}
}
Every time, when dhcpd.conf is modified, restart the dhcpd service as below:
# systemctl restart dhcpd
# systemctl status dhcpd
# systemctl enable dhcpd
Edit and configure TFTP server
TFTP (Trivial File Transfer Protocol) is used to transfer files from server to clients without any kind of authentication. In the case of PXE, tftp perform bootstrap loading.
TFTP server is needed to provide the following:
- initrd.img – The “boot loader” which will be loaded to RAM disk.
- vmlinuz – A compressed bootable Linux Kernel.
To configure tftp, edit the following configuration file.
[root@e26-linuxjb ~]# cat /etc/xinetd.d/tftp
# default: off
# description: The tftp server serves files using the trivial file transfer \
# protocol. The tftp protocol is often used to boot diskless \
# workstations, download configuration files to network-aware printers, \
# and to start the installation process for some operating systems.
service tftp
{
socket_type = dgram
protocol = udp
wait = yes
user = root
server = /usr/sbin/in.tftpd
server_args = -s /var/lib/tftpboot
disable = no
per_source = 11
cps = 100 2
flags = IPv4
}
All the network boot related files are to be placed in tftp root directory “/var/lib/tftpboot”
Run the following commands to copy required network boot files in ‘/var/lib/tftpboot/’
[root@pxe ~]# cp -v /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot
[root@pxe ~]# cp -v /usr/share/syslinux/menu.c32 /var/lib/tftpboot
[root@pxe ~]# cp -v /usr/share/syslinux/memdisk /var/lib/tftpboot
[root@pxe ~]# cp -v /usr/share/syslinux/mboot.c32 /var/lib/tftpboot
[root@pxe ~]# cp -v /usr/share/syslinux/chain.c32 /var/lib/tftpboot
[root@pxe ~]#
[root@pxe ~]# mkdir /var/lib/tftpboot/pxelinux.cfg
[root@pxe ~]# mkdir /var/lib/tftpboot/networkboot
Create subfolder in /var/lib/tftpboot/networkboot for each OS being configured for PXE boot. For example, in this case, RHEL 7.8 and CoreOS 4.6.8
[root@pxe ~]# mkdir /var/lib/tftpboot/networkboot/rhel78
[root@pxe ~]# mkdir /var/lib/tftpboot/networkboot/coreos46
Mount ISO file
Download the iso file for RHEL 7.8 and CoreOS 4.6.8 and move it to PXE server.
For example, run the following to download the CoreOS 4.6.8 iso.
# curl -L -J -O https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/4.6/latest/rhcos-4.6.8-x86_64-live.x86_64.iso
Create a sub folder in ‘/var/ftp/pub’ for each OS to store the boot image files.
# mkdir /var/ftp/pub/coreos468
# mkdir /var/ftp/pub/rhel78
Run the following commands to mount iso file both for rhel 7.8 and coreOS 4.6.8 and then copy its contents in ftp server’s directory ‘/var/ftp/pub/coreos468’
Perform the following for each OS iso file. In this example, I setup two OS, RHEL 7.8 and CoreOS 4.6.8
[root@pxe ~]# mount -o loop rhcos-4.6.8-x86_64-live.x86_64.iso /mnt/
mount: /dev/loop0 is write-protected, mounting read-only
[root@pxe ~]# cd /mnt/
[root@pxe mnt]# cp -av * /var/ftp/pub/coreos468
Content of coreos468 is shown as below:
[root@e26-linuxjb ~]# ls -ll /var/ftp/pub/coreos468
total 887512
dr-xr-xr-x. 3 root root 20 Dec 5 2020 EFI
dr-xr-xr-x. 3 root root 60 Dec 5 2020 images
dr-xr-xr-x. 2 root root 156 Dec 5 2020 isolinux
-r--r--r--. 1 root root 132 Dec 5 2020 zipl.prm
[root@e26-linuxjb ~]#
Content of RHEL 7.8 folder is shown is below
[root@e26-linuxjb ~]# ls -ll /var/ftp/pub/rhel78/
total 412
dr-xr-xr-x. 4 root root 54 Jun 26 21:50 addons
dr-xr-xr-x. 3 root root 18 Jun 26 21:50 EFI
-r--r--r--. 1 root root 8266 Jun 26 21:50 EULA
-r--r--r--. 1 root root 1455 Jun 26 21:50 extra_files.json
-r--r--r--. 1 root root 18092 Jun 26 21:50 GPL
dr-xr-xr-x. 3 root root 57 Jun 26 21:50 images
dr-xr-xr-x. 2 root root 217 Jun 26 21:50 isolinux
dr-xr-xr-x. 2 root root 43 Jun 26 21:50 LiveOS
-r--r--r--. 1 root root 114 Jun 26 21:50 media.repo
dr-xr-xr-x. 2 root root 282624 Jun 26 21:50 Packages
dr-xr-xr-x. 2 root root 4096 Jun 26 21:50 repodata
-rw-r--r--. 1 root root 963 Jun 26 21:50 rhel78.cfg
-r--r--r--. 1 root root 3375 Jun 26 21:50 RPM-GPG-KEY-redhat-beta
-r--r--r--. 1 root root 3211 Jun 26 21:50 RPM-GPG-KEY-redhat-release
-r--r--r--. 1 root root 1796 Jun 26 21:50 TRANS.TBL
[root@e26-linuxjb ~]#
Copy ISO file contents to FTP server folder
Copy Kernel file (vmlimz) and initrd file from mounted iso file
For CoreOS copy to ‘/var/lib/tftpboot/networkboot/coreos468’
For RHEL copy to ‘/var/lib/tftpboot/networkboot/rhel78’
# cp /var/ftp/pub/coreos468/images/pxeboot/* /var/lib/tftpboot/networkboot/coreos468/.
[root@e26-linuxjb ~]# ls -ll /var/lib/tftpboot/networkboot/coreos468
total 887508
-r--r--r--. 1 root root 79512484 Jun 28 11:58 initrd.img
-r--r--r--. 1 root root 820364800 Jun 28 11:59 rootfs.img
-r--r--r--. 1 root root 8924528 Jun 28 11:57 vmlinuz
[root@e26-linuxjb ~]#
[root@e26-linuxjb ~]#
[root@e26-linuxjb ~]# ls -ll /var/lib/tftpboot/networkboot/rhel78/
total 63148
-r--r--r--. 1 root root 57894232 Jun 25 20:25 initrd.img
-r-xr-xr-x. 1 root root 6762800 Jun 25 20:25 vmlinuz
[root@e26-linuxjb ~]#
Note: In case of CoreOS, you can also download kernel, initramfs, and rootfs from Red Hat Mirror site (https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/4.6/latest ) and store it in the /var/lib/tfpboot/networkboot/coreos468 folder instead of getting it from iso file.
unmount the iso file using ‘umount’ command
[root@pxe ~]# umount /mnt/
[root@pxe ~]#
Now you can verify the content of ftp server in the browser as shown below. Make sure your ftp service is running.
rhel78 folder contents from browser is shown below
You can also open the ftp site in window explorer to perform CRUD operations.
Configure grub.cfg for UEFI or pxelinux.cfg/default for creating PXE menu
[root@e26-linuxjb images]# cat /var/lib/tftpboot/grub.cfg
set timeout=60
# for bootstrap node
menuentry 'Install RHEL CoreOS 4.6.8 Bootstrap Node' --class fedora --class gnu-linux --class gnu --class os {
linuxefi /networkboot/coreos468/vmlinuz inst.repo=ftp://10.10.1.5/pub/coreos468 coreos.live.rootfs_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-live-rootfs.x86_64.img nomodeset rd.neednet=1 coreos.inst=yes coreos.inst.install_dev=sda coreos.inst.image_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-metal.x86_64.raw.gz coreos.inst.insecure coreos.inst.ignition_url=http://10.10.1.5:8080/ignition-install/bootstrap.ign
initrdefi /networkboot/coreos468/initrd.img
}
# for master node
menuentry 'Install RHEL CoreOS 4.6.8 Master Node' --class fedora --class gnu-linux --class gnu --class os {
linuxefi /networkboot/coreos468/vmlinuz inst.repo=ftp://10.10.1.5/pub/coreos468 coreos.live.rootfs_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-live-rootfs.x86_64.img nomodeset rd.neednet=1 coreos.inst=yes coreos.inst.install_dev=sda coreos.inst.image_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-metal.x86_64.raw.gz coreos.inst.insecure coreos.inst.ignition_url=http://10.10.1.5:8080/ignition-install/master.ign
initrdefi /networkboot/coreos468/initrd.img
}
# for worker node
menuentry 'Install RHEL CoreOS 4.6.8 Worker Node' --class fedora --class gnu-linux --class gnu --class os {
linuxefi /networkboot/coreos468/vmlinuz inst.repo=ftp://10.10.1.5/pub/coreos468 coreos.live.rootfs_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-live-rootfs.x86_64.img nomodeset rd.neednet=1 coreos.inst=yes coreos.inst.install_dev=sda coreos.inst.image_url=http://10.10.1.5:8080/ignition-install/rhcos-4.6.8-x86_64-metal.x86_64.raw.gz coreos.inst.insecure coreos.inst.ignition_url=http://10.10.1.5:8080/ignition-install/worker.ign
initrdefi /networkboot/coreos468/initrd.img
}
menuentry 'Install RHEL 7.8' {
linuxefi /networkboot/rhel78/vmlinuz inst.repo=ftp://10.10.1.5/pub/rhel78
initrdefi /networkboot/rhel78/initrd.img
}
[root@e26-linuxjb images]#
For BIOS boot, pxelinux.cfg/default is needed and following file can be used as a reference
[root@e26-linuxjb ~]# cat /var/lib/tftpboot/pxelinux.cfg/default
default menu.c32
prompt 0
timeout 30
MENU TITLE Cisco SJC LAB PXE Menu
LABEL rhel7_x64
MENU LABEL RHEL 7_X64
KERNEL /networkboot/rhel78/vmlinuz
APPEND initrd=/networkboot/rhel78/initrd.img inst.repo=ftp://10.10.1.5/pub/rhel78
ks=ftp://10.10.1.5/pub/rhel78/rhel78.cfg
[root@e26-linuxjb ~]#
Create KickStart
Before creating kickstart file, let’s first create the root password in an encrypted string because we will using that encrypted password string in kickstart file.
[root@pxe ~]# openssl passwd -1 Pxe@123#
$1$e2wrcGGX$tZPQKPsXVhNmbiGg53MN41
[root@pxe ~]#
System default kickstart file is placed under /root with name ‘anaconda-ks.cfg’. We will be creating a new kickstart under the folder /var/ftp/pub/rhel78 with the name ‘rhel78.cfg’
Copy the following content into the new kickstart file. Please modify the kickstart file as per your needs.
[root@pxe ~]# vi /var/ftp/pub/rhel78/rhel78.cfg
#platform=x86, AMD64, or Intel EM64T
#version=DEVEL
# Firewall configuration
firewall --disabled
# Install OS instead of upgrade
install
# Use FTP installation media
url --url="ftp://172.168.1.11/pub/"
# Root password
rootpw --iscrypted $1$e2wrcGGX$tZPQKPsXVhNmbiGg53MN41
# System authorization information
auth useshadow passalgo=sha512
# Use graphical install
graphical
firstboot disable
# System keyboard
keyboard us
# System language
lang en_US
# SELinux configuration
selinux disabled
# Installation logging level
logging level=info
# System timezone
timezone Europe/Amsterdam
# System bootloader configuration
bootloader location=mbr
clearpart --all --initlabel
part swap --asprimary --fstype="swap" --size=1024
part /boot --fstype xfs --size=300
part pv.01 --size=1 --grow
volgroup root_vg01 pv.01
logvol / --fstype xfs --name=lv_01 --vgname=root_vg01 --size=1 --grow
%packages
@^minimal
@core
%end
%addon com_redhat_kdump --disable --reserve-mb='auto'
%end
Start and enable services
[root@pxe ~]# systemctl start xinetd
[root@pxe ~]# systemctl enable xinetd
[root@pxe ~]# systemctl start dhcpd.service
[root@pxe ~]# systemctl enable dhcpd.service
Created symlink from /etc/systemd/system/multi-user.target.wants/dhcpd.service to /usr/lib/systemd/system/dhcpd.service.
[root@pxe ~]#
[root@pxe ~]# systemctl start vsftpd
[root@pxe ~]# systemctl enable vsftpd
Created symlink from /etc/systemd/system/multi-user.target.wants/vsftpd.service to /usr/lib/systemd/system/vsftpd.service.
[root@pxe ~]#
In Case SELinux is enabled, then set the following selinux rule for ftp server.
[root@pxe ~]# setsebool -P allow_ftpd_full_access 1
[root@pxe ~]#
Following is required for UEFI
cp /var/ftp/pub/rhel78/EFI/BOOT/grubx64.efi /var/lib/tftpboot/
Launch vKVM
Launch vKVM from Cisco Intersight and login to server vKVM console.
Click PowerPower Cycle System
Click Boot Device Select LAN
The following screen will display and launch the PXE boot menu as defined in grub.cfg