Zoetrope

(Re-)Installing Linux on a bare-metal Dell Poweredge server using CD's/DVD's is so passé. Instead use your servers network card to perform a PXE-boot and install it over the network.

Since all Dell Poweredge servers come with an IPMI BMC controller you can do this remotely from the office on a server in your datacenter. This how-to covers a remote install of a Dell Poweredge 2850 with an IPMI 1.5 BMC controller. This means we will use Serial-Over-Lan console redirection through a proxy program called Solproxy. If you've got a recent Poweredge server this step is not necessary.

This post assumes you have a working RHEL5/Centos5 install to serve as your provisioning server, and your bare-metal server has a working IPMI configuration. See this post by Lone Sysadmin how to configure the BMC on a Dell Poweredge if you have an OS running on it. Else you need to configure the BMC controller in the BIOS.

Here's how you use Cobbler to manage the TFTP boot server and DHCP server, and optionally build a mirror for several distro's from installation media or rsync mirrors...

First we need to install the needed software. I'm starting from a Centos 5 base install:

• yum install tftboot, dhcp, httpd, cobbler
• install the RPM containing the Solproxy software: yum install osabmcutil9g (for Dell PE 1850/2850 only) available from the linux.dell.com yum repositories
• change 'disable' to 'no' in /etc/xinetd.d/tftp
• service start xinetd && service start httpd
• service start dsm_bmu_solproxy32 to start the solproxy server.
• make sure your iptables configuration allows tftp (tcp-udp 69) and http traffic on your provisioning server

Let's see what cobbler needs to get it started:

cobbler check

The following potential problems were detected:
#0: The 'server' field in /var/lib/cobbler/settings must be set to something other than localhost,
or kickstarting features will not work. This should be a resolvable hostname or IP 
for the boot server as reachable by all machines that will use it.
#1: For PXE to be functional, the 'next_server' field in /var/lib/cobbler/settings must be set 
to something other than 127.0.0.1, and should match the IP of the boot server on the PXE network.

I'm not using the distro repository import/sync to manage cobbler distros/profiles so I need to have the vmlinuz and initrd images for Centos5. Do not copy these manually into /tftboot because cobbler wants absolute control about anything that goes into that directory and cobbler sync will clean this directory at will. (same thing applies to the /var/www/cobbler directory)

mkdir /var/lib/cobbler/images/centos5-x86_64/
wget http://centos.mirrors.skynet.be/pub/centos/5/os/x86_64/images/pxeboot/initrd.img
wget http://centos.mirrors.skynet.be/pub/centos/5/os/x86_64/images/pxeboot/vmlinuz

We need to add a distro to cobbler so it knows what kernel image and initrd to send when pxebooting. I'm also feeding it kernel-options to enable console redirection

cobbler distro add --name=centos5-x86_64 --kernel=/var/lib/cobbler/images/centos5-x86_64/vmlinuz --initrd=/var/lib/cobbler/images/centos5-x86_64/initrd.img --kopts="console=ttyS0,19200"

Now add a profile for a bare-metal server and tell it what distro and kickstart-file to use for this distro.

cobbler profile add --name=enterprise-centos5 --distro=centos5-x86_64 --kickstart=/var/lib/cobbler/kickstarts/enterprise_centos5-ks.cfg --ksmeta="console=ttyS0,19200"

Since I'm installing remotely and redirecting the console I'm adding a specific system for my bare-metal server based on its MAC address. Cobbler builds PXE-boot menu's for the configured distributions. If you would boot a server without a system profile you would have to navigate the menu and select which profile to install. If you don't select one, it will try to boot the server from the local hard-disk. This can be especially tricky using IPMI-console redirection on a server with an existing OS.

Adding a specific system to cobbler will put a config in /tftboot/pxelinux.cfg/00-XX-XX-XX-XX-XX-XX and when you PXE-boot the server it will automatically boot the right kernel and kickstart option.

cobbler system add --name=00:XX:XX:XX:XX:XX --profile=enterprise-centos5

Running cobbler report gives us a nice overview:

distro          : centos5-x86_64
kernel          : /var/lib/cobbler/images/centos5-x86_64/vmlinuz
initrd          : /var/lib/cobbler/images/centos5-x86_64/initrd.img
kernel options  : {'console': 'ttyS0,19200'}
architecture    : x86
ks metadata     : {'console': 'ttyS0,19200'}
breed           : redhat

profile         : enterprise-centos5
distro          : centos5-x86_64
kickstart       : /var/lib/cobbler/kickstarts/enterprise_centos5-ks.cfg
kernel options  : {}
ks metadata     : {'console': 'ttyS0,19200'}
virt file size  : 5
virt ram        : 512
repos           : []

system           : 00:XX:XX:XX:XX:XX
profile          : enterprise-centos5
kernel options   : {}
ks metadata      : {}
ip address       : None
mac address      : 00:XX:XX:XX:XX:XX
hostname         : 
pxe info set?    : True
config id        : 01-00-XX-XX-XX-XX-XX
netboot enabled? : 1
kickstart        : <>

Running cobbler sync will update /tftboot and /var/www/cobbler to reflect our changes. It will also restart the dhcpd config and restart it.

cobbler sync

sync distro: centos5-x86_64
sync profile: enterprise-centos5
Shutting down dhcpd:                                       [  OK  ]
Starting dhcpd:                                            [  OK  ]

Take a look at the Cobbler manpage if you want to use the more advanced features of Cobbler such as repository mirroring and kickstart templating. Also check out this article on Red Hat Magazine if you would want to use Cobbler without using PXE-boot.

So now we have Cobbler managing our tftp server to PXE-boot our servers. Now we need to remotely (re)boot our bare-metal server and watch our kickstarted install.

Let's telnet to the solproxy daemon running on localhost on port 623 and connect to our IPMI controller on the Poweredge 1850/2850:

[root@spock ~]# telnet localhost 623
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.


Powered by Avocent
BMC Management Utility - SOLProxy v3.0 Release 11
Copyright 2007 - Avocent Corporation(http://www.avocent.com/dell/bmu)


No connection

1:Connect to the Remote Server's BMC
2:Configure the Serial-Over-LAN for the Remote Server
3:Activate Console Redirection
4:Reboot and Activate Console Redirection
5:Help
6:Exit

Please select the item(press 1, 2, 3, 4, 5, 6):

Select 4:Reboot and Activate Console Redirection. This will (re)boot your server and will attempt to connecto the console (when it asks for Key:, just press enter).

Please select the item(press 1, 2, 3, 4, 5, 6):4
Server Address:your_ipmi_ip_address_here
Username:admin
Password:
Key: 

SOLProxy Status:Connected.

Powering on.

Your server will power-on and in a few moments you will be seeing the BIOS and disc controller information scroll by. When you see these it's time to tell the server it needs to try a PXE-boot. You do this by sending the ESC-@ escape sequence which translates in a F12 keyboard command.

Important Note: The SOL-connection is often very flaky and will disconnects. When that happens, just try Activate Console Redirection again. As soon as you see the BIOS etc. send the ESC-@ escape. You should see at least an Attempting PXE-boot statement in the upper-right hand side. This flaky behaviour is common on all my Poweredge 1850/2850 systems.

After a while you should see the Anaconda installer through the console redirection perform the installation using the provided kickstart. Remeber to try to activate the console redirection again if the solproxy software should lose its connection.

Here's my latest kickstart file I use to install a base Centos 5 (including Xen virtualization support) from a http mirror and do some post-install configuration:

#platform=x86, AMD64, or Intel EM64T
# System authorization information
auth  --useshadow  --enablemd5 
# System bootloader configuration
bootloader --location=mbr
# Use text mode install
text
# network configuration. eth0 and its gateway will be used to install centos 5 over the network
network --device eth0 --bootproto static --ip YOUR_PRIVATE_IP --netmask YOUR_PRIVATE_IP_NETMASK --nameserver NS_IP --gateway GATEWAY_IP

# Firewall configuration
firewall --enabled --http --ssh
# Run the Setup Agent on first boot
firstboot --disable
# System keyboard
keyboard us
# System language
lang en_US
# Installation logging level
#logging --level=debug
# Use network installation
url --url http://centos.mirrors.skynet.be/pub/centos/5/os/x86_64
#url --url http://ftp.belnet.be/packages/centos/5/os/x86_64
# Reboot after installation
reboot
#Root password
rootpw --iscrypted A_CRYPTED_PASSWORD_HERE

# SELinux configuration
selinux --enforcing
# Do not configure the X Window System
skipx
# System timezone
timezone --isUtc Europe/Brussels
# Install OS instead of upgrade
install

# Partition clearing information
clearpart --all --initlabel 
# partitioning and LVM physical volume creation
part /boot --fstype "ext3" --size=100
part pv.01 --size=100 --grow
volgroup raid10 --pesize=32768 pv.01
# multiple swap spaces (enabled with swapon /dev/raid10/swap1)
logvol swap --fstype swap --name=swap1 --vgname=raid10 --size=2048
logvol swap --fstype swap --name=swap2 --vgname=raid10 --size=2048
logvol swap --fstype swap --name=swap3 --vgname=raid10 --size=2048
logvol swap --fstype swap --name=swap4 --vgname=raid10 --size=2048
# setup the Logical Volumes
logvol / --fstype ext3 --name=slash --vgname=raid10 --size=5048

%packages
@base
@virtualization

%post
/usr/sbin/useradd -c "A User" -p 'CRYPTED PASSWORD' a_username
# disable useless console ttys
perl -p -i -e 's/(.*tty3)/#\1/' /etc/inittab
perl -p -i -e 's/(.*tty4)/#\1/' /etc/inittab
perl -p -i -e 's/(.*tty5)/#\1/' /etc/inittab
perl -p -i -e 's/(.*tty6)/#\1/' /etc/inittab
# disable services
chkconfig --levels 345 rpcgssd off
chkconfig --levels 345 netfs off
chkconfig --levels 345 rpcidmapd off
chkconfig --levels 345 cups off
chkconfig --levels 345 portmap off
chkconfig --levels 345 rpcsvcgssd off
chkconfig --levels 345 nfslock off
chkconfig --levels 345 gpm off
chkconfig --levels 345 psacct on
chkconfig --levels 345 avahi-daemon off
chkconfig --levels 345 avahi-dnsconfd off
# stop services
service rpcgssd stop
service netfs stop
service rpcidmapd stop
service cups stop
service portmap stop
service rpcsvcgssd stop
service nfslock stop
service gpm stop
service psacct start
service avahi-daemon stop
service avahi-dnsconfd stop
# delete services
chkconfig --del rpcgssd
chkconfig --del netfs
chkconfig --del rpcidmapd
chkconfig --del cups
chkconfig --del portmap
chkconfig --del rpcsvcgssd
chkconfig --del nfslock
chkconfig --del gpm
chkconfig --del avahi-daemon
chkconfig --del avahi-dnsconfd
# install postfix, remove sendmail
yum -y install postfix
chkconfig postfix on
service postfix start
rpm -e sendmail
# Configure some Yum repo's and install some packages
rpm -Uhv http://apt.sw.be/packages/rpmforge-release/rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm
yum -y install dstat iozone screen