IPMI raw notes

Frozen snapshot of the below url saved on our site in case this URL goes away

http://forum.likg.org.ua/linux-specific/ipmi-notes-very-raw-edition-t72....

I did a copy and paste, so all info does not display the same.

Lik`s forum

cPanel/WHM/OpenVZ/Xen Lik`s forum

Skip to content

Advanced search

* Board index ‹ General *nix Server Side Actions ‹ Linux specific
* Change font size
* Print view

* FAQ
* Register
* Login

IPMI notes (very RAW edition)
Linux specific actions
Forum rules
All information is given AS IS
Post a reply
9 posts • Page 1 of 1
IPMI notes (very RAW edition)

Postby lik » Mon Apr 13, 2009 11:12 am
IPMI on linux-based server

Running IPMI on Linux
What is IPMI?

IPMI is standard which allows remote server management, primarily developed by Intel. IPMI cards, known as Baseboard Management Cards (BMCs) are primitive computers in their own right and are operational all the time, so long as the server has a power source. The server itself does not need to be powered on, or the operating system operational for the BMC to work, it just needs a power source to be connected to the server.

The primary benefits of IPMI are:

- View server chassis and motherboard sensor output remotely, such as chassis status and intrusion detection.
- Ability to remotely power on, power off, reboot the server and flash the identification light.
- Ability to set up a console on a serial port and have the BMC redirect that console over a network port, which in cooperation with BIOS level console redirection, gives you the ability to view the BIOS, bootloader, bootup and shutdown procedures and console output should the machine hang or lock up, just as you would if you were interacting with the machine locally. This is called Serial Over Lan (SOL) and is available in IPMI v2.0 as a standard and using non-standard proprietary methods in v1.5.

Essentially, IPMI will save you from a few hundred to over a thousand GBP instead of buying a remote power control unit and SOL will save you the same amount again over buying an IP KVM.

IPMI Revisions

There are currently 3 IPMI revisions:

- IPMI v1.0 - Autonomous access, logging and control. IPMI messaging command sets, sensor data records and event messages. Access through system interfaces like memory mapped IO, I2C bus etc.
- IPMI v1.5 - Ability to send IPMI messages to BMC over LAN, LAN alerting. No SOL as part of the standard's specification but some vendor specific SOL implementations.
- IPMI v2.0 - Serial Over LAN enabling console redirection, access control, enhanced authentication, packet encryption using RCMP+, SMbus interface.

IPMI version 2.0 is desirable as it allows you to use SOL to get a remote console on the server as though it were local in cases where the operating system locks up and SSH or (heaven forbid) telnet access are not available due to the operating system being inoperable. v2.0 also allows you to encrypt the contents of the IPMI packets sent to remote systems and so protects the BMC passwords and your commands on the network. IPMI v1.5 still allows to you to power the system on and off and view sensor output, but does not support packet encryption (and therefore sends your BMC password over the network in plain text) and does not support SOL in any standardised way. Both 2.0 and 1.5 are in common usage and are both still sold on new servers.
Glossary

List of IPMI terms.

- BMC : Baseboard Management Controllers. IPMI compliant micro controllers that handle system event management. These are usually available as cPCI cards.
- GPCagent : A SuperMicro proprietary “Graceful Power” control agent. This agent provides graceful power control features for both Linux and windows platform. GPC means the OS will shutdown gracefully before a power shutdown. It is available from the SuperMicro website. It run as a daemon (smagent) on a Linux based managed system. It requires openIPMI kernel modules to be installed on the managed system to interact with the IPMI device.
- i2c : A low speed (>=400khz) system management interface supported on most embedded systems. It is similar to the SMBus.
- ipmicli : A SuperMicro proprietary command line interface for Linux, similar in function to IPMIView, available from the SuperMicro website.
- IPMItool : An opensource tool for accessing the IPMI device through either local or remote access. Its a command line tool that can used to perform various commands for reading and writing to the IPMI device. It is equivalent to the ipmicli proprietary tool except for console redirection which is not available on this tool.
- IPMIView : A SuperMicro proprietary java applet available from SuperMicro website. Runs on both windows (tested on windows2k) and Linux (fc3) platforms. This runs on the remote system and can be used to interact with the IPMI interface on the managed system. This software provides sensor monitoring, secure login, LAN/IP configuration, chassis power control and console redirection terminal. It also provides a graceful power shutdown/restart option that requires a daemon running on the managed system.
- ipnmac : A SuperMicro proprietary command line tool for Linux to set the IP and mac address for the ipmi interface. This tool can be used to set the address locally on the managed system.
Managed system : system which is to be managed using IPMI. The IPMI card is installed on this machine. IPMI v2.0 supports both local and remote access to the BMC. Local access is provided through a system interface like kCS (IO port). Remote access is provided through the onboard LAN interface (on IPMI supported motherboards).
- OpenIPMI : An opensource IPMI project that maintains linux drivers for the IPMI device. These drivers run on the managed system and provide a local interface to the IPMI card. They also support a primitive command line utility, equivalent to the ipmicli. The utility is meant more as a sample than a working tool.
- Remote system : System from which the IPMI enabled server is managed. This is usually over the network.
- SMbus : System Management bus. A low speed (<100khz) system management interface supported on most PC and server motherboards. It is similar to the I2C bus. The BMC uses the SMbus to communicate with motherboard sensors and Ethernet interface.
- SuperoDoctor : A SuperMicro proprietary IPMI tool. Verision II is a command line tool for local access on the managed system to IPMI interface. Version III is a GUI based tool for local access but works only on windows platform. This requires the openIPMI kernel modules to be installed on the managed machine.

Installation

This guide covers the installation of IPMI tools on Dell 1425 servers and Supermicro servers with a PDSMi+ motherboard. The instructions will be relevant for other server models, but I make no promises. I make references in the links section to Supermicro X7DVL based hardware which comes with a different IPMI BMC but I haven't spent much time investigating the hardware.

For Supermicro servers, open up the server chassis and make a note of the MAC address on the IPMI port. You're supposed need this later when flashing the BMC with it's firmware, however I found that it wasn't necessary to tell the BMC it's MAC address as it already knew and offered it as a default when asking for it. The wise amongst you will write this down anyway and compare it to what the flashing utility says. Beware also that the Supermicro docs erroneously tell you to get the MAC address from the LAN ports, not the BMC socket, this is wrong, you need the MAC written on the BMC's socket.

In the BIOS, set the console redirection to COM2, which is the BMC console port, it doesn't physically exist). Disable BIOS redirection after POST, choose a baud rate (19200 is default on PDSMI+ motherboards, so we used it as a default on everything to keep things tidy, Supermicro X7DVL motherboards which use a SIMIPMI BMC could use a number of different baud rates, your choice is up to you), a terminal type (vt100 works ok for me) and leave the other settings as they are.

Supermicro BMCs require you to boot from a CD and use their tool to flash the BMC prior to use, refer to the FTP link below and go up a few directories to get the latest IPMI CD image (the one shipped with the server caused me problems). Burn the image to a CD and boot from it. Use the utility to install the firmware for your IPMI version and then your motherboard version. I had to guess whether to use RCMP+ (meaning it supports encryption) or RCMP (meaning no encryption). I have the Supermicro AOC-IPMI20-E and it wouldn't work with RCMP+, even though it will upload a firmware for you. It worked after I reflashed it with the RCMP version.

When the process finished, use the ipnmac utility while still booted from the CD, by typing ipnmac (if you're not in the right directory you'll have to navigate using cd and dir commands to find it). Give it a unique IP address from any other interface on the machine or on your network. The BMC needs to be network addressable in its own right. Also give it the MAC address you took from the IPMI socket on the motherboard, not the one written on the LAN ports as suggested by the official Supermicro docs. As I said before, I found that the flashing utility offered the correct MAC address as a default at this stage, but it would be wise to check it against what you wrote down earlier.

Once this step is done, hit CTRL-Alt-Del and remove the CD.

For Dell 1425 servers (and probably other Dells), hit Alt-3 when prompted to enter the BMC setup and give it some unique network settings and some user settings (make your passwords secure!).

You can boot now into Linux, or from a Linux installer CD.

Note on network configuration:

Both Dell 1425s and Supermicro PDSMi+ motherboards use the first LAN port to redirect the IPMI traffic over when using SOL. For this reason, whether you intend to use SOL or not, it is a good idea to use the first LAN ports, almost certainly eth0, as your local network interface, rather than an Internet visible interface.

Once booted into Linux, install openipmi on all machines with a BMC and ipmitool on every machine from which you wish to run IPMI commands locally or to send IPMI commands to a remote machine. Red Hat, FC and Centos users will have to install OpenIPMI and OpenIPMI-tools. SUSE and other Linux users will have figure out what to do themselves for ipmitool. There are IPMItool packages on the website if they're not in your package-shallow distros ;). OpenIPMI is not required to send ipmi commands to remote machines. You only need OpenIPMI where you want to run IPMI commands locally, manage the BMC locally from the OS (which you want to do if you have a BMC in the machine) or I presume to do console redirection over the BMC.

Next you need to load the kernel modules. Fedora/Red Hat/Centos people just need to run setup, open the services tool and check the ipmi box then run /etc/init.d/ipmi start. On Ubuntu I had to do the following, Red Hat and derivatives could try this if their devices aren't found when starting the service. Try modprobing ipmi_si without any options at first, then build up the options if it fails. If you're not sure which method your BMC uses, try leaving out the type= parameter as the module will figure it out.

For kernel 2.6.x:

Code: Select all
modprobe ipmi_msghandler
modprobe ipmi_devintf
modprobe ipmi_si type=kcs ports=0xca8 regspacings=4

If ipmi_si won't load, look at the output of dmidecode for the base address of your IPMI BMC and then use that base address for the ports= module option. The default ports option is 0xca2 so if your BMC is at that address according to dmidecode, then you don't need this option. A SLES 10 user tells me that they did have to specify the ports value 0xca2 on an HP DL380 G5, so perhaps it isn't always the default.

Kernel 2.4 people will have to follow the Debian IPMI instructions, as you're living in a world I haven't encountered with IPMI. You should note that ipmi_si is called ipmi_si_drv and its regspacings option is called si_regspacings. You may also have to make your own device node if you're not using devfs, as documented in the Debian instructions. After modprobing the relevent modules successfully, ls -l /dev/ipmi0 to see if you have a device node before trying to create one.

If this works without errors then Fedora/RH/Centos people are set, Ubuntu/Debian people need to add the modules and options to /etc/modules.conf or maybe add the modprobe commands to /etc/init.d/local if you have no other way.

If you then cat /proc/devices, you should see your IPMI device listed and ls -l /dev/ipmi0 should show your device node.
Setting Up Serial Consoles

Neither of the 2 brands of BMCs I have set up have a physical serial port, they are logical and are managed by the BMC.

You should have set up your BIOS for console redirection earlier, so now we will do the bootloader and init.

To allow your bootloader to redirect over the BMC's serial port, edit /boot/grub/grub.conf, sometimes known as /boot/grub/menu.list add the following lines to grub.conf or menu.list:

Code: Select all
serial --unit=1 --speed=19200 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console

Disable splash screens by commenting out anything starting with splash outside of the OS boot menu section and remove any splash options from your kernel lines. A text console can't display them.

To make kernel messages output over your BMC, add console=tty0 console=ttyS1,19200n8r to the end of your kernel lines, so it should look something like the following:

title kernel 2.6.15-26-amd64-server
root (hd0,0)
kernel /vmlinuz-2.6.15-26-amd64-server root=/dev/sda2 ro quiet console=tty0 console=ttyS1,19200n8r
initrd /initrd.img-2.6.15-26-amd64-server
savedefault
boot

This gives you a serial console on the second serial port, which should be your BMC's serial port, at 19200Kb per second, which should match what you chose in the BIOS. The order of the console options above is important. The last listed will be the system's default console which will display the boot messages and kernel errors. This means that after halfway through the boot process, during shutdown and when there are kernel errors, only the serial console will see the messages. As you want to work remotely, this is the way it has to be. You can't have more than one default console. You can however interrupt grub at boot time and edit the kernel line for a single boot with different parameters if you need to see the default console locally but remember that they will persist until you reboot, which means that if you reboot but still want to see them locally, you'll have to interrupt and edit grub at boot time again and also, to see them remotely again, you need to remember to reboot.

About halfway through the bootup procedure, once the kernel has booted and loaded drivers for your hardware and has mounted the hard disks and so on, the bootloader hands over to init which brings up your services and network configurations etc. To get init and therefore your booting/booted Linux system redirecting the console over the BMC's serial port, edit /etc/inittab and add the following line to the console section for Debian/Ubuntu:

S1:2345:respawn:/sbin/getty -L ttyS1 19200 vt100

For Fedora/Red Hat/CentOS:

S1:2345:respawn:/sbin/agetty -h ttyS1 19200 vt100

This gives you a serial console on the second serial port, which should be your BMC's serial port, for the BIOS as configured earlier, the bootloader and init. Again, change 19200 to whatever you chose in your BIOS.

Users of Ubuntu (and presumably Debian and it's other derivative distributions) can read https://help.ubuntu.com/community/IPMI for assistance with all of this stuff. I guess it will be useful for everybody else too, if you can recognise where it is Ubuntu specific.

IPMI Commands

You should then be able to run some IPMI commands locally (probably need to be root as root owns the device node):

Code: Select all
ipmitool -I open chassis power status
Chassis Power is on

ipmitool -I open chassis status

System Power : on
Power Overload : false
Power Interlock : inactive
Main Power Fault : false
Power Control Fault : false
Power Restore Policy : always-off
Last Power Event :
Chassis Intrusion : inactive
Front-Panel Lockout : inactive
Drive Fault : false
Cooling/Fan Fault : false
Sleep Button Disable : allowed
Diag Button Disable : allowed
Reset Button Disable : allowed
Power Button Disable : allowed
Sleep Button Disabled: true
Diag Button Disabled : true
Reset Button Disabled: true
Power Button Disabled: true

Command dissection:

Code: Select all
ipmitool -I open chassis power status
-I open - use the local openipmi interface. It seems that when the interface is local, it can be omitted from the command and the command will default to the local interface.
chassis - run a command from the chassis set of commands.
power - run a command relating to power.
status - show the status of the chassis power status.

Thats pretty simple.

Assuming you get a local response, then try a remote command from another machine, you won't need to be root as you are talking to the BMC directly, rather than through the OS. IPMI 2.0 BMCs support both encrypted (RCMP+) and non-encrypted (RCMP) IPMI traffic sent over the network, 1.5 BMCs only support non-encrypted traffic. Encrypted lan traffic is specified by the lanplus parameter, where as non-encrypted network traffic is specified by the lan parameter.

Code: Select all
ipmitool -I lanplus -H 192.168.10.123 -U root -a chassis power status
Password:
Chassis Power is on

ipmitool -I lanplus -H 192.168.10.123 -U root -a chassis status
Password:
System Power : on
Power Overload : false
Power Interlock : inactive
Main Power Fault : false
Power Control Fault : false
Power Restore Policy : always-off
Last Power Event :
Chassis Intrusion : inactive
Front-Panel Lockout : inactive
Drive Fault : false
Cooling/Fan Fault : false
Sleep Button Disable : allowed
Diag Button Disable : allowed
Reset Button Disable : allowed
Power Button Disable : allowed
Sleep Button Disabled: true
Diag Button Disabled : true
Reset Button Disabled: true
Power Button Disabled: true

Command dissection:

Code: Select all
ipmitool -I lanplus -H 192.168.10.123 -U ipmiadmin -a chassis power status
-I lanplus - Run the command over the LAN with encryption for IPMI 2.0 hosts. You can use ''-I lan'' to send commands over the lan for 1.5 hosts as they don't support the encryption used by lanplus. 2.0 hosts can also use ''-I lan'' but why would you want to when 2.0 hosts can use encryption.
-H 192.168.10.123 - -H means host followed by an IP address or hostname.
-U root - -U specifies a user, followed by the username set up on the BMC, this is not a local or remote OS user.
-a - prompt for password. You can also possible to supply the password as an environment variable instead of using -a, but I guess this is less secure.
chassis power status - as per the local command dissection.

If that all works for you then you can a woop and a holler.

IPMItool has a familiar UNIX bash shell or Cisco IOS syntax, which allows you to complete the command as you go along. If you type in an incomplete command, IPMItool will provide you with a list of options that can be used to complete your command.

Simply typing ipmitool -I open will offer the top-level sub-commands, which you can then choose from to add to your command.

Code: Select all
ipmitool -I open
No command provided!
Commands:
raw Send a RAW IPMI request and print response
i2c Send an I2C Master Write-Read command and print response
lan Configure LAN Channels
chassis Get chassis status and set power state
event Send pre-defined events to MC
mc Management Controller status and global enables
sdr Print Sensor Data Repository entries and readings
sensor Print detailed sensor information
fru Print built-in FRU and scan SDR for FRU locators
sel Print System Event Log (SEL)
pef Configure Platform Event Filtering (PEF)
sol Configure IPMIv2.0 Serial-over-LAN
isol Configure IPMIv1.5 Serial-over-LAN
user Configure Management Controller users
channel Configure Management Controller channels
session Print session information
sunoem OEM Commands for Sun servers
shell Launch interactive IPMI shell
exec Run list of commands from file
set Set runtime variable for shell and exec[/quote]
[code]ipmitool -I open user
User Commands: summary []
list []
set name
set password []
disable
enable
test <16|20> [[/code]
[code]ipmitool -I open chassis
Chassis Commands: status, power, identify, policy, restart_cause, poh, bootdev

ipmitool -I open chassis power
chassis power Commands: status, on, off, cycle, reset, diag, soft[/code]
…And so on.

Beware, powering the server off or rebooting does not do a graceful shutdown like the OS would. They are the same as holding in the power button or pressing the reboot button. Supermicro offer a graceful shutdown daemon for Windows and Linux which will respond to the graceful showdown and reboot commands provided by Supermicro's graphical Java IPMI control application IPMIview.

[b]Setting Up the BMC[/b]

The Supermicro BMCs send out arp requests gratuitously which can degrade performance, so we will turn them off:
[code]ipmitool lan set 1 arp generate off[/code]
We will also set the BMC's netmask as well as it's default and backup gateways:
[code]ipmitool lan set 1 netmask 255.255.255.0
ipmitool lan set 1 defgw ipaddr 192.168.1.1
ipmitool lan set 1 bakgw ipaddr 192.168.1.2[/code]
You can also set the gateway MAC addresses if you want to.

And now set it's SNMP community name so that we can send SNMP traps:
[code]ipmitool lan set 1 snmp [/code]

[b]Getting a Remote Console Using IPMI[/b]

To get a Serial Over LAN console, assuming you have set up the various BMC redirections correctly, run:
[code]ipmitool -I lanplus -H 192.168.10.123 -U ipmiadmin -a sol activate[/code]
SOL only works on IPMI 2.0 BMCs and so only works with devices which support the encrypted lanplus interface (though my Supermicro's would do it without encryption using the graphical tool described below).

If SOL doesn't work or you simply want to quit your SOL session, you can use the key sequence:
[code]~.

to get out. It seems that inside a SOL session, all of ipmitool's SOL session commands start with the character ~.

Alternatively you may use a graphical tool as described below.

Graphical IPMI

Graphical IPMI tools make life easier as you don't have to remember commands or syntax and as most GUI tools are provided by the IPMI device vendors, they may offer vendor specific functionality that isn't available in the generic command line tools. Of course, in all cases except regarding vendor specific (and thus not standards compliant) functionality, the command line tools should provide all necessary functionality and so GUI tools are not essential to get IPMI working.

This section describes the installation and use of Supermicro's IPMIView. I am told that Dell's OpenManage suite also offers some GUI IPMI tools but I can't vouch for that as the Dell servers I used didn't seem to come with OpenManage. There are surely other GUI IPMI tools from other vendors, I just don't know about them.

There is a Java application from Supermicro called IPMIView which allows you to graphically manage servers with IPMI BMCs. The primary advantage of this tool is that SOL works easily and you can issue graceful reboot and shutdown commands if you install Supermicro's graceful shutdown daemon. The version of IPMIView which is shipped on the CD (as of late 2007) doesn't work on FC6 or F7, however the latest version from the FTP site does. Another version, shipped on the Supermicro FTP site for our dual processor machines is at ftp://ftp.supermicro.com/CDR-SIMIPMI_1. ... inistrator and this appears to work with Fedora 7 also.

Install as a regular user using sudo from the command line (by uncommenting the following line from /etc/sudoers if need to):

%wheel ALL=(ALL) ALL

…and adding your user to the wheel group:

usermod -G wheel

Make the installer executable:

chmod +x ./IPMIView-Linux_2.6.31_071005.bin

and run it as sudo:

sudo ./IPMIView-Linux_2.6.31_071005.bin

It should now run. When the install finishes you need to chmod +x the bundled JRE directory as it's root only by default:

sudo chmod -R +x /opt/SUPERMICRO/IPMIView/_jvm/

Now you should be able to run

/opt/SUPERMICRO/IPMIView/IPMIView20.bin

as a regular user and create a desktop icon for it by right-clicking the desktop, clicking Create Launcher and providing the command as above and any other meta-data you wish to add, like description, name and and a nice icon from the installation directory.

Using the IPMIView application is covered in the Supermicro documentation (seems to be the only thing covered in them too…)
Adding an IPMI User With IPMIView

You should now be able to search for the host's IPMI device address in IPMIView and log in with the username and password of ADMIN. You should add a new user, change it's user level to administrator and delete the ADMIN user.
Links
IPMI Standards

Intel IPMI page

Intel IPMI Standards Page
IPMI Software

IPMItool

OpenIPMI Linux IPMI kernel module support

Link to Supermicro patched version of IPMItool

Supermicro FTP server, Linux IPMI tools directory (version number in URL subject to change)

Newer version of IPMITool (version number in URL subject to change)

Supermicro bootable iso images for flashing BMCs (get the one with IPMI_Server_Managment.iso in the name)
IPMI Software Documentation

IPMItool Manual

OpenIPMI kernel module options

Supermicro IPMI manuals (version number in URL subject to change)

Supermicro IPMI Manuals for Supermicro X7DVL motherboard with SIM IPMI devices (version number in URL subject to change)

The April 2008 issue, number 89, of Linux Pro Magazine (aka Linux Magazine outside of Canada and the USA), contains a decent IPMI article written by Justin Penney. This doesn't appear to be web visible at the moment.
IPMI Howtos

Ubuntu IPMI documentation New for Ubuntu 8.04

Older Debian IPMI howto

Supermicro IPMI on Linux Contains errata on the Supermicro Docs

Another Linux IPMI doc

Yet another Linux IPMI doc
Vendor IPMI Documentation

Dell IPMI howto Contains useful sample commands.
Other IPMI Documentation

Cern IPMI presentation PDF containing demonstration of IPMI usage, including reference to a version of IPMItool patched by Supermicro.
Related Documentation
IPMI

This is intended to help fairly knowledgeable people get IPMI working on their hosts so they can issue remote commands to their hardware. I focus on Red Hat Enterprise Linux on a Dell, but it is likely to work on other hosts, distributions, and OSes, too. This works for me on Dell PowerEdge 1850, PowerEdge 2850, Dell PowerEdge 1950, and Dell PowerEdge 2950 hardware.

Dell PowerEdge 1650, 2650, and 1750 servers have an older implementation of IPMI which will let you issue commands locally, but not to these models over the network.

Before you begin:

The Baseboard Management Controller (BMC) is the thing that implements IPMI. It piggybacks on the first built-in NIC so you have to have that attached to the network on the hosts you wish to manage. It uses its own IP address (so you need an extra one).

If you are new to this get a server that is nearby to act as your test machine. Most of the IPMI commands that shut the host down also kill the BMC. If that happens you’ll need to go power the machine on manually. Until you figure out exactly what is okay and what isn’t you’ll be pushing the power button a lot.

You will need a Linux host to send the IPMI commands from. Undoubtedly there are ways to send these commands from other operating systems, but as I am a Linux guy I’ll use that as my example. Feel free to post comments addressing other OSes.

Getting the OS prepared:

1) Install IPMItool and the startup scripts. On Red Hat Enterprise Linux install the OpenIPMI, OpenIPMI-tools, OpenIPMI-libs, and OpenIPMI-devel packages. That will get you everything you need. There are similar packages available for other distributions (SuSE, Ubuntu, CentOS, etc.). You’ll need IPMItool on any machine you want to configure, and any machine you want to send commands from.

2) Enable the IPMI service:

/sbin/chkconfig ipmi on

3) Start the IPMI service, which will load the kernel modules for you:

/sbin/service ipmi start

Configure the BMC for Remote Usage:

1) There are two ways to configure the BMC. You can configure it through the boot-time menu (Ctrl-E), where you can set the management password and IP address information. Or, you can configure it with ipmitool from the OS. Replace my sample IP address, gateway, and netmask with your own:

/usr/bin/ipmitool -I open lan set 1 ipaddr 192.168.40.88
/usr/bin/ipmitool -I open lan set 1 defgw ipaddr 192.168.40.1
/usr/bin/ipmitool -I open lan set 1 netmask 255.255.255.0
/usr/bin/ipmitool -I open lan set 1 access on

2) Secure the BMC, so unauthorized people can’t power cycle your machines. To do this you want to change the default SNMP community, the “null” user password, and the root user password. First, set the SNMP community, either to a random string or something you know:

/usr/bin/ipmitool -I open lan set 1 snmp YOURSNMPCOMMUNITY

Then set the null user password to something random. Replace CRAPRANDOMSTRING with something random and secure:

/usr/bin/ipmitool -I open lan set 1 password CRAPRANDOMSTRING

Last, set the root user password to something you know:

/usr/bin/ipmitool -I open user set password 2 REMEMBERTHIS

Double-check your settings with:

/usr/bin/ipmitool -I open lan print 1

Trying it:

1) You can set an environment variable, IPMI_PASSWORD, with the password you used above. That will save some typing:

export IPMI_PASSWORD="REMEMBERTHIS"

If you use this substitute the “-a” in the following commands with a “-E”.

2) From another machine issue the following command, obviously replacing the IP with the target BMC’s IP:

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power status

You should get something like:

Chassis Power is on

If you get anything else, or nothing, double-check to make sure the BMC is set right, you entered the right password, and the IP it has is reachable from the machine you’re on. You can double-check your work via the Ctrl-E boot menu, too.

Beyond that, get familiar with:

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power off

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power cycle

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a sel list

For me, a “chassis power off” command kills the box. “SEL” is the system event log.

You can issue all of these commands locally, too:

/usr/bin/ipmitool sel list

Hopefully this helps a little. If you find any errors in this please leave me a comment or send me an email. Thanks!
Lik G

lik
Site Admin
Site Admin

Posts: 325
Joined: Sun Aug 10, 2008 9:32 am
Location: Kharkov

* Website
* ICQ

Top
Re: IPMI notes

Postby lik » Mon Apr 27, 2009 9:30 am
How-to setup IPMI SOL for Linux based Dell (and some other) servers

To setup IMPI Console
For Fedora/Red Hat/CentOS:

Edit the file /boot/grub/grub.conf, as follows. See the sample file that follows, which shows the changes described in this procedure.
1. Add the following two new lines in the general settings section of the file:
serial —unit=0 —speed=19200
terminal —timeout=10 serial
2. Append two options to the kernel line:
kernel …………. console=ttyS#, where ttyS# is dependant on which COM port is used and is dependaint on the chassis type.
3. If the /etc/grub/grub.conf contains a splashimage directive, you must comment it out.

Below is the maximum baud rate and Com port for each server chassis:
• 19200 for PowerEdge 1425SC and other PowerEdge x8xx systems using Com 1 (ttyS0)
• 57600 for PowerEdge x9xx systems without Dell Remote Access Controller 5 (DRAC 5) Using Com 2 (ttyS1)
• 115200 for PowerEdge x9xx systems with DRAC 5 Using Com 2
• 9600 is the Safe-Mode. Meaning that it should always work

Table D-1. Sample File: /boot/grub/grub.conf
# grub.conf generated by anaconda
#
Note that you do not have to rerun grub after making changes to this file
# NOTICE: You do not have a /boot partition. This means that
# all kernel and initrd paths are relative to /, e.g.
# root (hd0,0)
# kernel /boot/vmlinuz-version ro root=/dev/sdal
# initrd /boot/initrd-version.img
#boot=/dev/sda
default=0
timeout=10
#splashimage=(hd0,2)/grub/splash.xpm.gz

serial —unit=0 —speed=9600
terminal —timeout=10 serial

title Red Hat Linux Advanced Server (2.4.9-e.3smp)
root (hd0,0)
kernel /boot/vmlinuz-2.4.9-e.3smp ro root=/dev/sda1 hda=ide-scsi console=tty0 console=ttyS1,9600
initrd /boot/initrd-2.4.9-e.3smp.img
title Red Hat Linux Advanced Server-up (2.4.9-e.3
root (hd0,00)
kernel /boot/vmlinuz-2.4.9-e.3 ro root=/dev/sda1
initrd /boot/initrd-2.4.9-e.3.im

This example assumes use of COM2 and a baud of 9600

Additional information for editing the grub.conf:
1. You may need to disable GRUB’s graphical interface and use the text-based interface instead; otherwise, the GRUB screen will not be displayed in BMU console redirection. To do so, comment out the line starting with splashimage.
2. If you have multiple options in GRUB and you want all of them to start a console session through the BMU serial connection, add console=ttyS#, to all options. The preceding example shows console=ttyS#, added to only the first option.

Enabling Login to the Console After Boot
Edit the file /etc/inittab, as follows:
Add a new line to configure a getty on the COM1 serial port:

co:2345:respawn:/sbin/agetty -h -L 19200 ttyS1 vt100

See the sample file, which shows the addition of the new line.

Table D-2. Sample File: /etc/innitab
#
# inittab This file describes how the INIT process should set up
# the system in a certain run-level.
#
# Author: Miquel van Smoorenburg,
# Modified for RHS Linux by Marc Ewing and Donnie Barnes
#

# Default runlevel. The runlevels used by RHS are:
# 0 - halt (Do NOT set initdefault to this)
# 1 - Single user mode
# 2 - Multiuser, without NFS (The same as 3, if you do not have
# networking)
# 3 - Full multiuser mode
# 4 - unused
# 5 - X11
# 6 - reboot (Do NOT set initdefault to this)
#
id:3:initdefault:

# System initialization.
si::sysinit:/etc/rc.d/rc.sysinit

l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
l3:3:wait:/etc/rc.d/rc 3
l4:4:wait:/etc/rc.d/rc 4
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6

# Things to run in every runlevel.
ud::once:/sbin/update

# Trap CTRL-ALT-DELETE
ca::ctrlaltdel:/sbin/shutdown -t3 -r now

# When our UPS tells us power has failed, assume we have a few
# minutes of power left. Schedule a shutdown for 2 minutes from now.
# This does, of course, assume you have power installed and your
# UPS is connected and working correctly.
pf::powerfail:/sbin/shutdown -f -h +2 “Power Failure; System Shutting Down”
# If power was restored before the shutdown kicked in, cancel it.
pr:12345:powerokwait:/sbin/shutdown -c “Power Restored; Shutdown Cancelled”

# Run gettys in standard runlevels
ag:12345:respawn:/sbin/agetty -h -L ttyS# vt100
1:2345:respawn:/sbin/mingetty tty1
2:2345:respawn:/sbin/mingetty tty2
3:2345:respawn:/sbin/mingetty tty3
4:2345:respawn:/sbin/mingetty tty4
5:2345:respawn:/sbin/mingetty tty5
6:2345:respawn:/sbin/mingetty tty6

# Run xdm in runlevel 5
# xdm is now a separate service
x:5:respawn:/etc/X11/prefdm –nodaemon

Edit the file /etc/securetty, as follows:
If you need to use Com 2 chances are you will need to edit /etc/securetty and add the following on the first available line:
ttyS1
or you can just run echo ttyS1 >> /etc/securetty (You must use two >’s. Failure to do so will overwrite the file with ttyS1)

Next you will need to ensure that IPMI is setup. To do so run the following:

ipmitool lan set 1 ipaddr $IPMI-IP
ipmitool lan set 1 netmask $NETMASK
ipmitool lan set 1 defgw ipaddr $GATEWAY
ipmitool lan set 1 user
ipmitool lan set 1 access on
ipmitool lan set 1 arp respond on
ipmitool mc reset warm

where:
IPMI-IP is the IP listed in the IPMI interface ip type
NETMASK is the netmask of the subnet
GATEWAY is the primary gateway for the subnet

For the Penguin Atlus 1300 the lan channel is 6 rather than 1

For the BIOS you may have to:
find the Serial Redirect setting, if there is one, and make sure it is set to redirect after post/redirect in os/redirect after boot
set the redirect to go over NIC/BMC/IP rather than tweak the serial speed to 19200 (9600 on non dell).

Example - Dell:1425

Example – Most Dell x9xx
Lik G

lik
Site Admin
Site Admin

Posts: 325
Joined: Sun Aug 10, 2008 9:32 am
Location: Kharkov

* Website
* ICQ

Top
Re: IPMI notes

Postby lik » Mon Apr 27, 2009 11:00 am
http://www.tldp.org/HOWTO/Remote-Serial-Console-HOWTO/
Lik G

lik
Site Admin
Site Admin

Posts: 325
Joined: Sun Aug 10, 2008 9:32 am
Location: Kharkov

* Website
* ICQ

Top
IPMI

Postby lik » Tue Jun 09, 2009 9:02 am
With Ubuntu/Debian, the IPMITOOL script seems to have an error with DRAC3, which doesn't work the KCS module.
Here are some instructions that worked for Ubuntu, or might be required for newer versions of Debian on machines with a DRAC3 card:

1. Install IPMI

Code: Select all
apt-get install ipmitool
/usr/share/ipmitool/ipmi.init.basic

If this throws an error as follows ...

Code: Select all
Setting up OpenIPMI driver...
FATAL: Module ipmi_kcs_drv not found.

... then make a new file for debian/ubuntu:

Code: Select all
vi /usr/share/ipmitool/ipmi.init.basic

The top section looks like this:

Code: Select all
# load the ipmi modules
modprobe ipmi_msghandler
modprobe ipmi_devintf
if ! modprobe ipmi_kcs_drv ; then
modprobe ipmi_si # try new module name
fi

This script check doesn't work, and throws an error that it can't load the KCS driver.
Change it to:

Code: Select all
modprobe ipmi_msghandler
modprobe ipmi_devintf
modprobe ipmi_si # try new module name

Save as:

Code: Select all
/usr/share/ipmitool/ipmi.init.debian
chmod 755 /usr/share/ipmitool/ipmi.init.debian

Run this script at startup.

You may load ipmi_devintf and ipmi_si. The module ipmi_msghandler will be loaded automatically.

Code: Select all
sudo echo "ipmi_devintf" >> /etc/modules
sudo echo "ipmi_si" >> /etc/modules

... and it works.
Lik G

lik
Site Admin
Site Admin

Posts: 325
Joined: Sun Aug 10, 2008 9:32 am
Location: Kharkov

* Website
* ICQ

Top
IPMI on FreeBSD servers

Postby lik » Tue Jun 09, 2009 10:37 am
IPMI

The Intelligent Platform Management Interface (IPMI) specification defines a set of common interfaces to a computer system which system administrators can use to monitor system health and manage the system. Several dozen companies support IPMI. Dell, HP, Intel Corporation and NEC Corporation announced IPMI v1.0 on 1998-09-16, v1.5 on 2001-03-01, and v2.0 on 2004-02-14.

IPMI operates independently of the operating system (OS) and allows administrators to manage a system remotely even in the absence of the OS or the system management software, or even if the monitored system is not powered on. IPMI can also function when the OS has started, and offers enhanced features when used with the system management software. IPMI gives only the structure and format of the interfaces as a standard; implementation may vary.

For details, see http://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface

Installing

Before you can use IPMI on a FreeBSD server, you will need to load the impi kernel module (or compile it in the kernel).

Code: Select all
kldload ipmi
kldstat

OR

Code: Select all
vi /boot/loader.conf
ipmi_load="YES"

Code: Select all
shutdown -r now

Next you will need to install the ipmitool client to talk ipmi and get information about your server.

Code: Select all
cd /usr/ports/sysutils/ipmitool
make install clean

If your server hardware has a BMC (board management controller), then the following commands should work for you:

Code: Select all
ipmitool -I open chassis status
ipmitool -I open sel list
ipmitool -I open sdr type temperature
ipmitool -I open fru print

Supported Hardware

Dell: http://linux.dell.com/ipmi.shtml

On Dell 1750 systems, support for IMPI 1.0 is available using the ERA/O card. But after testing on Freebsd 7.0 STABLE, the BIOS doesn't seems to send information about the IPMI BMC. So it must be added manually via:

Code: Select all
vi /boot/device.hints
hint.ipmi.0.at=isa0
hint.ipmi.0.mode=KCS

Lik G

lik
Site Admin
Site Admin

Posts: 325
Joined: Sun Aug 10, 2008 9:32 am
Location: Kharkov

* Website
* ICQ

Top
Re: IPMI notes (very RAW edition)

Postby lik » Mon Oct 12, 2009 3:35 pm
http://www.freebsdwiki.net/index.php/IPMI
Lik G

lik
Site Admin
Site Admin

Posts: 325
Joined: Sun Aug 10, 2008 9:32 am
Location: Kharkov

* Website
* ICQ

Top
SuperMicro IPMI Text Console Redirection under Linux

Postby lik » Sat Dec 12, 2009 1:25 am
Instructions of IPMI Text Console Redirection under Linux

(1) Host A with IPMI BMC installed:
a) BIOS POST:
(i) Enable "Console Redirection" in BIOS Setup.

For example, COM2 / 19.2Kbps / 8N1

(ii) Disable "Enable Console Redirection after POST" in BIOS setup.

b) BOOT LOADER:
(i) For GRUB, add the following TWO lines into /boot/grub/grub.conf, but comment out "splashimage=(hd0,0)/grub/splash.xpm.gz"

Code: Select all
serial --unit=1 --speed=19200 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console
#splashimage=(hd0,0)/grub/splash.xpm.gz

(ii) Then add "serial console=ttyS1,19200n8" to the end of kernel /vmlinuz in /boot/grub/grub.conf.
For example:

Code: Select all
kernel /vmlinuz-2.6.5-1.358smp ro root=LABEL=/ rhgb quiet serial console=ttyS1,19200n8

This will result all boot message to be output to console ttyS1, and you will not see
all these boot messages in local console until login message promt up.

c) LINUX OS:
(i) Add the following line into /etc/inittab.

Code: Select all
s0:2345:respawn:/sbin/agetty ttyS1 19200

(ii) Edit /etc/securetty and add ttyS1
(2) Host B with IPMIView installed:
a) Install and run IPMIView.
b) Log in Host A with IPMI BMC installed as Admin.
c) Start Console Redirection in IPMIView right after the Host A reboots.
You will see the BIOS POST, the boot loader, and the Linux OS messages and
prompts.
Lik G

lik
Site Admin
Site Admin

Posts: 325
Joined: Sun Aug 10, 2008 9:32 am
Location: Kharkov

* Website
* ICQ

Top
IPMI Management Utilities projects

Postby lik » Tue Jun 01, 2010 10:25 am
Links with information related to the IPMI Management Utilities project.

ipmiutil project, sourceforge http://ipmiutil.sourceforge.net (current)
panicsel project, sourceforge http://panicsel.sourceforge.net (old)
IPMI Specification http://www.intel.com/design/servers/ipmi/index.htm
OSDL Carrier Grade Linux http://www.osdl.org/lab_activities/carrier_grade_linux/
Intel imb driver source http://downloadfinder.intel.com/scripts ... ver*source
Intel imb driver for Windows http://downloadfinder.intel.com/scripts ... imb+driver
OpenIPMI project http://sourceforge.net/projects/openipmi/
by Corey Minyard of MontaVista (home= http://openipmi.sourceforge.net)
ipmitools project by San Mehat http://sourceforge.net/projects/ipmitools/
valinux IPMI driver http://cvs.sf.net/cgi-bin/viewcvs.cgi/i ... hes/2.4.x/
GNU FreeIPMI library project http://www.gnu.org/software/freeipmi/
LANDesk IPMI driver/daemon http://www.landesk.com/Support/ (or see
Intel System Resource CD for Intel servers)
lm-sensors project http://secure.netroedge.com/~lm78/
dmidecode project http://www.nongnu.org/dmidecode/
IPMI vendor/mfg IDs http://www.iana.org/assignments/enterprise-numbers
Lik G

lik
Site Admin
Site Admin

Posts: 325
Joined: Sun Aug 10, 2008 9:32 am
Location: Kharkov

* Website
* ICQ

Top
Update IPMI firmware on Supermicro X7xxx/X8xxx motheboards

Postby lik » Fri Aug 06, 2010 5:13 am
IPMI update instructions for Supermicro X7DVL:

1. Download firmware update on your local PC:

Code: Select all
wget "ftp://ftp.supermicro.com/CDR-SIMIPMI_1.13_for_SIM_IPMI/Firmware/SIMxx/ugsim160.bin"

2. Print and backup current IPMI settings:

Code: Select all
ipmitool lan print

3. Disable IPMI driver (OpenIPMI or FreeIPMI) on the server and keep it so during the process of firmware update:

Code: Select all
/etc/init.d/ipmi stop
/etc/init.d/ipmievd stop

4. Log into IPMI web interface.
5. Go to the tab 'Maintenance' and click on 'Firmware Update'.
6. Click on the button 'Enter Update Mode'.
7. Choose the file with the firmware (ugsim160.bin downloaded at step #1) and click on the button 'Upload Firmware'.
8. Click on the button 'Start Upgrade'.
After that you will see a popup message: “Please wait 1 minute” and when you click on the button 'OK' on this popup message IPMI will reboot itself.
9. In case IPMI interface isn't accessible after update you will have to reset IPMI settings to the factory default with the help of IPMICFG utility.

IPMI update instructions for Supermicro X8DTL-IF/X8DTL-3F/X8DTL-6F:
1. Download firmware update on your local PC for your motherboard from http://www.supermicro.com/support/bios/Firmware.aspx
For X8DTL, for example, it would be IPMI8DTL131.zip
2. Print and backup current IPMI settings:

Code: Select all
ipmitool lan print

3. Disable IPMI driver (OpenIPMI or FreeIPMI) on the server and keep it so during the process of firmware update:

Code: Select all
/etc/init.d/ipmi stop
/etc/init.d/ipmievd stop

4. Log into IPMI web interface.
5. Click on the 'Maintenance' icon in the Menu bar.
6. Click on 'Enter Update Mode' button to enter the update mode. Follow the instruction on the page to browse and update your firmware on the IPMI card. Your IPMI firmware image has .bin extension (Aten_131_03092010.bin). Once device has entered the firmware update mode, the device will be reset even if you cancel the process of firmware updating.
7. In case IPMI interface isn't accessible after update you will have to reset IPMI settings to the factory default with the help of IPMICFG utility.

IPMICFG utility - how to reset IPMI to the factory defaults

In case if IPMI interface isn't accessible after update you will have to do the following:

1. Upload IPMICFG tool to the server (you will find required binaries inside the archive):

Code: Select all
wget "ftp://ftp.supermicro.com/utility/IPMICFG/Linux/IPMICFG-Linux_v1.20.zip"

2. Unload(if any) IPMI driver (OpenIPMI or FreeIPMI) on the server:

Code: Select all
/etc/init.d/ipmi stop
/etc/init.d/ipmievd stop

3. Run the following commands:

Reset to the factory defaults:

Code: Select all
./ipmicfg-linux.x86_64 -fd

Disable the DHCP:

Code: Select all
./ipmicfg-linux.x86_64 -dhcp off

Set IP (format:###.###.###.###):

Code: Select all
./ipmicfg-linux.x86_64 -m

Set Subnet Mask (format:###.###.###.###):

Code: Select all
./ipmicfg-linux.x86_64 -k

Set Gateway IP (format:###.###.###.###):

Code: Select all
./ipmicfg-linux.x86_64 -g

4. Log into IPMI using web interface and perform its basic tune. (default login credentials are Login: ADMIN; Password: ADMIN).
Lik G

lik
Site Admin
Site Admin

Posts: 325
Joined: Sun Aug 10, 2008 9:32 am
Location: Kharkov

* Website
* ICQ

Top
Display posts from previous: Sort by
Post a reply
9 posts • Page 1 of 1

Return to Linux specific
Jump to:

*

Related topics
Replies
Views
Last post

*

Useful MySql notes
by lik » Tue Mar 17, 2009 11:02 am
1 Replies
84 Views
Last post by lik View the latest post
Thu Apr 09, 2009 5:35 pm
Server Side Actions

*

DDoS notes
by lik » Thu Oct 01, 2009 3:53 am
0 Replies
30 Views
Last post by lik View the latest post
Thu Oct 01, 2009 3:53 am
Interesting links

*

OpenSSL notes
by lik » Tue Jan 26, 2010 3:09 pm
1 Replies
48 Views
Last post by lik View the latest post
Wed May 19, 2010 11:43 am
Server Side Actions

*

Fantastico De Luxe install notes
by lik » Tue Dec 09, 2008 9:41 am
0 Replies
244 Views
Last post by lik View the latest post
Tue Dec 09, 2008 9:41 am
Server Side Actions

* Board index
* The team • Delete all board cookies • All times are UTC + 2 hours [ DST ]

* News News
* Site map Site map
* SitemapIndex SitemapIndex
* RSS Feed RSS Feed
* Channel list Channel list

Powered by BB3 © Group

SEO improved
Rambler's Top100
cron