wiki:ipmi

Version 5 (modified by Jamie McClelland, 2 months ago) (diff)

--

IPMI

IPMI allows us to access a physical server removetly, as if we were in the colo with a monitor and keyboard plugged in. At least in theory. We used to use a simple serial cable to achieve this feat. However, new servers are no longer beings shipped with serial ports and motherboard/bios software no longer support them. Instead, we are expected to use IPMI.

IPMI provides a million ways to connect, but we are enabling two. Both are configured to use a local, non-routable IP address.

  • web-based access (we will setup nginx to proxy from a public address to the non-routable address)
  • serial over LAN access, using ipmiutil which will be run from a server in the cabinet

Unfortunately, IPMI is complex and varies considerably from computer to computer. Here are the steps used to connect john.

BIOS/Setup

When you first boot the computer, IPMI has to be enabled and assigned an IP address.

We use a non-routable IP address so the only way to connect to IPMI is via a machine in the colo itself.

Below are the screen shots of the bios and their settings.

Enable http redirections

On robideau, we have the following nginx configuration to allow web-based proxying:

#server {
#  listen 80;
#  server_name ipmi.john.mayfirst.org;
#  root /var/www/html;
#  location /.well-known/acme-challenge {
#    try_files $uri $uri/ =404;
#  }
#  location / {
#    return 301 https://$host$request_uri;
#  }
#}

server {
  listen 443;
  server_name ipmi.john.mayfirst.org;
  location /.well-known {
    root /var/www/html;
  }
  location / {
    access_log off;
    proxy_pass http://192.168.56.3:80;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
  }

  ssl on;
  ssl_certificate_key /etc/letsencrypt/live/ipmi.john.mayfirst.org/privkey.pem;
  ssl_certificate /etc/letsencrypt/live/ipmi.john.mayfirst.org/fullchain.pem;

  ssl_ciphers 'AES128+EECDH:AES128+EDH:!aNULL';

  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_session_cache shared:SSL:10m;

  ssl_prefer_server_ciphers on;
  ssl_dhparam /etc/ssl/dhparam.pem;

  add_header Strict-Transport-Security max-age=63072000;
  add_header X-Content-Type-Options nosniff;
}

The commented out section must be un-commented initially to allow lets encrypt to create a proper initial certificate.

Also, you need to create dlparams with:

openssl dhparam -out /etc/ssl/dhparam.pem 2048

And, of course, you need to create a DNS record so that a domain (e.g. ipmi.john.mayfirst.org) points to robideau's IP address.

Setup IPMI

Now, you can access the IPMI web configuration.

The default user pass is: ADMIN/ADMIN - case sensitive!

Here are some important changes:

  • Configuration -> Users
  • Change password for ADMIN user. NOTE make password 16 characters or less and keep out weird characters. It is possible to lock yourself out with this step.
  • Configuration -> Port
  • Only check off Web port (80) and Ikvm server port (5900)

Now, you can access the "monitor" by clicking Remove Control -> iKVM/HTML5

So-called Serial access

On robideau, or your jump host, install ipmiutil.

Then, create a simple bash script to help you remember the command:

ipmiutil sol -V 4 -a -N 192.168.56.3 -U ADMIN -Y

You will be prompted for the password.

Setup Linux

After installing linux, edit /etc/default/grub and change GRUB_CMDLINE_LINUX to:

GRUB_CMDLINE_LINUX="console=tty0 console=ttyS1,115200n8"

And add:

GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --unit0 --word=8 --parity=no --stop=1"

## Disable web config

Once you have working serial access, be sure to turn off nginx on robideau. We don't need to leave that open to the world. But it's handy to have in case you want to restart the physical server or something goes wrong with console access.

Attachments (7)

Download all attachments as: .zip