I help run a number of servers for friends and family. It's a win-win for them and me. I get practice as a sysadmin, they get more peace of mind. Of course, these servers aren't ones that I solely run -- other people log in often and can do various actions.
After one particularly harried night, I realize that a friend's server was shut down by their VPS provider. Not too surprising for the super cheap price, but it's not nice either way. I'm still working on setting up a monitoring server for all of the servers I have access to, but in the mean time I wanted a solution to tell me when the server was shutdown and when it finished booting.
I love using Pushover for things like this too. Super easy API, and generous free terms -- it's not likely that I'll need more than 7,500 alerts that the servers are booting and shutting down in a month!
Since every server is running systemd, I can take advantage of it and greatly
simplify the implementation. To start with, I create a basic shell script at
#!/bin/sh # This script is a basic 'service' launched and stopped by systemd. This will # send an alert out via Pushover to let me know that the server is shutting # down or finished booting. if [ "$1" = "boot" ]; then message="$(hostname -f) has finished booting" sound=bugle elif [ "$1" = "shutdown" ]; then message="$(hostname -f) is shutting down" sound=falling fi curl -s \ --form-string "token=token123" \ --form-string "user=user456" \ --form-string "message=$message" \ --form-string "sound=$sound" \ https://api.pushover.net/1/messages.json
I then set up the systemd service unit at
[Unit] Description=Send an alert via pushover that the system has finished booting After=network-online.target [Service] # Idle means that systemd will wait until the system is idle before executing Type=idle ExecStart=/usr/local/sbin/bootup-pushover-alert.sh boot ExecStop=/usr/local/sbin/bootup-pushover-alert.sh shutdown RemainAfterExit=yes Restart=no [Install] WantedBy=multi-user.target
And simply enabling the service:
systemctl enable boot-alert-pushover
A few interesting notes about this:
- This will not work for the virtual equivalent of pulling the power cord. That's where some sort of external monitoring would be useful. Nodeping and others are great at that.
- Setting the service type to idle means that systemd will delay executing the script until the server has (mostly?) finished booting.
- For simple shell scripts like this, you must set RemainAfterExit to yes otherwise ExecStop won't be called.
- Systemd does support calling a series of scripts in a directory when
the system shuts down. See
systemd-shutdown(8)for details about that. However, in my experience it didn't work.