Add Nagios plugins to LibreNMS and setup check_http

This is going to explain how to add Nagios plugins to your existing LibreNMS setup. We are going to use the check_http plugin to monitor a website and SSL certificate. The server we are setting this up on is running the following:
Server Version: 16.04.2 LTS (Xenial Xerus)
Apache Version: Apache/2.4.18 (Ubuntu)
MariaDB Version: 15.1 Distrib 10.0.29-MariaDB
PHP Version: 7.0.18

Install Nagios plugins:

root@stupiderror:~# apt install nagios-plugins

Edit config.php and add the following lines to integrate the Nagios plugins with LibreNMS:

root@stupiderror:~# vi /opt/librenms/config.php
$config['show_services']        = 1;
$config['nagios_plugins']       = '/usr/lib/nagios/plugins';

Now in LibreNMS you should see Services on the Navigation bar between Devices and Ports. Under Services if you choose Add Service you should have a list of all of the plugins under the Type drop down menu.

Edit the LibreNMS cron job and make sure it is running check-services.php every 5 minutes. This was already present for me, if it is missing add it.

root@stupiderror:~# vi /etc/cron.d/librenms
*/5  *    * * *   librenms    /opt/librenms/check-services.php >> /dev/null 2>&1

You can add a service from the Services menu or you can go to devices and choose a device. We are going to add the HTTP service to monitor the status code, response time, and SSL certificate expiration. Choose your device and then click on Services which should be in between Inventory and Logs. Click Add Service and choose http for the type. Fill in the description you want. For IP address fill in the website address such as stupiderror.com. If your site redirects to www then fill in something like www.stupiderror.com. For the Parameters we are going to use -S -w 3 -c 5. The -S tells it to connect with ssl so it uses HTTPS instead of HTTP. The -w says to put the service in a Warning state if it takes 3 seconds or longer to respond. The -c option tells it to put the service in a Critical state if it takes 5 seconds or longer to respond.

To monitor the expiration of the certificate click on Add Service again. Choose HTTP for type. For IP address fill in your domain name, something like www.stupiderror.com. For Parameters we can do -C 30,14. This says the service is good when the certificate is valid for more than 30 days. If the certificate is valid for 15 to 30 then days then change the state to Warning and if it is valid for 14 or less days then change the state to Critical. If you click on Details you can see graphs for these metrics.

For more information on check_http check out the man page: https://www.monitoring-plugins.org/doc/man/check_http.html

How to setup LibreNMS on Ubuntu 17.04

This is going to explain how to setup LibreNMS on an Ubuntu 17.04 server. For a production server you should stick with Ubuntu 16.04 as that seems to be what LibreNMS supports. We are going to setup LibreNMS on a server with the following:
Server Version: 17.04 (Zesty Zapus)
Apache Version: Apache/2.4.25 (Ubuntu)
MariaDB Version: 15.1 Distrib 10.1.22-MariaDB
PHP Version: 7.0.18

root@stupiderror:~# apt install apache2 fping git graphviz imagemagick libapache2-mod-fcgid libsnmp-dev mariadb-server mariadb-client mtr-tiny nmap php-cli php-curl php-fpm php-gd php-json php-mcrypt php-mysql php-pear php-snmp python-mysqldb php-net-ipv6 rrdcached rrdtool snmp snmpd whois
root@stupiderror:~# pear install Net_IPV4

LibreNMS needs php-net-ipv4 but this isn’t in the Zesty repositories. Instead we use pear to install it.
Next we will enable the apache modules and then create a database for LibreNMS.


root@stupiderror:~# a2enmod rewrite expires proxy_fcgi setenvif
root@stupiderror:~# a2enconf php7.0-fpm
root@stupiderror:~# phpenmod mcrypt
root@stupiderror:~# systemctl reload apache2.service
root@stupiderror:~# mysql -u root -p
Enter password:  (if you just installed hit enter to continue, no password has been set yet)
MariaDB [(none)]> create database librenms character set utf8 collate utf8_unicode_ci;
MariaDB [(none)]> create user 'librenms'@'localhost' identified by 'uniquepass';
MariaDB [(none)]> grant all privileges on librenms.* to 'librenms'@'localhost';
MariaDB [(none)]> flush privileges;
MariaDB [(none)]> quit

Edit /etc/mysql/mariadb.conf.d/50-server.cnf and add the following lines in bold under the [mysqld] section.


# this is only for the mysqld standalone daemon
[mysqld]

innodb_file_per_table=1
sql-mode=""
innodb_flush_log_at_trx_commit=0

#
# * Basic Settings

Create a librenms user, create folder structure and clone the git repository to install LibreNMS.


root@stupiderror:~# useradd librenms -d /opt/librenms -M -r
root@stupiderror:~# usermod -a -G librenms www-data
root@stupiderror:~# cd /opt
root@stupiderror:/opt# git clone https://github.com/librenms/librenms.git librenms
root@stupiderror:/opt# cd librenms/
root@stupiderror:/opt/librenms# mkdir rrd logs
root@stupiderror:/opt/librenms# chmod 755 rrd

Create an apache site configuration file for Librenms


root@stupiderror:/opt/librenms# vi /etc/apache2/sites-available/librenms.conf

  DocumentRoot /opt/librenms/html
  ServerName librenms.stupiderror.com
  CustomLog /opt/librenms/logs/access_log combined
  ErrorLog /opt/librenms/logs/error_log
  AllowEncodedSlashes NoDecode
  
    Require all granted
    AllowOverride All
    Options FollowSymLinks MultiViews
  

Setup SNMP v3 authentication


root@stupiderror:/opt/librenms# systemctl stop snmpd.service 
root@stupiderror:/opt/librenms# net-snmp-config --create-snmpv3-user -ro -a authPass -x privPass -X AES -A SHA librenms
root@stupiderror:/opt/librenms# curl -o /usr/bin/distro https://raw.githubusercontent.com/librenms/librenms-agent/master/snmp/distro
root@stupiderror:/opt/librenms# chmod +x /usr/bin/distro

Edit /etc/snmp/snmpd.conf and comment out the two lines with the default public string. Then scroll down to find the extend entries and comment out the tests and add the line with distro.


# rocommunity public  default    -V systemonly
# rocommunity6 public  default   -V systemonly

# extend    test1   /bin/echo  Hello, world!
# extend-sh test2   echo Hello, world! ; echo Hi there ; exit 35
#extend-sh test3   /bin/sh /tmp/shtest
extend distro /usr/bin/distro

Restart SNMPD, enable LibreNMS site, disable default site, setup maintenance tasks, and chown the LibreNMS files to librenms user and group.


root@stupiderror:/opt/librenms# systemctl start snmpd.service
root@stupiderror:/opt/librenms# a2ensite librenms.conf
root@stupiderror:/opt/librenms# a2dissite 000-default.conf
root@stupiderror:/opt/librenms# systemctl reload apache2.service
root@stupiderror:/opt/librenms# cp librenms.nonroot.cron /etc/cron.d/librenms
root@stupiderror:/opt/librenms# cp misc/librenms.logrotate /etc/logrotate.d/librenms
root@stupiderror:/opt/librenms# chown -R librenms:librenms /opt/librenms

Now open your browser and go to install.php at your site, in my example I go to http://librenms.stupiderror.com/install.php
All your PHP modules should be installed so you can click Next Stage
Fill in the password for the database you created for LibreNMS and click Next Stage.
Now you should see a status page, mine went to Step #96 and then updated the schema which went to 195. On the last line it says — Done. Click on Goto Add User
Create a user by filling in the info you want to use and click Add User.
Click Generate Config
Copy the config and save it to /opt/librenms/config.php and click Finish Install

Edit /opt/librenms/html/.htaccess and change the last RewriteRule which should like
RewriteRule ^(.*)$ index.php/$1/
change it to:
RewriteRule ^(.*)$ index.php

To enable RRDCached edit /opt/librenms/config.php and uncomment the following line:
$config[‘rrdcached’] = “unix:/var/run/rrdcached.sock”;

Edit /etc/default/rrdcached and modify the following lines like so:


WRITE_TIMEOUT=1800
WRITE_JITTER=1800
BASE_PATH=/opt/librenms/rrd/
SOCKGROUP=librenms
DAEMON_GROUP=librenms
DAEMON_USER=librenms
BASE_OPTIONS="-B -F -R"

root@stupiderror:/opt/librenms# systemctl restart rrdcached.service

Lets finish up with adding the LibreNMS server as a device to be monitored.
Log in to your LibreNMS web portal. Go to Device > Add Device
Fill in the following:
Hostname: 127.0.0.1
SNMP Version: v3
Port: 161

(Previously we used the net-snmp-config to add a user and credentials, I’m going to use the same values as I did above but hopefully you created your own credentials)
Auth Level: authPriv
Auth User Name: librenms
Auth Password: authPass
Auth Algorithm: SHA
Crypto Password: privPass
Crypto Algorithm: AES

Click Add Device

How to setup Lets Encrypt with Apache on Ubuntu 16.10

This is going to explain how to setup Let’s Encrypt for a WordPress site using Apache on Ubuntu. This setup requires you to have root access to the web server.
Server Version: 16.10 (Yakkety Yak)
Apache Version: Apache/2.4.18 (Ubuntu)

First lets generate a Diffie-Helman Parameters file (This will take a while, can easily take 5+ minutes):

stupiderror@localhost:~$ sudo -i
[sudo] password for stupiderror:
stupiderror@localhost:~# openssl dhparam -out /etc/ssl/private/dhparams_4096.pem 4096
Generating DH parameters, 4096 bit long safe prime, generator 2
This is going to take a long time
.....................................................................+..........

Now we will install certbot:

stupiderror@localhost:~# apt update
stupiderror@localhost:~# apt install python-certbot-apache
stupiderror@localhost:~# certbot --apache

Select the domain(s) that you want to enable HTTPS for.
Provide an email address.
Accept the Terms of Service.
Choose Easy or Secure. Easy will let your site be HTTP and HTTPS. Secure will make your site only HTTPS but will redirect anyone trying to use HTTP to HTTPS.

You can use a command like the following and avoid the menu, just substitute in your domain and email:

stupiderror@localhost:~# certbot --apache -d stupiderror.com -d www.stupiderror.com -m support@example.net --agree-tos

The last step is to add a cron entry to auto renew the certificate.

stupiderror@localhost:~# crontab -e
0 1 * * 0    /usr/bin/certbot renew >> /var/log/le-renew.log

Reset WordPress Password

This is going to explain how to reset a WordPress password utilizing MySQL.
This was performed on a server running the following:
Server Version: Ubuntu 16.04.1 LTS (Xenial Xerus)
Wordpress Version: 4.6.1
MySQL Version: 5.7.15
PHP Version: 7.0.8

First we will login to MySQL. If you don’t know your MySQL username, database, and/or password you can find that in wp-config.php at the root of your WordPress site.

/** The name of the database for WordPress */
define('DB_NAME', 'wordpress');

/** MySQL database username */
define('DB_USER', 'stupiderror');

/** MySQL database password */
define('DB_PASSWORD', 'WPpassword');

/** MySQL hostname */
define('DB_HOST', 'localhost');

Use this information to login to MySQL:

stupiderror@localhost:~/html$ mysql -u stupiderror -h localhost -p
Enter password:
mysql> use wordpress;
Database changed
mysql> show tables;
+-----------------------+
| Tables_in_wordpress   |
+-----------------------+
| wp_commentmeta        |
| ...                   |
| wp_users              |
| wp_wpeditor_settings  |
+-----------------------+
17 rows in set (0.00 sec)
mysql>

Most of the tables were truncated from the results. We are looking for a table ending in _users, in this case wp_users. Now lets find the user we want to reset the password for.

mysql> select id, user_login,user_pass,user_email from wp_users;
+----+------------+------------------------------------+-------------------+
| id | user_login | user_pass                          | user_email        |
+----+------------+------------------------------------+-------------------+
|  1 | Admin      | $P$B3pqWu6gdqNO/kzDB4ZtnDPntHIq8f. | adm@localhost.com |
+----+------------+------------------------------------+-------------------+
1 row in set (0.00 sec)
mysql>

If you have a modern version of PHP and WordPress your password hash should start with $P$
This means WordPress is using PHPass. We will use a small script to generate a new hash.
On the 6th line replace newPassword with the new password you want to use. Save this file to the root of your WordPress site.

<?php
  require( dirname(__FILE__) . '/wp-load.php' );
  require_once ABSPATH . WPINC . '/class-phpass.php';
  $hasher = new PasswordHash(8,TRUE);

  $hash = $hasher->HashPassword('newPassword');

  echo "Hash: $hash\r\n"
?>

Now generate a hash and then update the database with the new hash:

stupiderror@localhost:~/html$ php generateHash.php
Hash: $P$B3pqWu6gdqNO/kzDB4ZtnDPntHIq8f.

mysql> update wp_users set user_pass="$P$B3pqWu6gdqNO/kzDB4ZtnDPntHIq8f." where ID = 1;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0
mysql>

Now you should be able to login to your WordPress site with the new password.

Backup and Upgrade EdgeRouter Using CLI

This entry will show you how to upgrade an Ubiquiti EdgeRouter using the command line. Before we do an upgrade we will do a backup in case something goes wrong. After upgrading we will show how to restore a backed up configuration which shouldn’t be needed if everything goes as planned.

Backup EdgeRouter Using CLI
There are several ways to do this. One of the simplest ways is to output the configuration and then copy and paste it.

stupiderror@stupiderror:~$ cat /config/config.boot


This will dump your configuration to the screen. You can then copy the configuration and then paste it to a text file on your computer and save it.

You can use the save command to backup the config using scp, ftp, or tftp. Here is an example using ftp:

save ftp://stupiderror:stupidpassword@stupiderror.com/config.boot.backup

Update EdgeRouter Firmware Using CLI
We will show two ways to do this. You can have the router get the update directly and apply it or you can download it to your computer and upload it to the router. First lets see what version is currently running using the show version command:

stupiderror@stupiderror:~$ show version
Version:      v1.5.0
Build ID:     4677648
Build on:     06/20/14 13:01
Copyright:    2012-2014 Ubiquiti Networks, Inc.
HW model:     EdgeRouter PoE 5-Port
HW S/N:       1CC328D0C476
Uptime:       14:19:38 up 77 days,  1:25,  1 user,  load average: 0.18, 0.10, 0.07

Go to https://www.ubnt.com/download/ and use the drop down menus to select your router. Version 1.6.0 is available: ER-e100.v1.6.0.4716006.tar. First we will show how to do the upgrade from the router. We need the URL to download the new firmware from which isn’t provided here. We can build it ourselves though. http://dl.ubnt.com/firmwares/edgemax/v1.6.0/ This is the URL where the downloads are kept. You may need to change the end of the URL if you are getting a version other than v1.6.0. Now append the file name (ER-e100.v1.6.0.4716006.tar) provided on the download page to the URL which gives us: http://dl.ubnt.com/firmwares/edgemax/v1.6.0/ER-e100.v1.6.0.4716006.tar We then use the add system image command to tell it to retrieve and install the firmware:

stupiderror@stupiderror:~$ add system image http://dl.ubnt.com/firmwares/edgemax/v1.6.0/ER-e100.v1.6.0.4716006.tar
Version [v1.2.0.4574253.130626.1248] is about to be replaced
Are you sure you want to replace old version? (Yes/No) [Yes]: Yes
Trying to get upgrade file from http://dl.ubnt.com/firmwares/edgemax/v1.6.0/ER-e100.v1.6.0.4716006.tar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 70.1M  100 70.1M    0     0  2815k      0  0:00:25  0:00:25 --:--:-- 2863k
Download suceeded
Checking upgrade image...Done
Preparing to upgrade...Done
Copying upgrade image...Done
Removing old image...Done
Checking upgrade image...Done
Copying config data...Done
Finishing upgrade...Done
Upgrade completed
stupiderror@stupiderror:~$ reboot
Proceed with reboot? [confirm][y]

Broadcast message from root@stupiderror (pts/0) (Sun Feb 22 14:32:46 2015):
The system is going down for reboot NOW!

stupiderror@stupiderror:~$ show version
Version:      v1.6.0
Build ID:     4716006
Build on:     10/31/14 17:31
Copyright:    2012-2014 Ubiquiti Networks, Inc.
HW model:     EdgeRouter PoE 5-Port
HW S/N:       1CC328D0C476
Uptime:       14:37:21 up 4 min,  1 user,  load average: 0.41, 0.52, 0.24

After the file the is uploaded and installed we then have to reboot so the new firmware can be loaded. After the reboot is complete we log back in and show version to confirm we are using the new firmware.

Another way to upgrade it is to download the file to our computer and then we can upload it to the router. In this case when you go to https://www.ubnt.com/download/ and select your router from the down drop menus you can then click the Download button and save the file. To upload the file to the router use scp in *nix or pscp in windows. pscp is made by Simon Tatham, the same person who makes putty. pscp is used from the command prompt. You want to give a command similar to below, make sure you have the correct file name, username, and IP address.

c:\>pscp ER-e100.v1.6.0.4716006.tar stupiderror@192.168.1.1:


Once it uploads you then use the add system image command to install it:

stupiderror@stupiderror:~$ add system image ER-e100.v1.6.0.4716006.tar


Everything is the same as above. Once it finishes installing you then need to reboot the router. After it reboots you can log back in and use show version to confirm the new firmware is being used.

Restore EdgeRouter Configuration Using CLI
Use scp or pscp to copy your backed up configuration file to the router. Once it is copied you can then load it and save it.

c:\>pscp starter-config.boot ubnt@192.168.0.1:/tmp/

stupiderror@stupiderror:~$ configure
stupiderror@stupiderror# load /tmp/starter-config.boot
stupiderror@stupiderror# commit
stupiderror@stupiderror# save

Basic Cisco Router Configuration

This entry will show a basic working configuration for a Cisco router. This configuration is from a Cisco 1841 running IOS 12.4(3h). The complete configuration is at the bottom of this post. The configuration uses DHCP on the WAN interface so the router will obtain an IP address from the ISP. The configuration uses NAT/PAT to allow multiple computer and devices to get online. The router will act as a DHCP server for the LAN.

DHCP For LAN

ip dhcp excluded-address 192.168.70.1 192.168.70.99
ip dhcp excluded-address 192.168.70.110 192.168.70.254
!
ip dhcp pool LAN
   network 192.168.70.0 255.255.255.0
   dns-server 8.26.56.26 8.8.8.8
   default-router 192.168.70.1
   lease 8


FastEthernet0/1 is setup as our LAN interface and has the IP address 192.168.70.1. Here we are enabling DHCP for the 192.168.70.0/24 network. The router will now hand out all IP addresses on that network. Up above we exclude 192.168.70.1 through 192.168.70.99. Then we exclude 192.168.70.110 through 192.168.70.254. This limits our IP pool to 10 IP addresses: 192.168.70.100 through 192.168.70.109. The DHCP server will provide 8.26.56.26 and 8.8.8.8 as two DNS servers for the client to use. 192.168.70.1 is provided as the default gateway. lease 8 specifies that the lease lasts for 8 days. If you omit that it will default to 1 day. lease is specified in day hour minute. If you want the lease to last for 8 hours and 30 minutes you would do:

stupiderror(dhcp-config)#lease 0 8 30

PAT Configuration
Cisco calls this PAT (port address translation) pretty much everyone else refers to it as NAT (network address translation). This configuration will replace any LAN IP address that is going to the Internet with the IP address on FastEthernet0/0, which is acting as our WAN interface.

interface FastEthernet0/0
 description WAN Interface
 ip address dhcp
 ip nat outside
!
interface FastEthernet0/1
 description LAN Interface
 ip address 192.168.70.1 255.255.255.0
 ip nat inside
!
ip nat inside source list 1 interface FastEthernet0/0 overload
!
access-list 1 permit 192.168.70.0 0.0.0.255


On FastEthernet0/0 we have the command ip nat outside specifying it as the outside interface. On FastEthernet0/1 we have a similar command that specifies inside. We also have an access list that permits 192.168.70.0/24. Those three commands won’t do anything on their own. ip nat inside source list 1 interface FastEthernet0/0 overload is what ties everything together. This says on the inside interface apply source access-list 1 and overload it on interface FastEthernet0/0. We need to overload it because we can have up to 254 hosts on the LAN and we only have one IP address on the WAN (FastEthernet0/0). If a host were to get on the network that had an IP address outside of 192.168.70.0/24 and was somehow able to reach FastEthernet0/1 (would pretty much require a secondary IP address on the interface) the access-list would prevent it from being translated if it were to exit FastEthernet0/0.

Configuration

stupiderror#sh run
Building configuration...

*Feb 21 14:35:18.419: %SYS-5-CONFIG_I: Configured from console by console
Current configuration : 1439 bytes
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname stupiderror
!
boot-start-marker
boot-end-marker
!
enable secret 5 $1$4D28$g74YPSLPm4lhTQR2Iuy01.
!
no aaa new-model
!
resource policy
!
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
ip subnet-zero
ip cef
!
!
no ip dhcp use vrf connected
ip dhcp excluded-address 192.168.70.1 192.168.70.99
ip dhcp excluded-address 192.168.70.110 192.168.70.254
!
ip dhcp pool LAN
   network 192.168.70.0 255.255.255.0
   dns-server 64.7.11.2 8.8.8.8
   default-router 192.168.70.1
   lease 8
!
!
no ip domain lookup
ip domain name example.net
ip ssh version 2
!
!
!
!
username stupiderror privilege 15 secret 5 $1$BG7S$BAvySovWUZ4k58nfOKXSu9
!
!
!
!
!
interface FastEthernet0/0
 description WAN Interface
 ip address dhcp
 ip nat outside
 ip virtual-reassembly
 duplex auto
 speed auto
!
interface FastEthernet0/1
 description LAN Interface
 ip address 192.168.70.1 255.255.255.0
 ip nat inside
 ip virtual-reassembly
 duplex auto
 speed auto
!
interface Serial0/0/0
 no ip address
 shutdown
!
interface Serial0/1/0
 no ip address
 shutdown
!
ip classless
!
ip http server
no ip http secure-server
ip nat inside source list 1 interface FastEthernet0/0 overload
!
access-list 1 permit 192.168.70.0 0.0.0.255
!
!
control-plane
!
!
line con 0
line aux 0
line vty 0 4
 login local
 transport input ssh
line vty 5 15
 login local
 transport input ssh
!
end

Cisco Password Recovery

This entry shows how to reset the password on a Cisco router. We are doing this on a Cisco 1841. This will require taking the device down but the configuration will not be lost.

Console Connection
First we need to connect our computer to the console port on the Cisco router using a Cisco console cable or rollover cable. If your computer doesn’t have a serial port you will need a USB to DB9 serial adapter. We will be using an adapter made by Pluggable. This uses a prolific chipset which will allow us to send the break sequence we will need soon.
If you are using windows you can use a program like putty to make the serial connection. In the *nix world minicom works well.
You will need to specify the com port to use such as COM3
Speed: 9600
Data bits: 8
Stop bits: 1
Parity: None
Flow control: None
Make your connection to the router now.

Boot To ROMMON
We will show two ways to do this.
First we will get to rommon by sending the break sequence.
Power cycle the router and press break in your console session as soon as text starts to scroll.
Note: On a Lenovo Thinkpad T430 you press FN + B for break.
You should see this:

System Bootstrap, Version 12.4(13r)T, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 2006 by cisco Systems, Inc.
PLD version 0x10
GIO ASIC version 0x127
c1841 platform with 262144 Kbytes of main memory
Main memory is configured to 64 bit mode with parity disabled

Readonly ROMMON initialized
rommon 1 >

If you can't send the break key to get to rommon then you can try removing the flash. To remove the flash first power the router off. Use the black button to pop the compact flash out. Power the router on and it should boot to the rommon prompt. If you removed the compact flash then you need to power it down after changing the configuration register, do not type reset. Once the router is powered off you can then insert the compact flash and turn it back on.

Now type confreg 0x2141 followed by reset.

rommon 1 > confreg 0x2141
You must reset or power cycle for new config to take effect.
rommon 2 > reset

The router will now reboot

Recover Configuration and Reset Password
When it starts back up you should now see:

--- System Configuration Dialog ---

Would you like to enter the initial configuration dialog? [yes/no]:

Choose no. Then you want to enter enable mode and copy the start-up configuration to the running configuration. By going in to rommon and changing the configuration register we prevented the router from loading the startup-config. The router booted and has no configuration loaded. We can now enter enable mode and load the startup-config. At this point the router is up and running with its configuration and we are logged in. We can now change the password and overwrite the existing password which we don't know. Then we just need to restore the configuration register and save our changes.

Would you like to enter the initial configuration dialog? [yes/no]: no
Press RETURN to get started! 

Router>en
Router#copy start run
Destination filename [running-config]?
1430 bytes copied in 2.756 secs (519 bytes/sec)
stupiderror#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
stupiderror(config)#user stupiderror priv 15 secret 0 stupidpass
stupiderror(config)#enable secret 0 stupidpass
stupiderror(config)#config-reg 0x2102
stupiderror(config)#exit
*Feb 21 15:50:23.991: %SYS-5-CONFIG_I: Configured from console by console
stupiderror#wr mem
Building configuration...
[OK]


At this point the router should be back up and running with a password we now know.

Installing Google Authenticator for SSH

This entry shows how to install Google Authenticator and set it up to provide ssh with two-factor authentication. This was done on Ubuntu Server 14.10. This is software that runs on your server and mobile device. It does not require a Gmail account or any third party service.

Install On Server
Install libpam-google-authenticator package:

stupiderror@localhost:~$ sudo -i
[sudo] password for stupiderror:
root@localhost:~# apt-get install libpam-google-authenticator
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  libqrencode3
The following NEW packages will be installed:
  libpam-google-authenticator libqrencode3
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 59.3 kB of archives.
After this operation, 206 kB of additional disk space will be used.
Do you want to continue? [Y/n] y

Setup User
Now we will setup Google Authenticator for user stupiderror:

root@localhost:~# exit
logout
stupiderror@localhost:~$ google-authenticator

Do you want authentication tokens to be time-based (y/n) y
Your new secret key is: BHBWGPOSWB6K19QY
Your verification code is 674538
Your emergency scratch codes are:
  16572244
  40474860
  4213D034
  55121160
  04767458

Do you want me to update your "/home/stupiderror/.google_authenticator" file (y/n) y

Do you want to disallow multiple uses of the same authentication
token? This restricts you to one login about every 30s, but it increases
your chances to notice or even prevent man-in-the-middle attacks (y/n) y

By default, tokens are good for 30 seconds and in order to compensate for
possible time-skew between the client and the server, we allow an extra
token before and after the current time. If you experience problems with poor
time synchronization, you can increase the window from its default
size of 1:30min to about 4min. Do you want to do so (y/n) n

If the computer that you are logging into isn't hardened against brute-force
login attempts, you can enable rate-limiting for the authentication module.
By default, this limits attackers to no more than 3 login attempts every 30s.
Do you want to enable rate-limiting (y/n) y

We choose to make the one time tokens time based. By default they are displayed for 30 seconds and have a 60 second grace period. Next it shows a QR code which was omitted above. This assists with getting it setup on your phone which we will get to next. If you don’t want to use the QR code you can enter the secret key on your phone instead. It then provides 5 emergency scratch codes. Write these down and they can be used when you need to login and your phone isn’t available. They are not time sensitive and each code can only be used once.

We choose yes to disallow the authentication token from being used more than once. If you want to log in to the server and you look to open two ssh sessions this means you can use the verification code displayed on your mobile device and then must wait for the next verification code to be displayed to complete the second login. If an attacker is able to sniff your keystrokes this means they cannot use the verification code you just used to log in.

We also enable rate limiting to slow down brute force attacks. Sshguard is installed and works fine with this enabled.

Setup Phone
On your phone search for Google Authenticator App in the app store and install it. It is made by Google, Inc. Open Google Authenticator on your phone and press the + in the top right to add an account. Choose Scan barcode or Manual entry. If you choose Scan barcode then point the camera on your phone at the QR code that was displayed when you ran google-authenticator on the server. Once it sees the code it adds it and names it after the user and hostname of the server google-authenticator was run as. In this example it named it: stupiderror@localhost. If you would like to rename the entry then click the pencil icon in the top right on your mobile device. Then the name will have a pencil icon next to it and you can click the name and edit it.

If you choose manual entry then fill in the information requested. Account is just a friendly name for you to identify the server. You can set this up on multiple servers and/or multiple users and add them all. The account name allows you to differentiate them. For key you need to enter the secret key that was displayed after the QR code when you ran google-authenticator. In the above example the key is BHBWGPOSWB6K19QY. Do not use this key, you need to use the one that was generated for your account. Then enable or disable the time restriction, we are leaving it enabled.

Enable Google Authenticator On Server
The time and date settings are important. If the time and date on the server is too far off from the phone the time based verification codes will fail. Make sure the time and date are set correctly. You may want to make a crontab entry to sync it daily:

0 0 * * *      /usr/sbin/ntpdate -s pool.ntp.org


We need root to edit these files:

stupiderror@localhost:~$ sudo -i
[sudo] password for stupiderror:
root@localhost:~# vi /etc/ssh/sshd_config


Find this line:

ChallengeResponseAuthentication no


and change the no to yes

ChallengeResponseAuthentication yes


Edit /etc/pam.d/sshd

root@localhost:~# vi /etc/pam.d/sshd


Add the following line and it should be before “@include common-auth”:

auth	required			pam_google_authenticator.so


To enable these changes restart SSHD:

root@localhost:~# service ssh restart

Test It
I would not disconnect your current session because if something goes wrong you will need to login as a different user or fix it from the console.

Using keyboard-interactive authentication.
Verification code:
Using keyboard-interactive authentication.
Password:
Last login: example.net
stupiderror@localhost:~$

If it isn't working properly the best place to look for error messages is probably the auth.log

root@localhost:~# tail -f /var/log/auth.log