Ever since Google announced that Chrome would mark non-https connections as ‘Not Secure’ I’ve begun to fret about ssl certificates. These serve two purposes. First, they encrypt your data and prevent Man-in-the-middle attacks, and secondly, they verify that the site you visit is the site it claims to be. I used to think that the former was more important, but now I am more of the opinion that identity verification is most important, now that phishing attacks are commonplace. LetsEncrypt has recently stepped in to help solve this problem.

With this in mind, when I saw that my UniFi controller was marked:

I wanted to fix it.

LetsEncrypt

For a long time, certificates have been sold by certificate authorities, but now you can get them for free from LetsEncrypt. However, there are some provisos to be aware of.

First while you used to be able to get a 3 year certificate from a vendor, LetsEncrypt certs are 90 days, and must be renewed. Secondly, you have to be able to prove you control the name that the certificate is for. This makes things more complicated.

In effect, you either need to be able to prove you control the DNS entry of your server, or the server itself. This works great if you have a publicly-facing web site, but it’s more complex when you want to secure servers that are not accessible, like my internal UniFicontroller. Fortunately, pfSense makes this reasonably easy. I’m using *nix servers, so it’s really not. I’ll walk through what I did.

I’m pretty ok with the 90 day lifecycle, since it conforms to most password-rotation rules. It’s good security.

How LetsEncrypt Works (in my case)

There are several ways that LetsEncrypt will work, and since I can’t update my DNS via API, I chose to use the ‘Standalone HTTP server’ option. Essentially, you create a dns entry for the server behind the firewall you want:

unifi.barclayhowe.com

And I made this a cname that points to my firewall server. LetsEncrypt expects to find an HTTP server there on port 80, and it wants to see a secret on that server to show that the requester, DNS owner, and server owner are all the same. You will basically clutter up your DNS records with some extra aliases, but I think that’s ok, too.

LetsEncrypt will only let you do 5 calls per hour, so they have a staging environment that allows building and testing your solution.

Concept

The basic concept here is as follows:

1. Add DNS entry
2. Add acme (the LetsEncrypt client) to pfSense
3. Set up a port forward from port 80 to some random port (port 80 is already in use on my pfSense server on the LAN side, so the LetsEncrypt server can’t use it)
4. Set up the acme client to request a certificate for your internal server.
5. Extract, move and install the certificate on the internal server

Easy, right? Or course it is…

Let’s Do it

I assume that you have a DNS service and can add your entry, and if you’re using PFSense, you can add a package.

Adding the LetsEncrypt Staging Server

First, navigate to the Services -> Acme Certificates Service.

Then, click on ‘Account Keys’:

and click ‘Add’.

This is a form that requires a few actions to complete. It looks like this at first:

Give it a name (I’d include ‘staging’ or ‘production’ in the name, as it helps out later), optionally a description, and an email address. I chose the Acme V2 staging server (it supports some new stuff) Then Click ‘+ Create Account Key’:

Then click ‘Register ACME account key’.

When the key icon becomes a check, you are ready to ask for a certificate. Click ‘Save’

Then switch to the ‘General Settings’ tab and set both checkboxes:

These will enable your renewal, and the extraction of the certificates to a folder so you can easily move them to another server.

Then go to the ‘Certificates’ tab and click ‘+Add’:

The only important note here is that set my DomainName to unifi.barclayhowe.com, and I picked a random port (18882) to listen on that is not 80. I needed to do a port forward from port 80 on my WAN interface to port 18882 on my LAN interface for this to work, you will too. I don’t like having open port forwards in my firewall, but this is pretty benign. After you do something like this, I recommend an external port scan to check that you are still secure. The acme service will start a temporary HTTP server when it renews, then tear it down when it’s done.

Now click ‘Save’, and exit to the list screen and see your certificate setup ready to request its first certificate:

Now click ‘Issue/Renew’ next to your new certificate. The gear will turn, and after a bit you’ll see a lot of green text. If there is block that looks like:

You did it. Now you can now ssh into your server and type:

ls /conf/acme/unifi*

You will see:

Putting Them Where They Belong

Now that you have them, you need to get them to the server where the controller is. Doing this requires the following steps:

1. Package them into a p12 package. It’s a way of gluing several certificate components together into one file.
2. Copy them to the new server.
3. Run commands on that server to install the certificates into the controller’s java web server.

Up front, as a regular user, you can practice this with (all one line):

openssl pkcs12 -export -in unifi.barclayhowe.com.all.pem -inkey unifi.barclayhowe.com.key -out unifi.p12 -name unifi-CAfile unifi.barclayhowe.com.fullchain -caname root -passout pass:test1234

This will package the three files into a single file with a password that is ‘test1234’. The password is required by the java keystore – it won’t work without it.

Scp the file to the internal server:

scp /conf/acme/unifi.p12 someuser@internalserver:~/unifi.p12

Then, on that server:

keytool -importkeystore -deststorepass aircontrolenterprise -destkeypass aircontrolenterprise -destkeystore /opt/UniFi/data/keystore -srckeystore /home/someuser/unifi.p12 -srcstoretype PKCS12 -alias unifi -srcstorepass test1234

systemctl restart unifi

Seems simple, but since we need to renew this every 90 days, we need to take advantage of the acme service’s ability to run scripts once it has renewed the certificate.

Scripting it

Aside from a bit of scripting skill ( my scripts are definitely rough), we need a trusted user that can connect from the pfSense server to the internal server in order to copy the file. We’ll end up with two scripts:

copyUnifiCertificate.sh, which will be called from the certificate renewal page, and:

installUnifiCertificate.sh which will live in the home folder of the user on the internal server to install the certificate.

User setup

On your internal server, run the following commands:

adduser pfSenseCertCopier

su pfSenseCertCopier

cd

ssh-keygen -t ed25519 -a 100

When prompted : Enter file in which to save the key (/home/pfSenseCertCopier/.ssh/id_dsa): Press enter.

When prompted:

Enter passphrase (empty for no passphrase):
Enter same passphrase again:

Press enter twice. You don’t want a password here, since it will run winout user input.

ls -la .ssh

this will list the contents of the /ssh hidden folder, and it has 2 keys, a private and a public.

then copy the key you created to the ssh authorized keys folder for that user:

cp .ssh/id_ed25519.pub .ssh/authorized_keys

Set ownership and permissions: the user needs to own those files, and the directory has to be globally executable, and the authorized keys has to be globally readable. This lets the ssh service see them:

chown -R pfSenseCertCopier:pfSenseCertCopier /home/ pfSenseCertCopier/.ssh

chmod 700 /home/pfSenseCertCopier/.ssh

chmod 600 /home/pfSenseCertCopier/.ssh/authorized_keys

copy the private key to the firewall server:

scp .ssh/id_ed25519 root@firewallserver:~

then test:

ssh pfSenseCertCopier@192.168.1.233 ought to fail. You have no defined password.

ssh -i id_ed25519 pfSenseCertCopier@internalhost

Should work, since you have the private key. Success!

Then the script on the Internal server

First, you will need the copier user to be able to use sudo with no password to only execute one command, so execute:

sudo visudo on your internal server

and add the following line, which will allow this user to only be able to run 2 commands without a password:

pfSenseCertCopier       ALL=(root)      NOPASSWD:/usr/bin/keytool, /usr/bin/systemctl

Then edit your script:

cd /home/pfSenseCertCopier

su pfSenseCertCopier

vi installUnifiCertificate.sh

#!/bin/bash
sudo keytool -delete -alias unifi -keystore /opt/UniFi/data/keystore -storepass aircontrolenterprise > /dev/null 2>&1

sudo keytool -importkeystore -deststorepass aircontrolenterprise -destkeypass aircontrolenterprise -destkeystore /opt/UniFi/data/keystore -srckeystore ~/unifi.p12 -srcstoretype PKCS12 -alias unifi  -srcstorepass test1234 > /dev/null 2>&1


sudo systemctl restart unifi > /dev/null 2>&1

then

chmod 700 installUnifiCertificate.sh

Test it:

./installUnifiCertificate.sh

and it should return nothing, but when you open your controller, you should be greeted with:

Remote Testing

OK, lets see if our user can do this remotely:

From your firewall server, run:

ssh -t -i id_ed25519 pfSenseCertCopier@internalserver ./installUnifiCertificate.sh

This will use ssh to authenticate with your private key and run the command remotely. It should complete, and your unifi controller should still work.

Final Script

Now, one more script on your firewall server, in your root home folder:

Copy the private key to a better-named file:

mv id_ed25519 pfSenseCertCopier.id_ed25519

vi copyUnifiCertificate.sh

#!/bin/sh
openssl pkcs12 -export -in /conf/acme/unifi.barclayhowe.com.all.pem -inkey /conf/acme/unifi.ba
rclayhowe.com.key -out /conf/acme/unifi.p12 -name unifi -CAfile /conf/acme/unifi.barclayhowe.c
om.fullchain -caname root -passout pass:test1234

scp -i /root/pfSenseCertCopier.id_ed25519 /conf/acme/unifi.p12 pfSenseCertCopier@1internalserver:~


ssh -t -i /root/pfSenseCertCopier.id_ed25519 pfSenseCertCopier@internalserver ./installUnifiCertific
ate.sh

then

chmod 700 copyUnifiCertificate.sh

and test:

vi copyUnifiCertificate.sh

And the entire orchestration will run.

Now To Tie It Together

Back in pfSense, add the command /root/copyUnifiCertificate.sh to the actions list:

save, and click the ‘Issue/Renew’ button once more. On your internal server, if you run:

ls -la

in the home folder of your copier user, you should see a freshly-updated certificate.

And your controller should work.

Wrapping up

I think this is a pretty secure way of doing this, and you should be able to re-use the script on the internal server to update any arbitrary certificate usage. I looked at a lot of sites for this exact process, and hopefully this helps, since I could only find prices.

Update 10/14/18: I moved my controller to Ubuntu, so here is how to do that.

Update 1/6/19: Also added a guide for installing to Kibana.

What I’m listening to as I do this:

Gogol Bordello’s Gypsy Punks Underdog World Strike. Many years ago I was just getting into European Metal, especially some folk- and ancient music-influenced bands like In Extremo, when a friend suggested it. It’s crazy good, and it defies description. It’s a good uptempo album for linux work.