PKI - Using Public and Private Keys to access Linux via SSH

foobars ever, I have been on a voyage of discovery, learning how to access a remote Red Hat box ( RHEL 7.2 ) via SSH without passwords.

Long story short, this RHEL box is actually hosted on KVM using Open Stack, but that's not important right now.

My client is macOS Sierra, which is good to know, but not totally relevant to my voyage.

Having created the box using OpenStack, which means that I have my own public/private key pair associated with the box at build time, I wanted to add a second user into the mix.

I used the OpenStack web UI to create a second public/private key pair, which resulted in me downloading the private key in PEM format as a .CER file.

I validated this using the file command: -

file ~/Downloads/foobar.cer 

/Users/davidhay/Downloads/foobar.cer: PEM RSA private key

So, having the private key for the second user, I needed to ensure that the Red Hat box had the corresponding public key.

The OpenStack UI gives me the ability to view the newly created public key, via Compute -> Access & Security -> Key Pair Details, which returned the name, fingerprint and the public key.

The public key is returned as a string of text: -

ssh-rsa 2222B3Nz2C1yc2E2222D2Q2B222B2QCo23fBjN2DMrhlXsWb6J3+Wrdcy/j28EhylY+Uc8/DzmbCm1ycCEuCDUpBdldD+2YpTyjQkH8ot0r4DWQDCM52o/rw1wV58tZniO60jiwdEDYy6mO5oPdhib598UQs0Ny67dCRE7wW+2NVcrIs2F5DLWMnBxUlQcdxiiHmqDwZ/KF25SS9RvEuNDVhfd0+DkbnvWhtf2D12ODdeSMUJ2to+ztDDW+mf6tpDqQDwJRhITkn2dziDdDlpSkHSDZw4Q7QmkSNsuv9Tj1BWKFDfuIq+ns3SU6/sqi8QvIXc+KWRnk3893EJDYqBjNH5ut2itHjVnlMhx0Bh+Mpiilz/c5x Generated-by-Nova

In order to use this on the remote RHEL box, I merely needed to add that string into a file, authorized_keys, on the remote box, in the target user's home directory: -

/home/bob/.ssh/authorized_keys

( In this case, the target user is bob )

With this in place, I should simply be able to ssh into the target box using the private key: -

ssh -i ~/foobar/foobar.cer bob@bpm101

In other words, running the ssh command with the -i switch to assert an identity contained within the certificate ~/foobar/foobar.cer for the remote user bob on the remote host bpm101.

Alas, this failed with: -

Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

I turned on various levels of debug within the ssh command: -

ssh -v -i ~/foobar/foobar.cer bob@bpm101

which returned: -

debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/davidhay/foobar/foobar.cer
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: No more authentication methods to try.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

ssh -vv -i ~/foobar/foobar.cer bob@bpm101

which returned: -

debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/davidhay/foobar/foobar.cer
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

ssh -vvv -i ~/foobar/foobar.cer bob@bpm101

which returned: -

debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/davidhay/foobar/foobar.cer
debug3: sign_and_send_pubkey: RSA SHA256:oIgBs7ffmebquR2tOmhRDF1lrCs/8VBmU03OyGyslCA
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

all of which was useful, but not that revealing.

I also started digging into the SSH logs on the server: -

journalctl _COMM=sshd

Dec 29 10:11:24 bpm101.novalocal sshd[29765]: Connection closed by 10.1.1.1 [preauth]
Dec 29 10:26:02 bpm101.novalocal sshd[30604]: Connection closed by 10.1.1.1 [preauth]
Dec 29 10:27:45 bpm101.novalocal sshd[30684]: Connection closed by 10.1.1.1 [preauth]
Dec 29 10:30:12 bpm101.novalocal sshd[30834]: Connection closed by 10.1.1.1 [preauth]

So I was at my wit's end …..

Then, last night, I started to wonder how I could compare the newly created public AND private key.

The internet brought me this: -

How to check if a RSA public / private key pair matched

This made reference to the ssh-keygen command, which can be used as follows: -

ssh-keygen -y -e -f foobar/foobar.cer

This inspects the private key ( foobar.cer ) and returns the corresponding public key: -

---- BEGIN SSH2 PUBLIC KEY ----
Comment: "2048-bit RSA, converted by 

dave@mac"

AAAAB3NzaC1yc7EAAAAAAQABAAABAQAAutYt3P7YNNOWG6Ve/xTpeNnkNwAARBzb0jiA7/
A7XT5x0xOVXrwrlefIU6ZFBfR1QLc54xCWPk7j77u4ZfsUE57+rSfFFHAm3T7/H5GVW7Ct
tS7mB/UoiXJtfOo9uNHNSrSOOqAQKAtAAprKzghREhoroIeb65TkzpA6cfAAmcE+ySCTNS
li7sALtAnBwCwFJGQUBKot77Xce1q04uhr9sYknOvRAVmTaihYAeaSgLh+UTAIeJ4/AABP
7+gAyuHK6pPJ9WEyUGHh3HAUio8o3uAx8VoZfnE+Y79y5XIpWT6+xrSG4XskRjPAy+9JsR
bsm3jS3qi08T649/aLL67A
---- END SSH2 PUBLIC KEY ——

I compared this to the public key that I'd stored on the server: -

2222B3Nz2C1yc2E2222D2Q2B222B2QCo23fBjN2DMrhlXsWb6J3+Wrdcy/j28EhylY+Uc8/DzmbCm1ycCEuCDUpBdldD+2YpTyjQkH8ot0r4DWQDCM52o/rw1wV58tZniO60jiwdEDYy6mO5oPdhib598UQs0Ny67dCRE7wW+2NVcrIs2F5DLWMnBxUlQcdxiiHmqDwZ/KF25SS9RvEuNDVhfd0+DkbnvWhtf2D12ODdeSMUJ2to+ztDDW+mf6tpDqQDwJRhITkn2dziDdDlpSkHSDZw4Q7QmkSNsuv9Tj1BWKFDfuIq+ns3SU6/sqi8QvIXc+KWRnk3893EJDYqBjNH5ut2itHjVnlMhx0Bh+Mpiilz/c5x

which, of course, didn't match.

I updated the server's authorized-keys file: -

vi .ssh/authorized_keys 

adding: -

ssh-rsa AAAAB3NzaC1yc7EAAAAAAQABAAABAQAAutYt3P7YNNOWG6Ve/xTpeNnkNwAARBzb0jiA7/A7XT5x0xOVXrwrlefIU6ZFBfR1QLc54xCWPk7j77u4ZfsUE57+rSfFFHAm3T7/H5GVW7CttS7mB/UoiXJtfOo9uNHNSrSOOqAQKAtAAprKzghREhoroIeb65TkzpA6cfAAmcE+ySCTNSli7sALtAnBwCwFJGQUBKot77Xce1q04uhr9sYknOvRAVmTaihYAeaSgLh+UTAIeJ4/AABP7+gAyuHK6pPJ9WEyUGHh3HAUio8o3uAx8VoZfnE+Y79y5XIpWT6+xrSG4XskRjPAy+9JsRbsm3jS3qi08T649/aLL67A

Once I saved the file, I was then able to connect: -

ssh -i ~/foobar/foobar.cer bob@bpm101

without problems.

So, long story even shorter, in order to use SSH and public/private keys for authentication, the public key ( stored on the target server ) needs to match the private key ( stored on the client ).

Perhaps that's obvious :-)

It is now …..