How to configure mTLS step-by-step on Linux (NGINX/OpenSSL)

How to configure mTLS step-by-step on Linux (NGINX/OpenSSL)

 

Configuring mutual TLS (mTLS) on Linux using NGINX and OpenSSL involves creating a certificate authority (CA), issuing server and client certificates, and configuring NGINX to require client authentication. Here’s a clear step-by-step guide.

---

1. Create a Certificate Authority (CA)

This CA will sign both server and client certificates.

mkdir -p ~/mtls && cd ~/mtls

# Generate CA private key
openssl genrsa -out ca.key 4096

# Generate CA certificate
openssl req -x509 -new -nodes -key ca.key -sha256 -days 3650 -out ca.crt

Fill in details (Common Name can be something like MyRootCA).

---

2. Create Server Certificate

Generate server key

openssl genrsa -out server.key 2048

Create CSR (Certificate Signing Request)

openssl req -new -key server.key -out server.csr

Use your domain or IP as Common Name (e.g., example.com or server IP).

Sign server certificate with CA

openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
-out server.crt -days 825 -sha256

---

3. Create Client Certificate

Generate client key

openssl genrsa -out client.key 2048

Create CSR

openssl req -new -key client.key -out client.csr

Sign client certificate

openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
-out client.crt -days 825 -sha256

---

4. Configure NGINX for mTLS

Edit your NGINX config (e.g., /etc/nginx/sites-available/default or nginx.conf):

server {
    listen 443 ssl;
    server_name your_domain_or_ip;

    ssl_certificate /path/to/server.crt;
    ssl_certificate_key /path/to/server.key;

    # Trust CA for client cert verification
    ssl_client_certificate /path/to/ca.crt;

    # Require client certificate
    ssl_verify_client on;

    # Optional: depth of cert chain
    ssl_verify_depth 2;

    location / {
        return 200 "mTLS authentication successful\n";
    }
}

---

5. Test NGINX Configuration

sudo nginx -t
sudo systemctl reload nginx

---

6. Test mTLS Connection

Without client cert (should fail)

curl https://your_domain_or_ip

With client cert (should succeed)

curl --cert client.crt --key client.key https://your_domain_or_ip

---

7. Optional: Combine Client Cert + Key (for browsers)

openssl pkcs12 -export -out client.p12 \
-inkey client.key -in client.crt -certfile ca.crt

You can import this .p12 into a browser.

---

8. Debugging Tips

• Check logs:

sudo tail -f /var/log/nginx/error.log

• Common issues:

  • CN mismatch → fix server_name or certificate CN

  • Unknown CA → ensure ssl_client_certificate points to correct CA

  • Permission issues → ensure NGINX can read cert files

---

9. Optional Security Enhancements

Add stronger TLS settings:

ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;

---

Concept Summary

• CA → trusted authority

• Server cert → proves server identity

• Client cert → proves client identity

• NGINX → enforces verification using ssl_verify_client on

---