FrankTheDevop FrankTheDevop FrankTheDevop FrankTheDevop FrankTheDevop

Tools

SSL Termination Stack Setup: Let´s encrypt, HAProxy, Your Stack

150 150 admin

Hi everyone,

for a setup at work I needed an quick and easy way to terminate an SSL Connection without hassle. After a short research I found it feasible to use Let´s encrypt for free SSL Certificates.But it looked like a lot of work to create the certificate so I searched for an quicker and hassle free approach. I found one, but it still took me a few hours to figure out how to use it correctly. And I want to save you the time.
My setup looked like this:
– Domain hosted at GoDaddy.com
– Server hosted at Digital Ocean (digitalocean.com*)
– Docker in Swarm Mode
– Portainer as UI
The expected outcome is:
– 1 Stack to (re-)generate the certificate
– 1-x Worker Stacks
If your Website is hosted somewhere else than GoDaddy that is no problem as long as you find it in this list: https://github.com/Neilpang/acme.sh/blob/master/dnsapi/README.md.
Let’s dive into the work:
1. Create API Credentials for GoDaddy / your supported Provider. How you do it depends on the provider, refer to this list: https://github.com/Neilpang/acme.sh/blob/master/dnsapi/README.md.
Remember to create Production keys, e.g. GoDaddy allows to create sandbox keys, those won’t work.
2. Deploy this stack config for the generation stack:
version: '3.5'
services:
  acme:
    command: daemon
    deploy:
      placement:
        constraints:
          - node.role == manager
      resources:
        reservations:
          cpus: '0.01'
          memory: 50M
    environment:
      DEPLOY_HAPROXY_PEM_PATH: /haproxy
      DEPLOY_HAPROXY_RELOAD: for task in $$(docker service ps SSL_system_haproxy -f desired-state=running -q); do docker run --rm -v /var/run/docker.sock:/var/run/docker.sock datagridsys/skopos-plugin-swarm-exec task-exec $$task /reload.sh; done
      
    hostname: '{{ .Service.Name }}-{{ .Task.Slot }}'
    image: interaction/acme.sh
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - acme-data:/acme.sh
      - nginx-data:/www
      - system_haproxy-data:/haproxy
  nginx:
    deploy:
      resources:
        reservations:
          cpus: '0.01'
          memory: 20M
    healthcheck:
      test: curl -f http://localhost || exit 1
    hostname: '{{ .Service.Name }}-{{ .Task.Slot }}'
    image: interaction/acme.sh-nginx
    ports:
      - 80:80
    volumes:
      - nginx-data:/www

volumes:
  acme-data:
    driver: local
    name: 'acme-data'
  nginx-data:
    driver: local
    name: 'nginx-data'
  system_haproxy-data:
    external: true

 

3. Go into your acme Container, either by docker exec -it $containerhash /bin/sh or via your UI.
4. For GoDaddy issue the commands:
4.1 export GD_Key=$yourkey
4.2 export GD_Secret=$yoursecret
4.3 acme.sh —issue -d “$yourFQDN” —dns dns_gd —dnssleep 15
4.4 acme.sh —deploy -d “$yourFQDN” —deploy-hook haproxy
The first two Commands 4.2 and 4.2 will set the required environment variables for the acme.sh script. In 4.3 replace $yourFQDN with the (sub-)domain you want the certificate be created for, e.g. web.stack.example.com.
With the last Command 4.4 you deploy the Certificate and let the script restart your HAProxy.
Let look at your worker stack. Here is my definition:
version: '3.5'
services:
  system_haproxy:
    image: 'dockercloud/haproxy:1.6.6'
    depends_on:
      - web
    deploy:
      resources:
        limits:
          memory: 512M
        reservations:
          memory: 256M
    environment:
      - CERT_FOLDER=/haproxy
      - DOCKER_HOST=127.0.0.1
      - 'EXTRA_GLOBAL_SETTINGS="debug"'
      - 'STATS_AUTH=admin:$password'
      - 'STATS_PORT=1936'
      - DOCKER_TLS_VERIFY
      - DOCKER_HOST
      - DOCKER_CERT_PATH
    volumes:
      - system_haproxy-data:/haproxy
      - /var/run/docker.sock:/var/run/docker.sock
    ports:
      - '443:443'
      - '1936:1936'
      
  web:
    image: 'dockercloud/hello-world:v1.0.0-frank'
    hostname: '{{ .Service.Name }}-{{ .Task.Slot }}'
    environment:
      - SERVICE_PORTS=$yourport
      - FORCE_SSL=yes
      - SSL_CERT=/haproxy/$crtificatename.pem
      - 'VIRTUAL_HOST=https://$yourFQDN'

volumes:
  system_haproxy-data:
    external: true

 

So what do we have here? A HAProxy Container with my default configuration + SSL Ports and the ENV Var CERT_FOLDER pointing to the folder where the certificate(s) are located. That is needed for the start up as HAProxy recognises, that you want SSL Termination and requires one of multiple ways (for more details see https://github.com/docker/dockercloud-haproxy/tree/master#ssl-termination).
The second entry is a test container you can find on docker hub, I just changed it to another port to reflect my own requirements. Normally the image is ‘dockercloud/hello-world’.
The important things here are the Environment Variables. VIRTUAL_HOST is probably already known by you. You can set the scheme to https instead of http and HAProxy recognises it. You also need the SERVICE_PORTS set to the ports you want to use on this container.
What is probably new to you is FORCE_SSL and SSL_CERT. FORCE_SSL enforces that every access to this container will be done securely via HTTPS. And SSL_CERT points to the location of the certificate we generated earlier. It is the mount point of the external container that is shared with the acme container.
After you deployed both stacks and issued the four commands in the acme container you are ready to go. When you open $yourFQN in your browser you should see something similar to this picture:
Congratulations! You now have a SSL terminated Stack you can easily develop and have no dependencies inside your worker stack(s). I hope I could save you quite some time so you can enjoy the benefits!
You can find the two stack definitions here: https://github.com/FrankTheDevop/ssl-termination-stack.
Feel free to use them :).
Kind Regards,
Frank
* Affiliate Link

Node.js Tooling I – Processmanager PM2

150 150 Frank

The purpose of this post is to help you get started with tools for Node.js in general and Loopback specifically to ease you life as Developer and Operator.

Prerequisite

You need a Node.js based API (to follow through this article. PM2 supports other languages too).

PM2 Processmanager

The PM2 Processmanager is a mighty one with many different options, has various integrations and even supports an online monitoring.

In this Article I present you the basic usage of it to get started fast. In a later Article I will help you migrate to Docker and add the online monitoring of your processes.

Installation

The Installation is as easy as npm install pm2 -g. It is important that you install it globally.

How to use it

You can manually start an api by issuing pm2 start app.js, but then you have to specify all parameter on every start. So it could be fast to test, but I recommend writing a small configuration file for it name process.yml.

Configuration Syntax

PM2 offers multiple syntax variants for the configuration file. Currently they support Javascript, JSON and YAML format. I prefer to create this configuration in YAML so I will present it to you in this syntax. For the others please have a look at Process Configuration File Syntax (1)

process.yml

Most often my configuration looks like this:

apps:
  - script: path_to_startup_script
    name: api_name
    exec_mode: cluster
    instances: 1
    env:
      DEBUG_FD: 1
      DEBUG_COLORS: 1

 
apps is the root element.

Each script entry refers to an app or api that should be started. Here you define the path to the file that starts your api.

You can either define one app per app/api or you can even define a stack you want to start with multiple entries.

Personally I use one configuration file per api in development and one configuration for a stack in the staging environment before I deploy to docker.

The name describe the name you will see in the process manager when you list the running processes.

With exec_mode and the following instances things get interesting. If you define the mode fork you can only start up one instance of this app/api. But if you define cluster then you´re able to scale the app/api with one single command and pm2 will load balance it for you!

instances defines how many concurrent instances you want to launch of this app/api at startup. I set this normally to 1 and adjust on the fly according to the needs.
This way I can already get a first idea of the need to scale.

With the env you can specify environment variables you want to set.
DEBUG_FD: 1 tells Node.js to change the output stream to process.stdout.
DEBUG_COLORS: 1 will add colors to the pm2 log output. This is handy because you see on the first glance if the logs message is an error or not.

These have not been all possibly Attributes for the configuration file. If you want a tighter control have a look at the Configuration File Attributes (2).

After this explanation you will find my configurations for a Express based API and a Loopback based API.

Express Example

apps:
  - script: bin/www
    name: api
    exec_mode: cluster
    instances: 1
    env:
      DEBUG_FD: 1
      DEBUG_COLORS: 1
      NODE_ENV: staging

Loopback Example

apps:
  - script: server/server.js
    name: loopback_api
    exec_mode: cluster
    instances: 1
    env:
      DEBUG_FD: 1
      DEBUG_COLORS: 1

Commands

After we defined the configuration we need to interact with the Processmanager to start and stop our APIs and check the log output.

Listing currently running Processes

To view all currently running Processes issue pm2 list at a console.

Start an API with an process.x

Starting your API with an configuration file is as easy as the command pm2 start process.x (x is config.js, json or yml).
After this command PM2 starts your API with the specified configuration and outputs it list of currently running processes.

Stopping an API

You can stop your API with pm2 stop process.x.
Important to know is that PM2 just stops your API then, but won´t remove it from the prepared to run process list. If you want to remove it cleanly and make sure on the next start you have a clean plate you have to destroy it.

Destroying an prepared to run API entry

To remove a API entry from the prepared to run list you issue pm2 delete process.x

Check the Logs

To check all logs without filtering to one API you issue pm2 logs.

If you want to filter the logs to one specific API you can add it to the command like this pm2 logs name_of_your_api

If you have any question post them into the comments. Or feel free to send my an email.

Yours sincerely,

Frank

Sources:
(1) http://pm2.keymetrics.io/docs/usage/application-declaration/
(2) http://pm2.keymetrics.io/docs/usage/application-declaration/#attributes-available