Docker 101 - Docker Compose with Environment Variables

 

Let's modify the previous exp 'Flask + Redis' to make environment variables to include some sensitive data (password)

Flask + Redis (require password)

Sample Code Reference

docker-compose.yml

version: "3.9"

services:

  web:

    build:

      context: ./

      dockerfile: Dockerfile

    ports:

      - "5000:5000"

    volumes:

       - .:/code

     environment:

       # Setup a sensitive environment variable

       - REDIS_PASSWORD=my_pwd

     networks:

       - my-bridge

   redis-server:

     image: "redis:alpine"

     # Run server with password

     command: redis-server --requirepass my_pwd

     networks:

       - my-bridge

 networks:

   my-bridge:

First, we need to override the command of 'redis-server' service to make sure the redis server is protected by password.

Second, in 'web' service, we define redis password as an environment variable called 'REDIS_PASSWORD' for app.py to use.

app.py

import time

import os

import redis

from flask import Flask

app = Flask(__name__)

# get redis passwrod from environment variables

redis_password = os.environ["REDIS_PASSWORD"]

# Using StrictReis with password

cache = redis.StrictRedis(host="redis-server", port=6379,

password=redis_password)

def get_hit_count():

    retries = 5

    while True:

        try:

            return cache.incr("hits")

        except redis.exceptions.ConnectionError as exc:

            if retries == 0:

                raise exc

            retries -= 1

            time.sleep(0.5)

@app.route("/")

def hello():

    count = get_hit_count()

    return "Hello World! I have been seen {} times.\n".format(count)

In order to connect to a redis server with password, we need to use StrictRedis.

And we pass the redis password by reading environment variable we set in container instead of hard-code it in app.py.

Then after running 'docker compose up', we can see the same result like the previous exp.

But if we share this docker-compose.yml file to others, then they will know our redis credential.

In order to deal with this situation, docker compose allow us to setup default values for environment variables in .env file. Reference

First of all, create a .env file under the parent directory of docker-compose.yml.

.env

REDIS_PASSWORD=my_pwd

docker-compose.yml

version: "3.9"

services:

  web:

    build:

      context: ./

      dockerfile: Dockerfile

    ports:

      - "5000:5000"

    volumes:

       - .:/code

     environment:

# Bind shell/env_file environment variables to containers'

       - REDIS_PASSWORD=${REDIS_PASSWORD}

     networks:

       - my-bridge

   redis-server:

     image: "redis:alpine"

     # Run server with password

# (which is bound to shell/env_file environment variables )

     command: redis-server --requirepass ${REDIS_PASSWORD}

     networks:

       - my-bridge

 networks:

   my-bridge:

In docker-compose.yml, we can substitute variables by shell/env_file environment variables.

Then we can see the same result in browser.

Also, we can use the following command to see the value after impetrated by .env.

Ex

$ docker compose convert

Result:

name: compose-env-exp-2

  services:

    redis-server:

        command:

        - redis-server

        - --requirepass

        - my_pwd

        image: redis:alpine

        networks:

          my-bridge: null

    web:

        build:

          context: /home/fcheng/Exp/compose-env-exp-2

          dockerfile: Dockerfile

        environment:

          REDIS_PASSWORD: my_pwd

        networks:

          my-bridge: null

        ports:

        - mode: ingress

          target: 5000

          published: "5000"

          protocol: tcp

        volumes:

        - type: bind

          source: /home/fcheng/Exp/compose-env-exp-2

          target: /code

          bind:

            create_host_path: true

    networks:

      my-bridge:

        name: compose-env-exp-2_my-bridge

Also, we can define multiple .env file such as dev.env, staging.env and prod.env and using --env-file option to load it when running 'docker compose up'

For example, let's create a file called prod.env.

prod.env

REDIS_PASSWORD=my_prod_pwd

And we can compare the result between the following two commands.

Ex

$ docker compose convert

Result:

  name: compose-env-exp-2   services:   redis-server: command: - redis-server - --requirepass - my_pwd image: redis:alpine networks: my-bridge: null   web: build: context: /home/fcheng/Exp/compose-env-exp-2 dockerfile: Dockerfile environment: REDIS_PASSWORD: my_pwd networks: my-bridge: null ports: - mode: ingress target: 5000 published: "5000" protocol: tcp volumes: - type: bind source: /home/fcheng/Exp/compose-env-exp-2 target: /code bind: create_host_path: true   networks:   my-bridge: name: compose-env-exp-2_my-bridge

Ex

$ docker compose --env-file prod.env convert

Result:

name: compose-env-exp-2 services: redis-server: command: - redis-server - --requirepass - my_prod_pwd image: redis:alpine networks: my-bridge: null web: build: context: /home/fcheng/Exp/compose-env-exp-2 dockerfile: Dockerfile environment: REDIS_PASSWORD: my_prod_pwd networks: my-bridge: null ports: - mode: ingress target: 5000 published: "5000" protocol: tcp volumes: - type: bind source: /home/fcheng/Exp/compose-env-exp-2 target: /code bind: create_host_path: true networks: my-bridge: name: compose-env-exp-2_my-bridge