Docker 101 - Containerizing our first application.

August 30th, 2017 · Estimated reading time: 35 minutes.

In this post we will learn what Docker is, how to use it, when it is appropiate to use it and we’ll make our first docker container based on a real-world production scenario.

What is a container?

Containers can be conceptually thought of as an advanced version of chroot or a lighter alternative to virtualization.

A container is an isolated user-level instance of the OS. From the point of view of the applications running inside these instances the container behaves exactly like a real computer, and the application will have access only to the resources that are explicitly assigned to it.

Containers share the running kernel and general system calls of the host OS, which means that it is significantly less demanding than traditional virtualization resource-wise. The downside to this is that you can’t have an application running in a container with a different OS than the host.

Pros and cons of using containers.

Using containers has a few inmediate advantages:

  • We can package applications along with their dependencies, creating a portable version of the app and eliminating the dreaded “but it works on my machine“-scenario. This helps eliminate the proverbial walls between the Operations departament and Development’s.

  • It is significantly lighter than traditional OS virtualization, enabling a higher compute density in the same hardware.

  • Reduced the effort required to maintain the runtime environment, along with its complexity. Since the applications can be treated as an opaque self-contained bundle, the package and the behaviour are the same in testing and in production.

  • Since the infrastructure is declared as code, we benefit from the advantages of IaC: Infrastructure versioning, code as reliable documentation and straightforward deployment of new environments.

Of course, we must mention the disadvantages too:

  • Containers run on top of an additional abstraction layer compared to bare-metal.

  • Containers share the running kernel with the host. A bug/glitch in the running kernel affects all the running containers.

  • Container management is challenging, though tools like Docker Swarm and Google Kubernetes can mitigate this.

  • GUI applications don’t work well. While we can work around this using X11 forwarding, it’s not a straightforward solution.

Real-world examples of containerized applications.

What is Docker?

Docker is a software product that implements container support. Thanks to Docker, we can package applications along with their dependencies and libraries and execute them in an isolated environment.

Among other features, Docker provides support for isolated networking and directly exposing physical devices to the container. Docker can also download pre-built images from Docker Hub or a private repository.

What containers, and by extension Docker, are not.

Containers are not a general-purpose solution to the application packaging problem. While containerizing an application has many advantages, it will often not be a straightforward process and the time and effort invested might prove too high.

To use an obvious example, would it make sense to distribute Skype in a container-like package? No.

  • Skype is a monolithic application, and splitting it into separate container-like components makes no practical sense.

  • It would need to have access to the host OS’s audio server. On PulseAudio you could work around it by establising a network audio sink on the host, but we would have to run an additional audio server.

  • A container can not natively communicate with the display server. There are workarounds for this but they involve either X11 forwarding through the network or sharing the X11 socket and breaking the isolation paradigm

If Skype would benefit from the advantages of containerization (ease of deployment, isolation, being incorporated into a continuous integration pipeline) then these tradeoffs might be worthwile.

Installing Docker

We are going to be using Docker 17.06.1-ce on Ubuntu 17.04.

The most recent release and installation instructions for Ubuntu are available on Docker’s website along with the rest of the supported operating systems and architectures.

First step is making sure we have the tools to add HTTPS repositories to our package sources.

sudo apt-get install apt-transport-https ca-certificates curl software-properties-common

Let’s add Docker’s official GPG key so we can verify packages, afterwards we double-check that the GPG key matches.

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo apt-key fingerprint 0EBFCD88

# At the time of writing this article, the key fingerprint is: 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88.

There are three Docker release rings: stable, edge and test. Unless you really need functionality or fixes not present in the stable ring, you should stick to the stable production releases.

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

If you would like to switch release rings, add the word edge or test after the word stable in the previous command.

Now, let’s refresh the package list and install docker.

sudo apt-get update
sudo apt-get install docker-ce

Docker containers currently have to be managed by the root user (or the equivalent in your chosen platform). Due to Docker’s design and capabilities, there is no way around this requirement. There are plans to eventually remove this restriction, but at the time of writing this post there is no actual roadmap on this matter.

As a result of the above concerns, only trusted users should be allowed to have access to the Docker daemon or any of the related utilities.

Optional but recommended post-install steps.

Removing the need for sudo.

If you don’t want to use sudo when you use the docker command, create a Unix group called docker and add users to it. When the docker daemon starts, it makes its socket read/writable by the docker group.

Warning: The docker group grants privileges equivalent to the root user. For details on how this impacts security in your system, see Docker Daemon Attack Surface.

Creating the docker group and adding your user to it is a straightforward process.

sudo groupadd docker
sudo usermod -aG docker `whoami`

You will need to log out of the session completely, and log back in.

Configure the Docker daemon to start on boot.

sudo systemctl enable docker

Auto-completion of commands for Docker.

Docker contains a variety of commands and the image and containers are identified by their IDs or, if we manually specify so, their friendly names. It is possible to integrate shell auto-completion with Docker commands and parameters, making our job a little easier.

We are going to implement this for Bash on Ubuntu 16.04.

First of all, we need the bash-completion package. In the rare case that it’s not already installed in our distribution, we can install it via the package manager.

sudo apt install bash-completion

Last step is to download the completion script and place it in /etc/bash_completion.d/

curl -L https://raw.githubusercontent.com/docker/compose/master/contrib/completion/bash/docker-compose -o /etc/bash_completion.d/docker-compose

Every new instance of bash will make use of the newly extended completion funtionality.

Up-to-date instructions for Bash and other shells are available on the website.

Running our very first container, and a look behind the scenes.

Hello World.

To test our Docker installation we are going to run a container supplied by Docker, its only function being displaying a welcome message.

$ sudo docker run hello-world

It results in the following output in the terminal.

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
Digest: sha256:f3b3b28a45160805bb16542c9531888519430e9e6d6ffc09d72261b0d26ff74f
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To be able to understand what just happened, we need to take a step back and examine the process of creating a container.

Docker workflow and container lifecycle.

A container is composed of two layers: the base isolated environment in which the application or service will be executed, image in Docker parlance; and the application or service themselves that we are going to run. Both of these are specified in a Dockerfile, which is nothing more than a plain-text file named Dockerfile.

For example, let’s examine the hello-world Dockerfile.

FROM scratch
COPY hello /
# Separator, to be used in a few moments.
CMD ["/hello"]

All instructions in a Dockerfile refer to the process of composing the parent image, except the CMD instruction which specifies the command that will be executed when the container is run.

FROM is a reserved keyword that establishes the environment in which the image will be built on. scratch is a minimal Docker environment, but we could specify another Docker image as a base such as an Ubuntu or CentOS environment.

COPY copies the specified file to the destination, in this case hello to the root of the filesystem.

CMD executes a binary called hello when the container is run.

Let’s examine a more complex example, a Python 3.4.3 only runtime environment based on Debian Jessie.

Be careful to not confuse RUN with CMD. RUN is executed when building the parent image, and CMD is the instruction that is ran when launching the container. Think of CMD as the command that launches your service, and RUN executes image-building steps.

# Builds on top of a Jessie base image.
FROM buildpack-deps:jessie 

# Remove the bundled python version, and everything related.
RUN apt-get purge -y python.*

ENV LANG C.UTF-8

RUN gpg --keyserver ha.pool.sks-keyservers.net --recv-keys 97FC712E4C024BBEA48A61ED3A5CA953F73C700D

ENV PYTHON_VERSION 3.4.3

ENV PYTHON_PIP_VERSION 7.1.0

# Downloads Python 3.4.3 from the official sources, and compiles it.
RUN set -x \
	&& mkdir -p /usr/src/python \
	&& curl -SL "https://www.python.org/ftp/python/$PYTHON_VERSION/Python-$PYTHON_VERSION.tar.xz" -o python.tar.xz \
	&& curl -SL "https://www.python.org/ftp/python/$PYTHON_VERSION/Python-$PYTHON_VERSION.tar.xz.asc" -o python.tar.xz.asc \
	&& gpg --verify python.tar.xz.asc \
	&& tar -xJC /usr/src/python --strip-components=1 -f python.tar.xz \
	&& rm python.tar.xz* \
	&& cd /usr/src/python \
	&& ./configure --enable-shared --enable-unicode=ucs4 \
	&& make -j$(nproc) \
	&& make install \
	&& ldconfig \
	&& pip3 install --no-cache-dir --upgrade pip==$PYTHON_PIP_VERSION \
	&& find /usr/local \
		\( -type d -a -name test -o -name tests \) \
		-o \( -type f -a -name '*.pyc' -o -name '*.pyo' \) \
		-exec rm -rf '{}' + \
	&& rm -rf /usr/src/python

RUN cd /usr/local/bin \
	&& ln -s easy_install-3.4 easy_install \
	&& ln -s idle3 idle \
	&& ln -s pydoc3 pydoc \
	&& ln -s python3 python \
	&& ln -s python-config3 python-config

# This is the command that will be run when the container is launched.
CMD python3 -c 'print("Hello World, I am a blog.")'

If we build this image, Docker will include everything in the current directory and then execute these instructions.

docker build -t python3-hello-world .

After the process finishes, we can create a container based on the image and run it.

$ docker run python3-hello-world

Hello World, I am a blog.

A complete Dockerfile syntax and documentation reference are available on Docker’s website.

Adding persistent storage.

As stated previously, container storage is volatile by definition. That is, the changes are not saved and the state is discarded on shutdown. If we were to run a process that needs persistent storage, like a database server, this would not be ideal to say the least.

Docker offers a solution for this problem: mounts.

Mounts

There are three kinds of mounts, two of which we can use to offer storage that will persist outside of the container.

  • Volumes: are managed by docker, exist outside of the application’s lifecycle and are portable. Good for sharing data among containers, and the preferred way of storing data. Volumes are stored on the host filesystem, under /var/lib/docker/volumes.

  • Bind mounts are traditional folders on the host machine that are available inside the container on a specific path. Good for sharing data between the host system and the container.

  • tmpfs mounts: These are RAM filesystems that behave exactly like traditional tmpfs mounts and will be discarded on shutdown. Should only be used for temporary files, and it offers the speed advantage of being stored entirely on RAM.

Criteria for choosing between volumes, bind mounts or tmpfs.

Note: You may encounter the -v or –volume syntax on older documentation referring to data storage in Docker. This syntax has been deprecated, and it’s oficially replaced by the –mount syntax which we will use throughout this post. An extended explanation is available on the official documentation.

Volumes can be shared between containers, are portable and are easier to back up and restore. They are able to be populated with the appropiate contents on deployment, and can be marked read-only. A common example would be hosting a website, and allowing the user to upload modifications via FTP. By sharing the volume, the apache2 process is able to read /var/www from the same place that the FTP server is able to write; at the same time we are keeping everything isolated.

On a Dockerfile you would declare a Volume with the following syntax: VOLUME /var/log.

Bind mounts work best in a scenario in which we need to preserve data from the container. For example, we need to share source code with the container, keep the container’s /etc/resolf.conf synced with the host’s or keep API secrets up to date inside the container. It could also be used to write the container’s logs into specific folders on the host.

Bind mounts cannot be declared from a Dockerfile, since the directories specified are not portable by definition and might not exist at runtime on the host machine. To use a bind mount, you must launch the container with the –mount parameter as we will see in the example at the end of this post.

tmpfs mounts are best used when the application generates non-persistent state data, and we need to eliminate the disk as a potential bottleneck when storing and accesing these files.

Networking.

Although we can deploy our own custom networks, Docker installs 3 networks automatically: bridge which is the default network for all containers and is isolated from the host, none which means that the container will have no network connectivity and host will bind the container to the host’s network stack.

host networking means that the container will be reachable from outside the host, and that the network is not isolated. For instance, a web server that runs in port 80 inside the container is automatically reachable through the host’s port 80.

Docker Engine natively supports two kinds of networks: A bridge network, limited to a single host; and an overlay network, which can span across multiple hosts.

In this post we are going to stick to the default bridge network, to deploy our test application in a isolated form.

A full reference for the networking internals of Docker is available on the manual.

A hypothetical real-world production example

Description of the scenario

Our company, WidgetMaker Inc, ships embedded devices that run a custom Linux distribution. To deploy our application code to these devices, we use a legacy app that was custom made for our needs a few years ago. This app requires:

  • Ubuntu 12.04 with a specific set of libraries and versions to function properly.

  • The application needs hardware access to a serial device to flash the generated binaries to our physical widgets.

  • It needs network access to show a web control panel, through which the application is controlled. Security was not a concern at the time of the design, which means a malicious actor could easily access the control panel and interfere with the process.

  • It will not work properly if multiple instances of the application are run at the same time, meaning that we can only deploy one widget at the time. The current workaround to perform this process in paralell is to have a few physical machines replicating the required configuration, and to manually operate them.

  • Retooling this app, while technically feasible, would be impractical for both budget and time-to-market reasons and our current budget is better spent elsewhere.

Integrating this application into the general DevOps pipeline would allow us to save time and money:

  • The specific environment the app needs is now inmutable and easily recreated. If there is a sudden hardware failure, we are minutes away from having a replacement ready to work.

  • We can isolate the network, and only allow access following secure criteria.

  • We could automate the deploying process: Instead of a human operator triggering the flashing procedures, we could trigger the flashing process when the human plugs the widget into the serial adapter and perform an action to notify the user when the deployment is complete.

  • We can add automated unit testing to the deployment procedure: Through the use of tools like Jenkins we can make sure that every single widget operates properly after the deployment, and guarantee that the units we ship to our customers are working correctly.

Designing the container.

Determining the requirements.

  • Ubuntu 12.04, with a series of specific libraries and settings.

  • Passthrough of /dev/ttyACM0 from the host OS.

  • Isolated network, with port 80 accessible only from our development machine.

  • The container needs access to the folder hostOS:/workspace/widgetOS/deployment-image/ to get the latest version of the applications files for flashing purposes.

    • A technically better way to do this would be to have the app code and the binaries stored on a repository that we could clone on launch, to ensure that every component is up to date. Since we want to learn how to inject files and folders for this example, we will not.

Building the base image.

Docker Hub already contains a pre-built Ubuntu 12.04 image, but we are going to build it manually from scratch.

If you wanted to use the Docker Hub image, you would just run docker pull ubuntu:12.04 to download it. In this particular case the image does come from Canonical themselves, but Docker Hub offers no guarantees and anyone could upload an image and pretend that it is trustworthy.

Since this is a production scenario, it is not recommended to use Docker Hub unless we can fully certify that the image comes unaltered from a trusted source.

We are going to use debootstrap to install a base Ubuntu system into /tmp/precise-pangolin, and then we’ll make Docker use it as a base image before making any modifications.

First we need to import the relevant GPG keys to be able to authenticate the packages. In this particular case, we need to install the package ubuntu-keyring, and pass a parameter to debootstrap so it used the old keys for Ubuntu releases.

sudo apt-get install ubuntu-keyring

mkdir /tmp/precise-pangolin
cd /tmp/precise-pangolin
sudo debootstrap --keyring /usr/share/keyrings/ubuntu-archive-removed-keys.gpg  --arch=i386 precise precise http://old-releases.ubuntu.com/ubuntu

Now debootstrap will download all the base packages. After the process is finished, we import this into docker to use it as the base image.

$ sudo tar -C precise -c . | docker import - precise
sha256:08d585429b174a4c27c89c47adf6f7bbb76d7cf3a246ca67562f257a7c66dc2d

If we list the images, we can see that our newly created Precise Pangolin image is ready to use.

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
precise             latest              08d585429b17        About a minute ago   168MB

This will become our standard Ubuntu 12.04 base image. We are going to use it as a parent image, and we will customize it to our particular app needs.

Applying the necessary customizations with a Dockerfile.

We are going to write a Dockerfile that will incorporate the necessary requirements for our app, and they will be applied to the base Ubuntu image.

Docker builds the image through layers: think of layers as version-control, if you make a change Docker only has to rebuild from that change onwards since the previous layers remain unchanged.

Two of our requirements can’t be coded into the Dockerfile: shared folders and shared devices. We will instead satisfy them at launch time.

For the purposes of recreating this scenario here are the contents of the file requirements.txt, which is present in the same folder as the Dockerfile.

Flask
Redis

The following are the Dockerfile contents, along with comments explaining what the different directives are and how they help us fullfil the app’s requirements.

# Specify the image we want to build on top on.
# "Parent image", in Docker-speak.
FROM precise

# LABEL lets you specify metadata for the container. This metada can be shown with `docker inspect`.

LABEL	maintainer="josue@josuealvarezmoreno" \
		copyright="WidgetMaker Inc" \
		deviceSupportStatus.widgy="Stable" \
		deviceSupportStatus.widgeton="Release Candidate 3" \
		deviceSupportStatus.fidly="Beta"

# The EXPOSE instruction informs Docker that the container listens on the specified network ports at runtime. EXPOSE does not make the ports of the container accessible to the host. To do that, you must use either the -p flag to publish a range of ports or the -P flag to publish all the exposed ports.
EXPOSE 80

# With the ENV instruction we can set enviroment variables.
# In our case, the software checks for a specific enviroment variable as a form of license validation before flashing, ensuring that users can't accidentally damage their devices.
ENV magicNumber="8974c408d831"

# apt-get should be aware that this is not an interactive session, and assume sane defaults.
ENV DEBIAN_FRONTEND noninteractive

# These RUN commands are executed when building the image, and will mold the enviroment into the shape we need for the application to run.

RUN apt-get update
RUN apt-get install -y curl

WORKDIR /tmp/

RUN curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
RUN python get-pip.py


# The ADD instruction will copy the file or directory contents into the image.
# Any change to the specified file will invalidate the cache for all following instructions, causing the image to be rebuilt if changes are detected.

# Copies the latest requirements modifications to the appropiate directory.

ADD requirements.txt /tmp/

# Changes the working directory, equivalent to 'cd'.
WORKDIR /tmp

# Install the requirements for the python part of the app. This requirements file is located on /tmp/
RUN pip install -r requirements.txt

# A CMD instruction is only executed when launching the container, and there can be only one in a Dockerfile.
# Runs the actual deployment program
CMD python /workspace/widgetOS/deployment-image/flasher.py

Building and launching the container.

To build the container, we need to initiate the build process from the Dockerfile directory. We are going to call this container widget-deployer.

$ sudo docker build -t widget-deployer .

Note: The build process copies every file in the current directory into the Docker image.

Sending build context to Docker daemon  6.144kB
Step 1/14 : FROM precise
 ---> 0e0cf7094433
Step 2/14 : LABEL maintainer "josue@josuealvarezmoreno" copyright "WidgetMaker Inc" deviceSupportStatus.widgy "Stable" deviceSupportStatus.widgeton "Release Candidate 3" deviceSupportStatus.fidly "Beta"
 ---> Running in 545a4bbd6902
 ---> 1abe7d9bf44e
Removing intermediate container 545a4bbd6902
Step 3/14 : EXPOSE 80
 ---> Running in c72f0bf476ba
 ---> 82304f3016b5
Removing intermediate container c72f0bf476ba
Step 4/14 : ENV magicNumber "8974c408d831"
 ---> Running in 06560f51ec54
 ---> 02a10485b58c
Removing intermediate container 06560f51ec54
Step 5/14 : ENV DEBIAN_FRONTEND noninteractive
 ---> Running in 408e767a5a63
 ---> f6c07c4eafe3
Removing intermediate container 408e767a5a63
Step 6/14 : RUN apt-get update
 ---> Running in 9778028dce0e
Ign http://old-releases.ubuntu.com precise InRelease
Hit http://old-releases.ubuntu.com precise Release.gpg
Hit http://old-releases.ubuntu.com precise Release
Hit http://old-releases.ubuntu.com precise/main i386 Packages
Get:1 http://old-releases.ubuntu.com precise/main TranslationIndex [3706 B]
Get:2 http://old-releases.ubuntu.com precise/main Translation-en [726 kB]
Fetched 729 kB in 1s (479 kB/s)
Reading package lists...
 ---> d7e16bef7aab
Removing intermediate container 9778028dce0e
Step 7/14 : RUN apt-get install -y curl
 ---> Running in 90c016c6034c
Reading package lists...
Building dependency tree...
The following extra packages will be installed:
  ca-certificates krb5-locales libasn1-8-heimdal libcurl3 libgcrypt11
  libgnutls26 libgpg-error0 libgssapi-krb5-2 libgssapi3-heimdal
  libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal
  libhx509-5-heimdal libidn11 libk5crypto3 libkeyutils1 libkrb5-26-heimdal
  libkrb5-3 libkrb5support0 libldap-2.4-2 libp11-kit0 libroken18-heimdal
  librtmp0 libsasl2-2 libsasl2-modules libtasn1-3 libwind0-heimdal openssl
Suggested packages:
  rng-tools gnutls-bin krb5-doc krb5-user libsasl2-modules-otp
  libsasl2-modules-ldap libsasl2-modules-sql libsasl2-modules-gssapi-mit
  libsasl2-modules-gssapi-heimdal
The following NEW packages will be installed:
  ca-certificates curl krb5-locales libasn1-8-heimdal libcurl3 libgcrypt11
  libgnutls26 libgpg-error0 libgssapi-krb5-2 libgssapi3-heimdal
  libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal
  libhx509-5-heimdal libidn11 libk5crypto3 libkeyutils1 libkrb5-26-heimdal
  libkrb5-3 libkrb5support0 libldap-2.4-2 libp11-kit0 libroken18-heimdal
  librtmp0 libsasl2-2 libsasl2-modules libtasn1-3 libwind0-heimdal openssl
0 upgraded, 29 newly installed, 0 to remove and 0 not upgraded.
Need to get 3979 kB of archives.
After this operation, 12.1 MB of additional disk space will be used.
Get:1 http://old-releases.ubuntu.com/ubuntu/ precise/main libroken18-heimdal i386 1.6~git20120311.dfsg.1-2 [47.4 kB]
Get:2 http://old-releases.ubuntu.com/ubuntu/ precise/main libasn1-8-heimdal i386 1.6~git20120311.dfsg.1-2 [242 kB]
Get:3 http://old-releases.ubuntu.com/ubuntu/ precise/main libgpg-error0 i386 1.10-2ubuntu1 [14.4 kB]
Get:4 http://old-releases.ubuntu.com/ubuntu/ precise/main libgcrypt11 i386 1.5.0-3 [281 kB]
Get:5 http://old-releases.ubuntu.com/ubuntu/ precise/main libp11-kit0 i386 0.12-2ubuntu1 [33.6 kB]
Get:6 http://old-releases.ubuntu.com/ubuntu/ precise/main libtasn1-3 i386 2.10-1ubuntu1 [43.7 kB]
Get:7 http://old-releases.ubuntu.com/ubuntu/ precise/main libgnutls26 i386 2.12.14-5ubuntu3 [448 kB]
Get:8 http://old-releases.ubuntu.com/ubuntu/ precise/main libkrb5support0 i386 1.10+dfsg~beta1-2 [23.6 kB]
Get:9 http://old-releases.ubuntu.com/ubuntu/ precise/main libk5crypto3 i386 1.10+dfsg~beta1-2 [77.1 kB]
Get:10 http://old-releases.ubuntu.com/ubuntu/ precise/main libkeyutils1 i386 1.5.2-2 [7716 B]
Get:11 http://old-releases.ubuntu.com/ubuntu/ precise/main libkrb5-3 i386 1.10+dfsg~beta1-2 [367 kB]
Get:12 http://old-releases.ubuntu.com/ubuntu/ precise/main libgssapi-krb5-2 i386 1.10+dfsg~beta1-2 [120 kB]
Get:13 http://old-releases.ubuntu.com/ubuntu/ precise/main libhcrypto4-heimdal i386 1.6~git20120311.dfsg.1-2 [104 kB]
Get:14 http://old-releases.ubuntu.com/ubuntu/ precise/main libheimbase1-heimdal i386 1.6~git20120311.dfsg.1-2 [33.2 kB]
Get:15 http://old-releases.ubuntu.com/ubuntu/ precise/main libwind0-heimdal i386 1.6~git20120311.dfsg.1-2 [77.8 kB]
Get:16 http://old-releases.ubuntu.com/ubuntu/ precise/main libhx509-5-heimdal i386 1.6~git20120311.dfsg.1-2 [127 kB]
Get:17 http://old-releases.ubuntu.com/ubuntu/ precise/main libkrb5-26-heimdal i386 1.6~git20120311.dfsg.1-2 [240 kB]
Get:18 http://old-releases.ubuntu.com/ubuntu/ precise/main libheimntlm0-heimdal i386 1.6~git20120311.dfsg.1-2 [16.8 kB]
Get:19 http://old-releases.ubuntu.com/ubuntu/ precise/main libgssapi3-heimdal i386 1.6~git20120311.dfsg.1-2 [112 kB]
Get:20 http://old-releases.ubuntu.com/ubuntu/ precise/main libidn11 i386 1.23-2 [112 kB]
Get:21 http://old-releases.ubuntu.com/ubuntu/ precise/main libsasl2-2 i386 2.1.25.dfsg1-3 [69.5 kB]
Get:22 http://old-releases.ubuntu.com/ubuntu/ precise/main libldap-2.4-2 i386 2.4.28-1.1ubuntu4 [186 kB]
Get:23 http://old-releases.ubuntu.com/ubuntu/ precise/main librtmp0 i386 2.4~20110711.gitc28f1bab-1 [57.7 kB]
Get:24 http://old-releases.ubuntu.com/ubuntu/ precise/main openssl i386 1.0.1-4ubuntu3 [520 kB]
Get:25 http://old-releases.ubuntu.com/ubuntu/ precise/main ca-certificates all 20111211 [169 kB]
Get:26 http://old-releases.ubuntu.com/ubuntu/ precise/main libcurl3 i386 7.22.0-3ubuntu4 [242 kB]
Get:27 http://old-releases.ubuntu.com/ubuntu/ precise/main krb5-locales all 1.10+dfsg~beta1-2 [8886 B]
Get:28 http://old-releases.ubuntu.com/ubuntu/ precise/main libsasl2-modules i386 2.1.25.dfsg1-3 [60.5 kB]
Get:29 http://old-releases.ubuntu.com/ubuntu/ precise/main curl i386 7.22.0-3ubuntu4 [137 kB]
Preconfiguring packages ...
Fetched 3979 kB in 1s (2618 kB/s)
Selecting previously unselected package libroken18-heimdal.
(Reading database ... 9848 files and directories currently installed.)
Unpacking libroken18-heimdal (from .../libroken18-heimdal_1.6~git20120311.dfsg.1-2_i386.deb) ...
Selecting previously unselected package libasn1-8-heimdal.
Unpacking libasn1-8-heimdal (from .../libasn1-8-heimdal_1.6~git20120311.dfsg.1-2_i386.deb) ...
Selecting previously unselected package libgpg-error0.
Unpacking libgpg-error0 (from .../libgpg-error0_1.10-2ubuntu1_i386.deb) ...
Selecting previously unselected package libgcrypt11.
Unpacking libgcrypt11 (from .../libgcrypt11_1.5.0-3_i386.deb) ...
Selecting previously unselected package libp11-kit0.
Unpacking libp11-kit0 (from .../libp11-kit0_0.12-2ubuntu1_i386.deb) ...
Selecting previously unselected package libtasn1-3.
Unpacking libtasn1-3 (from .../libtasn1-3_2.10-1ubuntu1_i386.deb) ...
Selecting previously unselected package libgnutls26.
Unpacking libgnutls26 (from .../libgnutls26_2.12.14-5ubuntu3_i386.deb) ...
Selecting previously unselected package libkrb5support0.
Unpacking libkrb5support0 (from .../libkrb5support0_1.10+dfsg~beta1-2_i386.deb) ...
Selecting previously unselected package libk5crypto3.
Unpacking libk5crypto3 (from .../libk5crypto3_1.10+dfsg~beta1-2_i386.deb) ...
Selecting previously unselected package libkeyutils1.
Unpacking libkeyutils1 (from .../libkeyutils1_1.5.2-2_i386.deb) ...
Selecting previously unselected package libkrb5-3.
Unpacking libkrb5-3 (from .../libkrb5-3_1.10+dfsg~beta1-2_i386.deb) ...
Selecting previously unselected package libgssapi-krb5-2.
Unpacking libgssapi-krb5-2 (from .../libgssapi-krb5-2_1.10+dfsg~beta1-2_i386.deb) ...
Selecting previously unselected package libhcrypto4-heimdal.
Unpacking libhcrypto4-heimdal (from .../libhcrypto4-heimdal_1.6~git20120311.dfsg.1-2_i386.deb) ...
Selecting previously unselected package libheimbase1-heimdal.
Unpacking libheimbase1-heimdal (from .../libheimbase1-heimdal_1.6~git20120311.dfsg.1-2_i386.deb) ...
Selecting previously unselected package libwind0-heimdal.
Unpacking libwind0-heimdal (from .../libwind0-heimdal_1.6~git20120311.dfsg.1-2_i386.deb) ...
Selecting previously unselected package libhx509-5-heimdal.
Unpacking libhx509-5-heimdal (from .../libhx509-5-heimdal_1.6~git20120311.dfsg.1-2_i386.deb) ...
Selecting previously unselected package libkrb5-26-heimdal.
Unpacking libkrb5-26-heimdal (from .../libkrb5-26-heimdal_1.6~git20120311.dfsg.1-2_i386.deb) ...
Selecting previously unselected package libheimntlm0-heimdal.
Unpacking libheimntlm0-heimdal (from .../libheimntlm0-heimdal_1.6~git20120311.dfsg.1-2_i386.deb) ...
Selecting previously unselected package libgssapi3-heimdal.
Unpacking libgssapi3-heimdal (from .../libgssapi3-heimdal_1.6~git20120311.dfsg.1-2_i386.deb) ...
Selecting previously unselected package libidn11.
Unpacking libidn11 (from .../libidn11_1.23-2_i386.deb) ...
Selecting previously unselected package libsasl2-2.
Unpacking libsasl2-2 (from .../libsasl2-2_2.1.25.dfsg1-3_i386.deb) ...
Selecting previously unselected package libldap-2.4-2.
Unpacking libldap-2.4-2 (from .../libldap-2.4-2_2.4.28-1.1ubuntu4_i386.deb) ...
Selecting previously unselected package librtmp0.
Unpacking librtmp0 (from .../librtmp0_2.4~20110711.gitc28f1bab-1_i386.deb) ...
Selecting previously unselected package openssl.
Unpacking openssl (from .../openssl_1.0.1-4ubuntu3_i386.deb) ...
Selecting previously unselected package ca-certificates.
Unpacking ca-certificates (from .../ca-certificates_20111211_all.deb) ...
Selecting previously unselected package libcurl3.
Unpacking libcurl3 (from .../libcurl3_7.22.0-3ubuntu4_i386.deb) ...
Selecting previously unselected package krb5-locales.
Unpacking krb5-locales (from .../krb5-locales_1.10+dfsg~beta1-2_all.deb) ...
Selecting previously unselected package libsasl2-modules.
Unpacking libsasl2-modules (from .../libsasl2-modules_2.1.25.dfsg1-3_i386.deb) ...
Selecting previously unselected package curl.
Unpacking curl (from .../curl_7.22.0-3ubuntu4_i386.deb) ...
Setting up libroken18-heimdal (1.6~git20120311.dfsg.1-2) ...
Setting up libasn1-8-heimdal (1.6~git20120311.dfsg.1-2) ...
Setting up libgpg-error0 (1.10-2ubuntu1) ...
Setting up libgcrypt11 (1.5.0-3) ...
Setting up libp11-kit0 (0.12-2ubuntu1) ...
Setting up libtasn1-3 (2.10-1ubuntu1) ...
Setting up libgnutls26 (2.12.14-5ubuntu3) ...
Setting up libkrb5support0 (1.10+dfsg~beta1-2) ...
Setting up libk5crypto3 (1.10+dfsg~beta1-2) ...
Setting up libkeyutils1 (1.5.2-2) ...
Setting up libkrb5-3 (1.10+dfsg~beta1-2) ...
Setting up libgssapi-krb5-2 (1.10+dfsg~beta1-2) ...
Setting up libhcrypto4-heimdal (1.6~git20120311.dfsg.1-2) ...
Setting up libheimbase1-heimdal (1.6~git20120311.dfsg.1-2) ...
Setting up libwind0-heimdal (1.6~git20120311.dfsg.1-2) ...
Setting up libhx509-5-heimdal (1.6~git20120311.dfsg.1-2) ...
Setting up libkrb5-26-heimdal (1.6~git20120311.dfsg.1-2) ...
Setting up libheimntlm0-heimdal (1.6~git20120311.dfsg.1-2) ...
Setting up libgssapi3-heimdal (1.6~git20120311.dfsg.1-2) ...
Setting up libidn11 (1.23-2) ...
Setting up libsasl2-2 (2.1.25.dfsg1-3) ...
Setting up libldap-2.4-2 (2.4.28-1.1ubuntu4) ...
Setting up librtmp0 (2.4~20110711.gitc28f1bab-1) ...
Setting up openssl (1.0.1-4ubuntu3) ...
Setting up ca-certificates (20111211) ...
Updating certificates in /etc/ssl/certs... 152 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....done.
Setting up libcurl3 (7.22.0-3ubuntu4) ...
Setting up krb5-locales (1.10+dfsg~beta1-2) ...
Setting up libsasl2-modules (2.1.25.dfsg1-3) ...
Setting up curl (7.22.0-3ubuntu4) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
 ---> 16684411a89e
Removing intermediate container 90c016c6034c
Step 8/14 : WORKDIR /tmp/
 ---> 37e50202d457
Removing intermediate container 32057cba2fc6
Step 9/14 : RUN curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
 ---> Running in 17c757483d01
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1558k  100 1558k    0     0  3595k      0 --:--:-- --:--:-- --:--:-- 4793k
 ---> 383279846608
Removing intermediate container 17c757483d01
Step 10/14 : RUN python get-pip.py
 ---> Running in ac9cbc7aaf7a
Collecting pip
/tmp/tmpG9jzym/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
/tmp/tmpG9jzym/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
Collecting setuptools
  Downloading setuptools-36.3.0-py2.py3-none-any.whl (477kB)
Collecting wheel
  Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
Installing collected packages: pip, setuptools, wheel
Successfully installed pip-9.0.1 setuptools-36.3.0 wheel-0.29.0
/tmp/tmpG9jzym/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
 ---> ae059d740d83
Removing intermediate container ac9cbc7aaf7a
Step 11/14 : ADD requirements.txt /tmp/
 ---> 1f99f596bd4a
Removing intermediate container 41963c811b1c
Step 12/14 : WORKDIR /tmp
 ---> 04ea9ee57bfc
Removing intermediate container 568de414a28c
Step 13/14 : RUN pip install -r requirements.txt
 ---> Running in 43fcb99859f4
Collecting Flask (from -r requirements.txt (line 1))
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading Flask-0.12.2-py2.py3-none-any.whl (83kB)
Collecting Redis (from -r requirements.txt (line 2))
  Downloading redis-2.10.6-py2.py3-none-any.whl (64kB)
Collecting itsdangerous>=0.21 (from Flask->-r requirements.txt (line 1))
  Downloading itsdangerous-0.24.tar.gz (46kB)
Collecting click>=2.0 (from Flask->-r requirements.txt (line 1))
  Downloading click-6.7-py2.py3-none-any.whl (71kB)
Collecting Jinja2>=2.4 (from Flask->-r requirements.txt (line 1))
  Downloading Jinja2-2.9.6-py2.py3-none-any.whl (340kB)
Collecting Werkzeug>=0.7 (from Flask->-r requirements.txt (line 1))
  Downloading Werkzeug-0.12.2-py2.py3-none-any.whl (312kB)
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->Flask->-r requirements.txt (line 1))
  Downloading MarkupSafe-1.0.tar.gz
Building wheels for collected packages: itsdangerous, MarkupSafe
  Running setup.py bdist_wheel for itsdangerous: started
  Running setup.py bdist_wheel for itsdangerous: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/fc/a8/66/24d655233c757e178d45dea2de22a04c6d92766abfb741129a
  Running setup.py bdist_wheel for MarkupSafe: started
  Running setup.py bdist_wheel for MarkupSafe: finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/88/a7/30/e39a54a87bcbe25308fa3ca64e8ddc75d9b3e5afa21ee32d57
Successfully built itsdangerous MarkupSafe
Installing collected packages: itsdangerous, click, MarkupSafe, Jinja2, Werkzeug, Flask, Redis
Successfully installed Flask-0.12.2 Jinja2-2.9.6 MarkupSafe-1.0 Redis-2.10.6 Werkzeug-0.12.2 click-6.7 itsdangerous-0.24
 ---> 7eae346963a6
Removing intermediate container 43fcb99859f4
Step 14/14 : CMD python /workspace/widgetOS/deployment-image/flasher.py
 ---> Running in 12fd72c01d09
 ---> 33306df28e7c
Removing intermediate container 12fd72c01d09
Successfully built 33306df28e7c
Successfully tagged widget-deployer:latest

Sharing devices and folders

Adding a shared folder between the host and the container with a bind mount.

As we have seen already, mounts can’t be specified inside the Dockerfile for portability reasons. Thus, we will specify a read/write mount when launching the container. In our case, we want to share the /workspace folder.

Using the following syntax we can mount a folder inside the container.

--mount type=bind,source=<absolute path on host>,target=<destination on container>

Sharing a physical device with the container.

To allow passthrough access to a physical device we need to specify it at runtime:

--device <device path>

Running the container.

Launching the container is a very straightforward process. Once built, all we have to do is:

sudo docker run --device /dev/ttyACM0 --mount type=bind,source=/workspace,target=/workspace  widget-deployer

Once launched, we can inspect the running container’s metadata with the inspect command.

docker inspect widget-deployer
[
    {
        "Id": "sha256:33306df28e7c87155e70463be8053edb21575500daf74060672397e7c8fb0829",
        "RepoTags": [
            "widget-deployer:latest"
        ],
        "RepoDigests": [],
        "Parent": "sha256:7eae346963a686a0b67a8c3d30bc27f68ec6dc3fd7143cfe474d31278c2e9988",
        "Comment": "",
        "Created": "2017-08-30T07:48:11.591362982Z",
        "Container": "12fd72c01d093016912455b813cb6e0bd5f72ebc7c2f913f15582f1238af1813",
        "ContainerConfig": {
            "Hostname": "12fd72c01d09",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "magicNumber=8974c408d831",
                "DEBIAN_FRONTEND=noninteractive"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ",
                "CMD [\"/bin/sh\" \"-c\" \"python /workspace/widgetOS/deployment-image/flasher.py\"]"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:7eae346963a686a0b67a8c3d30bc27f68ec6dc3fd7143cfe474d31278c2e9988",
            "Volumes": null,
            "WorkingDir": "/tmp",
            "Entrypoint": null,
            "OnBuild": [],
            "Labels": {
                "copyright": "WidgetMaker Inc",
                "deviceSupportStatus.fidly": "Beta",
                "deviceSupportStatus.widgeton": "Release Candidate 3",
                "deviceSupportStatus.widgy": "Stable",
                "maintainer": "josue@josuealvarezmoreno"
            }
        },
        "DockerVersion": "17.06.1-ce",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "80/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "magicNumber=8974c408d831",
                "DEBIAN_FRONTEND=noninteractive"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "python /workspace/widgetOS/deployment-image/flasher.py"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:7eae346963a686a0b67a8c3d30bc27f68ec6dc3fd7143cfe474d31278c2e9988",
            "Volumes": null,
            "WorkingDir": "/tmp",
            "Entrypoint": null,
            "OnBuild": [],
            "Labels": {
                "copyright": "WidgetMaker Inc",
                "deviceSupportStatus.fidly": "Beta",
                "deviceSupportStatus.widgeton": "Release Candidate 3",
                "deviceSupportStatus.widgy": "Stable",
                "maintainer": "josue@josuealvarezmoreno"
            }
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 222672573,
        "VirtualSize": 222672573,
        "GraphDriver": {
            "Data": null,
            "Name": "aufs"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:3bb86384fde4b1091d120c6d7191e0527bf79ccceb768fde1243b749eceedc33",
                "sha256:6f1faf2a0e748987c7ee58e24c84bc908eec8ba5a1bdc0c060a402a3b1100480",
                "sha256:df0d2e8d340c5526f319bb7bab8fb8e298e0bd5445576cec7b16dc8184f99a13",
                "sha256:3f92e333309b33b824a6252899b24ad1cb864c288da06a3e63f4bb5ba5582124",
                "sha256:42e6185f24be913557cc64ea688499a9683fee14f299f80be16a9c2540b888e6",
                "sha256:60fb28685d5f6de80f6afbbd123c77c177e61b3a2a6ba66afd4747e4b8e53b20",
                "sha256:796de985371edcb6b0dcd0eadd802e3a8f475cd2cb66b12525a6246d97f87ec2"
            ]
        }
    }
]

Our app is now ready to use.

Running more than one instance of the app.

If we wanted to run many instances of the app in parallel, we would need:

  • An individual serial adapter per instance, passed through with the –device parameter. /dev/ttyACM1 for example.
  • To forward the exposed webserver port to a different port on the host, per instance. For example, we forward port 80 to port 1080.

The commandline would look like this.

docker run --device /dev/ttyACM1 --mount type=bind,source=/workspace,target=/workspace -p 1080:80 widget-deployer

And examining the running list of containers would show that the port is forwarded.

$ docker ps

CONTAINER ID        IMAGE               STATUS          PORTS                  
958b79eed6ca        widget-deployer     Up              127.0.0.1:1080->80/tcp
08a264882398        widget-deployer     Up              127.0.0.1:1081->80/tcp
92a8df94994b        widget-deployer     Up              127.0.0.1:1082->80/tcp
725c581d4dca        widget-deployer     Up              127.0.0.1:1083->80/tcp

Debugging a container: Stepping inside through a shell.

To launch a bash shell inside the container and inspect it’s contents, we can run the container and specify a custom command to run.

$ docker run --device /dev/ttyACM0 --mount type=bind,source=/workspace,target=/workspace -it widget-deployer /bin/bash 

root@44489cae5745:/tmp# ls /workspace/widgetOS/deployment-image/flasher.py 
/workspace/widgetOS/deployment-image/flasher.py

root@44489cae5745:/tmp# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
73: eth0@if74: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP 
    link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.2/16 scope global eth0
       valid_lft forever preferred_lft forever

Keep in mind that any manual changes made are lost on container termination, and they do not apply to newly launched instances.

Cleaning the images and temporary files we have created.

docker rm $(docker ps -a -q)
docker rmi $(docker images -q)
docker volume rm $(docker volume ls)

Conclusions.

Containers are a powerful and speedy addition to our devops workflow, and while they are not a one-size-fits-all solution, they do offer significant advantages over other application isolation alternatives.

In a future post we will explore how to use Docker Swarm to launch a complex network of containers, including web apps, load balancers and databases.