Monday, July 25, 2022

Docker 101 - Network



Network Drivers



List all networks (the list below is the default network created by Docker since we have not created any networks yet). Reference

Ex: 
$ docker network ls

Result:
NETWORK ID     NAME      DRIVER    SCOPE
09a5f34a0bb8   bridge    bridge    local
  f419db6a9e21   host host      local
  0175d39fdfe6   none null      local

*bridge:
The default network driver.
If you don’t specify a driver, this is the type of network you are creating.
Bridge networks are usually used when your applications run in standalone containers that need to communicate.

*host:
For standalone containers, remove network isolation between the container and the Docker host, and use the host’s networking directly.

*none(null):
 For this container, disable all networking.
 Usually used in conjunction with a custom network driver.


Bridge network



*In terms of networking: 
         A bridge network is a Link Layer device which forwards traffic between network segments.

*In terms of Docker:
         A bridge network uses a software bridge which allows containers connected to the same bridge network to communicate. It provides isolation from containers which are not connected to that bridge network.

Reference: document1, document2


Default bridge



When you start Docker, a default bridge network (also called bridge) is created automatically, and newly-started containers connect to it unless otherwise specified

First of all, let's check our network interface in our docker host.

Ex: 
$ ifconfig

Result:
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
inet6 fe80::42:62ff:fe8b:caa7  prefixlen 64  scopeid 0x20<link>
  ether 02:42:62:8b:ca:a7  txqueuelen 0  (Ethernet)
  RX packets 0  bytes 0 (0.0 B)
  RX errors 0  dropped 0  overruns 0  frame 0
  TX packets 9  bytes 1041 (1.0 KB)
  TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

'docker0' is the interface Docker created for us for default bridge network.

Ex: 
$ docker network inspect bridge

Result:
[
{
"Name": "bridge",
"Scope": "local",
"Driver": "bridge",
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.17.0.0/16",
"Gateway": "172.17.0.1"
}
]
},
"Containers": {},
"Labels": {},
...
}
]

Therefore, the gateway of the default bridge will be docker0 in Docker host.

Then, let's create two container without specifying network.

Ex: 
$ docker container run -d -it --name alpine_default_1 alpine sh
$ docker container run -d -it --name alpine_default_2 alpine sh

Result:
bd334a1aa24f97f61d048a7f459b8e7a93ac479805993f2b510fbae47c88abb7
  93122bbae5a31bf10041b2fd473244d34621ee563eabf496d33ba7346e10b8e5

Check the detail of each container separately.

Ex: 
$ docker container inspect bd33

Result:
"NetworkSettings": {
...
"Networks": {
"bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "09a5f34a0bb8118d5f064a6496c939a1ece5a817...",
"EndpointID": "755aa48ce8fdc5d6c750cd71f541d40069b2b06...",
"Gateway": "172.17.0.1",
"IPAddress": "172.17.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:11:00:02",
"DriverOpts": null
}
}
}

Ex: 
$ docker container inspect 9312

Result:
"NetworkSettings": {
...
    "Networks": {
"bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "09a5f34a0bb8118d5f064a6496c939a1ece5a817...",
"EndpointID": "9b9d9e1efa86c27ee1dd20f70bed3776f7f15ce...",
"Gateway": "172.17.0.1",
"IPAddress": "172.17.0.3",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:11:00:03",
"DriverOpts": null
}
    }
}

Ex: 
$ docker network inspect bridge

Result:
[
{
"Name": "bridge",
"Scope": "local",
"Driver": "bridge",
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.17.0.0/16",
"Gateway": "172.17.0.1"
}
]
},
"Containers": {
"93122bbae5a31bf10041b2fd473244d34621ee563eabf496d...": {
"Name": "alpine_default_2",
"EndpointID": "9b9d9e1efa86c27ee1dd20f70bed3776f7f15ce7...",
"MacAddress": "02:42:ac:11:00:03",
"IPv4Address": "172.17.0.3/16",
"IPv6Address": ""
},
"bd334a1aa24f97f61d048a7f459b8e7a93ac479805993f2b...": {
"Name": "alpine_default_1",
"EndpointID": "755aa48ce8fdc5d6c750cd71f541d40069b2b069...",
"MacAddress": "02:42:ac:11:00:02",
"IPv4Address": "172.17.0.2/16",
"IPv6Address": ""
}
},
"Labels": {}
}
]

Once it is all set, let's verify the connection between them.
Let's try to ping containers by IP address first.

Ex: 
$ docker container exec -it alpine_default_1 sh
/ # ping -c 1 172.17.0.3

Result:
PING 172.17.0.3 (172.17.0.3): 56 data bytes
64 bytes from 172.17.0.3: seq=0 ttl=64 time=0.056 ms

--- 172.17.0.3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.056/0.056/0.056 ms

Let's try to ping containers by the container name.

Ex: 
/ # ping -c 1 alpine_default_2

Result:
ping: bad address 'alpine_default_2'

Opps, since we are using the default bridge, we cannot use container name to find its IP address automatically.


User-defined Bridge



Docker allows us to create user-defined bridge, and actually the document indicates that we should use user-defined bridge instead of default bridge for those reasons.

Lets' create a user-defined bridge to dive deeper.

Ex: 
$ docker network create my-bridge

Result:
14988e726d808d3af667636c0e8051de92bca0d80a64f0f90aa65cbd555575fb

Ex: 
$ docker network ls

Result:
NETWORK ID     NAME        DRIVER    SCOPE
09a5f34a0bb8   bridge      bridge    local
f419db6a9e21   host        host      local
14988e726d80   my-bridge   bridge    local
0175d39fdfe6   none        null      local

Ex: 
$ docker network inspect my-bridge

Result:
[
{
"Name": "my-bridge",
"Scope": "local",
"Driver": "bridge",
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "172.18.0.0/16",
"Gateway": "172.18.0.1"
}
]
},
"Containers": {},
"Labels": {}
}
]

Ex: 
$ ifconfig

Result:
br-14988e726d80: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
inet 172.18.0.1  netmask 255.255.0.0  broadcast 172.18.255.255
ether 02:42:67:8c:86:48  txqueuelen 0  (Ethernet)
RX packets 0  bytes 0 (0.0 B)
RX errors 0  dropped 0  overruns 0  frame 0
TX packets 0  bytes 0 (0.0 B)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
inet6 fe80::42:62ff:fe8b:caa7  prefixlen 64  scopeid 0x20<link>
ether 02:42:62:8b:ca:a7  txqueuelen 0  (Ethernet)
RX packets 6  bytes 308 (308.0 B)
RX errors 0  dropped 0  overruns 0  frame 0
TX packets 15  bytes 1644 (1.6 KB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The gateway of the new created user-defined bridge will be br-14988e726d80 network in Docker host.

Then create containers and specify the network with this user-defined bridge.

Ex: 
$ docker container run -d -it --name alpine_user_defined_3 \
--network my-bridge alpine sh
$ docker container run -d -it --name alpine_user_defined_4 \
--network my-bridge alpine sh

Result:
8b43bd1b39467d8cacaf6a20e5f7c1cacbee288c55a3bfb73dba9cd89a9a6a6c
2e91d9793bc1fb4d740c5ca4df9a8c22f8fe4616c3c63ad388943bb248e464c0

Ex: 
$ docker container inspect 8b43

Result:
"NetworkSettings": {
"Networks": {
"my-bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"8b43bd1b3946"
],
"NetworkID": "14988e726d808d3af667636c0e8051de92bca0d80...",
"EndpointID": "fa8c9a1bbf57750bfed8c23049fdce56f7da7575...",
"Gateway": "172.18.0.1",
"IPAddress": "172.18.0.2",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:12:00:02",
"DriverOpts": null
}
}
}

Ex: 
$ docker container inspect 2e91

Result:
"NetworkSettings": {
"Networks": {
"my-bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"2e91d9793bc1"
],
"NetworkID": "14988e726d808d3af667636c0e8051de92bca0d80...",
"EndpointID": "f91c5cda192c8a9c48f2a46630ab542758dc0fe1...",
"Gateway": "172.18.0.1",
"IPAddress": "172.18.0.3",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:12:00:03",
"DriverOpts": null
}
}
}

Since we are using user-undefined bridge, we can use container name now.

Ex: 
$ docker container exec -it alpine_user_defined_3 sh
/ # ping -c 1 alpine_user_defined_4

Result:
PING alpine_user_defined_4 (172.18.0.3): 56 data bytes
64 bytes from 172.18.0.3: seq=0 ttl=64 time=0.135 ms

--- alpine_user_defined_4 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.135/0.135/0.135 ms


Port Expose



From the Dockerfile, the EXPOSE command is just as a type of documentation.
It informs the developers that the container created by this image will need to listens on the specified network ports at runtime.

Therefore, when running your container, you need to use -p to publish ports.
The exposed port is accessible on the host and the ports are available to any client that can reach the host. Reference


Ex: 
$ docker container run -d --name web1 \
--network my-bridge -p 8080:80 nginx

Result:
f8594b7f3db8824386923a38664e92b70363dfb8c82c45d46cb1e370aff6f3dc

Then we can access it with that port in our Docker host.

Ex: 
$ wget localhost:8080

Result:
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 615 [text/html]

index.html            100%[===============>]     615  --.-KB/s    in 0s

Let's check the iptables to see what rules Docker added.

Ex: 
$ sudo iptables -t nat -nvxL

Result:
Chain DOCKER (2 references)
target   prot opt source     destination
RETURN   all   --   0.0.0.0/0  0.0.0.0/0
RETURN   all   --   0.0.0.0/0  0.0.0.0/0
DNAT     tcp   --   0.0.0.0/0  0.0.0.0/0  tcp dpt:8080 to:172.18.0.2:80


Posted by at No comments:
Labels: ,

Friday, July 15, 2022

Docker 101 - Manage Data

  


Before introducing how to manage data in Docker, let's use the mysql image to do some experiment to see if data is persist in container or not.



MySQL database



Let's follow the official document to run a mysql container.


Ex:
$ docker container run -d -e MYSQL_ROOT_PASSWORD=secret_pwd mysql

Result:
Unable to find image 'mysql:latest' locally
  latest: Pulling from library/mysql
  e54b73e95ef3: Pull complete
  327840d38cb2: Pull complete
  642077275f5f: Pull complete
  e077469d560d: Pull complete
  cbf214d981a6: Pull complete
  7d1cc1ea1b3d: Pull complete
  d48f3c15cb80: Pull complete
  94c3d7b2c9ae: Pull complete
  f6cfbf240ed7: Pull complete
  e12b159b2a12: Pull complete
  4e93c6fd777f: Pull complete
  Digest: sha256:152cf187a3efc56afb0b3877b4d21e231d1d6eb828ca92210...
  Status: Downloaded newer image for mysql:latest
  3577ab8e698b32d39259372dc6e189386846b60ff11075c1acd981d101ad8c8d

Ex: Check the container status
$ docker container ls -a

Result:
CONTAINER ID   IMAGE     COMMAND                  CREATED          
3577ab8e698b   mysql     "docker-entrypoint.s…"   18 seconds ago  

STATUS          PORTS                 NAMES
Up 17 seconds   3306/tcp, 33060/tcp   youthful_mahavira

Then our MySQL container is up.
Then we can use 'sh' to run mysql commands and  create a new database.

Ex:
$ docker container exec -it 3577 sh
  # mysql -u root -p
  Enter password: (secret_pwd)

  mysql> CREATE DATABASE demo;
  mysql> SHOW DATABASES;

Result:
 +--------------------+
 | Database           |
  +--------------------+
  | demo               |
  | information_schema |
  | mysql              |
  | performance_schema |
  | sys                |
  +--------------------+
  5 rows in set (0.00 sec)

The data will be kept if we stop and restart this running container because those data are saved in writable container layer.

Ex:
$ docker container stop 3577
  $ docker container start 3577
  $ docker container exec -it 3577 sh
  $ mysql -u root -p
  Enter password: (secret_pwd)

  mysql> SHOW DATABASES;

Result:
+--------------------+
  | Database           |
  +--------------------+
  | demo               |
  | information_schema |
  | mysql              |
  | performance_schema |
  | sys                |
  +--------------------+
  5 rows in set (0.00 sec)

But if this container is removed and got recreated, then all data are lost.

Ex:
$ docker container stop 3577
  $ docker container rm 3577
  $ docker container run -d -e MYSQL_ROOT_PASSWORD=secret_pwd mysql

Result:
317d15e11298d1f1afbf2c3050fc002fdfde559e873c3a5ea48050dc570f2ea1

Ex:
$ docker container exec -it 317d sh
$ mysql -u root -p
  Enter password: (secret_pwd)

  mysql> SHOW DATABASES;

Result:
+--------------------+
| Database           |
  +--------------------+
  | information_schema |
  | mysql              |
  | performance_schema |
  | sys                |
  +--------------------+
  4 rows in set (0.00 sec)


How to manage data in docker?



There are two ways to manage Data in Docker: Volumes and Bind Mounts.






Volumes



Volumes are stored in a part of the host filesystem which is managed by Docker (/var/lib/docker/volumes/ on Linux)
Non-Docker processes should not modify this part of the filesystem.
Volumes are the best way to persist data in Docker.

Let's specific mount when running MySQL container.

Ex:
$ docker container run -d  \
    --mount source=mysql-data,target=/var/lib/mysql \
        -e MYSQL_ROOT_PASSWORD=secret_pwd mysql

Result:
66f2a3d07cc7

We can use the following command to check the volumes.

Ex:
$ docker volume ls

Result:
DRIVER    VOLUME NAME
  local     mysql-data

We even can check it with more details.

Ex:
$ docker volume inspect mysql-data

Result:
[
    {
      "CreatedAt": "2022-07-15T11:18:58-07:00",
      "Driver": "local",
      "Labels": null,
      "Mountpoint": "/var/lib/docker/volumes/mysql-data/_data",
      "Name": "mysql-data",
      "Options": null,
      "Scope": "local"
    }
  ]

Let's create a new database under this MySQL container.

Ex:
$ docker container exec -it 66f2 sh
  # mysql -u root -p
  Enter password: (secret_pwd)

  mysql> CREATE DATABASE demo;
  mysql> SHOW DATABASES;

Result:
+--------------------+
| Database           |
+--------------------+
  | demo               |
  | information_schema |
  | mysql              |
  | performance_schema |
  | sys                |
  +--------------------+
  5 rows in set (0.00 sec)

Then let's stop and remove this container.
And create a new container and mount the data to the same volume 'mysql-data'.

Ex:
$ docker container stop 66f2
$ docker container rm 66f2
  $ docker container run -d  \
      --mount source=mysql-data,target=/var/lib/mysql \
      -e MYSQL_ROOT_PASSWORD=secret_pwd mysql

Result:
3d2360f62b86b3da6cc76360f285dc2745122e8df8d5bf0be457f30e5e459ce4

Then we can find 'demo' database in this new MySQL container.

Ex:
$ docker container exec -it 3d23 sh
$ mysql -u root -p
  Enter password: (secret_pwd)

  mysql> SHOW DATABASES;

Result:
+--------------------+
| Database           |
  +--------------------+
  | demo               |
  | information_schema |
  | mysql              |
  | performance_schema |
  | sys                |
  +--------------------+
  5 rows in set (0.00 sec)


Bind Mounts



Bind mounts may be stored anywhere on the host system.
Non-Docker processes on the Docker host or a Docker container can modify them at any time.

We can use the following example to see how to launch a static page to nginx container and keep changing it in our host filesystem.

Ex:
$ touch index.html
$ vim index.html


<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Bind Mounts Exp</title>
</head>
<body>
<h2>Hello World</h2>
</body>
  </html>


Then we can mount the current folder to nginx container with the following commands.

Ex: $(pwd) means the current folder
$ docker container run -it -d  \
-p 8080:80  \
--name web  \
--mount type=bind,source=$(pwd),target=/usr/share/nginx/html  \
nginx

Result:
2b5cac189c984c07f384020b2ca31f88782d3f0a309573a52ce74d48303dfa8f

Once this nginx container is up, then we can check the browser (localshot:8080), and the page will just simply show 'Hello World'




We can change the index.html directly in our host filesystem, and once we refresh the browser, then we can see the page got changed.

Ex:
$ vim index.html

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Bind Mounts Exp</title>
</head>
<body>
<h2>Hello Docker</h2>
</body>
  </html>





Therefore, using bind mounts, we can make our development experience better.

Posted by at No comments:
Labels: ,
Subscribe to: Posts (Atom)