Easy service load balancing with docker-compose
In this blog post I am going to walk through how you can easily setup a load balanced web service using docker-compose. Our web service will have a single end point that returns a message. We will write the web service using Spark to keep things as simple as possible.
We will build the project using Maven
and we will make use of these two maven plugins:
- https://maven.apache.org/plugins/maven-jar-plugin/
- https://maven.apache.org/plugins/maven-shade-plugin/
The full working project is available here:
https://github.com/bltuckerdevblog/load-balancing-with-docker-compose
The first thing we will need to do is create our web service. Spark makes that easy:
package com.abnormallydriven.dockercompose;
import static spark.Spark.*;
public class Application {
public static void main(String[] args) {
get("/hello", (request, response) -> "Hello from Spark Java");
}
}
That's it! We can now build that jar with maven and run it using java -jar.
We can then go to localhost:4567/hello and see the response.
The next step on the road to a load balanced version of the this web service using docker-compose is to get the service into a docker container.
We can do that by creating a Dockerfile like this:
FROM openjdk:8
ADD hello-service.jar .
EXPOSE 4567
ENTRYPOINT java -jar hello-service.jar
Once we've done that you can deploy and run the service using docker run.
Once we have containerized our web service we need to get our load balancer ready to go. We will be using nginx to do the load balancing for our service instances. We will create a Dockerfile again to get our load balancer up and running.
FROM nginx
COPY nginx.conf /etc/nginx
EXPOSE 80 443
CMD ["nginx", "-g", "daemon off;"]
The nginx.conf file which is copied into our container when it's built looks like this
events {}
http {
upstream hello-service {
server hello-service-1:4567;
server hello-service-2:4567;
}
server {
listen 80;
location / {
proxy_pass http://hello-service/;
}
location /stats {
stub_status on;
allow all;
}
}
}
The nginx.conf file will tell our load balancer to proxy requests through to our back end service instances as well as to expose a statistics end point that we can inspect later. Now that we have our Dockerfiles setup we can get to the real fun with docker-compose.
The first thing we will need to do is create our docker-compose.yml file which will instruct docker-compose on how to bring our service to life.
The first line of our our yaml file will be the version declaration. We are using version 2.1 for our docker-compose file, the latest version at the time of this writing is 3.0 but it requires docker engine 1.13.
Once we have let docker-compose know what version our file will be in, we can get around to declaring our services. We do this with the services keyword in our docker-compose file. A basic example of a service declaration in a docker-compose file might look like this:
services:
hello-service-1:
build:
service/
The path to your Dockerfile is relative to the location of the docker-compose file.
You can see in our simple declaration that path appears under the build keyword in the docker-compose file.
We can set environment variables within our containers by passing them in via the docker-compose file. We will use this feature to prove that when we run our services with docker-compose that we actually have two load balanced containers running. Let's expand our service's declaration by adding an environment variable.
hello-service-1:
environment:
- service_name="Service Instance A"
build:
service/
Now our web service code will be able to access an environment variable in its container named "service_name". We can use the value of that environment variable to write our response. When we hit our service's load balanced http address we should see the response text alternate based on which instance replied to our request.
The last thing we will need to provide in our web service's docker-compose configuration is a health check. This is an important because we are using nginx as our load balancer and it will fail if the service's defined in its upstream are not available when it launches. We will use a health check along with the depends_on keyword in our load balancer's configuration to make sure that docker-compose does not attempt to start the load balancer until our services are up and running.
hello-service-1:
environment:
- service_name="Service Instance A"
build:
service/
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:4567/hello"]
interval: 10s
timeout: 10s
retries: 3
Our healthcheck declaration says to run the curl command against our hello endpoint. The health check will first be run after the interval time has passed after the container has started, and then it will be called repeatedly every interval seconds after that. A service is considered healthy after it successfully passes a health check, it will be marked as unhealthy after it fails the health check the retries number of times.
That is it for our first service's docker-compose configuration. Our second service will look exactly the same except we will be given a different environment variable value. Here's what our second service's configuration will look like:
hello-service-2:
environment:
- service_name="Service Instance B"
build:
service/
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:4567/hello"]
interval: 10s
timeout: 10s
retries: 3
Now lets declare our load balancer
hello-service-load-balancer:
build:
loadbalancer/
ports:
- "127.0.0.1:80:80"
depends_on:
hello-service-1:
condition: service_healthy
hello-service-2:
condition: service_healthy
We specify a Docker file via build just as we did for each of our web services, but we make use of the ports keyword now to declare that this container should expose port 80 to port 80 on our host machine. That way when we go to localhost in our browser we will be actually hitting this container.
We also make use of depends_on to tell docker-compose that this container depends on the other two services in our docker-compose file, the condition keyword specifies that docker-compose should wait until the services report as healthy before attempting to launch our load balancer.
If we leave the condition keyword off then docker will simply wait for our web service's containers to start before launching our load balancer. If we did that though and our web services had any kind of start up time before they could begin responding to requests then they might not respond to nginx as it starts up and then our nginx container would fail and we would not have a running load balancer.
Our entire docker-compose file should now look like this:
version: '2.1'
services:
hello-service-1:
environment:
- service_name="Service Instance A"
build:
service/
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:4567/hello"]
interval: 10s
timeout: 10s
retries: 3
hello-service-2:
environment:
- service_name="Service Instance B"
build:
service/
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:4567/hello"]
interval: 10s
timeout: 10s
retries: 3
hello-service-load-balancer:
build:
loadbalancer/
ports:
- "127.0.0.1:80:80"
depends_on:
hello-service-1:
condition: service_healthy
hello-service-2:
condition: service_healthy
We can bring our load balanced service to life with
docker-compose up -d
Once the service is up and running you should be able to navigate to localhost/hello and see either
Hello from "Service Instance B". or
Hello from "Service Instance B".
Each time you hit the address you should see the response alternate between services since by default the load balancer will perform round robin request routing.
When we are done we can bring our services back down cleanly using
docker-compose down
That's it! I would strongly recommend that you have a look at the docker-compose reference documentation to learn more!