# Docker Composition Configure environment variables and run Magistrala Docker Composition. > \*Note\*\*: `docker-compose` uses `.env` file to set all environment variables. Ensure that you run the command from the same location as .env file. ## Installation Follow the [official Docker Compose installation guide](https://docs.docker.com/compose/install/) to install Docker Compose. ## Usage Run the following commands from the project root directory. ```bash docker compose -f docker/docker-compose.yaml up ``` To start additional addon services: ```bash docker compose -f docker/addons//docker-compose.yaml up ``` To pull images from a specific release in `ghcr.io/absmach/magistrala`, change `MG_RELEASE_TAG` in `.env` before running these commands. ## Broker Configuration Magistrala supports configurable MQTT broker and Message broker, which also acts as an events store. Magistrala uses two types of brokers: 1. **MQTT_BROKER**: Handles MQTT communication between MQTT adapters and message broker. This can either be `RabbitMQ` or `NATS`. 2. **MESSAGE_BROKER**: Manages message exchange between Magistrala core, optional, and external services. This can either be `NATS` or `RabbitMQ`. This is used to store messages for distributed processing. Events store: This is used by Magistrala services to store events for distributed processing. Magistrala uses a single service to be the message broker and events store. This can either be `NATS` or `RabbitMQ`. Redis can also be used as an events store, but it requires a message broker to be deployed along with it for message exchange. ## Supported Combinations This is the same as MESSAGE_BROKER. This can either be `NATS` or `RabbitMQ` or `Redis`. If Redis is used as an events store, then RabbitMQ or NATS is used as a message broker. The current deployment strategy for Magistrala in `docker/docker-compose.yaml` is to use RabbitMQ as a MQTT_BROKER and NATS as a MESSAGE_BROKER and EVENTS_STORE. Depending on the desired setup, the following broker configurations are valid: - `MQTT_BROKER: RabbitMQ`, `MESSAGE_BROKER: NATS`, `EVENTS_STORE: NATS` - `MQTT_BROKER: RabbitMQ`, `MESSAGE_BROKER: NATS`, `EVENTS_STORE: Redis` - `MQTT_BROKER: RabbitMQ`, `MESSAGE_BROKER: RabbitMQ`, `EVENTS_STORE: RabbitMQ` - `MQTT_BROKER: RabbitMQ`, `MESSAGE_BROKER: RabbitMQ`, `EVENTS_STORE: Redis` - `MQTT_BROKER: NATS`, `MESSAGE_BROKER: RabbitMQ`, `EVENTS_STORE: RabbitMQ` - `MQTT_BROKER: NATS`, `MESSAGE_BROKER: RabbitMQ`, `EVENTS_STORE: Redis` - `MQTT_BROKER: NATS`, `MESSAGE_BROKER: NATS`, `EVENTS_STORE: NATS` - `MQTT_BROKER: NATS`, `MESSAGE_BROKER: NATS`, `EVENTS_STORE: Redis` > For non-default brokers (e.g. RabbitMQ as message broker), adjust the environment variables appropriately and rebuild Docker images. Example: ```bash MG_MESSAGE_BROKER_TYPE=msg_rabbitmq make dockers ``` Then in `.env`: ```text MG_MESSAGE_BROKER_TYPE=msg_rabbitmq MG_MESSAGE_BROKER_URL=${MG_RABBITMQ_URL} ``` For Redis as an events store, you would need to run RabbitMQ or NATS as a message broker. For example, to use Redis as an events store with rabbitmq as a message broker: ```bash MG_ES_TYPE=es_redis MG_MESSAGE_BROKER_TYPE=msg_rabbitmq make dockers ``` ```env MG_MESSAGE_BROKER_TYPE=msg_rabbitmq MG_MESSAGE_BROKER_URL=${MG_RABBITMQ_URL} MG_ES_TYPE=es_redis MG_ES_URL=${MG_REDIS_URL} ``` For MQTT broker other than RabbitMQ, you would need to change the `docker/.env`. For example, to use NATS as a MQTT broker: ```env MG_MQTT_BROKER_TYPE=nats MG_MQTT_BROKER_HEALTH_CHECK=${MG_NATS_HEALTH_CHECK} MG_MQTT_ADAPTER_MQTT_QOS=${MG_NATS_MQTT_QOS} MG_MQTT_ADAPTER_MQTT_TARGET_HOST=${MG_MQTT_BROKER_TYPE} MG_MQTT_ADAPTER_MQTT_TARGET_PORT=1883 MG_MQTT_ADAPTER_MQTT_TARGET_HEALTH_CHECK=${MG_MQTT_BROKER_HEALTH_CHECK} MG_MQTT_ADAPTER_WS_TARGET_HOST=${MG_MQTT_BROKER_TYPE} MG_MQTT_ADAPTER_WS_TARGET_PORT=8080 MG_MQTT_ADAPTER_WS_TARGET_PATH=${MG_NATS_WS_TARGET_PATH} ``` ### RabbitMQ configuration (as MQTT broker or MESSAGE_BROKER) ```yaml services: rabbitmq: image: rabbitmq:3.12.12-management-alpine container_name: magistrala-rabbitmq restart: on-failure environment: RABBITMQ_ERLANG_COOKIE: ${MG_RABBITMQ_COOKIE} RABBITMQ_DEFAULT_USER: ${MG_RABBITMQ_USER} RABBITMQ_DEFAULT_PASS: ${MG_RABBITMQ_PASS} RABBITMQ_DEFAULT_VHOST: ${MG_RABBITMQ_VHOST} ports: - ${MG_RABBITMQ_PORT}:${MG_RABBITMQ_PORT} - ${MG_RABBITMQ_HTTP_PORT}:${MG_RABBITMQ_HTTP_PORT} networks: - magistrala-base-net ``` ### Redis configuration (as events store) ```yaml services: redis: image: redis:7.2.4-alpine container_name: magistrala-es-redis restart: on-failure networks: - magistrala-base-net volumes: - magistrala-broker-volume:/data ``` ## Nginx Configuration Nginx is the entry point for all traffic to Magistrala. By using environment variables file at `docker/.env` you can modify the below given Nginx directive. | Environment Variable | Description | | ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `MG_PUBLIC_HOST` | Public DNS name for the Docker host. This value is used by UI URLs and Let's Encrypt certificate requests. | | `MG_UI_HOST` | Internal Compose hostname for the UI service. Defaults to `ui`. | | `MG_LETSENCRYPT_ENABLED` | Set to `true` to request and use a Let's Encrypt certificate. Set to `false` to comment out the Let's Encrypt cert/key paths and use the fallback Nginx certificate. | | `MG_LETSENCRYPT_EMAIL` | Email address used by Let's Encrypt for expiry and account notifications. Required when running the `letsencrypt` profile. | | `MG_LETSENCRYPT_STAGING` | Set to `true` to request staging certificates while testing. Set to `false` for trusted production certificates. | | `MG_LETSENCRYPT_FORCE_RENEWAL` | Set to `true` for one certbot run when replacing a staging certificate with a production certificate. Set it back to `false` after the production certificate is issued. | | `MG_NGINX_SERVER_NAME` | `MG_NGINX_SERVER_NAME` environmental variable is used to configure nginx directive `server_name`. If environmental variable `MG_NGINX_SERVER_NAME` is empty then default value `localhost` will set to `server_name`. | | `MG_NGINX_SERVER_CERT` | `MG_NGINX_SERVER_CERT` environmental variable is used to configure nginx directive `ssl_certificate`. If environmental variable `MG_NGINX_SERVER_CERT` is empty then by default server certificate in the path `docker/ssl/certs/magistrala-server.crt` will be assigned. | | `MG_NGINX_SERVER_KEY` | `MG_NGINX_SERVER_KEY` environmental variable is used to configure nginx directive `ssl_certificate_key`. If environmental variable `MG_NGINX_SERVER_KEY` is empty then by default server certificate key in the path `docker/ssl/certs/magistrala-server.key` will be assigned. | | `MG_NGINX_SERVER_CLIENT_CA` | `MG_NGINX_SERVER_CLIENT_CA` environmental variable is used to configure nginx directive `ssl_client_certificate`. If environmental variable `MG_NGINX_SERVER_CLIENT_CA` is empty then by default certificate in the path `docker/ssl/certs/ca.crt` will be assigned. | | `MG_NGINX_SERVER_DHPARAM` | `MG_NGINX_SERVER_DHPARAM` environmental variable is used to configure nginx directive `ssl_dhparam`. If environmental variable `MG_NGINX_SERVER_DHPARAM` is empty then by default file in the path `docker/ssl/dhparam.pem` will be assigned. | Adjust these values in `.env` to configure TLS / SSL behavior for your deployment. ### HTTPS UI with Let's Encrypt The Compose stack can request and renew a Let's Encrypt certificate with the optional `letsencrypt` profile. This secures the public Nginx entrypoint and serves the UI through `https://${MG_PUBLIC_HOST}/`. Plain UI requests to `/` are redirected to HTTPS, while API and messaging routes keep their existing protocol behavior. Certbot stores challenge files and issued certificates under ignored local paths in `docker/ssl/`. Prerequisites: - `MG_PUBLIC_HOST` must resolve to the Docker host. - Ports `80` and `443` must be reachable from the public internet. - Set `MG_LETSENCRYPT_EMAIL` before requesting a certificate. For a staging certificate, run one command from the project root: ```bash make run_tls host=example.com email=admin@example.com ``` For a trusted production certificate, set `staging=false`: ```bash make run_tls host=example.com email=admin@example.com staging=false ``` The target updates `docker/.env`, starts the Compose stack with the fallback certificate, runs certbot, switches Nginx to the issued certificate, and recreates Nginx. It also sets `MG_UI_DOCKER_ACCEPT_EULA=yes` for the UI container and configures public UI URLs to `https://${MG_PUBLIC_HOST}`. To configure the same instance without Let's Encrypt, use: ```bash make run_tls host=example.com letsencrypt=false ``` That command updates `docker/.env`, comments out `MG_NGINX_SERVER_CERT` and `MG_NGINX_SERVER_KEY`, stops certbot if it exists, and runs the stack with the fallback Nginx certificate. If you are replacing an existing valid certificate and want certbot to request a new one immediately, pass `force=true`: ```bash make run_tls host=example.com email=admin@example.com staging=false force=true ``` The generated certificate paths in `docker/.env` are: ```env MG_NGINX_SERVER_CERT=./ssl/letsencrypt/live//fullchain.pem MG_NGINX_SERVER_KEY=./ssl/letsencrypt/live//privkey.pem ``` The setup script comments or uncomments those values automatically. Operators should not need to edit them by hand. The certbot service keeps running and checks renewal twice a day. When a certificate is renewed, it sends a `HUP` signal to the Nginx process so new TLS handshakes use the renewed certificate. ## Makefile Integration The included `Makefile` defines build and Docker‑build targets for all Magistrala services. Key points: - `SERVICES`: list of core services (auth, clients, channels, http, coap, mqtt, ws, etc.) - `DOCKERS`, `DOCKERS_DEV`: build targets for production and development Docker images - `make dockers`, `make dockers_dev`: always tag images as `ghcr.io/absmach/magistrala/` - Build arguments embed version, commit hash, and build timestamp into the binary Build all services: ```bash make all # builds all services make dockers # builds all Docker images ``` Start services with Docker compose: ```bash docker compose -f docker/docker-compose.yaml up ``` To clean up: ```bash make cleandocker ``` To run tests(unit tests + API tests) ```bash make test ```