opensource/supermq
1
0
Fork 0
mirror of https://github.com/absmach/supermq.git synced 2026-06-23 06:30:22 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

NOISSUE - Remove Vault scripts and compose (#246)

Browse Source 
* remove vault from MG

Signed-off-by: Arvindh <arvindh91@gmail.com>

* rename env in readme

Signed-off-by: Arvindh <arvindh91@gmail.com>

---------

Signed-off-by: Arvindh <arvindh91@gmail.com>
This commit is contained in:
Arvindh
2025-07-08 17:48:32 +05:30
committed by GitHub
parent 5568589182
commit 148e2fbb7f
14 changed files with 5 additions and 1027 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker/README.md
+5 -5
View File
@@ -123,12 +123,12 @@ services:
Nginx is the entry point for all traffic to Magistrala.
By using environment variables file at `docker/.env` you can modify the below given Nginx directive.
`MG_NGINX_SERVER_NAME` environmental variable is used to configure nginx directive `server_name`. If environmental variable `MG_NGINX_SERVER_NAME` is empty then default value `localhost` will set to `server_name`.
`SMQ_NGINX_SERVER_NAME` environmental variable is used to configure nginx directive `server_name`. If environmental variable `SMQ_NGINX_SERVER_NAME` is empty then default value `localhost` will set to `server_name`.
`MG_NGINX_SERVER_CERT` environmental variable is used to configure nginx directive `ssl_certificate`. If environmental variable `MG_NGINX_SERVER_CERT` is empty then by default server certificate in the path `docker/ssl/certs/magistrala-server.crt` will be assigned.
`SMQ_NGINX_SERVER_CERT` environmental variable is used to configure nginx directive `ssl_certificate`. If environmental variable `SMQ_NGINX_SERVER_CERT` is empty then by default server certificate in the path `docker/ssl/certs/magistrala-server.crt` will be assigned.
`MG_NGINX_SERVER_KEY` environmental variable is used to configure nginx directive `ssl_certificate_key`. If environmental variable `MG_NGINX_SERVER_KEY` is empty then by default server certificate key in the path `docker/ssl/certs/magistrala-server.key` will be assigned.
`SMQ_NGINX_SERVER_KEY` environmental variable is used to configure nginx directive `ssl_certificate_key`. If environmental variable `SMQ_NGINX_SERVER_KEY` is empty then by default server certificate key in the path `docker/ssl/certs/magistrala-server.key` will be assigned.
`MG_NGINX_SERVER_CLIENT_CA` environmental variable is used to configure nginx directive `ssl_client_certificate`. If environmental variable `MG_NGINX_SERVER_CLIENT_CA` is empty then by default certificate in the path `docker/ssl/certs/ca.crt` will be assigned.
`SMQ_NGINX_SERVER_CLIENT_CA` environmental variable is used to configure nginx directive `ssl_client_certificate`. If environmental variable `SMQ_NGINX_SERVER_CLIENT_CA` is empty then by default certificate in the path `docker/ssl/certs/ca.crt` will be assigned.
`MG_NGINX_SERVER_DHPARAM` environmental variable is used to configure nginx directive `ssl_dhparam`. If environmental variable `MG_NGINX_SERVER_DHPARAM` is empty then by default file in the path `docker/ssl/dhparam.pem` will be assigned.
`SMQ_NGINX_SERVER_DHPARAM` environmental variable is used to configure nginx directive `ssl_dhparam`. If environmental variable `SMQ_NGINX_SERVER_DHPARAM` is empty then by default file in the path `docker/ssl/dhparam.pem` will be assigned.
docker/addons/vault/README.md
-290
View File
@@ -1,290 +0,0 @@
# Vault
This is Vault service deployment to be used with Magistrala.
When the Vault service is started, some initialization steps need to be done to set things up.
## Configuration
| Variable | Description | Default |
| :-------------------------------------- | ----------------------------------------------------------------------------- | ------------------------------------- |
| MG_VAULT_ADDR | Vault Address | http://vault:8200 |
| MG_VAULT_UNSEAL_KEY_1 | Vault unseal key | "" |
| MG_VAULT_UNSEAL_KEY_2 | Vault unseal key | "" |
| MG_VAULT_UNSEAL_KEY_3 | Vault unseal key | "" |
| MG_VAULT_TOKEN | Vault cli access token | "" |
| MG_VAULT_PKI_PATH | Vault secrets engine path for Root CA | pki |
| MG_VAULT_PKI_ROLE_NAME | Vault Root CA role name to issue intermediate CA | magistrala_int_ca |
| MG_VAULT_PKI_FILE_NAME | Root CA Certificates name used by`vault_set_pki.sh` | mg_root |
| MG_VAULT_PKI_CA_CN | Common name used for Root CA creation by`vault_set_pki.sh` | Magistrala Root Certificate Authority |
| MG_VAULT_PKI_CA_OU | Organization unit used for Root CA creation by`vault_set_pki.sh` | Magistrala |
| MG_VAULT_PKI_CA_O | Organization used for Root CA creation by`vault_set_pki.sh` | Magistrala |
| MG_VAULT_PKI_CA_C | Country used for Root CA creation by`vault_set_pki.sh` | FRANCE |
| MG_VAULT_PKI_CA_L | Location used for Root CA creation by`vault_set_pki.sh` | PARIS |
| MG_VAULT_PKI_CA_ST | State or Provisions used for Root CA creation by`vault_set_pki.sh` | PARIS |
| MG_VAULT_PKI_CA_ADDR | Address used for Root CA creation by`vault_set_pki.sh` | 5 Av. Anatole |
| MG_VAULT_PKI_CA_PO | Postal code used for Root CA creation by`vault_set_pki.sh` | 75007 |
| MG_VAULT_PKI_CLUSTER_PATH | Vault Root CA Cluster Path | http://localhost |
| MG_VAULT_PKI_CLUSTER_AIA_PATH | Vault Root CA Cluster AIA Path | http://localhost |
| MG_VAULT_PKI_INT_PATH | Vault secrets engine path for Intermediate CA | pki_int |
| MG_VAULT_PKI_INT_SERVER_CERTS_ROLE_NAME | Vault Intermediate CA role name to issue server certificate | magistrala_server_certs |
| MG_VAULT_PKI_INT_THINGS_CERTS_ROLE_NAME | Vault Intermediate CA role name to issue Things certificates | magistrala_things_certs |
| MG_VAULT_PKI_INT_FILE_NAME | Intermediate CA Certificates name used by`vault_set_pki.sh` | mg_root |
| MG_VAULT_PKI_INT_CA_CN | Common name used for Intermediate CA creation by`vault_set_pki.sh` | Magistrala Root Certificate Authority |
| MG_VAULT_PKI_INT_CA_OU | Organization unit used for Root CA creation by`vault_set_pki.sh` | Magistrala |
| MG_VAULT_PKI_INT_CA_O | Organization used for Intermediate CA creation by`vault_set_pki.sh` | Magistrala |
| MG_VAULT_PKI_INT_CA_C | Country used for Intermediate CA creation by`vault_set_pki.sh` | FRANCE |
| MG_VAULT_PKI_INT_CA_L | Location used for Intermediate CA creation by`vault_set_pki.sh` | PARIS |
| MG_VAULT_PKI_INT_CA_ST | State or Provisions used for Intermediate CA creation by`vault_set_pki.sh` | PARIS |
| MG_VAULT_PKI_INT_CA_ADDR | Address used for Intermediate CA creation by`vault_set_pki.sh` | 5 Av. Anatole |
| MG_VAULT_PKI_INT_CA_PO | Postal code used for Intermediate CA creation by`vault_set_pki.sh` | 75007 |
| MG_VAULT_PKI_INT_CLUSTER_PATH | Vault Intermediate CA Cluster Path | http://localhost |
| MG_VAULT_PKI_INT_CLUSTER_AIA_PATH | Vault Intermediate CA Cluster AIA Path | http://localhost |
| MG_VAULT_THINGS_CERTS_ISSUER_ROLEID | Vault Intermediate CA Things Certificate issuer AppRole authentication RoleID | magistrala |
| MG_VAULT_THINGS_CERTS_ISSUER_SECRET | Vault Intermediate CA Things Certificate issuer AppRole authentication Secret | magistrala |
## Setup
The following scripts are provided, which work on the running Vault service from within the `docker/addons/vault/scripts` directory.
### 1. `vault_init.sh`
Calls `vault operator init` to perform the initial vault initialization and generates a `docker/addons/vault/scripts/data/secrets` file which contains the Vault unseal keys and root tokens.
### 2. `vault_copy_env.sh`
After the initial setup, the Vault-related environment variables (`MG_VAULT_TOKEN`, `MG_VAULT_UNSEAL_KEY_1`, `MG_VAULT_UNSEAL_KEY_2`, `MG_VAULT_UNSEAL_KEY_3`) need to be updated in the `.env` file.
The `vault_copy_env.sh` script automatically retrieves these values from the `docker/addons/vault/scripts/data/secrets` file and updates the corresponding environment variables in your `.env` file.
Example:
```sh
Vault environment variables have been successfully set in ~/magistrala/docker/.env
```
### 3. `vault_unseal.sh`
This can be run after the initialization to unseal Vault, which is necessary for it to be used to store and/or get secrets.
This can be used if you don't want to restart the service.
The unseal environment variables need to be set in `.env` for the script to work (`MG_VAULT_TOKEN`,`MG_VAULT_UNSEAL_KEY_1`, `MG_VAULT_UNSEAL_KEY_2`, `MG_VAULT_UNSEAL_KEY_3`).
This script should not be necessary to run after the initial setup, since the Vault service unseals itself when starting the container.
Example output:
```bash
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed true
Total Shares 5
Threshold 3
Unseal Progress 1/3
Unseal Nonce 4c248cc8-e9f5-055e-319b-00ee06f998a0
Version 1.15.4
Build Date 2023-12-04T17:45:28Z
Storage Type file
HA Enabled false
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed true
Total Shares 5
Threshold 3
Unseal Progress 2/3
Unseal Nonce 4c248cc8-e9f5-055e-319b-00ee06f998a0
Version 1.15.4
Build Date 2023-12-04T17:45:28Z
Storage Type file
HA Enabled false
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Unseal Progress 3/3
Unseal Nonce 4c248cc8-e9f5-055e-319b-00ee06f998a0
Version 1.15.4
Build Date 2023-12-04T17:45:28Z
Storage Type file
HA Enabled false
```
### 4. vault_set_pki.sh
The `vault_set_pki.sh` script is responsible for generating the root certificate, intermediate certificate, and HTTPS server certificate. All generated certificates, keys, and CSR files are stored in the `docker/addons/vault/scripts/data` directory.
The script pulls necessary parameters for certificate generation from environment variables, which are, by default, loaded from `docker/.env`.
- Environment variables prefixed with `MG_VAULT_PKI` in the `docker/.env` file are used for generating the root CA.
- Environment variables prefixed with `MG_VAULT_PKI_INT` are used for generating the intermediate CA.
To skip generating the server certificate and key, you can pass the `--skip-server-cert` option to the script:
```sh
./vault_set_pki.sh --skip-server-cert
```
#### Troubleshooting:
If you encounter the following error:
```sh
jq command could not be found, please install it and try again.
```
Install `jq` using:
```sh
sudo apt-get update && sudo apt-get install -y jq
```
After installing `jq`, rerun the script.
### 5. `vault_create_approle.sh`
This script enables AppRole authorization in Vault. The certs service uses these AppRole credentials to issue and revoke certificates from the Vault intermediate CA.
Example output:
```sh
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
Key Value
--- -----
token <token_value>
token_accessor i6YVeKh4wQ4e0Aj0ONiyGw1Z
token_duration ∞
token_renewable false
token_policies ["root"]
identity_policies []
policies ["root"]
Creating new policy for AppRole
Successfully copied 2.56kB to magistrala-vault:/vault/magistrala_things_certs_issue.hcl
Success! Uploaded policy: magistrala_things_certs_issue
Enabling AppRole
Success! Enabled approle auth method at: approle/
Deleting old AppRole
Success! Data deleted (if it existed) at: auth/approle/role/magistrala_things_certs_issuer
Creating new AppRole
Success! Data written to: auth/approle/role/magistrala_things_certs_issuer
Writing custom role ID
Key Value
--- -----
role_id f23942b3-62b9-7456-784f-220ca3f703b9
Success! Data written to: auth/approle/role/magistrala_things_certs_issuer/role-id
Writing custom secret
Key Value
--- -----
secret_id 61d5a30f-634c-6027-f5b6-4934e6fc49b2
secret_id_accessor 1d744f6e-e0c2-5431-a87a-2b23fde584a7
secret_id_num_uses 0
secret_id_ttl 0s
Testing custom role ID and secret by logging in
Key Value
--- -----
token <token_value>
token_accessor 9cuwS4mrLHKhJQMv0pl9Bbg9
token_duration 1h
token_renewable true
token_policies ["default" "magistrala_things_certs_issue"]
identity_policies []
policies ["default" "magistrala_things_certs_issue"]
token_meta_role_name magistrala_things_certs_issuer
```
By default, the `vault_create_approle.sh` script tries to enable the AppRole authentication method. Certs service uses the approle credentials to issue and revoke things certificate from vault intermediate CA. If AppRole is already enabled, you can skip this step by passing the `--skip-enable-approle` argument:
```sh
./vault_create_approle.sh --skip-enable-approle
```
### 6. `vault_copy_certs.sh`
This script copies the required certificates and keys from `docker/addons/vault/scripts/data` to the `docker/ssl/certs` folder.
Example output:
```bash
Copying certificate files
'data/localhost.crt' -> '~/Documents/magistrala/docker/ssl/certs/magistrala-server.crt'
'data/localhost.key' -> '~/Documents/magistrala/docker/ssl/certs/magistrala-server.key'
'data/mg_int.key' -> '~/Documents/magistrala/docker/ssl/certs/ca.key'
'data/mg_int_bundle.crt' -> '~/Documents/magistrala/docker/ssl/certs/ca.crt'
```
## Custom `.env` Path Support
Vault scripts support specifying a custom `.env` file path using the `--env-file` argument. If this argument is not provided, the scripts will use the default `.env` file located at `docker/.env`.
To use a different `.env` file, include the `--env-file` argument followed by the path to your `.env` file when running the Vault scripts. Below are examples of how to execute each script with a custom `.env` file path:
```bash
./vault_init.sh --env-file /custom/path/.env
./vault_copy_env.sh --env-file /custom/path/.env
./vault_unseal.sh --env-file /custom/path/.env
./vault_set_pki.sh --env-file /custom/path/.env
./vault_create_approle.sh --env-file /custom/path/.env
./vault_copy_certs.sh --env-file /custom/path/.env
```
## Hashicorp Cloud Platform (HCP) Vault
To have the same PKI setup can done in Hashicorp Cloud Platform (HCP) Vault follow the below steps:
Requirement: [VAULT CLI](https://developer.hashicorp.com/vault/tutorials/getting-started/getting-started-install)
- Replace the environmental variable `MG_VAULT_ADDR` in `docker/.env` with HCP Vault address.
- Replace the environmental variable `MG_VAULT_TOKEN` in `docker/.env` with HCP Vault Admin token.
- Run script `vault_set_pki.sh` and `vault_create_approle.sh`.
- Optional step, run script `vault_copy_certs.sh` to copy certificates to magistrala default path.
## Vault CLI
It can also be useful to run the Vault CLI for inspection and administration work.
```bash
Usage: vault <command> [args]
Common commands:
read Read data and retrieves secrets
write Write data, configuration, and secrets
delete Delete secrets and configuration
list List data or secrets
login Authenticate locally
agent Start a Vault agent
server Start a Vault server
status Print seal and HA status
unwrap Unwrap a wrapped secret
Other commands:
audit Interact with audit devices
auth Interact with auth methods
debug Runs the debug command
kv Interact with Vault's Key-Value storage
lease Interact with leases
monitor Stream log messages from a Vault server
namespace Interact with namespaces
operator Perform operator-specific tasks
path-help Retrieve API help for paths
plugin Interact with Vault plugins and catalog
policy Interact with policies
print Prints runtime configurations
secrets Interact with secrets engines
ssh Initiate an SSH session
token Interact with tokens
```
If the Vault is setup through `docker/addons/vault`, then Vault CLI can be run directly using the Vault image in Docker: `docker run -it magistrala/vault:latest vault`
## Vault Web UI
If the Vault is setup through `docker/addons/vault`, Then Vault Web UI is accessible by default on `http://localhost:8200/ui`.
docker/addons/vault/config.hcl
-10
View File
@@ -1,10 +0,0 @@
storage "file" {
path = "/vault/file"
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = 1
}
ui = true
docker/addons/vault/docker-compose.yaml
-39
View File
@@ -1,39 +0,0 @@
# Copyright (c) Abstract Machines
# SPDX-License-Identifier: Apache-2.0
# This docker-compose file contains optional Vault service for Magistrala platform.
# Since this is optional, this file is dependent of docker-compose file
# from <project_root>/docker. In order to run these services, execute command:
# docker compose -f docker/docker-compose.yaml -f docker/addons/vault/docker-compose.yaml up
# from project root. Vault default port (8200) is exposed, so you can use Vault CLI tool for
# vault inspection and administration, as well as access the UI.
networks:
magistrala-base-net:
volumes:
magistrala-vault-volume:
services:
vault:
image: hashicorp/vault:1.15.4
container_name: magistrala-vault
ports:
- ${MG_VAULT_PORT}:8200
networks:
- magistrala-base-net
volumes:
- magistrala-vault-volume:/vault/file
- magistrala-vault-volume:/vault/logs
- ./config.hcl:/vault/config/config.hcl
- ./entrypoint.sh:/entrypoint.sh
environment:
VAULT_ADDR: http://127.0.0.1:${MG_VAULT_PORT}
MG_VAULT_PORT: ${MG_VAULT_PORT}
MG_VAULT_UNSEAL_KEY_1: ${MG_VAULT_UNSEAL_KEY_1}
MG_VAULT_UNSEAL_KEY_2: ${MG_VAULT_UNSEAL_KEY_2}
MG_VAULT_UNSEAL_KEY_3: ${MG_VAULT_UNSEAL_KEY_3}
entrypoint: /bin/sh
command: /entrypoint.sh
cap_add:
- IPC_LOCK
docker/addons/vault/entrypoint.sh
-25
View File
@@ -1,25 +0,0 @@
#!/usr/bin/dumb-init /bin/sh
# Copyright (c) Abstract Machines
# SPDX-License-Identifier: Apache-2.0
VAULT_CONFIG_DIR=/vault/config
docker-entrypoint.sh server &
VAULT_PID=$!
sleep 2
echo $MG_VAULT_UNSEAL_KEY_1
echo $MG_VAULT_UNSEAL_KEY_2
echo $MG_VAULT_UNSEAL_KEY_3
if [[ ! -z "${MG_VAULT_UNSEAL_KEY_1}" ]] &&
[[ ! -z "${MG_VAULT_UNSEAL_KEY_2}" ]] &&
[[ ! -z "${MG_VAULT_UNSEAL_KEY_3}" ]]; then
echo "Unsealing Vault"
vault operator unseal ${MG_VAULT_UNSEAL_KEY_1}
vault operator unseal ${MG_VAULT_UNSEAL_KEY_2}
vault operator unseal ${MG_VAULT_UNSEAL_KEY_3}
fi
wait $VAULT_PID
docker/addons/vault/scripts/.gitignore
-5
View File
@@ -1,5 +0,0 @@
# Copyright (c) Abstract Machines
# SPDX-License-Identifier: Apache-2.0
data
magistrala_things_certs_issue.hcl
docker/addons/vault/scripts/magistrala_things_certs_issue.template.hcl
-32
View File
@@ -1,32 +0,0 @@
# Allow issue certificate with role with default issuer from Intermediate PKI
path "${MG_VAULT_PKI_INT_PATH}/issue/${MG_VAULT_PKI_INT_THINGS_CERTS_ROLE_NAME}" {
capabilities = ["create", "update"]
}
## Revole certificate from Intermediate PKI
path "${MG_VAULT_PKI_INT_PATH}/revoke" {
capabilities = ["create", "update"]
}
## List Revoked Certificates from Intermediate PKI
path "${MG_VAULT_PKI_INT_PATH}/certs/revoked" {
capabilities = ["list"]
}
## List Certificates from Intermediate PKI
path "${MG_VAULT_PKI_INT_PATH}/certs" {
capabilities = ["list"]
}
## Read Certificate from Intermediate PKI
path "${MG_VAULT_PKI_INT_PATH}/cert/+" {
capabilities = ["read"]
}
path "${MG_VAULT_PKI_INT_PATH}/cert/+/raw" {
capabilities = ["read"]
}
path "${MG_VAULT_PKI_INT_PATH}/cert/+/raw/pem" {
capabilities = ["read"]
}
docker/addons/vault/scripts/vault_cmd.sh
-24
View File
@@ -1,24 +0,0 @@
#!/usr/bin/bash
# Copyright (c) Abstract Machines
# SPDX-License-Identifier: Apache-2.0
vault() {
if is_container_running "magistrala-vault"; then
docker exec -it magistrala-vault vault "$@"
else
if which vault &> /dev/null; then
$(which vault) "$@"
else
echo "magistrala-vault container or vault command not found. Please refer to the documentation: https://github.com/absmach/magistrala/blob/main/docker/addons/vault/README.md"
fi
fi
}
is_container_running() {
local container_name="$1"
if [ "$(docker inspect --format '{{.State.Running}}' "$container_name" 2>/dev/null)" = "true" ]; then
return 0
else
return 1
fi
}
docker/addons/vault/scripts/vault_copy_certs.sh
-86
View File
@@ -1,86 +0,0 @@
#!/usr/bin/bash
# Copyright (c) Abstract Machines
# SPDX-License-Identifier: Apache-2.0
set -euo pipefail
scriptdir="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"
# default env file path
env_file="docker/.env"
# default certs copy path
certs_copy_path="docker/ssl/certs/"
while [[ "$#" -gt 0 ]]; do
case $1 in
--env-file)
if [[ -z "${2:-}" ]]; then
echo "Error: --env-file requires a non-empty option argument."
exit 1
fi
env_file="$2"
if [[ ! -f "$env_file" ]]; then
echo "Error: .env file not found at $env_file"
exit 1
fi
shift
;;
--certs-copy-path)
if [[ -z "${2:-}" ]]; then
echo "Error: --certs-copy-path requires a non-empty option argument."
exit 1
fi
certs_copy_path="$2"
shift
;;
*)
echo "Error: Unknown parameter passed: $1"
exit 1
;;
esac
shift
done
readDotEnv() {
set -o allexport
source "$env_file"
set +o allexport
}
readDotEnv
server_name="localhost"
# Check if MG_NGINX_SERVER_NAME is set or not empty
if [ -n "${MG_NGINX_SERVER_NAME:-}" ]; then
server_name="$MG_NGINX_SERVER_NAME"
fi
echo "Copying certificate files to ${certs_copy_path}"
if [ -e "$scriptdir/data/${server_name}.crt" ]; then
cp -v "$scriptdir/data/${server_name}.crt" "${certs_copy_path}magistrala-server.crt"
else
echo "${server_name}.crt file not available"
fi
if [ -e "$scriptdir/data/${server_name}.key" ]; then
cp -v "$scriptdir/data/${server_name}.key" "${certs_copy_path}magistrala-server.key"
else
echo "${server_name}.key file not available"
fi
if [ -e "$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.key" ]; then
cp -v "$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.key" "${certs_copy_path}ca.key"
else
echo "$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.key file not available"
fi
if [ -e "$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}_bundle.crt" ]; then
cp -v "$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}_bundle.crt" "${certs_copy_path}ca.crt"
else
echo "$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}_bundle.crt file not available"
fi
exit 0
docker/addons/vault/scripts/vault_copy_env.sh
-46
View File
@@ -1,46 +0,0 @@
#!/usr/bin/bash
# Copyright (c) Abstract Machines
# SPDX-License-Identifier: Apache-2.0
set -euo pipefail
scriptdir="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"
# default env file path
env_file="docker/.env"
while [[ "$#" -gt 0 ]]; do
case $1 in
--env-file)
if [[ -z "${2:-}" ]]; then
echo "Error: --env-file requires a non-empty option argument."
exit 1
fi
env_file="$2"
if [[ ! -f "$env_file" ]]; then
echo "Error: .env file not found at $env_file"
exit 1
fi
shift
;;
*)
echo "Unknown parameter passed: $1"
exit 1
;;
esac
shift
done
write_env() {
if [ -e "$scriptdir/data/secrets" ]; then
sed -i "s,MG_VAULT_UNSEAL_KEY_1=.*,MG_VAULT_UNSEAL_KEY_1=$(awk -F ": " '$1 == "Unseal Key 1" {print $2}' $scriptdir/data/secrets)," "$env_file"
sed -i "s,MG_VAULT_UNSEAL_KEY_2=.*,MG_VAULT_UNSEAL_KEY_2=$(awk -F ": " '$1 == "Unseal Key 2" {print $2}' $scriptdir/data/secrets)," "$env_file"
sed -i "s,MG_VAULT_UNSEAL_KEY_3=.*,MG_VAULT_UNSEAL_KEY_3=$(awk -F ": " '$1 == "Unseal Key 3" {print $2}' $scriptdir/data/secrets)," "$env_file"
sed -i "s,MG_VAULT_TOKEN=.*,MG_VAULT_TOKEN=$(awk -F ": " '$1 == "Initial Root Token" {print $2}' $scriptdir/data/secrets)," "$env_file"
echo "Vault environment variables are set successfully in $env_file"
else
echo "Error: Source file '$scriptdir/data/secrets' not found."
fi
}
write_env
docker/addons/vault/scripts/vault_create_approle.sh
-122
View File
@@ -1,122 +0,0 @@
#!/usr/bin/bash
# Copyright (c) Abstract Machines
# SPDX-License-Identifier: Apache-2.0
set -euo pipefail
scriptdir="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"
# default env file path
env_file="docker/.env"
SKIP_ENABLE_APP_ROLE=""
while [[ "$#" -gt 0 ]]; do
case $1 in
--env-file)
if [[ -z "${2:-}" ]]; then
echo "Error: --env-file requires a non-empty option argument."
exit 1
fi
env_file="$2"
if [[ ! -f "$env_file" ]]; then
echo "Error: .env file not found at $env_file"
exit 1
fi
shift
;;
--skip-enable-approle)
SKIP_ENABLE_APP_ROLE="true"
;;
*)
echo "Unknown parameter passed: $1"
exit 1
;;
esac
shift
done
readDotEnv() {
set -o allexport
source "$env_file"
set +o allexport
}
source "$scriptdir/vault_cmd.sh"
vaultCreatePolicyFile() {
envsubst '
${MG_VAULT_PKI_INT_PATH}
${MG_VAULT_PKI_INT_THINGS_CERTS_ROLE_NAME}
' < "$scriptdir/magistrala_things_certs_issue.template.hcl" > "$scriptdir/magistrala_things_certs_issue.hcl"
}
vaultCreatePolicy() {
echo "Creating new policy for AppRole"
if is_container_running "magistrala-vault"; then
docker cp "$scriptdir/magistrala_things_certs_issue.hcl" magistrala-vault:/vault/magistrala_things_certs_issue.hcl
vault policy write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} magistrala_things_certs_issue /vault/magistrala_things_certs_issue.hcl
else
vault policy write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} magistrala_things_certs_issue "$scriptdir/magistrala_things_certs_issue.hcl"
fi
}
vaultEnableAppRole() {
if [[ "$SKIP_ENABLE_APP_ROLE" == "true" ]]; then
echo "Skipping Enable AppRole"
else
echo "Enabling AppRole"
vault auth enable -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} approle
fi
}
vaultDeleteRole() {
echo "Deleting old AppRole"
vault delete -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} auth/approle/role/magistrala_things_certs_issuer
}
vaultCreateRole() {
echo "Creating new AppRole"
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} auth/approle/role/magistrala_things_certs_issuer \
token_policies=magistrala_things_certs_issue secret_id_num_uses=0 \
secret_id_ttl=0 token_ttl=1h token_max_ttl=3h token_num_uses=0
}
vaultWriteCustomRoleID() {
echo "Writing custom role id"
vault read -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} auth/approle/role/magistrala_things_certs_issuer/role-id
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} auth/approle/role/magistrala_things_certs_issuer/role-id role_id=${MG_VAULT_THINGS_CERTS_ISSUER_ROLEID}
}
vaultWriteCustomSecret() {
echo "Writing custom secret"
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} -f auth/approle/role/magistrala_things_certs_issuer/secret-id
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} auth/approle/role/magistrala_things_certs_issuer/custom-secret-id secret_id=${MG_VAULT_THINGS_CERTS_ISSUER_SECRET} num_uses=0 ttl=0
}
vaultTestRoleLogin() {
echo "Testing custom roleid secret by logging in"
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} auth/approle/login \
role_id=${MG_VAULT_THINGS_CERTS_ISSUER_ROLEID} \
secret_id=${MG_VAULT_THINGS_CERTS_ISSUER_SECRET}
}
if ! command -v jq &> /dev/null; then
echo "jq command could not be found, please install it and try again."
exit 1
fi
readDotEnv
vault login -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_TOKEN}
vaultCreatePolicyFile
vaultCreatePolicy
vaultEnableAppRole
vaultDeleteRole
vaultCreateRole
vaultWriteCustomRoleID
vaultWriteCustomSecret
vaultTestRoleLogin
exit 0
docker/addons/vault/scripts/vault_init.sh
-46
View File
@@ -1,46 +0,0 @@
#!/usr/bin/bash
# Copyright (c) Abstract Machines
# SPDX-License-Identifier: Apache-2.0
set -euo pipefail
scriptdir="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"
# default env file path
env_file="docker/.env"
while [[ "$#" -gt 0 ]]; do
case $1 in
--env-file)
if [[ -z "${2:-}" ]]; then
echo "Error: --env-file requires a non-empty option argument."
exit 1
fi
env_file="$2"
if [[ ! -f "$env_file" ]]; then
echo "Error: .env file not found at $env_file"
exit 1
fi
shift
;;
*)
echo "Unknown parameter passed: $1"
exit 1
;;
esac
shift
done
readDotEnv() {
set -o allexport
source "$env_file"
set +o allexport
}
source "$scriptdir/vault_cmd.sh"
readDotEnv
mkdir -p "$scriptdir/data"
vault operator init -address="$MG_VAULT_ADDR" 2>&1 | tee >(sed -r 's/\x1b\[[0-9;]*m//g' > "$scriptdir/data/secrets")
docker/addons/vault/scripts/vault_set_pki.sh
-251
View File
@@ -1,251 +0,0 @@
#!/usr/bin/bash
# Copyright (c) Abstract Machines
# SPDX-License-Identifier: Apache-2.0
set -euo pipefail
scriptdir="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"
# edfault env file path
env_file="docker/.env"
SKIP_SERVER_CERT=""
while [[ "$#" -gt 0 ]]; do
case $1 in
--env-file)
if [[ -z "${2:-}" ]]; then
echo "Error: --env-file requires a non-empty option argument."
exit 1
fi
env_file="$2"
if [[ ! -f "$env_file" ]]; then
echo "Error: .env file not found at $env_file"
exit 1
fi
shift
;;
--skip-server-cert)
SKIP_SERVER_CERT="--skip-server-cert"
;;
*)
echo "Unknown parameter passed: $1"
exit 1
;;
esac
shift
done
readDotEnv() {
set -o allexport
source "$env_file"
set +o allexport
}
server_name="localhost"
# Check if MG_NGINX_SERVER_NAME is set or not empty
if [ -n "${MG_NGINX_SERVER_NAME:-}" ]; then
server_name="$MG_NGINX_SERVER_NAME"
fi
source "$scriptdir/vault_cmd.sh"
vaultEnablePKI() {
vault secrets enable -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} -path ${MG_VAULT_PKI_PATH} pki
vault secrets tune -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} -max-lease-ttl=87600h ${MG_VAULT_PKI_PATH}
}
vaultConfigPKIClusterPath() {
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_PATH}/config/cluster aia_path=${MG_VAULT_PKI_CLUSTER_AIA_PATH} path=${MG_VAULT_PKI_CLUSTER_PATH}
}
vaultConfigPKICrl() {
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_PATH}/config/crl expiry="5m" ocsp_disable=false ocsp_expiry=0 auto_rebuild=true auto_rebuild_grace_period="2m" enable_delta=true delta_rebuild_interval="1m"
}
vaultAddRoleToSecret() {
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_PATH}/roles/${MG_VAULT_PKI_ROLE_NAME} \
allow_any_name=true \
max_ttl="8760h" \
default_ttl="8760h" \
generate_lease=true
}
vaultGenerateRootCACertificate() {
echo "Generate root CA certificate"
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} -format=json ${MG_VAULT_PKI_PATH}/root/generate/exported \
common_name="\"$MG_VAULT_PKI_CA_CN\"" \
ou="\"$MG_VAULT_PKI_CA_OU\"" \
organization="\"$MG_VAULT_PKI_CA_O\"" \
country="\"$MG_VAULT_PKI_CA_C\"" \
locality="\"$MG_VAULT_PKI_CA_L\"" \
province="\"$MG_VAULT_PKI_CA_ST\"" \
street_address="\"$MG_VAULT_PKI_CA_ADDR\"" \
postal_code="\"$MG_VAULT_PKI_CA_PO\"" \
ttl=87600h | tee >(jq -r .data.certificate >"$scriptdir/data/${MG_VAULT_PKI_FILE_NAME}_ca.crt") \
>(jq -r .data.issuing_ca >"$scriptdir/data/${MG_VAULT_PKI_FILE_NAME}_issuing_ca.crt") \
>(jq -r .data.private_key >"$scriptdir/data/${MG_VAULT_PKI_FILE_NAME}_ca.key")
}
vaultSetupRootCAIssuingURLs() {
echo "Setup URLs for CRL and issuing"
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_PATH}/config/urls \
issuing_certificates="{{cluster_aia_path}}/v1/${MG_VAULT_PKI_PATH}/ca" \
crl_distribution_points="{{cluster_aia_path}}/v1/${MG_VAULT_PKI_PATH}/crl" \
ocsp_servers="{{cluster_aia_path}}/v1/${MG_VAULT_PKI_PATH}/ocsp" \
enable_templating=true
}
vaultGenerateIntermediateCAPKI() {
echo "Generate Intermediate CA PKI"
vault secrets enable -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} -path=${MG_VAULT_PKI_INT_PATH} pki
vault secrets tune -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} -max-lease-ttl=43800h ${MG_VAULT_PKI_INT_PATH}
}
vaultConfigIntermediatePKIClusterPath() {
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_INT_PATH}/config/cluster aia_path=${MG_VAULT_PKI_INT_CLUSTER_AIA_PATH} path=${MG_VAULT_PKI_INT_CLUSTER_PATH}
}
vaultConfigIntermediatePKICrl() {
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_INT_PATH}/config/crl expiry="5m" ocsp_disable=false ocsp_expiry=0 auto_rebuild=true auto_rebuild_grace_period="2m" enable_delta=true delta_rebuild_interval="1m"
}
vaultGenerateIntermediateCSR() {
echo "Generate intermediate CSR"
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} -format=json ${MG_VAULT_PKI_INT_PATH}/intermediate/generate/exported \
common_name="\"$MG_VAULT_PKI_INT_CA_CN\"" \
ou="\"$MG_VAULT_PKI_INT_CA_OU\""\
organization="\"$MG_VAULT_PKI_INT_CA_O\"" \
country="\"$MG_VAULT_PKI_INT_CA_C\"" \
locality="\"$MG_VAULT_PKI_INT_CA_L\"" \
province="\"$MG_VAULT_PKI_INT_CA_ST\"" \
street_address="\"$MG_VAULT_PKI_INT_CA_ADDR\"" \
postal_code="\"$MG_VAULT_PKI_INT_CA_PO\"" \
| tee >(jq -r .data.csr >"$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.csr") \
>(jq -r .data.private_key >"$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.key")
}
vaultSignIntermediateCSR() {
echo "Sign intermediate CSR"
if is_container_running "magistrala-vault"; then
docker cp "$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.csr" magistrala-vault:/vault/${MG_VAULT_PKI_INT_FILE_NAME}.csr
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} -format=json ${MG_VAULT_PKI_PATH}/root/sign-intermediate \
csr=@/vault/${MG_VAULT_PKI_INT_FILE_NAME}.csr ttl="8760h" \
ou="\"$MG_VAULT_PKI_INT_CA_OU\""\
organization="\"$MG_VAULT_PKI_INT_CA_O\"" \
country="\"$MG_VAULT_PKI_INT_CA_C\"" \
locality="\"$MG_VAULT_PKI_INT_CA_L\"" \
province="\"$MG_VAULT_PKI_INT_CA_ST\"" \
street_address="\"$MG_VAULT_PKI_INT_CA_ADDR\"" \
postal_code="\"$MG_VAULT_PKI_INT_CA_PO\"" \
| tee >(jq -r .data.certificate >"$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.crt") \
>(jq -r .data.issuing_ca >"$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}_issuing_ca.crt")
else
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} -format=json ${MG_VAULT_PKI_PATH}/root/sign-intermediate \
csr=@"$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.csr" ttl="8760h" \
ou="\"$MG_VAULT_PKI_INT_CA_OU\""\
organization="\"$MG_VAULT_PKI_INT_CA_O\"" \
country="\"$MG_VAULT_PKI_INT_CA_C\"" \
locality="\"$MG_VAULT_PKI_INT_CA_L\"" \
province="\"$MG_VAULT_PKI_INT_CA_ST\"" \
street_address="\"$MG_VAULT_PKI_INT_CA_ADDR\"" \
postal_code="\"$MG_VAULT_PKI_INT_CA_PO\"" \
| tee >(jq -r .data.certificate >"$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.crt") \
>(jq -r .data.issuing_ca >"$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}_issuing_ca.crt")
fi
}
vaultInjectIntermediateCertificate() {
echo "Inject Intermediate Certificate"
if is_container_running "magistrala-vault"; then
docker cp "$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.crt" magistrala-vault:/vault/${MG_VAULT_PKI_INT_FILE_NAME}.crt
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_INT_PATH}/intermediate/set-signed certificate=@/vault/${MG_VAULT_PKI_INT_FILE_NAME}.crt
else
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_INT_PATH}/intermediate/set-signed certificate=@"$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.crt"
fi
}
vaultGenerateIntermediateCertificateBundle() {
echo "Generate intermediate certificate bundle"
cat "$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}.crt" "$scriptdir/data/${MG_VAULT_PKI_FILE_NAME}_ca.crt" \
> "$scriptdir/data/${MG_VAULT_PKI_INT_FILE_NAME}_bundle.crt"
}
vaultSetupIntermediateIssuingURLs() {
echo "Setup URLs for CRL and issuing"
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_INT_PATH}/config/urls \
issuing_certificates="{{cluster_aia_path}}/v1/${MG_VAULT_PKI_INT_PATH}/ca" \
crl_distribution_points="{{cluster_aia_path}}/v1/${MG_VAULT_PKI_INT_PATH}/crl" \
ocsp_servers="{{cluster_aia_path}}/v1/${MG_VAULT_PKI_INT_PATH}/ocsp" \
enable_templating=true
}
vaultSetupServerCertsRole() {
if [ "$SKIP_SERVER_CERT" == "--skip-server-cert" ]; then
echo "Skipping server certificate role"
else
echo "Setup Server certificate role"
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_INT_PATH}/roles/${MG_VAULT_PKI_INT_SERVER_CERTS_ROLE_NAME} \
allow_subdomains=true \
max_ttl="4320h"
fi
}
vaultGenerateServerCertificate() {
if [ "$SKIP_SERVER_CERT" == "--skip-server-cert" ]; then
echo "Skipping generate server certificate"
else
echo "Generate server certificate"
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} -format=json ${MG_VAULT_PKI_INT_PATH}/issue/${MG_VAULT_PKI_INT_SERVER_CERTS_ROLE_NAME} \
common_name="$server_name" ttl="4320h" \
| tee >(jq -r .data.certificate >"$scriptdir/data/${server_name}.crt") \
>(jq -r .data.private_key >"$scriptdir/data/${server_name}.key")
fi
}
vaultSetupThingCertsRole() {
echo "Setup Thing Certs role"
vault write -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_PKI_INT_PATH}/roles/${MG_VAULT_PKI_INT_THINGS_CERTS_ROLE_NAME} \
allow_subdomains=true \
allow_any_name=true \
max_ttl="2160h"
}
vaultCleanupFiles() {
if is_container_running "magistrala-vault"; then
docker exec magistrala-vault sh -c 'rm -rf /vault/*.{crt,csr}'
fi
}
if ! command -v jq &> /dev/null; then
echo "jq command could not be found, please install it and try again."
exit 1
fi
readDotEnv
mkdir -p "$scriptdir/data"
vault login -namespace=${MG_VAULT_NAMESPACE} -address=${MG_VAULT_ADDR} ${MG_VAULT_TOKEN}
vaultEnablePKI
vaultConfigPKIClusterPath
vaultConfigPKICrl
vaultAddRoleToSecret
vaultGenerateRootCACertificate
vaultSetupRootCAIssuingURLs
vaultGenerateIntermediateCAPKI
vaultConfigIntermediatePKIClusterPath
vaultConfigIntermediatePKICrl
vaultGenerateIntermediateCSR
vaultSignIntermediateCSR
vaultInjectIntermediateCertificate
vaultGenerateIntermediateCertificateBundle
vaultSetupIntermediateIssuingURLs
vaultSetupServerCertsRole
vaultGenerateServerCertificate
vaultSetupThingCertsRole
vaultCleanupFiles
exit 0
docker/addons/vault/scripts/vault_unseal.sh
-46
View File
@@ -1,46 +0,0 @@
#!/usr/bin/bash
# Copyright (c) Abstract Machines
# SPDX-License-Identifier: Apache-2.0
set -euo pipefail
scriptdir="$(cd "$(dirname "${BASH_SOURCE[0]}")" >/dev/null 2>&1 && pwd)"
# default env file path
env_file="docker/.env"
while [[ "$#" -gt 0 ]]; do
case $1 in
--env-file)
if [[ -z "${2:-}" ]]; then
echo "Error: --env-file requires a non-empty option argument."
exit 1
fi
env_file="$2"
if [[ ! -f "$env_file" ]]; then
echo "Error: .env file not found at $env_file"
exit 1
fi
shift
;;
*)
echo "Unknown parameter passed: $1"
exit 1
;;
esac
shift
done
readDotEnv() {
set -o allexport
source "$env_file"
set +o allexport
}
source "$scriptdir/vault_cmd.sh"
readDotEnv
vault operator unseal -address=${MG_VAULT_ADDR} ${MG_VAULT_UNSEAL_KEY_1}
vault operator unseal -address=${MG_VAULT_ADDR} ${MG_VAULT_UNSEAL_KEY_2}
vault operator unseal -address=${MG_VAULT_ADDR} ${MG_VAULT_UNSEAL_KEY_3}