opensource/cocos
1
0
Fork 0
mirror of https://github.com/ultravioletrs/cocos.git synced 2026-08-07 07:14:50 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

COCOS-165 - Add Docker support (#180)

Browse Source 
* add docker support

* add copyright clause

* rebase docker support

* address blank lines

* update manual tests to include docker

* fix algo test

* fix docker command

* add docker doc

* fix AddDataset method

* fixed lin_reg.py

* rebsed docker implementation

* fix NewAlgorithm error

* change docker README.md based on rebase

* fix docker README

* fix docker.go gofumpt

* add option for datasets and results mount

* edit README for docker

* make docker container run command a part of docker image

* remove unused code

* make /cocos the default directory

* updated documentation

* removed docker dir

* rebased docker
This commit is contained in:
Danko Miladinovic
2024-08-21 16:42:05 +02:00
committed by GitHub
parent f906593492
commit ee8370406c
13 changed files with 357 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
test/manual/README.md
+6 -2
View File
@@ -60,7 +60,7 @@ cd ../..
# the host data information. To add the host data to the .json file that contains
# the information about the platform, run CLI with the host data in base64 format
# and the path of the backend_info.json file.:
./build/cocos-cli backend measurement '<host-data>' '<backend_info.json>'
./build/cocos-cli backend hostdata '<host-data>' '<backend_info.json>'
# For attested TLS, also define the path to the backend_info.json that contains reference values for the fields of the attestation report
export AGENT_GRPC_MANIFEST=./scripts/backend_info/backend_info.json
@@ -77,6 +77,10 @@ export AGENT_GRPC_ATTESTED_TLS=true
./build/cocos-cli algo test/manual/algo/lin_reg.py <private_key_file_path> -a python -r test/manual/algo/requirements.py
# 2023/09/21 10:43:53 Uploading algorithm binary: test/manual/algo/lin_reg.bin
# In order to run the Docker image, run the CLI program with the algorithm docker option
go run ./cmd/cli/main.go algo -a docker <path_to_docker_image.tar> <private_key_file_path>
# 2023/09/21 10:43:53 Uploading algorithm binary: <path_to_docker_image.tar>
# Run the CLI program with dataset input
./build/cocos-cli data test/manual/data/iris.csv <private_key_file_path>
# 2023/09/21 10:45:25 Uploading dataset CSV: test/manual/data/iris.csv
@@ -87,7 +91,7 @@ export AGENT_GRPC_ATTESTED_TLS=true
# 2023/09/21 10:45:40 Computation result retrieved and saved successfully!
```
Now there is a `result.bin` file in the current working directory. The file holds the trained logistic regression model. To test the model, run
Now there is a `result.zip` file in the current working directory. The file holds the trained logistic regression model. To test the model, run
```sh
python ./test/manual/algo/lin_reg.py predict results.zip ./test/manual/data
test/manual/agent-config/main.go
+1 -1
View File
@@ -32,7 +32,7 @@ func main() {
pubKeyFile := os.Args[3]
attestedTLSParam, err := strconv.ParseBool(os.Args[4])
if err != nil {
log.Fatalf("usage: %s <data-path> <algo-path> <attested-tls-bool>, <attested-tls-bool> must be a bool value", os.Args[0])
log.Fatalf("usage: %s <data-path> <algo-path> <public-key-path> <attested-tls-bool>, <attested-tls-bool> must be a bool value", os.Args[0])
}
attestedTLS := attestedTLSParam
test/manual/algo/README.md
+74 -2
View File
@@ -1,6 +1,6 @@
# Algorithm
Agent accepts binaries programs, python scripts, and wasm files. It runs them in a sandboxed environment and returns the output.
Agent accepts binaries programs, python scripts, Docker images and wasm files. It runs them in a sandboxed environment and returns the output.
## Python Example
@@ -38,7 +38,7 @@ python3 test/manual/algo/lin_reg.py predict results.zip test/manual/data
This will make inference on the results of the linear regression model.
To run the examples in the agent, you can use the following command:
To run the examples in the secure VM (SVM) by the Agent, you can use the following command:
```bash
go run ./test/computations/main.go ./test/manual/algo/lin_reg.py public.pem false ./test/manual/data/iris.csv
@@ -96,6 +96,78 @@ For addition example, you can use the following command:
./build/cocos-cli result ./private.pem
```
## Docker Example
Here we will use the docker with the linear regression example (`lin_reg.py`). Throughout the example, we assume that our current working directory is the directory in which the `cocos` repository is cloned. For example:
```bash
# ls
cocos
```
The docker image must have a `cocos` directory containing the `datasets` and `results` directories. The Agent will run this image inside the SVM and will mount the datasets and results onto the `/cocos/datasets` and `/cocos/results` directories inside the image. The docker image must also contain the command that will be run when the docker container is run.
The first step is to create a docker file. Use your favorite editor to create a file named `Dockerfile` in the current working directory and write in it the following code:
```bash
FROM python:3.9-slim
# set the working directory in the container
WORKDIR /cocos
RUN mkdir /cocos/results
RUN mkdir /cocos/datasets
COPY ./cocos/test/manual/algo/requirements.txt /cocos/requirements.txt
COPY ./cocos/test/manual/algo/lin_reg.py /cocos/lin_reg.py
# install dependencies
RUN pip install -r requirements.txt
# command to be run when the docker container is started
CMD ["python3", "/cocos/lin_reg.py"]
```
Next, run the build command and then save the docker image as a `tar` file.
```bash
docker build -t linreg .
docker save linreg > linreg.tar
```
In another window, you can run the following command:
```bash
sudo MANAGER_QEMU_SMP_MAXCPUS=4 MANAGER_GRPC_URL=localhost:7001 MANAGER_LOG_LEVEL=debug MANAGER_QEMU_USE_SUDO=false MANAGER_QEMU_ENABLE_SEV=false MANAGER_QEMU_SEV_CBITPOS=51 MANAGER_QEMU_ENABLE_SEV_SNP=false MANAGER_QEMU_OVMF_CODE_FILE=/usr/share/edk2/x64/OVMF_CODE.fd MANAGER_QEMU_OVMF_VARS_FILE=/usr/share/edk2/x64/OVMF_VARS.fd go run main.go
```
This command is run from the [manager main directory](../../../cmd/manager/). This will start the manager. Make sure you have already built the [qemu image](../../../hal/linux/README.md).
In another window, specify what kind of algorithm you want the Agent to run (docker):
```bash
./cocos/build/cocos-cli algo ./linreg.tar ./cocos/private.pem -a docker
```
make sure you have built the cocos-cli. This will upload the docker image.
Next we need to upload the dataset
```bash
./cocos/build/cocos-cli data ./cocos/test/manual/data/iris.csv ./cocos/private.pem
```
After some time when the results are ready, you can run the following command to get the results:
```bash
./cocos/build/cocos-cli results ./cocos/private.pem
```
This will return the results of the algorithm.
To make inference on the results, you can use the following command:
```bash
python3 ./cocos/test/manual/algo/lin_reg.py predict result.zip ./cocos/test/manual/data
```
## Wasm Example
More information on how to run wasm files can be found [here](https://github.com/ultravioletrs/ai/tree/main/burn-algorithms).