Compare commits

..

443 Commits

Author SHA1 Message Date
Sammy Kerata Oina 10037ad874 NOISSUE - Fix TLS connection (#329)
* fix tls

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* refactor

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-12-06 14:49:46 +01:00
Washington Kigani Kamadi ec426e58a2 COCOS - 209 - Restructure agent and manager gRPC config (#297)
* restructure grpc configs

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

enhance clients

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

restructure config

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

refactor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

rebase

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

rebase

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

use separate configuration

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

fix tests

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

fix config

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

refactor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

Lint

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

fix tests

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

add tests

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

add test case

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

add test case

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

refactor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

further refactor'

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

add tests

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

rebase

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* remove redundant code

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* fix test

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-12-04 20:32:46 +01:00
Sammy Kerata Oina 92a4f8bd32 NOISSUE - Fix loading of CA certs on agent (#321)
* debug connection

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* actual fix

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove debugs

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add unit test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* more tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* consolidate tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix client auth

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* debug

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* better handling

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-12-04 06:03:41 -07:00
Sammy Kerata Oina 0864eb69c9 NOISSUE - Update buildroot config (#320)
* update cfg

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* update workflow

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-29 13:58:28 +01:00
Danko Miladinovic 63f20ea34d NOISSUE - Remove ramfs (#322) 2024-11-29 12:48:03 +01:00
Sammy Kerata Oina bceb1727d1 NOISSUE - Use Mockery config (#323)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-28 15:11:13 +01:00
Sammy Kerata Oina febdba4f96 NOISSUE - Add quotes on cmdline (#318)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-25 17:10:35 +01:00
Sammy Kerata Oina d70bf052a5 NOISSUE - Prevent race conditions on manager (#316)
* fix panic

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* no races

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-22 05:07:38 -07:00
Sammy Kerata Oina 77d99d4beb PRISM-598 - Return correct port on run (#315)
* attestation policy field

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fmt

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* return correct port

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-22 10:45:57 +01:00
Sammy Kerata Oina 760c9bb580 NOISSUE - Rename backend info to attestation policy (#314)
* attestation policy field

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fmt

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-21 20:56:03 +01:00
dependabot[bot] 9c11f4b662 NOISSUE - Bump the go-dependency group across 1 directory with 7 updates (#313)
Bumps the go-dependency group with 3 updates in the / directory: [go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc](https://github.com/open-telemetry/opentelemetry-go-contrib), [golang.org/x/crypto](https://github.com/golang/crypto) and google.golang.org/protobuf.


Updates `go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc` from 0.56.0 to 0.57.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/zpages/v0.56.0...zpages/v0.57.0)

Updates `go.opentelemetry.io/otel/trace` from 1.31.0 to 1.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.31.0...v1.32.0)

Updates `golang.org/x/crypto` from 0.28.0 to 0.29.0
- [Commits](https://github.com/golang/crypto/compare/v0.28.0...v0.29.0)

Updates `golang.org/x/sync` from 0.8.0 to 0.9.0
- [Commits](https://github.com/golang/sync/compare/v0.8.0...v0.9.0)

Updates `google.golang.org/grpc` from 1.67.1 to 1.68.0
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.67.1...v1.68.0)

Updates `google.golang.org/protobuf` from 1.35.1 to 1.35.2

Updates `golang.org/x/term` from 0.25.0 to 0.26.0
- [Commits](https://github.com/golang/term/compare/v0.25.0...v0.26.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: go.opentelemetry.io/otel/trace
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-dependency
- dependency-name: golang.org/x/term
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-19 11:12:10 +01:00
Sammy Kerata Oina 5a22ac2eca COCOS-159 - Improve test coverage to 65% (#310)
* few atls tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove commented code

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add atls tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* new line

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add more test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* more test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add empty line and parallel test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* move const outside test case

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-19 08:13:33 +01:00
Danko Miladinovic d5941edb56 NOISSUE - Add information on OVMF verison, CPU type, CPU number and EOS version (#307)
* add manager OVMF verison information

* add Manager documentation

* add kernel cmd

* add SVMInfo test
2024-11-18 17:16:33 +01:00
Sammy Kerata Oina 04b51a6073 NOISSUE - Remove duplicate event (#308)
* remove duplicate event

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* generate string

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-11 08:04:30 -07:00
Sammy Kerata Oina 1e285e32b4 NOISSUE - Manifest checksum (#306)
* update backend info, and generate manifest checksun

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* update report

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-08 14:56:45 +01:00
Sammy Kerata Oina a3577da5b2 NOISSUE - aTLS minor fixes (#305)
* set minimum args

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* update cli and allow provisional firmware

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-07 18:05:41 +01:00
Sammy Kerata Oina 151262d039 set minimum args (#304)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-07 16:41:23 +01:00
Sammy Kerata Oina 9b7f105691 fix minor bugs (#303)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-07 14:40:23 +01:00
Sammy Kerata Oina 46b94204df NOISSUE - Improve file streaming (#295)
* improve file streaming

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* error check

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* empty line

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* send buffer test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* stream data and attestation

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fumpt

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* mocks

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* value check

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* more value checks

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fumpt

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* all  files

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-07 10:47:53 +01:00
Sammy Kerata Oina 01a619fd2a NOISSUE - Improve SDK initialization (#302)
* do not require agent connection for all commands

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* improve tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* more test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* blank line

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* ignore cmd

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-07 09:09:51 +01:00
Sammy Kerata Oina d3636de824 NOISSUE - Remove blocking on vsock (#301)
* no blocking

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* return error

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-06 02:31:04 +01:00
Sammy Kerata Oina 0380b2323a NOISSUE - Efficient cid assign (#300)
* efficient cid assign

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* order

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-11-05 17:46:34 +01:00
Danko Miladinovic 3951a6248a NOISSUE - Change guest to kernel version 6.12-rc6 (#299)
Co-authored-by: ultraviolet <cocosai@ultraviolet.local.pragmatic-it.com>
2024-11-05 16:04:17 +01:00
Danko Miladinovic e372cfc219 COCOS-192 - Add support for attested TLS (#279)
* add draft tls extension

* add client support for ipv6

* remove vscode

* add evidence request server payload

* clean up the code

* add fetch and verify for quote provider

* add build parameters for buildroot

* change Makefile to always enable CGO

* fix ci

* add malloc check for NULL

* add copyright

* renamed files and fix cgo lint

* fix cache test

* fix server tests

* remove ineffective assignment

* fix no-TLS connection

* add check for SSL_set_fd failure

* add tests for verification of attestation

* fix CI

* fix failing tests

* fix backend tests

* remove commented code

* separate verify and validate function

* fix failing test

* Simplify function name

---------

Co-authored-by: ultraviolet <cocosai@ultraviolet.local.pragmatic-it.com>
2024-11-04 19:10:34 +01:00
Danko Miladinovic 6f747190b9 NOISSUE - Add SEV-SNP support for kernel 6.11 (#298) 2024-11-04 12:33:45 +01:00
Sammy Kerata Oina f6a93fe2a1 COCOS-255 - Human readable attestation (#289)
* human readable attestation

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add tests

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix tests

Signed-off-by: SammyOina <sammyoina@gmail.com>

* err check

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix roundtrip

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-10-31 16:02:41 +01:00
Sammy Kerata Oina 534ad91623 NOISSUE - Remove redundant retry logic (#293)
* remove redundant logic

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove line

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix internal tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* all pb files

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-31 15:46:56 +01:00
Sammy Kerata Oina 69b8dfa3ea NOISSUE - Fix failing test (#292)
* fix failing test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove junk code

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix yml

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-30 17:17:04 +01:00
Sammy Kerata Oina fad3182638 NOISSUE - Refactor manager events and detangle service (#287)
* extract events service

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* major refactor and detangling

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* small fixes

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* handle tests better

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix race condition

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix race

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* use plain interface

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* move mutex

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-30 16:07:54 +01:00
Sammy Kerata Oina 20e7ea76e0 NOISSUE - Improve pkg tests (#286)
* add pkg tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* expect errors

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix assertions

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix test cases

Signed-off-by: SammyOina <sammyoina@gmail.com>

* coverage files

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add more test cases

Signed-off-by: SammyOina <sammyoina@gmail.com>

* improve tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* update test descriptions

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-10-30 16:04:30 +01:00
Sammy Kerata Oina 2a6fa8da25 NOISSUE - Add health check (#288)
* add health check

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add test case

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint and add test case

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* switch context

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-30 15:28:07 +01:00
Sammy Kerata Oina 6043ad150b COCOS-256 - Progress bar on downloads (#290)
* add progress bar for downloads

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* better error handling

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix test and refactor

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix failing test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add test coverage

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-29 12:33:00 +01:00
dependabot[bot] 765513b387 NOISSUE - Bump github.com/fatih/color (#291)
Bumps the go-dependency group with 1 update in the / directory: [github.com/fatih/color](https://github.com/fatih/color).


Updates `github.com/fatih/color` from 1.17.0 to 1.18.0
- [Release notes](https://github.com/fatih/color/releases)
- [Commits](https://github.com/fatih/color/compare/v1.17.0...v1.18.0)

---
updated-dependencies:
- dependency-name: github.com/fatih/color
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-29 09:49:54 +01:00
dependabot[bot] 3ca3d48926 Bump the go-dependency group across 1 directory with 2 updates (#285)
Bumps the go-dependency group with 1 update in the / directory: [go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc](https://github.com/open-telemetry/opentelemetry-go-contrib).


Updates `go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc` from 0.55.0 to 0.56.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/zpages/v0.55.0...zpages/v0.56.0)

Updates `go.opentelemetry.io/otel/trace` from 1.30.0 to 1.31.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.30.0...v1.31.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: go.opentelemetry.io/otel/trace
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-21 10:57:03 +02:00
Sammy Kerata Oina 184617da9e NOISSUE - Host data verification (#275)
* host data verification

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* update mocks

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* debug host data

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* debug

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* check device

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* imorove test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* missing header

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* update embed option

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* minor fixes

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* update deps

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* each case is unique

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* all files

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix coverage

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* resolve comments

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* improve coverage

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add test case

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* use consts

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add coverage

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* make sure pid is exited

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-14 11:12:49 +02:00
dependabot[bot] bb903c0170 NOISSUE - Bump the go-dependency group across 1 directory with 4 updates (#276)
Bumps the go-dependency group with 3 updates in the / directory: [golang.org/x/crypto](https://github.com/golang/crypto), [google.golang.org/grpc](https://github.com/grpc/grpc-go) and google.golang.org/protobuf.


Updates `golang.org/x/crypto` from 0.27.0 to 0.28.0
- [Commits](https://github.com/golang/crypto/compare/v0.27.0...v0.28.0)

Updates `google.golang.org/grpc` from 1.67.0 to 1.67.1
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.67.0...v1.67.1)

Updates `google.golang.org/protobuf` from 1.34.2 to 1.35.1

Updates `golang.org/x/term` from 0.24.0 to 0.25.0
- [Commits](https://github.com/golang/term/compare/v0.24.0...v0.25.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-dependency
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: golang.org/x/term
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-09 20:17:30 +02:00
Sammy Kerata Oina 18aa8ba785 NOISSUE - Add internal tests (#266)
* add internal tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix linter

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix race conditions

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove all races

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-09 20:01:11 +02:00
Sammy Kerata Oina db7f3c7a4b COCOS-278 - Abstract state machine (#280)
* abstract state machine

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* perpetual results consumption

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* async action

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix failing tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix failing test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-09 13:19:12 +02:00
Sammy Kerata Oina fb0fbaeb9a COCOS-253 - Improve CLI error handling (#277)
* decode errors

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* standardise error formatting

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix failing tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add errors tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* pass lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add test cases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-08 17:11:37 +02:00
Sammy Kerata Oina 7ef25674c4 add cli tests (#274)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-08 16:28:17 +02:00
Sammy Kerata Oina 034547d667 NOISSUE - Add VM state machine and filter on qemu logs (#272)
* add vm state machine and filter on qemu logs

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix failing test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix logging test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix failing test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-08 16:02:17 +02:00
Sammy Kerata Oina 643c132ff7 NOISSUE - Add pkg tests (#269)
* add pkg tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* rename function

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-08 15:50:50 +02:00
Sammy Kerata Oina 5e01ecdab7 add manager tests (#273)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-08 15:35:17 +02:00
Sammy Kerata Oina f6b69d65df NOISSUE - Add agent pkg tests (#271)
* add agent tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-10-08 15:29:21 +02:00
dependabot[bot] faaddc3571 NOISSUE - Bump google.golang.org/grpc (#270)
Bumps the go-dependency group with 1 update in the / directory: [google.golang.org/grpc](https://github.com/grpc/grpc-go).


Updates `google.golang.org/grpc` from 1.66.1 to 1.67.0
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.66.1...v1.67.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-02 15:55:44 +02:00
Smith Jilks 3d9fde39c2 NOISSUE - Enhance CLI (#250)
* Enhance CLI progressbar

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update cli error and success messages colors

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update cli emojis

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Add logs for cli interrupt by user

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Remove extra whitespaces

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update upload data emoji

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update cli main.go

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update cli errors

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update cli

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update cli

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update go sum

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Add progressbar tests

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Fix cli cmd error formating

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Add cli datasets, algo and result tests

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

---------

Signed-off-by: Jilks Smith <smithjilks@gmail.com>
2024-10-01 10:25:52 +02:00
b1ackd0t 63994d78b8 NOISSUE - Add Rust gitignore (#268)
* chore(backendinfo): Add rust build artefacts to gitignore

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* style: format file following rust linter guidelines

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* chore(CI): Add rust CI pipeline

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

---------

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
2024-09-30 11:49:18 +02:00
Sammy Kerata Oina 115c6c24c0 NOISSUE - Fix file format (#261)
* add coverage

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* use codecov

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* rename extension

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-27 10:52:52 +02:00
Sammy Kerata Oina c69dcd0e2d NOISSUE - Improve reliability of state machine test (#260)
* add sleep to prevent test failing

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add coverage

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* use codecov

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* create dir

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-26 11:59:26 +02:00
Sammy Kerata Oina 6c4819563c remove vnc (#259)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-25 21:36:40 +02:00
dependabot[bot] aa01ef795d Update sev requirement (#257)
Updates the requirements on [sev](https://github.com/virtee/sev) to permit the latest version.

Updates `sev` to 4.0.0
- [Commits](https://github.com/virtee/sev/compare/v3.1.1...v4.0.0)

---
updated-dependencies:
- dependency-name: sev
  dependency-type: direct:production
  dependency-group: rs-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-25 17:38:24 +02:00
dependabot[bot] 18102db235 Bump the go-dependency group across 1 directory with 8 updates (#258)
Bumps the go-dependency group with 3 updates in the / directory: [github.com/caarlos0/env/v11](https://github.com/caarlos0/env), [go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc](https://github.com/open-telemetry/opentelemetry-go-contrib) and [github.com/docker/docker](https://github.com/docker/docker).


Updates `github.com/caarlos0/env/v11` from 11.1.0 to 11.2.2
- [Release notes](https://github.com/caarlos0/env/releases)
- [Changelog](https://github.com/caarlos0/env/blob/main/.goreleaser.yml)
- [Commits](https://github.com/caarlos0/env/compare/v11.1.0...v11.2.2)

Updates `go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc` from 0.53.0 to 0.55.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/zpages/v0.53.0...zpages/v0.55.0)

Updates `go.opentelemetry.io/otel/trace` from 1.28.0 to 1.30.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.28.0...v1.30.0)

Updates `golang.org/x/crypto` from 0.25.0 to 0.27.0
- [Commits](https://github.com/golang/crypto/compare/v0.25.0...v0.27.0)

Updates `golang.org/x/sync` from 0.7.0 to 0.8.0
- [Commits](https://github.com/golang/sync/compare/v0.7.0...v0.8.0)

Updates `google.golang.org/grpc` from 1.65.0 to 1.66.1
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.65.0...v1.66.1)

Updates `github.com/docker/docker` from 27.1.0+incompatible to 27.3.1+incompatible
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v27.1.0...v27.3.1)

Updates `golang.org/x/term` from 0.22.0 to 0.24.0
- [Commits](https://github.com/golang/term/compare/v0.22.0...v0.24.0)

---
updated-dependencies:
- dependency-name: github.com/caarlos0/env/v11
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: go.opentelemetry.io/otel/trace
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
- dependency-name: golang.org/x/term
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-dependency
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-25 17:15:58 +02:00
Sammy Kerata Oina 8b37b35750 COCOS-154 - Fix HAL release pipeline (#254)
* increase release pipeline

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* update go and free some space

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* optimize

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix cache

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* free up space

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* modify

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove restrictions

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fifty gigs

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fourty gigs

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* old mbs

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove outdated actions

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* rename

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* weekly update

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-09-25 16:27:28 +02:00
Smith Jilks af3817d3b7 COCOS-242-Agent tests fail ocassionally due to a missing mock (#251)
* Clean up files after algo run

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Add test cleanup

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

---------

Signed-off-by: Jilks Smith <smithjilks@gmail.com>
2024-09-24 14:44:27 +02:00
Sammy Kerata Oina 5d5ae35e2b NOISSUE - Reduce message loss via vsock with acks (#252)
* state check within func

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* debug logs sending

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* debug message sending

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* ack messages

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* handle proto better

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* improve concurrency

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* improve manager handling

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove debug lines

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* sync next id

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* reduce locks

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-23 18:38:02 +02:00
Washington Kigani Kamadi df923f9b1f NOISSUE - Rename error to warning (#249)
* rename error to warning

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* update logging package

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-09-20 18:33:10 +02:00
Sammy Kerata Oina a7caa59137 NOISSUE - Fix race condition in tests (#248)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-20 11:59:56 +02:00
Sammy Kerata Oina eab3a06705 fix redundant logs (#247)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-20 11:09:31 +02:00
Washington Kigani Kamadi 5ff8b96311 add disconnected status (#246)
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-09-20 10:00:48 +02:00
Sammy Kerata Oina e266e91033 COCOS-238 - Add measurement directly on backend info file (#245)
* add measurement directly on backendinfo

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add host data

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-19 21:32:38 +02:00
Sammy Kerata Oina 4c09b4bea5 NOISSUE - Format log messages from agent (#244)
* downgrade mod

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add fields to logging

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-19 10:18:02 +02:00
Washington Kigani Kamadi 1546fbc4c2 NOISSUE - Use Constants for Run Events (#243)
* enhance timeline

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* fix: remove redundant event

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* use constant

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* lint

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* use typed constant for status

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* export agent status and state

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* ehance event states

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* use manager states and status

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* move algo-run to agent package

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* replace manager variable with constant

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* add manager states

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* remove typo

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-09-17 18:01:30 +02:00
Sammy Kerata Oina 2f4ca414cb NOISSUE - Stop computation gracefully (#241)
* stop gracefully

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* use constant

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-17 17:57:42 +02:00
Washington Kigani Kamadi c14a6338cc NOISSUE - Enhance event status (#235)
* enhance timeline

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* fix: remove redundant event

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* use constant

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* lint

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* use typed constant for status

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* refactor status

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* export agent status and state

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* ehance event states

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* fix tests

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* use manager states and status

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* move algo-run to agent package

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* replace literal with constant

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* replace manager variable with constant

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-09-17 15:58:15 +02:00
Sammy Kerata Oina 355f95771d NOISSUE - Use constants for log level (#240)
* use constants for log level

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-13 14:10:19 +02:00
Smith Jilks e26deb98e4 COCOS-143 - Add agent service tests (#170)
* Add agent service tests

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update agent service tests

* Fix agent service tests

* Improve agent service test coverage

* Improve agent service test coverage

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Fix tests

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Refactor and improve coverage

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

---------

Signed-off-by: Jilks Smith <smithjilks@gmail.com>
2024-09-12 16:54:09 +02:00
Sammy Kerata Oina 20ddb3aa29 restart always (#239)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-12 15:55:53 +02:00
Sammy Kerata Oina e0b828d0ae use syslog (#237)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-11 16:28:07 +02:00
Sammy Kerata Oina 46d24f928a NOISSUE - Add events for disconnection of agent (#233)
* add events

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* typo

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* group logs

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix error

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix initialization of goroutine

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add comment

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update comment

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove naked return

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-09-11 14:26:46 +02:00
Sammy Kerata Oina 8db88ccbde NOISSUE - Fix handling of runreq chunks (#234)
* fix handling of runreq chunks

Signed-off-by: SammyOina <sammyoina@gmail.com>

* copy ovmf vars

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint errors

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-09-06 17:49:05 +02:00
Sammy Kerata Oina c2a4b44769 NOISSUE - Cache and retry message sending (#222)
* cache and retry message sending

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* cache and retry message sending

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove safeconn

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* simplify retry

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* debug disconnect

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove debug

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* simplify

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-09-06 17:02:30 +02:00
Sammy Kerata Oina 51b129c3a2 NOISSUE - Flush Docker logs (#229)
* flush docker logs

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* show logs in realtime

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add tty

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove duplicate

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* python3

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* error check

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove capitalization

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-09-06 12:53:48 +02:00
Sammy Kerata Oina f848afeefd NOISSUE - Define sendFunc type (#232)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-05 14:32:04 +02:00
Sammy Kerata Oina 00980639d5 NOISSUE - Remove run channel (#231)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-05 12:27:06 +02:00
Sammy Kerata Oina 7ba34b93bc NOISSUE - Streamline message processing to prevent potential message loss (#228)
* fix dropping of message response from manager

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove change

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* simplify

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add message send timeout

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-09-03 11:29:07 +02:00
Sammy Kerata Oina 9ca045b06a COCOS-214 - Improve manager resiliance by tracking vms on restart (#219)
* track hanging vm processes

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix run test

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix stop computation

Signed-off-by: SammyOina <sammyoina@gmail.com>

* shutdown gracefully

Signed-off-by: SammyOina <sammyoina@gmail.com>

* check if process still exists

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* use const

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-30 18:08:11 +02:00
Sammy Kerata Oina e572793295 exit with error code (#225)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-30 15:32:23 +02:00
Sammy Kerata Oina 5383f4465b NOISSUE - Exit on network failures only (#227)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-30 15:30:51 +02:00
Sammy Kerata Oina dc349e1f1f NOISSUE - V-Sock reconnect for agent (#215)
* vsock reconnect

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* use backoff

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-30 15:29:39 +02:00
b1ackd0t 742bba5f00 NOISSUE - Add Dockerfile For IRIS Example (#220)
* feat(Docker): Add Dockerfile for testing

Add Dockerfile for testing linear regression algorithm

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* fix(docs): Update docker linear regression example

Resolves https://github.com/ultravioletrs/cocos/pull/220#discussion_r1732974631

---------

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
2024-08-29 22:32:31 +02:00
Sammy Kerata Oina bdfc5fd06d run manager using systemd (#213)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-28 23:11:49 +02:00
Sammy Kerata Oina 7a2789fb5f NOISSUE - Remove race condition returning before all go routines have completed (#221)
* remove race condition returning before all go routines have completed

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* refine

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-28 18:16:15 +02:00
Sammy Kerata Oina d0c99479db NOISSUE - Remove CID tracking (#218)
* remove cid tracking

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove unused code

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-27 02:30:00 +02:00
b1ackd0t 01c502e1a1 NOISSUE - Fix Algo Path (#216)
* fix(algorithm): fix running binary algorithm

This is the error
```
"algorithm": executable file not found in $PATH
```

* fix(algorithm): get absolute filepath for all algorithms

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* fix(lint): fix linting errors

* refactor: simplify algorithm filename

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

---------

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
2024-08-26 17:19:47 +02:00
Sammy Kerata Oina bdbeb4e976 COCOS-199 - Enable testing of SEV features on any machine (#205)
* make attestation embeddable

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* mock backend info

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* embed files

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* finish up

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-23 16:52:07 +02:00
Sammy Kerata Oina c402248515 COCOS-169 - Add support for algo arguments (#202)
* custom args

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* DEBUG

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* args bug

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* switch to slice

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add flags

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* switch to string array

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-23 16:51:35 +02:00
Washington Kigani Kamadi 7155027440 NOISSUE: Return Response on Computation Termination. (#211)
* send response to manager on computation termination

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* fix tests

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* refactor: enhance stop computation

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* remove comment and add event

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-08-23 15:30:57 +02:00
Danko Miladinovic 4c80b574e3 add ramfs to HAL (#212) 2024-08-23 15:30:04 +02:00
Danko Miladinovic ee8370406c COCOS-165 - Add Docker support (#180)
* add docker support

* add copyright clause

* rebase docker support

* address blank lines

* update manual tests to include docker

* fix algo test

* fix docker command

* add docker doc

* fix AddDataset method

* fixed lin_reg.py

* rebsed docker implementation

* fix NewAlgorithm error

* change docker README.md based on rebase

* fix docker README

* fix docker.go gofumpt

* add option for datasets and results mount

* edit README for docker

* make docker container run command a part of docker image

* remove unused code

* make /cocos the default directory

* updated documentation

* removed docker dir

* rebased docker
2024-08-21 16:42:05 +02:00
Sammy Kerata Oina f906593492 remove tmp directory (#204)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-21 11:55:36 +02:00
Sammy Kerata Oina 899bfb0ec5 COCOS-151 - Add compression/decompression option for CLI/Agent (#200)
* on the fly compression

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* rename file-hash to checksum

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* check error properly

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix lint

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix connection handling

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-21 11:54:52 +02:00
Sammy Kerata Oina e4ef1aae36 COCOS-181 - Add failed state for algo execution (#197)
* additional state failed

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* generate state string

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix logic

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix typo

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* conditional transition

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-21 00:02:09 +02:00
b1ackd0t 6aec555f4e NOISSUE - Start Port Search At Known Port (#207)
* feat(manager): start port search at minimum port range

When getting free port to be use we start at the minimum port range i.e 6000. This makes the agent url more deterministic since in most cases the agent port will be 6000

* fix(manager): Remove unused error channel

The error channel is not used

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

---------

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
2024-08-20 23:59:28 +02:00
Sammy Kerata Oina 066dacd46a NOISSUE - Fix docs (#203)
* fix docs

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix typos

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* cli

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add build instructions

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove file

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-20 23:58:37 +02:00
Sammy Kerata Oina 8f2cbd349d bump port range (#206)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-20 23:56:34 +02:00
Sammy Kerata Oina f596702e6e NOISSUE - Fix bug on same dataset provider for multiple datasets (#198)
* fix bug on same dataset provider

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix failing test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-20 23:25:13 +02:00
Sammy Kerata Oina 31391a36a1 NOISSUE - Add optional Filename (#195)
* add optional filename

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* cover all bases

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* maintain filename

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-09 13:09:11 +02:00
Sammy Kerata Oina 971e00aa68 NOISSUE - Select from port range (#196)
* select from port range

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix failing test

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove whitespace

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-08-08 15:48:12 +02:00
Washington Kigani Kamadi 18cfa7619e PRISM-337: Add Filename to Dataset (#191)
* add filename to dataset

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* update protoc

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-08-07 15:58:25 +02:00
b1ackd0t afc306a85b NOISSUE - Enable WASM Support and FileSystem Support (#189)
* feat(algorithm): Add wasm as an algo type

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* feat(algorithm): Use filesystem to store results

Move from unix socket for results storage to filesystem

* test: test new filesystem changes

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* refactor(files): rename resultFile to resultsFilePath

* feat(wasm-runtime): change from wasmtime to wasmedge

Wasmedge enables easier directory mapping to get results

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* feat(algorithm): send results as zipped directory

Create a new function to zip the results directory and send it back to the user

* fix(wasm): runtime argument

Fix the directory mapping for wasm runtime arguments

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* fix(errors): provide useful error message

* chore(gitignore): add results zip to gitignore

* feat(filesystem): Enable storing results on filesystem for python algos

* refactor: revert to upstream cocos repo

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* fix: remove AddDataset from algorithm interface

* fix: agent to handle results zipping

* test: test zipping directories

* refactor(agent): Handle file operations from agent

* test: run test inside eos

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* refactor(test): Document and test algos are running

Document steps on running the 2 python exampls and ensure they are running on eos

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* fix: remove witheDataset option

* test: test without dataset argument

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

---------

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
2024-08-06 19:06:48 +02:00
Smith Jilks 3c855e3b68 NOISSUE - Handle larger manifests exceeding the default grpc limit (#161)
* Handle larger manifests exceeding the default grpc limit

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update manager tests

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update manager tests

* Update manager client.go

* Update manager client.go

* Update manager client.go

* Update manager grpc server.go

* Update manager grpc server and client

---------

Signed-off-by: Jilks Smith <smithjilks@gmail.com>
2024-08-06 16:25:35 +02:00
Washington Kigani Kamadi 9161d30683 PRISM-312 : Fetch Backend Information (#187)
* fetch backend info

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

WIP

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* add id to grpc response

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* read backend information

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

revert changes in test server

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* update info json

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* test on dell machine

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* update protoc

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

update protoc

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* refactor fetch backend info

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* remove computation definition

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* refactor manager service creation

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* refactor manager service creation:

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* return config to main

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* add tests on test/computation

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* update backend info path

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* use sudo

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* comment out sev testing section

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* update backend info json location

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* handle failed execution

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* return error on failed execution:

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-08-01 15:02:50 +02:00
Sammy Kerata Oina e376cf35a4 feat: Add support for GCC and libstdc++ packages in cocos_defconfig (#188)
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-07-31 12:43:40 +02:00
Danko Miladinovic 24a76a1685 NOISSUE - Fix attested TLS and attestation CLI (#184)
* update code to reflect updated libraries

* fix attestation CLI

* update manual test for attestation CLI
2024-07-30 16:05:49 +02:00
b1ackd0t 6d170e9ca4 NOISSUE - Fix failing tests (#186)
* test(events): failing tests

* fix: Add datapath as optional parameter

If datapath is not provided if should work as assume dataset is empty
If an invalid i.e non existent path is given it should return an error
2024-07-26 12:05:26 +02:00
b1ackd0t c9ec42deb2 fix(computation): Allow computation with empty dataset (#185) 2024-07-26 11:22:13 +02:00
Washington Kigani Kamadi 7498800d2d update attestation command example (#183)
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-07-25 01:25:31 +02:00
Sammy Kerata Oina 46555a72ba NOISSUE - Error logs instead of failure on stderr (#182)
* warning logs

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* switch to error

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-07-24 14:28:51 +02:00
Sammy Kerata Oina 67d01e39be COCOS-155 - Add python algo support (#178)
* * feat(algorithm.go): add support for algorithm type context
* feat(python.go): implement Python algorithm runtime
* fix(cocos_defconfig): add IPTABLES package

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update proto

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* small fixes

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add metadata

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* debug

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* debug

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* chunk logger

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* debug logger

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* test lock

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add req file

Signed-off-by: SammyOina <sammyoina@gmail.com>

* stream result

Signed-off-by: SammyOina <sammyoina@gmail.com>

* test with venv

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix missing requirements file

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* result stream

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* modify test server

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* remove debugging and cleaning up

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* original repo

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* add missing header

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* downgrade protoc

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2024-07-23 16:38:03 +02:00
b1ackd0t 2f8109879c COCOS-168 - Allow running Computations without datasets (#175)
* feat(agent): Allow empty dataset

Allow running of algorithm with empty dataset since not all algorithms require datasets.
Allow state-machine transition from algo-received state to running state incase of no dataset provided

Fixes https://github.com/ultravioletrs/cocos/issues/168

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* chore(gitignore): Remove build artefacts

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* feat(algorithms): Add test algorithm for addition

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* refactor(addition): Modify addition algo to one file

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* fix(agent): move state transition to callback func

Move state transition from `receivingAlgorithm` to `running` to state call back function

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* feat(agent-event): Add `algoReceivedNoData` event

`algoReceivedNoData` is an event that is sent if we receive an algorithm and it should not have a dataset hence changes the state from `receivingAlgorithm` to `running`

* fix(agent-state): Change state depending on manifest

Change state from `receivingAlgorithm` to either `receivingData` if there is a dataset or `running` if there is no dataset provided

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

---------

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
2024-07-22 14:51:16 +02:00
Smith Jilks 88707c4668 NOISSUE - Fix Jaeger tracer (#177) 2024-07-22 14:49:42 +02:00
Washington Kigani Kamadi d76074ae41 COCOS-144 - Add Agent SDK Tests (#167)
* add tests and mocks

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

fix ci

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

update test

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

fix(agent/grpc): revert change

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

fix ci

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* refactor attestation and report tests

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

refactor tests

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

remove commented code

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

remove comment

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

remove comments

* add test cases

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

export agent errors

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

remove comm

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* fix tests

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-07-11 20:53:31 +02:00
Sammy Kerata Oina 2ceb1c3562 COCOS-152 - Refactor algorithm execution and add visibility through logging and events (#176)
* * feat(agent): add support for binary algorithm execution
*
* feat(agent/algorithm): add Algorithm interface and binary implementation
*
* feat(agent/algorithm/binary): implement Run method for binary algorithm execution
*
* feat(agent/algorithm/logging): implement Stdout and Stderr writers for algorithm logging
*
* feat(agent/algorithm/logging_test): add tests for Stdout and Stderr writers
*
* feat(agent/events): add Service interface for sending events
*
* feat(agent/events/mocks): add mock implementation for Service interface
*
* refactor(agent/service): update runComputation method to use binary algorithm implementation

Signed-off-by: SammyOina <sammyoina@gmail.com>

* * fix(logging.go): handle error when sending event in Write method of Stderr struct
* test(logging_test.go): add copyright header
* fix(backend_info.go): add missing type declaration in function signature
* fix(agent.go): rename progressbar variable to pb for clarity and consistency

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-07-11 12:24:19 +02:00
b1ackd0t dc16e8a997 NOISSUE - Use Magistrala Shared Packages (#174)
* refactor(env): remove internal env package

No need for this package since the parent package github.com/caarlos0/env does everything we need

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* refactor(jaeger): remove internal jaeger package

No need for this package since we can use magistrala exported package

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* refactor(metrics): remove internal metrics pkg

Use exported magistrala prometheus package

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* chore(dep): Update grpc and other dependencies

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

* style(linter): remove enabled by default linters

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>

---------

Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
2024-07-10 15:48:37 +02:00
Sammy Kerata Oina f4e3e8e09c COCOS-157 - Provide abstractions for VM management (#171)
* abstract vm creation and allow stopping computation

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor QEMU configuration loading and execution in main.go

Signed-off-by: SammyOina <sammyoina@gmail.com>

* * feat(agent-config): add support for sending agent configuration to manager

Signed-off-by: SammyOina <sammyoina@gmail.com>

* * chore(checkproto.yaml): update protoc-gen and protoc-grpc versions

Signed-off-by: SammyOina <sammyoina@gmail.com>

* * chore(auth): update mockery version to v2.43.2
* chore(main.go): update import path for vm package in agent
* chore(main.go): update import path for vm package in manager
* chore(go.mod): add github.com/google/logger v1.1.1 as a required dependency
* chore(manager_test.go): update import path for vm package in manager
* chore(logging.go): move logging.go to manager/qemu/vm package
* chore(logging_test.go): move logging_test.go to manager/qemu/vm package
* chore(vm_factory.go): rename vm_factory.go to provider.go in manager/qemu/vm/mocks package
* chore(vm.go): move vm.go to manager/qemu/vm package
* chore(vm.go): update import path for vm package in manager
* chore(vm_test.go): move vm_test.go to manager/qemu/vm package
* chore(vsock.go): move vsock.go to manager

Signed-off-by: SammyOina <sammyoina@gmail.com>

* * fix(main.go): change import path for 'github.com/ultravioletrs/cocos/manager/qemu/vm' to 'github.com/ultravioletrs/cocos/manager/vm'
* fix(main.go): change vsock.Dial argument from 'vm.VsockConfigPort' to 'qemu.VsockConfigPort'
* fix(main.go): change import path for 'github.com/ultravioletrs/cocos/manager/qemu' to 'github.com/ultravioletrs/cocos/manager/qemu'

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-07-10 15:33:54 +02:00
Sammy Kerata Oina d1c9834d86 NOISSUE - Fix user role on RPC call (#173)
* * fix(interceptors.go): change role from auth.DataProviderRole to auth.ConsumerRole in AuthenticateUser call

Signed-off-by: SammyOina <sammyoina@gmail.com>

* * test(interceptors_test.go): update role variable values in TestAuthUnaryInterceptor
* - Change role value from auth.DataProviderRole to auth.ConsumerRole in the "authorized result method" test case
* - Change role value from auth.DataProviderRole to auth.ConsumerRole in the "unauthorized result method" test case
* - Change role value from auth.DataProviderRole to auth.ConsumerRole in the "other method" test case

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-07-10 13:42:41 +02:00
Danko Miladinovic 654be60a80 COCOS-132 - Add progress bar for algo and data uploads (#162)
* add progress bar to CLI

* fix error handling

* fix comments errors

* add header

* add wraper for AlgoClient and DataClient

* add compile time check for wrapper structs

* refactor code
2024-07-09 09:35:40 +02:00
Danko Miladinovic 006897a57c COCOS-153 - Add host-data option (#163)
* add host_data option

* add CLI hostdata option and rename platform to backend

* move code for computation hash to a function

* rename getComputationHash to computationHash

* add default for backend information field switch
2024-07-08 15:32:13 +02:00
Sammy Kerata Oina 0e236bf2e8 * feat(hal/linux): add support for wasmtime package (#166)
* fix(hal/linux/Config.in): add source for wasmtime Config.in file

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-07-04 17:04:51 +02:00
Washington Kigani Kamadi ff5bade57d COCOS-145 : Support Other Private/Public Key Pair Algorithms (#147)
* feat: generate multiple key types using cli

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

pass cert to agent

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

fix ci

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

standardize key types

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

marshal public key

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

use openssl format for keys

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* update auth

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

update makefile

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

update agent

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

enhance parsing logic

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* pass key type as flag

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

fix ci

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

return key on successful signature verification

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

fix cli

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

fix ed25519 signing

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

ci

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* refactor code

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* fix(cli/result.go): dry code

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2024-07-03 22:56:18 +02:00
Danko Miladinovic 7c090fd19f NOISSUE - Add CLI option to download ASK and ARK (#149)
* add cli option to download ask and ark

* add ARK and ASK to cert chain of the attestation report

* fix spelling mistake

* add explanation for aTLS testing

* remove commented code
2024-07-03 16:07:19 +02:00
Sammy Kerata Oina 30092c6a45 COCOS-135 - Refactor gRPC auth interceptors and update dependencies (#140)
* Implement gRPC auth interceptors and update dependencies

Introduced gRPC authentication interceptors for unary and stream calls in the agent's API to enhance security. These interceptors verify signatures based on roles before handling requests, rejecting unauthorized ones. The authentication logic was refactored, adding relevant error handling.

Added mocks for authentication interfaces to facilitate testing and replaced direct errors with wrapped ones to provide additional context.

The Makefile now includes a version pin for the Mockery tool, ensuring consistent mock generation across environments.

Updated the gRPC server setup in `grpc.go` to utilize the new interceptors. This change aims to reinforce the API's access control by authenticating users according to their roles and is expected to prevent unauthorized access to sensitive endpoints.

Lastly, dependencies in go.mod and go.sum were updated, most notably the inclusion of testify for improved testing capabilities.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix linting errors

Signed-off-by: SammyOina <sammyoina@gmail.com>

* * chore(Makefile): remove unused variables and update 'mocks' target

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-07-01 21:39:35 +02:00
Sammy Kerata Oina 617b595559 Add Docker support to system configuration (#164)
Added support for Docker in the system configuration file. This includes packages like libseccomp, CA certificates, Docker CLI, Docker Compose, Docker Engine, containerd, and runc.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-07-01 21:37:58 +02:00
Sammy Kerata Oina 4c4161ccfb Fix capitalized Vnc field and increment Vnc on run (#139)
Corrected the struct field 'vnc' to 'Vnc' to match Go's public field naming conventions and to make it exportable. Each QEMU virtual machine's VNC display number now increments upon instantiation to avoid display conflicts.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-06-06 16:34:13 +02:00
Sammy Kerata Oina ff64c10ac3 NOISSUE - Add new key management command to CLI (#136)
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-29 13:09:54 +02:00
Danko Miladinovic 0574abc228 NOISSUE - Add Rust script for fetching platform data (#133)
* add rust program for fetching platform data

* fix new line error

* add CLI options to add the measurement to platform_info.json file

* add documentation for platform info testing

* add explanation for sev-snp-measure

* delete excess space

* fix minor errors

* fix minor errors

* add file permision constant
2024-05-28 18:08:07 +02:00
Sammy Kerata Oina 2ce112cc1b COCOS-103 - User authN and AuthZ using digital signatures (#128)
* Update Go to 1.22 and enhance security features

- Upgraded the Go version in GitHub Actions workflows to 1.22.x for latest features and security patches.
- Added RSA public key field `UserKey` in `Dataset` and `Algorithm` to reinforce data integrity and encryption.
- Refactored `Result` method in `agentService` to use `containsID` for improved readability and potential performance benefits.
- Updated `grpcserver.New` and `internal/server/grpc` invocations to pass `agent.Service` by value in line with recommended Go practices.
- Introduced `grpc.StreamInterceptor` with no args in `Server.Start` which seems to be an initial step for future stream interceptor configuration.

These changes prepare for stronger data security measures, maintain compatibility with the latest Go features, and improve code quality regarding service struct usage. Potential follow-up is needed to configure the stream interceptor and to ensure the new RSA key field is appropriately utilized in data handling.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor auth system and protocol buffers

Enhanced the authentication system by adding context support and an improved user-role model. Implemented robust RSA public key verification for users and a restructured interceptor logic specific to stream types, streamlining the auth process. Updated protocol buffers and associated structures to accommodate user keys as byte slices, aligning with standard cryptographic practice. CLI commands for algorithms and datasets now require a private key file path argument for signing, strengthening security during interactions.

This comprehensive overhaul addresses security and efficiency considerations in the RPC framework and aligns with best practices for key handling. By streamlining and securing the user authentication process, the agent service's reliability is greatly improved, directly impacting the robustness of the entire computation pipeline.

- Refactored auth: added role-based user validation, context handling
- Reworked interceptors: separated stream types, fortified signature checks
- Updated protocol buffers: user public keys as byte slices for standard compatibility
- Enhanced CLI: introduced private key argument, ensuring secure algorithm and dataset submission
- Improved server and SDK contracts to align with auth changes

Related issues:
- Implements user roles and auth context [#103]
- CLI security enhancement for private key management

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Updated PEM decoding for key parsing in CLI and tests

Added `encoding/pem` to decode PEM blocks when parsing private and public keys across CLI commands and test computation scenarios, ensuring compatibility with key files. This enhances robustness in key handling by supporting PEM encoded keys. The update also includes registration of a new Keys command in the CLI.

Refactored code is now compliant with common key formats, addressing potential parsing issues.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix auth signature encoding and improve CLI usage example

The authentication system now decodes base64 strings before verifying signatures to align with the expected format. Additionally, the signature generation now encodes the output in base64, ensuring consistency across the auth process.

The CLI help message for the `result` command is enhanced by providing a usage example, making it more user-friendly and informative.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor containsID to handle dynamic fields

Updated the `containsID` function to accept a field name parameter, enabling dynamic field lookup within the reflection logic. This change facilitates the use of the function for various struct fields, improving code reusability and flexibility.

CLI command 'data' now requires an additional argument for the private key file path, outlined in the usage example update, reinforcing command clarity and user guidance.

Resolves issues with hardcoded field lookups and enhances CLI usability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Remove extraneous newline in key generation log output

A redundant newline after the success message in the key generation command was removed to clean up log output formatting. This change ensures a more consistent and professional appearance of the CLI tool's messages.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Implemented auth service in gRPC startup

Added authentication services to the gRPC server initialization to enforce security measures. The gRPC server's New function now includes an `authSvc` parameter, requiring instantiation of the auth service before starting the server. Failure to create the auth service results in a fatal error, halting the process to avoid running without protection. Tests have been updated to include `nil` values for the auth service parameter to maintain their functionality without authentication.

Refactored `grpcserver.New` to accept the new auth service, and updated the main agent startup logic to create and inject the auth service. Added the auth middleware interceptors to the server options, which ensures that each gRPC call will undergo authentication.

This change is a step towards secure communication, and affected components should now consider the authentication requirement.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor config read logic and update agent setup

Improved the configuration reading in `cmd/agent/main.go` to handle larger payloads by reading data in chunks and checking for EOF, ensuring that all config data is captured even if it exceeds the initial buffer size. Enhanced the `test/manual/agent-config/main.go` to require additional command-line arguments, improving the setup process by explicitly requiring paths for data, algorithm, and public key as well as a boolean for attested TLS. Also updated the hashing method to SHA3 for the algorithm and data files, and included the hash and public keys as part of the agent, dataset, and result consumer configurations. These changes will make the agent setup more robust and provide better integrity checks for the involved files.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor run method to agentService

Moved the run function into agentService for better encapsulation and maintainability. This refactoring includes capturing both stdout and stderr during algorithm execution, enabling more informative debugging through enhanced logging. Consequentially, the run method now references members through the service instance, aligning with object-oriented best practices and improving code coherence.

Resolves issue with insufficient execution details when computations fail.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor computation data handling to use filepaths

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor error logging and ensure consistency

Replaced usage of the standard log package with a custom logger for error reporting to standardize error logging throughout the application. Additionally, introduced graceful shutdown by returning from the main function rather than forcing exit when failing to create auth service, aligning the application's error handling strategy.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor auth initialization and key file handling

Improved the readability and maintainability of the authentication service initialization by adding line breaks for logical separation. Also, standardized key filenames in the CLI key generation by introducing constants, enhancing code clarity and reducing the likelihood of file-naming errors.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor auth verification logic for improved security

Removed an extraneous line in the `verifySignature` function that was not necessary for the signature verification process. This change simplifies the code and improves readability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor payload structures to simplify API

Removed the 'provider', 'id', 'consumer' fields from protocol buffers, gRPC services, and related functions across various files to streamline the data model and align with the new authentication system based on cryptographic verification rather than string identifiers. This results in more efficient data handling and a reduction in unnecessary payload data, while enhancing security by making entity validation strictly cryptographic.

The changes affect agent-SDK interactions, CLI tools, and related services, ensuring only the necessary data (algorithm/data bytes, user keys, and hashes) is transmitted and processed. Consequently, the core computation algorithm and dataset handlers now rely on indexes derived from context to associate data with respective manifest entries, thus maintaining the ability to link to specific computation manifests without relying on explicit IDs in the payload. Additionally, refactored authentication methods now enforce role-based security seamlessly through metadata.

This approach enhances privacy by avoiding transmission of potentially sensitive strings over the network and by ensuring that only internal indices, not globally interpretable identifiers, are used to process computations.

Aligned with the broader architectural goal of simplifying and securing the platform's core services, this change paves the way for upcoming revisions to the authentication scheme that will further consolidate role-based security and improve system integrity.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance CLI security with key paths

Removed the section on running computations from the CLI README as it may no longer be necessary or the functionality has been moved elsewhere. Required private key file paths for algorithm, dataset upload, and result retrieval commands to enhance security. This change associates each action with a specific identity, ensuring secure and traceable operations. Additionally, updated the manual test commands to reflect this new requirement.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix ci

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix fmt

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-28 14:10:13 +02:00
Sammy Kerata Oina aebe01a873 NOISSUE - Updated README with static binary instructions (#130)
Enhanced the manual algorithm testing documentation by including steps to install additional dependencies and guidance on creating static binaries. This ensures wider compatibility and ease of deployment for python programs tested with this framework.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-22 14:40:34 +02:00
Sammy Kerata Oina 4c681569f1 NOISSUE - Add sha3-256 file hashing command to CLI (#129)
Introduced a new command for the CLI to calculate the sha3-256 hash of a given file. The update enhances the utility's cryptography toolkit by allowing users to easily generate and verify file integrity directly through the command-line interface.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-20 16:56:42 +02:00
Sammy Kerata Oina c274521faf Refactor computation data handling to use filepaths (#126)
Changed the internal representation of algorithms and datasets within the service from byte slices to file paths, writing received data directly to temp files. This modification allows for handling potentially large data sets without the need to load them entirely into memory, improving the memory efficiency and scalability of the service. Additionally, it aligns the call signature of external algorithms with the new approach, updating documentation and examples accordingly. Updated the linear regression example for consistency with the new data handling process.

Resolves issues with memory bloat when processing large datasets.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-13 17:16:39 +02:00
Sammy Kerata Oina 226704cf0d COCOS-122 - Enable streaming RPCs for Algo and Data services (#123)
* Enable streaming RPCs for Algo and Data services

Modified the gRPC service definitions for Algo and Data methods to use stream processing, enabling the handling of larger datasets and algorithms without being limited by memory restrictions. This allows client and server to send chunks of data sequentially rather than requiring the entire payload to be loaded into memory at once.

Updated server implementations to accumulate data from multiple chunks, allowing for more efficient processing and communication when dealing with large files. Client implementations have been adjusted to segment and send data in a streaming fashion.

Removed previously existing synchronous client code as it became redundant with the new streaming approach, streamlining the client's communication patterns with the gRPC backend.

This change allows for better resource management, especially in systems with constraints on memory, improving overall scalability and performance of the data and algorithm processing pipeline.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor algorithm ID check logic

Simplify the algorithm validation logic in the agent service by replacing the previous containment check with direct ID comparison. This change streamlines the error handling for undeclared algorithms and hash mismatches, while also ensuring clear and direct provider validation. The modifications enhance the readability and maintainability of the code without altering functionality.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Updated README to build single-file executable with PyInstaller

Modified the PyInstaller command in the manual testing README to bundle the linear regression script into a single executable file. This simplifies distribution and execution of the script by eliminating the need for multiple dependency files.

Ref: Optimization of deployment process
Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-13 13:14:50 +02:00
Danko Miladinovic 479598a663 COCOS-116 -Separated networking setup from agent start up script (#118)
* added network setup script

* removed fixed name enp0s2 from code

* fixed small typo

* changed the capitalization of a letter
2024-05-01 17:03:28 +02:00
Sammy Kerata Oina 8d082567d7 COCOS-101 - Switch to self-contained algorithms as executables (#119)
* Switch to self-contained algorithms as executables

Transitioned from using Python scripts to self-contained binary executables for running algorithms, improving modularity and reducing dependencies. This change removes the reliance on a Python environment, as evident by the removal of Python setup and packages from the build configuration. The service now creates temporary executable files for algorithm runs, handling all permissions and cleanup, enhancing security and maintaining clean execution states. A warning is logged if computation fails, aiding in debugging. Additionally, updated manual tests to reflect these changes in the agent's handling of algorithms.

Refactors:
- Removed Python runtime const since it's no longer needed.
- Updated documentation and test commands to reflect the change from .py to .bin for algorithm files.

Build config:
- Removed Python and pip packages to reduce the build size and complexity.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update agent service.go file with new constants and file permission

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refine singular usage of 'algorithm' across modules

Standardized terminology throughout the project to refer to 'algorithm' in the singular form rather than plural. Streamlined various documentations, string constants, function names, and variable names to bring cohesiveness and eliminate ambiguity when handling algorithms across README files, CLI interfaces, and internal API representations.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix state names and indices in state_string.go

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-04-29 15:48:17 +02:00
Sammy Kerata Oina 4b5000d107 NOISSUE - Allow termination of manager (#95)
* Implement manager client heartbeat mechanism

Introduced a heartbeat system in the manager service to maintain an active link with client agents. The updates involve sending periodic heartbeat signals controlled via the new environment variable `MANAGER_HEARTBEAT_INTERVAL`, defaulting to 1 second. The protobuf definitions, service interfaces, and server logic have been updated accordingly to handle the incoming signals. This change allows better tracking of active client connections and could be used for features like auto-reconnect or resource cleanup for lost connections in the future.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance gRPC server to handle client termination

Introduce the capability for the gRPC server in the manager module to process termination requests from clients. Server and client proto buffers have been updated to include a Termination message, allowing clients to signal the server to stop processing. Handling for communication and termination flows involving context cancellation have been added across server implementation, manager client, and main execution logic to properly respond to termination signals. This ensures clean shutdowns and resource deallocation when a client requests termination, improving the robustness of the system.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor gRPC server context handling

Removed the redundant storage of context in grpcServer struct and modified the NewServer constructor to exclude the context parameter. Updated the Process method to derive context from the stream directly, ensuring better context scoping and adherence to proper gRPC patterns. This change simplifies the server's context management and ensures contexts are more accurately associated with their corresponding streams, which can improve debugging and request cancellation behavior.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update protoc to v4.25.3 and remove heartbeat system

Upgraded protoc version to 4.25.3, ensuring compatibility with the latest protocol buffer features and potential performance improvements. Additionally, removed the entire heartbeat system, involving both its gRPC server implementation and associated client-side logic. This elimination suggests a shift in the service health-check strategy, potentially to a more modern or efficient model.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Removed heartbeat functionality for manager service

The heartbeat feature and its related configuration have been removed across the application. This includes the removal of the Heartbeat method from the Service interface, the associated interval setting in the config structure, and related environment variable documentation. The change implies an architectural shift towards alternative methods of service availability checking or potentially a move to stateless service design. The cleanup removes unnecessary code, thus simplifying the overall service logic.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Introduce Terminate message and streamline gRPC server logic

- Added a new `Terminate` protobuf message to handle termination requests more explicitly.
- Removed redundant context cancels in gRPC server stream processing to rely on context propagation.
- Updated channel usage to match the altered stream message types; this aligns message handling with the updated protobuf definitions.
- Passed client authentication info to service run function to enhance the security during service execution.
- Utilized stream's context directly in error groups for improved clarity and error handling scope.
- Refactored message descriptors in protobuf to maintain correct ordering after introducing new message types.

These changes enhance the codebase's maintainability, simplify the logic, and align with the updated messaging protocol.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update agent configuration in main.go

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-04-29 14:08:42 +02:00
Sammy Kerata Oina 64f7e7f7fd NOISSUE - Refactor single algorithm processing (#117)
* Refactor single algorithm processing

Simplified the agent service's algorithm handling logic to process a single algorithm instead of multiple. This change:
- Removed the `Algorithms` type and associated stringer implementation.
- Updated the state machine and service logic to expect a singular algorithm, aligning the agent's internal state transitions with the new model.
- Adjusted the manager service and computations test server to mirror these changes in their respective payload structures, ensuring API and test consistency.
- Altered README files to reflect the simplified interaction model and removed outdated descriptions.
- Reverted the protoc-gen-go version used for generating protobuf files to maintain compatibility with the rest of the codebase.

The single-algorithm approach streamlines the computation running process, reducing complexity and potential error conditions. It directly impacts how external services will construct and send computation requests.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update protoc-gen-go version to v1.33.0

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor variable name in computations.go and grpc.go

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-04-17 13:34:54 +02:00
Danko Miladinovic 3a14896555 NOISSUE - Attested TLS (#99)
* added initial code for attested TLS

* added client validation and verification

* fixed bugs for attested TLS

* updated README for manual testing

* fixed CI errors

* removed SNP pollicy from agent config

* added attested TLS config param to AgentConfig

* generated manager.pb.go for protoc v25.2

* updated proto-gen-go version on CI

* generated agent.pb.go to match newest proto gen version

* define errors for error handling

* fixed comments
2024-04-16 15:19:19 +02:00
Drasko DRASKOVIC b57bafd310 Fix usage (#114)
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2024-04-16 00:42:47 +02:00
Drasko DRASKOVIC 9e6fb31d90 Fix links and add features (#113)
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2024-04-15 22:26:41 +02:00
Sammy Kerata Oina 2b760ec207 NOISSUE - Streamline client identification (#105)
* Update protoc to v4.25.3 and streamline client identification

Protocol Buffer compiler `protoc` has been upgraded to version 4.25.3. This update unifies the version across multiple generated files to ensure compatibility and take advantage of any bug fixes and performance improvements in the new release.

Additionally, the client identification process has been refined. The redundant `WhoAmI` message and the corresponding checks have been removed from the gRPC server implementation. Clients are now identified via their address from the `Process` stream's context as soon as a connection is established, simplifying the code and potentially reducing handshake time.

This change sets the foundation for a leaner communication protocol between manager and agents, and could contribute to lower latencies in client-server interactions.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix goroutine bug in grpc server

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update PROTOC_VERSION to 25.3

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor server reference in readme

Updated the deployment instructions in the README to point to the correct server location after consolidating test server documentation. Removed obsolete `manager-server` module and associated main entry point to align with the new architecture and streamline workflows.

Resolves issues with outdated links and cluttered repository structure.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-04-15 22:08:37 +02:00
Sammy Kerata Oina 2e612a2867 COCOS-108 - Enhanced state machine Synchronization (#112)
* Enhanced state machine synchronization

Introduced a wait group to the state machine to manage go-routine synchronization effectively. The Start method now ensures that the state machine's execution is tracked, preventing potential early termination of go-routines during state transitions. The test suite is updated to wait for the go-routine to complete before proceeding, ensuring more reliable test execution.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Remove unnecessary code and fix test case in agent package

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-04-15 18:55:58 +02:00
Drasko DRASKOVIC f2f57ac413 Fix license (#107)
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2024-04-14 18:05:16 +02:00
Drasko Draskovic d50a6fcfda Fix releases link
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2024-04-14 17:55:29 +02:00
Drasko DRASKOVIC d323fd3b7e Fix docs link (#106)
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2024-04-14 17:49:10 +02:00
Filip Bugarski 35c2a9e92a Update cocos documentation (#89)
* Update cocos documentation

Signed-off-by: fbugarski <filipbugarski@gmail.com>

* inserted go run commands into separate lines

* updated syntax higlighting to bash

---------

Signed-off-by: fbugarski <filipbugarski@gmail.com>
Co-authored-by: Danko Miladinovic <danko@etf.rs>
2024-04-13 18:33:10 +02:00
Sammy Kerata Oina 5c406743e0 NOISSUE - Add attesatation validation/verification options (#87)
* Standardize attestation report data size to 64 bytes

Redefined the report data for attestation requests from a variable byte slice to a fixed 64-byte array across various critical components. This change enforces a standardized data length, simplifying validation logic and ensuring consistency when handling the attestation data for both the server and client end-points, logging functionalities, and the CLI tool. Updated attestation data handling in the SDK to accommodate the new fixed-length constraint.

By strictly adhering to the 64-byte requirement, the modifications promote robust input checking and prevent potential issues related to dynamic data length processing.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor attestation validation logic

Streamlined attestation validation in the CLI with a new configuration parsing approach. Introduced a retry mechanism with timeout and backoff for HTTPS getters that facilitate remote trust validation. Clarified expected data lengths in proto comments across attestation and manager services, ensuring consistency and correctness of the cryptographic data fields.

This modification enhances maintainability by replacing a verbose struct with a configurable object model, thus simplifying parameter handling. Robustness is improved through failure-retrieval strategies during remote validation, and additional in-code documentation specifies cryptographic constraints for critical data elements.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add new attestation validation flags

Enhanced attestation validation in the CLI with additional flag support for FAMILY_ID, IMAGE_ID, REPORT_ID, REPORT_ID_MA, MEASUREMENT, CHIP_ID, and TCB-related fields. This update enables fine-grained control over attestation report criteria, ensuring stricter validation conforming to protocol requirements. Adjusted the default value for MinimumGuestSvn to align with the new policy specifications.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance attestation policy configuration

Introduced several new configuration flags to the CLI's attestation policy, including options for minimum AMD-SP firmware builds, revocation list checking, and network restrictions. Expanded trust criteria parameters with the support for trusted author and identity key hashes. The default VMPL setting is now explicitly initialized. These changes improve security controls and offer more detailed attestation verification settings.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Simplify attestation validation process

Refactored the attestation CLI command to improve usability by removing the need for explicitly provided report data as an argument. Default values for configuration parameters have been centralized into constants for maintainability. Additionally, integrated data parsing functions to streamline the validation checks with robust error handling for cases such as invalid hex strings and file I/O issues.

This enhancement reduces user error and the complexity of entering attestation data by parsing relevant information from within the environment, all while maintaining the same security standards.

The process of marking flags as required for CLI commands has been corrected to prevent runtime errors and improve command reliability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refined attestation validation logic

Enhanced attestation command handling by removing the hardcoded report data length and replacing it with the standard SHA-512 hash size, ensuring dynamic compatibility with hash lengths. Introduced size constants to improve code readability and enforce explicit length checks on attestation input data, raising errors when requirements are unmet. This preventative measure aims to avert potential runtime errors stemming from unexpected data sizes. The change also simplifies the minimum guest SVN description for clarity.

References to removed unnecessary whitespace maintain code cleanliness. These adjustments culminate in a more robust and maintainable attestation process, in accordance with the best practices for secure handling of cryptographic data and compliance with AMD's ABI format specifications.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhanced attestation documentation and command syntax

Updated CLI documentation to include detailed descriptions of the attestation retrieval and validation process, along with a comprehensive list of new flags for the validation command, providing users with extensive control over the attestation verification process. Updated command syntax with flags for specifying report data, improving command clarity and consistency. These changes ensure better user guidance and offer a more robust attestation handling experience.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enforce report data size validation uniformly

Refactored report data size validation to ensure it is performed consistently across the system. Removed the hard-coded length check from attestation request validation in favor of a centralized verification based on the sha512 digest size. Also, eliminated fallback mechanism to the SHA-512 hash of report data and made the length requirement explicit, leading to immediate failure if not met. This harmonizes the error handling logic, reinforces data integrity checks, and simplifies debugging by removing ambiguous length correction behavior. Adjusted related validation functions to use a common utility, enhancing maintainability.

Fix spelling errors in log messages for attestation validation commands to improve clarity in output.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Initialize attestation config with defaults

Enhanced the attestation command initialization by setting default values for the `Config` struct to prevent nil pointer exceptions and potentially streamline configuration handling. Moreover, corrected a flag declaration by removing shorthand.P support for 'permit_provisional_software', aligning it with other flags for consistency. This change should improve stability and user experience while configuring attestation policy.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* nil check prior

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix typos

Signed-off-by: SammyOina <sammyoina@gmail.com>

* use file for attestation report

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refine attestation validation with default policy values

Adjusted default values and constants related to the attestation configuration for enhanced clarity and consistency. Changed the default minimum guest SVN value to align with new policy requirements and introduced a default guest policy constant. Additionally, standardized byte array placeholders for attestation validation fields to prevent potential issues with uninitialized bytes and ensure robust input validation checks. These changes promote more intuitive and secure attestation validation procedures.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refine attestation defaults and flag requirements

Updated the default guest policy to use hexadecimal notation for clarity and added a default minimum version for the firmware API version flag. Made 'chip_id', 'measurement', 'report_id', and 'report_id_ma' flags mandatory to ensure critical attestation data is provided by the user, enhancing validation robustness. Added a check for 'measurement' field length for consistency in input validation.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Optimize attestation file format

Standardized attestation file format across the codebase by moving the 'attestationFilePath' constant from the function scope to a higher scope within the package. Switched the file extension from .txt to .bin to reflect binary data storage, improving consistency and potential performance benefits from binary file handling.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Optimize default values for attestation validation

Removed unnecessary empty initializers and replaced hardcoded empty values with nil for optional fields in attestation validation flags. Introduced a constant defaultReportIdMa for the REPORT_ID_MA field to provide a meaningful default instead of an empty one. Streamlined the command setup by removing the requirement flags for fields that are now optional and unchecked if unset, reducing burdens on the user for providing unnecessary input.

The change simplifies the command's usage and eliminates the need for users to specify values for certain fields that can be optional, improving the command's flexibility and user experience. Additionally, setting a specific default for REPORT_ID_MA ensures clarity in cases where this is not provided by the user.

This update enhances the maintainability and usability of the attestation validation functionality.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor hardcoded attestation report size

Introduced a constant `size64` to replace the previously hardcoded array size of `64` used across various Attestation methods. This change enhances code maintainability and readability by centralizing the size definition, making it easier to update in the future if necessary.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Updated protocol buffer dependencies and cleaned CLI flags

Protobuf compiler versions were updated to v4.25.2 across various RPC service definitions for agent and manager packages, ensuring compatibility with the latest features and fixes. In the CLI attestation code, unnecessary flags (`DisallowNetwork` and `PermitProvisionalSoftware`) were removed to streamline user options, reflecting a more secure and user-focused configuration interface. Added an input validation check in `attestation.go` for CA bundle presence when a product name is set, enhancing the reliability of attestation validation. Also rectified a bug in `grpc.go` by fixing an incorrect variable assignment for reading the key file content, thereby preventing potential TLS-related errors.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Updated protoc version and adjusted proto file paths

Bumped the protocol compiler version to 25.2 to align with our dependency updates and ensure compatibility. Additionally, revised the proto file paths under the continuous integration setup, moving from 'manager/' to 'pkg/manager/' to reflect the recent directory restructuring. The changes ensure that proto file checks and comparisons are conducted in the correct file locations, preventing potential build and sync issues in future developments.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance hash size consistency in gRPC attestation

Aligned the hard-coded byte array size for `ReportData` with `sha512.Size` constant to ensure consistency and maintainability in gRPC attestation requests and responses. This change mitigates the risk of future errors if the SHA512 hash size standard is altered.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Optimize agent-config listener loop

Removed the goroutine wrapping the listener loop in 'agent-config' manual test to streamline and simplify the connection handling logic. This change ensures the loop runs on the main goroutine, improving the readability and maintainability of the code.

Refactoring assists in avoiding potential race conditions and makes the server's flow easier to understand for future debugging and development.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor report data size checks to use constant

Centralize the report data size definition by replacing various size checks and array declarations with a constant `ReportDataSize`. This streamlines code maintenance and ensures consistency across all instances where report data size is validated or used. The changes remove direct references to the hash function output size, decoupling the report data size from the hash function's characteristics and allowing easier updates if the data size requirement changes.

These modifications bolster the codebase's flexibility for potential adjustments in security protocols or data handling specifications.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor config JSON handling in attestation CLI

Introduced an `exampleJSONConfig` constant containing a sample configuration to improve code readability and maintainability. Previously, the example JSON configuration for the attestation CLI was an inline string, making the code cluttered and less maintainable. With this change, the example configuration is now stored as a constant, resulting in cleaner command flag setup and enhanced clarity. This constant is used in the command flag description to guide users when providing their custom configuration. Additionally, refactored the command flags related to the configuration by aligning and sorting them for better code organization.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-04-09 10:33:26 +02:00
Smith Jilks b6f116474d Cocos-91 - Update Makefile and cocos documentation (#93)
* Update Makefile

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update cocos documentation

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update cocos documentation

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update Makefile

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update Makefile

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update Makefile

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update cocos documentation

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Remove install rule from Makefile

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update agent README.md

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update agent README.md

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Update README.md files

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Remove INSTALL_DIR from Makefile

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

* Fix comments in README.md files

Signed-off-by: Jilks Smith <smithjilks@gmail.com>

---------

Signed-off-by: Jilks Smith <smithjilks@gmail.com>
2024-04-08 15:59:38 +02:00
Sammy Kerata Oina 3e3e02f132 Enhance security with SHA-3 and command-line args (#97)
Updated the hashing algorithm from SHA-256 to SHA-3 for improved security and reliability in the computation service. Modified the service to accept dataset and algorithm paths as command-line arguments, enhancing flexibility and user control. Increased robustness of the service's error handling by incorporating file read validations and updated usage instructions in both the README and the program's startup error messages.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-04-06 23:45:13 +02:00
Danko Miladinovic 4b38ac09dd NOISSUE - Added SEV-SNP support (#98)
* added SEV-SNP support

* updated documentation for SEV-SNP

* changed variable names to adhere to the GO naming convention

* changed the variable name from V-sock to Vsock
2024-03-22 10:55:09 +01:00
Danko Miladinovic 1a322f8ecb NOISSUE - Fix network interface problem for agent (#96)
* Added bus/addr for pcie net

* Remove SCSI parameters

* Remove bus parameter

* Removed blank lines
2024-03-15 16:37:41 +01:00
Sammy Kerata Oina 8c2caec395 NOISSUE - Use standard GitHub templates for issues and PRs (#92)
Implemented new GitHub templates to streamline the contribution process. Bug report and feature request templates guide users in creating detailed and actionable issues, while a universal pull request template ensures systematically structured submissions. These additions facilitate more efficient reviews, and prioritize issues effectively. Additionally, configured the repository to disallow blank issues, encouraging use of the templates for better project management.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-03-13 13:06:51 +01:00
Sammy Kerata Oina de60358c02 NOISSUE - Enforce 32-byte length check for hashes (#86)
* Enforce 32-byte length check for hashes

Introduced a validation step in the manager service to ensure both algorithm and dataset hashes are of byte length 32 before provisioning VMs, preventing runtime errors due to invalid hash lengths. The test manager-server now generates a valid 32-byte hash using SHA256 for mock data to align with the new validation requirement.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* use equal comparison

Signed-off-by: SammyOina <sammyoina@gmail.com>

* use a constant

Signed-off-by: SammyOina <sammyoina@gmail.com>

* use constants

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-24 22:22:25 +01:00
Sammy Kerata Oina 997fb3bf48 COCOS-83 - Add hash verification for datasets and algorithms (#84)
* Add hash verification for datasets and algorithms

Enhanced data integrity checking by incorporating hash fields in Dataset and Algorithm structures, and modified the corresponding service logic to validate these hashes during processing. The update includes SHA-3 for hash computation, replacing the former SHA-256 usage, and ensures the provided data matches the expected hash from the manifest to prevent processing malformed or corrupted data.

- Introduce `Hash` field to both Dataset and Algorithm structs to store the expected hash value.
- Implement SHA-3 hashing within service methods that process the data, ensuring consistency with newly added `Hash` fields.
- Add error handling for hash mismatches, preventing further processing and alerting to potential data integrity issues.
- Update Protocol Buffers serialization to accommodate the new hash fields for gRPC communication.
- Modify manager service's Run method to pass the hash information when creating agent configurations.

Go module dependencies were updated to include the new SHA-3 package and upgrade Go version to 1.21.6 for improved stability and compatibility.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Remove identifiers from protobuf and related code

The protobuf definitions and related service handling code have been revised to drop specific identifier fields (`AlgorithmID` and `DatasetID`) to simplify API responses and internal function signatures. These removals streamline the overall data flow between components, reduce unnecessary data transmission, and lead to an aligned server-client expectation where identifiers are no longer a part of the response payload. Consequently, these changes simplify the logic within various functions and client commands, reinforcing encapsulation by ensuring that internal identifiers do not need to be managed or exposed unnecessarily.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-19 15:31:35 +01:00
Sammy Kerata Oina 722b463b6a NOISSUE - Use a single listener for logs and events (#82)
* add handler

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor gRPC and Protobuf integration for manager service

- Shifted Protobuf message definitions to a separate package `pkg/manager`.
- Updated references throughout the codebase to import and use the new package for gRPC service definitions.
- Enhanced AgentLog message with additional fields `level` and `timestamp`.
- Removed direct dependencies on old Protobuf-generated types in favor of the new package.
- Deleted obsolete Protobuf-generated files as they are now superseded by the new `pkg/manager`.
- Streamlined event publishing and gRPC handling in the manager service to use the updated Protobuf messages.

This refactoring improves modularity by centralizing Protobuf message definitions and decouples internal representation from the gRPC interface, aligning with best practices for microservice architecture. Additionally, the enriched logging structure paves the way for more detailed and fine-grained log analysis.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor vsock event/log handling and config

Streamlined event and log services in the manager by moving vsock listening functions out of `managerService` initialization and into dedicated `RetrieveAgentEventsLogs` methods. This change decouples the manager service creation from the actual start of log listening, adding clarity and flexibility in service management. Also moved logging middleware invocation outside of network handling loops to avoid unnecessary overhead.

Additionally, the agent's vsock port configuration is now dynamically passed to the `New` function in the `events` package instead of relying on a hardcoded constant, allowing for greater configurability and testability.

Finally, updated message structures for event and log sending to conform with the `ClientStreamMessage` definitions. These modifications should improve parsing and handling consistency and prepare our system for future enhancements related to inter-process communication.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix linting errors

Signed-off-by: SammyOina <sammyoina@gmail.com>

* correct path to generated files

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix comments

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove uneccessary comments

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-15 17:02:05 +01:00
Sammy Kerata Oina 938dd6cb78 NOISSUE - mTLS support across services (#71)
* Implemented mTLS support across services

Extended gRPC configuration to support mutual TLS (mTLS) in agent and manager components for enhanced security. This includes the loading of Certificate Authority (CA) certificates, server, and client certificates, and keys. Updated README documentation to reflect the new environment variables required for mTLS configuration. Additionally, streamlined secure gRPC client connection setup and logging messages to indicate whether a service is running with TLS, mTLS, or without TLS.

The change ensures secure communication between services by verifying both client and server identities, thus addressing potential security concerns in network-level interactions.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance agent cert handling and update copyright

- Implement function to create certificate files for the agent configuration dynamically, ensuring file paths are updated to reflect newly created files. This improves the agent's setup process by automating the certificate handling.
- Update copyright clause to reflect the new owning entity, Ultraviolet, affirming correct attribution and compliance with legal requirements.
- Refactor gRPC client connection code to remove redundant package alias, streamlining the codebase and improving readability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor cert loading with fallbacks

Removed redundant certificate file creation logic in the agent module and introduced a more robust loading mechanism in the gRPC server module to support direct byte content aside from file paths. This change simplifies the initial setup process for the agent by removing the need to create certificate files preemptively, thereby streamlining deployment in environments with varying filesystem access. It supports using certificate contents directly, enhancing compatibility with in-memory configurations or environments where file storage may not be ideal.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-08 10:07:51 +01:00
Sammy Kerata Oina e86860b9ea NOISSUE - Enhanced gRPC server concurrency with errgroup (#81)
* Enhanced gRPC server concurrency with errgroup

Refactored the gRPC server's `Process` method to handle incoming messages and service execution concurrently using an `errgroup.Group`. Added a context to the `grpcServer` struct and propagated it to support graceful shutdowns and better request handling. Changed the `Service` interface's `Run` method to use a channel for transmitting `ComputationRunReq`, facilitating non-blocking operation and streamlining the request-response cycle.

Asynchronous handling of stream requests with `errgroup` allows simultaneous processing and error management, improving the server's responsiveness and fault tolerance. This update paves the way for more robust service implementations that require concurrent operations.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove trailing space

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-08 10:05:41 +01:00
Sammy Kerata Oina e2fb7ea88d NOISSUE - Add test server (#80)
* add test server

Signed-off-by: SammyOina <sammyoina@gmail.com>

* clean up and update docs

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update docs

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-07 16:16:27 +01:00
Sammy Kerata Oina 6d0100c096 COCOS-73 - Remove server (#76)
* Refactor GRPC manager service and client

The manager service and client have been restructured for stream communication, facilitating real-time agent events, logs, and run responses. The `Run` RPC is replaced by the `Process` stream RPC, enabling bidirectional streaming between clients and the manager service. This allows continuous interchange of different message types including `WhoAmIRequest`, `AgentLog`, `AgentEvent`, and `RunResponse`.

Several message types have been adjusted and new fields introduced, like `AgentPort` in `RunResponse` and various agent-config attributes including CA files and instance IDs, to support TLS client authentication and distinguish between agent instances.

We've also incorporated `google.protobuf.Timestamp` in `AgentEvent` for precise event logging. The client code reflects these modifications with updated method calls and stream handling logic for ongoing communication. Moreover, the updates necessitate corresponding changes throughout service, grpc, and sdk layers to interoperate with the new streaming approach.

The transition to streaming paves the way for a more interactive, flexible communication system that can accommodate future expansion and real-time monitoring features.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add computation id to run response

Signed-off-by: SammyOina <sammyoina@gmail.com>

* rename request

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-07 15:32:35 +01:00
Sammy Kerata Oina 8975e28437 COCOS-77 - Stream agent logs and events over gRPC (#78)
* Refactor GRPC manager service and client

The manager service and client have been restructured for stream communication, facilitating real-time agent events, logs, and run responses. The `Run` RPC is replaced by the `Process` stream RPC, enabling bidirectional streaming between clients and the manager service. This allows continuous interchange of different message types including `WhoAmIRequest`, `AgentLog`, `AgentEvent`, and `RunResponse`.

Several message types have been adjusted and new fields introduced, like `AgentPort` in `RunResponse` and various agent-config attributes including CA files and instance IDs, to support TLS client authentication and distinguish between agent instances.

We've also incorporated `google.protobuf.Timestamp` in `AgentEvent` for precise event logging. The client code reflects these modifications with updated method calls and stream handling logic for ongoing communication. Moreover, the updates necessitate corresponding changes throughout service, grpc, and sdk layers to interoperate with the new streaming approach.

The transition to streaming paves the way for a more interactive, flexible communication system that can accommodate future expansion and real-time monitoring features.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update GitHub Actions to Latest Versions

Upgraded GitHub Actions 'checkout' to version 4 and 'setup-go' to version 5 across various workflow files to leverage the latest features and improvements for better performance and reliability. This also ensures compatibility with Go version 1.21.x which is specified in the workflows.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor event handling and logging

Reworked event and log processing to use channels instead of direct HTTP calls. Removed obsolete events package and consolidated event structures, leading to cleaner and more maintainable code. Updated agent events to use channels, enhanced error handling in log forwarding, and simplified manager `New` function signature to accept an event channel directly.

- Removed `events` and `agentevents` packages to reduce complexity.
- Replaced direct event server communication with internal channel usage.
- Introduced `AgentEvent` struct in events.go for standardized event objects.
- Adapted `managerService` to dispatch events and logs through channels.
- Streamlined manager construction by removing the now-unnecessary event service and host IP parameters.

This change results in a more robust and easier to extend event and log management system within the agent-manager interaction.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix ci

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove unused code

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add comments

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-07 12:08:39 +01:00
Sammy Kerata Oina 3e2be03047 NOISSUE - Attest validation and verification (#74)
* Expand CLI with Manager Service and Enhanced Attestation Commands

The CLI has been updated to interact not only with the Agent service but also with the Manager service, reflecting an expanded scope of operations. As part of this update, 'manager' subcommands have now been incorporated, ensuring users can seamlessly initiate computations through the manager service using the CLI.

Furthermore, attestation functionality has greatly improved. In addition to retrieving attestations from the agent, users can now validate and verify them directly from the CLI, bolstering security measures. New subcommands under 'attestation' provide clear and specific actions for attestation management.

Additionally, the command structure has been refactored to categorize commands under 'agent' or 'manager' prefixes, promoting better command organization and a more intuitive user experience.

These enhancements are part of ongoing efforts to provide robust and secure tools for service interaction and computational task management in distributed networks.

Relevant documentation adjustments and example usage have been updated in README files to align with these functional improvements.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* capitalize comments

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-07 12:02:41 +01:00
Sammy Kerata Oina 046b549079 Remove HTTP server support and streamline config (#75)
The HTTP server-related code, documentation, and configurations have been removed as part of a shift towards prioritizing gRPC for service communication. This update includes deletions of HTTP host and port configs across various components, the manager HTTP API alongside its Swagger definition, and the removal of related scaffolding and utility code. This change simplifies the overall architecture and eliminates redundant HTTP support, focusing on optimizing gRPC performance and security features.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-07 11:03:46 +01:00
Sammy Kerata Oina 6f42135ec0 NOISSUE - Fetch attestation report (#66)
* Improve attestation and update deps

Enhanced the attestation process to require additional report data and a variable privacy level, catering to varying security requirements. The `Run` method's context parameter was removed to simplify invocation following API alignment. Dependencies got updated, notably adding `github.com/google/go-sev-guest` for attestation, and some were indirectly bumped. These updates ensure the agent service leverages the latest available features and improvements for security and stability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Extend AttestationRequest with data fields

Enhanced the AttestationRequest message structure by introducing 'report_data' and 'vpml' fields to encapsulate additional attestation context. Adjusted service and middleware layers to incorporate these new parameters, enabling richer attestation workflows. The GRPC endpoint, logging, and metrics layers have been updated to handle the additional data, catering to more secure and nuanced attestation use cases.

Refactors Attestation command in CLI to accept report data argument and include an optional 'vpml' flag for specifying virtual machine privilege level. This change empowers users to request attestation with specific context directly from the CLI.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor attestation to remove vpml dependency

Removed the `vpml` field from the AttestationRequest protobuf, and all associated handling code, streamlining the attestation process. Code adjustments were made in gRPC client, server, and service layers to ensure proper request construction and validation that focuses only on report data. This change simplifies the attestation API and its usage, reducing error risk and improving maintainability. It reflects a decision to decouple the protocol from specific vendor measurement levels, potentially due to a shift towards a more generalized attestation procedure.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance CLI usability and streamline SDK attestation

- Added a usage example to the CLI attestation command for better user guidance.
- Simplified the SDK attestation function by removing an unused parameter and directly populating the request structure with report data to align function parameters with actual usage requirements.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-02-06 17:55:42 +01:00
Sammy Kerata Oina e5c8243f5f COCOS-62 - Add EOS images to release (#72)
* Add build and release GitHub workflow for tagged commits

Introduced a new GitHub Actions workflow to automate building and releasing for tagged commits. The workflow checks out the required repositories, builds the project using Buildroot configurations, and creates a release with the resultant kernel and rootfs artifacts. This streamlines the release process, ensuring consistent and reproducible builds.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update GitHub Actions to Latest Versions

Upgraded GitHub Actions in the workflow to latest or more recent versions to leverage improvements and maintain compatibility. This includes using version 4 for checkout steps and switching to 'latest' for release creation and asset upload actions, ensuring the use of up-to-date features and potential bug fixes.

Ref: Internal upgrade protocol for CI/CD consistency.
Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix workflow

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Optimized checkout step in CI workflow

Ensured the checked-out repository's reference matches the current workflow's triggering reference. This alignment can result in more predictable builds, especially when the workflow is triggered by pushes to branches other than the default or by tagged commits.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* test repo

Signed-off-by: SammyOina <sammyoina@gmail.com>

* test modified module

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update and upgrade Ubuntu in GitHub actions workflow

Ensure the GitHub actions runner has the latest Ubuntu updates before checking out the repository. This change significantly reduces the chances of encountering bugs or security vulnerabilities stemming from outdated packages, creating a more reliable and secure CI/CD environment.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* restore repo

Signed-off-by: SammyOina <sammyoina@gmail.com>

* install golang

Signed-off-by: SammyOina <sammyoina@gmail.com>

* create tags only for release

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-31 00:08:01 +01:00
Sammy Kerata Oina 8be6129889 NOISSUE - Fix bug on receiving logs and events, update deprecated methods (#68)
* Ensure graceful shutdown and improve connection handling

Refined the network connection handling in both agent events and logs to continuously process incoming data in a loop, enabling the services to handle more than a single message per connection. Additionally, instituted a deferred close operation for the event service to guarantee resources are cleanly released upon the application's termination.

Resolves potential resource leakage and enhances log processing efficiency.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Improve robustness in event and log handling

Altered handling in agent event and log services to continue processing incoming data rather than returning early upon encountering errors. This ensures that a single erroneous data point does not prematurely halt the processing loop, improving the robustness and reliability of the services. Potential errors are now reported and logged, yet the system remains operational to handle subsequent data.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Upgrade gRPC telemetry to use StatsHandler

Switched from using gRPC's UnaryInterceptor for telemetry to the more comprehensive StatsHandler provided by otelgrpc. This enhances telemetry collection by allowing the capture of a wider range of RPC stats, leading to improved monitoring and observability of the gRPC server.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* use constants

Signed-off-by: sammy <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
Signed-off-by: sammy <sammyoina@gmail.com>
2024-01-25 15:30:27 +01:00
Sammy Kerata Oina f2567830b3 NOISSUE - Remove docker (#67)
* Remove Docker-related build functionality

Removed the Docker build scripts, Dockerfile, and docker-compose definitions from the Makefile and relevant directories. Updated documentation by stripping out references to building and running Docker containers. This change likely reflects a shift towards a different deployment strategy or a move away from Docker as a dependency for builds and runtime. This simplification could lead to less complexity in the build process and reduce maintenance overhead associated with Docker configurations.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Removed Docker image build and publish workflow

The workflow for building and publishing a Docker image on pushes to the main branch has been removed. This step likely reflects a change in deployment strategy or a move to a different CI/CD pipeline. Continuous deployment might now be managed by another service or process, adhering to updated operational requirements or infrastructure changes.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-25 15:29:16 +01:00
Sammy Kerata Oina 55c19ca6b5 NOISSUE - Refactor SDK and CLI (#69)
* Refactor CLI commands to use CLI struct

Introduced a CLI struct to encapsulate dependencies such as agentSDK and managerSDK previously passed to command constructors. This shift towards a more object-oriented approach simplifies command creation and enhances code readability. CLI-related commands are now methods on the CLI struct, accessing shared services directly, which streamlines the setup of CLI command hierarchies and reduces the need to pass dependencies around.

Removed the OpenAPI specification file as it was likely deemed obsolete or no longer necessary to be included with the CLI binary, suggesting a possible shift in how the API is managed or documented.

Lastly, the main application now sets up two distinct command groups for 'agent' and 'manager', with respective commands organized under them, making the CLI tool's structure more intuitive for users.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix ci

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-25 15:07:33 +01:00
Sammy Kerata Oina 0c77167be1 COCOS-56 - Pass agent computation events to manager (#63)
* Simplify event handling and config

Streamlined event service interface by consolidating `SendEvent` and introducing `SendRaw`. Removed `notification_server_url` and `instance_id` parameters from several event publication calls to leverage centralized event construction. This change not only cleans up redundancy in event-related code but also simplifies the configuration data flow across the system, making it easier to manage and less error-prone. Uniform event generation now improves consistency and maintainability.

Refactored configuration management in the agent and manager services. Removed notifications URL from the agent configuration, relying on a simplification that assumes a single source of events. Updated Manager Port to VsockConfigPort for clarity and consistency across vsock communication.

These modifications should facilitate easier integration and extension of event and configuration systems in the future.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor error handling in agent event forwarding

Introduced context and error channel handling to the agent event forwarding process. The logger now warns on errors during forward operations asynchronously, allowing for non-blocking error reporting. Additionally, reliance on the global logger was removed in favor of passing error information via channels, improving modularity and error flow control.

Resolves issue with silent forwarding failures by providing a means to alert system operators without halting the service. This enhancement makes the error reporting more robust and reactive while maintaining service continuity.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove unused field

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update package name

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-22 16:36:34 +01:00
Sammy Kerata Oina 34c3bbdbd8 COCOS-55 - Pass agent logs to manager via vsock (#64)
* Simplify event handling and config

Streamlined event service interface by consolidating `SendEvent` and introducing `SendRaw`. Removed `notification_server_url` and `instance_id` parameters from several event publication calls to leverage centralized event construction. This change not only cleans up redundancy in event-related code but also simplifies the configuration data flow across the system, making it easier to manage and less error-prone. Uniform event generation now improves consistency and maintainability.

Refactored configuration management in the agent and manager services. Removed notifications URL from the agent configuration, relying on a simplification that assumes a single source of events. Updated Manager Port to VsockConfigPort for clarity and consistency across vsock communication.

These modifications should facilitate easier integration and extension of event and configuration systems in the future.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor error handling in agent event forwarding

Introduced context and error channel handling to the agent event forwarding process. The logger now warns on errors during forward operations asynchronously, allowing for non-blocking error reporting. Additionally, reliance on the global logger was removed in favor of passing error information via channels, improving modularity and error flow control.

Resolves issue with silent forwarding failures by providing a means to alert system operators without halting the service. This enhancement makes the error reporting more robust and reactive while maintaining service continuity.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove unused field

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance agent logging via vsock connection

Redirected agent logging to use a vsock connection instead of standard output, improving the process isolation and enabling centralized log management. The change involved dialing to the specified vsock log port and initializing the logger with the vsock connection rather than stdout.

Additionally, the manager service now maintains a map of agent vsock cids to computation IDs, providing better tracking of computation resources. A routine to retrieve logs from agents was also initiated during the service setup to facilitate log collection.

Consequential to these changes is the removal of a redundant os package import in the agent's main.go, further cleaning up the dependencies.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fail gracefully

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Updated backoff strategy and VM configurations

- Added `github.com/cenkalti/backoff` to direct dependencies for robust retry logic in agent configuration sending.
- Modified the vsock logs port to align with the updated port range standards.
- Enclosed kernel console arguments in quotes to ensure proper parsing in QEMU configurations.
- Implemented exponential backoff when sending agent configurations to handle transient failures.

Refactors:
- Streamlined creation of `AgentConfig` within the computation setup to avoid unnecessary initializations when `c.AgentConfig` is not nil.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor command execution and improve argument construction

Consolidated the error handling in the command execution function for better readability. In the QEMU configuration, the argument assembly process is enhanced for clarity and correctness; the VNC parameter is now separate, and string quoting is handled properly for kernel parameters. These changes result in more maintainable code and prevent potential formatting issues during QEMU argument parsing.

Resolves issues with argument construction in QEMU config module.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refine default config handling and unpacking

Improved the agent configuration by dynamically setting default values for the log level and port if they are not specified in the incoming configuration. Also streamlined configuration unpacking in the endpoint and service layers, reducing redundancy and ensuring all required fields are correctly copied over to the Manager's configuration structure. This change ensures better fault tolerance and more maintainable code by handling edge cases where configuration values might be missing.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* rename dir

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Ensure runRes.Empty() reflects non-empty state

Changed the always-true return value of the `runRes.Empty()` method to `false` to accurately indicate the presence of a response body. This adjustment ensures downstream handling of API responses aligns with actual content state.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Replace mglog with slog across codebase

Updated various components to replace the `mglog` logger implementation with the `slog` logger. This change affects logging initialization and calls throughout the codebase including the agent, manager, and internal server components. Transitioning to `slog` is part of a broader shift to standardize the logging mechanism to improve maintainability and consistency.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-22 14:13:47 +01:00
Sammy Kerata Oina 55afe4c038 COCOS-49 - Pass agent configuration and computation via vsock (#57)
* Optimize QEMU launch and add V-sock support

Refactored QEMU argument construction and launching logic by removing the dependency on 'agent.Computation'. This simplification makes the VM creation process more streamlined. Additionally, introduced V-sock capabilities in the QEMU configuration to facilitate improved guest-host communication. Updated the README to include kernel module setup instructions for the new V-sock feature.

The V-sock implementation enables VMs to use a consistent communication channel that is not affected by network configuration changes, enhancing reliability and potential interoperability with host services. It's important to ensure that the necessary kernel modules are loaded as part of the setup process, as documented.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add vsock-based communication to manager

Introduced virtual socket (vsock) communication abilities in the manager package by implementing a new socket service. This includes establishing a vsock listener and stub methods for sending computation results and cleaning up resources. The addition provides the groundwork for interprocess communication between guest and host in virtualized environments.

- Integrated the `mdlayher/vsock` library for handling virtual socket operations.
- Created a new `sockService` struct to encapsulate vsock listener handling.
- Implemented `NewVsock` constructor to initialize the listener with domain value `3`.
- Added placeholder methods for future computation sending and service closing logic.

This enhancement targets scenarios where efficient VM-to-host communication is required.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove env

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor agent config and use vsock

Introduce `AgentConfig` struct to group agent-related configurations, and update `Computation` struct to include the new `AgentConfig` field. Replace command-line computation extraction with vsock-based config retrieval for robustness and decoupling. The agent configuration is now read from a vsock connection during runtime, allowing for more dynamic and flexible deployments. Adjusted the main agent application logic to support these configuration changes, and corresponding changes have been made in the manager to facilitate vsock communication.

This approach aligns with modern practices for microservices by streamlining configuration management and reducing reliance on static command-line parameters. Moreover, it enhances the scalability of the agent service by allowing configuration to be managed externally.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor agent config and remove deprecated code

Consolidated agent configuration management into a single `AgentConfig` message and pruned deprecated Protobuf `ComputationReq`, `DatasetReq`, and `AlgorithmReq` messages. Adapted corresponding manager service logic to the new configuration structure. These modifications align with updated manager API schema, facilitate clearer configuration handling, and improve maintainability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* send configuration

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Switch agent to listen mode for manager connections

Previously, the agent established a connection to the manager using a direct dial. This change shifts the setup to where the agent listens on a specified port and accepts incoming connections. It ensures that the agent properly handles incoming requests by initiating a listening socket and waiting for the manager to connect, enhancing the system's flexibility in connection management. This adjustment also includes graceful closure of the listening socket.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-19 17:24:51 +01:00
Danko Miladinovic 2bb3b5650a Add support for vSockets in guest kernel (#61)
* Add python pip to buildroot

* Use DHCP for agent

* Use systemd for naming network interfaces

* Removed eth0 dhcp setting

* Add ETH_IFACE variable

* Added explanation for ETH_IFACE name

* Further explain ETH_IFACE value

* Add guest kernel support for VSOCKETS
2024-01-18 14:41:05 +01:00
Sammy Kerata Oina b4a49fb56c COCOS-58 - Remove refernece to network.link file (#59)
* Optimize QEMU launch and add V-sock support

Refactored QEMU argument construction and launching logic by removing the dependency on 'agent.Computation'. This simplification makes the VM creation process more streamlined. Additionally, introduced V-sock capabilities in the QEMU configuration to facilitate improved guest-host communication. Updated the README to include kernel module setup instructions for the new V-sock feature.

The V-sock implementation enables VMs to use a consistent communication channel that is not affected by network configuration changes, enhancing reliability and potential interoperability with host services. It's important to ensure that the necessary kernel modules are loaded as part of the setup process, as documented.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add vsock-based communication to manager

Introduced virtual socket (vsock) communication abilities in the manager package by implementing a new socket service. This includes establishing a vsock listener and stub methods for sending computation results and cleaning up resources. The addition provides the groundwork for interprocess communication between guest and host in virtualized environments.

- Integrated the `mdlayher/vsock` library for handling virtual socket operations.
- Created a new `sockService` struct to encapsulate vsock listener handling.
- Implemented `NewVsock` constructor to initialize the listener with domain value `3`.
- Added placeholder methods for future computation sending and service closing logic.

This enhancement targets scenarios where efficient VM-to-host communication is required.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Revert "Optimize QEMU launch and add V-sock support"

This reverts commit 44557c39d9.

* Revert "Add vsock-based communication to manager"

This reverts commit d1d3c01f3b.

* Remove unnecessary network link file installation

Network configuration is being handled elsewhere, making the installation of the `00-network.link` file redundant. Eliminated the line from the agent Makefile to prevent potential conflicts and streamline the initialization process.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-17 15:14:53 +01:00
Danko Miladinovic c25cfec84c Use DHCP for VM networking (#54)
* Add python pip to buildroot

* Use DHCP for agent

* Use systemd for naming network interfaces

* Removed eth0 dhcp setting

* Add ETH_IFACE variable

* Added explanation for ETH_IFACE name

* Further explain ETH_IFACE value
2024-01-15 19:27:07 +01:00
Danko Miladinovic cca75fc5cc Add python pip to buildroot (#51) 2024-01-12 17:51:26 +01:00
Filip Bugarski d24ce2b5bd NOISSUE - Fix Magistrala dependency version (#48)
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2024-01-12 14:44:41 +01:00
Sammy Kerata Oina 200f207673 fix logger (#47)
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-12 14:34:54 +01:00
Danko Miladinovic 004f286900 Use systemd for HAL startup (#46) 2024-01-12 13:33:46 +01:00
Danko Miladinovic 0a0f757833 Use systemd for HAL boot (#40)
* Use systemd for HAL boot

* Removed unnecessary mkdir command

* Add  newline at the end of file

* Configure eth0 network interface

* Keep kernel naming scheme

* Fix agent.mk file for network interfaces
2024-01-11 22:05:42 +01:00
Sammy Kerata Oina 1167aeb53f NOISSUE - Update documentation (#20)
* Fix bug in agent service and state

The commit fixes a bug in the agent service and state files.
Previously, the condition to check the state in the agent service
was incorrect. It was checking the state directly instead of
using the GetState() method. This has been fixed by using the
GetState() method to check the state.

Additionally, a new GetState() method has been added to the
StateMachine struct in the state file. This method retrieves
the current state by acquiring a lock and returning the state
value.

The changes have been tested and verified to resolve the bug
and improve the accuracy of state checking in the agent service.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix bug in agent state machine

The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in state machine

The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in StateMachine.Start()

The StateMachine.Start() method was experiencing a race condition
when multiple events were being processed concurrently. This was
caused by not properly locking and unlocking the state machine
before and after updating the state. This commit fixes the issue
by adding proper locking and unlocking around the state update
operation. Additionally, the logging statement has been updated
to include the previous and next states for better debugging.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add magistrala dep

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove mainflux

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger.

The commit message should be:

"Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger"

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix notification topic in agent service and update NATS ports in Docker environment variables

The agent service's notification topic was incorrectly set to "channels.manager" instead of "agent". This commit fixes the issue by updating the notification topic.

Additionally, the NATS ports in the Docker environment variables were incorrect. The COCOS_NATS_PORT and COCOS_NATS_HTTP_PORT have been updated to the correct values.

These changes ensure that the agent service uses the correct notification topic and the NATS ports are properly configured.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor Makefile and add new targets for building Docker images

The Makefile has been refactored to include new targets for building Docker images. The `make_docker` and `make_docker_dev` functions have been defined to handle the Docker build process. The `dockers` and `dockers_dev` targets have been added to build the Docker images for all services and development environments respectively.

This commit introduces changes to the Makefile to improve the build process and provide support for Docker images.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update readme

Signed-off-by: SammyOina <sammyoina@gmail.com>

* wrap env vars

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor main.go in cmd/cli

The main.go file in cmd/cli has been refactored to improve code readability and maintainability. The defURL constant has been removed as it is no longer needed. Additionally, unnecessary whitespace has been removed.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix linting

Signed-off-by: sammy <sammyoina@gmail.com>

* rename cocos-ai to cocos

Signed-off-by: sammy <sammyoina@gmail.com>

* Updated README with NATS setup instructions and correct systemd path

Expanded the project's README to include setup instructions for the NATS server, which is now a necessary component for agent and manager communication. Additionally, the systemd service file path has been corrected from 'systemd' to 'init/systemd' ensuring the agent's proper installation and configuration as a daemon. This facilitates a smoother setup experience and reflects the dependency on NATS for push notifications.

Ref: Agent and manager communication enhancement
Signed-off-by: SammyOina <sammyoina@gmail.com>

* Added message broker URL to agent service config

Included the environment variable for the message broker URL in the systemd service configuration to facilitate agent communication with the messaging system. This ensures the agent can connect to the designated message broker for event publishing and subscription.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update docs to hal

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Remove TLS and timeout config from gRPC client

Refactored gRPC client and associated API code by removing unnecessary TLS configuration and timeout settings. Simplified the communication setup by trusting the environment to enforce security policies and handle operation durations, instead of hardcoding these within the application.

This brings about a cleaner, more maintainable codebase and shifts responsibility for security configurations out of the code, aligning with infrastructure-as-code practices and enabling easier scalability and environment-specific adjustments.

This change also affects the API documentation and example commands, which have been updated accordingly to reflect the simplification and to guide users with the streamlined setup process.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
Signed-off-by: sammy <sammyoina@gmail.com>
2024-01-11 15:19:34 +01:00
Sammy Kerata Oina a5e6cae92c COCOS-44 - Add agent address to run responses (#45)
* Add agent address to run responses

The manager service's Run method now retrieves the agent address upon successful computation execution, providing more informative responses across gRPC, HTTP, and logging endpoints. This change improves service transparency by returning the agent's address to be used by client services, making the manager service's external communication more comprehensive. Updated the `RunResponse` structure in the corresponding protocol buffers definition and response handling in gRPC and HTTP APIs, ensuring that agent address information is serialized appropriately. It also necessitates a slight adjustment in the QEMU configuration to manage port forwarding rules more dynamically, simplifying the process as only agent-relevant ports are incremented with each new computation. This extra detail in responses aids in debugging and offers better integration capabilities for clients.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Allocate dynamic ports for VM guests and expand error handling

Refactored service initialization to accept host IP and incorporated dynamic port allocation for VM guests, replacing the prior static increment method. Introduced a new error type 'ErrFailedToAllocatePort' to capture instances where the system is unable to find a free port. Integrated a third-party error package for improved error wrapping and context. These changes prevent port conflicts between VM guests and enhance error diagnostics for service operations.

Resolves issue with static port allocation leading to conflicts.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add HOST_IP to service configuration

Extend service configuration to include the host machine's IP address, allowing instances to be aware of their deployment environment. This update passes the new HostIP field to the service constructor, ensuring the service can now operate with host-specific logic.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Populate AgentAddress in gRPC Responses

Enhanced the gRPC encode/decode functions to properly populate the 'AgentAddress' field in 'RunResponse' objects. This ensures that consumers of the gRPC interface receive complete response data, which previously omitted the important 'AgentAddress' information. The change impacts both server-side response encoding and client-side response decoding, aligning the implementation with the expected interface contract.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix ci

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-11 15:08:35 +01:00
Sammy Kerata Oina c29ef354fa COCOS-24 - Add events service via http (#27)
* add status endpoint

Signed-off-by: sammy <sammyoina@gmail.com>

* feat: Update code generation tools to latest versions

Update the code generation tools, including protoc-gen-go and
protoc-gen-go-grpc, to their latest versions (v1.31.0 and v1.3.0,
respectively). This ensures compatibility with the latest features and
improvements. The updated tools also require gRPC-Go v1.32.0 or later.

The new versions bring important updates and bug fixes, enhancing the
performance and stability of the generated code. By staying up-to-date
with the latest tooling, we can take advantage of the latest
functionality and ensure a smooth development experience.

No code changes are included in this commit. These updates only impact
the code generation process.

Signed-off-by: sammy <sammyoina@gmail.com>

* Update Go version to 1.21.x

Signed-off-by: sammy <sammyoina@gmail.com>

* Refactor agent and manager services to publish event notifications

The refactoring includes changes to the agent and manager services to incorporate event notifications. By publishing events, the services can inform subscribers about the current state of the computation or any updates. Specifically, the `agentService` now includes a `cmpHash` field to store the SHA-256 hash of the computation, which is subsequently used when publishing events. The `agentService` and `managerService` now use the `publisher` interface to publish events to the topic "manager". Notably, the removed `pubsub.go` file is no longer necessary.

This commit improves the service architecture by allowing subscribers to receive relevant updates and monitor the progress of computations. It enhances the overall system by providing more transparency and enabling better coordination between the agent and manager services.

Signed-off-by: sammy <sammyoina@gmail.com>

* Improve generated Go file comparison in checkproto workflow

Refactor the file comparison logic in the checkproto workflow to use the `-p` flag instead of `-s` for improved accuracy. This change ensures that the generated Go files are thoroughly compared with the original ones, detecting any discrepancies and preventing out-of-sync files from passing the validation. By using the `-p` flag, we now check both the contents and the metadata of the files, providing more robust synchronization checks. This update enhances the reliability of the checkproto workflow and helps maintain consistency between the proto files and their corresponding generated Go files.

Signed-off-by: sammy <sammyoina@gmail.com>

* Update file comparison command to detect differences line by line

The code change updates the file comparison command used in the CI workflow to detect differences line by line instead of only reporting the first difference encountered. This change improves the accuracy of detecting inconsistencies between the original protobuf files and the generated Go files. Previously, only the first difference was reported, leading to potential missed issues. By comparing the files line by line, we can now detect and report all differences accurately. This change enhances the reliability of our CI pipeline and ensures that the generated Go files stay in sync with the protobuf files.

Signed-off-by: sammy <sammyoina@gmail.com>

* add event exporting to external server

Signed-off-by: sammy <sammyoina@gmail.com>

* feat: Add support for notification server URL

The commit adds a new environment variable, `COCOS_NOTIFICATION_SERVER_URL`, which allows specifying the server to receive notification events from the agent. This addition provides flexibility to configure the notification server URL based on the deployment environment. This change enables seamless integration with different notification server instances and enhances the extensibility of the system. It resolves the need to modify the code directly when changing the server URL.

Signed-off-by: sammy <sammyoina@gmail.com>

* Refactor gRPC client and server, remove unused handlers

The commit refactors the gRPC client and server code by removing the unused `nopDecoder` and `status` handlers from the client and server, respectively. This cleanup reduces code clutter and improves maintenance. No significant consequences are expected.

Signed-off-by: sammy <sammyoina@gmail.com>

* Ensure generated Go files stay in sync with proto files during the CI workflow

Fixes an issue in the CI workflow where proto files and their corresponding generated Go files were not being properly compared for synchronization. Previously, the `cmp -l` command was used, which only printed differing byte positions, leading to false negatives. This has been corrected by using `cmp -s` instead, which outputs nothing if the files are identical. This change ensures that any differences between the proto files and their generated Go files will be detected, helping to maintain consistency and accuracy in the codebase.

Signed-off-by: sammy <sammyoina@gmail.com>

* Enhance notification payload and endpoint

Extended the notification system to include 'status' and 'details' in the payload, improving traceability and debugging. Adapted the serialized JSON structure for clarity and added an 'originator' field to track the source service. Transitioned to a generalized event endpoint, facilitating a more streamlined event handling process.

Refactors POST request to a more appropriate endpoint and updates the notification service interface to reflect new payload requirements.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor event notification logic

Removed the legacy notifications package and consolidated event notification functionality using the new internal events service. Modified agent, manager, and main application code to use this service for consistent event reporting and error handling workflows across services. This change simplifies event management, improves error visibility, and allows for more maintainable code by centralizing event-related logic. The substitution of verbose state-specific publishEvent calls with generic status reporting aligns with the new service's capabilities.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Expand agent protobuf message types and improve error handling

The protobuf definition for agent messages has been updated to include an additional message type, facilitating future data structure expansions. Additionally, error handling for event sending in the main agent execution has been enhanced to log errors when sending 'init' events fail, ensuring issues are properly tracked. The unused `notificationTopic` constant in the manager service has been removed for cleaner code maintenance.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Optimize JSON parsing and fix header omission

Removed unnecessary unquoting of a JSON string before unmarshaling, streamlining the computation value extraction process. Also corrected a missing Content-Type header in the event sending function, ensuring proper handling of JSON requests by recipients. These changes improve performance and communication reliability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* align vars

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Remove computation request timeout feature

The timeout feature for computation requests has been removed to simplify the computation execution flow. This involved changes across multiple files, including protobuf definitions, HTTP endpoint handling, and the internal computation logic. We eliminated the timeout field, associated logic, and error handling to ensure the system no longer supports timeouts for computations, mitigating any unintended timeout impacts on long-running processes.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: sammy <sammyoina@gmail.com>
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-11 14:37:14 +01:00
Sammy Kerata Oina b55b6c9734 NOISSUE - Refactor computation parameter naming and remove tracing (#42)
Standardized the variable name for computation parameter across various service and middleware implementations to improve code consistency. This enhances readability and maintainability.

Additionally, tracing functionality has been removed completely, signifying a potential shift in the telemetry strategy or removal of an unused feature. The run command associated with cli package has also been deleted, suggesting a change in the CLI tool's capabilities or its command handling strategy.

The service interface's Run method definition and its implementations now align with the updated parameter naming, ensuring coherence throughout the codebase.

Removed files:
- The entire `tracing` package has been deleted.
- Removed the `cli/run.go` file, indicating the removal of the 'run' command from the CLI.

Please note that these changes may affect any existing workflows or systems that rely on the removed tracing capabilities or the CLI run command, and appropriate adjustments will need to be made to accommodate these updates.

Refer to internal documentation or release notes for migration paths or additional context.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-10 11:52:37 +01:00
Sammy Kerata Oina 4d9f7b1a45 COCOS-25 - Remove networking between manager and agent (#43)
* Enable passing computation details to QEMU args

Refactored the QEMU argument construction to accept computation details as JSON. The changes include updating function signatures to pass an `agent.Computation` object and serialize it into the QEMU boot parameters. These modifications allow for custom computations to be specified per VM, facilitating more dynamic and configurable virtual machine creation. This update also integrates timing and data transformation necessary for the agent computation setup.

Resolves computation configurability for virtual machines.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enable extraction of computation value from cmdline

Introduced a feature to parse the computation parameter directly from the machine's command line, thus enabling dynamic configuration of agent computations without the need for a preset environment. This change enhances the agent's ability to adapt to different computation requirements on startup. A fallback warning informs of cases where the computation parameter was not found, ensuring traceability of operational state during service initialization.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Remove RunRequest and RunResponse from Agent gRPC

Refactor the Agent service by deleting the RunRequest and RunResponse messages along with the corresponding method implementations in the gRPC interface. This simplifies the gRPC protocol by removing an obsolete RPC method, streamlining communication between services. The deletion also impacts the manager service, which no longer uses the removed RPC method. Consequently, the service definitions across related files like endpoints, servers, and clients have been adjusted. This adjustment enforces a more direct approach for starting computations, reflecting a shift toward manager-initiated computation execution.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-09 20:07:45 +01:00
Sammy Kerata Oina 46a91ccbc3 remove nats (#41)
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-09 19:56:31 +01:00
Sammy Kerata Oina 8c5bb52c56 COCOS-7 - Switch qemu config to use kernel and initrd images (#39)
* Switch qemu config to use kernel and initrd images

Refactored qemu configuration to load the system using separate kernel and initial ramdisk images instead of a QCOW2 image. Removed previous disk image configurations and replaced them with directives to specify the kernel (`bzImage`) and root filesystem (`rootfs.cpio.gz`) files. Adjusted the constructQemuArgs function accordingly to set the kernel path, kernel command line options, and initrd path. Updated CreateVM function to handle copying of the new kernel and rootfs images. These changes align the virtual machine setup process with a more modular and potentially customizable boot configuration.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

* capcomments

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-08 18:05:35 +01:00
Sammy Kerata Oina ee7159a406 COCOS-35 - Refactor RunRequest to use structured Computation (#38)
* Refactor RunRequest to use structured Computation

The protobuf and associated service implementations for the RunRequest message were refactored to replace the raw Computation byte slice with a structured ComputationReq object. This allows clearer and more type-safe manipulation of computation requests. The grpc, http, and agent service layers were updated to build and parse ComputationReq accordingly. The ComputationReq structure includes details like IDs, names, time stamps, and metadata, forming a well-defined contract for computation tasks.

This change aligns with efforts to standardize request formats and improve clarity in inter-service communication. It impacts all systems interfacing with the RunRequest service and thus requires coordinated updates to the entire stack.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Initialize metadata maps and handle nil values

Improved the robustness of metadata handling in gRPC endpoints and SDK by initializing metadata maps and explicitly checking for nil values before converting them. This ensures that both the agent's gRPC endpoint and the SDK properly handle cases where metadata fields may be uninitialized or contain nil values, preventing potential null pointer exceptions.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor computation request handling

Refactored the endpoint to construct Computation object from gRPC request, incorporating structpb for metadata handling and timestamppb for StartTime and EndTime fields. The management service and API requests are also updated to align with these changes, improving type safety and ensuring data is correctly marshalled when making service calls.

Resolves data marshalling issues for computation requests.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* use singular

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove unuse fields

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove unused fields

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-08 17:51:46 +01:00
Sammy Kerata Oina ee1f5ebb3d NOISSUE - Optimize context deadline and error handling (#34)
Improved the timing logic for setting computation deadlines by replacing the channel-based pattern with a direct time addition. This change enhances the performance by preventing unnecessary channel operations.

Additionally, relocated the select block that checks for computation timeout and data reception errors. This alteration ensures that the socket connection acceptance routine starts before any timeouts or errors are checked, thereby avoiding potential race conditions and making the code more resilient to early timeout scenarios.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-04 12:53:08 +01:00
Sammy Kerata Oina 7ea8e48dbb NOISSUE - Optimized Docker build (#32)
Refactored the Dockerfile to use multi-stage builds, improving the image size and separation of concerns. Removed `qemu-system-x86_64` from the initial build stage and included it in a new, minimal final image based on `alpine:latest`. This change minimizes the production image size by avoiding unnecessary build dependencies in the final artefact. Additionally, the CA certificates are now copied into the final stage to maintain SSL functionality.

The development Dockerfile has been cleaned to match new conventions.

Resolves issues with bloated Docker images and unneeded dependencies in production.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-04 12:35:29 +01:00
Sammy Kerata Oina a3c5c765b8 NOISSUE - Connect to agent when vm is created (#33)
* Add gRPC TLS config and update protoc versions

Enhanced manager service with TLS configuration options and timeout settings for gRPC communication with agent services. Updated corresponding protobuf definitions to include new fields for CA certificates, TLS status, and timeouts. The added TLS support ensures secure inter-service communication while flexible timeout configurations improve the robustness of network interactions.

The following adjustments and additions were made:
- Included new gRPC configuration fields (ca_certs, client_tls, timeout) for agent communication in protobuf definitions.
- Injected agent gRPC configuration into service endpoints, ensuring secure TLS setup and compliance with provided settings.
- Revised main service function signatures to accept the gRPC configuration object.
- Incremented port forwarding counters post-computation to avoid port conflicts.
- Conducted compatibility update of protobuf version comments to reflect minor version bump (v4.25.0 to v4.25.1).

This change impacts service deployment that requires proper configuration of TLS credentials and mindful determination of timeouts for efficient network use.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update proto

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Set default timeout for agent configuration

Introduced a default timeout of 60 seconds in both gRPC and HTTP endpoints for agent configuration when none is specified. This change ensures that operations do not hang indefinitely and provides a reasonable default for client interactions. Additionally, the instantiation of a new agent client is now outside the retry loop to avoid repeated setup on transient failures.

Refactors service logic to optimize client connection handling by moving the agent client setup to occur before attempting retries, which should reduce overhead and improve clarity in error situations.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Subject: Move agent client creation into retry loop

Body:
Refactored the service manager's Run method to initialize the agent gRPC client inside the exponential backoff retry loop. This change addresses intermittent connection issues by reattempting client creation on temporary network failures, ensuring a robust setup before calling the Run method on the client.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-04 12:34:57 +01:00
Sammy Kerata Oina 34c5472485 NOISSUE - Remove multiple brokers (#31)
* remove multiple brokers

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Standardize build process on NATS messaging

Removed conditional build flags for different message broker types, making NATS the default and only option. Simplified the Makefile by eliminating the toggleable message broker configuration and related docker-based setup. Updated the NATS URL in docker `.env` and docker-compose service dependency to reflect this change. This adjustment streamlines the build process and reduces configuration complexity.

Resolves issues with inconsistent messaging setups across various environments.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor message broker configuration

Standardized the environment variable for the message broker URL across services and updated documentation to reflect this change. Renamed the environment variable from COCOS_NATS_URL to COCOS_MESSAGE_BROKER_URL to improve clarity and maintain consistency in service configuration. This alteration facilitates future support for different message broker backends beyond NATS without further variable name changes.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-12-24 19:45:04 +01:00
Sammy Kerata Oina 1fec5fe3d4 add repo readme (#30)
Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-12-24 15:46:31 +01:00
Dusan Borovcanin 4c53fd26e3 NOISSUE - Fix CI Docker build
Signed-off-by: Dusan Borovcanin <borovcanindusan1@gmail.com>
2023-12-21 17:43:03 +01:00
Sammy Kerata Oina 668552256f NOISSUE - Create and publish a Docker image (#29)
* Create and publish a Docker image

This commit adds a new file `.github/workflows/build.yml` which contains the workflow configuration for creating and publishing a Docker image. The workflow is triggered on push events to the `main` branch. It sets environment variables for the registry and image name. The job runs on `ubuntu-latest` and has necessary permissions for reading contents and writing packages. The steps include checking out the repository, logging in to the container registry using the GitHub token, and extracting metadata for Docker.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor build workflow and update Makefile

The build workflow has been refactored to build the Manager image instead of the API Gateway image. The Makefile has also been updated.

The changes include:
- Renaming the build step to "Build Manager and push Docker image"
- Updating the image name to include "/manager"
- Updating the build arguments to use "SVC=manager"
- Updating the tags to use the output from the metadata action

These changes ensure that the correct Docker image is built and pushed.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove labels

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-12-21 17:26:34 +01:00
Danko Miladinovic 3e7fc09233 Update documentation (#28)
* Modify external.desc and README

* Added a blank line at end of external.desc file
2023-12-08 16:45:02 +01:00
Drasko DRASKOVIC 0dac01d752 NOISSUE - Add HAL (#26)
* Add HAL

Signed-off-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>

* Add Agent package in HAL

Signed-off-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>

---------

Signed-off-by: Drasko DRASKOVIC <drasko.draskovic@gmail.com>
2023-12-05 15:08:49 +01:00
Sammy Kerata Oina 18508796c1 add linters (#23)
- switchTrue
      - importShadow
      - httpNoBody
      - paramTypeCombine
      - emptyStringTest
      - builtinShadow
      - exposedSyncMutex
      - importas

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-24 18:59:42 +01:00
Sammy Kerata Oina 5adf0349a3 COCOS-9 - Dockerize manager (#18)
* Fix bug in agent service and state

The commit fixes a bug in the agent service and state files.
Previously, the condition to check the state in the agent service
was incorrect. It was checking the state directly instead of
using the GetState() method. This has been fixed by using the
GetState() method to check the state.

Additionally, a new GetState() method has been added to the
StateMachine struct in the state file. This method retrieves
the current state by acquiring a lock and returning the state
value.

The changes have been tested and verified to resolve the bug
and improve the accuracy of state checking in the agent service.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix bug in agent state machine

The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in state machine

The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in StateMachine.Start()

The StateMachine.Start() method was experiencing a race condition
when multiple events were being processed concurrently. This was
caused by not properly locking and unlocking the state machine
before and after updating the state. This commit fixes the issue
by adding proper locking and unlocking around the state update
operation. Additionally, the logging statement has been updated
to include the previous and next states for better debugging.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add magistrala dep

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove mainflux

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger.

The commit message should be:

"Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger"

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE

The Makefile has been updated to fix an issue with setting the DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE variables. The USER_REPO variable is now used to generate the DOCKER_PROJECT name following the Docker Compose guidelines. Additionally, the COCOS_MESSAGE_BROKER_TYPE variable is now properly set to "nats" if it is empty. This ensures that the correct values are used when compiling and installing the service.

Summary:
Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE

Details:
- Update USER_REPO variable to generate DOCKER_PROJECT name
- Set COCOS_MESSAGE_BROKER_TYPE to "nats" if empty

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix notification topic in agent service and update NATS ports in Docker environment variables

The agent service's notification topic was incorrectly set to "channels.manager" instead of "agent". This commit fixes the issue by updating the notification topic.

Additionally, the NATS ports in the Docker environment variables were incorrect. The COCOS_NATS_PORT and COCOS_NATS_HTTP_PORT have been updated to the correct values.

These changes ensure that the agent service uses the correct notification topic and the NATS ports are properly configured.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add Dockerfiles for production and development environments

This commit adds two new Dockerfiles, one for the production environment and one for the development environment. The production Dockerfile sets up the necessary dependencies and builds the application, while the development Dockerfile simply copies the built application. Both Dockerfiles include the necessary SSL certificates for the application to work properly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor Makefile and add new targets for building Docker images

The Makefile has been refactored to include new targets for building Docker images. The `make_docker` and `make_docker_dev` functions have been defined to handle the Docker build process. The `dockers` and `dockers_dev` targets have been added to build the Docker images for all services and development environments respectively.

This commit introduces changes to the Makefile to improve the build process and provide support for Docker images.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix makefile to generate Docker images for services

- Added DOCKERS and DOCKERS_DEV variables to generate Docker images for services
- Updated the dockers target to include DOCKERS
- Updated the dockers_dev target to include DOCKERS_DEV
- Updated the $(DOCKERS) and $(DOCKERS_DEV) targets to call the respective make_docker and make_docker_dev functions

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor Makefile and Dockerfile

The Makefile has been refactored to remove unnecessary sed commands and improve code readability. The changes include:

- Removed sed commands related to changing the broker in docker-compose.yml for nats and rabbitmq profiles.
- Removed the eer target from the Makefile.

The Dockerfile has been modified to include the WORKDIR directive.

These changes improve the maintainability and readability of the codebase.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix NATS URL in .env file and add Manager configuration in docker-compose.yml

The NATS URL in the .env file was updated to use the correct hostname for the broker. Additionally, the Manager configuration was added to the docker-compose.yml file.

This commit fixes the NATS URL in the .env file and adds the necessary Manager configuration to the docker-compose.yml file.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix incorrect host and port values in Makefile and .env

The commit updates the host and port values in the Makefile and .env files to fix incorrect values. The host and port values are updated to "cocos-manager" and "7003" respectively. This ensures that the correct host and port are used for the Manager HTTP and gRPC services.

This commit resolves the issue with the incorrect host and port values and ensures that the Manager services are configured correctly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Dockerfile and docker-compose.yml configurations

The Dockerfile and docker-compose.yml configurations have been fixed to address the following issues:

- Removed unnecessary COPY commands in the Dockerfile.
- Removed the "privileged" flag in the docker-compose.yml file.

These changes ensure that the Docker image is built correctly and the container is launched with the appropriate configurations.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove manual assignment

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update protoc CI

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add paths

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove profiles

Signed-off-by: SammyOina <sammyoina@gmail.com>

* restore default config

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Docker project name in Makefile

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-24 18:57:31 +01:00
Sammy Kerata Oina 0f831706f9 NOISSUE - Add CoCos-AI Manager API specification (#19)
* Add CoCos-AI Manager API specification

This commit adds the CoCos-AI Manager API specification
in the form of a YAML file. The specification includes
information about the title, description, contact,
license, and version of the API. It also defines the
servers where the API is hosted and the paths and
operations available, such as running computation on
a virtual machine.

The API specification is based on OpenAPI 3.0.1 and
provides a clear and concise overview of the CoCos-AI
Manager service.

The commit also includes a link to the CoCos-AI repository
and the license information.

This commit is necessary to provide a clear and documented
API specification for the CoCos-AI Manager service.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor go.mod and go.sum files

The go.mod and go.sum files have been refactored to remove the go.opentelemetry.io/contrib/propagators/jaeger package, which is no longer needed. This package was causing compatibility issues with the current version of the project. The refactoring ensures that the project is using the latest compatible versions of the required dependencies.

This commit removes the go.opentelemetry.io/contrib/propagators/jaeger package from the go.mod file and updates the go.sum file accordingly.

Note: The go.mod file now uses go.opentelemetry.io/otel v1.19.0 and go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.19.0.

Please review the changes to ensure compatibility and functionality.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix typo

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-24 18:54:54 +01:00
Sammy Kerata Oina 298878c96d NOISSUE - Rename module to cocos (#22)
* add stringer

Signed-off-by: SammyOina <sammyoina@gmail.com>

* rename module to cocos

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix fmt.Stringer implementation in computations.go

The fmt.Stringer implementation for Datasets and Algorithms in computations.go was fixed to correctly use pointers.

This commit addresses the issue where the String() method for Datasets and Algorithms in computations.go was not correctly implemented. The fix ensures that the String() method now correctly marshals the data to JSON and returns the string representation.

The changes made in this commit will improve the functionality and accuracy of the String() method for Datasets and Algorithms.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-22 13:15:19 +01:00
Dušan Borovčanin 7b31e21818 NOISSUE - Fix attestation gRPC call (#21)
Signed-off-by: Dusan Borovcanin <borovcanindusan1@gmail.com>
2023-11-21 12:25:24 +01:00
Sammy Kerata Oina 4ea9ff6531 NOISSUE - Add message broker on agent and manager (#17)
* Fix bug in agent state machine

The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in state machine

The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in StateMachine.Start()

The StateMachine.Start() method was experiencing a race condition
when multiple events were being processed concurrently. This was
caused by not properly locking and unlocking the state machine
before and after updating the state. This commit fixes the issue
by adding proper locking and unlocking around the state update
operation. Additionally, the logging statement has been updated
to include the previous and next states for better debugging.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add magistrala dep

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove mainflux

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix agentService New function to include messaging.Publisher parameter

The agentService New function has been updated to include a messaging.Publisher parameter. This change allows the agent service to publish messages to a messaging system. The messaging.Publisher parameter has been added to the agentService struct and the New function signature has been updated accordingly. This change ensures that the agent service can communicate with other components using the messaging system.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor service.go state functions

The commit refactors the state functions in the service.go file.
The functions for each state have been modified to use the svc.publishEvent
method to publish events with appropriate messages.

- Refactor state functions in service.go
- Use svc.publishEvent to publish events with messages for each state

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix computation run event publishing and add pubsub functionality

The computation run event publishing in the agent service was fixed to correctly call the publishEvent function. Additionally, the pubsub functionality was added to the manager package.

- Fixed computation run event publishing in agent service
- Added pubsub functionality to manager package

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix license header in pubsub.go file

The commit fixes the license header in the pubsub.go file.
The copyright and SPDX-License-Identifier have been added
to comply with the Apache-2.0 license.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger.

The commit message should be:

"Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger"

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE

The Makefile has been updated to fix an issue with setting the DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE variables. The USER_REPO variable is now used to generate the DOCKER_PROJECT name following the Docker Compose guidelines. Additionally, the COCOS_MESSAGE_BROKER_TYPE variable is now properly set to "nats" if it is empty. This ensures that the correct values are used when compiling and installing the service.

Summary:
Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE

Details:
- Update USER_REPO variable to generate DOCKER_PROJECT name
- Set COCOS_MESSAGE_BROKER_TYPE to "nats" if empty

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Makefile Docker profile assignment and build flags

The Makefile was updated to fix the assignment of the Docker profile and build flags. The Docker profile is now assigned based on the value of COCOS_MESSAGE_BROKER_TYPE, and if it is not provided, the default value is set to "nats". The build flags were also updated to include the COCOS_MESSAGE_BROKER_TYPE value as a tag for the Go build process.

This commit addresses the issue with the Docker profile assignment and ensures that the correct build flags are used during the build process.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix makefile

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix notification topic in agent service and update NATS ports in Docker environment variables

The agent service's notification topic was incorrectly set to "channels.manager" instead of "agent". This commit fixes the issue by updating the notification topic.

Additionally, the NATS ports in the Docker environment variables were incorrect. The COCOS_NATS_PORT and COCOS_NATS_HTTP_PORT have been updated to the correct values.

These changes ensure that the agent service uses the correct notification topic and the NATS ports are properly configured.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add pubsub

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update protoc

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-10 16:12:20 +01:00
Sammy Kerata Oina 4958f9468f NOISSUE - Implement State-machine (#16)
* add state machine

Signed-off-by: SammyOina <sammyoina@gmail.com>

* This commit addresses the following changes:

- In agent/service.go:
  - Added a goroutine to start the StateMachine
  - Sent the start event to the StateMachine

- In agent/state.go:
  - Added a copyright notice and license identifier

These changes ensure that the StateMachine starts correctly and that the agent/state.go file includes the necessary copyright and license information.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in agent service initialization

The commit fixes a race condition in the agent service initialization.
Previously, the `New` function in `agent/service.go` would start the state machine
goroutine before initializing the `sm` field. This could lead to a race condition
where the state machine would access uninitialized fields.

To fix this, the `New` function now takes a `context.Context` argument and passes it
to the `Start` method of the state machine. Additionally, a `sync.Mutex` has been
added to the `StateMachine` struct to ensure thread safety.

This commit addresses the race condition and ensures proper initialization of the agent service.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix sending events when algorithms and datasets are received

This commit fixes a bug where events were not being sent when algorithms and datasets were received in the agent service. The bug was causing the events to not be triggered, leading to incorrect behavior. This commit adds the necessary code to send the events when algorithms and datasets are received, ensuring that the events are properly triggered.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix bug: computation results not ready

The computation results were not being marked as ready,
resulting in an error when trying to access them.
This commit fixes the issue by adding a check for
the computation results before returning them.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix bug in agent service and state

The commit fixes a bug in the agent service and state files.
Previously, the condition to check the state in the agent service
was incorrect. It was checking the state directly instead of
using the GetState() method. This has been fixed by using the
GetState() method to check the state.

Additionally, a new GetState() method has been added to the
StateMachine struct in the state file. This method retrieves
the current state by acquiring a lock and returning the state
value.

The changes have been tested and verified to resolve the bug
and improve the accuracy of state checking in the agent service.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* check for time outs

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix bug in agent state machine

The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in state machine

The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in StateMachine.Start()

The StateMachine.Start() method was experiencing a race condition
when multiple events were being processed concurrently. This was
caused by not properly locking and unlocking the state machine
before and after updating the state. This commit fixes the issue
by adding proper locking and unlocking around the state update
operation. Additionally, the logging statement has been updated
to include the previous and next states for better debugging.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-10 14:41:05 +01:00
Washington Kigani Kamadi 4eea9c1a4c Add attestation endpoint (#5)
* vendor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Return agent changes

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* remove vendor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Add context to sdk functions

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Fix comment

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Add context to functions

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Remove context variable

Signed-off-by: WashingtonKK
<washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2023-11-10 14:05:17 +01:00
Sammy Kerata Oina 8c7d4a76ec NOISSUE - Remove HTTP server from agent and update agent service (#15)
* Update agent.pb.go to include a new field "Provider" in the AlgoRequest struct.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix build

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix undeclared consumer error in agent service

The commit fixes an error in the agent service where an undeclared consumer was requesting results in the computation manifest. The error is resolved by adding an error message for the undeclared consumer case.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-10 13:52:25 +01:00
Sammy Kerata Oina da590bfb63 NOISSUE - Add linters and config file (#12)
* vendor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* remove vendor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
Co-authored-by: WashingtonKK <washingtonkigan@gmail.com>
2023-11-10 13:49:05 +01:00
Sammy Kerata Oina 70785b96e6 return hashes on agent service. (#14)
Co-authored-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-11-01 13:23:01 +01:00
Sammy Kerata Oina f1f8f95653 NOISSUE - Fix dependencies (#13)
* vendor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Return agent changes

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Add missing import

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* remove vendor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Fix formatting

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Formatting errors

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Update agent/api/grpc/client.go

Signed off: WashingtonKK

washingtonkigan@gmail.com

Co-authored-by: Sammy Kerata Oina <44265300+SammyOina@users.noreply.github.com>

* add linters and fix

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update ci

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove deprecated dependencies and
 use local agent and manager

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update mainflux

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Jaeger URL in agent and manager main.go files

The Jaeger URL in the agent and manager main.go files was incorrect. This commit fixes the Jaeger URL by updating it to "http://localhost::4318/v1/traces".

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
Signed-off-by: SammyOina <sammyoina@gmail.com>
Co-authored-by: WashingtonKK <washingtonkigan@gmail.com>
2023-10-27 22:12:53 +02:00
Washington Kigani Kamadi 7d47ed7a20 Fix CI (#6)
* Fix CI

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Fix CI

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Use single make

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2023-10-20 17:25:14 +02:00
Drasko Draskovic 79d1834173 Fix compilation
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2023-10-19 18:23:53 +02:00
Drasko Draskovic a63cafc4d0 Fix non-vendor compilation
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2023-10-19 17:58:23 +02:00
Drasko Draskovic 63d8035e46 Fix dir structure
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2023-10-19 17:34:38 +02:00
Drasko Draskovic 96e3c48fcb Remove vendoring
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2023-10-19 17:31:55 +02:00
Drasko DRASKOVIC 91c9274885 Merge pull request #2 from SammyOina/CA-1
CA-1 - Fix build
2023-10-19 15:31:00 +02:00
SammyOina f9a2fd0b96 Fix Makefile version and commit variables
The Makefile was updated to fix the version and commit variables.
Previously, the VERSION variable was using "git describe" to get the latest tag,
but it was failing when there were no tags available.
The updated VERSION variable now uses "git describe --always" to fallback to the commit hash if no tags are available.
The COMMIT variable was also updated to use "git rev-parse HEAD" to get the commit hash.

Summary:
Fix Makefile version and commit variables

Body:
- Update VERSION variable to use "git describe --always" to fallback to commit hash
- Update COMMIT variable to use "git rev-parse HEAD" to get commit hash

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-10-19 12:58:20 +03:00
SammyOina 7cb04499e8 Update Go version to 1.21.x in GitHub Actions workflow
The Go version in the GitHub Actions workflow was updated from 1.20.x to 1.21.x. This change ensures that the correct version of Go is used during the workflow execution.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-10-19 12:30:00 +03:00
SammyOina 7657f8bd7a fix build
Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-10-19 12:22:52 +03:00
Drasko Draskovic dd08b7d0b1 Fix vendoring
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2023-10-18 21:35:35 +02:00
Drasko Draskovic e9d143a7d3 Merge remote-tracking branch 'manager/main'
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2023-10-18 12:59:28 +02:00
Drasko Draskovic f690c286f2 Merge remote-tracking branch 'agent/main'
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2023-10-18 12:50:08 +02:00
Drasko DRASKOVIC 194fd29d60 Initial commit 2023-10-18 12:05:53 +02:00
Darko Draskovic 740a631287 Merge pull request #45 from ultravioletrs/qemuExponentialBackoff
Qemu VM exponential backoff
2023-10-09 17:21:16 +02:00
WashingtonKK 514cc18bf8 Remove debug logs
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2023-10-09 17:46:13 +03:00
WashingtonKK d96b3414c5 Add backoff
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2023-10-09 17:00:40 +03:00
WashingtonKK c8d1ad4793 vendor docs
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2023-10-09 17:00:19 +03:00
WashingtonKK 7302e82173 Fix variables
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2023-10-09 15:56:25 +03:00
WashingtonKK 2d6487790b Reduce time delay
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2023-10-09 14:59:20 +03:00
WashingtonKK 0ae70d9d75 Add retry loop
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
2023-10-09 14:43:42 +03:00
Darko Draskovic 2e0f3e9240 Merge pull request #38 from ultravioletrs/remove_domain_endpoint
Remove create libvirt domain and create QEMU VM endpoints
2023-09-21 15:52:56 +02:00
Darko Draskovic 4f05805600 Add instructions for AMD-SEV tests (#46)
* Use .venv for AMD-SEV tests

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Add instructions for VM

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

---------

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-21 15:38:56 +02:00
Drasko DRASKOVIC 5171882634 Merge pull request #45 from ultravioletrs/algo_data
Add basic business logic of training a model on a dataset
2023-09-21 11:43:29 +02:00
Darko Draskovic b0b22aeed3 Add manual test procedure for CLI
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-21 10:47:31 +02:00
Darko Draskovic 16cb51563c Unexport AgentSDK by renaming to agentSDK
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-21 10:22:57 +02:00
Darko Draskovic b78b15efb6 Fix algorithms cobra command to send file and not string
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-21 10:20:33 +02:00
Darko Draskovic 10f9ec8517 Rename results.go to result.go
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-19 15:37:58 +02:00
Darko Draskovic 7cb1916e7a Move unix socket handling to separate pkg and update api yaml doc
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-19 13:58:02 +02:00
Darko Draskovic 0ab28aa3cc Replace stdout pipe with unix socket
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-19 00:35:23 +02:00
Darko Draskovic 54e03f3373 Refactor /result to perform actual computation
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-18 18:09:19 +02:00
Darko Draskovic 91dd8dd7b4 Add py script std out capture by in memory io.Writer
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-18 17:00:31 +02:00
Darko Draskovic e9f4d0c8d4 Add run func to apply python script (algo) to csv data (dataset)
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-18 15:52:51 +02:00
Darko Draskovic 4d09a9f4c3 Add basic business logic of training a model on a dataset and returning a model
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-18 15:08:27 +02:00
Darko Draskovic 115ccc9f7b Remove print to console statements
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 16:53:43 +02:00
Darko Draskovic 70840e2a62 Update proto files
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 15:03:43 +02:00
Darko Draskovic 83270dc9bc Move libvirt connect func to libvirt package
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 14:57:16 +02:00
Darko Draskovic 4354f0574c Add separate libvirt package and move qemu related functions to qemu package
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 14:57:16 +02:00
Darko Draskovic db991b529c Remove custom settings from go vendor system
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 14:57:16 +02:00
Darko Draskovic ec865c8d68 Add instructions for focal server preparation
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 14:57:16 +02:00
Darko Draskovic da7705de61 Refactor README to use blob for computation run req
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 14:57:16 +02:00
Darko Draskovic de9feccc51 Refactor run service func to create vm
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 14:57:16 +02:00
Darko Draskovic 0049ce2b28 Add temporary sh script to start vm used for testing
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 14:57:16 +02:00
Darko Draskovic c4cc8b48bf Remove create libvirt domain and create QEMU VM endpoints
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 14:57:16 +02:00
Darko Draskovic eb230b5aa6 Merge pull request #40 from ultravioletrs/checkproto
Bump proto version in github actions
2023-09-07 14:56:47 +02:00
Darko Draskovic e54d2731cd Bump proto version in github actions
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-07 14:15:45 +02:00
Drasko DRASKOVIC a3c4664497 Merge pull request #43 from ultravioletrs/focal
Refactor README to simplify VM setup procedure
2023-09-05 16:51:47 +02:00
Darko Draskovic 38558dbe53 Remove obsolete agent in VM configuration info
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-09-05 16:32:37 +02:00
Darko Draskovic d70d9fe7e7 Remove alpine and packer related files
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-18 17:14:08 +02:00
Darko Draskovic c3443c5375 Refactor README to simplify VM setup procedure
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-18 13:14:39 +02:00
Drasko DRASKOVIC 2cf6e1ac3e Merge pull request #10 from ultravioletrs/manger_SEV
Refactor manager to start AMD SEV encrypted VM
2023-08-16 15:21:48 +02:00
Darko Draskovic 5028946731 Replace literal strings by consts and properly format godoc comments
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-16 11:08:38 +02:00
Darko Draskovic b3ad784ee0 Add file.go and temporary file creation to enable multi instance VM launch
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-15 21:47:23 +02:00
Darko Draskovic 0b38e94787 Add uuid to VM creation config id fields
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-15 20:28:04 +02:00
Darko Draskovic 1536c6cbf5 Revert to env var VM parametrization
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-15 16:03:26 +02:00
Darko Draskovic 6b4a7080e7 Add CreateQemuVM endoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-15 15:32:42 +02:00
Darko Draskovic 4137ad3513 Amend VM launch troubleshooting instructions with port forwaring settings
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-15 11:53:55 +02:00
Darko Draskovic 5558a4f72b Add ExecutableAndArgs generator func to qemu and qemuCmd struct to service
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-14 22:32:36 +02:00
Darko Draskovic 08f26af910 Add VM launch troubleshooting instructions
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-14 20:45:26 +02:00
Darko Draskovic 88e05aea0b Refactor README to use focal-server-cloudimg-amd64.img
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-14 16:22:56 +02:00
Darko Draskovic ce6131e5fd Add programmatic qemu bin abs path config
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-13 01:47:32 +02:00
Darko Draskovic 475004cc84 Add use sudo option to qemu command ctor
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-13 01:03:51 +02:00
Darko Draskovic 51e69d8ab4 Replace sh script for ctor of qemu cmd line with go code
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-12 23:14:36 +02:00
Darko Draskovic d7aca933bd Move cmd funcs to internal/cmd.go
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-10 16:49:26 +02:00
Darko Draskovic 7e21a17317 Remove tmp file creation and capture stdout with buffer
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-10 10:57:34 +02:00
Darko Draskovic 1012d2cb0e Add sudo env var
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-09 16:05:40 +02:00
Darko Draskovic 3caaf6a631 Add a tmp file to read qemu cmd line constr output
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-09 15:57:37 +02:00
Darko Draskovic 9c545d3c7c Replace cmd.Output() with cmd.Start()
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-08 18:10:47 +02:00
Darko Draskovic fd68c9712b Merge pull request #36 from ultravioletrs/CLIReadme
Write Readme.md for Agent CLI
2023-08-08 17:13:19 +02:00
fbugarski dab603fb5f Makefile clean, and Readme installation part fix.
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-08-08 16:50:25 +02:00
Darko Draskovic ff47905247 Remove sudo necessity
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-08 14:29:14 +02:00
fbugarski 44f742fdb0 Add missing parts to Readme
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-08-07 18:39:42 +02:00
Darko Draskovic 6294117de8 Add ConstructQemuCommand and fix guestgrpc bug in sh
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-07 18:37:44 +02:00
Darko Draskovic ada7cb96b7 Parse env vars for qemu cmd line config
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-07 17:46:46 +02:00
Darko Draskovic f6d5b616d2 Add uefi bios vars, exec cmd line and log to file parameters
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-08-07 17:21:14 +02:00
fbugarski a29737b296 Add instalation to Readme
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-08-07 16:53:27 +02:00
fbugarski 5762136b0a fix remarks
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-08-07 15:57:25 +02:00
fbugarski 95a6957f4e Organize help output
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-08-04 16:33:05 +02:00
fbugarski 38cdf9405b remove extra code from makefile
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-08-04 13:39:59 +02:00
fbugarski 1a1785e1c7 Write Readme.md for Agent CLI
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-08-04 13:19:44 +02:00
Darko Draskovic b92806c3b5 Refactor dom.xml to start focal-server-cloudimg-amd64.qcow2
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-07-31 17:32:59 +02:00
Darko Draskovic b92c698800 Add host-guest http and grpc port parametrization
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-07-31 12:23:49 +02:00
Darko Draskovic eaef7ddf9a Add proper formatting
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-07-31 12:23:49 +02:00
Darko Draskovic 10e2a6d0da Refactor manager to start AMD SEV encrypted VM
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-07-31 12:23:49 +02:00
Drasko DRASKOVIC 6aff8ab0bd Merge pull request #26 from ultravioletrs/NOISSUE-update
NOISSUE - Change Default Port For GRPC Client To 7001
2023-07-28 15:49:11 +02:00
rodneyosodo 4dac375b87 Change Default Port For Client To 7001 as in main
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-27 13:51:25 +03:00
Darko Draskovic 376f204f54 Merge pull request #25 from ultravioletrs/NOISSUE-update
NOISSUE - Update Version of Agent
2023-07-27 12:40:59 +02:00
rodneyosodo 2847a28a0f Update Version of Agent
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-27 13:35:16 +03:00
Darko Draskovic c2240066f9 Merge pull request #33 from ultravioletrs/NOISSUE-cleanup
NOISSUE - Update ldflags
2023-07-27 12:29:42 +02:00
Darko Draskovic f561f6c4c9 Merge pull request #34 from ultravioletrs/NOISSUE-update
NOISSUE - Upgrade Go and Dependencies
2023-07-27 12:29:07 +02:00
Darko Draskovic a7ce9b1d87 Merge pull request #23 from ultravioletrs/NOISSUE-cleanup
DRAFT - Update ProtoC workflow
2023-07-27 12:28:12 +02:00
Darko Draskovic c9a7695b64 Merge pull request #24 from ultravioletrs/NOISSUE-update
DRAFT - Upgrade Go and Dependencies
2023-07-27 12:27:21 +02:00
rodneyosodo 15d97f64cb Upgrade Go and Dependencies
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-27 12:23:45 +03:00
rodneyosodo b82b2c6269 Upgrade Go and Dependencies
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-27 12:20:28 +03:00
rodneyosodo f73c829d27 Update ProtoC workflow
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-27 12:10:59 +03:00
rodneyosodo 20962362b6 Update ldflags
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-27 12:06:34 +03:00
Darko Draskovic ac4d20e07e Merge pull request #22 from ultravioletrs/NOISSUE-rmmfxkit
NOISSUE - Remove MFXKIT Dep
2023-07-27 11:02:18 +02:00
Darko Draskovic bd884cbcc4 Merge pull request #32 from ultravioletrs/NOISSUE-rmmfxkit
NOISSUE - Remove Docker Composition
2023-07-27 11:00:07 +02:00
rodneyosodo f5f645db0a Remove MFXKIT Dep
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-27 11:59:59 +03:00
rodneyosodo d802f65450 Remove Env File
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-27 11:29:44 +03:00
rodneyosodo f6f7152015 Remove Docker Composition
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-27 11:20:15 +03:00
Darko Draskovic 9747a0b18d Merge pull request #31 from ultravioletrs/NOISSUE-rmmfxkit
NOISSUE - Remove MFXKIT Dep
2023-07-26 17:04:55 +02:00
rodneyosodo 0bf5985f5b Rename Package Name
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-26 18:03:24 +03:00
rodneyosodo d1b40f9a06 Remove MFXKIT Dep
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-26 18:01:14 +03:00
Darko Draskovic 972e15376f Merge pull request #30 from ultravioletrs/NOISSUE-cleanup
NOISSUE - Update Protocol Buffer workflow
2023-07-26 16:59:25 +02:00
rodneyosodo d2fff75874 Update ProtoC workflow
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-26 15:34:29 +03:00
Darko Draskovic 5c3413ff74 Merge pull request #21 from ultravioletrs/NOISSUE-servicelayertracing
NOISSUE - Add Service Layer Tracing
2023-07-26 13:17:46 +02:00
rodneyosodo 897e58faec Update MANAGER_JAEGER_URL
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-26 14:06:41 +03:00
rodneyosodo f5d3b7c15b Move From OpenTracing To OpenTelemetry
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 18:56:53 +03:00
Darko Draskovic 0d3630a31b Merge pull request #29 from ultravioletrs/NOISSUE-servicelayertracing
NOISSUE - Add Service Layer Tracing
2023-07-25 17:44:25 +02:00
rodneyosodo 8ea5dda282 Propagate context
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 18:38:16 +03:00
rodneyosodo c88d77ac6f Add context to agent functions
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 17:16:51 +03:00
rodneyosodo e46a2f137b Move From OpenTracing To OpenTelemetry
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 17:03:29 +03:00
rodneyosodo 60070be4d5 Add Service Layer Tracing
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 16:24:27 +03:00
rodneyosodo fb596ab1ee Add Service Layer Tracing
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 16:07:18 +03:00
Darko Draskovic f0e7c183e0 Merge pull request #20 from ultravioletrs/NOISSUE-agentclient
NOISSUE - Connect To Agent
2023-07-25 14:58:08 +02:00
rodneyosodo f379123aa7 Remove Empty Line
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 15:51:48 +03:00
rodneyosodo 54072fc052 Connect To Agent
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 15:42:17 +03:00
Darko Draskovic 2175ae3f1a Merge pull request #19 from ultravioletrs/NOISSUE-grpcclient
NOISSUE - Add GRPC Client
2023-07-25 14:33:24 +02:00
Darko Draskovic a903ca23cd Merge pull request #27 from ultravioletrs/NOISSUE-grpcclient
NOISSUE - Add GRPC Client
2023-07-25 13:55:43 +02:00
rodneyosodo d5ac5fd3a9 Add GRPC Client
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 14:39:48 +03:00
rodneyosodo 7fec10fb56 Remove IsSecure
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 14:38:50 +03:00
rodneyosodo b6809fa5b2 Rename newClient
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 14:32:15 +03:00
rodneyosodo 9e2af2c525 Rename agentGRPCConfig
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 13:58:52 +03:00
rodneyosodo 9c74a0e7db Add GRPC Client
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 13:08:24 +03:00
Darko Draskovic 7c9a5315a7 Merge pull request #18 from ultravioletrs/NOISSUE-updateserver
NOISSUE - Move HTTP and GRPC Server Implementations To Internal
2023-07-25 10:45:43 +02:00
Darko Draskovic 5bce3a8839 Merge pull request #25 from ultravioletrs/NOISSUE-updateserver
NOISSUE - Move HTTP and GRPC Server Implementations To Internal
2023-07-25 10:36:58 +02:00
rodneyosodo 9484a44eb2 Rename stopAllServers
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 10:51:07 +03:00
rodneyosodo 150b7ab81c Move HTTP and GRPC Server Implementations To Internal
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 21:32:06 +03:00
rodneyosodo a1b4a21cb6 Reformat stopAllServer
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 19:42:11 +03:00
rodneyosodo b82c62d9b4 Add Inline Err Check
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 19:21:05 +03:00
rodneyosodo 94a22d2282 Use Channel of Size 1
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 19:06:49 +03:00
rodneyosodo 7ac8372720 Rename StopHandler
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 18:57:44 +03:00
rodneyosodo 76b2bed007 Fix Remarks
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 15:52:00 +03:00
rodneyosodo 885f4c9be1 Move HTTP and GRPC Server Implementations To Internal
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 14:57:05 +03:00
Darko Draskovic 03d637b5bf Merge pull request #17 from ultravioletrs/NOISSUE-updateMF
NOISSUE - Update Mainflux Dependency
2023-07-24 11:41:26 +02:00
Darko Draskovic 0f15d8c731 Merge pull request #16 from ultravioletrs/NOISSUE-metric
NOISSUE - Add Metric To Internal Package
2023-07-24 11:40:51 +02:00
rodneyosodo 0d9aa5d88e Update Mainflux Module
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 12:29:45 +03:00
rodneyosodo 300cabbaca Add Metric To Internal Package
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 12:25:19 +03:00
Darko Draskovic bb88a71eef Merge pull request #24 from ultravioletrs/NOISSUE-updateMF
NOISSUE - Update Mainflux Dependency
2023-07-24 11:22:10 +02:00
rodneyosodo 8505deedc6 Update Mainflux Module
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 12:16:30 +03:00
Darko Draskovic 6cecedd1a1 Merge pull request #23 from ultravioletrs/NOISSUE-metric
NOISSUE - Add Metric To Internal Package
2023-07-24 11:15:11 +02:00
rodneyosodo c09383ac72 Add Metric To Internal Package
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 12:00:28 +03:00
Darko Draskovic 8c87ef947b Merge pull request #15 from ultravioletrs/NOISSUE-env
NOISSUE - Change How To Load Env Variables
2023-07-21 18:25:17 +02:00
Darko Draskovic 3a57c8d7a4 Merge pull request #22 from ultravioletrs/NOISSUE-env
NOISSUE - Change How To Load Env Variables
2023-07-21 18:24:46 +02:00
rodneyosodo 1829afd123 Change How To Load Env Variables
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-21 19:20:35 +03:00
rodneyosodo 2c354c3977 Remove MFXKIT Env Vars
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-21 19:18:26 +03:00
rodneyosodo d59b9293cb Initialixe Epty Vars More Idomatic
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-21 19:02:54 +03:00
rodneyosodo b5e64528b1 Make config items Public
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-21 18:55:30 +03:00
rodneyosodo f18e82efa7 Change How To Load Env Variables
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-21 18:20:40 +03:00
Drasko DRASKOVIC 5b60f079b5 Merge pull request #12 from ultravioletrs/docs
Amend README with default domain creation instructions
2023-07-19 13:35:50 +02:00
Darko Draskovic 52e681c28d Amend README with default domain creation instructions
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-07-19 10:43:25 +02:00
Darko Draskovic a3619098dc Merge pull request #19 from ultravioletrs/CLISDK
Connect CLI and SDK
2023-07-17 17:45:14 +02:00
fbugarski bd61a3bd9f Remove println
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-17 16:59:30 +02:00
fbugarski f4ff0feb3c fix err check
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-17 16:39:01 +02:00
fbugarski cc078a1910 Add err check
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-17 16:32:22 +02:00
fbugarski 8f0fd5a1fa Add run, and fix response ,and loglevel
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-17 16:23:22 +02:00
fbugarski 311dc7e1a7 Fix load config
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-17 12:53:46 +02:00
fbugarski cfb0918c95 Remove unused
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-17 12:36:42 +02:00
fbugarski 74c9c306e6 Fix logger error and parse duration
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-17 12:28:15 +02:00
fbugarski 31b1f397d6 Move cfg to main
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-14 18:09:48 +02:00
fbugarski 30d6f97976 Remove unused
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-14 14:16:32 +02:00
fbugarski 5b1403edfc Agent timeout
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-14 13:49:27 +02:00
fbugarski 2ec9df770c Fix grpc connection
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-14 11:21:35 +02:00
Darko Draskovic 62455e35b2 Merge pull request #9 from ultravioletrs/NOISSUE-ci
NOISSUE - Add CI
2023-07-13 09:40:25 +02:00
rodneyosodo 074186dd55 Add Error Check
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-12 19:06:44 +03:00
fbugarski 9a33eb828c Connect CLI and SDK
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-12 14:33:47 +02:00
Darko Draskovic 5938760f8f Merge pull request #16 from ultravioletrs/NOISSUE-gotidy
NOISSUE - Add CI
2023-07-12 14:21:41 +02:00
rodneyosodo 2c54aa78d3 Update Build Command
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-12 13:34:56 +03:00
rodneyosodo d32fd050f7 Initial Commit: Add CI
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-10 15:12:00 +03:00
rodneyosodo f4e71e74b2 Initial Commit: Add CI
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-10 15:06:45 +03:00
Dušan Borovčanin 45a92ec842 Merge pull request #18 from ultravioletrs/agentsdk
Agent service SDK
2023-07-07 12:53:20 +02:00
fbugarski 87ea08b08a Add comments and fix url's
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-07 12:43:34 +02:00
fbugarski 0b4fbd5efe Implement SDK funcs
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-07 10:49:17 +02:00
fbugarski f3cc5848ef DRAFT - Agent service SDK
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-07-07 06:24:43 +02:00
Darko Draskovic 71b5617ab3 Add wait time for Linux VM boot
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-26 17:49:30 +02:00
Drasko DRASKOVIC a917c4ba0c Merge pull request #13 from ultravioletrs/cli_openapi
Add open api spec for Agent CLI
2023-06-26 16:05:47 +02:00
fbugarski 93b47a5897 Add initial structs for CLI commands
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-06-26 11:53:02 +02:00
Drasko DRASKOVIC 6bd89b08fd Merge pull request #15 from ultravioletrs/c_onciseness
Change client to c for consistency and conciseness
2023-06-22 19:46:38 +02:00
Darko Draskovic 6b7a81799c Change client to c for consistency and conciseness
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-22 19:41:16 +02:00
Darko Draskovic 05ce62c158 Merge pull request #14 from ultravioletrs/grpc_algo_data_result
Add grpc server stub for algo, data and result
2023-06-22 19:37:55 +02:00
Darko Draskovic b5c38308f4 Add Result grpc endpoint and service stub
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-22 14:51:05 +02:00
Darko Draskovic 1c140ba4ee Fix grpc client encode run func to start with user domain req
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-22 11:11:35 +02:00
Darko Draskovic 5277f44e41 Add Algo and Data grpc client funcs
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-21 15:34:26 +02:00
Darko Draskovic 4036dd3c85 Add grpc server stub for algo, data and result
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-21 13:52:45 +02:00
Drasko DRASKOVIC 62234dc7b1 Merge pull request #12 from ultravioletrs/cocosagent.service
Add systemd cocos agent init script
2023-06-20 15:33:40 +02:00
Darko Draskovic ce4726081a Rename cocosagent.service to cocos-agent.service
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-20 15:31:23 +02:00
Darko Draskovic 5296340326 Use /run/cocos for agent.pid file and create /var/log/cocos via systemd script
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-20 15:29:07 +02:00
Darko Draskovic d4e3811640 Use different stderr and stdout agent log files
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-20 15:03:52 +02:00
Darko Draskovic c8d2e96437 Fix Alpine script and README to use /cocos/agent name and location
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-20 12:48:44 +02:00
Darko Draskovic 53370ba0f6 Add /var/log/agent and /cocos dirs to systemd script
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-19 17:02:59 +02:00
Darko Draskovic 5a03d2ae4a Remove /status endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-19 16:32:46 +02:00
Darko Draskovic 5496af3af2 Remove redundant setup-cocosagent.sh script
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-19 16:07:45 +02:00
Darko Draskovic 01a9f27deb Remove /computation manifest and add /result endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-19 14:34:29 +02:00
Darko Draskovic d212c77e22 Replace algorithm/dataset endpoint with algorithms/datasets/endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-19 14:16:20 +02:00
Darko Draskovic 4be83a9888 Add open api spec for Agent CLI
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-19 12:50:33 +02:00
Darko Draskovic 1e9fa759dd Add log files and setup script
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-19 12:15:21 +02:00
Darko Draskovic 066c8ecfe9 Add systemd cococ agent init script
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-15 10:48:43 +02:00
Darko Draskovic f2c58adef6 Add instructions for checking if ports on guest are open
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-14 16:19:19 +02:00
Darko Draskovic 72b56ee40a Merge pull request #8 from ultravioletrs/update-gomod
NOISSUE - Update Version For Agent to Match Latest
2023-06-14 10:26:56 +02:00
rodneyosodo b7c6eadf4f Update Version For Agent to Match Latest
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-06-14 10:24:46 +03:00
Darko Draskovic e05aca0b74 Add instructions related to VM management
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-13 20:41:59 +02:00
Darko Draskovic 49feeb96ee Add instructions for checking if ports on guest are open
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-12 15:54:17 +02:00
Darko Draskovic 5030a20af6 Remove redundant agent dir
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-12 15:18:18 +02:00
Darko Draskovic 7b18467800 Remove redundant manager dir
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-30 14:17:34 +02:00
Darko Draskovic a6d0734212 Refactor run request to use slice of bytes to represent computation
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-30 11:27:43 +02:00
Darko Draskovic d3acfd1718 Add agent gRPC client
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-30 11:26:45 +02:00
Darko Draskovic a150433982 Add instructions related to VM management
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-26 12:54:44 +02:00
Darko Draskovic cea3e059fe Add packer json for alpine qcow2 creation
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-23 14:11:55 +02:00
Darko Draskovic 5e2ee315f6 Add instructions related to VM management
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-19 14:34:55 +02:00
Darko Draskovic 79ce0e34f9 Add instructions for interaction with cocos computations microservice
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-19 14:34:07 +02:00
Darko Draskovic 90cea0d143 Add copy files and run computation instructions
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-18 18:42:51 +02:00
Darko Draskovic 2e8603208b Add fun to replace rel with abs paths in xml files
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-18 15:31:00 +02:00
Darko Draskovic 4191560130 Fix wrong xml filepath creation
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-17 15:14:17 +02:00
Darko Draskovic e9e9b8d15c Fix VM creation error when snapshot does not exist
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-17 14:43:37 +02:00
Darko Draskovic f02a500c2d Fix agent rc script to use grpc addr visible from host
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-17 14:03:53 +02:00
Darko Draskovic c1f9fa6328 Add agent grpc svc and client and Run endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-15 16:24:58 +02:00
Darko Draskovic 750bed0d76 Add run svc endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-12 12:52:00 +02:00
Darko Draskovic 306fa302dc Add OpenRC agent script
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-04-18 16:14:23 +02:00
Darko Draskovic 780c620b30 Add Makefile targets for build and copy to VM agent
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-04-18 10:12:30 +02:00
Darko Draskovic 7383186143 Add libvirt funcs to revert to snapshot
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-04-11 14:43:11 +02:00
Darko Draskovic 023d28ea22 Fix vol and dom config to use qcow2 format to enable snapshots
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-04-10 18:17:59 +02:00
Darko Draskovic 16fb3710ac Add Run client endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-04-10 17:28:10 +02:00
Darko Draskovic fe7c3d9014 Add Run gRPC endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-31 17:17:02 +02:00
Darko Draskovic eb4a0943f6 Add Run endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-31 16:55:32 +02:00
Darko Draskovic b400210738 Add gRPC server start routine
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-29 15:27:22 +02:00
Darko Draskovic 7c12be6a24 Add grpc client skeleton
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-29 11:02:04 +02:00
Darko Draskovic 6f014156b3 Add grpc server skeleton
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-28 17:17:12 +02:00
Darko Draskovic 122b214299 Remove ping endpoint and svc func
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-28 15:07:15 +02:00
Darko Draskovic 8831a4f957 Add create qemu domain endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-28 14:10:01 +02:00
Darko Draskovic e25abcf5e0 Replace 'cocosvm' with 'manager'
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-27 17:01:03 +02:00
Darko Draskovic ad60e95736 Rename mfxkit to agent
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-27 16:37:52 +02:00
Darko Draskovic 6ec53d074f Add CLI skeleton
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-27 15:40:14 +02:00
Darko Draskovic 35c62cb273 first commit 2023-03-27 13:56:05 +02:00
Darko Draskovic 4d3bcf99b4 Move xml strings to xml files
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-22 14:01:17 +01:00
Darko Draskovic 39c5cc2dd5 Add virtual machine creation
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-15 15:38:31 +01:00
Darko Draskovic 278331dc81 Add libvirt
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-15 11:50:39 +01:00
Darko Draskovic b7a1fbb2fa Rename mfxkit to cocos
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-14 15:47:26 +01:00
Darko Draskovic 22c743c4dd first commit 2023-03-14 15:12:23 +01:00
1518 changed files with 29166 additions and 456979 deletions
+49
View File
@@ -0,0 +1,49 @@
name: Bug Report
description: File a bug/issue report. Make sure to search to see if an issue already exists for the bug you encountered.
title: "Bug: <title>"
labels: ["bug", "needs-review", "help wanted"]
body:
- type: textarea
attributes:
label: What were you trying to achieve?
description: A clear and concise description of what the bug is.
validations:
required: true
- type: textarea
attributes:
label: What are the expected results?
description: A concise description of what you expected to happen.
validations:
required: true
- type: textarea
attributes:
label: What are the received results?
description: A concise description of what you received.
validations:
required: true
- type: textarea
attributes:
label: Steps To Reproduce
description: What are the steps to reproduce the issue?
placeholder: |
1. In this environment...
2. With this config...
3. Run '...'
4. See error...
validations:
required: false
- type: textarea
attributes:
label: In what environment did you encounter the issue?
description: A concise description of the environment you encountered the issue in.
validations:
required: true
- type: textarea
attributes:
label: Additional information you deem important
description: |
Links? References? Anything that will give us more context about the issue you are encountering!
Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
validations:
required: false
+1
View File
@@ -0,0 +1 @@
blank_issues_enabled: false
@@ -0,0 +1,36 @@
name: Feature Request
description: File a feature request. Make sure to search to see if a request already exists for the feature you are requesting.
title: "Feature: <title>"
labels: ["enchancement", "needs-review"]
body:
- type: textarea
attributes:
label: Is your feature request related to a problem? Please describe.
description: A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
validations:
required: true
- type: textarea
attributes:
label: Describe the feature you are requesting, as well as the possible use case(s) for it.
description: A clear and concise description of what you want to happen.
validations:
required: true
- type: dropdown
attributes:
label: Indicate the importance of this feature to you.
description: This will help us prioritize the feature request.
options:
- Must-have
- Should-have
- Nice-to-have
validations:
required: true
- type: textarea
attributes:
label: Anything else?
description: |
Links? References? Anything that will give us more context about the feature that you are requesting.
Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
validations:
required: false
+66
View File
@@ -0,0 +1,66 @@
<!--
Pull request title should be `COCOS-XXX - description` or `NOISSUE - description` where XXX is ID of the issue that this PR relate to.
Please review the [CONTRIBUTING.md](https://github.com/ultravioletrs/cocos/blob/master/CONTRIBUTING.md) file for detailed contributing guidelines.
For Work In Progress Pull Requests, please use the Draft PR feature, see https://github.blog/2019-02-14-introducing-draft-pull-requests/ for further details.
For a timely review/response, please avoid force-pushing additional commits if your PR already received reviews or comments.
- Provide tests for your changes.
- Use descriptive commit messages.
- Comment your code where appropriate.
- Squash your commits
- Update any related documentation.
-->
# What type of PR is this?
<!--This represents the type of PR you are submitting.
For example:
This is a bug fix because it fixes the following issue: #1234
This is a feature because it adds the following functionality: ...
This is a refactor because it changes the following functionality: ...
This is a documentation update because it updates the following documentation: ...
This is a dependency update because it updates the following dependencies: ...
This is an optimization because it improves the following functionality: ...
-->
## What does this do?
<!--
Please provide a brief description of what this PR is intended to do.
Include List any changes that modify/break current functionality.
-->
## Which issue(s) does this PR fix/relate to?
<!--
For pull requests that relate or close an issue, please include them below. We like to follow [Github's guidance on linking issues to pull requests](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue).
For example having the text: "Resolves #1234" would connect the current pull request to issue 1234. And when we merge the pull request, Github will automatically close the issue.
-->
- Related Issue #
- Resolves #
## Have you included tests for your changes?
<!--If you have not included tests, please explain why.
For example:
Yes, I have included tests for my changes.
No, I have not included tests because I do not know how to.
-->
## Did you document any new/modified feature?
<!--If you have not included documentation, please explain why.
For example:
Yes, I have updated the documentation for the new feature.
No, I have not updated the documentation because I do not know how to.
-->
### Notes
<!--Please provide any additional information you feel is important.-->
+22
View File
@@ -0,0 +1,22 @@
version: 2
updates:
- package-ecosystem: "cargo"
directory: "/scripts/attestation_policy"
schedule:
interval: "weekly"
day: "monday"
groups:
rs-dependencies:
patterns:
- "*"
- package-ecosystem: "gomod"
directories:
- "/"
schedule:
interval: "weekly"
day: "monday"
groups:
go-dependency:
patterns:
- "*"
+82
View File
@@ -0,0 +1,82 @@
name: CI
on:
push:
branches:
- main
paths:
- "manager/manager.proto"
- "pkg/manager/*.pb.go"
- "agent/agent.proto"
- "agent/*.pb.go"
pull_request:
branches:
- main
paths:
- "manager/manager.proto"
- "pkg/manager/*.pb.go"
- "agent/agent.proto"
- "agent/*.pb.go"
jobs:
checkproto:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.23.x
- name: Set up protoc
run: |
PROTOC_VERSION=28.1
PROTOC_GEN_VERSION=v1.34.2
PROTOC_GRPC_VERSION=v1.5.1
# Download and install protoc
PROTOC_ZIP=protoc-$PROTOC_VERSION-linux-x86_64.zip
curl -0L -o $PROTOC_ZIP https://github.com/protocolbuffers/protobuf/releases/download/v$PROTOC_VERSION/$PROTOC_ZIP
unzip -o $PROTOC_ZIP -d protoc3
sudo mv protoc3/bin/* /usr/local/bin/
sudo mv protoc3/include/* /usr/local/include/
rm -rf $PROTOC_ZIP protoc3
# Install protoc-gen-go and protoc-gen-go-grpc
go install google.golang.org/protobuf/cmd/protoc-gen-go@$PROTOC_GEN_VERSION
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@$PROTOC_GRPC_VERSION
# Add protoc to the PATH
export PATH=$PATH:/usr/local/bin/protoc
- name: Set up Cocos-AI
run: |
# Rename .pb.go files to .pb.go.tmp to prevent conflicts
for p in $(ls pkg/manager/*.pb.go); do
mv $p $p.tmp
done
for p in $(ls agent/*.pb.go); do
mv $p $p.tmp
done
# Generate Go files from protobuf definitions
make protoc
# Compare generated Go files with the original ones
for p in $(ls pkg/manager/*.pb.go); do
if ! cmp -s $p $p.tmp; then
echo "Proto file and generated Go file $p are out of sync!"
exit 1
fi
done
for p in $(ls agent/*.pb.go); do
if ! cmp -s $p $p.tmp; then
echo "Proto file and generated Go file $p are out of sync!"
exit 1
fi
done
-39
View File
@@ -1,39 +0,0 @@
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
test:
strategy:
matrix:
go-version: [1.19.x]
os: [ubuntu-latest]
runs-on: ${{ matrix.os }}
steps:
- name: Install Go
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
- name: Checkout code
uses: actions/checkout@v2
- name: Run go vet
run: go vet ./...
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
with:
version: latest
args: --no-config --disable-all --enable gosimple --enable govet --enable unused --enable deadcode --timeout 3m
- name: Build
run: go build -v ./...
- name: Run tests
run: go test -mod=vendor -v --race -covermode=atomic -coverprofile cover.out ./...
+73
View File
@@ -0,0 +1,73 @@
name: Build and Release
on:
push:
tags:
- '*'
jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 120
steps:
- name: Free Disk Space
run: |
sudo apt-get clean
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf "/usr/local/share/boost"
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
df -h
- name: Update Ubuntu
run: |
sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get clean
sudo apt-get autoremove -y
df -h
- name: Maximize build space
uses: easimon/maximize-build-space@master
with:
root-reserve-mb: 35000
swap-size-mb: 1024
remove-dotnet: 'true'
remove-android: 'true'
- name: Check free space
run: |
echo "Free space:"
df -h
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.23.x
cache-dependency-path: "go.sum"
- name: Checkout cocos
uses: actions/checkout@v4
with:
repository: 'ultravioletrs/cocos'
path: cocos
- name: Checkout buildroot
uses: actions/checkout@v4
with:
repository: 'buildroot/buildroot'
path: buildroot
ref: 2024.11-rc2
- name: Build
run: |
cd buildroot
make BR2_EXTERNAL=../cocos/hal/linux cocos_defconfig
make
- name: Release
uses: softprops/action-gh-release@v2
with:
files: |
buildroot/output/images/bzImage
buildroot/output/images/rootfs.cpio.gz
+61
View File
@@ -0,0 +1,61 @@
name: CI
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
ci:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.23.x
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v1.60
- name: Build
run: |
make
- name: Create coverage directory
run: |
mkdir coverage
- name: Run Agent tests
run: go test -v --race -covermode=atomic -coverprofile coverage/agent.out ./agent/...
- name: Run cli tests
run: go test -v --race -covermode=atomic -coverprofile coverage/cli.out ./cli/...
- name: Run cmd tests
run: go test -v --race -covermode=atomic -coverprofile coverage/cmd.out ./cmd/...
- name: Run internal tests
run: go test -v --race -covermode=atomic -coverprofile coverage/internal.out ./internal/...
- name: Run pkg tests
run: go test -v --race -covermode=atomic -coverprofile coverage/pkg.out ./pkg/...
- name: Run manager tests
run: sudo go test -v --race -covermode=atomic -coverprofile coverage/manager.out ./manager/...
- name: Upload results to Codecov
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage/*.out
codecov_yml_path: codecov.yml
verbose: true
+41
View File
@@ -0,0 +1,41 @@
name: Rust CI Pipeline
on:
push:
branches:
- main
paths:
- "scripts/attestation_policy/**"
- ".github/workflows/rust.yaml"
pull_request:
branches:
- main
paths:
- "scripts/attestation_policy/**"
- ".github/workflows/rust.yaml"
env:
CARGO_TERM_COLOR: always
jobs:
rust-check:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./scripts/attestation_policy
steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Check cargo
run: cargo check --release --all-targets
- name: Check formatting
run: cargo fmt --all -- --check
- name: Run linter
run: cargo clippy -- -D warnings
- name: Build for all features
run: cargo build --release --all-features
+26
View File
@@ -1 +1,27 @@
build
cmd/manager/img
.cov
*.pem
dist/
*.zip
*.spec
*.tar
# Generated by Cargo
# will have compiled files and executables
debug/
target/
# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
Cargo.lock
# These are backup files generated by rustfmt
**/*.rs.bk
# MSVC Windows builds of rustc generate these, which store debugging information
*.pdb
+80
View File
@@ -0,0 +1,80 @@
run:
timeout: 3m
issues:
max-issues-per-linter: 10
max-same-issues: 10
linters-settings:
importas:
no-unaliased: true
no-extra-aliases: false
alias:
- pkg: github.com/absmach/magistrala/logger
alias: mglog
gocritic:
enabled-checks:
- dupImport
- importShadow
- httpNoBody
- paramTypeCombine
- emptyStringTest
- builtinShadow
- exposedSyncMutex
disabled-checks:
- appendAssign
enabled-tags:
- diagnostic
disabled-tags:
- performance
- style
- experimental
- opinionated
stylecheck:
checks: ["-ST1000", "-ST1003", "-ST1020", "-ST1021", "-ST1022"]
goheader:
template: |-
Copyright (c) Ultraviolet
SPDX-License-Identifier: Apache-2.0
linters:
disable-all: true
enable:
- importas
- gocritic
- gosimple
- errcheck
- govet
- unused
- goconst
- godot
- godox
- ineffassign
- misspell
- stylecheck
- whitespace
- gci
- gofmt
- goimports
- loggercheck
- goheader
- asasalint
- asciicheck
- bidichk
- contextcheck
- decorder
- dogsled
- errchkjson
- errname
- execinquery
- copyloopvar
- ginkgolinter
- gocheckcompilerdirectives
- gofumpt
- goprintffuncname
- importas
- makezero
- mirror
- nakedret
- dupword
@@ -1,11 +1,11 @@
# Contributing to Mainflux
# Contributing to CoCos
The following is a set of guidelines to contribute to Mainflux and its libraries, which are
hosted on the [Mainflux Organization](https://github.com/mainflux) on GitHub.
The following is a set of guidelines to contribute to CoCos and its libraries, which are
hosted on the [Ultraviolet Organization](https://github.com/ultravioletrs) on GitHub.
This project adheres to the [Contributor Covenant 1.2](http://contributor-covenant.org/version/1/2/0).
By participating, you are expected to uphold this code. Please report unacceptable behavior to
[abuse@mainflux.com](mailto:abuse@mainflux.com).
[abuse@ultraviolet.rs](mailto:abuse@ultraviolet.rs).
## Reporting issues
@@ -13,7 +13,7 @@ Reporting issues are a great way to contribute to the project. We are perpetuall
thorough bug report.
Before raising a new issue, check [our issue
list](https://github.com/mainflux/mainflux/issues) to determine if it already contains the
list](https://github.com/ultravioletrs/cocos/issues) to determine if it already contains the
problem that you are facing.
A good bug report shouldn't leave others needing to chase you for more information. Please be as detailed as possible. The following questions might serve as a template for writing a detailed
@@ -41,23 +41,23 @@ To contribute to the project, [fork](https://help.github.com/articles/fork-a-rep
clone your fork repository, and configure the remotes:
```
git clone https://github.com/<your-username>/mainflux.git
cd mainflux
git remote add upstream https://github.com/mainflux/mainflux.git
git clone https://github.com/<your-username>/cocos.git
cd cocos
git remote add upstream https://github.com/ultravioletrs/cocos.git
```
If your cloned repository is behind the upstream commits, then get the latest changes from upstream:
```
git checkout master
git pull --rebase upstream master
git pull --rebase upstream main
```
Create a new topic branch from `master` using the naming convention `MF-[issue-number]`
Create a new topic branch from `master` using the naming convention `COCOS-[issue-number]`
to help us keep track of your contribution scope:
```
git checkout -b MF-[issue-number]
git checkout -b COCOS-[issue-number]
```
Commit your changes in logical chunks. When you are ready to commit, make sure
@@ -74,13 +74,13 @@ and `user.email` git configs, you can sign your commit automatically with `git c
Locally merge (or rebase) the upstream development branch into your topic branch:
```
git pull --rebase upstream master
git pull --rebase upstream main
```
Push your topic branch up to your fork:
```
git push origin MF-[issue-number]
git push origin COCOS-[issue-number]
```
[Open a Pull Request](https://help.github.com/articles/using-pull-requests/) with a clear title
+4 -14
View File
@@ -1,6 +1,7 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
https://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
@@ -175,24 +176,13 @@
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Copyright 2015-2020 Mainflux
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
+44 -63
View File
@@ -1,81 +1,62 @@
COCOS_DOCKER_IMAGE_NAME_PREFIX ?= cocos
BUILD_DIR = build
SERVICES = computations datasets
DOCKERS = $(addprefix docker_,$(SERVICES))
DOCKERS_DEV = $(addprefix docker_dev_,$(SERVICES))
CGO_ENABLED ?= 0
SERVICES = manager agent cli
ATTESTATION_POLICY = attestation_policy
CGO_ENABLED ?= 1
GOARCH ?= amd64
VERSION ?= $(shell git describe --abbrev=0 --tags)
VERSION ?= $(shell git describe --abbrev=0 --tags --always)
COMMIT ?= $(shell git rev-parse HEAD)
TIME ?= $(shell date +%F_%T)
EMBED_ENABLED ?= 0
INSTALL_DIR ?= /usr/local/bin
CONFIG_DIR ?= /etc/cocos
SERVICE_NAME ?= cocos-manager
SERVICE_DIR ?= /etc/systemd/system
SERVICE_FILE = init/systemd/$(SERVICE_NAME).service
define compile_service
CGO_ENABLED=$(CGO_ENABLED) GOOS=$(GOOS) GOARCH=$(GOARCH) GOARM=$(GOARM) \
go build -mod=vendor -ldflags "-s -w \
-X 'github.com/ultravioletrs/cocos/internal/http.BuildTime=$(TIME)' \
-X 'github.com/ultravioletrs/cocos/internal/http.Version=$(VERSION)' \
-X 'github.com/ultravioletrs/cocos/internal/http.Commit=$(COMMIT)'" \
go build -ldflags "-s -w \
-X 'github.com/absmach/magistrala.BuildTime=$(TIME)' \
-X 'github.com/absmach/magistrala.Version=$(VERSION)' \
-X 'github.com/absmach/magistrala.Commit=$(COMMIT)'" \
$(if $(filter 1,$(EMBED_ENABLED)),-tags "embed",) \
-o ${BUILD_DIR}/cocos-$(1) cmd/$(1)/main.go
endef
define make_docker
$(eval svc=$(subst docker_,,$(1)))
docker build \
--no-cache \
--build-arg SVC=$(svc) \
--build-arg GOARCH=$(GOARCH) \
--build-arg GOARM=$(GOARM) \
--build-arg VERSION=$(VERSION) \
--build-arg COMMIT=$(COMMIT) \
--build-arg TIME=$(TIME) \
--tag=$(COCOS_DOCKER_IMAGE_NAME_PREFIX)/$(svc) \
-f docker/Dockerfile .
endef
define make_docker_dev
$(eval svc=$(subst docker_dev_,,$(1)))
docker build \
--no-cache \
--build-arg SVC=$(svc) \
--tag=$(COCOS_DOCKER_IMAGE_NAME_PREFIX)/$(svc) \
-f docker/Dockerfile.dev ./build
endef
.PHONY: all $(SERVICES) $(ATTESTATION_POLICY) install clean
all: $(SERVICES)
.PHONY: all $(SERVICES) dockers dockers_dev latest release
$(SERVICES):
$(call compile_service,$@)
$(ATTESTATION_POLICY):
$(MAKE) -C ./scripts/attestation_policy
protoc:
protoc -I. --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative agent/agent.proto
protoc -I. --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative manager/manager.proto
protoc -I. --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative agent/events/events.proto
mocks:
mockery --config ./mockery.yml
install: $(SERVICES)
install -d $(INSTALL_DIR)
install $(BUILD_DIR)/cocos-cli $(INSTALL_DIR)/cocos-cli
install $(BUILD_DIR)/cocos-manager $(INSTALL_DIR)/cocos-manager
install -d $(CONFIG_DIR)
install cocos-manager.env $(CONFIG_DIR)/cocos-manager.env
clean:
rm -rf ${BUILD_DIR}
rm -rf $(BUILD_DIR)
cleandocker:
# Stops containers and removes containers, networks, volumes, and images created by up
docker-compose -f docker/docker-compose.yml down --rmi all -v --remove-orphans
run: install_service
sudo systemctl start $(SERVICE_NAME).service
ifdef pv
# Remove unused volumes
docker volume ls -f name=$(MF_DOCKER_IMAGE_NAME_PREFIX) -f dangling=true -q | xargs -r docker volume rm
endif
stop:
sudo systemctl stop $(SERVICE_NAME).service
install:
cp ${BUILD_DIR}/* $(GOBIN)
test:
go test -mod=vendor -v -race -count 1 -tags test $(shell go list ./... | grep -v 'vendor\|cmd')
$(SERVICES):
$(call compile_service,$(@))
$(DOCKERS):
$(call make_docker,$(@),$(GOARCH))
$(DOCKERS_DEV):
$(call make_docker_dev,$(@))
dockers: $(DOCKERS)
dockers_dev: $(DOCKERS_DEV)
run:
docker-compose -f docker/docker-compose.yml up
install_service:
sudo install -m 644 $(SERVICE_FILE) $(SERVICE_DIR)/$(SERVICE_NAME).service
sudo systemctl daemon-reload
+34 -37
View File
@@ -1,4 +1,8 @@
# Cocos AI
[![codecov](https://codecov.io/gh/ultravioletrs/cocos/graph/badge.svg?token=HX01LR01K9)](https://codecov.io/gh/ultravioletrs/cocos)
![Go report card](https://goreportcard.com/badge/github.com/ultravioletrs/cocos)
[Cocos AI (Confdential Computing System for AI/ML)][cocos] is a platform for secure multiparty computation (SMPC)
based on the [Confidential Computing][cc] and [Trusted Execution Environments (TEEs)][tee].
@@ -7,62 +11,55 @@ based on the [Confidential Computing][cc] and [Trusted Execution Environments (T
</p>
With Cocos AI it becomes possible to run AI/ML workloads on combined datasets from multiple organizations
while guaranteeing the privacy and security of the data and the algorithms.
while guaranteeing the privacy and security of the data and the algorithm.
Data is always encrypted, protected by hardware secure enclaves (Trusted Execution Environments),
attested via secure remote attestation protocols, and invisible to cloud processors or any other
3rd party to which computation is offloaded.
## Install
The following prerequisites are needed to run Cocos:
## Features
- [Docker](https://docs.docker.com/install/)
- [Docker compose](https://docs.docker.com/compose/install/)
Cocos AI is implementing the following features:
### Build Docker Images
Currenty, there is no Docker registry, so Cocos images must be built by hand:
```
make dockers
```
For this you might be needing to setup DNS servers in your `/etc/resolf.conf` as explained
[here](https://github.com/docker/cli/issues/2618) (i.e. add Google's `nameserver 8.8.8.8`).
### Run Composition
Once the images are built (`docker images` command should show you `cocos/computations` and `cocos/datasets`),
composition can be run:
```bash
docker-compose -f docker/docker-compose.yml up
```
This will bring up the Cocos docker services and interconnect them. This command can also be executed using the project's included Makefile:
```bash
make run
```
- TEE enablement, deployment and monitoring (secure VM manager)
- HAL for TEEs based on hardened Linux kernel, secure bootloader and custom-tailored embedded rootfs for minimal TCB
- In-enclave agent, netowrking controller and other system software
- Encrypted asynchronous data transfer and result delivery
- API for programmable platform manipulation
- HW and SW supported attestation with verification tools
- CLI for system interaction
## Usage
The quickest way to start using Cocos is via the CLI. The latest version can be downloaded from the [official releases page][rel].
It can also be built and used from the project's root directory:
Clone the repo and create binaries:
```bash
make cli
./build/cocos-cli version
git clone git@github.com:ultravioletrs/cocos.git
make
```
Additional details on using the CLI can be found in the [CLI documentation](https://docs.mainflux.io/cli).
This will create 3 binaries:
```bash
ls build/
# cocos-agent cocos-cli cocos-manager
```
- Manager can be deployed on the AMD SEV-SNP host
- Agent can be built into [EOS][eos]-based HAL
- CLI can be used to communicate to remote Agent.
## Documentation
Official documentation is hosted at [Cocos official docs page][docs]. Documentation is auto-generated, checkout the instructions on [official docs repository](https://github.com/cocos/docs).
Project documentation is hosted at [Cocos AI official docs page][docs].
Documentation is generated from the [docs repository](https://github.com/ultravioletrs/docs).
## License
Cocos AI is a proprietary product created by Ultraviolet company.
Cocos AI is published under permissive open-source [Apache-2.0](LICENSE) license.
[cc]: https://confidentialcomputing.io/white-papers-reports/
[cocos]: https://cocos.ai/
[rel]: https://github.com/ultraviolet/cocos/releases
[rel]: https://github.com/ultravioletrs/cocos/releases
[tee]: https://en.wikipedia.org/wiki/Trusted_execution_environment
[docs]: https://docs.cocos.ultraviolet.rs
[cli]: https://docs.cocos.ultraviolet.rs/cli
[eos]: https://github.com/ultravioletrs/eos
+39
View File
@@ -0,0 +1,39 @@
# Agent
Agent service provides a barebones HTTP and gRPC API and Service interface implementation for the development of the agent service.
## Configuration
The service is configured using the environment variables from the following table. Note that any unset variables will be replaced with their default values.
| Variable | Description | Default |
| ----------------------------- | ------------------------------------------------------ | ------------------------------ |
| AGENT_LOG_LEVEL | Log level for agent service (debug, info, warn, error) | info |
| AGENT_GRPC_HOST | Agent service gRPC host | "" |
| AGENT_GRPC_PORT | Agent service gRPC port | 7002 |
| AGENT_GRPC_SERVER_CERT | Path to gRPC server certificate in pem format | "" |
| AGENT_GRPC_SERVER_KEY | Path to gRPC server key in pem format | "" |
| AGENT_GRPC_SERVER_CA_CERTS | Path to gRPC server CA certificate | "" |
| AGENT_GRPC_CLIENT_CA_CERTS | Path to gRPC client CA certificate | "" |
## Deployment
To start the service outside of the container, execute the following shell script:
```bash
# Download the latest version of the service
git clone git@github.com:ultravioletrs/cocos.git
cd cocos
# Compile the service
make agent
# Run the service
./build/cocos-agent
```
## Usage
For more information about service capabilities and its usage, please check out the [README documentation](../README.md).
+596
View File
@@ -0,0 +1,596 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.34.2
// protoc v5.28.1
// source: agent/agent.proto
package agent
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type AlgoRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Algorithm []byte `protobuf:"bytes,1,opt,name=algorithm,proto3" json:"algorithm,omitempty"`
Requirements []byte `protobuf:"bytes,2,opt,name=requirements,proto3" json:"requirements,omitempty"`
}
func (x *AlgoRequest) Reset() {
*x = AlgoRequest{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_agent_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *AlgoRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*AlgoRequest) ProtoMessage() {}
func (x *AlgoRequest) ProtoReflect() protoreflect.Message {
mi := &file_agent_agent_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use AlgoRequest.ProtoReflect.Descriptor instead.
func (*AlgoRequest) Descriptor() ([]byte, []int) {
return file_agent_agent_proto_rawDescGZIP(), []int{0}
}
func (x *AlgoRequest) GetAlgorithm() []byte {
if x != nil {
return x.Algorithm
}
return nil
}
func (x *AlgoRequest) GetRequirements() []byte {
if x != nil {
return x.Requirements
}
return nil
}
type AlgoResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
}
func (x *AlgoResponse) Reset() {
*x = AlgoResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_agent_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *AlgoResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*AlgoResponse) ProtoMessage() {}
func (x *AlgoResponse) ProtoReflect() protoreflect.Message {
mi := &file_agent_agent_proto_msgTypes[1]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use AlgoResponse.ProtoReflect.Descriptor instead.
func (*AlgoResponse) Descriptor() ([]byte, []int) {
return file_agent_agent_proto_rawDescGZIP(), []int{1}
}
type DataRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Dataset []byte `protobuf:"bytes,1,opt,name=dataset,proto3" json:"dataset,omitempty"`
Filename string `protobuf:"bytes,2,opt,name=filename,proto3" json:"filename,omitempty"`
}
func (x *DataRequest) Reset() {
*x = DataRequest{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_agent_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *DataRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*DataRequest) ProtoMessage() {}
func (x *DataRequest) ProtoReflect() protoreflect.Message {
mi := &file_agent_agent_proto_msgTypes[2]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use DataRequest.ProtoReflect.Descriptor instead.
func (*DataRequest) Descriptor() ([]byte, []int) {
return file_agent_agent_proto_rawDescGZIP(), []int{2}
}
func (x *DataRequest) GetDataset() []byte {
if x != nil {
return x.Dataset
}
return nil
}
func (x *DataRequest) GetFilename() string {
if x != nil {
return x.Filename
}
return ""
}
type DataResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
}
func (x *DataResponse) Reset() {
*x = DataResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_agent_proto_msgTypes[3]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *DataResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*DataResponse) ProtoMessage() {}
func (x *DataResponse) ProtoReflect() protoreflect.Message {
mi := &file_agent_agent_proto_msgTypes[3]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use DataResponse.ProtoReflect.Descriptor instead.
func (*DataResponse) Descriptor() ([]byte, []int) {
return file_agent_agent_proto_rawDescGZIP(), []int{3}
}
type ResultRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
}
func (x *ResultRequest) Reset() {
*x = ResultRequest{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_agent_proto_msgTypes[4]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *ResultRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*ResultRequest) ProtoMessage() {}
func (x *ResultRequest) ProtoReflect() protoreflect.Message {
mi := &file_agent_agent_proto_msgTypes[4]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use ResultRequest.ProtoReflect.Descriptor instead.
func (*ResultRequest) Descriptor() ([]byte, []int) {
return file_agent_agent_proto_rawDescGZIP(), []int{4}
}
type ResultResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
File []byte `protobuf:"bytes,1,opt,name=file,proto3" json:"file,omitempty"`
}
func (x *ResultResponse) Reset() {
*x = ResultResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_agent_proto_msgTypes[5]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *ResultResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*ResultResponse) ProtoMessage() {}
func (x *ResultResponse) ProtoReflect() protoreflect.Message {
mi := &file_agent_agent_proto_msgTypes[5]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use ResultResponse.ProtoReflect.Descriptor instead.
func (*ResultResponse) Descriptor() ([]byte, []int) {
return file_agent_agent_proto_rawDescGZIP(), []int{5}
}
func (x *ResultResponse) GetFile() []byte {
if x != nil {
return x.File
}
return nil
}
type AttestationRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
ReportData []byte `protobuf:"bytes,1,opt,name=report_data,json=reportData,proto3" json:"report_data,omitempty"` // Should be of length 64.
}
func (x *AttestationRequest) Reset() {
*x = AttestationRequest{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_agent_proto_msgTypes[6]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *AttestationRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*AttestationRequest) ProtoMessage() {}
func (x *AttestationRequest) ProtoReflect() protoreflect.Message {
mi := &file_agent_agent_proto_msgTypes[6]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use AttestationRequest.ProtoReflect.Descriptor instead.
func (*AttestationRequest) Descriptor() ([]byte, []int) {
return file_agent_agent_proto_rawDescGZIP(), []int{6}
}
func (x *AttestationRequest) GetReportData() []byte {
if x != nil {
return x.ReportData
}
return nil
}
type AttestationResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
File []byte `protobuf:"bytes,1,opt,name=file,proto3" json:"file,omitempty"`
}
func (x *AttestationResponse) Reset() {
*x = AttestationResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_agent_proto_msgTypes[7]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *AttestationResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*AttestationResponse) ProtoMessage() {}
func (x *AttestationResponse) ProtoReflect() protoreflect.Message {
mi := &file_agent_agent_proto_msgTypes[7]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use AttestationResponse.ProtoReflect.Descriptor instead.
func (*AttestationResponse) Descriptor() ([]byte, []int) {
return file_agent_agent_proto_rawDescGZIP(), []int{7}
}
func (x *AttestationResponse) GetFile() []byte {
if x != nil {
return x.File
}
return nil
}
var File_agent_agent_proto protoreflect.FileDescriptor
var file_agent_agent_proto_rawDesc = []byte{
0x0a, 0x11, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2f, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x12, 0x05, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x22, 0x4f, 0x0a, 0x0b, 0x41, 0x6c,
0x67, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1c, 0x0a, 0x09, 0x61, 0x6c, 0x67,
0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x09, 0x61, 0x6c,
0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x22, 0x0a, 0x0c, 0x72, 0x65, 0x71, 0x75, 0x69,
0x72, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x0c, 0x72,
0x65, 0x71, 0x75, 0x69, 0x72, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x22, 0x0e, 0x0a, 0x0c, 0x41,
0x6c, 0x67, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x43, 0x0a, 0x0b, 0x44,
0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x18, 0x0a, 0x07, 0x64, 0x61,
0x74, 0x61, 0x73, 0x65, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x64, 0x61, 0x74,
0x61, 0x73, 0x65, 0x74, 0x12, 0x1a, 0x0a, 0x08, 0x66, 0x69, 0x6c, 0x65, 0x6e, 0x61, 0x6d, 0x65,
0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x66, 0x69, 0x6c, 0x65, 0x6e, 0x61, 0x6d, 0x65,
0x22, 0x0e, 0x0a, 0x0c, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x22, 0x0f, 0x0a, 0x0d, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x22, 0x24, 0x0a, 0x0e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f,
0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28,
0x0c, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x22, 0x35, 0x0a, 0x12, 0x41, 0x74, 0x74, 0x65, 0x73,
0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1f, 0x0a,
0x0b, 0x72, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x5f, 0x64, 0x61, 0x74, 0x61, 0x18, 0x01, 0x20, 0x01,
0x28, 0x0c, 0x52, 0x0a, 0x72, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x44, 0x61, 0x74, 0x61, 0x22, 0x29,
0x0a, 0x13, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x73,
0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x01, 0x20,
0x01, 0x28, 0x0c, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x32, 0xfd, 0x01, 0x0a, 0x0c, 0x41, 0x67,
0x65, 0x6e, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x33, 0x0a, 0x04, 0x41, 0x6c,
0x67, 0x6f, 0x12, 0x12, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41, 0x6c, 0x67, 0x6f, 0x52,
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x13, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41,
0x6c, 0x67, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x28, 0x01, 0x12,
0x33, 0x0a, 0x04, 0x44, 0x61, 0x74, 0x61, 0x12, 0x12, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e,
0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x13, 0x2e, 0x61, 0x67,
0x65, 0x6e, 0x74, 0x2e, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x22, 0x00, 0x28, 0x01, 0x12, 0x39, 0x0a, 0x06, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x12, 0x14,
0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52, 0x65, 0x71,
0x75, 0x65, 0x73, 0x74, 0x1a, 0x15, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x52, 0x65, 0x73,
0x75, 0x6c, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x30, 0x01, 0x12,
0x48, 0x0a, 0x0b, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x19,
0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69,
0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1a, 0x2e, 0x61, 0x67, 0x65, 0x6e,
0x74, 0x2e, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x73,
0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x30, 0x01, 0x42, 0x09, 0x5a, 0x07, 0x2e, 0x2f, 0x61,
0x67, 0x65, 0x6e, 0x74, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (
file_agent_agent_proto_rawDescOnce sync.Once
file_agent_agent_proto_rawDescData = file_agent_agent_proto_rawDesc
)
func file_agent_agent_proto_rawDescGZIP() []byte {
file_agent_agent_proto_rawDescOnce.Do(func() {
file_agent_agent_proto_rawDescData = protoimpl.X.CompressGZIP(file_agent_agent_proto_rawDescData)
})
return file_agent_agent_proto_rawDescData
}
var file_agent_agent_proto_msgTypes = make([]protoimpl.MessageInfo, 8)
var file_agent_agent_proto_goTypes = []any{
(*AlgoRequest)(nil), // 0: agent.AlgoRequest
(*AlgoResponse)(nil), // 1: agent.AlgoResponse
(*DataRequest)(nil), // 2: agent.DataRequest
(*DataResponse)(nil), // 3: agent.DataResponse
(*ResultRequest)(nil), // 4: agent.ResultRequest
(*ResultResponse)(nil), // 5: agent.ResultResponse
(*AttestationRequest)(nil), // 6: agent.AttestationRequest
(*AttestationResponse)(nil), // 7: agent.AttestationResponse
}
var file_agent_agent_proto_depIdxs = []int32{
0, // 0: agent.AgentService.Algo:input_type -> agent.AlgoRequest
2, // 1: agent.AgentService.Data:input_type -> agent.DataRequest
4, // 2: agent.AgentService.Result:input_type -> agent.ResultRequest
6, // 3: agent.AgentService.Attestation:input_type -> agent.AttestationRequest
1, // 4: agent.AgentService.Algo:output_type -> agent.AlgoResponse
3, // 5: agent.AgentService.Data:output_type -> agent.DataResponse
5, // 6: agent.AgentService.Result:output_type -> agent.ResultResponse
7, // 7: agent.AgentService.Attestation:output_type -> agent.AttestationResponse
4, // [4:8] is the sub-list for method output_type
0, // [0:4] is the sub-list for method input_type
0, // [0:0] is the sub-list for extension type_name
0, // [0:0] is the sub-list for extension extendee
0, // [0:0] is the sub-list for field type_name
}
func init() { file_agent_agent_proto_init() }
func file_agent_agent_proto_init() {
if File_agent_agent_proto != nil {
return
}
if !protoimpl.UnsafeEnabled {
file_agent_agent_proto_msgTypes[0].Exporter = func(v any, i int) any {
switch v := v.(*AlgoRequest); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_agent_agent_proto_msgTypes[1].Exporter = func(v any, i int) any {
switch v := v.(*AlgoResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_agent_agent_proto_msgTypes[2].Exporter = func(v any, i int) any {
switch v := v.(*DataRequest); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_agent_agent_proto_msgTypes[3].Exporter = func(v any, i int) any {
switch v := v.(*DataResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_agent_agent_proto_msgTypes[4].Exporter = func(v any, i int) any {
switch v := v.(*ResultRequest); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_agent_agent_proto_msgTypes[5].Exporter = func(v any, i int) any {
switch v := v.(*ResultResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_agent_agent_proto_msgTypes[6].Exporter = func(v any, i int) any {
switch v := v.(*AttestationRequest); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_agent_agent_proto_msgTypes[7].Exporter = func(v any, i int) any {
switch v := v.(*AttestationResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_agent_agent_proto_rawDesc,
NumEnums: 0,
NumMessages: 8,
NumExtensions: 0,
NumServices: 1,
},
GoTypes: file_agent_agent_proto_goTypes,
DependencyIndexes: file_agent_agent_proto_depIdxs,
MessageInfos: file_agent_agent_proto_msgTypes,
}.Build()
File_agent_agent_proto = out.File
file_agent_agent_proto_rawDesc = nil
file_agent_agent_proto_goTypes = nil
file_agent_agent_proto_depIdxs = nil
}
+44
View File
@@ -0,0 +1,44 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
syntax = "proto3";
package agent;
option go_package = "./agent";
service AgentService {
rpc Algo(stream AlgoRequest) returns (AlgoResponse) {}
rpc Data(stream DataRequest) returns (DataResponse) {}
rpc Result(ResultRequest) returns (stream ResultResponse) {}
rpc Attestation(AttestationRequest) returns (stream AttestationResponse) {}
}
message AlgoRequest {
bytes algorithm = 1;
bytes requirements = 2;
}
message AlgoResponse {}
message DataRequest {
bytes dataset = 1;
string filename = 2;
}
message DataResponse {}
message ResultRequest {
}
message ResultResponse {
bytes file = 1;
}
message AttestationRequest {
bytes report_data = 1; // Should be of length 64.
}
message AttestationResponse {
bytes file = 1;
}
+230
View File
@@ -0,0 +1,230 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.5.1
// - protoc v5.28.1
// source: agent/agent.proto
package agent
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.64.0 or later.
const _ = grpc.SupportPackageIsVersion9
const (
AgentService_Algo_FullMethodName = "/agent.AgentService/Algo"
AgentService_Data_FullMethodName = "/agent.AgentService/Data"
AgentService_Result_FullMethodName = "/agent.AgentService/Result"
AgentService_Attestation_FullMethodName = "/agent.AgentService/Attestation"
)
// AgentServiceClient is the client API for AgentService service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type AgentServiceClient interface {
Algo(ctx context.Context, opts ...grpc.CallOption) (grpc.ClientStreamingClient[AlgoRequest, AlgoResponse], error)
Data(ctx context.Context, opts ...grpc.CallOption) (grpc.ClientStreamingClient[DataRequest, DataResponse], error)
Result(ctx context.Context, in *ResultRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[ResultResponse], error)
Attestation(ctx context.Context, in *AttestationRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[AttestationResponse], error)
}
type agentServiceClient struct {
cc grpc.ClientConnInterface
}
func NewAgentServiceClient(cc grpc.ClientConnInterface) AgentServiceClient {
return &agentServiceClient{cc}
}
func (c *agentServiceClient) Algo(ctx context.Context, opts ...grpc.CallOption) (grpc.ClientStreamingClient[AlgoRequest, AlgoResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &AgentService_ServiceDesc.Streams[0], AgentService_Algo_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[AlgoRequest, AlgoResponse]{ClientStream: stream}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type AgentService_AlgoClient = grpc.ClientStreamingClient[AlgoRequest, AlgoResponse]
func (c *agentServiceClient) Data(ctx context.Context, opts ...grpc.CallOption) (grpc.ClientStreamingClient[DataRequest, DataResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &AgentService_ServiceDesc.Streams[1], AgentService_Data_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[DataRequest, DataResponse]{ClientStream: stream}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type AgentService_DataClient = grpc.ClientStreamingClient[DataRequest, DataResponse]
func (c *agentServiceClient) Result(ctx context.Context, in *ResultRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[ResultResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &AgentService_ServiceDesc.Streams[2], AgentService_Result_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[ResultRequest, ResultResponse]{ClientStream: stream}
if err := x.ClientStream.SendMsg(in); err != nil {
return nil, err
}
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type AgentService_ResultClient = grpc.ServerStreamingClient[ResultResponse]
func (c *agentServiceClient) Attestation(ctx context.Context, in *AttestationRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[AttestationResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &AgentService_ServiceDesc.Streams[3], AgentService_Attestation_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[AttestationRequest, AttestationResponse]{ClientStream: stream}
if err := x.ClientStream.SendMsg(in); err != nil {
return nil, err
}
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type AgentService_AttestationClient = grpc.ServerStreamingClient[AttestationResponse]
// AgentServiceServer is the server API for AgentService service.
// All implementations must embed UnimplementedAgentServiceServer
// for forward compatibility.
type AgentServiceServer interface {
Algo(grpc.ClientStreamingServer[AlgoRequest, AlgoResponse]) error
Data(grpc.ClientStreamingServer[DataRequest, DataResponse]) error
Result(*ResultRequest, grpc.ServerStreamingServer[ResultResponse]) error
Attestation(*AttestationRequest, grpc.ServerStreamingServer[AttestationResponse]) error
mustEmbedUnimplementedAgentServiceServer()
}
// UnimplementedAgentServiceServer must be embedded to have
// forward compatible implementations.
//
// NOTE: this should be embedded by value instead of pointer to avoid a nil
// pointer dereference when methods are called.
type UnimplementedAgentServiceServer struct{}
func (UnimplementedAgentServiceServer) Algo(grpc.ClientStreamingServer[AlgoRequest, AlgoResponse]) error {
return status.Errorf(codes.Unimplemented, "method Algo not implemented")
}
func (UnimplementedAgentServiceServer) Data(grpc.ClientStreamingServer[DataRequest, DataResponse]) error {
return status.Errorf(codes.Unimplemented, "method Data not implemented")
}
func (UnimplementedAgentServiceServer) Result(*ResultRequest, grpc.ServerStreamingServer[ResultResponse]) error {
return status.Errorf(codes.Unimplemented, "method Result not implemented")
}
func (UnimplementedAgentServiceServer) Attestation(*AttestationRequest, grpc.ServerStreamingServer[AttestationResponse]) error {
return status.Errorf(codes.Unimplemented, "method Attestation not implemented")
}
func (UnimplementedAgentServiceServer) mustEmbedUnimplementedAgentServiceServer() {}
func (UnimplementedAgentServiceServer) testEmbeddedByValue() {}
// UnsafeAgentServiceServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to AgentServiceServer will
// result in compilation errors.
type UnsafeAgentServiceServer interface {
mustEmbedUnimplementedAgentServiceServer()
}
func RegisterAgentServiceServer(s grpc.ServiceRegistrar, srv AgentServiceServer) {
// If the following call pancis, it indicates UnimplementedAgentServiceServer was
// embedded by pointer and is nil. This will cause panics if an
// unimplemented method is ever invoked, so we test this at initialization
// time to prevent it from happening at runtime later due to I/O.
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
t.testEmbeddedByValue()
}
s.RegisterService(&AgentService_ServiceDesc, srv)
}
func _AgentService_Algo_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(AgentServiceServer).Algo(&grpc.GenericServerStream[AlgoRequest, AlgoResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type AgentService_AlgoServer = grpc.ClientStreamingServer[AlgoRequest, AlgoResponse]
func _AgentService_Data_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(AgentServiceServer).Data(&grpc.GenericServerStream[DataRequest, DataResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type AgentService_DataServer = grpc.ClientStreamingServer[DataRequest, DataResponse]
func _AgentService_Result_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(ResultRequest)
if err := stream.RecvMsg(m); err != nil {
return err
}
return srv.(AgentServiceServer).Result(m, &grpc.GenericServerStream[ResultRequest, ResultResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type AgentService_ResultServer = grpc.ServerStreamingServer[ResultResponse]
func _AgentService_Attestation_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(AttestationRequest)
if err := stream.RecvMsg(m); err != nil {
return err
}
return srv.(AgentServiceServer).Attestation(m, &grpc.GenericServerStream[AttestationRequest, AttestationResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type AgentService_AttestationServer = grpc.ServerStreamingServer[AttestationResponse]
// AgentService_ServiceDesc is the grpc.ServiceDesc for AgentService service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var AgentService_ServiceDesc = grpc.ServiceDesc{
ServiceName: "agent.AgentService",
HandlerType: (*AgentServiceServer)(nil),
Methods: []grpc.MethodDesc{},
Streams: []grpc.StreamDesc{
{
StreamName: "Algo",
Handler: _AgentService_Algo_Handler,
ClientStreams: true,
},
{
StreamName: "Data",
Handler: _AgentService_Data_Handler,
ClientStreams: true,
},
{
StreamName: "Result",
Handler: _AgentService_Result_Handler,
ServerStreams: true,
},
{
StreamName: "Attestation",
Handler: _AgentService_Attestation_Handler,
ServerStreams: true,
},
},
Metadata: "agent/agent.proto",
}
+29
View File
@@ -0,0 +1,29 @@
// Code generated by "stringer -type=AgentEvent"; DO NOT EDIT.
package agent
import "strconv"
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[Start-0]
_ = x[ManifestReceived-1]
_ = x[AlgorithmReceived-2]
_ = x[DataReceived-3]
_ = x[RunComplete-4]
_ = x[ResultsConsumed-5]
_ = x[RunFailed-6]
}
const _AgentEvent_name = "StartManifestReceivedAlgorithmReceivedDataReceivedRunCompleteResultsConsumedRunFailed"
var _AgentEvent_index = [...]uint8{0, 5, 21, 38, 50, 61, 76, 85}
func (i AgentEvent) String() string {
if i < 0 || i >= AgentEvent(len(_AgentEvent_index)-1) {
return "AgentEvent(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _AgentEvent_name[_AgentEvent_index[i]:_AgentEvent_index[i+1]]
}
+30
View File
@@ -0,0 +1,30 @@
// Code generated by "stringer -type=AgentState"; DO NOT EDIT.
package agent
import "strconv"
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[Idle-0]
_ = x[ReceivingManifest-1]
_ = x[ReceivingAlgorithm-2]
_ = x[ReceivingData-3]
_ = x[Running-4]
_ = x[ConsumingResults-5]
_ = x[Complete-6]
_ = x[Failed-7]
}
const _AgentState_name = "IdleReceivingManifestReceivingAlgorithmReceivingDataRunningConsumingResultsCompleteFailed"
var _AgentState_index = [...]uint8{0, 4, 21, 39, 52, 59, 75, 83, 89}
func (i AgentState) String() string {
if i < 0 || i >= AgentState(len(_AgentState_index)-1) {
return "AgentState(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _AgentState_name[_AgentState_index[i]:_AgentState_index[i+1]]
}
+49
View File
@@ -0,0 +1,49 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package algorithm
import (
"context"
"google.golang.org/grpc/metadata"
)
type AlgorithType string
const (
AlgoTypeBin AlgorithType = "bin"
AlgoTypePython AlgorithType = "python"
AlgoTypeWasm AlgorithType = "wasm"
AlgoTypeDocker AlgorithType = "docker"
AlgoTypeKey = "algo_type"
AlgoArgsKey = "algo_args"
ResultsDir = "results"
DatasetsDir = "datasets"
AlgoWorkingDir = "/cocos"
)
func AlgorithmTypeToContext(ctx context.Context, algoType string) context.Context {
return metadata.AppendToOutgoingContext(ctx, AlgoTypeKey, algoType)
}
func AlgorithmTypeFromContext(ctx context.Context) string {
return metadata.ValueFromIncomingContext(ctx, AlgoTypeKey)[0]
}
func AlgorithmArgsToContext(ctx context.Context, algoArgs []string) context.Context {
for _, arg := range algoArgs {
ctx = metadata.AppendToOutgoingContext(ctx, AlgoArgsKey, arg)
}
return ctx
}
func AlgorithmArgsFromContext(ctx context.Context) []string {
return metadata.ValueFromIncomingContext(ctx, AlgoArgsKey)
}
// Algorithm is an interface that specifies the API for an algorithm.
type Algorithm interface {
// Run executes the algorithm and returns the result.
Run() error
}
+48
View File
@@ -0,0 +1,48 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package binary
import (
"fmt"
"io"
"log/slog"
"os/exec"
"github.com/ultravioletrs/cocos/agent/algorithm"
"github.com/ultravioletrs/cocos/agent/algorithm/logging"
"github.com/ultravioletrs/cocos/agent/events"
)
var _ algorithm.Algorithm = (*binary)(nil)
type binary struct {
algoFile string
stderr io.Writer
stdout io.Writer
args []string
}
func NewAlgorithm(logger *slog.Logger, eventsSvc events.Service, algoFile string, args []string) algorithm.Algorithm {
return &binary{
algoFile: algoFile,
stderr: &logging.Stderr{Logger: logger, EventSvc: eventsSvc},
stdout: &logging.Stdout{Logger: logger},
args: args,
}
}
func (b *binary) Run() error {
cmd := exec.Command(b.algoFile, b.args...)
cmd.Stderr = b.stderr
cmd.Stdout = b.stdout
if err := cmd.Start(); err != nil {
return fmt.Errorf("error starting algorithm: %v", err)
}
if err := cmd.Wait(); err != nil {
return fmt.Errorf("algorithm execution error: %v", err)
}
return nil
}
+100
View File
@@ -0,0 +1,100 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package binary
import (
"bytes"
"log/slog"
"os"
"testing"
"github.com/ultravioletrs/cocos/agent/algorithm/logging"
"github.com/ultravioletrs/cocos/agent/events/mocks"
)
func TestNewAlgorithm(t *testing.T) {
logger := slog.New(slog.NewTextHandler(os.Stdout, nil))
eventsSvc := new(mocks.Service)
algoFile := "/path/to/algo"
args := []string{"arg1", "arg2"}
algo := NewAlgorithm(logger, eventsSvc, algoFile, args)
b, ok := algo.(*binary)
if !ok {
t.Fatalf("NewAlgorithm did not return a *binary")
}
if b.algoFile != algoFile {
t.Errorf("Expected algoFile to be %s, got %s", algoFile, b.algoFile)
}
if len(b.args) != len(args) {
t.Errorf("Expected %d args, got %d", len(args), len(b.args))
}
for i, arg := range args {
if b.args[i] != arg {
t.Errorf("Expected arg %d to be %s, got %s", i, arg, b.args[i])
}
}
if _, ok := b.stderr.(*logging.Stderr); !ok {
t.Errorf("Expected stderr to be *algorithm.Stderr")
}
if _, ok := b.stdout.(*logging.Stdout); !ok {
t.Errorf("Expected stdout to be *algorithm.Stdout")
}
}
func TestBinaryRun(t *testing.T) {
tests := []struct {
name string
algoFile string
args []string
expectedError bool
}{
{
name: "Successful execution",
algoFile: "echo",
args: []string{"Hello, World!"},
expectedError: false,
},
{
name: "Non-existent binary",
algoFile: "non_existent_binary",
args: []string{},
expectedError: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
logger := slog.New(slog.NewTextHandler(os.Stdout, nil))
eventsSvc := new(mocks.Service)
b := NewAlgorithm(logger, eventsSvc, tt.algoFile, tt.args).(*binary)
var stdout, stderr bytes.Buffer
b.stdout = &stdout
b.stderr = &stderr
err := b.Run()
if tt.expectedError && err == nil {
t.Errorf("Expected an error, but got none")
}
if !tt.expectedError && err != nil {
t.Errorf("Unexpected error: %v", err)
}
if !tt.expectedError {
if stdout.Len() == 0 {
t.Errorf("Expected non-empty stdout")
}
}
})
}
}
+178
View File
@@ -0,0 +1,178 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package docker
import (
"bufio"
"context"
"fmt"
"io"
"log/slog"
"os"
"path"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/mount"
"github.com/docker/docker/client"
"github.com/ultravioletrs/cocos/agent/algorithm"
"github.com/ultravioletrs/cocos/agent/algorithm/logging"
"github.com/ultravioletrs/cocos/agent/events"
)
const (
containerName = "agent_container"
datasetsMountPath = "/cocos/datasets"
resultsMountPath = "/cocos/results"
)
var _ algorithm.Algorithm = (*docker)(nil)
type docker struct {
algoFile string
logger *slog.Logger
stderr io.Writer
stdout io.Writer
}
func NewAlgorithm(logger *slog.Logger, eventsSvc events.Service, algoFile string) algorithm.Algorithm {
d := &docker{
algoFile: algoFile,
logger: logger,
stderr: &logging.Stderr{Logger: logger, EventSvc: eventsSvc},
stdout: &logging.Stdout{Logger: logger},
}
return d
}
func (d *docker) Run() error {
ctx := context.Background()
// Create a new Docker client.
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
return fmt.Errorf("could not create a new Docker client: %v", err)
}
// Open the Docker image tar file.
imageFile, err := os.Open(d.algoFile)
if err != nil {
return fmt.Errorf("could not open Docker image: %v", err)
}
defer imageFile.Close()
// Load the Docker image from the tar file.
resp, err := cli.ImageLoad(ctx, imageFile, true)
if err != nil {
return fmt.Errorf("could not load Docker image from file: %v", err)
}
defer resp.Body.Close()
// List the loaded images to get the image ID.
images, err := cli.ImageList(ctx, image.ListOptions{})
if err != nil {
return fmt.Errorf("could not get the Docker image list: %v", err)
}
var imageID string = ""
var dockerImageName string = ""
for _, image := range images {
for _, tag := range image.RepoTags {
imageID = image.ID
dockerImageName = tag
break
}
}
if imageID == "" {
return fmt.Errorf("could not find image ID")
}
// Create and start the container.
respContainer, err := cli.ContainerCreate(ctx, &container.Config{
Image: dockerImageName,
Tty: true,
AttachStdout: true,
AttachStderr: true,
}, &container.HostConfig{
Mounts: []mount.Mount{
{
Type: mount.TypeBind,
Source: path.Join(algorithm.AlgoWorkingDir, algorithm.DatasetsDir),
Target: datasetsMountPath,
},
{
Type: mount.TypeBind,
Source: path.Join(algorithm.AlgoWorkingDir, algorithm.ResultsDir),
Target: resultsMountPath,
},
},
}, nil, nil, containerName)
if err != nil {
return fmt.Errorf("could not create a Docker container: %v", err)
}
if err := cli.ContainerStart(ctx, respContainer.ID, container.StartOptions{}); err != nil {
return fmt.Errorf("could not start a Docker container: %v", err)
}
stdout, err := cli.ContainerLogs(ctx, respContainer.ID, container.LogsOptions{ShowStdout: true, Follow: true})
if err != nil {
return fmt.Errorf("could not read stdout from the container: %v", err)
}
defer stdout.Close()
go func() {
if err := writeToOut(stdout, d.stdout); err != nil {
d.logger.Warn(fmt.Sprintf("could not write to stdout: %v", err))
}
}()
stderr, err := cli.ContainerLogs(ctx, respContainer.ID, container.LogsOptions{ShowStderr: true, Follow: true})
if err != nil {
d.logger.Warn(fmt.Sprintf("could not read stderr from the container: %v", err))
}
defer stderr.Close()
go func() {
if err := writeToOut(stderr, d.stderr); err != nil {
d.logger.Warn(fmt.Sprintf("could not write to stderr: %v", err))
}
}()
statusCh, errCh := cli.ContainerWait(ctx, respContainer.ID, container.WaitConditionNotRunning)
select {
case err := <-errCh:
if err != nil {
return fmt.Errorf("could not wait for a Docker container: %v", err)
}
case <-statusCh:
}
defer func() {
if err = cli.ContainerRemove(ctx, respContainer.ID, container.RemoveOptions{Force: true}); err != nil {
d.logger.Warn(fmt.Sprintf("error could not remove container: %v", err))
}
if _, err := cli.ImageRemove(ctx, imageID, image.RemoveOptions{Force: true}); err != nil {
d.logger.Warn(fmt.Sprintf("error could not remove image: %v", err))
}
}()
return nil
}
func writeToOut(readCloser io.ReadCloser, ioWriter io.Writer) error {
scanner := bufio.NewScanner(readCloser)
for scanner.Scan() {
if _, err := ioWriter.Write(scanner.Bytes()); err != nil {
return fmt.Errorf("error writing to output: %v", err)
}
}
if err := scanner.Err(); err != nil {
return fmt.Errorf("error reading container logs error: %v", err)
}
return nil
}
+29
View File
@@ -0,0 +1,29 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package docker
import (
"log/slog"
"os"
"testing"
"github.com/stretchr/testify/assert"
"github.com/ultravioletrs/cocos/agent/algorithm/logging"
"github.com/ultravioletrs/cocos/agent/events/mocks"
)
// TestNewAlgorithm tests the NewAlgorithm function.
func TestNewAlgorithm(t *testing.T) {
logger := slog.New(slog.NewTextHandler(os.Stdout, nil))
eventsSvc := new(mocks.Service)
algoFile := "/path/to/algo.tar"
algo := NewAlgorithm(logger, eventsSvc, algoFile)
d, ok := algo.(*docker)
assert.True(t, ok, "NewAlgorithm should return a *docker")
assert.Equal(t, algoFile, d.algoFile, "algoFile should be set correctly")
assert.NotNil(t, d.logger, "logger should be set")
assert.IsType(t, &logging.Stderr{}, d.stderr, "stderr should be of type *algorithm.Stderr")
assert.IsType(t, &logging.Stdout{}, d.stdout, "stdout should be of type *algorithm.Stdout")
}
+78
View File
@@ -0,0 +1,78 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package logging
import (
"bytes"
"encoding/json"
"io"
"log/slog"
"github.com/ultravioletrs/cocos/agent/events"
)
var (
_ io.Writer = &Stdout{}
_ io.Writer = &Stderr{}
)
const (
bufSize = 1024
algorithmRun = "AlgorithmRun"
warningStatus = "Warning"
)
type Stdout struct {
Logger *slog.Logger
}
// Write implements io.Writer.
func (s *Stdout) Write(p []byte) (n int, err error) {
inBuf := bytes.NewBuffer(p)
buf := make([]byte, bufSize)
for {
n, err := inBuf.Read(buf)
if err != nil {
if err == io.EOF {
break
}
return len(p) - inBuf.Len(), err
}
s.Logger.Debug(string(buf[:n]))
}
return len(p), nil
}
type Stderr struct {
Logger *slog.Logger
EventSvc events.Service
}
// Write implements io.Writer.
func (s *Stderr) Write(p []byte) (n int, err error) {
inBuf := bytes.NewBuffer(p)
buf := make([]byte, bufSize)
for {
n, err := inBuf.Read(buf)
if err != nil {
if err == io.EOF {
break
}
return len(p) - inBuf.Len(), err
}
s.Logger.Error(string(buf[:n]))
}
if err := s.EventSvc.SendEvent(algorithmRun, warningStatus, json.RawMessage{}); err != nil {
return len(p), err
}
return len(p), nil
}
+86
View File
@@ -0,0 +1,86 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package logging
import (
"strings"
"testing"
mglog "github.com/absmach/magistrala/logger"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/ultravioletrs/cocos/agent/events/mocks"
"github.com/ultravioletrs/cocos/pkg/manager"
)
func TestStdoutWrite(t *testing.T) {
tests := []struct {
name string
input string
expected []string
}{
{
name: "Single line",
input: "Hello, World!",
expected: []string{"Hello, World!"},
},
{
name: "Multiple lines",
input: "Line 1\nLine 2\nLine 3",
expected: []string{"Line 1\nLine 2\nLine 3"},
},
{
name: "Long input",
input: strings.Repeat("a", bufSize+100),
expected: []string{strings.Repeat("a", bufSize), strings.Repeat("a", 100)},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
stdout := &Stdout{Logger: mglog.NewMock()}
n, err := stdout.Write([]byte(tt.input))
assert.NoError(t, err)
assert.Equal(t, len(tt.input), n)
})
}
}
func TestStderrWrite(t *testing.T) {
tests := []struct {
name string
input string
expected []string
}{
{
name: "Single line",
input: "Error: Something went wrong",
expected: []string{"Error: Something went wrong"},
},
{
name: "Multiple lines",
input: "Error 1\nError 2\nError 3",
expected: []string{"Error 1\nError 2\nError 3"},
},
{
name: "Long input",
input: strings.Repeat("e", bufSize+100),
expected: []string{strings.Repeat("e", bufSize), strings.Repeat("e", 100)},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockEventService := mocks.NewService(t)
mockEventService.On("SendEvent", "AlgorithmRun", manager.Warning.String(), mock.Anything).Return(nil)
stderr := &Stderr{Logger: mglog.NewMock(), EventSvc: mockEventService}
n, err := stderr.Write([]byte(tt.input))
assert.NoError(t, err)
assert.Equal(t, len(tt.input), n)
mockEventService.AssertExpectations(t)
})
}
}
+105
View File
@@ -0,0 +1,105 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package python
import (
"context"
"fmt"
"io"
"log/slog"
"os"
"os/exec"
"path/filepath"
"github.com/ultravioletrs/cocos/agent/algorithm"
"github.com/ultravioletrs/cocos/agent/algorithm/logging"
"github.com/ultravioletrs/cocos/agent/events"
"google.golang.org/grpc/metadata"
)
const (
PyRuntime = "python3"
PyRuntimeKey = "python_runtime"
)
func PythonRunTimeToContext(ctx context.Context, runtime string) context.Context {
return metadata.AppendToOutgoingContext(ctx, PyRuntimeKey, runtime)
}
func PythonRunTimeFromContext(ctx context.Context) string {
return metadata.ValueFromIncomingContext(ctx, PyRuntimeKey)[0]
}
var _ algorithm.Algorithm = (*python)(nil)
type python struct {
algoFile string
stderr io.Writer
stdout io.Writer
runtime string
requirementsFile string
args []string
}
func NewAlgorithm(logger *slog.Logger, eventsSvc events.Service, runtime, requirementsFile, algoFile string, args []string) algorithm.Algorithm {
p := &python{
algoFile: algoFile,
stderr: &logging.Stderr{Logger: logger, EventSvc: eventsSvc},
stdout: &logging.Stdout{Logger: logger},
requirementsFile: requirementsFile,
args: args,
}
if runtime != "" {
p.runtime = runtime
} else {
p.runtime = PyRuntime
}
return p
}
func (p *python) Run() error {
venvPath := "venv"
createVenvCmd := exec.Command(p.runtime, "-m", "venv", venvPath)
createVenvCmd.Stderr = p.stderr
createVenvCmd.Stdout = p.stdout
if err := createVenvCmd.Run(); err != nil {
return fmt.Errorf("error creating virtual environment: %v", err)
}
pythonPath := filepath.Join(venvPath, "bin", "python")
updatePipCmd := exec.Command(pythonPath, "-m", "pip", "install", "--upgrade", "pip")
updatePipCmd.Stderr = p.stderr
updatePipCmd.Stdout = p.stdout
if err := updatePipCmd.Run(); err != nil {
return fmt.Errorf("error updating pip: %v", err)
}
if p.requirementsFile != "" {
rcmd := exec.Command(pythonPath, "-m", "pip", "install", "-r", p.requirementsFile)
rcmd.Stderr = p.stderr
rcmd.Stdout = p.stdout
if err := rcmd.Run(); err != nil {
return fmt.Errorf("error installing requirements: %v", err)
}
}
args := append([]string{p.algoFile}, p.args...)
cmd := exec.Command(pythonPath, args...)
cmd.Stderr = p.stderr
cmd.Stdout = p.stdout
if err := cmd.Start(); err != nil {
return fmt.Errorf("error starting algorithm: %v", err)
}
if err := cmd.Wait(); err != nil {
return fmt.Errorf("algorithm execution error: %v", err)
}
if err := os.RemoveAll(venvPath); err != nil {
return fmt.Errorf("error removing virtual environment: %v", err)
}
return nil
}
+148
View File
@@ -0,0 +1,148 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package python
import (
"bytes"
"context"
"io"
"log/slog"
"os"
"path/filepath"
"strings"
"testing"
"github.com/ultravioletrs/cocos/agent/algorithm/logging"
"github.com/ultravioletrs/cocos/agent/events/mocks"
"google.golang.org/grpc/metadata"
)
const runtime = "python3"
func TestPythonRunTimeToContext(t *testing.T) {
ctx := context.Background()
newCtx := PythonRunTimeToContext(ctx, runtime)
md, ok := metadata.FromOutgoingContext(newCtx)
if !ok {
t.Fatal("Expected metadata in context")
}
values := md.Get(PyRuntimeKey)
if len(values) != 1 || values[0] != runtime {
t.Errorf("Expected runtime %s, got %v", runtime, values)
}
}
func TestPythonRunTimeFromContext(t *testing.T) {
ctx := metadata.NewIncomingContext(context.Background(), metadata.Pairs(PyRuntimeKey, runtime))
got := PythonRunTimeFromContext(ctx)
if got != runtime {
t.Errorf("Expected runtime %s, got %s", runtime, got)
}
}
func TestNewAlgorithm(t *testing.T) {
logger := &slog.Logger{}
eventsSvc := new(mocks.Service)
requirementsFile := "requirements.txt"
algoFile := "algorithm.py"
args := []string{"--arg1", "value1"}
algo := NewAlgorithm(logger, eventsSvc, runtime, requirementsFile, algoFile, args)
p, ok := algo.(*python)
if !ok {
t.Fatal("Expected *python type")
}
if p.runtime != runtime {
t.Errorf("Expected runtime %s, got %s", runtime, p.runtime)
}
if p.requirementsFile != requirementsFile {
t.Errorf("Expected requirementsFile %s, got %s", requirementsFile, p.requirementsFile)
}
if p.algoFile != algoFile {
t.Errorf("Expected algoFile %s, got %s", algoFile, p.algoFile)
}
if len(p.args) != len(args) {
t.Errorf("Expected %d args, got %d", len(args), len(p.args))
}
}
func TestRun(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "python-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
scriptContent := []byte("print('Hello, World!')")
scriptPath := filepath.Join(tmpDir, "test_script.py")
if err := os.WriteFile(scriptPath, scriptContent, 0o644); err != nil {
t.Fatal(err)
}
eventsSvc := new(mocks.Service)
var stdout, stderr bytes.Buffer
algo := &python{
algoFile: scriptPath,
stderr: io.MultiWriter(&stderr, &logging.Stderr{Logger: slog.Default(), EventSvc: eventsSvc}),
stdout: io.MultiWriter(&stdout, &logging.Stdout{Logger: slog.Default()}),
runtime: "python3",
}
err = algo.Run()
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
expectedOutput := "Hello, World!\n"
if !strings.Contains(stdout.String(), expectedOutput) {
t.Errorf("Expected output to contain %q, got %q", expectedOutput, stdout.String())
}
}
func TestRunWithRequirements(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "python-test")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(tmpDir)
scriptContent := []byte("import requests\nprint(requests.__version__)")
scriptPath := filepath.Join(tmpDir, "test_script.py")
if err := os.WriteFile(scriptPath, scriptContent, 0o644); err != nil {
t.Fatal(err)
}
requirementsContent := []byte("requests==2.26.0")
requirementsPath := filepath.Join(tmpDir, "requirements.txt")
if err := os.WriteFile(requirementsPath, requirementsContent, 0o644); err != nil {
t.Fatal(err)
}
eventsSvc := new(mocks.Service)
var stdout, stderr bytes.Buffer
algo := &python{
algoFile: scriptPath,
requirementsFile: requirementsPath,
stderr: io.MultiWriter(&stderr, &logging.Stderr{Logger: slog.Default(), EventSvc: eventsSvc}),
stdout: io.MultiWriter(&stdout, &logging.Stdout{Logger: slog.Default()}),
runtime: "python3",
}
err = algo.Run()
if err != nil {
t.Fatalf("Unexpected error: %v", err)
}
if !strings.Contains(stdout.String(), "2.26.0") {
t.Errorf("Expected output to contain requests version 2.26.0, got %q", stdout.String())
}
}
+82
View File
@@ -0,0 +1,82 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package algorithm_test
import (
"os"
"testing"
"github.com/ultravioletrs/cocos/agent/algorithm"
"github.com/ultravioletrs/cocos/internal"
)
func TestZipDirectory(t *testing.T) {
cases := []struct {
name string
directories []string
files []string
expected []string
}{
{
name: "empty directory",
directories: []string{"testdata"},
},
{
name: "single file",
files: []string{"file1.txt"},
},
{
name: "directory with single file",
directories: []string{"testdata"},
expected: []string{"testdata/file1.txt"},
},
{
name: "directory with multiple files",
directories: []string{"testdata"},
expected: []string{
"testdata/file1.txt",
"testdata/file2.txt",
"testdata/file3.txt",
},
},
{
name: "nested directories",
directories: []string{"testdata", "testdata/nested"},
expected: []string{
"testdata/nested/file1.txt",
"testdata/nested/file2.txt",
"testdata/nested/file3.txt",
},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if err := os.Mkdir(algorithm.ResultsDir, 0o755); err != nil {
t.Fatalf("error creating results directory: %s", err.Error())
}
defer func() {
if err := os.RemoveAll(algorithm.ResultsDir); err != nil {
t.Fatalf("error removing results directory and its contents: %s", err.Error())
}
}()
for _, dir := range tc.directories {
if dir != "" {
if err := os.Mkdir(algorithm.ResultsDir+"/"+dir, 0o755); err != nil {
t.Fatalf("error creating test directory: %s", err.Error())
}
}
}
for _, file := range tc.files {
if _, err := os.Create(algorithm.ResultsDir + "/" + file); err != nil {
t.Fatalf("error creating test file: %s", err.Error())
}
}
if _, err := internal.ZipDirectoryToMemory(algorithm.ResultsDir); err != nil {
t.Errorf("ZipDirectory() error = %v", err)
}
})
}
}
+54
View File
@@ -0,0 +1,54 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package wasm
import (
"fmt"
"io"
"log/slog"
"os/exec"
"github.com/ultravioletrs/cocos/agent/algorithm"
"github.com/ultravioletrs/cocos/agent/algorithm/logging"
"github.com/ultravioletrs/cocos/agent/events"
)
const wasmRuntime = "wasmedge"
var mapDirOption = []string{"--dir", ".:" + algorithm.ResultsDir}
var _ algorithm.Algorithm = (*wasm)(nil)
type wasm struct {
algoFile string
stderr io.Writer
stdout io.Writer
args []string
}
func NewAlgorithm(logger *slog.Logger, eventsSvc events.Service, algoFile string, args []string) algorithm.Algorithm {
return &wasm{
algoFile: algoFile,
stderr: &logging.Stderr{Logger: logger, EventSvc: eventsSvc},
stdout: &logging.Stdout{Logger: logger},
args: args,
}
}
func (w *wasm) Run() error {
args := append(mapDirOption, w.algoFile)
args = append(args, w.args...)
cmd := exec.Command(wasmRuntime, args...)
cmd.Stderr = w.stderr
cmd.Stdout = w.stdout
if err := cmd.Start(); err != nil {
return fmt.Errorf("error starting algorithm: %v", err)
}
if err := cmd.Wait(); err != nil {
return fmt.Errorf("algorithm execution error: %v", err)
}
return nil
}
+89
View File
@@ -0,0 +1,89 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package wasm
import (
"log/slog"
"os"
"os/exec"
"testing"
"github.com/ultravioletrs/cocos/agent/algorithm/logging"
"github.com/ultravioletrs/cocos/agent/events/mocks"
)
func TestNewAlgorithm(t *testing.T) {
logger := slog.New(slog.NewTextHandler(os.Stdout, nil))
eventsSvc := new(mocks.Service)
algoFile := "test.wasm"
args := []string{"arg1", "arg2"}
algo := NewAlgorithm(logger, eventsSvc, algoFile, args)
w, ok := algo.(*wasm)
if !ok {
t.Fatalf("NewAlgorithm did not return a *wasm")
}
if w.algoFile != algoFile {
t.Errorf("Expected algoFile to be %s, got %s", algoFile, w.algoFile)
}
if len(w.args) != len(args) {
t.Errorf("Expected %d args, got %d", len(args), len(w.args))
}
_, ok = w.stderr.(*logging.Stderr)
if !ok {
t.Errorf("Expected stderr to be *algorithm.Stderr")
}
_, ok = w.stdout.(*logging.Stdout)
if !ok {
t.Errorf("Expected stdout to be *algorithm.Stdout")
}
}
func TestRunError(t *testing.T) {
// Mock exec.Command to return an error
execCommand = mockExecCommandError
defer func() { execCommand = exec.Command }()
logger := slog.New(slog.NewTextHandler(os.Stdout, nil))
eventsSvc := new(mocks.Service)
algoFile := "test.wasm"
args := []string{"arg1", "arg2"}
w := NewAlgorithm(logger, eventsSvc, algoFile, args).(*wasm)
err := w.Run()
if err == nil {
t.Errorf("Run() should have returned an error")
}
}
func mockExecCommand(command string, args ...string) *exec.Cmd {
cs := []string{"-test.run=TestHelperProcess", "--", command}
cs = append(cs, args...)
cmd := exec.Command(os.Args[0], cs...)
cmd.Env = []string{"GO_WANT_HELPER_PROCESS=1"}
return cmd
}
func mockExecCommandError(command string, args ...string) *exec.Cmd {
cmd := mockExecCommand(command, args...)
cmd.Env = append(cmd.Env, "GO_WANT_HELPER_PROCESS_ERROR=1")
return cmd
}
func TestHelperProcess(t *testing.T) {
if os.Getenv("GO_WANT_HELPER_PROCESS") != "1" {
return
}
if os.Getenv("GO_WANT_HELPER_PROCESS_ERROR") == "1" {
os.Exit(1)
}
os.Exit(0)
}
var execCommand = exec.Command
+6
View File
@@ -0,0 +1,6 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
// Package api contains API-related concerns: endpoint definitions, middlewares
// and all resource representations.
package api
+5
View File
@@ -0,0 +1,5 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
// Package grpc contains implementation of kit service gRPC API.
package grpc
+80
View File
@@ -0,0 +1,80 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package grpc
import (
"context"
"github.com/go-kit/kit/endpoint"
"github.com/ultravioletrs/cocos/agent"
)
func algoEndpoint(svc agent.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(algoReq)
if err := req.validate(); err != nil {
return algoRes{}, err
}
algo := agent.Algorithm{Algorithm: req.Algorithm, Requirements: req.Requirements}
err := svc.Algo(ctx, algo)
if err != nil {
return algoRes{}, err
}
return algoRes{}, nil
}
}
func dataEndpoint(svc agent.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(dataReq)
if err := req.validate(); err != nil {
return dataRes{}, err
}
dataset := agent.Dataset{Dataset: req.Dataset, Filename: req.Filename}
err := svc.Data(ctx, dataset)
if err != nil {
return dataRes{}, err
}
return dataRes{}, nil
}
}
func resultEndpoint(svc agent.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(resultReq)
if err := req.validate(); err != nil {
return resultRes{}, err
}
file, err := svc.Result(ctx)
if err != nil {
return resultRes{}, err
}
return resultRes{File: file}, nil
}
}
func attestationEndpoint(svc agent.Service) endpoint.Endpoint {
return func(ctx context.Context, request interface{}) (interface{}, error) {
req := request.(attestationReq)
if err := req.validate(); err != nil {
return attestationRes{}, err
}
file, err := svc.Attestation(ctx, req.ReportData)
if err != nil {
return attestationRes{}, err
}
return attestationRes{File: file}, nil
}
}
+173
View File
@@ -0,0 +1,173 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package grpc
import (
"context"
"errors"
"testing"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/mocks"
"golang.org/x/crypto/sha3"
)
const svcErr = "Service Error"
func TestAlgoEndpoint(t *testing.T) {
svc := new(mocks.Service)
tests := []struct {
name string
req algoReq
expectedErr bool
}{
{
name: "Success",
req: algoReq{Algorithm: []byte("algorithm")},
},
{
name: "Validation Error",
req: algoReq{},
expectedErr: true,
},
{
name: "Service Error",
req: algoReq{Algorithm: []byte("algorithm")},
expectedErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.name == svcErr {
svc.On("Algo", context.Background(), agent.Algorithm{Algorithm: tt.req.Algorithm}).Return(errors.New("")).Once()
} else {
svc.On("Algo", context.Background(), agent.Algorithm{Algorithm: tt.req.Algorithm}).Return(nil).Once()
}
endpoint := algoEndpoint(svc)
_, err := endpoint(context.Background(), tt.req)
if (err != nil) != tt.expectedErr {
t.Errorf("algoEndpoint() error = %v, expectedErr %v", err, tt.expectedErr)
}
})
}
}
func TestDataEndpoint(t *testing.T) {
svc := new(mocks.Service)
tests := []struct {
name string
req dataReq
expectedErr bool
}{
{
name: "Success",
req: dataReq{Dataset: []byte("dataset")},
},
{
name: "Validation Error",
req: dataReq{},
expectedErr: true,
},
{
name: "Service Error",
req: dataReq{Dataset: []byte("dataset")},
expectedErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.name == svcErr {
svc.On("Data", context.Background(), agent.Dataset{Dataset: tt.req.Dataset}).Return(errors.New("")).Once()
} else {
svc.On("Data", context.Background(), agent.Dataset{Dataset: tt.req.Dataset}).Return(nil).Once()
}
endpoint := dataEndpoint(svc)
_, err := endpoint(context.Background(), tt.req)
if (err != nil) != tt.expectedErr {
t.Errorf("dataEndpoint() error = %v, expectedErr %v", err, tt.expectedErr)
}
})
}
}
func TestResultEndpoint(t *testing.T) {
svc := new(mocks.Service)
tests := []struct {
name string
req resultReq
expectedErr bool
}{
{
name: "Success",
req: resultReq{},
},
{
name: "Service Error",
req: resultReq{},
expectedErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.name == svcErr {
svc.On("Result", context.Background()).Return([]byte{}, errors.New("")).Once()
} else {
svc.On("Result", context.Background()).Return([]byte{}, nil).Once()
}
endpoint := resultEndpoint(svc)
res, err := endpoint(context.Background(), tt.req)
if (err != nil) != tt.expectedErr {
t.Errorf("resultEndpoint() error = %v, expectedErr %v", err, tt.expectedErr)
}
if err == nil {
_, ok := res.(resultRes)
if !ok {
t.Errorf("resultEndpoint() returned unexpected type %T", res)
}
}
})
}
}
func TestAttestationEndpoint(t *testing.T) {
svc := new(mocks.Service)
tests := []struct {
name string
req attestationReq
expectedErr bool
}{
{
name: "Success",
req: attestationReq{ReportData: sha3.Sum512([]byte("report data"))},
},
{
name: "Service Error",
req: attestationReq{ReportData: sha3.Sum512([]byte("report data"))},
expectedErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if tt.name == svcErr {
svc.On("Attestation", context.Background(), tt.req.ReportData).Return([]byte{}, errors.New("")).Once()
} else {
svc.On("Attestation", context.Background(), tt.req.ReportData).Return([]byte{}, nil).Once()
}
endpoint := attestationEndpoint(svc)
res, err := endpoint(context.Background(), tt.req)
if (err != nil) != tt.expectedErr {
t.Errorf("attestationEndpoint() error = %v, expectedErr %v", err, tt.expectedErr)
}
if err == nil {
_, ok := res.(attestationRes)
if !ok {
t.Errorf("attestationEndpoint() returned unexpected type %T", res)
}
}
})
}
}
+74
View File
@@ -0,0 +1,74 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package grpc
import (
"context"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/auth"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
type authInterceptor struct {
auth auth.Authenticator
}
type wrappedServerStream struct {
grpc.ServerStream
ctx context.Context
}
func (s *wrappedServerStream) Context() context.Context {
return s.ctx
}
func NewAuthInterceptor(authSvc auth.Authenticator) (grpc.UnaryServerInterceptor, grpc.StreamServerInterceptor) {
ai := &authInterceptor{auth: authSvc}
return ai.AuthUnaryInterceptor(), ai.AuthStreamInterceptor()
}
func (s *authInterceptor) AuthStreamInterceptor() grpc.StreamServerInterceptor {
return func(srv interface{}, stream grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
switch info.FullMethod {
case agent.AgentService_Algo_FullMethodName:
if _, err := s.auth.AuthenticateUser(stream.Context(), auth.AlgorithmProviderRole); err != nil {
return status.Errorf(codes.Unauthenticated, "%v", err.Error())
}
return handler(srv, stream)
case agent.AgentService_Data_FullMethodName:
ctx, err := s.auth.AuthenticateUser(stream.Context(), auth.DataProviderRole)
if err != nil {
return status.Errorf(codes.Unauthenticated, "%s", err.Error())
}
wrapped := &wrappedServerStream{ServerStream: stream, ctx: ctx}
return handler(srv, wrapped)
case agent.AgentService_Result_FullMethodName:
ctx, err := s.auth.AuthenticateUser(stream.Context(), auth.ConsumerRole)
if err != nil {
return status.Errorf(codes.Unauthenticated, "%v", err.Error())
}
wrapped := &wrappedServerStream{ServerStream: stream, ctx: ctx}
return handler(srv, wrapped)
default:
return handler(srv, stream)
}
}
}
func (s *authInterceptor) AuthUnaryInterceptor() grpc.UnaryServerInterceptor {
return func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
switch info.FullMethod {
case agent.AgentService_Result_FullMethodName:
ctx, err := s.auth.AuthenticateUser(ctx, auth.ConsumerRole)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "%v", err.Error())
}
return handler(ctx, req)
default:
return handler(ctx, req)
}
}
}
+152
View File
@@ -0,0 +1,152 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package grpc
import (
"context"
"testing"
"github.com/stretchr/testify/mock"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/auth"
"github.com/ultravioletrs/cocos/agent/auth/mocks"
"google.golang.org/grpc"
"google.golang.org/grpc/metadata"
)
func TestAuthUnaryInterceptor(t *testing.T) {
authmock := new(mocks.Authenticator)
tests := []struct {
name string
authorized bool
method string
role auth.UserRole
wantErr bool
}{
{
name: "authorized result method",
authorized: true,
method: agent.AgentService_Result_FullMethodName,
role: auth.ConsumerRole,
wantErr: false,
},
{
name: "unauthorized result method",
authorized: false,
method: agent.AgentService_Result_FullMethodName,
role: auth.ConsumerRole,
wantErr: true,
},
{
name: "other method",
authorized: false,
method: "OtherMethod",
role: auth.ConsumerRole,
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
switch tt.authorized {
case true:
mockCall := authmock.On("AuthenticateUser", context.Background(), tt.role).Return(context.Background(), nil)
mockCall.Once()
case false:
mockCall := authmock.On("AuthenticateUser", context.Background(), tt.role).Return(context.Background(), auth.ErrMissingMetadata)
mockCall.Once()
}
unaryInt, _ := NewAuthInterceptor(authmock)
_, err := unaryInt(context.Background(), nil, &grpc.UnaryServerInfo{FullMethod: tt.method}, func(ctx context.Context, req interface{}) (interface{}, error) {
return nil, nil
})
if tt.wantErr && err == nil {
t.Errorf("expected error, got none")
} else if !tt.wantErr && err != nil {
t.Errorf("unexpected error: %v", err)
}
})
}
}
func TestAuthStreamInterceptor(t *testing.T) {
authmock := new(mocks.Authenticator)
tests := []struct {
name string
authorized bool
method string
role auth.UserRole
wantErr bool
}{
{
name: "authorized algo method",
authorized: true,
method: agent.AgentService_Algo_FullMethodName,
role: auth.AlgorithmProviderRole,
wantErr: false,
},
{
name: "unauthorized algo method",
authorized: false,
method: agent.AgentService_Algo_FullMethodName,
role: auth.AlgorithmProviderRole,
wantErr: true,
},
{
name: "authorized data method",
authorized: true,
method: agent.AgentService_Data_FullMethodName,
role: auth.DataProviderRole,
wantErr: false,
},
{
name: "unauthorized data method",
authorized: false,
method: agent.AgentService_Data_FullMethodName,
role: auth.DataProviderRole,
wantErr: true,
},
{
name: "other method",
authorized: false,
method: "OtherMethod",
role: auth.DataProviderRole,
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
switch tt.authorized {
case true:
mockCall := authmock.On("AuthenticateUser", mock.Anything, tt.role).Return(context.Background(), nil)
mockCall.Once()
case false:
mockCall := authmock.On("AuthenticateUser", mock.Anything, tt.role).Return(context.Background(), auth.ErrMissingMetadata)
mockCall.Once()
}
_, streamInt := NewAuthInterceptor(authmock)
err := streamInt(nil, &mockServerStream{ctx: metadata.NewIncomingContext(context.Background(), metadata.Pairs())}, &grpc.StreamServerInfo{FullMethod: tt.method}, func(srv interface{}, stream grpc.ServerStream) error {
return nil
})
if tt.wantErr && err == nil {
t.Errorf("expected error, got none")
} else if !tt.wantErr && err != nil {
t.Errorf("unexpected error: %v", err)
}
})
}
}
type mockServerStream struct {
grpc.ServerStream
ctx context.Context
}
func (m *mockServerStream) Context() context.Context {
return m.ctx
}
+46
View File
@@ -0,0 +1,46 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package grpc
import (
"errors"
)
type algoReq struct {
Algorithm []byte `protobuf:"bytes,1,opt,name=algorithm,proto3" json:"algorithm,omitempty"`
Requirements []byte
}
func (req algoReq) validate() error {
if len(req.Algorithm) == 0 {
return errors.New("algorithm binary is required")
}
return nil
}
type dataReq struct {
Dataset []byte `protobuf:"bytes,1,opt,name=dataset,proto3" json:"dataset,omitempty"`
Filename string
}
func (req dataReq) validate() error {
if len(req.Dataset) == 0 {
return errors.New("dataset CSV file is required")
}
return nil
}
type resultReq struct{}
func (req resultReq) validate() error {
// No request parameters to validate, so no validation logic needed
return nil
}
type attestationReq struct {
ReportData [64]byte
}
func (req attestationReq) validate() error {
return nil
}
+15
View File
@@ -0,0 +1,15 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package grpc
type algoRes struct{}
type dataRes struct{}
type resultRes struct {
File []byte `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
}
type attestationRes struct {
File []byte
}
+219
View File
@@ -0,0 +1,219 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package grpc
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"github.com/go-kit/kit/transport/grpc"
"github.com/ultravioletrs/cocos/agent"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
)
const (
bufferSize = 1024 * 1024
FileSizeKey = "file-size"
)
var _ agent.AgentServiceServer = (*grpcServer)(nil)
type grpcServer struct {
algo grpc.Handler
data grpc.Handler
result grpc.Handler
attestation grpc.Handler
agent.UnimplementedAgentServiceServer
}
// NewServer returns new AgentServiceServer instance.
func NewServer(svc agent.Service) agent.AgentServiceServer {
return &grpcServer{
algo: grpc.NewServer(
algoEndpoint(svc),
decodeAlgoRequest,
encodeAlgoResponse,
),
data: grpc.NewServer(
dataEndpoint(svc),
decodeDataRequest,
encodeDataResponse,
),
result: grpc.NewServer(
resultEndpoint(svc),
decodeResultRequest,
encodeResultResponse,
),
attestation: grpc.NewServer(
attestationEndpoint(svc),
decodeAttestationRequest,
encodeAttestationResponse,
),
}
}
func decodeAlgoRequest(_ context.Context, grpcReq interface{}) (interface{}, error) {
req := grpcReq.(*agent.AlgoRequest)
return algoReq{
Algorithm: req.Algorithm,
Requirements: req.Requirements,
}, nil
}
func encodeAlgoResponse(_ context.Context, response interface{}) (interface{}, error) {
return &agent.AlgoResponse{}, nil
}
func decodeDataRequest(_ context.Context, grpcReq interface{}) (interface{}, error) {
req := grpcReq.(*agent.DataRequest)
return dataReq{
Dataset: req.Dataset,
Filename: req.Filename,
}, nil
}
func encodeDataResponse(_ context.Context, response interface{}) (interface{}, error) {
return &agent.DataResponse{}, nil
}
func decodeResultRequest(_ context.Context, grpcReq interface{}) (interface{}, error) {
return resultReq{}, nil
}
func encodeResultResponse(_ context.Context, response interface{}) (interface{}, error) {
res := response.(resultRes)
return &agent.ResultResponse{
File: res.File,
}, nil
}
func decodeAttestationRequest(_ context.Context, grpcReq interface{}) (interface{}, error) {
req := grpcReq.(*agent.AttestationRequest)
if len(req.ReportData) != agent.ReportDataSize {
return nil, errors.New("malformed report data, expect 64 bytes")
}
return attestationReq{ReportData: [agent.ReportDataSize]byte(req.ReportData)}, nil
}
func encodeAttestationResponse(_ context.Context, response interface{}) (interface{}, error) {
res := response.(attestationRes)
return &agent.AttestationResponse{
File: res.File,
}, nil
}
// Algo implements agent.AgentServiceServer.
func (s *grpcServer) Algo(stream agent.AgentService_AlgoServer) error {
var algoFile, reqFile []byte
for {
algoChunk, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
return status.Error(codes.Internal, err.Error())
}
algoFile = append(algoFile, algoChunk.Algorithm...)
reqFile = append(reqFile, algoChunk.Requirements...)
}
_, res, err := s.algo.ServeGRPC(stream.Context(), &agent.AlgoRequest{Algorithm: algoFile, Requirements: reqFile})
if err != nil {
return err
}
ar := res.(*agent.AlgoResponse)
return stream.SendAndClose(ar)
}
// Data implements agent.AgentServiceServer.
func (s *grpcServer) Data(stream agent.AgentService_DataServer) error {
var dataFile []byte
var filename string
for {
dataChunk, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
return status.Error(codes.Internal, err.Error())
}
dataFile = append(dataFile, dataChunk.Dataset...)
filename = dataChunk.Filename
}
_, res, err := s.data.ServeGRPC(stream.Context(), &agent.DataRequest{Dataset: dataFile, Filename: filename})
if err != nil {
return err
}
ar := res.(*agent.DataResponse)
return stream.SendAndClose(ar)
}
func (s *grpcServer) Result(req *agent.ResultRequest, stream agent.AgentService_ResultServer) error {
_, res, err := s.result.ServeGRPC(stream.Context(), req)
if err != nil {
return err
}
rr := res.(*agent.ResultResponse)
if err := stream.SetHeader(metadata.New(map[string]string{FileSizeKey: fmt.Sprint(len(rr.File))})); err != nil {
return status.Error(codes.Internal, err.Error())
}
resultBuffer := bytes.NewBuffer(rr.File)
buf := make([]byte, bufferSize)
for {
n, err := resultBuffer.Read(buf)
if err == io.EOF {
break
}
if err != nil {
return status.Error(codes.Internal, err.Error())
}
if err := stream.Send(&agent.ResultResponse{File: buf[:n]}); err != nil {
return status.Error(codes.Internal, err.Error())
}
}
return nil
}
func (s *grpcServer) Attestation(req *agent.AttestationRequest, stream agent.AgentService_AttestationServer) error {
_, res, err := s.attestation.ServeGRPC(stream.Context(), req)
if err != nil {
return err
}
rr := res.(*agent.AttestationResponse)
if err := stream.SetHeader(metadata.New(map[string]string{FileSizeKey: fmt.Sprint(len(rr.File))})); err != nil {
return status.Error(codes.Internal, err.Error())
}
attestationBuffer := bytes.NewBuffer(rr.File)
buf := make([]byte, bufferSize)
for {
n, err := attestationBuffer.Read(buf)
if err == io.EOF {
break
}
if err != nil {
return status.Error(codes.Internal, err.Error())
}
if err := stream.Send(&agent.AttestationResponse{File: buf[:n]}); err != nil {
return status.Error(codes.Internal, err.Error())
}
}
return nil
}
+213
View File
@@ -0,0 +1,213 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package grpc
import (
"context"
"io"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/mocks"
"google.golang.org/grpc"
"google.golang.org/grpc/metadata"
)
type MockAgentService_AlgoServer struct {
grpc.ServerStream
mock.Mock
ctx context.Context
}
func (m *MockAgentService_AlgoServer) Context() context.Context {
return m.ctx
}
func (m *MockAgentService_AlgoServer) Recv() (*agent.AlgoRequest, error) {
args := m.Called()
return args.Get(0).(*agent.AlgoRequest), args.Error(1)
}
func (m *MockAgentService_AlgoServer) SendAndClose(resp *agent.AlgoResponse) error {
args := m.Called(resp)
return args.Error(0)
}
type MockAgentService_DataServer struct {
grpc.ServerStream
mock.Mock
ctx context.Context
}
func (m *MockAgentService_DataServer) Context() context.Context {
return m.ctx
}
func (m *MockAgentService_DataServer) Recv() (*agent.DataRequest, error) {
args := m.Called()
return args.Get(0).(*agent.DataRequest), args.Error(1)
}
func (m *MockAgentService_DataServer) SendAndClose(resp *agent.DataResponse) error {
args := m.Called(resp)
return args.Error(0)
}
type MockAgentService_ResultServer struct {
grpc.ServerStream
mock.Mock
ctx context.Context
}
func (m *MockAgentService_ResultServer) Context() context.Context {
return m.ctx
}
func (m *MockAgentService_ResultServer) SetHeader(metadata.MD) error {
return nil
}
func (m *MockAgentService_ResultServer) Send(resp *agent.ResultResponse) error {
args := m.Called(resp)
return args.Error(0)
}
type MockAgentService_AttestationServer struct {
grpc.ServerStream
mock.Mock
ctx context.Context
}
func (m *MockAgentService_AttestationServer) Context() context.Context {
return m.ctx
}
func (m *MockAgentService_AttestationServer) Send(resp *agent.AttestationResponse) error {
args := m.Called(resp)
return args.Error(0)
}
func (m *MockAgentService_AttestationServer) SetHeader(metadata.MD) error {
return nil
}
func TestAlgo(t *testing.T) {
mockService := new(mocks.Service)
server := NewServer(mockService)
mockStream := &MockAgentService_AlgoServer{ctx: context.Background()}
mockStream.On("Recv").Return(&agent.AlgoRequest{Algorithm: []byte("algo"), Requirements: []byte("req")}, nil).Once()
mockStream.On("Recv").Return(&agent.AlgoRequest{}, io.EOF)
mockStream.On("SendAndClose", &agent.AlgoResponse{}).Return(nil)
mockService.On("Algo", context.Background(), agent.Algorithm{Algorithm: []byte("algo"), Requirements: []byte("req")}).Return(nil)
err := server.Algo(mockStream)
assert.NoError(t, err)
mockStream.AssertExpectations(t)
mockService.AssertExpectations(t)
}
func TestData(t *testing.T) {
mockService := new(mocks.Service)
server := NewServer(mockService)
mockStream := &MockAgentService_DataServer{ctx: context.Background()}
mockStream.On("Recv").Return(&agent.DataRequest{Dataset: []byte("data"), Filename: "test.txt"}, nil).Once()
mockStream.On("Recv").Return(&agent.DataRequest{}, io.EOF)
mockStream.On("SendAndClose", &agent.DataResponse{}).Return(nil)
mockService.On("Data", context.Background(), agent.Dataset{Dataset: []byte("data"), Filename: "test.txt"}).Return(nil)
err := server.Data(mockStream)
assert.NoError(t, err)
mockStream.AssertExpectations(t)
mockService.AssertExpectations(t)
}
func TestResult(t *testing.T) {
mockService := new(mocks.Service)
server := NewServer(mockService)
mockStream := &MockAgentService_ResultServer{ctx: context.Background()}
mockService.On("Result", mock.Anything).Return([]byte("result data"), nil)
mockStream.On("Send", mock.AnythingOfType("*agent.ResultResponse")).Return(nil)
err := server.Result(&agent.ResultRequest{}, mockStream)
assert.NoError(t, err)
mockStream.AssertExpectations(t)
mockService.AssertExpectations(t)
}
func TestAttestation(t *testing.T) {
mockService := new(mocks.Service)
server := NewServer(mockService)
mockStream := &MockAgentService_AttestationServer{ctx: context.Background()}
mockStream.On("Send", mock.AnythingOfType("*agent.AttestationResponse")).Return(nil)
reportData := [agent.ReportDataSize]byte{}
mockService.On("Attestation", mock.Anything, reportData).Return([]byte("attestation data"), nil)
err := server.Attestation(&agent.AttestationRequest{ReportData: reportData[:]}, mockStream)
assert.NoError(t, err)
mockService.AssertExpectations(t)
}
func TestDecodeAlgoRequest(t *testing.T) {
req := &agent.AlgoRequest{Algorithm: []byte("algo"), Requirements: []byte("req")}
decoded, err := decodeAlgoRequest(context.Background(), req)
assert.NoError(t, err)
assert.Equal(t, algoReq{Algorithm: []byte("algo"), Requirements: []byte("req")}, decoded)
}
func TestEncodeAlgoResponse(t *testing.T) {
encoded, err := encodeAlgoResponse(context.Background(), algoRes{})
assert.NoError(t, err)
assert.Equal(t, &agent.AlgoResponse{}, encoded)
}
func TestDecodeDataRequest(t *testing.T) {
req := &agent.DataRequest{Dataset: []byte("data"), Filename: "test.txt"}
decoded, err := decodeDataRequest(context.Background(), req)
assert.NoError(t, err)
assert.Equal(t, dataReq{Dataset: []byte("data"), Filename: "test.txt"}, decoded)
}
func TestEncodeDataResponse(t *testing.T) {
encoded, err := encodeDataResponse(context.Background(), dataRes{})
assert.NoError(t, err)
assert.Equal(t, &agent.DataResponse{}, encoded)
}
func TestDecodeResultRequest(t *testing.T) {
decoded, err := decodeResultRequest(context.Background(), &agent.ResultRequest{})
assert.NoError(t, err)
assert.Equal(t, resultReq{}, decoded)
}
func TestEncodeResultResponse(t *testing.T) {
encoded, err := encodeResultResponse(context.Background(), resultRes{File: []byte("result")})
assert.NoError(t, err)
assert.Equal(t, &agent.ResultResponse{File: []byte("result")}, encoded)
}
func TestDecodeAttestationRequest(t *testing.T) {
reportData := [agent.ReportDataSize]byte{}
req := &agent.AttestationRequest{ReportData: reportData[:]}
decoded, err := decodeAttestationRequest(context.Background(), req)
assert.NoError(t, err)
assert.Equal(t, attestationReq{ReportData: reportData}, decoded)
}
func TestEncodeAttestationResponse(t *testing.T) {
encoded, err := encodeAttestationResponse(context.Background(), attestationRes{File: []byte("attestation")})
assert.NoError(t, err)
assert.Equal(t, &agent.AttestationResponse{File: []byte("attestation")}, encoded)
}
+80
View File
@@ -0,0 +1,80 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
//go:build !test
// +build !test
package api
import (
"context"
"fmt"
"log/slog"
"time"
"github.com/ultravioletrs/cocos/agent"
)
var _ agent.Service = (*loggingMiddleware)(nil)
type loggingMiddleware struct {
logger *slog.Logger
svc agent.Service
}
// LoggingMiddleware adds logging facilities to the core service.
func LoggingMiddleware(svc agent.Service, logger *slog.Logger) agent.Service {
return &loggingMiddleware{logger, svc}
}
func (lm *loggingMiddleware) Algo(ctx context.Context, algorithm agent.Algorithm) (err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method Algo took %s to complete", time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors", message))
}(time.Now())
return lm.svc.Algo(ctx, algorithm)
}
func (lm *loggingMiddleware) Data(ctx context.Context, dataset agent.Dataset) (err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method Data took %s to complete", time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors", message))
}(time.Now())
return lm.svc.Data(ctx, dataset)
}
func (lm *loggingMiddleware) Result(ctx context.Context) (response []byte, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method Result took %s to complete", time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors", message))
}(time.Now())
return lm.svc.Result(ctx)
}
func (lm *loggingMiddleware) Attestation(ctx context.Context, reportData [agent.ReportDataSize]byte) (response []byte, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method Attestation took %s to complete", time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors", message))
}(time.Now())
return lm.svc.Attestation(ctx, reportData)
}
+69
View File
@@ -0,0 +1,69 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
//go:build !test
// +build !test
package api
import (
"context"
"time"
"github.com/go-kit/kit/metrics"
"github.com/ultravioletrs/cocos/agent"
)
var _ agent.Service = (*metricsMiddleware)(nil)
type metricsMiddleware struct {
counter metrics.Counter
latency metrics.Histogram
svc agent.Service
}
// MetricsMiddleware instruments core service by tracking request count and
// latency.
func MetricsMiddleware(svc agent.Service, counter metrics.Counter, latency metrics.Histogram) agent.Service {
return &metricsMiddleware{
counter: counter,
latency: latency,
svc: svc,
}
}
func (ms *metricsMiddleware) Algo(ctx context.Context, algorithm agent.Algorithm) error {
defer func(begin time.Time) {
ms.counter.With("method", "algo").Add(1)
ms.latency.With("method", "algo").Observe(time.Since(begin).Seconds())
}(time.Now())
return ms.svc.Algo(ctx, algorithm)
}
func (ms *metricsMiddleware) Data(ctx context.Context, dataset agent.Dataset) error {
defer func(begin time.Time) {
ms.counter.With("method", "data").Add(1)
ms.latency.With("method", "data").Observe(time.Since(begin).Seconds())
}(time.Now())
return ms.svc.Data(ctx, dataset)
}
func (ms *metricsMiddleware) Result(ctx context.Context) ([]byte, error) {
defer func(begin time.Time) {
ms.counter.With("method", "result").Add(1)
ms.latency.With("method", "result").Observe(time.Since(begin).Seconds())
}(time.Now())
return ms.svc.Result(ctx)
}
func (ms *metricsMiddleware) Attestation(ctx context.Context, reportData [agent.ReportDataSize]byte) ([]byte, error) {
defer func(begin time.Time) {
ms.counter.With("method", "attestation").Add(1)
ms.latency.With("method", "attestation").Observe(time.Since(begin).Seconds())
}(time.Now())
return ms.svc.Attestation(ctx, reportData)
}
+173
View File
@@ -0,0 +1,173 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package auth
import (
"context"
"crypto"
"crypto/ecdsa"
"crypto/ed25519"
"crypto/rsa"
"crypto/sha256"
"crypto/x509"
"encoding/base64"
"github.com/absmach/magistrala/pkg/errors"
"github.com/ultravioletrs/cocos/agent"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
)
type UserRole string
const (
UserMetadataKey = "user-id"
SignatureMetadataKey = "signature"
ConsumerRole UserRole = "consumer"
DataProviderRole UserRole = "data-provider"
AlgorithmProviderRole UserRole = "algorithm-provider"
)
var (
ErrMissingMetadata = errors.New("missing metadata")
ErrInvalidMetadata = errors.New("invalid metadata")
ErrSignatureVerificationFailed = errors.New("signature verification failed")
)
type Authenticator interface {
AuthenticateUser(ctx context.Context, role UserRole) (context.Context, error)
}
type service struct {
resultConsumers []interface{}
datasetProviders []interface{}
algorithmProvider interface{}
}
func New(manifest agent.Computation) (Authenticator, error) {
s := &service{}
for _, rc := range manifest.ResultConsumers {
pubKey, err := x509.ParsePKIXPublicKey(rc.UserKey)
if err != nil {
return nil, err
}
pKey, err := decodePublicKey(pubKey)
if err != nil {
return nil, err
}
s.resultConsumers = append(s.resultConsumers, pKey)
}
for _, dp := range manifest.Datasets {
pubKey, err := x509.ParsePKIXPublicKey(dp.UserKey)
if err != nil {
return nil, err
}
pKey, err := decodePublicKey(pubKey)
if err != nil {
return nil, err
}
s.datasetProviders = append(s.datasetProviders, pKey)
}
pubKey, err := x509.ParsePKIXPublicKey(manifest.Algorithm.UserKey)
if err != nil {
return nil, err
}
pKey, err := decodePublicKey(pubKey)
if err != nil {
return nil, err
}
s.algorithmProvider = pKey
return s, nil
}
func extractSignature(md metadata.MD) (string, error) {
signature := md.Get(SignatureMetadataKey)
if len(signature) != 1 {
return "", status.Errorf(codes.Unauthenticated, "invalid metadata")
}
return signature[0], nil
}
func verifySignature(role UserRole, signature string, publicKey any) error {
hash := sha256.Sum256([]byte(role))
sigByte, err := base64.StdEncoding.DecodeString(signature)
if err != nil {
return err
}
var ok bool
switch publicKey := publicKey.(type) {
case *rsa.PublicKey:
if err = rsa.VerifyPKCS1v15(publicKey, crypto.SHA256, hash[:], sigByte); err != nil {
return err
}
return nil
case *ecdsa.PublicKey:
ok = ecdsa.VerifyASN1(publicKey, hash[:], sigByte)
case ed25519.PublicKey:
ok = ed25519.Verify(publicKey, []byte(role), sigByte)
}
if !ok {
return ErrSignatureVerificationFailed
}
return nil
}
func (s *service) AuthenticateUser(ctx context.Context, role UserRole) (context.Context, error) {
md, ok := metadata.FromIncomingContext(ctx)
if !ok {
return nil, ErrMissingMetadata
}
signature, err := extractSignature(md)
if err != nil {
return nil, errors.Wrap(err, ErrInvalidMetadata)
}
switch role {
case ConsumerRole:
for i, rc := range s.resultConsumers {
if err := verifySignature(role, signature, rc); err == nil {
return agent.IndexToContext(ctx, i), nil
}
}
case DataProviderRole:
for _, dp := range s.datasetProviders {
if err := verifySignature(role, signature, dp); err == nil {
return ctx, nil
}
}
case AlgorithmProviderRole:
if err := verifySignature(role, signature, s.algorithmProvider); err == nil {
return ctx, nil
}
}
return ctx, ErrSignatureVerificationFailed
}
func decodePublicKey(key any) (pubKey any, err error) {
switch key := key.(type) {
case *rsa.PublicKey:
return key, nil
case *ecdsa.PublicKey:
return key, nil
case ed25519.PublicKey:
return key, nil
default:
return nil, errors.New("unsupported public key type")
}
}
+158
View File
@@ -0,0 +1,158 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package auth
import (
"context"
"crypto"
"crypto/ecdsa"
"crypto/ed25519"
"crypto/elliptic"
"crypto/rand"
"crypto/rsa"
"crypto/sha256"
"crypto/x509"
"encoding/base64"
"testing"
"github.com/absmach/magistrala/pkg/errors"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/ultravioletrs/cocos/agent"
"google.golang.org/grpc/metadata"
)
func TestAuthenticateUser(t *testing.T) {
resultConsumerKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
require.NoError(t, err)
dataProviderKey, err := rsa.GenerateKey(rand.Reader, 2048)
require.NoError(t, err)
pubEd25519Key, algorithmProviderKey, err := ed25519.GenerateKey(rand.Reader)
require.NoError(t, err)
resultConsumerPubKey, err := x509.MarshalPKIXPublicKey(&resultConsumerKey.PublicKey)
require.NoError(t, err)
dataProviderPubKey, err := x509.MarshalPKIXPublicKey(&dataProviderKey.PublicKey)
require.NoError(t, err)
algorithmProviderPubKey, err := x509.MarshalPKIXPublicKey(pubEd25519Key)
require.NoError(t, err)
manifest := agent.Computation{
ResultConsumers: []agent.ResultConsumer{{UserKey: resultConsumerPubKey}},
Datasets: []agent.Dataset{{UserKey: dataProviderPubKey}},
Algorithm: agent.Algorithm{UserKey: algorithmProviderPubKey},
}
auth, err := New(manifest)
if err != nil {
t.Fatalf("failed to create authenticator: %v", err)
}
testCases := []struct {
name string
role UserRole
key any
expectedErr error
}{
{
name: "valid result consumer",
role: ConsumerRole,
key: resultConsumerKey,
expectedErr: nil,
},
{
name: "valid data provider",
role: DataProviderRole,
key: dataProviderKey,
expectedErr: nil,
},
{
name: "valid algorithm provider",
role: AlgorithmProviderRole,
key: algorithmProviderKey,
expectedErr: nil,
},
{
name: "invalid role",
role: "invalid-role",
key: resultConsumerKey,
expectedErr: ErrSignatureVerificationFailed,
},
{
name: "invalid key",
role: ConsumerRole,
key: dataProviderKey,
expectedErr: ErrSignatureVerificationFailed,
},
{
name: "missing signature",
role: ConsumerRole,
key: resultConsumerKey,
expectedErr: ErrInvalidMetadata,
},
{
name: "missing metadata",
role: ConsumerRole,
key: resultConsumerKey,
expectedErr: ErrMissingMetadata,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
signature, err := signRole(tc.role, tc.key)
if err != nil {
t.Fatalf("failed to sign role: %v", err)
}
ctx := context.Background()
switch tc.name {
case "missing signature":
ctx = metadata.NewIncomingContext(ctx, metadata.Pairs())
case "missing metadata":
default:
ctx = metadata.NewIncomingContext(ctx, metadata.Pairs(SignatureMetadataKey, signature))
}
ctx, err = auth.AuthenticateUser(ctx, tc.role)
assert.True(t, errors.Contains(err, tc.expectedErr), "expected error %v, got %v", tc.expectedErr, err)
if err == nil {
switch id, ok := agent.IndexFromContext(ctx); {
case tc.role == ConsumerRole:
assert.True(t, ok, "expected index in context")
assert.Equal(t, 0, id, "expected index 0 in context")
default:
assert.False(t, ok, "expected no index in context")
}
}
})
}
}
func signRole(role UserRole, key crypto.PrivateKey) (string, error) {
var signature []byte
var err error
switch k := key.(type) {
case ed25519.PrivateKey:
signature, err = k.Sign(rand.Reader, []byte(role), crypto.Hash(0))
case *rsa.PrivateKey, *ecdsa.PrivateKey:
hash := sha256.Sum256([]byte(role))
signer := key.(crypto.Signer)
signature, err = signer.Sign(rand.Reader, hash[:], crypto.SHA256)
default:
return "", errors.New("unsupported key type")
}
if err != nil {
return "", err
}
return base64.StdEncoding.EncodeToString(signature), nil
}
+100
View File
@@ -0,0 +1,100 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
// Code generated by mockery v2.43.2. DO NOT EDIT.
package mocks
import (
context "context"
auth "github.com/ultravioletrs/cocos/agent/auth"
mock "github.com/stretchr/testify/mock"
)
// Authenticator is an autogenerated mock type for the Authenticator type
type Authenticator struct {
mock.Mock
}
type Authenticator_Expecter struct {
mock *mock.Mock
}
func (_m *Authenticator) EXPECT() *Authenticator_Expecter {
return &Authenticator_Expecter{mock: &_m.Mock}
}
// AuthenticateUser provides a mock function with given fields: ctx, role
func (_m *Authenticator) AuthenticateUser(ctx context.Context, role auth.UserRole) (context.Context, error) {
ret := _m.Called(ctx, role)
if len(ret) == 0 {
panic("no return value specified for AuthenticateUser")
}
var r0 context.Context
var r1 error
if rf, ok := ret.Get(0).(func(context.Context, auth.UserRole) (context.Context, error)); ok {
return rf(ctx, role)
}
if rf, ok := ret.Get(0).(func(context.Context, auth.UserRole) context.Context); ok {
r0 = rf(ctx, role)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(context.Context)
}
}
if rf, ok := ret.Get(1).(func(context.Context, auth.UserRole) error); ok {
r1 = rf(ctx, role)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// Authenticator_AuthenticateUser_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'AuthenticateUser'
type Authenticator_AuthenticateUser_Call struct {
*mock.Call
}
// AuthenticateUser is a helper method to define mock.On call
// - ctx context.Context
// - role auth.UserRole
func (_e *Authenticator_Expecter) AuthenticateUser(ctx interface{}, role interface{}) *Authenticator_AuthenticateUser_Call {
return &Authenticator_AuthenticateUser_Call{Call: _e.mock.On("AuthenticateUser", ctx, role)}
}
func (_c *Authenticator_AuthenticateUser_Call) Run(run func(ctx context.Context, role auth.UserRole)) *Authenticator_AuthenticateUser_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context), args[1].(auth.UserRole))
})
return _c
}
func (_c *Authenticator_AuthenticateUser_Call) Return(_a0 context.Context, _a1 error) *Authenticator_AuthenticateUser_Call {
_c.Call.Return(_a0, _a1)
return _c
}
func (_c *Authenticator_AuthenticateUser_Call) RunAndReturn(run func(context.Context, auth.UserRole) (context.Context, error)) *Authenticator_AuthenticateUser_Call {
_c.Call.Return(run)
return _c
}
// NewAuthenticator creates a new instance of Authenticator. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewAuthenticator(t interface {
mock.TestingT
Cleanup(func())
}) *Authenticator {
mock := &Authenticator{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
+88
View File
@@ -0,0 +1,88 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package agent
import (
"context"
"encoding/json"
"fmt"
"google.golang.org/grpc/metadata"
)
var _ fmt.Stringer = (*Datasets)(nil)
type AgentConfig struct {
LogLevel string `json:"log_level,omitempty"`
Host string `json:"host,omitempty"`
Port string `json:"port,omitempty"`
CertFile string `json:"cert_file,omitempty"`
KeyFile string `json:"server_key,omitempty"`
ServerCAFile string `json:"server_ca_file,omitempty"`
ClientCAFile string `json:"client_ca_file,omitempty"`
AttestedTls bool `json:"attested_tls,omitempty"`
}
type Computation struct {
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Datasets Datasets `json:"datasets,omitempty"`
Algorithm Algorithm `json:"algorithm,omitempty"`
ResultConsumers []ResultConsumer `json:"result_consumers,omitempty"`
AgentConfig AgentConfig `json:"agent_config,omitempty"`
}
type ResultConsumer struct {
UserKey []byte `json:"user_key,omitempty"`
}
func (d *Datasets) String() string {
dat, err := json.Marshal(d)
if err != nil {
return ""
}
return string(dat)
}
type Dataset struct {
Dataset []byte `json:"-"`
Hash [32]byte `json:"hash,omitempty"`
UserKey []byte `json:"user_key,omitempty"`
Filename string `json:"filename,omitempty"`
}
type Datasets []Dataset
type Algorithm struct {
Algorithm []byte `json:"-"`
Hash [32]byte `json:"hash,omitempty"`
UserKey []byte `json:"user_key,omitempty"`
Requirements []byte `json:"-"`
}
type ManifestIndexKey struct{}
func IndexToContext(ctx context.Context, index int) context.Context {
return context.WithValue(ctx, ManifestIndexKey{}, index)
}
func IndexFromContext(ctx context.Context) (int, bool) {
index, ok := ctx.Value(ManifestIndexKey{}).(int)
return index, ok
}
const DecompressKey = "decompress"
func DecompressFromContext(ctx context.Context) bool {
vals := metadata.ValueFromIncomingContext(ctx, DecompressKey)
if len(vals) == 0 {
return false
}
return vals[0] == "true"
}
func DecompressToContext(ctx context.Context, decompress bool) context.Context {
return metadata.AppendToOutgoingContext(ctx, DecompressKey, fmt.Sprintf("%t", decompress))
}
+133
View File
@@ -0,0 +1,133 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package agent
import (
"context"
"encoding/json"
"reflect"
"testing"
"google.golang.org/grpc/metadata"
)
func TestDatasetsString(t *testing.T) {
datasets := Datasets{
{
Hash: [32]byte{1, 2, 3},
UserKey: []byte("user_key"),
Filename: "test.dat",
},
}
expected := `[{"hash":[1,2,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"user_key":"dXNlcl9rZXk=","filename":"test.dat"}]`
result := datasets.String()
if result != expected {
t.Errorf("Datasets.String() = %v, want %v", result, expected)
}
}
func TestIndexToContext(t *testing.T) {
ctx := context.Background()
index := 5
newCtx := IndexToContext(ctx, index)
result, ok := IndexFromContext(newCtx)
if !ok {
t.Errorf("IndexFromContext() ok = false, want true")
}
if result != index {
t.Errorf("IndexFromContext() = %v, want %v", result, index)
}
}
func TestDecompressFromContext(t *testing.T) {
tests := []struct {
name string
ctx context.Context
expected bool
}{
{
name: "No decompress metadata",
ctx: context.Background(),
expected: false,
},
{
name: "Decompress true",
ctx: metadata.NewIncomingContext(
context.Background(),
metadata.Pairs(DecompressKey, "true"),
),
expected: true,
},
{
name: "Decompress false",
ctx: metadata.NewIncomingContext(
context.Background(),
metadata.Pairs(DecompressKey, "false"),
),
expected: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := DecompressFromContext(tt.ctx)
if result != tt.expected {
t.Errorf("DecompressFromContext() = %v, want %v", result, tt.expected)
}
})
}
}
func TestDecompressToContext(t *testing.T) {
ctx := context.Background()
decompress := true
newCtx := DecompressToContext(ctx, decompress)
md, ok := metadata.FromOutgoingContext(newCtx)
if !ok {
t.Errorf("metadata.FromOutgoingContext() ok = false, want true")
}
vals := md.Get(DecompressKey)
if len(vals) != 1 {
t.Errorf("len(md.Get(DecompressKey)) = %v, want 1", len(vals))
}
if vals[0] != "true" {
t.Errorf("md.Get(DecompressKey)[0] = %v, want 'true'", vals[0])
}
}
func TestAgentConfigJSON(t *testing.T) {
config := AgentConfig{
LogLevel: "info",
Host: "localhost",
Port: "8080",
CertFile: "cert.pem",
KeyFile: "key.pem",
ServerCAFile: "server_ca.pem",
ClientCAFile: "client_ca.pem",
AttestedTls: true,
}
data, err := json.Marshal(config)
if err != nil {
t.Fatalf("Failed to marshal AgentConfig: %v", err)
}
var unmarshaledConfig AgentConfig
err = json.Unmarshal(data, &unmarshaledConfig)
if err != nil {
t.Fatalf("Failed to unmarshal AgentConfig: %v", err)
}
if !reflect.DeepEqual(config, unmarshaledConfig) {
t.Errorf("Unmarshaled config does not match original. Got %+v, want %+v", unmarshaledConfig, config)
}
}
+4
View File
@@ -0,0 +1,4 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package agent
+46
View File
@@ -0,0 +1,46 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package events
import (
"encoding/json"
"io"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/types/known/timestamppb"
)
type service struct {
service string
computationID string
conn io.Writer
}
type Service interface {
SendEvent(event, status string, details json.RawMessage) error
}
func New(svc, computationID string, conn io.Writer) (Service, error) {
return &service{
service: svc,
computationID: computationID,
conn: conn,
}, nil
}
func (s *service) SendEvent(event, status string, details json.RawMessage) error {
body := EventsLogs{Message: &EventsLogs_AgentEvent{AgentEvent: &AgentEvent{
EventType: event,
Timestamp: timestamppb.Now(),
ComputationId: s.computationID,
Originator: s.service,
Status: status,
Details: details,
}}}
protoBody, err := proto.Marshal(&body)
if err != nil {
return err
}
_, err = s.conn.Write(protoBody)
return err
}
+405
View File
@@ -0,0 +1,405 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.34.2
// protoc v5.28.1
// source: agent/events/events.proto
package events
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
timestamppb "google.golang.org/protobuf/types/known/timestamppb"
reflect "reflect"
sync "sync"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type AgentEvent struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
EventType string `protobuf:"bytes,1,opt,name=event_type,json=eventType,proto3" json:"event_type,omitempty"`
Timestamp *timestamppb.Timestamp `protobuf:"bytes,2,opt,name=timestamp,proto3" json:"timestamp,omitempty"`
ComputationId string `protobuf:"bytes,3,opt,name=computation_id,json=computationId,proto3" json:"computation_id,omitempty"`
Details []byte `protobuf:"bytes,4,opt,name=details,proto3" json:"details,omitempty"`
Originator string `protobuf:"bytes,5,opt,name=originator,proto3" json:"originator,omitempty"`
Status string `protobuf:"bytes,6,opt,name=status,proto3" json:"status,omitempty"`
}
func (x *AgentEvent) Reset() {
*x = AgentEvent{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_events_events_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *AgentEvent) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*AgentEvent) ProtoMessage() {}
func (x *AgentEvent) ProtoReflect() protoreflect.Message {
mi := &file_agent_events_events_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use AgentEvent.ProtoReflect.Descriptor instead.
func (*AgentEvent) Descriptor() ([]byte, []int) {
return file_agent_events_events_proto_rawDescGZIP(), []int{0}
}
func (x *AgentEvent) GetEventType() string {
if x != nil {
return x.EventType
}
return ""
}
func (x *AgentEvent) GetTimestamp() *timestamppb.Timestamp {
if x != nil {
return x.Timestamp
}
return nil
}
func (x *AgentEvent) GetComputationId() string {
if x != nil {
return x.ComputationId
}
return ""
}
func (x *AgentEvent) GetDetails() []byte {
if x != nil {
return x.Details
}
return nil
}
func (x *AgentEvent) GetOriginator() string {
if x != nil {
return x.Originator
}
return ""
}
func (x *AgentEvent) GetStatus() string {
if x != nil {
return x.Status
}
return ""
}
type AgentLog struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Message string `protobuf:"bytes,1,opt,name=message,proto3" json:"message,omitempty"`
ComputationId string `protobuf:"bytes,2,opt,name=computation_id,json=computationId,proto3" json:"computation_id,omitempty"`
Level string `protobuf:"bytes,3,opt,name=level,proto3" json:"level,omitempty"`
Timestamp *timestamppb.Timestamp `protobuf:"bytes,4,opt,name=timestamp,proto3" json:"timestamp,omitempty"`
}
func (x *AgentLog) Reset() {
*x = AgentLog{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_events_events_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *AgentLog) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*AgentLog) ProtoMessage() {}
func (x *AgentLog) ProtoReflect() protoreflect.Message {
mi := &file_agent_events_events_proto_msgTypes[1]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use AgentLog.ProtoReflect.Descriptor instead.
func (*AgentLog) Descriptor() ([]byte, []int) {
return file_agent_events_events_proto_rawDescGZIP(), []int{1}
}
func (x *AgentLog) GetMessage() string {
if x != nil {
return x.Message
}
return ""
}
func (x *AgentLog) GetComputationId() string {
if x != nil {
return x.ComputationId
}
return ""
}
func (x *AgentLog) GetLevel() string {
if x != nil {
return x.Level
}
return ""
}
func (x *AgentLog) GetTimestamp() *timestamppb.Timestamp {
if x != nil {
return x.Timestamp
}
return nil
}
type EventsLogs struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// Types that are assignable to Message:
//
// *EventsLogs_AgentLog
// *EventsLogs_AgentEvent
Message isEventsLogs_Message `protobuf_oneof:"message"`
}
func (x *EventsLogs) Reset() {
*x = EventsLogs{}
if protoimpl.UnsafeEnabled {
mi := &file_agent_events_events_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *EventsLogs) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*EventsLogs) ProtoMessage() {}
func (x *EventsLogs) ProtoReflect() protoreflect.Message {
mi := &file_agent_events_events_proto_msgTypes[2]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use EventsLogs.ProtoReflect.Descriptor instead.
func (*EventsLogs) Descriptor() ([]byte, []int) {
return file_agent_events_events_proto_rawDescGZIP(), []int{2}
}
func (m *EventsLogs) GetMessage() isEventsLogs_Message {
if m != nil {
return m.Message
}
return nil
}
func (x *EventsLogs) GetAgentLog() *AgentLog {
if x, ok := x.GetMessage().(*EventsLogs_AgentLog); ok {
return x.AgentLog
}
return nil
}
func (x *EventsLogs) GetAgentEvent() *AgentEvent {
if x, ok := x.GetMessage().(*EventsLogs_AgentEvent); ok {
return x.AgentEvent
}
return nil
}
type isEventsLogs_Message interface {
isEventsLogs_Message()
}
type EventsLogs_AgentLog struct {
AgentLog *AgentLog `protobuf:"bytes,1,opt,name=agent_log,json=agentLog,proto3,oneof"`
}
type EventsLogs_AgentEvent struct {
AgentEvent *AgentEvent `protobuf:"bytes,2,opt,name=agent_event,json=agentEvent,proto3,oneof"`
}
func (*EventsLogs_AgentLog) isEventsLogs_Message() {}
func (*EventsLogs_AgentEvent) isEventsLogs_Message() {}
var File_agent_events_events_proto protoreflect.FileDescriptor
var file_agent_events_events_proto_rawDesc = []byte{
0x0a, 0x19, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2f, 0x65, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x2f, 0x65,
0x76, 0x65, 0x6e, 0x74, 0x73, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x06, 0x65, 0x76, 0x65,
0x6e, 0x74, 0x73, 0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74,
0x6f, 0x62, 0x75, 0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x22, 0xde, 0x01, 0x0a, 0x0a, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x45, 0x76,
0x65, 0x6e, 0x74, 0x12, 0x1d, 0x0a, 0x0a, 0x65, 0x76, 0x65, 0x6e, 0x74, 0x5f, 0x74, 0x79, 0x70,
0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x65, 0x76, 0x65, 0x6e, 0x74, 0x54, 0x79,
0x70, 0x65, 0x12, 0x38, 0x0a, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x18,
0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d,
0x70, 0x52, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x12, 0x25, 0x0a, 0x0e,
0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x69, 0x64, 0x18, 0x03,
0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f,
0x6e, 0x49, 0x64, 0x12, 0x18, 0x0a, 0x07, 0x64, 0x65, 0x74, 0x61, 0x69, 0x6c, 0x73, 0x18, 0x04,
0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x64, 0x65, 0x74, 0x61, 0x69, 0x6c, 0x73, 0x12, 0x1e, 0x0a,
0x0a, 0x6f, 0x72, 0x69, 0x67, 0x69, 0x6e, 0x61, 0x74, 0x6f, 0x72, 0x18, 0x05, 0x20, 0x01, 0x28,
0x09, 0x52, 0x0a, 0x6f, 0x72, 0x69, 0x67, 0x69, 0x6e, 0x61, 0x74, 0x6f, 0x72, 0x12, 0x16, 0x0a,
0x06, 0x73, 0x74, 0x61, 0x74, 0x75, 0x73, 0x18, 0x06, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x73,
0x74, 0x61, 0x74, 0x75, 0x73, 0x22, 0x9b, 0x01, 0x0a, 0x08, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x4c,
0x6f, 0x67, 0x12, 0x18, 0x0a, 0x07, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x18, 0x01, 0x20,
0x01, 0x28, 0x09, 0x52, 0x07, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x12, 0x25, 0x0a, 0x0e,
0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x69, 0x64, 0x18, 0x02,
0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f,
0x6e, 0x49, 0x64, 0x12, 0x14, 0x0a, 0x05, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x18, 0x03, 0x20, 0x01,
0x28, 0x09, 0x52, 0x05, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x12, 0x38, 0x0a, 0x09, 0x74, 0x69, 0x6d,
0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67,
0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54,
0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x52, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74,
0x61, 0x6d, 0x70, 0x22, 0x7f, 0x0a, 0x0a, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x4c, 0x6f, 0x67,
0x73, 0x12, 0x2f, 0x0a, 0x09, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f, 0x6c, 0x6f, 0x67, 0x18, 0x01,
0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x65, 0x76, 0x65, 0x6e, 0x74, 0x73, 0x2e, 0x41, 0x67,
0x65, 0x6e, 0x74, 0x4c, 0x6f, 0x67, 0x48, 0x00, 0x52, 0x08, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x4c,
0x6f, 0x67, 0x12, 0x35, 0x0a, 0x0b, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f, 0x65, 0x76, 0x65, 0x6e,
0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x65, 0x76, 0x65, 0x6e, 0x74, 0x73,
0x2e, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x48, 0x00, 0x52, 0x0a, 0x61,
0x67, 0x65, 0x6e, 0x74, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x42, 0x09, 0x0a, 0x07, 0x6d, 0x65, 0x73,
0x73, 0x61, 0x67, 0x65, 0x42, 0x0a, 0x5a, 0x08, 0x2e, 0x2f, 0x65, 0x76, 0x65, 0x6e, 0x74, 0x73,
0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (
file_agent_events_events_proto_rawDescOnce sync.Once
file_agent_events_events_proto_rawDescData = file_agent_events_events_proto_rawDesc
)
func file_agent_events_events_proto_rawDescGZIP() []byte {
file_agent_events_events_proto_rawDescOnce.Do(func() {
file_agent_events_events_proto_rawDescData = protoimpl.X.CompressGZIP(file_agent_events_events_proto_rawDescData)
})
return file_agent_events_events_proto_rawDescData
}
var file_agent_events_events_proto_msgTypes = make([]protoimpl.MessageInfo, 3)
var file_agent_events_events_proto_goTypes = []any{
(*AgentEvent)(nil), // 0: events.AgentEvent
(*AgentLog)(nil), // 1: events.AgentLog
(*EventsLogs)(nil), // 2: events.EventsLogs
(*timestamppb.Timestamp)(nil), // 3: google.protobuf.Timestamp
}
var file_agent_events_events_proto_depIdxs = []int32{
3, // 0: events.AgentEvent.timestamp:type_name -> google.protobuf.Timestamp
3, // 1: events.AgentLog.timestamp:type_name -> google.protobuf.Timestamp
1, // 2: events.EventsLogs.agent_log:type_name -> events.AgentLog
0, // 3: events.EventsLogs.agent_event:type_name -> events.AgentEvent
4, // [4:4] is the sub-list for method output_type
4, // [4:4] is the sub-list for method input_type
4, // [4:4] is the sub-list for extension type_name
4, // [4:4] is the sub-list for extension extendee
0, // [0:4] is the sub-list for field type_name
}
func init() { file_agent_events_events_proto_init() }
func file_agent_events_events_proto_init() {
if File_agent_events_events_proto != nil {
return
}
if !protoimpl.UnsafeEnabled {
file_agent_events_events_proto_msgTypes[0].Exporter = func(v any, i int) any {
switch v := v.(*AgentEvent); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_agent_events_events_proto_msgTypes[1].Exporter = func(v any, i int) any {
switch v := v.(*AgentLog); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_agent_events_events_proto_msgTypes[2].Exporter = func(v any, i int) any {
switch v := v.(*EventsLogs); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
}
file_agent_events_events_proto_msgTypes[2].OneofWrappers = []any{
(*EventsLogs_AgentLog)(nil),
(*EventsLogs_AgentEvent)(nil),
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_agent_events_events_proto_rawDesc,
NumEnums: 0,
NumMessages: 3,
NumExtensions: 0,
NumServices: 0,
},
GoTypes: file_agent_events_events_proto_goTypes,
DependencyIndexes: file_agent_events_events_proto_depIdxs,
MessageInfos: file_agent_events_events_proto_msgTypes,
}.Build()
File_agent_events_events_proto = out.File
file_agent_events_events_proto_rawDesc = nil
file_agent_events_events_proto_goTypes = nil
file_agent_events_events_proto_depIdxs = nil
}
+33
View File
@@ -0,0 +1,33 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
syntax = "proto3";
package events;
import "google/protobuf/timestamp.proto";
option go_package = "./events";
message AgentEvent {
string event_type = 1;
google.protobuf.Timestamp timestamp = 2;
string computation_id = 3;
bytes details = 4;
string originator = 5;
string status = 6;
}
message AgentLog {
string message = 1;
string computation_id = 2;
string level = 3;
google.protobuf.Timestamp timestamp = 4;
}
message EventsLogs {
oneof message {
AgentLog agent_log = 1;
AgentEvent agent_event = 2;
}
}
+64
View File
@@ -0,0 +1,64 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package events
import (
"bytes"
"encoding/json"
"errors"
"testing"
"time"
"github.com/stretchr/testify/assert"
"google.golang.org/protobuf/proto"
)
type mockConn struct {
writeErr error
buf bytes.Buffer
}
func (m *mockConn) Write(p []byte) (n int, err error) {
if m.writeErr != nil {
return 0, m.writeErr
}
return m.buf.Write(p)
}
func TestSendEventSuccess(t *testing.T) {
mockConnection := &mockConn{}
svc, err := New("test_service", "12345", mockConnection)
assert.NoError(t, err)
details := json.RawMessage(`{"key": "value"}`)
err = svc.SendEvent("test_event", "success", details)
assert.NoError(t, err)
var writtenMessage EventsLogs
err = proto.Unmarshal(mockConnection.buf.Bytes(), &writtenMessage)
assert.NoError(t, err)
assert.Equal(t, "test_event", writtenMessage.GetAgentEvent().EventType)
assert.Equal(t, "12345", writtenMessage.GetAgentEvent().ComputationId)
assert.Equal(t, "test_service", writtenMessage.GetAgentEvent().Originator)
assert.Equal(t, "success", writtenMessage.GetAgentEvent().Status)
now := time.Now()
eventTimestamp := writtenMessage.GetAgentEvent().GetTimestamp().AsTime()
assert.WithinDuration(t, now, eventTimestamp, 1*time.Second)
}
func TestSendEventFailure(t *testing.T) {
mockConnection := &mockConn{writeErr: errors.New("write error")}
svc, err := New("test_service", "12345", mockConnection)
assert.NoError(t, err)
details := json.RawMessage(`{"key": "value"}`)
err = svc.SendEvent("test_event", "failure", details)
assert.Error(t, err)
assert.Equal(t, "write error", err.Error())
}
+87
View File
@@ -0,0 +1,87 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
// Code generated by mockery v2.43.2. DO NOT EDIT.
package mocks
import (
json "encoding/json"
mock "github.com/stretchr/testify/mock"
)
// Service is an autogenerated mock type for the Service type
type Service struct {
mock.Mock
}
type Service_Expecter struct {
mock *mock.Mock
}
func (_m *Service) EXPECT() *Service_Expecter {
return &Service_Expecter{mock: &_m.Mock}
}
// SendEvent provides a mock function with given fields: event, status, details
func (_m *Service) SendEvent(event string, status string, details json.RawMessage) error {
ret := _m.Called(event, status, details)
if len(ret) == 0 {
panic("no return value specified for SendEvent")
}
var r0 error
if rf, ok := ret.Get(0).(func(string, string, json.RawMessage) error); ok {
r0 = rf(event, status, details)
} else {
r0 = ret.Error(0)
}
return r0
}
// Service_SendEvent_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'SendEvent'
type Service_SendEvent_Call struct {
*mock.Call
}
// SendEvent is a helper method to define mock.On call
// - event string
// - status string
// - details json.RawMessage
func (_e *Service_Expecter) SendEvent(event interface{}, status interface{}, details interface{}) *Service_SendEvent_Call {
return &Service_SendEvent_Call{Call: _e.mock.On("SendEvent", event, status, details)}
}
func (_c *Service_SendEvent_Call) Run(run func(event string, status string, details json.RawMessage)) *Service_SendEvent_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(string), args[1].(string), args[2].(json.RawMessage))
})
return _c
}
func (_c *Service_SendEvent_Call) Return(_a0 error) *Service_SendEvent_Call {
_c.Call.Return(_a0)
return _c
}
func (_c *Service_SendEvent_Call) RunAndReturn(run func(string, string, json.RawMessage) error) *Service_SendEvent_Call {
_c.Call.Return(run)
return _c
}
// NewService creates a new instance of Service. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewService(t interface {
mock.TestingT
Cleanup(func())
}) *Service {
mock := &Service{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
+252
View File
@@ -0,0 +1,252 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
// Code generated by mockery v2.43.2. DO NOT EDIT.
package mocks
import (
context "context"
agent "github.com/ultravioletrs/cocos/agent"
mock "github.com/stretchr/testify/mock"
)
// Service is an autogenerated mock type for the Service type
type Service struct {
mock.Mock
}
type Service_Expecter struct {
mock *mock.Mock
}
func (_m *Service) EXPECT() *Service_Expecter {
return &Service_Expecter{mock: &_m.Mock}
}
// Algo provides a mock function with given fields: ctx, algorithm
func (_m *Service) Algo(ctx context.Context, algorithm agent.Algorithm) error {
ret := _m.Called(ctx, algorithm)
if len(ret) == 0 {
panic("no return value specified for Algo")
}
var r0 error
if rf, ok := ret.Get(0).(func(context.Context, agent.Algorithm) error); ok {
r0 = rf(ctx, algorithm)
} else {
r0 = ret.Error(0)
}
return r0
}
// Service_Algo_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Algo'
type Service_Algo_Call struct {
*mock.Call
}
// Algo is a helper method to define mock.On call
// - ctx context.Context
// - algorithm agent.Algorithm
func (_e *Service_Expecter) Algo(ctx interface{}, algorithm interface{}) *Service_Algo_Call {
return &Service_Algo_Call{Call: _e.mock.On("Algo", ctx, algorithm)}
}
func (_c *Service_Algo_Call) Run(run func(ctx context.Context, algorithm agent.Algorithm)) *Service_Algo_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context), args[1].(agent.Algorithm))
})
return _c
}
func (_c *Service_Algo_Call) Return(_a0 error) *Service_Algo_Call {
_c.Call.Return(_a0)
return _c
}
func (_c *Service_Algo_Call) RunAndReturn(run func(context.Context, agent.Algorithm) error) *Service_Algo_Call {
_c.Call.Return(run)
return _c
}
// Attestation provides a mock function with given fields: ctx, reportData
func (_m *Service) Attestation(ctx context.Context, reportData [64]byte) ([]byte, error) {
ret := _m.Called(ctx, reportData)
if len(ret) == 0 {
panic("no return value specified for Attestation")
}
var r0 []byte
var r1 error
if rf, ok := ret.Get(0).(func(context.Context, [64]byte) ([]byte, error)); ok {
return rf(ctx, reportData)
}
if rf, ok := ret.Get(0).(func(context.Context, [64]byte) []byte); ok {
r0 = rf(ctx, reportData)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).([]byte)
}
}
if rf, ok := ret.Get(1).(func(context.Context, [64]byte) error); ok {
r1 = rf(ctx, reportData)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// Service_Attestation_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Attestation'
type Service_Attestation_Call struct {
*mock.Call
}
// Attestation is a helper method to define mock.On call
// - ctx context.Context
// - reportData [64]byte
func (_e *Service_Expecter) Attestation(ctx interface{}, reportData interface{}) *Service_Attestation_Call {
return &Service_Attestation_Call{Call: _e.mock.On("Attestation", ctx, reportData)}
}
func (_c *Service_Attestation_Call) Run(run func(ctx context.Context, reportData [64]byte)) *Service_Attestation_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context), args[1].([64]byte))
})
return _c
}
func (_c *Service_Attestation_Call) Return(_a0 []byte, _a1 error) *Service_Attestation_Call {
_c.Call.Return(_a0, _a1)
return _c
}
func (_c *Service_Attestation_Call) RunAndReturn(run func(context.Context, [64]byte) ([]byte, error)) *Service_Attestation_Call {
_c.Call.Return(run)
return _c
}
// Data provides a mock function with given fields: ctx, dataset
func (_m *Service) Data(ctx context.Context, dataset agent.Dataset) error {
ret := _m.Called(ctx, dataset)
if len(ret) == 0 {
panic("no return value specified for Data")
}
var r0 error
if rf, ok := ret.Get(0).(func(context.Context, agent.Dataset) error); ok {
r0 = rf(ctx, dataset)
} else {
r0 = ret.Error(0)
}
return r0
}
// Service_Data_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Data'
type Service_Data_Call struct {
*mock.Call
}
// Data is a helper method to define mock.On call
// - ctx context.Context
// - dataset agent.Dataset
func (_e *Service_Expecter) Data(ctx interface{}, dataset interface{}) *Service_Data_Call {
return &Service_Data_Call{Call: _e.mock.On("Data", ctx, dataset)}
}
func (_c *Service_Data_Call) Run(run func(ctx context.Context, dataset agent.Dataset)) *Service_Data_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context), args[1].(agent.Dataset))
})
return _c
}
func (_c *Service_Data_Call) Return(_a0 error) *Service_Data_Call {
_c.Call.Return(_a0)
return _c
}
func (_c *Service_Data_Call) RunAndReturn(run func(context.Context, agent.Dataset) error) *Service_Data_Call {
_c.Call.Return(run)
return _c
}
// Result provides a mock function with given fields: ctx
func (_m *Service) Result(ctx context.Context) ([]byte, error) {
ret := _m.Called(ctx)
if len(ret) == 0 {
panic("no return value specified for Result")
}
var r0 []byte
var r1 error
if rf, ok := ret.Get(0).(func(context.Context) ([]byte, error)); ok {
return rf(ctx)
}
if rf, ok := ret.Get(0).(func(context.Context) []byte); ok {
r0 = rf(ctx)
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).([]byte)
}
}
if rf, ok := ret.Get(1).(func(context.Context) error); ok {
r1 = rf(ctx)
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// Service_Result_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Result'
type Service_Result_Call struct {
*mock.Call
}
// Result is a helper method to define mock.On call
// - ctx context.Context
func (_e *Service_Expecter) Result(ctx interface{}) *Service_Result_Call {
return &Service_Result_Call{Call: _e.mock.On("Result", ctx)}
}
func (_c *Service_Result_Call) Run(run func(ctx context.Context)) *Service_Result_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context))
})
return _c
}
func (_c *Service_Result_Call) Return(_a0 []byte, _a1 error) *Service_Result_Call {
_c.Call.Return(_a0, _a1)
return _c
}
func (_c *Service_Result_Call) RunAndReturn(run func(context.Context) ([]byte, error)) *Service_Result_Call {
_c.Call.Return(run)
return _c
}
// NewService creates a new instance of Service. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewService(t interface {
mock.TestingT
Cleanup(func())
}) *Service {
mock := &Service{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
+207
View File
@@ -0,0 +1,207 @@
// Code generated by mockery v2.43.2. DO NOT EDIT.
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package mocks
import (
context "context"
agent "github.com/ultravioletrs/cocos/agent"
metadata "google.golang.org/grpc/metadata"
mock "github.com/stretchr/testify/mock"
)
// AgentService_AlgoClient is an autogenerated mock type for the AgentService_AlgoClient type
type AgentService_AlgoClient struct {
mock.Mock
}
// CloseAndRecv provides a mock function with given fields:
func (_m *AgentService_AlgoClient) CloseAndRecv() (*agent.AlgoResponse, error) {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for CloseAndRecv")
}
var r0 *agent.AlgoResponse
var r1 error
if rf, ok := ret.Get(0).(func() (*agent.AlgoResponse, error)); ok {
return rf()
}
if rf, ok := ret.Get(0).(func() *agent.AlgoResponse); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(*agent.AlgoResponse)
}
}
if rf, ok := ret.Get(1).(func() error); ok {
r1 = rf()
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// CloseSend provides a mock function with given fields:
func (_m *AgentService_AlgoClient) CloseSend() error {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for CloseSend")
}
var r0 error
if rf, ok := ret.Get(0).(func() error); ok {
r0 = rf()
} else {
r0 = ret.Error(0)
}
return r0
}
// Context provides a mock function with given fields:
func (_m *AgentService_AlgoClient) Context() context.Context {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for Context")
}
var r0 context.Context
if rf, ok := ret.Get(0).(func() context.Context); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(context.Context)
}
}
return r0
}
// Header provides a mock function with given fields:
func (_m *AgentService_AlgoClient) Header() (metadata.MD, error) {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for Header")
}
var r0 metadata.MD
var r1 error
if rf, ok := ret.Get(0).(func() (metadata.MD, error)); ok {
return rf()
}
if rf, ok := ret.Get(0).(func() metadata.MD); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(metadata.MD)
}
}
if rf, ok := ret.Get(1).(func() error); ok {
r1 = rf()
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// RecvMsg provides a mock function with given fields: m
func (_m *AgentService_AlgoClient) RecvMsg(m interface{}) error {
ret := _m.Called(m)
if len(ret) == 0 {
panic("no return value specified for RecvMsg")
}
var r0 error
if rf, ok := ret.Get(0).(func(interface{}) error); ok {
r0 = rf(m)
} else {
r0 = ret.Error(0)
}
return r0
}
// Send provides a mock function with given fields: _a0
func (_m *AgentService_AlgoClient) Send(_a0 *agent.AlgoRequest) error {
ret := _m.Called(_a0)
if len(ret) == 0 {
panic("no return value specified for Send")
}
var r0 error
if rf, ok := ret.Get(0).(func(*agent.AlgoRequest) error); ok {
r0 = rf(_a0)
} else {
r0 = ret.Error(0)
}
return r0
}
// SendMsg provides a mock function with given fields: m
func (_m *AgentService_AlgoClient) SendMsg(m interface{}) error {
ret := _m.Called(m)
if len(ret) == 0 {
panic("no return value specified for SendMsg")
}
var r0 error
if rf, ok := ret.Get(0).(func(interface{}) error); ok {
r0 = rf(m)
} else {
r0 = ret.Error(0)
}
return r0
}
// Trailer provides a mock function with given fields:
func (_m *AgentService_AlgoClient) Trailer() metadata.MD {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for Trailer")
}
var r0 metadata.MD
if rf, ok := ret.Get(0).(func() metadata.MD); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(metadata.MD)
}
}
return r0
}
// NewAgentService_AlgoClient creates a new instance of AgentService_AlgoClient. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewAgentService_AlgoClient(t interface {
mock.TestingT
Cleanup(func())
}) *AgentService_AlgoClient {
mock := &AgentService_AlgoClient{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
+207
View File
@@ -0,0 +1,207 @@
// Code generated by mockery v2.43.2. DO NOT EDIT.
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package mocks
import (
context "context"
agent "github.com/ultravioletrs/cocos/agent"
metadata "google.golang.org/grpc/metadata"
mock "github.com/stretchr/testify/mock"
)
// AgentService_DataClient is an autogenerated mock type for the AgentService_DataClient type
type AgentService_DataClient struct {
mock.Mock
}
// CloseAndRecv provides a mock function with given fields:
func (_m *AgentService_DataClient) CloseAndRecv() (*agent.DataResponse, error) {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for CloseAndRecv")
}
var r0 *agent.DataResponse
var r1 error
if rf, ok := ret.Get(0).(func() (*agent.DataResponse, error)); ok {
return rf()
}
if rf, ok := ret.Get(0).(func() *agent.DataResponse); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(*agent.DataResponse)
}
}
if rf, ok := ret.Get(1).(func() error); ok {
r1 = rf()
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// CloseSend provides a mock function with given fields:
func (_m *AgentService_DataClient) CloseSend() error {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for CloseSend")
}
var r0 error
if rf, ok := ret.Get(0).(func() error); ok {
r0 = rf()
} else {
r0 = ret.Error(0)
}
return r0
}
// Context provides a mock function with given fields:
func (_m *AgentService_DataClient) Context() context.Context {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for Context")
}
var r0 context.Context
if rf, ok := ret.Get(0).(func() context.Context); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(context.Context)
}
}
return r0
}
// Header provides a mock function with given fields:
func (_m *AgentService_DataClient) Header() (metadata.MD, error) {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for Header")
}
var r0 metadata.MD
var r1 error
if rf, ok := ret.Get(0).(func() (metadata.MD, error)); ok {
return rf()
}
if rf, ok := ret.Get(0).(func() metadata.MD); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(metadata.MD)
}
}
if rf, ok := ret.Get(1).(func() error); ok {
r1 = rf()
} else {
r1 = ret.Error(1)
}
return r0, r1
}
// RecvMsg provides a mock function with given fields: m
func (_m *AgentService_DataClient) RecvMsg(m interface{}) error {
ret := _m.Called(m)
if len(ret) == 0 {
panic("no return value specified for RecvMsg")
}
var r0 error
if rf, ok := ret.Get(0).(func(interface{}) error); ok {
r0 = rf(m)
} else {
r0 = ret.Error(0)
}
return r0
}
// Send provides a mock function with given fields: _a0
func (_m *AgentService_DataClient) Send(_a0 *agent.DataRequest) error {
ret := _m.Called(_a0)
if len(ret) == 0 {
panic("no return value specified for Send")
}
var r0 error
if rf, ok := ret.Get(0).(func(*agent.DataRequest) error); ok {
r0 = rf(_a0)
} else {
r0 = ret.Error(0)
}
return r0
}
// SendMsg provides a mock function with given fields: m
func (_m *AgentService_DataClient) SendMsg(m interface{}) error {
ret := _m.Called(m)
if len(ret) == 0 {
panic("no return value specified for SendMsg")
}
var r0 error
if rf, ok := ret.Get(0).(func(interface{}) error); ok {
r0 = rf(m)
} else {
r0 = ret.Error(0)
}
return r0
}
// Trailer provides a mock function with given fields:
func (_m *AgentService_DataClient) Trailer() metadata.MD {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for Trailer")
}
var r0 metadata.MD
if rf, ok := ret.Get(0).(func() metadata.MD); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(metadata.MD)
}
}
return r0
}
// NewAgentService_DataClient creates a new instance of AgentService_DataClient. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewAgentService_DataClient(t interface {
mock.TestingT
Cleanup(func())
}) *AgentService_DataClient {
mock := &AgentService_DataClient{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
+407
View File
@@ -0,0 +1,407 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package agent
import (
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"os"
"path/filepath"
"slices"
sync "sync"
"github.com/google/go-sev-guest/client"
"github.com/ultravioletrs/cocos/agent/algorithm"
"github.com/ultravioletrs/cocos/agent/algorithm/binary"
"github.com/ultravioletrs/cocos/agent/algorithm/docker"
"github.com/ultravioletrs/cocos/agent/algorithm/python"
"github.com/ultravioletrs/cocos/agent/algorithm/wasm"
"github.com/ultravioletrs/cocos/agent/events"
"github.com/ultravioletrs/cocos/agent/statemachine"
"github.com/ultravioletrs/cocos/internal"
"golang.org/x/crypto/sha3"
)
var _ Service = (*agentService)(nil)
//go:generate stringer -type=AgentState
type AgentState int
const (
Idle AgentState = iota
ReceivingManifest
ReceivingAlgorithm
ReceivingData
Running
ConsumingResults
Complete
Failed
)
//go:generate stringer -type=AgentEvent
type AgentEvent int
const (
Start AgentEvent = iota
ManifestReceived
AlgorithmReceived
DataReceived
RunComplete
ResultsConsumed
RunFailed
)
//go:generate stringer -type=Status
type Status uint8
const (
IdleState Status = iota
InProgress
Ready
Completed
Terminated
Warning
Starting
)
const (
// ReportDataSize is the size of the report data expected by the attestation service.
ReportDataSize = 64
algoFilePermission = 0o700
)
var (
// ErrMalformedEntity indicates malformed entity specification (e.g.
// invalid username or password).
ErrMalformedEntity = errors.New("malformed entity specification")
// ErrUnauthorizedAccess indicates missing or invalid credentials provided
// when accessing a protected resource.
ErrUnauthorizedAccess = errors.New("missing or invalid credentials provided")
// ErrUndeclaredAlgorithm indicates algorithm was not declared in computation manifest.
ErrUndeclaredDataset = errors.New("dataset not declared in computation manifest")
// ErrAllManifestItemsReceived indicates no new computation manifest items expected.
ErrAllManifestItemsReceived = errors.New("all expected manifest Items have been received")
// ErrUndeclaredConsumer indicates the consumer requesting results in not declared in computation manifest.
ErrUndeclaredConsumer = errors.New("result consumer is undeclared in computation manifest")
// ErrResultsNotReady indicates the computation results are not ready.
ErrResultsNotReady = errors.New("computation results are not yet ready")
// ErrStateNotReady agent received a request in the wrong state.
ErrStateNotReady = errors.New("agent not expecting this operation in the current state")
// ErrHashMismatch provided algorithm/dataset does not match hash in manifest.
ErrHashMismatch = errors.New("malformed data, hash does not match manifest")
// ErrFileNameMismatch provided dataset filename does not match filename in manifest.
ErrFileNameMismatch = errors.New("malformed data, filename does not match manifest")
// ErrAllResultsConsumed indicates all results have been consumed.
ErrAllResultsConsumed = errors.New("all results have been consumed by declared consumers")
// ErrAttestationFailed attestation failed.
ErrAttestationFailed = errors.New("failed to get raw quote")
)
// Service specifies an API that must be fullfiled by the domain service
// implementation, and all of its decorators (e.g. logging & metrics).
type Service interface {
Algo(ctx context.Context, algorithm Algorithm) error
Data(ctx context.Context, dataset Dataset) error
Result(ctx context.Context) ([]byte, error)
Attestation(ctx context.Context, reportData [ReportDataSize]byte) ([]byte, error)
}
type agentService struct {
mu sync.Mutex
computation Computation // Holds the current computation request details.
algorithm algorithm.Algorithm // Filepath to the algorithm received for the computation.
result []byte // Stores the result of the computation.
sm statemachine.StateMachine // Manages the state transitions of the agent service.
runError error // Stores any error encountered during the computation run.
eventSvc events.Service // Service for publishing events related to computation.
quoteProvider client.QuoteProvider // Provider for generating attestation quotes.
logger *slog.Logger // Logger for the agent service.
resultsConsumed bool // Indicates if the results have been consumed.
}
var _ Service = (*agentService)(nil)
// New instantiates the agent service implementation.
func New(ctx context.Context, logger *slog.Logger, eventSvc events.Service, cmp Computation, quoteProvider client.QuoteProvider) Service {
sm := statemachine.NewStateMachine(Idle)
svc := &agentService{
sm: sm,
eventSvc: eventSvc,
quoteProvider: quoteProvider,
logger: logger,
computation: cmp,
}
transitions := []statemachine.Transition{
{From: Idle, Event: Start, To: ReceivingManifest},
{From: ReceivingManifest, Event: ManifestReceived, To: ReceivingAlgorithm},
}
if len(cmp.Datasets) == 0 {
transitions = append(transitions, statemachine.Transition{From: ReceivingAlgorithm, Event: AlgorithmReceived, To: Running})
} else {
transitions = append(transitions, statemachine.Transition{From: ReceivingAlgorithm, Event: AlgorithmReceived, To: ReceivingData})
transitions = append(transitions, statemachine.Transition{From: ReceivingData, Event: DataReceived, To: Running})
}
transitions = append(transitions, []statemachine.Transition{
{From: Running, Event: RunComplete, To: ConsumingResults},
{From: Running, Event: RunFailed, To: Failed},
{From: ConsumingResults, Event: ResultsConsumed, To: Complete},
}...)
for _, t := range transitions {
sm.AddTransition(t)
}
sm.SetAction(Idle, svc.publishEvent(IdleState.String()))
sm.SetAction(ReceivingManifest, svc.publishEvent(InProgress.String()))
sm.SetAction(ReceivingAlgorithm, svc.publishEvent(InProgress.String()))
sm.SetAction(ReceivingData, svc.publishEvent(InProgress.String()))
sm.SetAction(Running, svc.runComputation)
sm.SetAction(ConsumingResults, svc.publishEvent(Ready.String()))
sm.SetAction(Complete, svc.publishEvent(Completed.String()))
sm.SetAction(Failed, svc.publishEvent(Failed.String()))
go func() {
if err := sm.Start(ctx); err != nil {
logger.Error(err.Error())
}
}()
sm.SendEvent(Start)
defer sm.SendEvent(ManifestReceived)
return svc
}
func (as *agentService) Algo(ctx context.Context, algo Algorithm) error {
if as.sm.GetState() != ReceivingAlgorithm {
return ErrStateNotReady
}
as.mu.Lock()
defer as.mu.Unlock()
if as.algorithm != nil {
return ErrAllManifestItemsReceived
}
hash := sha3.Sum256(algo.Algorithm)
if hash != as.computation.Algorithm.Hash {
return ErrHashMismatch
}
currentDir, err := os.Getwd()
if err != nil {
return fmt.Errorf("error getting current directory: %v", err)
}
f, err := os.Create(filepath.Join(currentDir, "algo"))
if err != nil {
return fmt.Errorf("error creating algorithm file: %v", err)
}
if _, err := f.Write(algo.Algorithm); err != nil {
return fmt.Errorf("error writing algorithm to file: %v", err)
}
if err := os.Chmod(f.Name(), algoFilePermission); err != nil {
return fmt.Errorf("error changing file permissions: %v", err)
}
if err := f.Close(); err != nil {
return fmt.Errorf("error closing file: %v", err)
}
algoType := algorithm.AlgorithmTypeFromContext(ctx)
if algoType == "" {
algoType = string(algorithm.AlgoTypeBin)
}
args := algorithm.AlgorithmArgsFromContext(ctx)
switch algoType {
case string(algorithm.AlgoTypeBin):
as.algorithm = binary.NewAlgorithm(as.logger, as.eventSvc, f.Name(), args)
case string(algorithm.AlgoTypePython):
var requirementsFile string
if len(algo.Requirements) > 0 {
fr, err := os.CreateTemp("", "requirements.txt")
if err != nil {
return fmt.Errorf("error creating requirments file: %v", err)
}
if _, err := fr.Write(algo.Requirements); err != nil {
return fmt.Errorf("error writing requirements to file: %v", err)
}
if err := fr.Close(); err != nil {
return fmt.Errorf("error closing file: %v", err)
}
requirementsFile = fr.Name()
}
runtime := python.PythonRunTimeFromContext(ctx)
as.algorithm = python.NewAlgorithm(as.logger, as.eventSvc, runtime, requirementsFile, f.Name(), args)
case string(algorithm.AlgoTypeWasm):
as.algorithm = wasm.NewAlgorithm(as.logger, as.eventSvc, f.Name(), args)
case string(algorithm.AlgoTypeDocker):
as.algorithm = docker.NewAlgorithm(as.logger, as.eventSvc, f.Name())
}
if err := os.Mkdir(algorithm.DatasetsDir, 0o755); err != nil {
return fmt.Errorf("error creating datasets directory: %v", err)
}
if as.algorithm != nil {
as.sm.SendEvent(AlgorithmReceived)
}
return nil
}
func (as *agentService) Data(ctx context.Context, dataset Dataset) error {
if as.sm.GetState() != ReceivingData {
return ErrStateNotReady
}
as.mu.Lock()
defer as.mu.Unlock()
if len(as.computation.Datasets) == 0 {
return ErrAllManifestItemsReceived
}
hash := sha3.Sum256(dataset.Dataset)
matched := false
for i, d := range as.computation.Datasets {
if hash == d.Hash {
if d.Filename != "" && d.Filename != dataset.Filename {
return ErrFileNameMismatch
}
as.computation.Datasets = slices.Delete(as.computation.Datasets, i, i+1)
if DecompressFromContext(ctx) {
if err := internal.UnzipFromMemory(dataset.Dataset, algorithm.DatasetsDir); err != nil {
return fmt.Errorf("error decompressing dataset: %v", err)
}
} else {
f, err := os.Create(fmt.Sprintf("%s/%s", algorithm.DatasetsDir, dataset.Filename))
if err != nil {
return fmt.Errorf("error creating dataset file: %v", err)
}
if _, err := f.Write(dataset.Dataset); err != nil {
return fmt.Errorf("error writing dataset to file: %v", err)
}
if err := f.Close(); err != nil {
return fmt.Errorf("error closing file: %v", err)
}
}
matched = true
break
}
}
if !matched {
return ErrUndeclaredDataset
}
if len(as.computation.Datasets) == 0 {
defer as.sm.SendEvent(DataReceived)
}
return nil
}
func (as *agentService) Result(ctx context.Context) ([]byte, error) {
currentState := as.sm.GetState()
if currentState != ConsumingResults && currentState != Complete && currentState != Failed {
return []byte{}, ErrResultsNotReady
}
index, ok := IndexFromContext(ctx)
if !ok {
return []byte{}, ErrUndeclaredConsumer
}
as.mu.Lock()
defer as.mu.Unlock()
if index < 0 || index >= len(as.computation.ResultConsumers) {
return []byte{}, ErrUndeclaredConsumer
}
if !as.resultsConsumed && currentState == ConsumingResults {
as.resultsConsumed = true
defer as.sm.SendEvent(ResultsConsumed)
}
return as.result, as.runError
}
func (as *agentService) Attestation(ctx context.Context, reportData [ReportDataSize]byte) ([]byte, error) {
rawQuote, err := as.quoteProvider.GetRawQuote(reportData)
if err != nil {
return []byte{}, err
}
return rawQuote, nil
}
func (as *agentService) runComputation(state statemachine.State) {
as.publishEvent(Starting.String())(state)
as.logger.Debug("computation run started")
defer func() {
if as.runError != nil {
as.sm.SendEvent(RunFailed)
} else {
as.sm.SendEvent(RunComplete)
}
}()
if err := os.Mkdir(algorithm.ResultsDir, 0o755); err != nil {
as.runError = fmt.Errorf("error creating results directory: %s", err.Error())
as.logger.Warn(as.runError.Error())
as.publishEvent(Failed.String())(state)
return
}
defer func() {
if err := os.RemoveAll(algorithm.ResultsDir); err != nil {
as.logger.Warn(fmt.Sprintf("error removing results directory and its contents: %s", err.Error()))
}
if err := os.RemoveAll(algorithm.DatasetsDir); err != nil {
as.logger.Warn(fmt.Sprintf("error removing datasets directory and its contents: %s", err.Error()))
}
}()
as.publishEvent(InProgress.String())(state)
if err := as.algorithm.Run(); err != nil {
as.runError = err
as.logger.Warn(fmt.Sprintf("failed to run computation: %s", err.Error()))
as.publishEvent(Failed.String())(state)
return
}
results, err := internal.ZipDirectoryToMemory(algorithm.ResultsDir)
if err != nil {
as.runError = err
as.logger.Warn(fmt.Sprintf("failed to zip results: %s", err.Error()))
as.publishEvent(Failed.String())(state)
return
}
as.publishEvent(Completed.String())(state)
as.result = results
}
func (as *agentService) publishEvent(status string) statemachine.Action {
return func(state statemachine.State) {
if err := as.eventSvc.SendEvent(state.String(), status, json.RawMessage{}); err != nil {
as.logger.Warn(err.Error())
}
}
}
+406
View File
@@ -0,0 +1,406 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package agent
import (
"context"
"crypto/rand"
"log"
"os"
"testing"
"time"
mglog "github.com/absmach/magistrala/logger"
"github.com/absmach/magistrala/pkg/errors"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/ultravioletrs/cocos/agent/algorithm"
"github.com/ultravioletrs/cocos/agent/algorithm/python"
"github.com/ultravioletrs/cocos/agent/events/mocks"
"github.com/ultravioletrs/cocos/agent/statemachine"
smmocks "github.com/ultravioletrs/cocos/agent/statemachine/mocks"
"github.com/ultravioletrs/cocos/pkg/attestation/quoteprovider"
mocks2 "github.com/ultravioletrs/cocos/pkg/attestation/quoteprovider/mocks"
"golang.org/x/crypto/sha3"
"google.golang.org/grpc/metadata"
)
var (
algoPath = "../test/manual/algo/lin_reg.py"
reqPath = "../test/manual/algo/requirements.txt"
dataPath = "../test/manual/data/iris.csv"
)
const datasetFile = "iris.csv"
func TestAlgo(t *testing.T) {
events := new(mocks.Service)
evCall := events.On("SendEvent", mock.Anything, mock.Anything, mock.Anything).Return(nil)
defer evCall.Unset()
qp, err := quoteprovider.GetQuoteProvider()
require.NoError(t, err)
algo, err := os.ReadFile(algoPath)
require.NoError(t, err)
algoHash := sha3.Sum256(algo)
reqFile, err := os.ReadFile(reqPath)
require.NoError(t, err)
testCases := []struct {
name string
err error
algo Algorithm
algoType string
}{
{
name: "Test Algo successfully",
algo: Algorithm{
Algorithm: algo,
Hash: algoHash,
},
algoType: "python",
err: nil,
},
{
name: "Test Algo successfully with requirements file",
algo: Algorithm{
Algorithm: algo,
Hash: algoHash,
Requirements: reqFile,
},
algoType: "python",
err: nil,
},
{
name: "Test Algo type binary successfully",
algo: Algorithm{
Algorithm: algo,
Hash: algoHash,
},
algoType: "bin",
err: nil,
},
{
name: "Test Algo type wasm successfully",
algo: Algorithm{
Algorithm: algo,
Hash: algoHash,
},
algoType: "wasm",
err: nil,
},
{
name: "Test Algo type docker successfully",
algo: Algorithm{
Algorithm: algo,
Hash: algoHash,
},
algoType: "docker",
err: nil,
},
{
name: "Test algo hash mismatch",
algo: Algorithm{},
algoType: "python",
err: ErrHashMismatch,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
err = os.RemoveAll("datasets")
require.NoError(t, err)
ctx := metadata.NewIncomingContext(context.Background(),
metadata.Pairs(algorithm.AlgoTypeKey, tc.algoType, python.PyRuntimeKey, python.PyRuntime),
)
ctx, cancel := context.WithCancel(ctx)
defer cancel()
svc := New(ctx, mglog.NewMock(), events, testComputation(t), qp)
time.Sleep(300 * time.Millisecond)
err = svc.Algo(ctx, tc.algo)
assert.True(t, errors.Contains(err, tc.err), "expected %v, got %v", tc.err, err)
t.Cleanup(func() {
err = os.RemoveAll("venv")
err = os.RemoveAll("algo")
err = os.RemoveAll("datasets")
})
})
}
}
func TestData(t *testing.T) {
events := new(mocks.Service)
evCall := events.On("SendEvent", mock.Anything, mock.Anything, mock.Anything).Return(nil)
defer evCall.Unset()
qp, err := quoteprovider.GetQuoteProvider()
require.NoError(t, err)
algo, err := os.ReadFile(algoPath)
require.NoError(t, err)
algoHash := sha3.Sum256(algo)
alg := Algorithm{
Hash: algoHash,
Algorithm: algo,
}
data, err := os.ReadFile(dataPath)
require.NoError(t, err)
dataHash := sha3.Sum256(data)
cases := []struct {
name string
data Dataset
err error
}{
{
name: "Test data successfully",
data: Dataset{
Hash: dataHash,
Dataset: data,
Filename: datasetFile,
},
},
{
name: "Test State not ready",
data: Dataset{
Dataset: data,
Hash: dataHash,
Filename: datasetFile,
},
err: ErrStateNotReady,
},
{
name: "Test File name does not match manifest",
data: Dataset{
Dataset: data,
Hash: dataHash,
Filename: "invalid",
},
err: ErrFileNameMismatch,
},
{
name: "Test dataset not declared in manifest",
data: Dataset{
Filename: datasetFile,
},
err: ErrUndeclaredDataset,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
ctx := metadata.NewIncomingContext(context.Background(),
metadata.Pairs(
algorithm.AlgoTypeKey, "python",
python.PyRuntimeKey, python.PyRuntime),
)
if tc.err != ErrUndeclaredDataset {
ctx = IndexToContext(ctx, 0)
}
ctx, cancel := context.WithCancel(ctx)
defer cancel()
comp := testComputation(t)
svc := New(ctx, mglog.NewMock(), events, comp, qp)
time.Sleep(300 * time.Millisecond)
if tc.err != ErrStateNotReady {
_ = svc.Algo(ctx, alg)
time.Sleep(300 * time.Millisecond)
}
err = svc.Data(ctx, tc.data)
assert.True(t, errors.Contains(err, tc.err), "expected %v, got %v", tc.err, err)
t.Cleanup(func() {
_ = os.RemoveAll("datasets")
_ = os.RemoveAll("results")
err = os.RemoveAll("venv")
err = os.RemoveAll("algo")
})
})
}
}
func TestResult(t *testing.T) {
events := new(mocks.Service)
evCall := events.On("SendEvent", mock.Anything, mock.Anything, mock.Anything).Return(nil)
defer evCall.Unset()
qp, err := quoteprovider.GetQuoteProvider()
require.NoError(t, err)
cases := []struct {
name string
err error
setup func(svc *agentService)
ctxSetup func(ctx context.Context) context.Context
state statemachine.State
}{
{
name: "Test results not ready",
err: ErrResultsNotReady,
setup: func(svc *agentService) {
},
state: Running,
},
{
name: "Test undeclared consumer",
err: ErrUndeclaredConsumer,
setup: func(svc *agentService) {
svc.computation.ResultConsumers = []ResultConsumer{{UserKey: []byte("user")}}
},
ctxSetup: func(ctx context.Context) context.Context {
return ctx
},
state: ConsumingResults,
},
{
name: "Test results consumed and event sent",
err: nil,
setup: func(svc *agentService) {
svc.computation.ResultConsumers = []ResultConsumer{{UserKey: []byte("key")}}
},
ctxSetup: func(ctx context.Context) context.Context {
return IndexToContext(ctx, 0)
},
state: ConsumingResults,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
ctx := metadata.NewIncomingContext(context.Background(),
metadata.Pairs(algorithm.AlgoTypeKey, "python", python.PyRuntimeKey, python.PyRuntime),
)
if tc.ctxSetup != nil {
ctx = tc.ctxSetup(ctx)
}
sm := new(smmocks.StateMachine)
sm.On("Start", ctx).Return(nil)
sm.On("GetState").Return(tc.state)
sm.On("SendEvent", mock.Anything).Return()
svc := &agentService{
sm: sm,
eventSvc: events,
quoteProvider: qp,
computation: testComputation(t),
}
go func() {
if err := svc.sm.Start(ctx); err != nil {
t.Errorf("Error starting state machine: %v", err)
}
}()
tc.setup(svc)
_, err := svc.Result(ctx)
t.Cleanup(func() {
_ = os.RemoveAll("datasets")
_ = os.RemoveAll("results")
})
assert.ErrorIs(t, err, tc.err, "expected %v, got %v", tc.err, err)
})
}
}
func TestAttestation(t *testing.T) {
events := new(mocks.Service)
qp := new(mocks2.QuoteProvider)
evCall := events.On("SendEvent", mock.Anything, mock.Anything, mock.Anything).Return(nil)
defer evCall.Unset()
cases := []struct {
name string
reportData [ReportDataSize]byte
rawQuote []uint8
err error
}{
{
name: "Test attestation successful",
reportData: generateReportData(),
rawQuote: make([]uint8, 0),
err: nil,
},
{
name: "Test attestation failed",
reportData: generateReportData(),
rawQuote: nil,
err: ErrAttestationFailed,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
ctx := metadata.NewIncomingContext(context.Background(),
metadata.Pairs(algorithm.AlgoTypeKey, "python", python.PyRuntimeKey, python.PyRuntime),
)
ctx, cancel := context.WithCancel(ctx)
defer cancel()
getQuote := qp.On("GetRawQuote", mock.Anything).Return(tc.rawQuote, tc.err)
if tc.err != ErrAttestationFailed {
getQuote = qp.On("GetRawQuote", mock.Anything).Return(tc.reportData, nil)
}
defer getQuote.Unset()
svc := New(ctx, mglog.NewMock(), events, testComputation(t), qp)
time.Sleep(300 * time.Millisecond)
_, err := svc.Attestation(ctx, tc.reportData)
assert.True(t, errors.Contains(err, tc.err), "expected %v, got %v", tc.err, err)
})
}
}
func generateReportData() [ReportDataSize]byte {
bytes := make([]byte, ReportDataSize)
_, err := rand.Read(bytes)
if err != nil {
log.Fatalf("Failed to generate random bytes: %v", err)
}
return [64]byte(bytes)
}
func testComputation(t *testing.T) Computation {
algo, err := os.ReadFile(algoPath)
require.NoError(t, err)
algoHash := sha3.Sum256(algo)
data, err := os.ReadFile(dataPath)
require.NoError(t, err)
dataHash := sha3.Sum256(data)
return Computation{
ID: "1",
Name: "sample computation",
Description: "sample description",
Datasets: []Dataset{{Hash: dataHash, UserKey: []byte("key"), Dataset: data, Filename: datasetFile}},
Algorithm: Algorithm{Hash: algoHash, UserKey: []byte("key"), Algorithm: algo},
ResultConsumers: []ResultConsumer{{UserKey: []byte("key")}},
AgentConfig: AgentConfig{
Port: "7002",
LogLevel: "debug",
AttestedTls: false,
},
}
}
+252
View File
@@ -0,0 +1,252 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package agent
import (
"context"
sync "sync"
"testing"
"time"
"github.com/ultravioletrs/cocos/agent/statemachine"
)
type MockState int
type MockEvent int
func (s MockState) String() string {
return []string{"State1", "State2", "State3"}[s]
}
func (e MockEvent) String() string {
return []string{"Event1", "Event2", "Event3"}[e]
}
const (
State1 MockState = iota
State2
State3
)
const (
Event1 MockEvent = iota
Event2
Event3
)
func TestNewStateMachine(t *testing.T) {
sm := statemachine.NewStateMachine(State1)
if sm == nil {
t.Fatal("NewStateMachine returned nil")
}
if sm.GetState() != State1 {
t.Errorf("Initial state not set correctly, got %v, want %v", sm.GetState(), State1)
}
}
func TestAddTransition(t *testing.T) {
sm := statemachine.NewStateMachine(State1)
sm.AddTransition(statemachine.Transition{From: State1, Event: Event1, To: State2})
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
go func() {
if err := sm.Start(ctx); err != context.Canceled {
t.Errorf("Start returned error: %v", err)
}
}()
sm.SendEvent(Event1)
time.Sleep(50 * time.Millisecond)
if sm.GetState() != State2 {
t.Errorf("Transition not applied correctly, got state %v, want %v", sm.GetState(), State2)
}
}
func TestSetAction(t *testing.T) {
sm := statemachine.NewStateMachine(State1)
var wg sync.WaitGroup
wg.Add(1)
sm.SetAction(State2, func(s statemachine.State) {
defer wg.Done()
})
sm.AddTransition(statemachine.Transition{From: State1, Event: Event1, To: State2})
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
go func() {
if err := sm.Start(ctx); err != context.Canceled {
t.Errorf("Start returned error: %v", err)
}
}()
sm.SendEvent(Event1)
wg.Wait()
if ctx.Err() != nil {
t.Error("Action was not called within the expected time")
}
}
func TestInvalidTransition(t *testing.T) {
sm := statemachine.NewStateMachine(State1)
sm.AddTransition(statemachine.Transition{From: State1, Event: Event1, To: State2})
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
errChan := make(chan error)
go func() {
errChan <- sm.Start(ctx)
}()
sm.SendEvent(Event2)
select {
case err := <-errChan:
if err == nil {
t.Errorf("Expected invalid transition error, got: %v", err)
}
case <-time.After(150 * time.Millisecond):
t.Error("Timeout waiting for invalid transition error")
}
}
func TestMultipleTransitions(t *testing.T) {
sm := statemachine.NewStateMachine(State1)
sm.AddTransition(statemachine.Transition{From: State1, Event: Event1, To: State2})
sm.AddTransition(statemachine.Transition{From: State2, Event: Event2, To: State3})
sm.AddTransition(statemachine.Transition{From: State3, Event: Event3, To: State1})
ctx, cancel := context.WithTimeout(context.Background(), 200*time.Millisecond)
defer cancel()
go func() {
if err := sm.Start(ctx); err != context.Canceled {
t.Errorf("Start returned error: %v", err)
}
}()
transitions := []struct {
event MockEvent
want MockState
}{
{Event1, State2},
{Event2, State3},
{Event3, State1},
}
for _, tt := range transitions {
sm.SendEvent(tt.event)
time.Sleep(50 * time.Millisecond)
if sm.GetState() != tt.want {
t.Errorf("After event %v, got state %v, want %v", tt.event, sm.GetState(), tt.want)
}
}
}
func TestConcurrency(t *testing.T) {
sm := statemachine.NewStateMachine(State1)
sm.AddTransition(statemachine.Transition{From: State1, Event: Event1, To: State2})
sm.AddTransition(statemachine.Transition{From: State2, Event: Event2, To: State1})
ctx, cancel := context.WithTimeout(context.Background(), 500*time.Millisecond)
defer cancel()
go func() {
if err := sm.Start(ctx); err == nil {
t.Errorf("Expected context error, got nil")
}
}()
for i := 0; i < 100; i++ {
go func() {
sm.SendEvent(Event1)
sm.SendEvent(Event2)
}()
}
time.Sleep(400 * time.Millisecond)
finalState := sm.GetState()
if finalState != State1 && finalState != State2 {
t.Errorf("Unexpected final state: %v", finalState)
}
}
func TestAgentEventString(t *testing.T) {
tests := []struct {
event AgentEvent
want string
}{
{Start, "Start"},
{ManifestReceived, "ManifestReceived"},
{AlgorithmReceived, "AlgorithmReceived"},
{DataReceived, "DataReceived"},
{RunComplete, "RunComplete"},
{ResultsConsumed, "ResultsConsumed"},
{RunFailed, "RunFailed"},
{AgentEvent(-1), "AgentEvent(-1)"},
}
for _, tt := range tests {
if got := tt.event.String(); got != tt.want {
t.Errorf("AgentEvent.String() = %v, want %v", got, tt.want)
}
}
}
func TestAgentStateString(t *testing.T) {
tests := []struct {
state AgentState
want string
}{
{Idle, "Idle"},
{ReceivingManifest, "ReceivingManifest"},
{ReceivingAlgorithm, "ReceivingAlgorithm"},
{ReceivingData, "ReceivingData"},
{Running, "Running"},
{ConsumingResults, "ConsumingResults"},
{Complete, "Complete"},
{Failed, "Failed"},
{AgentState(-1), "AgentState(-1)"},
}
for _, tt := range tests {
if got := tt.state.String(); got != tt.want {
t.Errorf("AgentState.String() = %v, want %v", got, tt.want)
}
}
}
func TestStatusString(t *testing.T) {
tests := []struct {
status Status
want string
}{
{IdleState, "IdleState"},
{InProgress, "InProgress"},
{Ready, "Ready"},
{Completed, "Completed"},
{Terminated, "Terminated"},
{Warning, "Warning"},
{Starting, "Starting"},
{Status(uint8(8)), "Status(8)"},
}
for _, tt := range tests {
if got := tt.status.String(); got != tt.want {
t.Errorf("Status.String() = %v, want %v", got, tt.want)
}
}
}
+233
View File
@@ -0,0 +1,233 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
// Code generated by mockery v2.43.2. DO NOT EDIT.
package mocks
import (
context "context"
mock "github.com/stretchr/testify/mock"
statemachine "github.com/ultravioletrs/cocos/agent/statemachine"
)
// StateMachine is an autogenerated mock type for the StateMachine type
type StateMachine struct {
mock.Mock
}
type StateMachine_Expecter struct {
mock *mock.Mock
}
func (_m *StateMachine) EXPECT() *StateMachine_Expecter {
return &StateMachine_Expecter{mock: &_m.Mock}
}
// AddTransition provides a mock function with given fields: t
func (_m *StateMachine) AddTransition(t statemachine.Transition) {
_m.Called(t)
}
// StateMachine_AddTransition_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'AddTransition'
type StateMachine_AddTransition_Call struct {
*mock.Call
}
// AddTransition is a helper method to define mock.On call
// - t statemachine.Transition
func (_e *StateMachine_Expecter) AddTransition(t interface{}) *StateMachine_AddTransition_Call {
return &StateMachine_AddTransition_Call{Call: _e.mock.On("AddTransition", t)}
}
func (_c *StateMachine_AddTransition_Call) Run(run func(t statemachine.Transition)) *StateMachine_AddTransition_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(statemachine.Transition))
})
return _c
}
func (_c *StateMachine_AddTransition_Call) Return() *StateMachine_AddTransition_Call {
_c.Call.Return()
return _c
}
func (_c *StateMachine_AddTransition_Call) RunAndReturn(run func(statemachine.Transition)) *StateMachine_AddTransition_Call {
_c.Call.Return(run)
return _c
}
// GetState provides a mock function with given fields:
func (_m *StateMachine) GetState() statemachine.State {
ret := _m.Called()
if len(ret) == 0 {
panic("no return value specified for GetState")
}
var r0 statemachine.State
if rf, ok := ret.Get(0).(func() statemachine.State); ok {
r0 = rf()
} else {
if ret.Get(0) != nil {
r0 = ret.Get(0).(statemachine.State)
}
}
return r0
}
// StateMachine_GetState_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'GetState'
type StateMachine_GetState_Call struct {
*mock.Call
}
// GetState is a helper method to define mock.On call
func (_e *StateMachine_Expecter) GetState() *StateMachine_GetState_Call {
return &StateMachine_GetState_Call{Call: _e.mock.On("GetState")}
}
func (_c *StateMachine_GetState_Call) Run(run func()) *StateMachine_GetState_Call {
_c.Call.Run(func(args mock.Arguments) {
run()
})
return _c
}
func (_c *StateMachine_GetState_Call) Return(_a0 statemachine.State) *StateMachine_GetState_Call {
_c.Call.Return(_a0)
return _c
}
func (_c *StateMachine_GetState_Call) RunAndReturn(run func() statemachine.State) *StateMachine_GetState_Call {
_c.Call.Return(run)
return _c
}
// SendEvent provides a mock function with given fields: event
func (_m *StateMachine) SendEvent(event statemachine.Event) {
_m.Called(event)
}
// StateMachine_SendEvent_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'SendEvent'
type StateMachine_SendEvent_Call struct {
*mock.Call
}
// SendEvent is a helper method to define mock.On call
// - event statemachine.Event
func (_e *StateMachine_Expecter) SendEvent(event interface{}) *StateMachine_SendEvent_Call {
return &StateMachine_SendEvent_Call{Call: _e.mock.On("SendEvent", event)}
}
func (_c *StateMachine_SendEvent_Call) Run(run func(event statemachine.Event)) *StateMachine_SendEvent_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(statemachine.Event))
})
return _c
}
func (_c *StateMachine_SendEvent_Call) Return() *StateMachine_SendEvent_Call {
_c.Call.Return()
return _c
}
func (_c *StateMachine_SendEvent_Call) RunAndReturn(run func(statemachine.Event)) *StateMachine_SendEvent_Call {
_c.Call.Return(run)
return _c
}
// SetAction provides a mock function with given fields: state, action
func (_m *StateMachine) SetAction(state statemachine.State, action statemachine.Action) {
_m.Called(state, action)
}
// StateMachine_SetAction_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'SetAction'
type StateMachine_SetAction_Call struct {
*mock.Call
}
// SetAction is a helper method to define mock.On call
// - state statemachine.State
// - action statemachine.Action
func (_e *StateMachine_Expecter) SetAction(state interface{}, action interface{}) *StateMachine_SetAction_Call {
return &StateMachine_SetAction_Call{Call: _e.mock.On("SetAction", state, action)}
}
func (_c *StateMachine_SetAction_Call) Run(run func(state statemachine.State, action statemachine.Action)) *StateMachine_SetAction_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(statemachine.State), args[1].(statemachine.Action))
})
return _c
}
func (_c *StateMachine_SetAction_Call) Return() *StateMachine_SetAction_Call {
_c.Call.Return()
return _c
}
func (_c *StateMachine_SetAction_Call) RunAndReturn(run func(statemachine.State, statemachine.Action)) *StateMachine_SetAction_Call {
_c.Call.Return(run)
return _c
}
// Start provides a mock function with given fields: ctx
func (_m *StateMachine) Start(ctx context.Context) error {
ret := _m.Called(ctx)
if len(ret) == 0 {
panic("no return value specified for Start")
}
var r0 error
if rf, ok := ret.Get(0).(func(context.Context) error); ok {
r0 = rf(ctx)
} else {
r0 = ret.Error(0)
}
return r0
}
// StateMachine_Start_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Start'
type StateMachine_Start_Call struct {
*mock.Call
}
// Start is a helper method to define mock.On call
// - ctx context.Context
func (_e *StateMachine_Expecter) Start(ctx interface{}) *StateMachine_Start_Call {
return &StateMachine_Start_Call{Call: _e.mock.On("Start", ctx)}
}
func (_c *StateMachine_Start_Call) Run(run func(ctx context.Context)) *StateMachine_Start_Call {
_c.Call.Run(func(args mock.Arguments) {
run(args[0].(context.Context))
})
return _c
}
func (_c *StateMachine_Start_Call) Return(_a0 error) *StateMachine_Start_Call {
_c.Call.Return(_a0)
return _c
}
func (_c *StateMachine_Start_Call) RunAndReturn(run func(context.Context) error) *StateMachine_Start_Call {
_c.Call.Return(run)
return _c
}
// NewStateMachine creates a new instance of StateMachine. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewStateMachine(t interface {
mock.TestingT
Cleanup(func())
}) *StateMachine {
mock := &StateMachine{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
+112
View File
@@ -0,0 +1,112 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package statemachine
import (
"context"
"fmt"
"sync"
)
type State interface {
String() string
}
type Event interface {
String() string
}
type Action func(State)
type Transition struct {
From State
Event Event
To State
}
type StateMachine interface {
AddTransition(t Transition)
SetAction(state State, action Action)
GetState() State
SendEvent(event Event)
Start(ctx context.Context) error
}
type stateMachine struct {
mu sync.Mutex
currentState State
transitions map[State]map[Event]State
actions map[State]Action
eventChan chan Event
}
func NewStateMachine(initialState State) StateMachine {
return &stateMachine{
currentState: initialState,
transitions: make(map[State]map[Event]State),
actions: make(map[State]Action),
eventChan: make(chan Event),
}
}
func (sm *stateMachine) AddTransition(t Transition) {
sm.mu.Lock()
defer sm.mu.Unlock()
if _, ok := sm.transitions[t.From]; !ok {
sm.transitions[t.From] = make(map[Event]State)
}
sm.transitions[t.From][t.Event] = t.To
}
func (sm *stateMachine) SetAction(state State, action Action) {
sm.mu.Lock()
defer sm.mu.Unlock()
sm.actions[state] = action
}
func (sm *stateMachine) GetState() State {
sm.mu.Lock()
defer sm.mu.Unlock()
return sm.currentState
}
func (sm *stateMachine) SendEvent(event Event) {
sm.eventChan <- event
}
func (sm *stateMachine) Start(ctx context.Context) error {
for {
select {
case event := <-sm.eventChan:
if err := sm.handleEvent(event); err != nil {
return err
}
case <-ctx.Done():
return ctx.Err()
}
}
}
func (sm *stateMachine) handleEvent(event Event) error {
sm.mu.Lock()
currentState := sm.currentState
nextState, valid := sm.transitions[currentState][event]
sm.mu.Unlock()
if !valid {
return fmt.Errorf("invalid transition: %v -> %v", currentState, event)
}
sm.mu.Lock()
sm.currentState = nextState
action := sm.actions[nextState]
sm.mu.Unlock()
if action != nil {
go action(nextState)
}
return nil
}
+29
View File
@@ -0,0 +1,29 @@
// Code generated by "stringer -type=Status"; DO NOT EDIT.
package agent
import "strconv"
func _() {
// An "invalid array index" compiler error signifies that the constant values have changed.
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[IdleState-0]
_ = x[InProgress-1]
_ = x[Ready-2]
_ = x[Completed-3]
_ = x[Terminated-4]
_ = x[Warning-5]
_ = x[Starting-6]
}
const _Status_name = "IdleStateInProgressReadyCompletedTerminatedWarningStarting"
var _Status_index = [...]uint8{0, 9, 19, 24, 33, 43, 50, 58}
func (i Status) String() string {
if i >= Status(len(_Status_index)-1) {
return "Status(" + strconv.FormatInt(int64(i), 10) + ")"
}
return _Status_name[_Status_index[i]:_Status_index[i+1]]
}
-471
View File
@@ -1,471 +0,0 @@
openapi: 3.0.1
info:
title: Computations service
description: HTTP API for managing computations
version: "1.0.0"
paths:
/computations:
post:
summary: Add new computation
description: |
Adds new computation to the list of computations owned by user identified using
the provided access token.
tags:
- computations
requestBody:
$ref: "#/components/requestBodies/ComputationCreateReq"
responses:
'201':
$ref: "#/components/responses/ComputationRes"
'400':
description: Failed due to malformed JSON.
'401':
description: Missing or invalid access token provided.
'409':
description: Entity already exist.
'415':
description: Missing or invalid content type.
'422':
description: Database can't process request.
'500':
$ref: "#/components/responses/ServiceError"
get:
summary: Retrieves computations
description: |
Retrieves a list of computations. Due to performance concerns, data
is retrieved in subsets. The API user must ensure that the entire
dataset is consumed either by making subsequent requests, or by
increasing the subset size of the initial request.
tags:
- computations
parameters:
- $ref: "#/components/parameters/Limit"
- $ref: "#/components/parameters/Offset"
- $ref: "#/components/parameters/Name"
- $ref: "#/components/parameters/Order"
- $ref: "#/components/parameters/Direction"
- $ref: "#/components/parameters/Metadata"
responses:
'200':
$ref: "#/components/responses/ComputationsPageRes"
'400':
description: Failed due to malformed query parameters.
'401':
description: Missing or invalid access token provided.
'404':
description: A non-existent entity request.
'422':
description: Database can't process request.
'500':
$ref: "#/components/responses/ServiceError"
/computations/search:
post:
summary: Search and retrieves computations
description: |
Retrieves a list of computations with name and metadata filtering.
Due to performance concerns, data is retrieved in subsets.
The API user must ensure that the entire
dataset is consumed either by making subsequent requests, or by
increasing the subset size of the initial request.
tags:
- computations
requestBody:
$ref: "#/components/requestBodies/ComputationsSearchReq"
responses:
'200':
$ref: "#/components/responses/ComputationsPageRes"
'400':
description: Failed due to malformed query parameters.
'401':
description: Missing or invalid access token provided.
'404':
description: A non-existent entity request.
'422':
description: Unprocessable Entity
'500':
$ref: "#/components/responses/ServiceError"
/computations/{Id}:
get:
summary: Retrieves computation info
tags:
- computation
parameters:
- $ref: "#/components/parameters/Id"
responses:
'200':
$ref: "#/components/responses/ComputationRes"
'401':
description: Missing or invalid access token provided.
'404':
description: Computation does not exist.
'422':
description: Database can't process request.
'500':
$ref: "#/components/responses/ServiceError"
put:
summary: Updates computation info
description: |
Update is performed by replacing the current resource data with values
provided in a request payload.
tags:
- computations
parameters:
- $ref: "#/components/parameters/Id"
requestBody:
$ref: "#/components/requestBodies/ComputationUpdateReq"
responses:
'200':
description: Computation updated.
'400':
description: Failed due to malformed JSON.
'401':
description: Missing or invalid access token provided.
'404':
description: Computation does not exist.
'415':
description: Missing or invalid content type.
'500':
$ref: "#/components/responses/ServiceError"
delete:
summary: Removes a computation
description: |
Removes a computation.
tags:
- computation
parameters:
- $ref: "#/components/parameters/Id"
responses:
'204':
description: Computation removed.
'400':
description: Failed due to malformed ID.
'401':
description: Missing or invalid access token provided.
'500':
$ref: "#/components/responses/ServiceError"
components:
schemas:
Key:
type: string
format: uuid
description: |
Computation key that is used for computation auth. If there is
not one provided service will generate one in UUID
format.
Identity:
type: object
properties:
id:
type: string
format: uuid
description: Computation unique identifier. This can be either
provided by the user or left blank. If the user provides a UUID,
it would be validated. If there is not one provided then
the service will generate one in UUID format.
ComputationReqSchema:
type: object
properties:
name:
type: string
description: Free-form name.
description:
type: string
description: Free-form description
datasets:
type: array
description: An array of datasets.
minItems: 0
uniqueItems: true
items:
type: string
algorithms:
type: array
description: An array of algorithms.
minItems: 1
uniqueItems: true
items:
type: string
startTime:
type: number
description: Start timestamp of the computations
endTime:
type: number
description: End of the computation
status:
type: string
description: Current status of the computation
owner:
type: string
description: ID of the user who started computation
datasetProviders:
type: array
description: List of IDs of users who provided datasets
minItems: 0
uniqueItems: true
items:
type: string
algorithmProviders:
type: array
description: List of IDs of users who provided algorithms
minItems: 1
uniqueItems: true
items:
type: string
ttl:
type: number
description: Period for computation to live in an inactive state
metadata:
type: object
description: Arbitrary, object-encoded data.
ComputationsReqSchema:
type: object
properties:
name:
type: string
description: Name filter. Filtering is performed as a case-insensitive partial match.
metadata:
type: object
description: Metadata filter. Filtering is performed matching the parameter with metadata on top level. Parameter is json.
total:
type: integer
description: Total number of items.
offset:
type: integer
description: Number of items to skip during retrieval.
default: 0
minimum: 0
limit:
type: integer
description: Size of the subset to retrieve.
default: 10
maximum: 100
minimum: 1
order:
type: string
description: Order type.
default: id
enum:
- name
- id
dir:
type: string
description: Order direction.
default: desc
enum:
- asc
- desc
ComputationResSchema:
type: object
properties:
id:
type: string
format: uuid
description: Unique identifier generated by the service.
name:
type: string
description: Free-form name.
key:
type: string
format: uuid
description: Auto-generated access key.
metadata:
type: object
description: Arbitrary, object-encoded data.
required:
- id
- type
- key
ComputationsPage:
type: object
properties:
computations:
type: array
minItems: 0
uniqueItems: true
items:
$ref: "#/components/schemas/ComputationResSchema"
total:
type: integer
description: Total number of items.
offset:
type: integer
description: Number of items to skip during retrieval.
limit:
type: integer
description: Maximum number of items to return in one page.
required:
- computations
parameters:
Id:
name: Id
description: Unique identifier.
in: path
schema:
type: string
format: uuid
required: true
Limit:
name: limit
description: Size of the subset to retrieve.
in: query
schema:
type: integer
default: 10
maximum: 100
minimum: 1
required: false
Offset:
name: offset
description: Number of items to skip during retrieval.
in: query
schema:
type: integer
default: 0
minimum: 0
required: false
Name:
name: name
description: Name filter. Filtering is performed as a case-insensitive partial match.
in: query
schema:
type: string
required: false
Order:
name: order
description: Order type.
in: query
schema:
type: string
default: id
enum:
- name
- id
required: false
Direction:
name: dir
description: Order direction.
in: query
schema:
type: string
default: desc
enum:
- asc
- desc
required: false
Metadata:
name: metadata
description: Metadata filter. Filtering is performed matching the parameter with metadata on top level. Parameter is json.
in: query
required: false
schema:
type: object
additionalProperties: {}
requestBodies:
ComputationCreateReq:
description: JSON-formatted document describing the new computaton.
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/ComputationReqSchema"
ComputationsCreateReq:
description: JSON-formatted document describing the new computation.
required: true
content:
application/json:
schema:
type: object
properties:
key:
$ref: "#/components/schemas/Key"
computations:
type: array
items:
$ref: "#/components/schemas/ComputationsReqSchema"
ComputationUpdateReq:
description: Arbitrary, object-encoded data.
required: true
content:
application/json:
schema:
type: object
properties:
name:
type: string
description: Free-form name.
metadata:
type: object
ComputationsSearchReq:
description: JSON-formatted document describing search parameters.
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/ComputationReqSchema"
KeyUpdateReq:
required: true
description: JSON containing computations.
content:
application/json:
schema:
type: object
properties:
key:
type: string
format: uuid
description: Computation key that is used for computation auth.
responses:
CreateComputationRes:
description: Computation registered.
headers:
Location:
content:
text/plain:
schema:
type: string
description: Created computation's relative URL.
example: /computations/{id}
ComputationRes:
description: Data retrieved.
content:
application/json:
schema:
$ref: "#/components/schemas/ComputationResSchema"
ComputationsPageRes:
description: Data retrieved.
content:
application/json:
schema:
$ref: "#/components/schemas/ComputationsPage"
IdentityRes:
description: ID returned.
content:
application/json:
schema:
$ref: "#/components/schemas/Identity"
ServiceError:
description: Unexpected server-side error occurred.
content:
application/json:
schema:
type: string
format: byte
HealthRes:
description: Service Health Check.
content:
application/json:
schema:
$ref: "./schemas/HealthInfo.yml"
securitySchemes:
bearerAuth:
type: http
scheme: bearer
bearerFormat: JWT
description: |
* Users access: "Authorization: Bearer <user_token>"
security:
- bearerAuth: []
-446
View File
@@ -1,446 +0,0 @@
openapi: 3.0.1
info:
title: Datasets service
description: HTTP API for managing datasets
version: "1.0.0"
paths:
/datasets:
post:
summary: Add new dataset
description: |
Adds new dataset to the list of datasets owned by user identified using
the provided access token.
tags:
- datasets
requestBody:
$ref: "#/components/requestBodies/DatasetCreateReq"
responses:
'201':
$ref: "#/components/responses/DatasetRes"
'400':
description: Failed due to malformed JSON.
'401':
description: Missing or invalid access token provided.
'409':
description: Entity already exist.
'415':
description: Missing or invalid content type.
'422':
description: Database can't process request.
'500':
$ref: "#/components/responses/ServiceError"
get:
summary: Retrieves datasets
description: |
Retrieves a list of datasets. Due to performance concerns, data
is retrieved in subsets. The API user must ensure that the entire
dataset is consumed either by making subsequent requests, or by
increasing the subset size of the initial request.
tags:
- datasets
parameters:
- $ref: "#/components/parameters/Limit"
- $ref: "#/components/parameters/Offset"
- $ref: "#/components/parameters/Name"
- $ref: "#/components/parameters/Order"
- $ref: "#/components/parameters/Direction"
- $ref: "#/components/parameters/Metadata"
responses:
'200':
$ref: "#/components/responses/DatasetsPageRes"
'400':
description: Failed due to malformed query parameters.
'401':
description: Missing or invalid access token provided.
'404':
description: A non-existent entity request.
'422':
description: Database can't process request.
'500':
$ref: "#/components/responses/ServiceError"
/datasets/search:
post:
summary: Search and retrieves datasets
description: |
Retrieves a list of datasets with name and metadata filtering.
Due to performance concerns, data is retrieved in subsets.
The API user must ensure that the entire
dataset is consumed either by making subsequent requests, or by
increasing the subset size of the initial request.
tags:
- datasets
requestBody:
$ref: "#/components/requestBodies/DatasetsSearchReq"
responses:
'200':
$ref: "#/components/responses/DatasetsPageRes"
'400':
description: Failed due to malformed query parameters.
'401':
description: Missing or invalid access token provided.
'404':
description: A non-existent entity request.
'422':
description: Unprocessable Entity
'500':
$ref: "#/components/responses/ServiceError"
/datasets/{Id}:
get:
summary: Retrieves dataset info
tags:
- dataset
parameters:
- $ref: "#/components/parameters/Id"
responses:
'200':
$ref: "#/components/responses/DatasetRes"
'401':
description: Missing or invalid access token provided.
'404':
description: Dataset does not exist.
'422':
description: Database can't process request.
'500':
$ref: "#/components/responses/ServiceError"
put:
summary: Updates dataset info
description: |
Update is performed by replacing the current resource data with values
provided in a request payload.
tags:
- datasets
parameters:
- $ref: "#/components/parameters/Id"
requestBody:
$ref: "#/components/requestBodies/DatasetUpdateReq"
responses:
'200':
description: dataset updated.
'400':
description: Failed due to malformed JSON.
'401':
description: Missing or invalid access token provided.
'404':
description: Dataset does not exist.
'415':
description: Missing or invalid content type.
'500':
$ref: "#/components/responses/ServiceError"
delete:
summary: Removes a dataset
description: |
Removes a dataset.
tags:
- dataset
parameters:
- $ref: "#/components/parameters/Id"
responses:
'204':
description: Dataset removed.
'400':
description: Failed due to malformed ID.
'401':
description: Missing or invalid access token provided.
'500':
$ref: "#/components/responses/ServiceError"
components:
schemas:
DatasetReqSchema:
type: object
properties:
name:
type: string
description: Free-form name.
description:
type: string
description: Free-form description
datasets:
type: array
description: An array of datasets.
minItems: 0
uniqueItems: true
items:
type: string
algorithms:
type: array
description: An array of algorithms.
minItems: 1
uniqueItems: true
items:
type: string
startTime:
type: number
description: Start timestamp of the datasets
endTime:
type: number
description: End of the dataset
status:
type: string
description: Current status of the dataset
owner:
type: string
description: ID of the user who started dataset
datasetProviders:
type: array
description: List of IDs of users who provided datasets
minItems: 0
uniqueItems: true
items:
type: string
algorithmProviders:
type: array
description: List of IDs of users who provided algorithms
minItems: 1
uniqueItems: true
items:
type: string
ttl:
type: number
description: Period for dataset to live in an inactive state
metadata:
type: object
description: Arbitrary, object-encoded data.
DatasetsReqSchema:
type: object
properties:
name:
type: string
description: Name filter. Filtering is performed as a case-insensitive partial match.
metadata:
type: object
description: Metadata filter. Filtering is performed matching the parameter with metadata on top level. Parameter is json.
total:
type: integer
description: Total number of items.
offset:
type: integer
description: Number of items to skip during retrieval.
default: 0
minimum: 0
limit:
type: integer
description: Size of the subset to retrieve.
default: 10
maximum: 100
minimum: 1
order:
type: string
description: Order type.
default: id
enum:
- name
- id
dir:
type: string
description: Order direction.
default: desc
enum:
- asc
- desc
DatasetResSchema:
type: object
properties:
id:
type: string
format: uuid
description: Unique identifier generated by the service.
name:
type: string
description: Free-form name.
key:
type: string
format: uuid
description: Auto-generated access key.
metadata:
type: object
description: Arbitrary, object-encoded data.
required:
- id
- type
- key
DatasetsPage:
type: object
properties:
datasets:
type: array
minItems: 0
uniqueItems: true
items:
$ref: "#/components/schemas/DatasetResSchema"
total:
type: integer
description: Total number of items.
offset:
type: integer
description: Number of items to skip during retrieval.
limit:
type: integer
description: Maximum number of items to return in one page.
required:
- datasets
parameters:
Id:
name: Id
description: Unique identifier.
in: path
schema:
type: string
format: uuid
required: true
Limit:
name: limit
description: Size of the subset to retrieve.
in: query
schema:
type: integer
default: 10
maximum: 100
minimum: 1
required: false
Offset:
name: offset
description: Number of items to skip during retrieval.
in: query
schema:
type: integer
default: 0
minimum: 0
required: false
Name:
name: name
description: Name filter. Filtering is performed as a case-insensitive partial match.
in: query
schema:
type: string
required: false
Order:
name: order
description: Order type.
in: query
schema:
type: string
default: id
enum:
- name
- id
required: false
Direction:
name: dir
description: Order direction.
in: query
schema:
type: string
default: desc
enum:
- asc
- desc
required: false
Metadata:
name: metadata
description: Metadata filter. Filtering is performed matching the parameter with metadata on top level. Parameter is json.
in: query
required: false
schema:
type: object
additionalProperties: {}
requestBodies:
DatasetCreateReq:
description: JSON-formatted document describing the new dataset.
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/DatasetReqSchema"
DatasetsCreateReq:
description: JSON-formatted document describing the new dataset.
required: true
content:
application/json:
schema:
type: object
properties:
datasets:
type: array
items:
$ref: "#/components/schemas/DatasetsReqSchema"
DatasetUpdateReq:
description: Arbitrary, object-encoded data.
required: true
content:
application/json:
schema:
type: object
properties:
name:
type: string
description: Free-form name.
metadata:
type: object
DatasetsSearchReq:
description: JSON-formatted document describing search parameters.
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/DatasetReqSchema"
KeyUpdateReq:
required: true
description: JSON containing datasets.
content:
application/json:
schema:
type: object
properties:
key:
type: string
format: uuid
description: Thing key that is used for thing auth.
responses:
CreateDatasetRes:
description: Dataset registered.
headers:
Location:
content:
text/plain:
schema:
type: string
description: Created dataset's relative URL.
example: /datasets/{id}
DatasetRes:
description: Data retrieved.
content:
application/json:
schema:
$ref: "#/components/schemas/DatasetResSchema"
DatasetsPageRes:
description: Data retrieved.
content:
application/json:
schema:
$ref: "#/components/schemas/DatasetsPage"
ServiceError:
description: Unexpected server-side error occurred.
content:
application/json:
schema:
type: string
format: byte
HealthRes:
description: Service Health Check.
content:
application/json:
schema:
$ref: "./schemas/HealthInfo.yml"
securitySchemes:
bearerAuth:
type: http
scheme: bearer
bearerFormat: JWT
description: |
* Users access: "Authorization: Bearer <user_token>"
security:
- bearerAuth: []
@@ -1,84 +0,0 @@
{
"info": {
"_postman_id": "5ed31077-ca87-496e-9b98-b8793ce7c5d2",
"name": "CoCoS",
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
},
"item": [
{
"name": "Create Computation",
"request": {
"method": "POST",
"header": [],
"body": {
"mode": "raw",
"raw": "{\n \"name\": \"Calculate salary\",\n \"description\": \"my test computation\",\n \"ttl\": 1243\n}",
"options": {
"raw": {
"language": "json"
}
}
},
"url": {
"raw": "http://localhost:9000/computations",
"protocol": "http",
"host": [
"localhost"
],
"port": "9000",
"path": [
"computations"
]
}
},
"response": []
},
{
"name": "View Computation",
"request": {
"method": "GET",
"header": [],
"url": {
"raw": "http://localhost:9000/computations/c89b53d3-c088-4b5a-addd-51a90ea94325",
"protocol": "http",
"host": [
"localhost"
],
"port": "9000",
"path": [
"computations",
"c89b53d3-c088-4b5a-addd-51a90ea94325"
]
}
},
"response": []
}
],
"event": [
{
"listen": "prerequest",
"script": {
"type": "text/javascript",
"exec": [
""
]
}
},
{
"listen": "test",
"script": {
"type": "text/javascript",
"exec": [
""
]
}
}
],
"variable": [
{
"key": "jwt",
"value": "",
"type": "string"
}
]
}
BIN
View File
Binary file not shown.
+12
View File
@@ -0,0 +1,12 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
//go:build embed
// +build embed
package cocosai
import _ "embed"
//go:embed attestation.bin
var EmbeddedAttestation []byte
+2
View File
@@ -0,0 +1,2 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
+102
View File
@@ -0,0 +1,102 @@
# Agent CLI
This repository contains the command-line interface (CLI) tool for interacting with the Agent and manager service. The CLI allows you to perform various tasks such as running computations, uploading algorithm and datasets, and retrieving results.
## Build
From the project root:
```bash
make cli
```
## Usage
#### Get attestation
Retrieves attestation information from the SEV guest and saves it to a file.
To retrieve attestation from agent, use the following command:
```bash
./build/cocos-cli attestation get '<report_data>'
```
#### Validate attestation
Validates the retrieved attestation information against a specified policy and checks its authenticity.
To validate and verify attestation from agent, use the following command:
```bash
./build/cocos-cli attestation validate '<attestation>' --report_data '<report_data>'
```
##### Flags
- --config: Path to a JSON file containing the validation configuration. This can be used to override individual flags.
- --report_data: Hex-encoded string representing the attestation report data. (Required for the validate command)
- --host_data: Hex-encoded string representing the expected HOST_DATA field.
- --family_id: Hex-encoded string representing the expected FAMILY_ID field.
- --image_id: Hex-encoded string representing the expected IMAGE_ID field.
- --report_id: Hex-encoded string representing the expected REPORT_ID field.
- --report_id_ma: Hex-encoded string representing the expected REPORT_ID_MA field.
- --measurement: Hex-encoded string representing the expected MEASUREMENT field.
- --chip_id: Hex-encoded string representing the expected CHIP_ID field.
- --minimum_tcb: Minimum acceptable value for CURRENT_TCB, COMMITTED_TCB, and REPORTED_TCB.
- --minimum_launch_tcb: Minimum acceptable value for LAUNCH_TCB.
- --minimum_guest_svn: Minimum acceptable SnpPolicy.
- --minimum_build: Minimum 8-bit build number for AMD-SP firmware.
- --check_crl: Download and check the CRL for revoked certificates (default: false).
- --disallow_network: If true, disallows network downloads for verification (default: false).
- --timeout: Duration to continue retrying failed HTTP requests (default: 2 minutes).
- --max_retry_delay: Maximum duration to wait between HTTP request retries (default: 30 seconds).
- --require_author_key: Require that AUTHOR_KEY_EN is set to 1 (default: false).
- --require_id_block: Require that the VM was launched with an ID_BLOCK signed by a trusted key (default: false).
- --permit_provisional_software: Allow provisional firmware (default: false).
- --platform_info: Maximum acceptable PLATFORM_INFO field bit-wise (optional).
- --minimum_version: Minimum AMD-SP firmware API version (major.minor) (optional).
- --trusted_author_keys: Paths to x.509 certificates of trusted author keys (optional).
- --trusted_author_key_hashes: Hex-encoded SHA-384 hashes of trusted author keys (optional).
- --trusted_id_keys: Paths to x.509 certificates of trusted identity keys (optional).
- --trusted_id_key_hashes: Hex-encoded SHA-384 hashes of trusted identity keys (optional).
- --product: AMD product name for the chip that generated the attestation report (optional).
- --stepping: Machine stepping for the chip that generated the attestation report (optional).
- --CA_bundles_paths: Paths to CA bundles for the AMD product (optional).
- --CA_bundles: PEM format CA bundles for the AMD product (optional).
#### Upload Algorithm
To upload an algorithm, use the following command:
```bash
./build/cocos-cli algo /path/to/algorithm <private_key_file_path>
```
##### Flags
- -a, --algorithm string Algorithm type to run (default "bin")
- --args stringArray Arguments to pass to the algorithm
- --python-runtime string Python runtime to use (default "python3")
- -r, --requirements string Python requirements file
#### Upload Dataset
To upload a dataset, use the following command:
```bash
./build/cocos-cli data /path/to/dataset.csv <private_key_file_path>
```
Users can also upload directories which will be compressed on transit. Once received by agent they will be stored as compressed files or decompressed if the user passed the decompression argument.
##### Flags
- -d, --decompress Decompress the dataset on agent
#### Retrieve result
To retrieve the computation result, use the following command:
```bash
./build/cocos-cli result <private_key_file_path>
```
#### Checksum
When defining the manifest dataset and algorithm checksums are required. This can be done as below:
```bash
./build/cocos-cli checksum <path_to_dataset_or_algorithm>
```
+180
View File
@@ -0,0 +1,180 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"bytes"
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"errors"
"os"
"testing"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/ultravioletrs/cocos/pkg/sdk/mocks"
)
const algorithmFile = "test_algo_file.py"
func generateRSAPrivateKeyFile(fileName string) error {
privateKey, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return err
}
privKeyFile, err := os.Create(fileName)
if err != nil {
return err
}
defer privKeyFile.Close()
privateKeyPEM := &pem.Block{
Type: rsaKeyType,
Bytes: x509.MarshalPKCS1PrivateKey(privateKey),
}
err = pem.Encode(privKeyFile, privateKeyPEM)
if err != nil {
return err
}
return nil
}
func TestAlgorithmCmd(t *testing.T) {
tests := []struct {
name string
setupMock func(*mocks.SDK)
setupFiles func() error
args []string
connectErr error
expectedOutput string
cleanup func()
}{
{
name: "successful upload",
setupMock: func(m *mocks.SDK) {
m.On("Algo", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
},
setupFiles: func() error {
if err := os.WriteFile(algorithmFile, []byte("test algorithm"), 0o644); err != nil {
return err
}
return generateRSAPrivateKeyFile(privateKeyFile)
},
args: []string{algorithmFile, privateKeyFile},
expectedOutput: "Successfully uploaded algorithm",
cleanup: func() {
os.Remove(privateKeyFile)
os.Remove(algorithmFile)
},
},
{
name: "missing algorithm file",
setupMock: func(m *mocks.SDK) {
m.On("Algo", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
},
args: []string{"non_existent_algo_file.py", privateKeyFile},
expectedOutput: "Error reading algorithm file",
},
{
name: "missing private key file",
setupMock: func(m *mocks.SDK) {
m.On("Algo", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
},
setupFiles: func() error {
return os.WriteFile(algorithmFile, []byte("test algorithm"), 0o644)
},
args: []string{algorithmFile, "non_existent_private_key.pem"},
expectedOutput: "Error reading private key file",
cleanup: func() {
os.Remove(algorithmFile)
},
},
{
name: "upload failure",
setupMock: func(m *mocks.SDK) {
m.On("Algo", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(errors.New("failed to upload algorithm due to error"))
},
setupFiles: func() error {
if err := os.WriteFile(algorithmFile, []byte("test algorithm"), 0o644); err != nil {
return err
}
return generateRSAPrivateKeyFile(privateKeyFile)
},
args: []string{algorithmFile, privateKeyFile},
expectedOutput: "Failed to upload algorithm",
cleanup: func() {
os.Remove(privateKeyFile)
os.Remove(algorithmFile)
},
},
{
name: "invalid private key",
setupMock: func(m *mocks.SDK) {
m.On("Algo", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
},
setupFiles: func() error {
if err := os.WriteFile(algorithmFile, []byte("test algorithm"), 0o644); err != nil {
return err
}
privKeyFile, err := os.Create(privateKeyFile)
if err != nil {
return err
}
defer privKeyFile.Close()
_, err = privKeyFile.WriteString("invalid private key")
return err
},
args: []string{algorithmFile, privateKeyFile},
expectedOutput: "Error decoding private key",
cleanup: func() {
os.Remove(algorithmFile)
os.Remove(privateKeyFile)
},
},
{
name: "connection error",
setupMock: func(m *mocks.SDK) {
},
connectErr: errors.New("failed to connect to agent"),
args: []string{algorithmFile, privateKeyFile},
expectedOutput: "Failed to connect to agent",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockSDK := new(mocks.SDK)
if tt.setupMock != nil {
tt.setupMock(mockSDK)
}
testCLI := CLI{
agentSDK: mockSDK,
connectErr: tt.connectErr,
}
if tt.setupFiles != nil {
err := tt.setupFiles()
require.NoError(t, err)
}
cmd := testCLI.NewAlgorithmCmd()
buf := new(bytes.Buffer)
cmd.SetOut(buf)
cmd.SetArgs(tt.args)
err := cmd.Execute()
require.NoError(t, err)
require.Contains(t, buf.String(), tt.expectedOutput)
if tt.cleanup != nil {
tt.cleanup()
}
})
}
}
+96
View File
@@ -0,0 +1,96 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"context"
"encoding/pem"
"os"
"github.com/fatih/color"
"github.com/spf13/cobra"
"github.com/ultravioletrs/cocos/agent/algorithm"
"github.com/ultravioletrs/cocos/agent/algorithm/python"
"google.golang.org/grpc/metadata"
)
var (
pythonRuntime string
algoType string
requirementsFile string
algoArgs []string
)
func (cli *CLI) NewAlgorithmCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "algo",
Short: "Upload an algorithm binary",
Example: "algo <algo_file> <private_key_file_path>",
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
if cli.connectErr != nil {
printError(cmd, "Failed to connect to agent: %v ❌ ", cli.connectErr)
return
}
algorithmFile := args[0]
cmd.Println("Uploading algorithm file:", algorithmFile)
algorithm, err := os.Open(algorithmFile)
if err != nil {
printError(cmd, "Error reading algorithm file: %v ❌ ", err)
return
}
defer algorithm.Close()
var req *os.File
if requirementsFile != "" {
req, err = os.Open(requirementsFile)
if err != nil {
printError(cmd, "Error reading requirments file: %v ❌ ", err)
return
}
defer req.Close()
}
privKeyFile, err := os.ReadFile(args[1])
if err != nil {
printError(cmd, "Error reading private key file: %v ❌ ", err)
return
}
pemBlock, _ := pem.Decode(privKeyFile)
privKey, err := decodeKey(pemBlock)
if err != nil {
printError(cmd, "Error decoding private key: %v ❌ ", err)
return
}
ctx := metadata.NewOutgoingContext(cmd.Context(), metadata.New(make(map[string]string)))
if err := cli.agentSDK.Algo(addAlgoMetadata(ctx), algorithm, req, privKey); err != nil {
printError(cmd, "Failed to upload algorithm due to error: %v ❌ ", err)
return
}
cmd.Println(color.New(color.FgGreen).Sprint("Successfully uploaded algorithm! ✔ "))
},
}
cmd.Flags().StringVarP(&algoType, "algorithm", "a", string(algorithm.AlgoTypeBin), "Algorithm type to run")
cmd.Flags().StringVar(&pythonRuntime, "python-runtime", python.PyRuntime, "Python runtime to use")
cmd.Flags().StringVarP(&requirementsFile, "requirements", "r", "", "Python requirements file")
cmd.Flags().StringArrayVar(&algoArgs, "args", []string{}, "Arguments to pass to the algorithm")
return cmd
}
func addAlgoMetadata(ctx context.Context) context.Context {
ctx = algorithm.AlgorithmTypeToContext(ctx, algoType)
ctx = algorithm.AlgorithmArgsToContext(ctx, algoArgs)
ctx = python.PythonRunTimeToContext(ctx, pythonRuntime)
return ctx
}
+701
View File
@@ -0,0 +1,701 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"encoding/hex"
"encoding/json"
"fmt"
"os"
"strconv"
"strings"
"time"
"github.com/absmach/magistrala/pkg/errors"
"github.com/fatih/color"
"github.com/google/go-sev-guest/abi"
"github.com/google/go-sev-guest/proto/check"
"github.com/google/go-sev-guest/proto/sevsnp"
"github.com/google/go-sev-guest/tools/lib/report"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/pkg/attestation/quoteprovider"
"google.golang.org/protobuf/encoding/protojson"
"google.golang.org/protobuf/types/known/wrapperspb"
)
const (
defaultMinimumTcb = 0
defaultMinimumLaunchTcb = 0
defaultMinimumGuestSvn = 0
defaultGuestPolicy = 0x0000000000030000
defaultMinimumBuild = 0
defaultCheckCrl = false
defaultTimeout = 2 * time.Minute
defaultMaxRetryDelay = 30 * time.Second
defaultRequireAuthor = false
defaultRequireIdBlock = false
defaultMinVersion = "0.0"
size16 = 16
size32 = 32
size48 = 48
size64 = 64
attestationFilePath = "attestation.bin"
attestationJson = "attestation.json"
sevProductNameMilan = "Milan"
sevProductNameGenoa = "Genoa"
exampleJSONConfig = `
{
"rootOfTrust":{
"product":"test_product",
"cabundlePaths":[
"test_cabundlePaths"
],
"cabundles":[
"test_Cabundles"
],
"checkCrl":true,
"disallowNetwork":true
},
"policy":{
"minimumGuestSvn":1,
"policy":"1",
"familyId":"AQIDBAUGBwgJCgsMDQ4PEA==",
"imageId":"AQIDBAUGBwgJCgsMDQ4PEA==",
"vmpl":0,
"minimumTcb":"1",
"minimumLaunchTcb":"1",
"platformInfo":"1",
"requireAuthorKey":true,
"reportData":"J+60aXs8btm8VcGgaJYURGeNCu0FIyWMFXQ7ZUlJDC0FJGJizJsOzDIXgQ75UtPC+Zqe0A3dvnnf5VEeQ61RTg==",
"measurement":"8s78ewoX7Xkfy1qsgVnkZwLDotD768Nqt6qTL5wtQOxHsLczipKM6bhDmWiHLdP4",
"hostData":"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw=",
"reportId":"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw=",
"reportIdMa":"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw=",
"chipId":"J+60aXs8btm8VcGgaJYURGeNCu0FIyWMFXQ7ZUlJDC0FJGJizJsOzDIXgQ75UtPC+Zqe0A3dvnnf5VEeQ61RTg==",
"minimumBuild":1,
"minimumVersion":"0.90",
"permitProvisionalFirmware":true,
"requireIdBlock":true,
"trustedAuthorKeys":[
"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw="
],
"trustedAuthorKeyHashes":[
"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw="
],
"trustedIdKeys":[
"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw="
],
"trustedIdKeyHashes":[
"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw="
],
"product":{
"name":"SEV_PRODUCT_MILAN",
"stepping":1,
"machineStepping":1
}
}
}
`
)
var (
cfg = check.Config{Policy: &check.Policy{}, RootOfTrust: &check.RootOfTrust{}}
cfgString string
timeout time.Duration
maxRetryDelay time.Duration
platformInfo string
stepping string
trustedAuthorKeys []string
trustedAuthorHashes []string
trustedIdKeys []string
trustedIdKeyHashes []string
attestationFile string
attestation []byte
empty16 = [size16]byte{}
empty32 = [size32]byte{}
empty64 = [size64]byte{}
defaultReportIdMa = []byte{255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255}
getJsonAttestation bool
errReportSize = errors.New("attestation contents too small")
)
func (cli *CLI) NewAttestationCmd() *cobra.Command {
return &cobra.Command{
Use: "attestation [command]",
Short: "Get and validate attestations",
Run: func(cmd *cobra.Command, args []string) {
cmd.Printf("Get and validate attestations\n\n")
cmd.Printf("Usage:\n %s [command]\n\n", cmd.CommandPath())
cmd.Printf("Available Commands:\n")
// Filter out "completion" command
availableCommands := make([]*cobra.Command, 0)
for _, subCmd := range cmd.Commands() {
if subCmd.Name() != "completion" {
availableCommands = append(availableCommands, subCmd)
}
}
for _, subCmd := range availableCommands {
cmd.Printf(" %-15s%s\n", subCmd.Name(), subCmd.Short)
}
cmd.Printf("\nFlags:\n")
cmd.Flags().VisitAll(func(flag *pflag.Flag) {
cmd.Printf(" -%s, --%s %s\n", flag.Shorthand, flag.Name, flag.Usage)
})
cmd.Printf("\nUse \"%s [command] --help\" for more information about a command.\n", cmd.CommandPath())
},
}
}
func (cli *CLI) NewGetAttestationCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "get",
Short: "Retrieve attestation information from agent. Report data expected in hex enoded string of length 64 bytes.",
Example: "get <report_data>",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
if cli.connectErr != nil {
printError(cmd, "Failed to connect to agent: %v ❌ ", cli.connectErr)
return
}
cmd.Println("Getting attestation")
reportData, err := hex.DecodeString(args[0])
if err != nil {
printError(cmd, "Error decoding report data: %v ❌ ", err)
return
}
if len(reportData) != agent.ReportDataSize {
msg := color.New(color.FgRed).Sprintf("report data must be a hex encoded string of length %d bytes ❌ ", agent.ReportDataSize)
cmd.Println(msg)
return
}
filename := attestationFilePath
if getJsonAttestation {
filename = attestationJson
}
attestationFile, err := os.Create(filename)
if err != nil {
printError(cmd, "Error creating attestation file: %v ❌ ", err)
return
}
if err := cli.agentSDK.Attestation(cmd.Context(), [agent.ReportDataSize]byte(reportData), attestationFile); err != nil {
printError(cmd, "Failed to get attestation due to error: %v ❌ ", err)
return
}
if err := attestationFile.Close(); err != nil {
printError(cmd, "Error closing attestation file: %v ❌ ", err)
return
}
if getJsonAttestation {
result, err := os.ReadFile(filename)
if err != nil {
printError(cmd, "Error reading attestation file: %v ❌ ", err)
return
}
result, err = attesationToJSON(result)
if err != nil {
printError(cmd, "Error converting attestation to json: %v ❌ ", err)
return
}
if err := os.WriteFile(filename, result, 0o644); err != nil {
printError(cmd, "Error writing attestation file: %v ❌ ", err)
return
}
}
cmd.Println("Attestation result retrieved and saved successfully!")
},
}
cmd.Flags().BoolVarP(&getJsonAttestation, "json", "j", false, "Get attestation in json format")
return cmd
}
func attesationToJSON(report []byte) ([]byte, error) {
if len(report) < abi.ReportSize {
return nil, errors.Wrap(errReportSize, fmt.Errorf("attestation contents too small (0x%x bytes). Want at least 0x%x bytes", len(report), abi.ReportSize))
}
attestationPB, err := abi.ReportCertsToProto(report[:abi.ReportSize])
if err != nil {
return nil, err
}
return json.MarshalIndent(attestationPB, "", " ")
}
func attesationFromJSON(reportFile []byte) ([]byte, error) {
var attestationPB sevsnp.Attestation
if err := json.Unmarshal(reportFile, &attestationPB); err != nil {
return nil, err
}
return report.Transform(&attestationPB, "bin")
}
func isFileJSON(filename string) bool {
return strings.HasSuffix(filename, ".json")
}
func (cli *CLI) NewValidateAttestationValidationCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "validate",
Short: "Validate and verify attestation information. The report is provided as a file path.",
Example: "validate <attestation_report_file_path>",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
cmd.Println("Checking attestation")
attestationFile = string(args[0])
if err := parseConfig(); err != nil {
printError(cmd, "Error parsing config: %v ❌ ", err)
return
}
if err := parseHashes(); err != nil {
printError(cmd, "Error parsing hashes: %v ❌ ", err)
return
}
if err := parseFiles(); err != nil {
printError(cmd, "Error parsing files: %v ❌ ", err)
return
}
// This format is the attestation report in AMD's specified ABI format, immediately
// followed by the certificate table bytes.
if len(attestation) < abi.ReportSize {
msg := color.New(color.FgRed).Sprintf("attestation contents too small (0x%x bytes). Want at least 0x%x bytes ❌ ", len(attestation), abi.ReportSize)
cmd.Println(msg)
return
}
if err := parseUints(); err != nil {
printError(cmd, "Error parsing uints: %v ❌ ", err)
return
}
cfg.Policy.Vmpl = wrapperspb.UInt32(0)
if err := validateInput(); err != nil {
printError(cmd, "Error validating input: %v ❌ ", err)
return
}
if err := quoteprovider.VerifyAndValidate(attestation, &cfg); err != nil {
printError(cmd, "Attestation validation and verification failed with error: %v ❌ ", err)
return
}
cmd.Println("Attestation validation and verification is successful!")
},
}
cmd.Flags().StringVar(
&cfgString,
"config",
"",
"Serialized json check.Config protobuf. This will overwrite individual flags. Unmarshalled as json. Example: "+exampleJSONConfig,
)
cmd.Flags().BytesHexVar(
&cfg.Policy.HostData,
"host_data",
empty32[:],
"The expected HOST_DATA field as a hex string. Must encode 32 bytes. Unchecked if unset.",
)
cmd.Flags().BytesHexVar(
&cfg.Policy.ReportData,
"report_data",
empty64[:],
"The expected REPORT_DATA field as a hex string. Must encode 64 bytes. Must be set.",
)
cmd.Flags().BytesHexVar(
&cfg.Policy.FamilyId,
"family_id",
empty16[:],
"The expected FAMILY_ID field as a hex string. Must encode 16 bytes. Unchecked if unset.",
)
cmd.Flags().BytesHexVar(
&cfg.Policy.ImageId,
"image_id",
empty16[:],
"The expected IMAGE_ID field as a hex string. Must encode 16 bytes. Unchecked if unset.",
)
cmd.Flags().BytesHexVar(
&cfg.Policy.ReportId,
"report_id",
nil,
"The expected REPORT_ID field as a hex string. Must encode 32 bytes. Unchecked if unset.",
)
cmd.Flags().BytesHexVar(
&cfg.Policy.ReportIdMa,
"report_id_ma",
defaultReportIdMa,
"The expected REPORT_ID_MA field as a hex string. Must encode 32 bytes. Unchecked if unset.",
)
cmd.Flags().BytesHexVar(
&cfg.Policy.Measurement,
"measurement",
nil,
"The expected MEASUREMENT field as a hex string. Must encode 48 bytes. Unchecked if unset.",
)
cmd.Flags().BytesHexVar(
&cfg.Policy.ChipId,
"chip_id",
nil,
"The expected MEASUREMENT field as a hex string. Must encode 48 bytes. Unchecked if unset.",
)
cmd.Flags().Uint64Var(
&cfg.Policy.MinimumTcb,
"minimum_tcb",
defaultMinimumTcb,
"The minimum acceptable value for CURRENT_TCB, COMMITTED_TCB, and REPORTED_TCB.",
)
cmd.Flags().Uint64Var(
&cfg.Policy.MinimumLaunchTcb,
"minimum_lauch_tcb",
defaultMinimumLaunchTcb,
"The minimum acceptable value for LAUNCH_TCB.",
)
cmd.Flags().Uint64Var(
&cfg.Policy.Policy,
"guest_policy",
defaultGuestPolicy,
"The most acceptable guest SnpPolicy.",
)
cmd.Flags().Uint32Var(
&cfg.Policy.MinimumGuestSvn,
"minimum_guest_svn",
defaultMinimumGuestSvn,
"The most acceptable GUEST_SVN.",
)
cmd.Flags().Uint32Var(
&cfg.Policy.MinimumBuild,
"minimum_build",
defaultMinimumBuild,
"The 8-bit minimum build number for AMD-SP firmware",
)
cmd.Flags().BoolVar(
&cfg.RootOfTrust.CheckCrl,
"check_crl",
defaultCheckCrl,
"Download and check the CRL for revoked certificates.",
)
cmd.Flags().DurationVar(
&timeout,
"timeout",
defaultTimeout,
"Duration to continue to retry failed HTTP requests.",
)
cmd.Flags().DurationVar(
&maxRetryDelay,
"max_retry_delay",
defaultMaxRetryDelay,
"Maximum Duration to wait between HTTP request retries.",
)
cmd.Flags().BoolVar(
&cfg.Policy.RequireAuthorKey,
"require_author_key",
defaultRequireAuthor,
"Require that AUTHOR_KEY_EN is 1.",
)
cmd.Flags().BoolVar(
&cfg.Policy.RequireIdBlock,
"require_id_block",
defaultRequireIdBlock,
"Require that the VM was launch with an ID_BLOCK signed by a trusted id key or author key",
)
cmd.Flags().StringVar(
&platformInfo,
"platform_info",
"",
"The maximum acceptable PLATFORM_INFO field bit-wise. May be empty or a 64-bit unsigned integer",
)
cmd.Flags().StringVar(
&cfg.Policy.MinimumVersion,
"minimum_version",
defaultMinVersion,
"Minimum AMD-SP firmware API version (major.minor). Each number must be 8-bit non-negative.",
)
cmd.Flags().StringArrayVar(
&trustedAuthorKeys,
"trusted_author_keys",
[]string{},
"Paths to x.509 certificates of trusted author keys",
)
cmd.Flags().StringArrayVar(
&trustedAuthorHashes,
"trusted_author_key_hashes",
[]string{},
"Hex-encoded SHA-384 hash values of trusted author keys in AMD public key format",
)
cmd.Flags().StringArrayVar(
&trustedIdKeys,
"trusted_id_keys",
[]string{},
"Paths to x.509 certificates of trusted author keys",
)
cmd.Flags().StringArrayVar(
&trustedIdKeyHashes,
"trusted_id_key_hashes",
[]string{},
"Hex-encoded SHA-384 hash values of trusted identity keys in AMD public key format",
)
cmd.Flags().StringVar(
&cfg.RootOfTrust.ProductLine,
"product",
"",
"The AMD product name for the chip that generated the attestation report.",
)
cmd.Flags().StringVar(
&stepping,
"stepping",
"",
"The machine stepping for the chip that generated the attestation report. Default unchecked.",
)
cmd.Flags().StringArrayVar(
&cfg.RootOfTrust.CabundlePaths,
"CA_bundles_paths",
[]string{},
"Paths to CA bundles for the AMD product. Must be in PEM format, ASK, then ARK certificates. If unset, uses embedded root certificates.",
)
cmd.Flags().StringArrayVar(
&cfg.RootOfTrust.Cabundles,
"CA_bundles",
[]string{},
"PEM format CA bundles for the AMD product. Combined with contents of cabundle_paths.",
)
if err := cmd.MarkFlagRequired("report_data"); err != nil {
printError(cmd, "Failed to mark flag as required: %v ❌ ", err)
return nil
}
if err := cmd.MarkFlagRequired("product"); err != nil {
printError(cmd, "Failed to mark flag as required: %v ❌ ", err)
return nil
}
return cmd
}
// parseConfig decodes config passed as json for check.Config struct.
// example
/* {
"rootOfTrust":{
"product":"test_product",
"cabundlePaths":[
"test_cabundlePaths"
],
"cabundles":[
"test_Cabundles"
],
"checkCrl":true,
"disallowNetwork":true
},
"policy":{
"minimumGuestSvn":1,
"policy":"1",
"familyId":"AQIDBAUGBwgJCgsMDQ4PEA==",
"imageId":"AQIDBAUGBwgJCgsMDQ4PEA==",
"vmpl":0,
"minimumTcb":"1",
"minimumLaunchTcb":"1",
"platformInfo":"1",
"requireAuthorKey":true,
"reportData":"J+60aXs8btm8VcGgaJYURGeNCu0FIyWMFXQ7ZUlJDC0FJGJizJsOzDIXgQ75UtPC+Zqe0A3dvnnf5VEeQ61RTg==",
"measurement":"8s78ewoX7Xkfy1qsgVnkZwLDotD768Nqt6qTL5wtQOxHsLczipKM6bhDmWiHLdP4",
"hostData":"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw=",
"reportId":"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw=",
"reportIdMa":"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw=",
"chipId":"J+60aXs8btm8VcGgaJYURGeNCu0FIyWMFXQ7ZUlJDC0FJGJizJsOzDIXgQ75UtPC+Zqe0A3dvnnf5VEeQ61RTg==",
"minimumBuild":1,
"minimumVersion":"0.90",
"permitProvisionalFirmware":true,
"requireIdBlock":true,
"trustedAuthorKeys":[
"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw="
],
"trustedAuthorKeyHashes":[
"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw="
],
"trustedIdKeys":[
"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw="
],
"trustedIdKeyHashes":[
"GSvLKpfu59Y9QOF6vhq0vQsOIvb4+5O/UOHLGLBTkdw="
],
"product":{
"name":"SEV_PRODUCT_MILAN",
"stepping":1,
"machineStepping":1
}
}
}*/
func parseConfig() error {
if cfgString == "" {
return nil
}
if err := protojson.Unmarshal([]byte(cfgString), &cfg); err != nil {
return err
}
// Populate fields that should not be nil
if cfg.RootOfTrust == nil {
cfg.RootOfTrust = &check.RootOfTrust{}
}
if cfg.Policy == nil {
cfg.Policy = &check.Policy{}
}
return nil
}
func parseHashes() error {
for _, hash := range trustedAuthorHashes {
hashBytes, err := hex.DecodeString(hash)
if err != nil {
return err
}
cfg.Policy.TrustedAuthorKeyHashes = append(cfg.Policy.TrustedAuthorKeyHashes, hashBytes)
}
for _, hash := range trustedIdKeyHashes {
hashBytes, err := hex.DecodeString(hash)
if err != nil {
return err
}
cfg.Policy.TrustedIdKeyHashes = append(cfg.Policy.TrustedIdKeyHashes, hashBytes)
}
return nil
}
func parseFiles() error {
file, err := os.ReadFile(attestationFile)
if err != nil {
return err
}
attestation = file
if isFileJSON(attestationFile) {
attestation, err = attesationFromJSON(attestation)
if err != nil {
return err
}
}
for _, path := range trustedAuthorKeys {
file, err := os.ReadFile(path)
if err != nil {
return err
}
cfg.Policy.TrustedAuthorKeys = append(cfg.Policy.TrustedAuthorKeys, file)
}
for _, path := range trustedIdKeys {
file, err := os.ReadFile(path)
if err != nil {
return err
}
cfg.Policy.TrustedIdKeys = append(cfg.Policy.TrustedIdKeys, file)
}
return nil
}
func parseUints() error {
if stepping != "" {
if base := getBase(stepping); base == 10 {
num, err := strconv.ParseUint(stepping, getBase(stepping), 8)
if err != nil {
return err
}
cfg.Policy.Product.MachineStepping = wrapperspb.UInt32(uint32(num))
} else {
num, err := strconv.ParseUint(stepping[2:], base, 8)
if err != nil {
return err
}
cfg.Policy.Product.MachineStepping = wrapperspb.UInt32(uint32(num))
}
}
if platformInfo != "" {
if base := getBase(platformInfo); base == 10 {
num, err := strconv.ParseUint(platformInfo, getBase(platformInfo), 8)
if err != nil {
return err
}
cfg.Policy.PlatformInfo = wrapperspb.UInt64(num)
} else {
num, err := strconv.ParseUint(platformInfo[2:], base, 8)
if err != nil {
return err
}
cfg.Policy.PlatformInfo = wrapperspb.UInt64(num)
}
}
return nil
}
func getBase(val string) int {
switch {
case strings.HasPrefix(val, "0x"):
return 16
case strings.HasPrefix(val, "0o"):
return 8
case strings.HasPrefix(val, "0b"):
return 2
default:
return 10
}
}
func validateInput() error {
if len(cfg.RootOfTrust.CabundlePaths) != 0 || len(cfg.RootOfTrust.Cabundles) != 0 && cfg.RootOfTrust.ProductLine == "" {
return fmt.Errorf("product name must be set if CA bundles are provided")
}
if err := validateFieldLength("report_data", cfg.Policy.ReportData, size64); err != nil {
return err
}
if err := validateFieldLength("host_data", cfg.Policy.HostData, size32); err != nil {
return err
}
if err := validateFieldLength("family_id", cfg.Policy.FamilyId, size16); err != nil {
return err
}
if err := validateFieldLength("image_id", cfg.Policy.ImageId, size16); err != nil {
return err
}
if err := validateFieldLength("report_id", cfg.Policy.ReportId, size32); err != nil {
return err
}
if err := validateFieldLength("report_id_ma", cfg.Policy.ReportIdMa, size32); err != nil {
return err
}
if err := validateFieldLength("measurement", cfg.Policy.Measurement, size48); err != nil {
return err
}
if err := validateFieldLength("chip_id", cfg.Policy.ChipId, size64); err != nil {
return err
}
for _, hash := range cfg.Policy.TrustedAuthorKeyHashes {
if err := validateFieldLength("trusted_author_key_hash", hash, size48); err != nil {
return err
}
}
for _, hash := range cfg.Policy.TrustedIdKeyHashes {
if err := validateFieldLength("trusted_id_key_hash", hash, size48); err != nil {
return err
}
}
return nil
}
func validateFieldLength(fieldName string, field []byte, expectedLength int) error {
if field != nil && len(field) != expectedLength {
return fmt.Errorf("%s length should be at least %d bytes long", fieldName, expectedLength)
}
return nil
}
+140
View File
@@ -0,0 +1,140 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"encoding/base64"
"fmt"
"os"
"github.com/absmach/magistrala/pkg/errors"
"github.com/google/go-sev-guest/proto/check"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"google.golang.org/protobuf/encoding/protojson"
)
type fieldType int
const (
measurementField fieldType = iota
hostDataField
)
const (
// 0o744 file permission gives RWX permission to the user and only the R permission to others.
filePermission = 0o744
// Length of the expected host data and measurement field in bytes.
hostDataLength = 32
measurementLength = 48
)
var (
errDecode = errors.New("base64 string could not be decoded")
errDataLength = errors.New("data does not have an adequate length")
errReadingAttestationPolicyFile = errors.New("error while reading the attestation policy file")
errUnmarshalJSON = errors.New("failed to unmarshal json")
errMarshalJSON = errors.New("failed to marshal json")
errWriteFile = errors.New("failed to write to file")
errAttestationPolicyField = errors.New("the specified field type does not exist in the attestation policy")
)
func (cli *CLI) NewAttestationPolicyCmd() *cobra.Command {
return &cobra.Command{
Use: "policy [command]",
Short: "Change attestation policy",
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("Change attestation policy\n\n")
fmt.Printf("Usage:\n %s [command]\n\n", cmd.CommandPath())
fmt.Printf("Available Commands:\n")
// Filter out "completion" command
availableCommands := make([]*cobra.Command, 0)
for _, subCmd := range cmd.Commands() {
if subCmd.Name() != "completion" {
availableCommands = append(availableCommands, subCmd)
}
}
for _, subCmd := range availableCommands {
fmt.Printf(" %-15s%s\n", subCmd.Name(), subCmd.Short)
}
fmt.Printf("\nFlags:\n")
cmd.Flags().VisitAll(func(flag *pflag.Flag) {
fmt.Printf(" -%s, --%s %s\n", flag.Shorthand, flag.Name, flag.Usage)
})
fmt.Printf("\nUse \"%s [command] --help\" for more information about a command.\n", cmd.CommandPath())
},
}
}
func (cli *CLI) NewAddMeasurementCmd() *cobra.Command {
return &cobra.Command{
Use: "measurement",
Short: "Add measurement to the attestation policy file. The value should be in base64. The second parameter is attestation_policy.json file",
Example: "measurement <measurement> <attestation_policy.json>",
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
if err := changeAttestationConfiguration(args[1], args[0], measurementLength, measurementField); err != nil {
printError(cmd, "Error could not change measurement data: %v ❌ ", err)
return
}
},
}
}
func (cli *CLI) NewAddHostDataCmd() *cobra.Command {
return &cobra.Command{
Use: "hostdata",
Short: "Add host data to the attestation policy file. The value should be in base64. The second parameter is attestation_policy.json file",
Example: "hostdata <host-data> <attestation_policy.json>",
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
if err := changeAttestationConfiguration(args[1], args[0], hostDataLength, hostDataField); err != nil {
printError(cmd, "Error could not change host data: %v ❌ ", err)
return
}
},
}
}
func changeAttestationConfiguration(fileName, base64Data string, expectedLength int, field fieldType) error {
data, err := base64.StdEncoding.DecodeString(base64Data)
if err != nil {
return errDecode
}
if len(data) != expectedLength {
return errDataLength
}
ac := check.Config{Policy: &check.Policy{}, RootOfTrust: &check.RootOfTrust{}}
attestationPolicy, err := os.ReadFile(fileName)
if err != nil {
return errors.Wrap(errReadingAttestationPolicyFile, err)
}
if err = protojson.Unmarshal(attestationPolicy, &ac); err != nil {
return errors.Wrap(errUnmarshalJSON, err)
}
switch field {
case measurementField:
ac.Policy.Measurement = data
case hostDataField:
ac.Policy.HostData = data
default:
return errAttestationPolicyField
}
fileJson, err := protojson.Marshal(&ac)
if err != nil {
return errors.Wrap(errMarshalJSON, err)
}
if err = os.WriteFile(fileName, fileJson, filePermission); err != nil {
return errors.Wrap(errWriteFile, err)
}
return nil
}
+131
View File
@@ -0,0 +1,131 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"encoding/base64"
"os"
"testing"
"github.com/google/go-sev-guest/proto/check"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/protobuf/encoding/protojson"
)
func TestChangeAttestationConfiguration(t *testing.T) {
tmpfile, err := os.CreateTemp("", "attestation_policy.json")
require.NoError(t, err)
defer os.Remove(tmpfile.Name())
initialConfig := check.Config{Policy: &check.Policy{}, RootOfTrust: &check.RootOfTrust{}}
initialJSON, err := protojson.Marshal(&initialConfig)
require.NoError(t, err)
err = os.WriteFile(tmpfile.Name(), initialJSON, 0o644)
require.NoError(t, err)
tests := []struct {
name string
base64Data string
expectedLength int
field fieldType
expectError bool
errorType error
}{
{
name: "Valid Measurement",
base64Data: base64.StdEncoding.EncodeToString(make([]byte, measurementLength)),
expectedLength: measurementLength,
field: measurementField,
expectError: false,
},
{
name: "Valid Host Data",
base64Data: base64.StdEncoding.EncodeToString(make([]byte, hostDataLength)),
expectedLength: hostDataLength,
field: hostDataField,
expectError: false,
},
{
name: "Invalid Base64",
base64Data: "Invalid Base64",
expectedLength: measurementLength,
field: measurementField,
expectError: true,
errorType: errDecode,
},
{
name: "Invalid Data Length",
base64Data: base64.StdEncoding.EncodeToString(make([]byte, measurementLength-1)),
expectedLength: measurementLength,
field: measurementField,
expectError: true,
errorType: errDataLength,
},
{
name: "Invalid Field Type",
base64Data: base64.StdEncoding.EncodeToString(make([]byte, measurementLength)),
expectedLength: measurementLength,
field: fieldType(999),
expectError: true,
errorType: errAttestationPolicyField,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := changeAttestationConfiguration(tmpfile.Name(), tt.base64Data, tt.expectedLength, tt.field)
if tt.expectError {
assert.Error(t, err)
assert.ErrorIs(t, err, tt.errorType)
} else {
assert.NoError(t, err)
content, err := os.ReadFile(tmpfile.Name())
require.NoError(t, err)
config := check.Config{Policy: &check.Policy{}, RootOfTrust: &check.RootOfTrust{}}
err = protojson.Unmarshal(content, &config)
require.NoError(t, err)
decodedData, _ := base64.StdEncoding.DecodeString(tt.base64Data)
if tt.field == measurementField {
assert.Equal(t, decodedData, config.Policy.Measurement)
} else if tt.field == hostDataField {
assert.Equal(t, decodedData, config.Policy.HostData)
}
}
})
}
}
func TestNewAttestationPolicyCmd(t *testing.T) {
cli := &CLI{}
cmd := cli.NewAttestationPolicyCmd()
assert.Equal(t, "policy [command]", cmd.Use)
assert.Equal(t, "Change attestation policy", cmd.Short)
assert.NotNil(t, cmd.Run)
}
func TestNewAddMeasurementCmd(t *testing.T) {
cli := &CLI{}
cmd := cli.NewAddMeasurementCmd()
assert.Equal(t, "measurement", cmd.Use)
assert.Equal(t, "Add measurement to the attestation policy file. The value should be in base64. The second parameter is attestation_policy.json file", cmd.Short)
assert.Equal(t, "measurement <measurement> <attestation_policy.json>", cmd.Example)
assert.NotNil(t, cmd.Run)
}
func TestNewAddHostDataCmd(t *testing.T) {
cli := &CLI{}
cmd := cli.NewAddHostDataCmd()
assert.Equal(t, "hostdata", cmd.Use)
assert.Equal(t, "Add host data to the attestation policy file. The value should be in base64. The second parameter is attestation_policy.json file", cmd.Short)
assert.Equal(t, "hostdata <host-data> <attestation_policy.json>", cmd.Example)
assert.NotNil(t, cmd.Run)
}
+450
View File
@@ -0,0 +1,450 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"bytes"
"encoding/hex"
"encoding/json"
"fmt"
"os"
"testing"
"github.com/absmach/magistrala/pkg/errors"
"github.com/google/go-sev-guest/abi"
"github.com/google/go-sev-guest/proto/check"
"github.com/google/go-sev-guest/proto/sevsnp"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/pkg/sdk/mocks"
)
func TestNewAttestationCmd(t *testing.T) {
mockSDK := new(mocks.SDK)
cli := &CLI{agentSDK: mockSDK}
cmd := cli.NewAttestationCmd()
assert.Equal(t, "attestation [command]", cmd.Use)
assert.Equal(t, "Get and validate attestations", cmd.Short)
var buf bytes.Buffer
cmd.SetOut(&buf)
cmd.SetOutput(&buf)
reportData := bytes.Repeat([]byte{0x01}, agent.ReportDataSize)
mockSDK.On("Attestation", mock.Anything, [agent.ReportDataSize]byte(reportData), mock.Anything).Return(nil)
cmd.SetArgs([]string{hex.EncodeToString(reportData)})
err := cmd.Execute()
assert.NoError(t, err)
assert.Contains(t, buf.String(), "Get and validate attestations")
}
func TestNewGetAttestationCmd(t *testing.T) {
validattestation, err := os.ReadFile("../attestation.bin")
require.NoError(t, err)
testCases := []struct {
name string
args []string
mockResponse []byte
mockError error
expectedErr string
expectedOut string
}{
{
name: "successful attestation retrieval",
args: []string{hex.EncodeToString(bytes.Repeat([]byte{0x01}, agent.ReportDataSize))},
mockResponse: []byte("mock attestation"),
mockError: nil,
expectedOut: "Attestation result retrieved and saved successfully!",
},
{
name: "invalid report data (decoding error)",
args: []string{"invalid"},
mockResponse: nil,
mockError: errors.New("error"),
expectedErr: "Error decoding report data",
},
{
name: "invalid report data size",
args: []string{hex.EncodeToString(bytes.Repeat([]byte{0x01}, 32))},
mockResponse: nil,
mockError: errors.New("error"),
expectedErr: "report data must be a hex encoded string of length 64 bytes",
},
{
name: "invalid report data hex",
args: []string{"invalid"},
mockResponse: nil,
mockError: errors.New("error"),
expectedErr: "Error decoding report data",
},
{
name: "failed to get attestation",
args: []string{hex.EncodeToString(bytes.Repeat([]byte{0x01}, agent.ReportDataSize))},
mockResponse: nil,
mockError: errors.New("error"),
expectedErr: "Failed to get attestation due to error",
},
{
name: "JSON report error",
args: []string{hex.EncodeToString(bytes.Repeat([]byte{0x01}, agent.ReportDataSize)), "--json"},
mockResponse: []byte("mock attestation"),
mockError: nil,
expectedErr: "Error converting attestation to json",
},
{
name: "successful JSON report",
args: []string{hex.EncodeToString(bytes.Repeat([]byte{0x01}, agent.ReportDataSize)), "--json"},
mockResponse: validattestation,
mockError: nil,
expectedOut: "Attestation result retrieved and saved successfully!",
},
{
name: "connection error",
args: []string{hex.EncodeToString(bytes.Repeat([]byte{0x01}, agent.ReportDataSize))},
mockResponse: nil,
mockError: errors.New("failed to connect to agent"),
expectedErr: "Failed to connect to agent",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
t.Cleanup(func() {
os.Remove(attestationFilePath)
os.Remove(attestationJson)
})
mockSDK := new(mocks.SDK)
cli := &CLI{agentSDK: mockSDK}
if tc.name == "connection error" {
cli.connectErr = errors.New("failed to connect to agent")
}
cmd := cli.NewGetAttestationCmd()
var buf bytes.Buffer
cmd.SetOutput(&buf)
mockSDK.On("Attestation", mock.Anything, [agent.ReportDataSize]byte(bytes.Repeat([]byte{0x01}, agent.ReportDataSize)), mock.Anything).Return(tc.mockError).Run(func(args mock.Arguments) {
_, err := args.Get(2).(*os.File).Write(tc.mockResponse)
require.NoError(t, err)
})
cmd.SetArgs(tc.args)
err := cmd.Execute()
if tc.expectedErr != "" {
assert.Contains(t, buf.String(), tc.expectedErr)
} else {
assert.NoError(t, err)
assert.Contains(t, buf.String(), tc.expectedOut)
}
})
}
}
func TestNewValidateAttestationValidationCmd(t *testing.T) {
cli := &CLI{}
cmd := cli.NewValidateAttestationValidationCmd()
assert.Equal(t, "validate", cmd.Use)
assert.Equal(t, "Validate and verify attestation information. The report is provided as a file path.", cmd.Short)
assert.Equal(t, fmt.Sprint(defaultMinimumTcb), cmd.Flag("minimum_tcb").Value.String())
assert.Equal(t, fmt.Sprint(defaultMinimumLaunchTcb), cmd.Flag("minimum_lauch_tcb").Value.String())
assert.Equal(t, fmt.Sprint(defaultGuestPolicy), cmd.Flag("guest_policy").Value.String())
assert.Equal(t, fmt.Sprint(defaultMinimumGuestSvn), cmd.Flag("minimum_guest_svn").Value.String())
assert.Equal(t, fmt.Sprint(defaultMinimumBuild), cmd.Flag("minimum_build").Value.String())
assert.Equal(t, defaultCheckCrl, cmd.Flag("check_crl").Value.String() == "true")
assert.Equal(t, fmt.Sprint(defaultTimeout), cmd.Flag("timeout").Value.String())
assert.Equal(t, fmt.Sprint(defaultMaxRetryDelay), cmd.Flag("max_retry_delay").Value.String())
}
func TestParseConfig(t *testing.T) {
cfgString = ""
err := parseConfig()
assert.NoError(t, err)
assert.NotNil(t, cfg.RootOfTrust)
assert.NotNil(t, cfg.Policy)
cfgString = `{"rootOfTrust":{"product":"test_product"},"policy":{"minimumGuestSvn":1}}`
err = parseConfig()
assert.NoError(t, err)
assert.Equal(t, "test_product", cfg.RootOfTrust.Product)
assert.Equal(t, uint32(1), cfg.Policy.MinimumGuestSvn)
cfgString = `{"invalid_json"`
err = parseConfig()
assert.Error(t, err)
}
func TestParseHashes(t *testing.T) {
trustedAuthorHashes = []string{"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}
trustedIdKeyHashes = []string{"fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210"}
cfg = check.Config{}
if cfg.Policy == nil {
cfg.Policy = &check.Policy{}
}
err := parseHashes()
assert.NoError(t, err)
assert.Len(t, cfg.Policy.TrustedAuthorKeyHashes, 1)
assert.Len(t, cfg.Policy.TrustedIdKeyHashes, 1)
trustedAuthorHashes = []string{"invalid_hash"}
err = parseHashes()
assert.Error(t, err)
}
func TestParseFiles(t *testing.T) {
attestationFile = "test_attestation.bin"
authorKeyFile := "test_author_key.pem"
idKeyFile := "test_id_key.pem"
err := os.WriteFile(attestationFile, []byte("test attestation"), 0o644)
assert.NoError(t, err)
err = os.WriteFile(authorKeyFile, []byte("test author key"), 0o644)
assert.NoError(t, err)
err = os.WriteFile(idKeyFile, []byte("test id key"), 0o644)
assert.NoError(t, err)
trustedAuthorKeys = []string{authorKeyFile}
trustedIdKeys = []string{idKeyFile}
err = parseFiles()
assert.NoError(t, err)
assert.Equal(t, []byte("test attestation"), attestation)
assert.Len(t, cfg.Policy.TrustedAuthorKeys, 1)
assert.Len(t, cfg.Policy.TrustedIdKeys, 1)
os.Remove(attestationFile)
os.Remove(authorKeyFile)
os.Remove(idKeyFile)
attestationFile = "non_existent_file.bin"
err = parseFiles()
assert.Error(t, err)
}
func TestParseUints(t *testing.T) {
stepping = "10"
platformInfo = "0xFF"
cfg = check.Config{}
if cfg.Policy == nil {
cfg.Policy = &check.Policy{
Product: &sevsnp.SevProduct{},
}
}
err := parseUints()
assert.NoError(t, err)
assert.Equal(t, uint32(10), cfg.Policy.Product.MachineStepping.Value)
assert.Equal(t, uint64(255), cfg.Policy.PlatformInfo.Value)
stepping = "invalid"
err = parseUints()
assert.Error(t, err)
stepping = "10"
platformInfo = "invalid"
err = parseUints()
assert.Error(t, err)
}
func TestValidateInput(t *testing.T) {
cfg = check.Config{}
if cfg.Policy == nil {
cfg.Policy = &check.Policy{}
}
if cfg.RootOfTrust == nil {
cfg.RootOfTrust = &check.RootOfTrust{}
}
cfg.Policy.ReportData = make([]byte, 64)
cfg.Policy.HostData = make([]byte, 32)
cfg.Policy.FamilyId = make([]byte, 16)
cfg.Policy.ImageId = make([]byte, 16)
cfg.Policy.ReportId = make([]byte, 32)
cfg.Policy.ReportIdMa = make([]byte, 32)
cfg.Policy.Measurement = make([]byte, 48)
cfg.Policy.ChipId = make([]byte, 64)
err := validateInput()
assert.NoError(t, err)
cfg.Policy.ReportData = make([]byte, 32)
err = validateInput()
assert.Error(t, err)
}
func TestGetBase(t *testing.T) {
assert.Equal(t, 16, getBase("0xFF"))
assert.Equal(t, 8, getBase("0o77"))
assert.Equal(t, 2, getBase("0b1010"))
assert.Equal(t, 10, getBase("123"))
}
func TestAttestationToJSON(t *testing.T) {
validReport, err := os.ReadFile("../attestation.bin")
require.NoError(t, err)
tests := []struct {
name string
input []byte
err error
}{
{
name: "Valid report",
input: validReport,
err: nil,
},
{
name: "Invalid report size",
input: make([]byte, abi.ReportSize-1),
err: errReportSize,
},
{
name: "Nil input",
input: nil,
err: errReportSize,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := attesationToJSON(tt.input)
assert.True(t, errors.Contains(err, tt.err))
if tt.err != nil {
assert.Nil(t, got)
return
}
require.NotNil(t, got)
var js map[string]interface{}
err = json.Unmarshal(got, &js)
assert.NoError(t, err)
})
}
}
func TestAttestationFromJSON(t *testing.T) {
tests := []struct {
name string
input []byte
err error
validate func(t *testing.T, output []byte)
}{
{
name: "Valid JSON",
input: func() []byte {
att := &sevsnp.Attestation{
Report: &sevsnp.Report{
CurrentTcb: 1,
FamilyId: make([]byte, 16),
ImageId: make([]byte, 16),
ReportData: make([]byte, 64),
Measurement: make([]byte, 48),
HostData: make([]byte, 32),
IdKeyDigest: make([]byte, 48),
AuthorKeyDigest: make([]byte, 48),
ReportId: make([]byte, 32),
ReportIdMa: make([]byte, 32),
ChipId: make([]byte, 64),
Signature: make([]byte, 512),
},
}
data, err := json.Marshal(att)
require.NoError(t, err)
return data
}(),
err: nil,
validate: func(t *testing.T, output []byte) {
assert.NotEmpty(t, output)
},
},
{
name: "Invalid JSON",
input: []byte(`{"invalid": json`),
err: errors.New("invalid character 'j' looking for beginning of value"),
validate: func(t *testing.T, output []byte) {
assert.Nil(t, output)
},
},
{
name: "Empty input",
input: []byte{},
err: errors.New("unexpected end of JSON input"),
validate: func(t *testing.T, output []byte) {
assert.Nil(t, output)
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := attesationFromJSON(tt.input)
assert.True(t, errors.Contains(err, tt.err))
tt.validate(t, got)
})
}
}
func TestIsFileJSON(t *testing.T) {
tests := []struct {
name string
filename string
want bool
}{
{
name: "Valid JSON extension",
filename: "test.json",
want: true,
},
{
name: "Valid JSON extension with path",
filename: "/path/to/test.json",
want: true,
},
{
name: "Invalid extension",
filename: "test.txt",
want: false,
},
{
name: "No extension",
filename: "test",
want: false,
},
{
name: "JSON in filename",
filename: "json.txt",
want: false,
},
{
name: "Empty string",
filename: "",
want: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := isFileJSON(tt.filename)
assert.Equal(t, tt.want, got)
})
}
}
func TestRoundTrip(t *testing.T) {
originalReport, err := os.ReadFile("../attestation.bin")
require.NoError(t, err)
jsonData, err := attesationToJSON(originalReport)
require.NoError(t, err)
require.NotNil(t, jsonData)
roundTripReport, err := attesationFromJSON(jsonData)
require.NoError(t, err)
require.NotNil(t, roundTripReport)
}
+78
View File
@@ -0,0 +1,78 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"fmt"
"os"
"path"
"github.com/google/go-sev-guest/abi"
"github.com/google/go-sev-guest/kds"
"github.com/google/go-sev-guest/proto/check"
"github.com/google/go-sev-guest/verify/trust"
"github.com/spf13/cobra"
"github.com/ultravioletrs/cocos/pkg/clients/grpc"
)
const (
caBundleName = "ask_ark.pem"
filePermisionKeys = 0o766
)
func (cli *CLI) NewCABundleCmd(fileSavePath string) *cobra.Command {
return &cobra.Command{
Use: "ca-bundle",
Short: "Fetch AMD SEV-SNPs CA Bundle (ASK and ARK)",
Example: "ca-bundle <path_to_platform_info_json>",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
attestationConfiguration := check.Config{Policy: &check.Policy{}, RootOfTrust: &check.RootOfTrust{}}
err := grpc.ReadAttestationPolicy(args[0], &attestationConfiguration)
if err != nil {
printError(cmd, "Error while reading manifest: %v ❌ ", err)
return
}
product := attestationConfiguration.RootOfTrust.ProductLine
getter := trust.DefaultHTTPSGetter()
caURL := kds.ProductCertChainURL(abi.VcekReportSigner, product)
bundle, err := getter.Get(caURL)
if err != nil {
message := fmt.Sprintf("Error fetching ARK and ASK from AMD KDS for product: %s", product)
message += ", error: %v ❌ "
printError(cmd, message, err)
return
}
err = os.MkdirAll(path.Join(fileSavePath, product), filePermisionKeys)
if err != nil {
message := fmt.Sprintf("Error while creating directory for product name %s", product)
message += ", error: %v ❌ "
printError(cmd, message, err)
return
}
bundlePath := path.Join(fileSavePath, product, caBundleName)
if err = saveToFile(bundlePath, bundle); err != nil {
printError(cmd, "Error while saving ARK-ASK to file: %v ❌ ", err)
return
}
},
}
}
func saveToFile(fileSavePath string, content []byte) error {
file, err := os.OpenFile(fileSavePath, os.O_CREATE|os.O_RDWR|os.O_TRUNC, filePermisionKeys)
if err != nil {
return err
}
if _, err := file.Write(content); err != nil {
return err
}
return nil
}
+59
View File
@@ -0,0 +1,59 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"bytes"
"os"
"path"
"testing"
"github.com/stretchr/testify/assert"
)
func TestNewCABundleCmd(t *testing.T) {
cli := &CLI{}
tempDir, err := os.MkdirTemp("", "ca-bundle-test")
assert.NoError(t, err)
defer os.RemoveAll(tempDir)
manifestContent := []byte(`{"root_of_trust": {"product_line": "Milan"}}`)
manifestPath := path.Join(tempDir, "manifest.json")
err = os.WriteFile(manifestPath, manifestContent, 0o644)
assert.NoError(t, err)
cmd := cli.NewCABundleCmd(tempDir)
cmd.SetArgs([]string{manifestPath})
output := &bytes.Buffer{}
cmd.SetOutput(output)
err = cmd.Execute()
assert.NoError(t, err)
expectedFilePath := path.Join(tempDir, "Milan", caBundleName)
_, err = os.Stat(expectedFilePath)
assert.NoError(t, err)
content, err := os.ReadFile(expectedFilePath)
assert.NoError(t, err)
assert.NotNil(t, content)
}
func TestSaveToFile(t *testing.T) {
tempDir, err := os.MkdirTemp("", "save-to-file-test")
assert.NoError(t, err)
defer os.RemoveAll(tempDir)
filePath := path.Join(tempDir, "test-file.txt")
content := []byte("test content")
err = saveToFile(filePath, content)
assert.NoError(t, err)
savedContent, err := os.ReadFile(filePath)
assert.NoError(t, err)
assert.Equal(t, content, savedContent)
_, err = os.Stat(filePath)
assert.NoError(t, err)
}
+95
View File
@@ -0,0 +1,95 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"encoding/base64"
"encoding/hex"
"encoding/json"
"os"
"github.com/spf13/cobra"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/internal"
"golang.org/x/crypto/sha3"
)
var (
ismanifest bool
toBase64 bool
)
func (cli *CLI) NewFileHashCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "checksum",
Short: "Compute the sha3-256 hash of a file",
Example: "checksum <file>",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
path := args[0]
if ismanifest {
hash, err := manifestChecksum(path)
if err != nil {
printError(cmd, "Error computing hash: %v ❌ ", err)
return
}
cmd.Println("Hash of manifest file:", hashOut(hash))
return
}
hash, err := internal.ChecksumHex(path)
if err != nil {
printError(cmd, "Error computing hash: %v ❌ ", err)
return
}
cmd.Println("Hash of file:", hashOut(hash))
},
}
cmd.Flags().BoolVarP(&ismanifest, "manifest", "m", false, "Compute the hash of the manifest file")
cmd.Flags().BoolVarP(&toBase64, "base64", "b", false, "Output the hash in base64")
return cmd
}
func manifestChecksum(path string) (string, error) {
file, err := os.ReadFile(path)
if err != nil {
return "", err
}
var cmp agent.Computation
if err := json.Unmarshal(file, &cmp); err != nil {
return "", err
}
jsonBytes, err := json.Marshal(cmp)
if err != nil {
return "", err
}
sum := sha3.Sum256(jsonBytes)
return hex.EncodeToString(sum[:]), nil
}
func hashOut(hashHex string) string {
if toBase64 {
return hexToBase64(hashHex)
}
return hashHex
}
func hexToBase64(hexStr string) string {
decoded, err := hex.DecodeString(hexStr)
if err != nil {
return ""
}
return base64.StdEncoding.EncodeToString(decoded)
}
+230
View File
@@ -0,0 +1,230 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"bytes"
"fmt"
"os"
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
func TestNewFileHashCmd(t *testing.T) {
cli := &CLI{}
cmd := cli.NewFileHashCmd()
if cmd.Use != "checksum" {
t.Errorf("Expected Use to be 'checksum', got %s", cmd.Use)
}
if cmd.Short != "Compute the sha3-256 hash of a file" {
t.Errorf("Expected Short to be 'Compute the sha3-256 hash of a file', got %s", cmd.Short)
}
if cmd.Example != "checksum <file>" {
t.Errorf("Expected Example to be 'checksum <file>', got %s", cmd.Example)
}
}
func TestNewFileHashCmdRun(t *testing.T) {
testCases := []struct {
name string
isManifest bool
toBase64 bool
expectedOut string
expectedErr string
}{
{
name: "Valid file",
isManifest: false,
toBase64: false,
expectedOut: "Hash of file:",
expectedErr: "",
},
{
name: "Valid manifest file",
isManifest: true,
toBase64: false,
expectedOut: "Hash of manifest file:",
expectedErr: "",
},
{
name: "Valid file with base64 output",
isManifest: false,
toBase64: true,
expectedOut: "Hash of file:",
expectedErr: "",
},
{
name: "Non-existent file",
isManifest: false,
toBase64: false,
expectedOut: "Error computing hash:",
expectedErr: "",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
cli := &CLI{}
cmd := cli.NewFileHashCmd()
var output bytes.Buffer
cmd.SetOut(&output)
cmd.SetErr(&output)
err := cmd.Flags().Set("manifest", fmt.Sprint(tc.isManifest))
assert.Nil(t, err)
err = cmd.Flags().Set("base64", fmt.Sprint(tc.toBase64))
assert.Nil(t, err)
if tc.name == "Non-existent file" {
cmd.SetArgs([]string{"non_existent_file.txt"})
} else {
content := []byte("{}")
tmpfile, err := os.CreateTemp("", "example")
if err != nil {
t.Fatal(err)
}
defer os.Remove(tmpfile.Name())
if _, err := tmpfile.Write(content); err != nil {
t.Fatal(err)
}
if err := tmpfile.Close(); err != nil {
t.Fatal(err)
}
cmd.SetArgs([]string{tmpfile.Name()})
}
err = cmd.Execute()
if err != nil {
t.Fatalf("Error executing command: %v", err)
}
out := output.String()
if !strings.Contains(out, tc.expectedOut) {
t.Errorf("Expected output to contain '%s', got '%s'", tc.expectedOut, out)
}
if tc.expectedErr != "" && !strings.Contains(out, tc.expectedErr) {
t.Errorf("Expected output to contain '%s', got '%s'", tc.expectedErr, out)
}
})
}
}
func TestManifestChecksum(t *testing.T) {
testCases := []struct {
name string
jsonContent string
expectedSum string
}{
{
name: "Valid manifest file",
jsonContent: `{
"id": "1234",
"name": "Example Computation",
"description": "This is an example computation"
}`,
expectedSum: "868825367c32c4b6d621d5d95e2890f233d8554df2348ab743aac2663a936f08",
},
{
name: "Invalid JSON",
jsonContent: `{`,
expectedSum: "",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
f, err := os.CreateTemp("", "test")
assert.Nil(t, err)
t.Cleanup(func() {
os.Remove(f.Name())
})
_, err = f.WriteString(tc.jsonContent)
assert.NoError(t, err)
err = f.Close()
assert.Nil(t, err)
hash, err := manifestChecksum(f.Name())
if tc.expectedSum == "" && err == nil {
t.Errorf("Expected error, got nil")
}
if tc.expectedSum != "" && err != nil {
t.Errorf("Unexpected error: %v", err)
}
if hash != tc.expectedSum {
t.Errorf("Expected hash %s, got %s", tc.expectedSum, hash)
}
})
}
}
func TestHexToBase64(t *testing.T) {
testCases := []struct {
name string
hexInput string
expectedOut string
}{
{
name: "Valid hex input",
hexInput: "48656c6c6f",
expectedOut: "SGVsbG8=",
},
{
name: "Invalid hex input",
hexInput: "invalid-hex",
expectedOut: "",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
out := hexToBase64(tc.hexInput)
if out != tc.expectedOut {
t.Errorf("Expected %s, got %s", tc.expectedOut, out)
}
})
}
}
func TestHashOut(t *testing.T) {
testCases := []struct {
name string
hashHex string
toBase64 bool
expectedOut string
}{
{
name: "Hex output",
hashHex: "48656c6c6f",
toBase64: false,
expectedOut: "48656c6c6f",
},
{
name: "Base64 output",
hashHex: "48656c6c6f",
toBase64: true,
expectedOut: "SGVsbG8=",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
toBase64 = tc.toBase64
out := hashOut(tc.hashHex)
if out != tc.expectedOut {
t.Errorf("Expected %s, got %s", tc.expectedOut, out)
}
})
}
}
+118
View File
@@ -0,0 +1,118 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"context"
"crypto/x509"
"encoding/pem"
"os"
"path"
"github.com/absmach/magistrala/pkg/errors"
"github.com/fatih/color"
"github.com/spf13/cobra"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/internal"
"google.golang.org/grpc/metadata"
)
var decompressDataset bool
func (cli *CLI) NewDatasetsCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "data",
Short: "Upload a dataset",
Example: "data <dataset_path> <private_key_file_path>",
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
if cli.connectErr != nil {
printError(cmd, "Failed to connect to agent: %v ❌ ", cli.connectErr)
return
}
datasetPath := args[0]
cmd.Println("Uploading dataset:", datasetPath)
f, err := os.Stat(datasetPath)
if err != nil {
printError(cmd, "Error reading dataset file: %v ❌ ", err)
return
}
var dataset *os.File
if f.IsDir() {
dataset, err = internal.ZipDirectoryToTempFile(datasetPath)
if err != nil {
printError(cmd, "Error zipping dataset directory: %v ❌ ", err)
return
}
defer dataset.Close()
defer os.Remove(dataset.Name())
} else {
dataset, err = os.Open(datasetPath)
if err != nil {
printError(cmd, "Error reading dataset file: %v ❌ ", err)
return
}
defer dataset.Close()
}
privKeyFile, err := os.ReadFile(args[1])
if err != nil {
printError(cmd, "Error reading private key file: %v ❌ ", err)
return
}
pemBlock, _ := pem.Decode(privKeyFile)
privKey, err := decodeKey(pemBlock)
if err != nil {
printError(cmd, "Error decoding private key: %v ❌ ", err)
return
}
ctx := metadata.NewOutgoingContext(cmd.Context(), metadata.New(make(map[string]string)))
if err := cli.agentSDK.Data(addDatasetMetadata(ctx), dataset, path.Base(datasetPath), privKey); err != nil {
printError(cmd, "Failed to upload dataset due to error: %v ❌ ", err)
return
}
cmd.Println(color.New(color.FgGreen).Sprint("Successfully uploaded dataset! ✔ "))
},
}
cmd.Flags().BoolVarP(&decompressDataset, "decompress", "d", false, "Decompress the dataset on agent")
return cmd
}
func decodeKey(b *pem.Block) (interface{}, error) {
if b == nil {
return nil, errors.New("error decoding key")
}
switch b.Type {
case rsaKeyType:
privKey, err := x509.ParsePKCS8PrivateKey(b.Bytes)
if err != nil {
privKey, err = x509.ParsePKCS1PrivateKey(b.Bytes)
if err != nil {
return nil, err
}
}
return privKey, nil
case ecdsaKeyType:
privKey, err := x509.ParseECPrivateKey(b.Bytes)
if err != nil {
return nil, err
}
return privKey, nil
default:
return nil, errors.New("error decoding key")
}
}
func addDatasetMetadata(ctx context.Context) context.Context {
return agent.DecompressToContext(ctx, decompressDataset)
}
+158
View File
@@ -0,0 +1,158 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"bytes"
"errors"
"os"
"testing"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/ultravioletrs/cocos/pkg/sdk/mocks"
)
func createTempDatasetFile(content string) (string, error) {
tmpFile, err := os.CreateTemp("", "dataset-*.txt")
if err != nil {
return "", err
}
defer tmpFile.Close()
_, err = tmpFile.WriteString(content)
if err != nil {
return "", err
}
return tmpFile.Name(), nil
}
func TestDatasetsCmd(t *testing.T) {
tests := []struct {
name string
setupMock func(*mocks.SDK)
setupFiles func() (string, error)
connectErr error
expectedOutput string
cleanup func(string, string)
}{
{
name: "successful upload",
setupMock: func(m *mocks.SDK) {
m.On("Data", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
},
setupFiles: func() (string, error) {
datasetFile, err := createTempDatasetFile("test dataset content")
if err != nil {
return "", err
}
err = generateRSAPrivateKeyFile(privateKeyFile)
return datasetFile, err
},
expectedOutput: "Successfully uploaded dataset",
cleanup: func(datasetFile, privateKeyFile string) {
os.Remove(datasetFile)
os.Remove(privateKeyFile)
},
},
{
name: "missing dataset file",
setupMock: func(m *mocks.SDK) {
m.On("Data", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
},
setupFiles: func() (string, error) {
return "", nil
},
expectedOutput: "Error reading dataset file",
},
{
name: "missing private key file",
setupMock: func(m *mocks.SDK) {
m.On("Data", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
},
setupFiles: func() (string, error) {
return createTempDatasetFile("test dataset content")
},
expectedOutput: "Error reading private key file",
cleanup: func(datasetFile, _ string) {
os.Remove(datasetFile)
},
},
{
name: "upload failure",
setupMock: func(m *mocks.SDK) {
m.On("Data", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(errors.New("failed to upload algorithm due to error"))
},
setupFiles: func() (string, error) {
datasetFile, err := createTempDatasetFile("test dataset content")
if err != nil {
return "", err
}
err = generateRSAPrivateKeyFile(privateKeyFile)
return datasetFile, err
},
expectedOutput: "Failed to upload dataset due to error",
cleanup: func(datasetFile, privateKeyFile string) {
os.Remove(datasetFile)
os.Remove(privateKeyFile)
},
},
{
name: "invalid private key",
setupMock: func(m *mocks.SDK) {
m.On("Data", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(nil)
},
setupFiles: func() (string, error) {
datasetFile, err := createTempDatasetFile("test dataset content")
if err != nil {
return "", err
}
err = os.WriteFile(privateKeyFile, []byte("invalid private key"), 0o644)
return datasetFile, err
},
expectedOutput: "Error decoding private key",
cleanup: func(datasetFile, privateKeyFile string) {
os.Remove(datasetFile)
os.Remove(privateKeyFile)
},
},
{
name: "connection error",
setupMock: func(m *mocks.SDK) {
},
setupFiles: func() (string, error) { return "", nil },
connectErr: errors.New("failed to connect to agent"),
expectedOutput: "Failed to connect to agent",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockSDK := new(mocks.SDK)
if tt.setupMock != nil {
tt.setupMock(mockSDK)
}
testCLI := CLI{
agentSDK: mockSDK,
connectErr: tt.connectErr,
}
datasetFile, err := tt.setupFiles()
require.NoError(t, err)
cmd := testCLI.NewDatasetsCmd()
buf := new(bytes.Buffer)
cmd.SetOut(buf)
cmd.SetArgs([]string{datasetFile, privateKeyFile})
err = cmd.Execute()
require.NoError(t, err)
require.Contains(t, buf.String(), tt.expectedOutput)
if tt.cleanup != nil {
tt.cleanup(datasetFile, privateKeyFile)
}
})
}
}
+49
View File
@@ -0,0 +1,49 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"github.com/absmach/magistrala/pkg/errors"
"github.com/fatih/color"
"github.com/spf13/cobra"
"github.com/ultravioletrs/cocos/agent/auth"
"github.com/ultravioletrs/cocos/pkg/clients/grpc/agent"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
var (
errAgentUnavailable = errors.New("agent is unavailable on the current address")
errDigitalSignatureVerificationFailed = errors.New("digital signature verification failed, check the provided public key")
)
func decodeErros(err error) error {
statusErr, ok := status.FromError(err)
if ok {
switch statusErr.Code() {
case codes.PermissionDenied:
return errDigitalSignatureVerificationFailed
case codes.Unavailable:
return errAgentUnavailable
case codes.Unknown:
return err
}
}
switch {
case errors.Contains(err, auth.ErrSignatureVerificationFailed):
return auth.ErrSignatureVerificationFailed
case errors.Contains(err, agent.ErrAgentServiceUnavailable):
return agent.ErrAgentServiceUnavailable
default:
return err
}
}
func printError(cmd *cobra.Command, message string, err error) {
if !Verbose {
err = decodeErros(err)
}
msg := color.New(color.FgRed).Sprintf(message, err)
cmd.Println(msg)
}
+110
View File
@@ -0,0 +1,110 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"bytes"
"errors"
"testing"
mgerrors "github.com/absmach/magistrala/pkg/errors"
"github.com/fatih/color"
"github.com/spf13/cobra"
"github.com/ultravioletrs/cocos/agent/auth"
"github.com/ultravioletrs/cocos/pkg/clients/grpc/agent"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
func TestDecodeErros(t *testing.T) {
tests := []struct {
name string
input error
expected error
}{
{
name: "Permission Denied",
input: status.Error(codes.PermissionDenied, "permission denied"),
expected: errDigitalSignatureVerificationFailed,
},
{
name: "Unavailable",
input: status.Error(codes.Unavailable, "service unavailable"),
expected: errAgentUnavailable,
},
{
name: "Unknown",
input: status.Error(codes.Unknown, "unknown error"),
expected: status.Error(codes.Unknown, "unknown error"),
},
{
name: "Signature Verification Failed",
input: mgerrors.Wrap(auth.ErrSignatureVerificationFailed, errors.New("wrapped error")),
expected: auth.ErrSignatureVerificationFailed,
},
{
name: "Other Error",
input: errors.New("other error"),
expected: errors.New("other error"),
},
{
name: "Agent Service Unavailable",
input: mgerrors.Wrap(agent.ErrAgentServiceUnavailable, errors.New("wrapped error")),
expected: agent.ErrAgentServiceUnavailable,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := decodeErros(tt.input)
if result.Error() != tt.expected.Error() {
t.Errorf("decodeErros(%v) = %v, want %v", tt.input, result, tt.expected)
}
})
}
}
func TestPrintError(t *testing.T) {
// Save the original color.NoColor value and restore it after the test
origNoColor := color.NoColor
color.NoColor = true
defer func() { color.NoColor = origNoColor }()
tests := []struct {
name string
message string
err error
verbose bool
expected string
}{
{
name: "Non-verbose mode",
message: "Error: %s",
err: status.Error(codes.PermissionDenied, "permission denied"),
verbose: false,
expected: "Error: digital signature verification failed, check the provided public key\n",
},
{
name: "Verbose mode",
message: "Error: %s",
err: status.Error(codes.PermissionDenied, "permission denied"),
verbose: true,
expected: "Error: rpc error: code = PermissionDenied desc = permission denied\n",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
Verbose = tt.verbose
cmd := &cobra.Command{}
buf := new(bytes.Buffer)
cmd.SetOut(buf)
printError(cmd, tt.message, tt.err)
if got := buf.String(); got != tt.expected {
t.Errorf("printError() output = %q, want %q", got, tt.expected)
}
})
}
}
+140
View File
@@ -0,0 +1,140 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"crypto/ecdsa"
"crypto/ed25519"
"crypto/elliptic"
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"os"
"github.com/spf13/cobra"
)
const (
keyBitSize = 4096
rsaKeyType = "PRIVATE KEY"
ecdsaKeyType = "EC PRIVATE KEY"
ed25519KeyType = "PRIVATE KEY"
publicKeyType = "PUBLIC KEY"
publicKeyFile = "public.pem"
privateKeyFile = "private.pem"
ECDSA = "ecdsa"
ED25519 = "ed25519"
)
var KeyType string
func (cli *CLI) NewKeysCmd() *cobra.Command {
return &cobra.Command{
Use: "keys",
Short: "Generate a new public/private key pair",
Long: "Generates a new public/private key pair using an algorithm of the users choice.\n" +
"Supported algorithms are RSA, ecdsa, and ed25519.",
Example: "./build/cocos-cli keys -k rsa",
Args: cobra.ExactArgs(0),
Run: func(cmd *cobra.Command, args []string) {
switch KeyType {
case ECDSA:
privEcdsaKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
printError(cmd, "Error generating keys: %v ❌ ", err)
return
}
pubKeyBytes, err := x509.MarshalPKIXPublicKey(&privEcdsaKey.PublicKey)
if err != nil {
printError(cmd, "Error marshalling public key: %v ❌ ", err)
return
}
if err := generateAndWriteKeys(privEcdsaKey, pubKeyBytes, ecdsaKeyType); err != nil {
printError(cmd, "Error generating and writing keys: %v ❌ ", err)
return
}
case ED25519:
pubEd25519Key, privEd25519Key, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
printError(cmd, "Error generating keys: %v ❌ ", err)
return
}
pubKey, err := x509.MarshalPKIXPublicKey(pubEd25519Key)
if err != nil {
printError(cmd, "Error marshalling public key: %v ❌ ", err)
return
}
if err := generateAndWriteKeys(privEd25519Key, pubKey, ed25519KeyType); err != nil {
printError(cmd, "Error generating and writing keys: %v ❌ ", err)
return
}
default:
privKey, err := rsa.GenerateKey(rand.Reader, keyBitSize)
if err != nil {
printError(cmd, "Error generating keys: %v ❌ ", err)
return
}
pubKeyBytes, err := x509.MarshalPKIXPublicKey(&privKey.PublicKey)
if err != nil {
printError(cmd, "Error marshalling public key: %v ❌ ", err)
return
}
if err := generateAndWriteKeys(privKey, pubKeyBytes, rsaKeyType); err != nil {
printError(cmd, "Error generating and writing keys: %v ❌ ", err)
return
}
}
cmd.Printf("Successfully generated public/private key pair of type: %s", KeyType)
},
}
}
func generateAndWriteKeys(privKey interface{}, pubKeyBytes []byte, keyType string) error {
privFile, err := os.Create(privateKeyFile)
if err != nil {
return err
}
defer privFile.Close()
var b []byte
switch privKey := privKey.(type) {
case *rsa.PrivateKey:
b = x509.MarshalPKCS1PrivateKey(privKey)
case *ecdsa.PrivateKey:
b, err = x509.MarshalECPrivateKey(privKey)
case ed25519.PrivateKey:
b, err = x509.MarshalPKCS8PrivateKey(privKey)
}
if err != nil {
return err
}
if err := pem.Encode(privFile, &pem.Block{
Type: keyType,
Bytes: b,
}); err != nil {
return err
}
pubFile, err := os.Create(publicKeyFile)
if err != nil {
return err
}
defer pubFile.Close()
if err := pem.Encode(pubFile, &pem.Block{
Type: publicKeyType,
Bytes: pubKeyBytes,
}); err != nil {
return err
}
return nil
}
+92
View File
@@ -0,0 +1,92 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"crypto/ecdsa"
"crypto/ed25519"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"os"
"testing"
)
func TestNewKeysCmd(t *testing.T) {
cli := &CLI{}
cmd := cli.NewKeysCmd()
if cmd.Use != "keys" {
t.Errorf("Expected Use to be 'keys', got %s", cmd.Use)
}
if cmd.Short != "Generate a new public/private key pair" {
t.Errorf("Unexpected Short description: %s", cmd.Short)
}
}
func TestGenerateAndWriteKeys(t *testing.T) {
tests := []struct {
name string
keyType string
}{
{"RSA", "rsa"},
{"ECDSA", "ecdsa"},
{"ED25519", "ed25519"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
KeyType = tt.keyType
cmd := (&CLI{}).NewKeysCmd()
cmd.Run(cmd, []string{})
if _, err := os.Stat(privateKeyFile); os.IsNotExist(err) {
t.Errorf("Private key file was not created")
}
if _, err := os.Stat(publicKeyFile); os.IsNotExist(err) {
t.Errorf("Public key file was not created")
}
privKeyData, err := os.ReadFile(privateKeyFile)
if err != nil {
t.Fatalf("Failed to read private key file: %v", err)
}
privPem, _ := pem.Decode(privKeyData)
if privPem == nil {
t.Fatalf("Failed to decode private key PEM")
}
var privKey interface{}
switch tt.keyType {
case "rsa":
privKey, err = x509.ParsePKCS1PrivateKey(privPem.Bytes)
case "ecdsa":
privKey, err = x509.ParseECPrivateKey(privPem.Bytes)
case "ed25519":
privKey, err = x509.ParsePKCS8PrivateKey(privPem.Bytes)
}
if err != nil {
t.Fatalf("Failed to parse private key: %v", err)
}
switch tt.keyType {
case "rsa":
if _, ok := privKey.(*rsa.PrivateKey); !ok {
t.Errorf("Expected RSA private key, got %T", privKey)
}
case "ecdsa":
if _, ok := privKey.(*ecdsa.PrivateKey); !ok {
t.Errorf("Expected ECDSA private key, got %T", privKey)
}
case "ed25519":
if _, ok := privKey.(ed25519.PrivateKey); !ok {
t.Errorf("Expected ED25519 private key, got %T", privKey)
}
}
os.Remove(privateKeyFile)
os.Remove(publicKeyFile)
})
}
}
+67
View File
@@ -0,0 +1,67 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"encoding/pem"
"os"
"github.com/fatih/color"
"github.com/spf13/cobra"
)
const (
resultFilePrefix = "results"
resultFileExt = ".zip"
resultfilename = "results.zip"
)
func (cli *CLI) NewResultsCmd() *cobra.Command {
return &cobra.Command{
Use: "result",
Short: "Retrieve computation result file",
Example: "result <private_key_file_path> <optional_file_name.zip>",
Args: cobra.MinimumNArgs(1),
Run: func(cmd *cobra.Command, args []string) {
if cli.connectErr != nil {
printError(cmd, "Failed to connect to agent: %v ❌ ", cli.connectErr)
return
}
cmd.Println("⏳ Retrieving computation result file")
privKeyFile, err := os.ReadFile(args[0])
if err != nil {
printError(cmd, "Error reading private key file: %v ❌ ", err)
return
}
filename := resultfilename
if len(args) > 1 {
filename = args[1]
}
pemBlock, _ := pem.Decode(privKeyFile)
privKey, err := decodeKey(pemBlock)
if err != nil {
printError(cmd, "Error decoding private key: %v ❌ ", err)
return
}
resultFile, err := os.Create(filename)
if err != nil {
printError(cmd, "Error creating result file: %v ❌ ", err)
return
}
defer resultFile.Close()
if err = cli.agentSDK.Result(cmd.Context(), privKey, resultFile); err != nil {
printError(cmd, "Error retrieving computation result: %v ❌ ", err)
return
}
cmd.Println(color.New(color.FgGreen).Sprintf("Computation result retrieved and saved successfully as %s! ✔ ", filename))
},
}
}
+177
View File
@@ -0,0 +1,177 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"bytes"
"errors"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/ultravioletrs/cocos/pkg/sdk/mocks"
)
const compResult = "Test computation result"
func TestResultsCmd_MultipleExecutions(t *testing.T) {
mockSDK := new(mocks.SDK)
mockSDK.On("Result", mock.Anything, mock.Anything, mock.Anything).Return(nil).Run(func(args mock.Arguments) {
_, err := args.Get(2).(*os.File).WriteString(compResult)
require.NoError(t, err)
})
testCLI := CLI{agentSDK: mockSDK}
err := generateRSAPrivateKeyFile(privateKeyFile)
require.NoError(t, err)
cmd := testCLI.NewResultsCmd()
buf := new(bytes.Buffer)
cmd.SetOut(buf)
cmd.SetArgs([]string{privateKeyFile})
for i := 0; i < 3; i++ {
err = cmd.Execute()
require.NoError(t, err)
require.Contains(t, buf.String(), "Computation result retrieved and saved successfully")
buf.Reset()
}
files, err := filepath.Glob("results*.zip")
require.NoError(t, err)
t.Cleanup(func() {
for _, file := range files {
os.Remove(file)
}
os.Remove(privateKeyFile)
})
}
func TestResultsCmd_InvalidPrivateKey(t *testing.T) {
mockSDK := new(mocks.SDK)
mockSDK.On("Result", mock.Anything, mock.Anything, mock.Anything).Return(nil).Run(func(args mock.Arguments) {
_, err := args.Get(2).(*os.File).WriteString(compResult)
require.NoError(t, err)
})
testCLI := CLI{agentSDK: mockSDK}
invalidPrivateKey, err := os.CreateTemp("", "invalid_private_key.pem")
require.NoError(t, err)
err = invalidPrivateKey.Close()
require.NoError(t, err)
t.Cleanup(func() {
err := os.Remove(invalidPrivateKey.Name())
require.NoError(t, err)
})
cmd := testCLI.NewResultsCmd()
buf := new(bytes.Buffer)
cmd.SetOut(buf)
cmd.SetArgs([]string{invalidPrivateKey.Name()})
err = cmd.Execute()
require.NoError(t, err)
require.Contains(t, buf.String(), "Error decoding private key")
mockSDK.AssertNotCalled(t, "Result", mock.Anything, mock.Anything, mock.Anything)
}
func TestResultsCmd(t *testing.T) {
tests := []struct {
name string
setupMock func(*mocks.SDK)
setupFiles func() (string, error)
connectErr error
expectedOutput string
cleanup func()
}{
{
name: "successful result retrieval",
setupMock: func(m *mocks.SDK) {
m.On("Result", mock.Anything, mock.Anything, mock.Anything).Return(nil).Run(func(args mock.Arguments) {
_, err := args.Get(2).(*os.File).WriteString(compResult)
require.NoError(t, err)
})
},
setupFiles: func() (string, error) {
return privateKeyFile, generateRSAPrivateKeyFile(privateKeyFile)
},
expectedOutput: "Computation result retrieved and saved successfully",
cleanup: func() {
files, _ := filepath.Glob("results*.zip")
for _, file := range files {
os.Remove(file)
}
os.Remove(privateKeyFile)
},
},
{
name: "missing private key file",
setupMock: func(m *mocks.SDK) {
m.On("Result", mock.Anything, mock.Anything, mock.Anything).Return(nil).Run(func(args mock.Arguments) {
_, err := args.Get(2).(*os.File).WriteString(compResult)
require.NoError(t, err)
})
},
setupFiles: func() (string, error) {
return "non_existent_private_key.pem", nil
},
expectedOutput: "Error reading private key file",
},
{
name: "result retrieval failure",
setupMock: func(m *mocks.SDK) {
m.On("Result", mock.Anything, mock.Anything, mock.Anything).Return(errors.New("error retrieving computation result"))
},
setupFiles: func() (string, error) {
return privateKeyFile, generateRSAPrivateKeyFile(privateKeyFile)
},
expectedOutput: "error retrieving computation result",
cleanup: func() {
os.Remove(privateKeyFile)
},
},
{
name: "connection error",
setupMock: func(m *mocks.SDK) {
},
setupFiles: func() (string, error) { return "", nil },
connectErr: errors.New("failed to connect to agent"),
expectedOutput: "Failed to connect to agent",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
mockSDK := new(mocks.SDK)
if tt.setupMock != nil {
tt.setupMock(mockSDK)
}
testCLI := CLI{
agentSDK: mockSDK,
connectErr: tt.connectErr,
}
file, err := tt.setupFiles()
require.NoError(t, err)
cmd := testCLI.NewResultsCmd()
buf := new(bytes.Buffer)
cmd.SetOut(buf)
cmd.SetArgs([]string{file})
err = cmd.Execute()
require.NoError(t, err)
require.Contains(t, buf.String(), tt.expectedOutput)
if tt.cleanup != nil {
tt.cleanup()
}
})
}
}
+44
View File
@@ -0,0 +1,44 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"context"
"github.com/spf13/cobra"
"github.com/ultravioletrs/cocos/pkg/clients/grpc"
"github.com/ultravioletrs/cocos/pkg/clients/grpc/agent"
"github.com/ultravioletrs/cocos/pkg/sdk"
)
var Verbose bool
type CLI struct {
agentSDK sdk.SDK
config grpc.AgentClientConfig
client grpc.Client
connectErr error
}
func New(config grpc.AgentClientConfig) *CLI {
return &CLI{
config: config,
}
}
func (c *CLI) InitializeSDK(cmd *cobra.Command) error {
agentGRPCClient, agentClient, err := agent.NewAgentClient(context.Background(), c.config)
if err != nil {
c.connectErr = err
return err
}
cmd.Println("🔗 Connected to agent using ", agentGRPCClient.Secure())
c.client = agentGRPCClient
c.agentSDK = sdk.NewAgentSDK(agentClient)
return nil
}
func (c *CLI) Close() {
c.client.Close()
}
+264
View File
@@ -0,0 +1,264 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"log"
"log/slog"
"os"
"time"
mglog "github.com/absmach/magistrala/logger"
"github.com/absmach/magistrala/pkg/prometheus"
"github.com/cenkalti/backoff/v4"
"github.com/google/go-sev-guest/abi"
"github.com/google/go-sev-guest/client"
"github.com/mdlayher/vsock"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/api"
agentgrpc "github.com/ultravioletrs/cocos/agent/api/grpc"
"github.com/ultravioletrs/cocos/agent/auth"
"github.com/ultravioletrs/cocos/agent/events"
agentlogger "github.com/ultravioletrs/cocos/internal/logger"
"github.com/ultravioletrs/cocos/internal/server"
grpcserver "github.com/ultravioletrs/cocos/internal/server/grpc"
ackvsock "github.com/ultravioletrs/cocos/internal/vsock"
managerevents "github.com/ultravioletrs/cocos/manager/events"
"github.com/ultravioletrs/cocos/manager/qemu"
"github.com/ultravioletrs/cocos/pkg/attestation/quoteprovider"
"golang.org/x/crypto/sha3"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
)
const (
svcName = "agent"
defSvcGRPCPort = "7002"
retryInterval = 5 * time.Second
)
func main() {
ctx, cancel := context.WithCancel(context.Background())
g, ctx := errgroup.WithContext(ctx)
cfg, err := readConfig()
if err != nil {
log.Fatalf("failed to read agent configuration from vsock %s", err.Error())
}
conn, err := dialVsock()
if err != nil {
log.Fatal(err)
}
defer conn.Close()
ackConn := ackvsock.NewAckWriter(conn)
var exitCode int
defer mglog.ExitWithError(&exitCode)
var level slog.Level
if err := level.UnmarshalText([]byte(cfg.AgentConfig.LogLevel)); err != nil {
log.Println(err)
exitCode = 1
return
}
handler := agentlogger.NewProtoHandler(ackConn, &slog.HandlerOptions{Level: level}, cfg.ID)
logger := slog.New(handler)
eventSvc, err := events.New(svcName, cfg.ID, ackConn)
if err != nil {
logger.Error(fmt.Sprintf("failed to create events service %s", err.Error()))
exitCode = 1
return
}
qp, err := quoteprovider.GetQuoteProvider()
if err != nil {
logger.Error(fmt.Sprintf("failed to create quote provider %s", err.Error()))
exitCode = 1
return
}
if err := verifyManifest(cfg, qp); err != nil {
logger.Error(err.Error())
exitCode = 1
return
}
setDefaultValues(&cfg)
svc := newService(ctx, logger, eventSvc, cfg, qp)
agentGrpcServerConfig := server.AgentConfig{
ServerConfig: server.ServerConfig{
BaseConfig: server.BaseConfig{
Host: cfg.AgentConfig.Host,
Port: cfg.AgentConfig.Port,
CertFile: cfg.AgentConfig.CertFile,
KeyFile: cfg.AgentConfig.KeyFile,
ServerCAFile: cfg.AgentConfig.ServerCAFile,
ClientCAFile: cfg.AgentConfig.ClientCAFile,
},
},
AttestedTLS: cfg.AgentConfig.AttestedTls,
}
registerAgentServiceServer := func(srv *grpc.Server) {
reflection.Register(srv)
agent.RegisterAgentServiceServer(srv, agentgrpc.NewServer(svc))
}
authSvc, err := auth.New(cfg)
if err != nil {
logger.Error(fmt.Sprintf("failed to create auth service %s", err.Error()))
exitCode = 1
return
}
gs := grpcserver.New(ctx, cancel, svcName, agentGrpcServerConfig, registerAgentServiceServer, logger, qp, authSvc)
g.Go(func() error {
for {
if _, err := io.Copy(io.Discard, conn); err != nil {
log.Printf("vsock connection lost: %v, reconnecting...", err)
conn.Close()
conn, err = dialVsock()
if err != nil {
log.Fatal("failed to reconnect: ", err)
}
}
time.Sleep(retryInterval)
}
})
g.Go(func() error {
return gs.Start()
})
g.Go(func() error {
return server.StopHandler(ctx, cancel, logger, svcName, gs)
})
if err := g.Wait(); err != nil {
logger.Error(fmt.Sprintf("%s service terminated: %s", svcName, err))
}
}
func newService(ctx context.Context, logger *slog.Logger, eventSvc events.Service, cmp agent.Computation, qp client.QuoteProvider) agent.Service {
svc := agent.New(ctx, logger, eventSvc, cmp, qp)
svc = api.LoggingMiddleware(svc, logger)
counter, latency := prometheus.MakeMetrics(svcName, "api")
svc = api.MetricsMiddleware(svc, counter, latency)
return svc
}
func readConfig() (agent.Computation, error) {
l, err := vsock.Listen(qemu.VsockConfigPort, nil)
if err != nil {
return agent.Computation{}, err
}
defer l.Close()
conn, err := l.Accept()
if err != nil {
return agent.Computation{}, err
}
defer conn.Close()
var buffer []byte
for {
chunk := make([]byte, 1024)
n, err := conn.Read(chunk)
if err != nil {
if err == io.EOF {
break
}
return agent.Computation{}, err
}
buffer = append(buffer, chunk[:n]...)
}
ac := agent.Computation{
AgentConfig: agent.AgentConfig{},
}
if err := json.Unmarshal(buffer, &ac); err != nil {
return agent.Computation{}, err
}
return ac, nil
}
func setDefaultValues(cfg *agent.Computation) {
if cfg.AgentConfig.LogLevel == "" {
cfg.AgentConfig.LogLevel = "info"
}
if cfg.AgentConfig.Port == "" {
cfg.AgentConfig.Port = defSvcGRPCPort
}
}
func isTEE() bool {
_, err := os.Stat("/dev/sev-guest")
return !os.IsNotExist(err)
}
func dialVsock() (*vsock.Conn, error) {
var conn *vsock.Conn
var err error
err = backoff.Retry(func() error {
conn, err = vsock.Dial(vsock.Host, managerevents.ManagerVsockPort, nil)
if err == nil {
log.Println("vsock connection established")
return nil
}
log.Printf("vsock connection failed, retrying in %s... Error: %v", retryInterval, err)
return err
}, backoff.NewExponentialBackOff())
if err != nil {
return nil, err
}
return conn, nil
}
func verifyManifest(cfg agent.Computation, qp client.QuoteProvider) error {
if !isTEE() {
return nil
}
ar, err := qp.GetRawQuote(sha3.Sum512([]byte(cfg.ID)))
if err != nil {
return err
}
arProto, err := abi.ReportCertsToProto(ar[:abi.ReportSize])
if err != nil {
return err
}
cfgBytes, err := json.Marshal(cfg)
if err != nil {
return err
}
mcHash := sha3.Sum256(cfgBytes)
if arProto.Report.HostData == nil {
return fmt.Errorf("manifest verification failed: HostData is nil")
}
if !bytes.Equal(arProto.Report.HostData, mcHash[:]) {
return fmt.Errorf("manifest verification failed")
}
return nil
}
+94
View File
@@ -0,0 +1,94 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package main
import (
"context"
"log/slog"
"os"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/events/mocks"
qpmocks "github.com/ultravioletrs/cocos/pkg/attestation/quoteprovider/mocks"
)
func TestSetDefaultValues(t *testing.T) {
tests := []struct {
name string
input agent.Computation
expected agent.Computation
}{
{
name: "Empty config",
input: agent.Computation{
AgentConfig: agent.AgentConfig{},
},
expected: agent.Computation{
AgentConfig: agent.AgentConfig{
LogLevel: "info",
Port: "7002",
},
},
},
{
name: "Partial config",
input: agent.Computation{
AgentConfig: agent.AgentConfig{
LogLevel: "debug",
},
},
expected: agent.Computation{
AgentConfig: agent.AgentConfig{
LogLevel: "debug",
Port: "7002",
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
setDefaultValues(&tt.input)
assert.Equal(t, tt.expected, tt.input)
})
}
}
func TestNewService(t *testing.T) {
ctx := context.Background()
logger := slog.New(slog.NewTextHandler(os.Stdout, nil))
eventSvc := new(mocks.Service)
eventSvc.On("SendEvent", mock.Anything, mock.Anything, mock.Anything).Return(nil)
cmp := agent.Computation{
ID: "test-computation",
AgentConfig: agent.AgentConfig{
LogLevel: "info",
Port: "7002",
},
}
qp := new(qpmocks.QuoteProvider)
svc := newService(ctx, logger, eventSvc, cmp, qp)
assert.NotNil(t, svc)
}
func TestVerifyManifest(t *testing.T) {
cfg := agent.Computation{
ID: "test-computation",
AgentConfig: agent.AgentConfig{
LogLevel: "info",
Port: "7002",
},
}
mockQP := new(qpmocks.QuoteProvider)
mockQP.On("GetRawQuote", mock.Anything).Return([]byte{}, nil)
err := verifyManifest(cfg, mockQP)
assert.NoError(t, err)
}
+154
View File
@@ -0,0 +1,154 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package main
import (
"fmt"
"os"
"os/signal"
"path"
"syscall"
"github.com/caarlos0/env/v11"
"github.com/fatih/color"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
"github.com/ultravioletrs/cocos/cli"
"github.com/ultravioletrs/cocos/pkg/clients/grpc"
cmd "github.com/virtee/sev-snp-measure-go/sevsnpmeasure/cmd"
)
const (
svcName = "cli"
envPrefixAgentGRPC = "AGENT_GRPC_"
completion = "completion"
filePermision = 0o755
cocosDirectory = ".cocos"
)
type config struct {
LogLevel string `env:"AGENT_LOG_LEVEL" envDefault:"info"`
}
func main() {
rootCmd := &cobra.Command{
Use: "cocos-cli [command]",
Short: "CLI application for CoCos Service API",
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("CLI application for CoCos Service API\n\n")
fmt.Printf("Usage:\n %s [command]\n\n", cmd.CommandPath())
fmt.Printf("Available Commands:\n")
// Filter out "completion" command
availableCommands := make([]*cobra.Command, 0)
for _, subCmd := range cmd.Commands() {
if subCmd.Name() != completion {
availableCommands = append(availableCommands, subCmd)
}
}
for _, subCmd := range availableCommands {
fmt.Printf(" %-15s%s\n", subCmd.Name(), subCmd.Short)
}
fmt.Printf("\nFlags:\n")
cmd.Flags().VisitAll(func(flag *pflag.Flag) {
fmt.Printf(" -%s, --%s %s\n", flag.Shorthand, flag.Name, flag.Usage)
})
fmt.Printf("\nUse \"%s [command] --help\" for more information about a command.\n", cmd.CommandPath())
},
}
signalChan := make(chan os.Signal, 1)
signal.Notify(signalChan, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-signalChan
fmt.Println()
rootCmd.Println(color.New(color.FgRed).Sprint("Operation aborted by user!"))
os.Exit(2)
}()
var cfg config
if err := env.Parse(&cfg); err != nil {
message := color.New(color.FgRed).Sprintf("failed to load %s configuration : %s", svcName, err)
rootCmd.Println(message)
return
}
homePath, err := os.UserHomeDir()
if err != nil {
message := color.New(color.FgRed).Sprintf("failed to fetch user home directory: %s", err)
rootCmd.Println(message)
return
}
directoryCachePath := path.Join(homePath, cocosDirectory)
if err := os.MkdirAll(directoryCachePath, filePermision); err != nil {
message := color.New(color.FgRed).Sprintf("failed to create directory %s : %s", directoryCachePath, err)
rootCmd.Println(message)
return
}
agentGRPCConfig := grpc.AgentClientConfig{}
if err := env.ParseWithOptions(&agentGRPCConfig, env.Options{Prefix: envPrefixAgentGRPC}); err != nil {
message := color.New(color.FgRed).Sprintf("failed to load %s gRPC client configuration : %s", svcName, err)
rootCmd.Println(message)
return
}
cliSVC := cli.New(agentGRPCConfig)
if err := cliSVC.InitializeSDK(rootCmd); err == nil {
defer cliSVC.Close()
}
rootCmd.PersistentFlags().BoolVarP(&cli.Verbose, "verbose", "v", false, "Enable verbose output")
keysCmd := cliSVC.NewKeysCmd()
attestationCmd := cliSVC.NewAttestationCmd()
attestationPolicyCmd := cliSVC.NewAttestationPolicyCmd()
// Agent Commands
rootCmd.AddCommand(cliSVC.NewAlgorithmCmd())
rootCmd.AddCommand(cliSVC.NewDatasetsCmd())
rootCmd.AddCommand(cliSVC.NewResultsCmd())
rootCmd.AddCommand(attestationCmd)
rootCmd.AddCommand(cliSVC.NewFileHashCmd())
rootCmd.AddCommand(attestationPolicyCmd)
rootCmd.AddCommand(keysCmd)
rootCmd.AddCommand(cliSVC.NewCABundleCmd(directoryCachePath))
// Attestation commands
attestationCmd.AddCommand(cliSVC.NewGetAttestationCmd())
attestationCmd.AddCommand(cliSVC.NewValidateAttestationValidationCmd())
// measure.
rootCmd.AddCommand(cmd.NewRootCmd())
// Flags
keysCmd.PersistentFlags().StringVarP(
&cli.KeyType,
"key-type",
"k",
"rsa",
"User Key type",
)
// Attestation Policy commands
attestationPolicyCmd.AddCommand(cliSVC.NewAddMeasurementCmd())
attestationPolicyCmd.AddCommand(cliSVC.NewAddHostDataCmd())
if err := rootCmd.Execute(); err != nil {
logErrorCmd(*rootCmd, err)
return
}
}
func logErrorCmd(cmd cobra.Command, err error) {
boldRed := color.New(color.FgRed, color.Bold)
boldRed.Fprintf(cmd.ErrOrStderr(), "\nerror: ")
fmt.Fprintf(cmd.ErrOrStderr(), "%s\n\n", color.RedString(err.Error()))
}
-208
View File
@@ -1,208 +0,0 @@
package main
import (
"context"
"fmt"
"io"
"io/ioutil"
"log"
"os"
"strconv"
"time"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/logger"
"github.com/mainflux/mainflux/pkg/uuid"
"github.com/opentracing/opentracing-go"
jconfig "github.com/uber/jaeger-client-go/config"
"github.com/ultravioletrs/clients"
authapi "github.com/ultravioletrs/clients/api/grpc"
"github.com/ultravioletrs/cocos/computations"
"github.com/ultravioletrs/cocos/computations/api/api"
"github.com/ultravioletrs/cocos/computations/postgres"
"github.com/ultravioletrs/cocos/internal/db"
"github.com/ultravioletrs/cocos/internal/env"
"github.com/ultravioletrs/cocos/internal/errors"
"github.com/ultravioletrs/cocos/internal/http"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
)
const (
defLogLevel = "debug"
defDBHost = "localhost"
defDBPort = "5432"
defDBUser = "cocos"
defDBPass = "cocos"
defDB = "computations"
defDBSSLMode = "disable"
defDBSSLCert = ""
defDBSSLKey = ""
defDBSSLRootCert = ""
defHTTPPort = "9000"
defServerCert = ""
defServerKey = ""
defJaegerURL = ""
defAuthTLS = "false"
defAuthCACerts = ""
defAuthURL = "localhost:8181"
defAuthTimeout = "1s"
envLogLevel = "COCOS_COMPUTATIONS_LOG_LEVEL"
envDBHost = "COCOS_COMPUTATIONS_DB_HOST"
envDBPort = "COCOS_COMPUTATIONS_DB_PORT"
envDBUser = "COCOS_COMPUTATIONS_DB_USER"
envDBPass = "COCOS_COMPUTATIONS_DB_PASS"
envDB = "COCOS_COMPUTATIONS_DB"
envDBSSLMode = "COCOS_COMPUTATIONS_DB_SSL_MODE"
envDBSSLCert = "COCOS_COMPUTATIONS_DB_SSL_CERT"
envDBSSLKey = "COCOS_COMPUTATIONS_DB_SSL_KEY"
envDBSSLRootCert = "COCOS_COMPUTATIONS_DB_SSL_ROOT_CERT"
envHTTPPort = "COCOS_COMPUTATIONS_HTTP_PORT"
envServerCert = "COCOS_COMPUTATIONS_SERVER_CERT"
envServerKey = "COCOS_COMPUTATIONS_SERVER_KEY"
envJaegerURL = "MF_JAEGER_URL"
envAuthTLS = "COCOS_COMPUTATIONS_AUTH_CLIENT_TLS"
envAuthCACerts = "COCOS_COMPUTATIONS_AUTH_CA_CERTS"
envAuthURL = "COCOS_COMPUTATIONS_AUTH_GRPC_URL"
envAuthTimeout = "COCOS_COMPUTATIONS_AUTH_GRPC_TIMEOUT"
)
const svcName = "Computations"
type config struct {
logLevel string
dbConfig db.Config
httpPort string
serverCert string
serverKey string
jaegerURL string
authTLS bool
authCACerts string
authURL string
authTimeout time.Duration
}
func loadConfig() config {
authTimeout, err := time.ParseDuration(mainflux.Env(envAuthTimeout, defAuthTimeout))
if err != nil {
log.Fatalf("Invalid %s value: %s", envAuthTimeout, err.Error())
}
tls, err := strconv.ParseBool(mainflux.Env(envAuthTLS, defAuthTLS))
if err != nil {
log.Fatalf("Invalid value passed for %s\n", envAuthTLS)
}
dbConfig := db.Config{
Host: env.Load(envDBHost, defDBHost),
Port: env.Load(envDBPort, defDBPort),
User: env.Load(envDBUser, defDBUser),
Pass: env.Load(envDBPass, defDBPass),
Name: env.Load(envDB, defDB),
SSLMode: env.Load(envDBSSLMode, defDBSSLMode),
SSLCert: env.Load(envDBSSLCert, defDBSSLCert),
SSLKey: env.Load(envDBSSLKey, defDBSSLKey),
SSLRootCert: env.Load(envDBSSLRootCert, defDBSSLRootCert),
Migrations: postgres.Migrations,
}
return config{
dbConfig: dbConfig,
authTLS: tls,
authTimeout: authTimeout,
logLevel: env.Load(envLogLevel, defLogLevel),
httpPort: env.Load(envHTTPPort, defHTTPPort),
serverCert: env.Load(envServerCert, defServerCert),
serverKey: env.Load(envServerKey, defServerKey),
authCACerts: env.Load(envAuthCACerts, defAuthCACerts),
jaegerURL: env.Load(envJaegerURL, defJaegerURL),
authURL: env.Load(envAuthURL, defAuthURL),
}
}
func main() {
cfg := loadConfig()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
g, ctx := errgroup.WithContext(ctx)
l, err := logger.New(os.Stdout, cfg.logLevel)
if err != nil {
log.Fatalf(err.Error())
}
db, err := db.Connect(cfg.dbConfig)
if err != nil {
log.Fatalf(err.Error())
}
authTracer, closer := initJaeger("auth", cfg.jaegerURL, l)
defer closer.Close()
auth, close := connectToAuth(cfg, authTracer, l)
if close != nil {
defer close()
}
repo := postgres.NewRepository(db)
idp := uuid.New()
svc := computations.NewService(repo, idp, auth)
svc = api.LoggingMiddleware(svc, l)
h := api.MakeHandler(svc, nil, l)
g.Go(func() error {
return http.StartServer(ctx, svcName, cfg.httpPort, cfg.serverCert, cfg.serverKey, h, l)
})
g.Go(errors.Handle(svcName, l, ctx, cancel))
if err := g.Wait(); err != nil {
l.Error(fmt.Sprintf("%s service terminated: %s", svcName, err))
}
}
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
if url == "" {
return opentracing.NoopTracer{}, ioutil.NopCloser(nil)
}
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger: %s", err))
os.Exit(1)
}
return tracer, closer
}
func connectToAuth(cfg config, tracer opentracing.Tracer, logger logger.Logger) (clients.AuthServiceClient, func() error) {
var opts []grpc.DialOption
if cfg.authTLS {
if cfg.authCACerts != "" {
tpc, err := credentials.NewClientTLSFromFile(cfg.authCACerts, "")
if err != nil {
logger.Error(fmt.Sprintf("Failed to create tls credentials: %s", err))
os.Exit(1)
}
opts = append(opts, grpc.WithTransportCredentials(tpc))
}
} else {
opts = append(opts, grpc.WithInsecure())
logger.Info("gRPC communication is not encrypted")
}
conn, err := grpc.Dial(cfg.authURL, opts...)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to auth service: %s", err))
os.Exit(1)
}
return authapi.NewClient(tracer, conn, cfg.authTimeout), conn.Close
}
-141
View File
@@ -1,141 +0,0 @@
package main
import (
"context"
"fmt"
"log"
"os"
"strconv"
"time"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/logger"
"github.com/mainflux/mainflux/pkg/uuid"
"github.com/ultravioletrs/cocos/datasets"
"github.com/ultravioletrs/cocos/datasets/api"
"github.com/ultravioletrs/cocos/datasets/postgres"
"github.com/ultravioletrs/cocos/internal/db"
"github.com/ultravioletrs/cocos/internal/env"
"github.com/ultravioletrs/cocos/internal/errors"
"github.com/ultravioletrs/cocos/internal/http"
"golang.org/x/sync/errgroup"
)
const (
defLogLevel = "debug"
defDBHost = "localhost"
defDBPort = "5432"
defDBUser = "cocos"
defDBPass = "cocos"
defDB = "datasets"
defDBSSLMode = "disable"
defDBSSLCert = ""
defDBSSLKey = ""
defDBSSLRootCert = ""
defHTTPPort = "9001"
defServerCert = ""
defServerKey = ""
defAuthTLS = "false"
defAuthCACerts = ""
defAuthURL = "localhost:8181"
defAuthTimeout = "1s"
envLogLevel = "COCOS_DATASETS_LOG_LEVEL"
envDBHost = "COCOS_DATASETS_DB_HOST"
envDBPort = "COCOS_DATASETS_DB_PORT"
envDBUser = "COCOS_DATASETS_DB_USER"
envDBPass = "COCOS_DATASETS_DB_PASS"
envDB = "COCOS_DATASETS_DB"
envDBSSLMode = "COCOS_DATASETS_DB_SSL_MODE"
envDBSSLCert = "COCOS_DATASETS_DB_SSL_CERT"
envDBSSLKey = "COCOS_DATASETS_DB_SSL_KEY"
envDBSSLRootCert = "COCOS_DATASETS_DB_SSL_ROOT_CERT"
envHTTPPort = "COCOS_DATASETS_HTTP_PORT"
envServerCert = "COCOS_DATASETS_SERVER_CERT"
envServerKey = "COCOS_DATASETS_SERVER_KEY"
envAuthTLS = "COCOS_DATASETS_AUTH_CLIENT_TLS"
envAuthCACerts = "COCOS_DATASETS_AUTH_CA_CERTS"
envAuthURL = "COCOS_DATASETS_AUTH_GRPC_URL"
envAuthTimeout = "COCOS_DATASETS_AUTH_GRPC_TIMEOUT"
)
const svcName = "Datasets"
type config struct {
logLevel string
dbConfig db.Config
httpPort string
serverCert string
serverKey string
authTLS bool
authCACerts string
authURL string
authTimeout time.Duration
}
func loadConfig() config {
authTimeout, err := time.ParseDuration(mainflux.Env(envAuthTimeout, defAuthTimeout))
if err != nil {
log.Fatalf("Invalid %s value: %s", envAuthTimeout, err.Error())
}
tls, err := strconv.ParseBool(mainflux.Env(envAuthTLS, defAuthTLS))
if err != nil {
log.Fatalf("Invalid value passed for %s\n", envAuthTLS)
}
dbConfig := db.Config{
Host: env.Load(envDBHost, defDBHost),
Port: env.Load(envDBPort, defDBPort),
User: env.Load(envDBUser, defDBUser),
Pass: env.Load(envDBPass, defDBPass),
Name: env.Load(envDB, defDB),
SSLMode: env.Load(envDBSSLMode, defDBSSLMode),
SSLCert: env.Load(envDBSSLCert, defDBSSLCert),
SSLKey: env.Load(envDBSSLKey, defDBSSLKey),
SSLRootCert: env.Load(envDBSSLRootCert, defDBSSLRootCert),
Migrations: postgres.Migrations,
}
return config{
dbConfig: dbConfig,
authTLS: tls,
authTimeout: authTimeout,
logLevel: env.Load(envLogLevel, defLogLevel),
httpPort: env.Load(envHTTPPort, defHTTPPort),
serverCert: env.Load(envServerCert, defServerCert),
serverKey: env.Load(envServerKey, defServerKey),
authCACerts: env.Load(envAuthCACerts, defAuthCACerts),
authURL: env.Load(envAuthURL, defAuthURL),
}
}
func main() {
cfg := loadConfig()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
g, ctx := errgroup.WithContext(ctx)
l, err := logger.New(os.Stdout, cfg.logLevel)
if err != nil {
log.Fatalf(err.Error())
}
db, err := db.Connect(cfg.dbConfig)
if err != nil {
log.Fatalf(err.Error())
}
repo := postgres.NewRepository(db)
idp := uuid.New()
svc := datasets.NewService(repo, idp)
svc = api.LoggingMiddleware(svc, l)
h := api.MakeHandler(svc, nil, l)
g.Go(func() error {
return http.StartServer(ctx, svcName, cfg.httpPort, cfg.serverCert, cfg.serverKey, h, l)
})
g.Go(errors.Handle(svcName, l, ctx, cancel))
if err := g.Wait(); err != nil {
l.Error(fmt.Sprintf("%s service terminated: %s", svcName, err))
}
}
+171
View File
@@ -0,0 +1,171 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package main
import (
"context"
"fmt"
"log"
"log/slog"
"net/url"
"os"
"os/signal"
"strings"
"syscall"
mglog "github.com/absmach/magistrala/logger"
"github.com/absmach/magistrala/pkg/jaeger"
"github.com/absmach/magistrala/pkg/prometheus"
"github.com/absmach/magistrala/pkg/uuid"
"github.com/caarlos0/env/v11"
"github.com/ultravioletrs/cocos/manager"
"github.com/ultravioletrs/cocos/manager/api"
managerapi "github.com/ultravioletrs/cocos/manager/api/grpc"
"github.com/ultravioletrs/cocos/manager/events"
"github.com/ultravioletrs/cocos/manager/qemu"
"github.com/ultravioletrs/cocos/manager/tracing"
pkggrpc "github.com/ultravioletrs/cocos/pkg/clients/grpc"
managergrpc "github.com/ultravioletrs/cocos/pkg/clients/grpc/manager"
"go.opentelemetry.io/otel/trace"
"golang.org/x/sync/errgroup"
)
const (
svcName = "manager"
envPrefixGRPC = "MANAGER_GRPC_"
envPrefixQemu = "MANAGER_QEMU_"
clientBufferSize = 100
)
type config struct {
LogLevel string `env:"MANAGER_LOG_LEVEL" envDefault:"info"`
JaegerURL url.URL `env:"COCOS_JAEGER_URL" envDefault:"http://localhost:4318"`
TraceRatio float64 `env:"COCOS_JAEGER_TRACE_RATIO" envDefault:"1.0"`
InstanceID string `env:"MANAGER_INSTANCE_ID" envDefault:""`
AttestationPolicyBinary string `env:"MANAGER_ATTESTATION_POLICY_BINARY" envDefault:"../../build"`
EosVersion string `env:"MANAGER_EOS_VERSION" envDefault:""`
}
func main() {
ctx, cancel := context.WithCancel(context.Background())
g, ctx := errgroup.WithContext(ctx)
var cfg config
if err := env.Parse(&cfg); err != nil {
log.Fatalf("failed to load %s configuration : %s", svcName, err)
}
logger, err := mglog.New(os.Stdout, cfg.LogLevel)
if err != nil {
log.Fatal(err.Error())
}
var exitCode int
defer mglog.ExitWithError(&exitCode)
if cfg.InstanceID == "" {
if cfg.InstanceID, err = uuid.New().ID(); err != nil {
logger.Error(fmt.Sprintf("Failed to generate instance ID: %s", err))
exitCode = 1
return
}
}
tp, err := jaeger.NewProvider(ctx, svcName, cfg.JaegerURL, cfg.InstanceID, cfg.TraceRatio)
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger: %s", err))
}
defer func() {
if err := tp.Shutdown(ctx); err != nil {
logger.Error(fmt.Sprintf("Error shutting down tracer provider: %v", err))
}
}()
tracer := tp.Tracer(svcName)
qemuCfg := qemu.Config{}
if err := env.ParseWithOptions(&qemuCfg, env.Options{Prefix: envPrefixQemu}); err != nil {
logger.Error(fmt.Sprintf("failed to load QEMU configuration: %s", err))
exitCode = 1
return
}
args := qemuCfg.ConstructQemuArgs()
logger.Info(strings.Join(args, " "))
managerGRPCConfig := pkggrpc.ManagerClientConfig{}
if err := env.ParseWithOptions(&managerGRPCConfig, env.Options{Prefix: envPrefixGRPC}); err != nil {
logger.Error(fmt.Sprintf("failed to load %s gRPC client configuration : %s", svcName, err))
exitCode = 1
return
}
managerGRPCClient, managerClient, err := managergrpc.NewManagerClient(managerGRPCConfig)
if err != nil {
logger.Error(err.Error())
exitCode = 1
return
}
defer managerGRPCClient.Close()
pc, err := managerClient.Process(ctx)
if err != nil {
logger.Error(err.Error())
exitCode = 1
return
}
eventsChan := make(chan *manager.ClientStreamMessage, clientBufferSize)
svc, err := newService(logger, tracer, qemuCfg, eventsChan, cfg.AttestationPolicyBinary, cfg.EosVersion)
if err != nil {
logger.Error(err.Error())
exitCode = 1
return
}
eventsSvc, err := events.New(logger, svc.ReportBrokenConnection, eventsChan)
if err != nil {
logger.Error(err.Error())
exitCode = 1
return
}
go eventsSvc.Listen(ctx)
mc := managerapi.NewClient(pc, svc, eventsChan, logger)
g.Go(func() error {
ch := make(chan os.Signal, 1)
signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM)
defer signal.Stop(ch)
select {
case <-ch:
logger.Info("Received signal, shutting down...")
cancel()
return nil
case <-ctx.Done():
return ctx.Err()
}
})
g.Go(func() error {
return mc.Process(ctx, cancel)
})
if err := g.Wait(); err != nil {
logger.Error(fmt.Sprintf("%s service terminated: %s", svcName, err))
}
}
func newService(logger *slog.Logger, tracer trace.Tracer, qemuCfg qemu.Config, eventsChan chan *manager.ClientStreamMessage, attestationPolicyPath string, eosVersion string) (manager.Service, error) {
svc, err := manager.New(qemuCfg, attestationPolicyPath, logger, eventsChan, qemu.NewVM, eosVersion)
if err != nil {
return nil, err
}
svc = api.LoggingMiddleware(svc, logger)
counter, latency := prometheus.MakeMetrics(svcName, "api")
svc = api.MetricsMiddleware(svc, counter, latency)
svc = tracing.New(svc, tracer)
return svc, nil
}
+60
View File
@@ -0,0 +1,60 @@
# Environment Configuration for Cocos
# Jaeger Tracing
COCOS_JAEGER_URL=http://localhost:4318
COCOS_JAEGER_TRACE_RATIO=1.0
# Manager Service Configuration
MANAGER_INSTANCE_ID=
MANAGER_ATTESTATION_POLICY_BINARY=../../build
MANAGER_GRPC_CLIENT_CERT=
MANAGER_GRPC_CLIENT_KEY=
MANAGER_GRPC_SERVER_CA_CERTS=
MANAGER_GRPC_URL=localhost:7001
MANAGER_GRPC_TIMEOUT=60s
MANAGER_EOS_VERSION=""
# QEMU Configuration
MANAGER_QEMU_MEMORY_SIZE=25G
MANAGER_QEMU_MEMORY_SLOTS=5
MANAGER_QEMU_MAX_MEMORY=30G
MANAGER_QEMU_OVMF_CODE_IF=pflash
MANAGER_QEMU_OVMF_CODE_FORMAT=raw
MANAGER_QEMU_OVMF_CODE_UNIT=0
MANAGER_QEMU_OVMF_CODE_FILE=/usr/share/OVMF/x64/OVMF_CODE.fd
MANAGER_QEMU_OVMF_VERSION=edk2-stable202408
MANAGER_QEMU_OVMF_CODE_READONLY=on
MANAGER_QEMU_OVMF_VARS_IF=pflash
MANAGER_QEMU_OVMF_VARS_FORMAT=raw
MANAGER_QEMU_OVMF_VARS_UNIT=1
MANAGER_QEMU_OVMF_VARS_FILE=/usr/share/OVMF/x64/OVMF_VARS.fd
MANAGER_QEMU_NETDEV_ID=vmnic
MANAGER_QEMU_HOST_FWD_AGENT=7020
MANAGER_QEMU_GUEST_FWD_AGENT=7002
MANAGER_QEMU_VIRTIO_NET_PCI_DISABLE_LEGACY=on
MANAGER_QEMU_VIRTIO_NET_PCI_IOMMU_PLATFORM=true
MANAGER_QEMU_VIRTIO_NET_PCI_ADDR=0x2
MANAGER_QEMU_VIRTIO_NET_PCI_ROMFILE=
MANAGER_QEMU_DISK_IMG_KERNEL_FILE=/home/sammyk/Documents/cocos-ai/cmd/manager/img/bzImage
MANAGER_QEMU_DISK_IMG_ROOTFS_FILE=/home/sammyk/Documents/cocos-ai/cmd/manager/img/rootfs.cpio.gz
MANAGER_QEMU_SEV_ID=sev0
MANAGER_QEMU_SEV_CBITPOS=51
MANAGER_QEMU_SEV_REDUCED_PHYS_BITS=1
MANAGER_QEMU_HOST_DATA=
MANAGER_QEMU_VSOCK_ID=vhost-vsock-pci0
MANAGER_QEMU_VSOCK_GUEST_CID=3
MANAGER_QEMU_VSOCK_VNC=0
MANAGER_QEMU_BIN_PATH=qemu-system-x86_64
MANAGER_QEMU_USE_SUDO=true
MANAGER_QEMU_ENABLE_SEV=false
MANAGER_QEMU_ENABLE_SEV_SNP=false
MANAGER_QEMU_ENABLE_KVM=true
MANAGER_QEMU_MACHINE=q35
MANAGER_QEMU_CPU=EPYC
MANAGER_QEMU_SMP_COUNT=4
MANAGER_QEMU_SMP_MAXCPUS=16
MANAGER_QEMU_MEM_ID=ram1
MANAGER_QEMU_KERNEL_HASH=false
MANAGER_QEMU_NO_GRAPHIC=true
MANAGER_QEMU_MONITOR=pty
MANAGER_QEMU_HOST_FWD_RANGE=6100-6200
+13
View File
@@ -0,0 +1,13 @@
# Copyright (c) Ultraviolet
# SPDX-License-Identifier: Apache-2.0
coverage:
ignore:
- "test/*"
- "cmd/**"
- "**/mocks/**"
- "mocks/**"
- "**/*.pb.go"
- "**/logging.go"
- "**/metrics.go"
- "**/tracing.go"

Some files were not shown because too many files have changed in this diff Show More