mirror of
https://github.com/ultravioletrs/cocos.git
synced 2026-06-23 04:10:25 +00:00
0ffc2d17cfd2dcfef111e91ea7e02b189e466c9d
27 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Sammy Kerata Oina
|
5377dd4d7f |
NOISSUE - Prepare cocos for v0.8.0 (#512)
* Refactor mock interfaces to use 'any' instead of 'interface{}' for improved type safety and readability across multiple files in the manager and pkg directories.
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* Update Go version to 1.25.x in CI workflows and remove obsolete Go package files
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* Add mock implementations for various components in the attestation and SDK packages
- Created mock for MeasurementProvider in pkg/attestation/cmdconfig/mocks/mocks_test.go
- Created mock for Provider in pkg/attestation/mocks/mocks_test.go
- Created mock for Client in pkg/clients/grpc/mocks/mocks_test.go
- Created mock for SDK in pkg/sdk/mocks/mocks_test.go
These mocks are generated using mockery and are intended for unit testing purposes.
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* Remove autogenerated mock files and update mock usage in tests
- Deleted mocks for gRPC clients in pkg/clients/grpc/mocks/mocks_test.go and pkg/sdk/mocks/mocks_test.go.
- Updated test files in pkg/progressbar/progress_test.go to use the new mock structure without type parameters for gRPC client interfaces.
- Refactored mock generation in pkg/sdk/mocks/sdk.go to streamline the mock creation process and ensure consistency across mock methods.
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* Update protobuf generated files for events and manager
- Bump protoc-gen-go version from v1.36.5 to v1.36.8 in events.pb.go and manager.pb.go.
- Refactor raw descriptor definitions in events.pb.go and manager.pb.go to use string concatenation for better readability and maintainability.
- Ensure compatibility with the latest protobuf specifications and improve code generation consistency.
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* Update test commands to use GOTOOLCHAIN for consistent Go version handling
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* Fix GOTOOLCHAIN usage in test command for consistency
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
---------
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
|
||
|
Sammy Kerata Oina
|
ccab296b62 |
NOISSUE - Downgrade Golang version for HAL (#494)
CI / lint (push) Has been cancelled
CI / test (agent) (push) Has been cancelled
CI / test (cli) (push) Has been cancelled
CI / test (cmd) (push) Has been cancelled
CI / test (internal) (push) Has been cancelled
CI / test (manager, true) (push) Has been cancelled
CI / test (pkg) (push) Has been cancelled
CI / upload-coverage (push) Has been cancelled
* Update Go version to 1.24.x in CI workflows and fix supermq version in go.mod Signed-off-by: Sammy Oina <sammyoina@gmail.com> * Refactor CI workflow to separate linting and testing jobs, and streamline test execution for multiple modules Signed-off-by: Sammy Oina <sammyoina@gmail.com> * Downgrade Go version from 1.23.10 to 1.23.8 in go.mod Signed-off-by: Sammy Oina <sammyoina@gmail.com> --------- Signed-off-by: Sammy Oina <sammyoina@gmail.com> |
||
 dorcaslitunya
|
94c169febb |
COCOS-394 Cloud Provider Attestation Service Integration (#421)
* Add token measurement command Add Azure cloud attestation fetching Add ability to fetch azure attestation token Remove gcp changes Remove gcp changes Add Azure attestation support Modify pipeline proto checks Update protoc version Fix failing CI fetch token as a file Convert jwt to json Small bug fix -- correct file name for attestation token Fix failing CI Modify protoc version Update protoc version Update protoc version Update protoc version Add changes to allow passing vtpm nonce Add PR review changes to refactor the code Refactor name change to AttestationResult Refactor name change to AttestationResult Return report as json Format files properly Fix attestaton changes Modify changes based on PR review Add more test coverage Correct bug in Server test Rename "FetchAttestationResult" to "AttestationResult" Send token as part of stream Fix CI NOISSUE - Add DisconnectReq message and TTL support for VM creation (#428) * feat: Add DisconnectReq message and TTL support for VM creation - Introduced DisconnectReq message in cvms.proto to handle disconnection requests. - Enhanced CreateReq in manager.proto to include a TTL field for virtual machines. - Updated CLI to accept TTL as a command-line flag during VM creation. - Modified manager service to remove VMs after the specified TTL duration. - Adjusted gRPC client connection handling in agent main.go to support new client structure. - Added mock implementation for gRPC client to facilitate testing. Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix: Mark server URL flag as required with error handling Signed-off-by: Sammy Oina <sammyoina@gmail.com> --------- Signed-off-by: Sammy Oina <sammyoina@gmail.com> COCOS-407 - Add support for Linux IMA (#429) * Added a feature which enables users to fetch IMA measurements and verify them * Added a feature which enables users to fetch IMA measurements and verify them * fixed lint error * fixed according to comments * fixed according to comments * fixed according to comments * fixed according to comments * final bug fix Add token measurement command Add Azure cloud attestation fetching Add ability to fetch azure attestation token Remove gcp changes Remove gcp changes Add Azure attestation support Modify pipeline proto checks Update protoc version Fix failing CI fetch token as a file Convert jwt to json Small bug fix -- correct file name for attestation token Fix failing CI Modify protoc version Update protoc version Update protoc version Update protoc version Add changes to allow passing vtpm nonce Add PR review changes to refactor the code Refactor name change to AttestationResult Refactor name change to AttestationResult Return report as json Format files properly Fix attestaton changes Modify changes based on PR review Add more test coverage Correct bug in Server test Rename "FetchAttestationResult" to "AttestationResult" Send token as part of stream Fix CI Rebase changes to main Refactor after rebase * Add Azure attestation * COCOS-395 - Cloud Provider Firmware Integration (#415) * add CC platform identification capability * add token verification * add snp azure * add azure snp report verification * fix linter errors * fix agent tests * expand the CC provider * fix azure atls * rebase branch * add nonce check for azure token * rename package attestations * remove alias attestations --------- Co-authored-by: Ubuntu <azureuser@UVCTestCVM.bu0p0zdolasezg1jifpyqhaxuc.dx.internal.cloudapp.net> * Add token measurement command Add Azure cloud attestation fetching Add ability to fetch azure attestation token Remove gcp changes Remove gcp changes Add Azure attestation support Modify pipeline proto checks Update protoc version Fix failing CI fetch token as a file Convert jwt to json Small bug fix -- correct file name for attestation token Fix failing CI Modify protoc version Update protoc version Update protoc version Update protoc version Add changes to allow passing vtpm nonce Add PR review changes to refactor the code Refactor name change to AttestationResult Refactor name change to AttestationResult Return report as json Format files properly Fix attestaton changes Modify changes based on PR review Add more test coverage Correct bug in Server test Rename "FetchAttestationResult" to "AttestationResult" Send token as part of stream Fix CI NOISSUE - Add DisconnectReq message and TTL support for VM creation (#428) * feat: Add DisconnectReq message and TTL support for VM creation - Introduced DisconnectReq message in cvms.proto to handle disconnection requests. - Enhanced CreateReq in manager.proto to include a TTL field for virtual machines. - Updated CLI to accept TTL as a command-line flag during VM creation. - Modified manager service to remove VMs after the specified TTL duration. - Adjusted gRPC client connection handling in agent main.go to support new client structure. - Added mock implementation for gRPC client to facilitate testing. Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix: Mark server URL flag as required with error handling Signed-off-by: Sammy Oina <sammyoina@gmail.com> --------- Signed-off-by: Sammy Oina <sammyoina@gmail.com> COCOS-407 - Add support for Linux IMA (#429) * Added a feature which enables users to fetch IMA measurements and verify them * Added a feature which enables users to fetch IMA measurements and verify them * fixed lint error * fixed according to comments * fixed according to comments * fixed according to comments * fixed according to comments * final bug fix Add token measurement command Add Azure cloud attestation fetching Add ability to fetch azure attestation token Remove gcp changes Remove gcp changes Add Azure attestation support Modify pipeline proto checks Update protoc version Fix failing CI fetch token as a file Convert jwt to json Small bug fix -- correct file name for attestation token Fix failing CI Modify protoc version Update protoc version Update protoc version Update protoc version Add changes to allow passing vtpm nonce Add PR review changes to refactor the code Refactor name change to AttestationResult Refactor name change to AttestationResult Return report as json Format files properly Fix attestaton changes Modify changes based on PR review Add more test coverage Correct bug in Server test Rename "FetchAttestationResult" to "AttestationResult" Send token as part of stream Fix CI Rebase changes to main Refactor after rebase * Rebase with main * Modify tests to accomodate changes * Use env vars appropriately * Use env vars appropriately * Use caps in err name --------- Co-authored-by: Danko Miladinovic <72250944+danko-miladinovic@users.noreply.github.com> Co-authored-by: Ubuntu <azureuser@UVCTestCVM.bu0p0zdolasezg1jifpyqhaxuc.dx.internal.cloudapp.net> |
||
|
Sammy Kerata Oina
|
c14f1d7b6c |
COCOS-391- GCP Attestation policy (#405)
* Add AgentGrpcHost configuration to agent server Signed-off-by: Sammy Oina <sammyoina@gmail.com> * Add SHA1 support to PcrValues and implement GCP attestation functions Signed-off-by: Sammy Oina <sammyoina@gmail.com> * Add GCP attestation policy and OVMF download commands Signed-off-by: Sammy Oina <sammyoina@gmail.com> * Add vTPM attestation support and update protobuf versions Signed-off-by: Sammy Oina <sammyoina@gmail.com> * Remove Host field from AgentConfig and update related references Signed-off-by: Sammy Oina <sammyoina@gmail.com> * Update GCP attestation policy to accept vCPU count as an argument Signed-off-by: Sammy Oina <sammyoina@gmail.com> * Add SHA512 digest verification for OVMF file in GCP download command Signed-off-by: Sammy Oina <sammyoina@gmail.com> * Update OVMF object name format in GCP attestation package Signed-off-by: Sammy Oina <sammyoina@gmail.com> * Refactor attestation policy structure to use nested Config field Signed-off-by: Sammy Oina <sammyoina@gmail.com> --------- Signed-off-by: Sammy Oina <sammyoina@gmail.com> |
||
 Danko Miladinovic
|
67f939fc66 |
COCOS-326 - Add vTPM support to CoCoS (#376)
* manager, cli and agent vtpm support * rebase and changed atls for vtpm * deleted unused code * changed chekproto.yaml script so it find the manager proto file correctly * fixe manager proto version * fix agent tests * fix server agent test * fix attestation test * fix attestation test gofumpt * created dummy RWC for TPM * fix comment * add default PCR values * rebase main * fix rust ci and missing header * changed embedded attestation to VMPL 2 * fix unused impot * fix pkg test * address attestation type * fix agent attestation test * add prc15 check * fix comments * fix cli tests * add doc * add mock for LeveledQuoteProvider when SEV-SNP device is not found Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix manager reading attestation policy * refactor PCR value checks and update attestation policy values Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix tests for sev and grpc --------- Signed-off-by: Sammy Oina <sammyoina@gmail.com> Co-authored-by: Sammy Oina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
ecad6514f3 |
COCOS-344 - New agent structure (#350)
* new agent structure Signed-off-by: Sammy Oina <sammyoina@gmail.com> * minor fixes and testing Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix lint Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix tests Signed-off-by: Sammy Oina <sammyoina@gmail.com> * cvm tests fix Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix test Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix cli test Signed-off-by: Sammy Oina <sammyoina@gmail.com> * rename Signed-off-by: Sammy Oina <sammyoina@gmail.com> * rename cvm to cvms plural Signed-off-by: Sammy Oina <sammyoina@gmail.com> * rename service Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix tests Signed-off-by: Sammy Oina <sammyoina@gmail.com> * remove context Signed-off-by: Sammy Oina <sammyoina@gmail.com> * refactor: reorder parameters in NewAlgorithm functions and update CVMClient to CVMSClient Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix(tests): update SendEvent mock to include an additional parameter Signed-off-by: Sammy Oina <sammyoina@gmail.com> * move expectations Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix(tests): move event initialization to the correct scope in service tests Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix(tests): update SendEvent mock to use EXPECT instead of On in service tests Signed-off-by: Sammy Oina <sammyoina@gmail.com> --------- Signed-off-by: Sammy Oina <sammyoina@gmail.com> |
||
|
Danko Miladinovic
|
d5941edb56 |
NOISSUE - Add information on OVMF verison, CPU type, CPU number and EOS version (#307)
* add manager OVMF verison information * add Manager documentation * add kernel cmd * add SVMInfo test |
||
|
Sammy Kerata Oina
|
6043ad150b |
COCOS-256 - Progress bar on downloads (#290)
* add progress bar for downloads Signed-off-by: Sammy Oina <sammyoina@gmail.com> * better error handling Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix test and refactor Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix failing test Signed-off-by: Sammy Oina <sammyoina@gmail.com> * add test coverage Signed-off-by: Sammy Oina <sammyoina@gmail.com> --------- Signed-off-by: Sammy Oina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
46d24f928a |
NOISSUE - Add events for disconnection of agent (#233)
* add events Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix lint Signed-off-by: Sammy Oina <sammyoina@gmail.com> * typo Signed-off-by: Sammy Oina <sammyoina@gmail.com> * group logs Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix error Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix initialization of goroutine Signed-off-by: Sammy Oina <sammyoina@gmail.com> * add comment Signed-off-by: SammyOina <sammyoina@gmail.com> * update comment Signed-off-by: SammyOina <sammyoina@gmail.com> * fix lint Signed-off-by: SammyOina <sammyoina@gmail.com> * remove naked return Signed-off-by: Sammy Oina <sammyoina@gmail.com> --------- Signed-off-by: Sammy Oina <sammyoina@gmail.com> Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
8db88ccbde |
NOISSUE - Fix handling of runreq chunks (#234)
* fix handling of runreq chunks Signed-off-by: SammyOina <sammyoina@gmail.com> * copy ovmf vars Signed-off-by: SammyOina <sammyoina@gmail.com> * fix lint errors Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
 b1ackd0t
|
afc306a85b |
NOISSUE - Enable WASM Support and FileSystem Support (#189)
* feat(algorithm): Add wasm as an algo type Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * feat(algorithm): Use filesystem to store results Move from unix socket for results storage to filesystem * test: test new filesystem changes Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * refactor(files): rename resultFile to resultsFilePath * feat(wasm-runtime): change from wasmtime to wasmedge Wasmedge enables easier directory mapping to get results Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * feat(algorithm): send results as zipped directory Create a new function to zip the results directory and send it back to the user * fix(wasm): runtime argument Fix the directory mapping for wasm runtime arguments Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * fix(errors): provide useful error message * chore(gitignore): add results zip to gitignore * feat(filesystem): Enable storing results on filesystem for python algos * refactor: revert to upstream cocos repo Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * fix: remove AddDataset from algorithm interface * fix: agent to handle results zipping * test: test zipping directories * refactor(agent): Handle file operations from agent * test: run test inside eos Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * refactor(test): Document and test algos are running Document steps on running the 2 python exampls and ensure they are running on eos Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * fix: remove witheDataset option * test: test without dataset argument Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> --------- Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> |
||
|
Sammy Kerata Oina
|
67d01e39be |
COCOS-155 - Add python algo support (#178)
* * feat(algorithm.go): add support for algorithm type context * feat(python.go): implement Python algorithm runtime * fix(cocos_defconfig): add IPTABLES package Signed-off-by: SammyOina <sammyoina@gmail.com> * update proto Signed-off-by: Sammy Oina <sammyoina@gmail.com> * small fixes Signed-off-by: Sammy Oina <sammyoina@gmail.com> * add metadata Signed-off-by: Sammy Oina <sammyoina@gmail.com> * debug Signed-off-by: Sammy Oina <sammyoina@gmail.com> * debug Signed-off-by: Sammy Oina <sammyoina@gmail.com> * chunk logger Signed-off-by: Sammy Oina <sammyoina@gmail.com> * debug logger Signed-off-by: Sammy Oina <sammyoina@gmail.com> * test lock Signed-off-by: Sammy Oina <sammyoina@gmail.com> * add req file Signed-off-by: SammyOina <sammyoina@gmail.com> * stream result Signed-off-by: SammyOina <sammyoina@gmail.com> * test with venv Signed-off-by: Sammy Oina <sammyoina@gmail.com> * fix missing requirements file Signed-off-by: Sammy Oina <sammyoina@gmail.com> * result stream Signed-off-by: Sammy Oina <sammyoina@gmail.com> * modify test server Signed-off-by: Sammy Oina <sammyoina@gmail.com> * remove debugging and cleaning up Signed-off-by: Sammy Oina <sammyoina@gmail.com> * original repo Signed-off-by: Sammy Oina <sammyoina@gmail.com> * add missing header Signed-off-by: Sammy Oina <sammyoina@gmail.com> * downgrade protoc Signed-off-by: Sammy Oina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> Signed-off-by: Sammy Oina <sammyoina@gmail.com> |
||
|
b1ackd0t
|
dc16e8a997 |
NOISSUE - Use Magistrala Shared Packages (#174)
* refactor(env): remove internal env package No need for this package since the parent package github.com/caarlos0/env does everything we need Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * refactor(jaeger): remove internal jaeger package No need for this package since we can use magistrala exported package Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * refactor(metrics): remove internal metrics pkg Use exported magistrala prometheus package Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * chore(dep): Update grpc and other dependencies Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> * style(linter): remove enabled by default linters Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> --------- Signed-off-by: Rodney Osodo <socials@rodneyosodo.com> |
||
|
Sammy Kerata Oina
|
f4e3e8e09c |
COCOS-157 - Provide abstractions for VM management (#171)
* abstract vm creation and allow stopping computation Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor QEMU configuration loading and execution in main.go Signed-off-by: SammyOina <sammyoina@gmail.com> * * feat(agent-config): add support for sending agent configuration to manager Signed-off-by: SammyOina <sammyoina@gmail.com> * * chore(checkproto.yaml): update protoc-gen and protoc-grpc versions Signed-off-by: SammyOina <sammyoina@gmail.com> * * chore(auth): update mockery version to v2.43.2 * chore(main.go): update import path for vm package in agent * chore(main.go): update import path for vm package in manager * chore(go.mod): add github.com/google/logger v1.1.1 as a required dependency * chore(manager_test.go): update import path for vm package in manager * chore(logging.go): move logging.go to manager/qemu/vm package * chore(logging_test.go): move logging_test.go to manager/qemu/vm package * chore(vm_factory.go): rename vm_factory.go to provider.go in manager/qemu/vm/mocks package * chore(vm.go): move vm.go to manager/qemu/vm package * chore(vm.go): update import path for vm package in manager * chore(vm_test.go): move vm_test.go to manager/qemu/vm package * chore(vsock.go): move vsock.go to manager Signed-off-by: SammyOina <sammyoina@gmail.com> * * fix(main.go): change import path for 'github.com/ultravioletrs/cocos/manager/qemu/vm' to 'github.com/ultravioletrs/cocos/manager/vm' * fix(main.go): change vsock.Dial argument from 'vm.VsockConfigPort' to 'qemu.VsockConfigPort' * fix(main.go): change import path for 'github.com/ultravioletrs/cocos/manager/qemu' to 'github.com/ultravioletrs/cocos/manager/qemu' Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
2ce112cc1b |
COCOS-103 - User authN and AuthZ using digital signatures (#128)
* Update Go to 1.22 and enhance security features - Upgraded the Go version in GitHub Actions workflows to 1.22.x for latest features and security patches. - Added RSA public key field `UserKey` in `Dataset` and `Algorithm` to reinforce data integrity and encryption. - Refactored `Result` method in `agentService` to use `containsID` for improved readability and potential performance benefits. - Updated `grpcserver.New` and `internal/server/grpc` invocations to pass `agent.Service` by value in line with recommended Go practices. - Introduced `grpc.StreamInterceptor` with no args in `Server.Start` which seems to be an initial step for future stream interceptor configuration. These changes prepare for stronger data security measures, maintain compatibility with the latest Go features, and improve code quality regarding service struct usage. Potential follow-up is needed to configure the stream interceptor and to ensure the new RSA key field is appropriately utilized in data handling. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor auth system and protocol buffers Enhanced the authentication system by adding context support and an improved user-role model. Implemented robust RSA public key verification for users and a restructured interceptor logic specific to stream types, streamlining the auth process. Updated protocol buffers and associated structures to accommodate user keys as byte slices, aligning with standard cryptographic practice. CLI commands for algorithms and datasets now require a private key file path argument for signing, strengthening security during interactions. This comprehensive overhaul addresses security and efficiency considerations in the RPC framework and aligns with best practices for key handling. By streamlining and securing the user authentication process, the agent service's reliability is greatly improved, directly impacting the robustness of the entire computation pipeline. - Refactored auth: added role-based user validation, context handling - Reworked interceptors: separated stream types, fortified signature checks - Updated protocol buffers: user public keys as byte slices for standard compatibility - Enhanced CLI: introduced private key argument, ensuring secure algorithm and dataset submission - Improved server and SDK contracts to align with auth changes Related issues: - Implements user roles and auth context [#103] - CLI security enhancement for private key management Signed-off-by: SammyOina <sammyoina@gmail.com> * Updated PEM decoding for key parsing in CLI and tests Added `encoding/pem` to decode PEM blocks when parsing private and public keys across CLI commands and test computation scenarios, ensuring compatibility with key files. This enhances robustness in key handling by supporting PEM encoded keys. The update also includes registration of a new Keys command in the CLI. Refactored code is now compliant with common key formats, addressing potential parsing issues. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix auth signature encoding and improve CLI usage example The authentication system now decodes base64 strings before verifying signatures to align with the expected format. Additionally, the signature generation now encodes the output in base64, ensuring consistency across the auth process. The CLI help message for the `result` command is enhanced by providing a usage example, making it more user-friendly and informative. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor containsID to handle dynamic fields Updated the `containsID` function to accept a field name parameter, enabling dynamic field lookup within the reflection logic. This change facilitates the use of the function for various struct fields, improving code reusability and flexibility. CLI command 'data' now requires an additional argument for the private key file path, outlined in the usage example update, reinforcing command clarity and user guidance. Resolves issues with hardcoded field lookups and enhances CLI usability. Signed-off-by: SammyOina <sammyoina@gmail.com> * Remove extraneous newline in key generation log output A redundant newline after the success message in the key generation command was removed to clean up log output formatting. This change ensures a more consistent and professional appearance of the CLI tool's messages. Signed-off-by: SammyOina <sammyoina@gmail.com> * Implemented auth service in gRPC startup Added authentication services to the gRPC server initialization to enforce security measures. The gRPC server's New function now includes an `authSvc` parameter, requiring instantiation of the auth service before starting the server. Failure to create the auth service results in a fatal error, halting the process to avoid running without protection. Tests have been updated to include `nil` values for the auth service parameter to maintain their functionality without authentication. Refactored `grpcserver.New` to accept the new auth service, and updated the main agent startup logic to create and inject the auth service. Added the auth middleware interceptors to the server options, which ensures that each gRPC call will undergo authentication. This change is a step towards secure communication, and affected components should now consider the authentication requirement. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor config read logic and update agent setup Improved the configuration reading in `cmd/agent/main.go` to handle larger payloads by reading data in chunks and checking for EOF, ensuring that all config data is captured even if it exceeds the initial buffer size. Enhanced the `test/manual/agent-config/main.go` to require additional command-line arguments, improving the setup process by explicitly requiring paths for data, algorithm, and public key as well as a boolean for attested TLS. Also updated the hashing method to SHA3 for the algorithm and data files, and included the hash and public keys as part of the agent, dataset, and result consumer configurations. These changes will make the agent setup more robust and provide better integrity checks for the involved files. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor run method to agentService Moved the run function into agentService for better encapsulation and maintainability. This refactoring includes capturing both stdout and stderr during algorithm execution, enabling more informative debugging through enhanced logging. Consequentially, the run method now references members through the service instance, aligning with object-oriented best practices and improving code coherence. Resolves issue with insufficient execution details when computations fail. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor computation data handling to use filepaths Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor error logging and ensure consistency Replaced usage of the standard log package with a custom logger for error reporting to standardize error logging throughout the application. Additionally, introduced graceful shutdown by returning from the main function rather than forcing exit when failing to create auth service, aligning the application's error handling strategy. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor auth initialization and key file handling Improved the readability and maintainability of the authentication service initialization by adding line breaks for logical separation. Also, standardized key filenames in the CLI key generation by introducing constants, enhancing code clarity and reducing the likelihood of file-naming errors. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor auth verification logic for improved security Removed an extraneous line in the `verifySignature` function that was not necessary for the signature verification process. This change simplifies the code and improves readability. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor payload structures to simplify API Removed the 'provider', 'id', 'consumer' fields from protocol buffers, gRPC services, and related functions across various files to streamline the data model and align with the new authentication system based on cryptographic verification rather than string identifiers. This results in more efficient data handling and a reduction in unnecessary payload data, while enhancing security by making entity validation strictly cryptographic. The changes affect agent-SDK interactions, CLI tools, and related services, ensuring only the necessary data (algorithm/data bytes, user keys, and hashes) is transmitted and processed. Consequently, the core computation algorithm and dataset handlers now rely on indexes derived from context to associate data with respective manifest entries, thus maintaining the ability to link to specific computation manifests without relying on explicit IDs in the payload. Additionally, refactored authentication methods now enforce role-based security seamlessly through metadata. This approach enhances privacy by avoiding transmission of potentially sensitive strings over the network and by ensuring that only internal indices, not globally interpretable identifiers, are used to process computations. Aligned with the broader architectural goal of simplifying and securing the platform's core services, this change paves the way for upcoming revisions to the authentication scheme that will further consolidate role-based security and improve system integrity. Signed-off-by: SammyOina <sammyoina@gmail.com> * Enhance CLI security with key paths Removed the section on running computations from the CLI README as it may no longer be necessary or the functionality has been moved elsewhere. Required private key file paths for algorithm, dataset upload, and result retrieval commands to enhance security. This change associates each action with a specific identity, ensuring secure and traceable operations. Additionally, updated the manual test commands to reflect this new requirement. Signed-off-by: SammyOina <sammyoina@gmail.com> * fix ci Signed-off-by: SammyOina <sammyoina@gmail.com> * fix fmt Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
|
Danko Miladinovic
|
3a14896555 |
NOISSUE - Attested TLS (#99)
* added initial code for attested TLS * added client validation and verification * fixed bugs for attested TLS * updated README for manual testing * fixed CI errors * removed SNP pollicy from agent config * added attested TLS config param to AgentConfig * generated manager.pb.go for protoc v25.2 * updated proto-gen-go version on CI * generated agent.pb.go to match newest proto gen version * define errors for error handling * fixed comments |
||
|
Sammy Kerata Oina
|
2b760ec207 |
NOISSUE - Streamline client identification (#105)
* Update protoc to v4.25.3 and streamline client identification Protocol Buffer compiler `protoc` has been upgraded to version 4.25.3. This update unifies the version across multiple generated files to ensure compatibility and take advantage of any bug fixes and performance improvements in the new release. Additionally, the client identification process has been refined. The redundant `WhoAmI` message and the corresponding checks have been removed from the gRPC server implementation. Clients are now identified via their address from the `Process` stream's context as soon as a connection is established, simplifying the code and potentially reducing handshake time. This change sets the foundation for a leaner communication protocol between manager and agents, and could contribute to lower latencies in client-server interactions. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix goroutine bug in grpc server Signed-off-by: SammyOina <sammyoina@gmail.com> * Update PROTOC_VERSION to 25.3 Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor server reference in readme Updated the deployment instructions in the README to point to the correct server location after consolidating test server documentation. Removed obsolete `manager-server` module and associated main entry point to align with the new architecture and streamline workflows. Resolves issues with outdated links and cluttered repository structure. Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
5c406743e0 |
NOISSUE - Add attesatation validation/verification options (#87)
* Standardize attestation report data size to 64 bytes Redefined the report data for attestation requests from a variable byte slice to a fixed 64-byte array across various critical components. This change enforces a standardized data length, simplifying validation logic and ensuring consistency when handling the attestation data for both the server and client end-points, logging functionalities, and the CLI tool. Updated attestation data handling in the SDK to accommodate the new fixed-length constraint. By strictly adhering to the 64-byte requirement, the modifications promote robust input checking and prevent potential issues related to dynamic data length processing. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor attestation validation logic Streamlined attestation validation in the CLI with a new configuration parsing approach. Introduced a retry mechanism with timeout and backoff for HTTPS getters that facilitate remote trust validation. Clarified expected data lengths in proto comments across attestation and manager services, ensuring consistency and correctness of the cryptographic data fields. This modification enhances maintainability by replacing a verbose struct with a configurable object model, thus simplifying parameter handling. Robustness is improved through failure-retrieval strategies during remote validation, and additional in-code documentation specifies cryptographic constraints for critical data elements. Signed-off-by: SammyOina <sammyoina@gmail.com> * Add new attestation validation flags Enhanced attestation validation in the CLI with additional flag support for FAMILY_ID, IMAGE_ID, REPORT_ID, REPORT_ID_MA, MEASUREMENT, CHIP_ID, and TCB-related fields. This update enables fine-grained control over attestation report criteria, ensuring stricter validation conforming to protocol requirements. Adjusted the default value for MinimumGuestSvn to align with the new policy specifications. Signed-off-by: SammyOina <sammyoina@gmail.com> * Enhance attestation policy configuration Introduced several new configuration flags to the CLI's attestation policy, including options for minimum AMD-SP firmware builds, revocation list checking, and network restrictions. Expanded trust criteria parameters with the support for trusted author and identity key hashes. The default VMPL setting is now explicitly initialized. These changes improve security controls and offer more detailed attestation verification settings. Signed-off-by: SammyOina <sammyoina@gmail.com> * Simplify attestation validation process Refactored the attestation CLI command to improve usability by removing the need for explicitly provided report data as an argument. Default values for configuration parameters have been centralized into constants for maintainability. Additionally, integrated data parsing functions to streamline the validation checks with robust error handling for cases such as invalid hex strings and file I/O issues. This enhancement reduces user error and the complexity of entering attestation data by parsing relevant information from within the environment, all while maintaining the same security standards. The process of marking flags as required for CLI commands has been corrected to prevent runtime errors and improve command reliability. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refined attestation validation logic Enhanced attestation command handling by removing the hardcoded report data length and replacing it with the standard SHA-512 hash size, ensuring dynamic compatibility with hash lengths. Introduced size constants to improve code readability and enforce explicit length checks on attestation input data, raising errors when requirements are unmet. This preventative measure aims to avert potential runtime errors stemming from unexpected data sizes. The change also simplifies the minimum guest SVN description for clarity. References to removed unnecessary whitespace maintain code cleanliness. These adjustments culminate in a more robust and maintainable attestation process, in accordance with the best practices for secure handling of cryptographic data and compliance with AMD's ABI format specifications. Signed-off-by: SammyOina <sammyoina@gmail.com> * Enhanced attestation documentation and command syntax Updated CLI documentation to include detailed descriptions of the attestation retrieval and validation process, along with a comprehensive list of new flags for the validation command, providing users with extensive control over the attestation verification process. Updated command syntax with flags for specifying report data, improving command clarity and consistency. These changes ensure better user guidance and offer a more robust attestation handling experience. Signed-off-by: SammyOina <sammyoina@gmail.com> * Enforce report data size validation uniformly Refactored report data size validation to ensure it is performed consistently across the system. Removed the hard-coded length check from attestation request validation in favor of a centralized verification based on the sha512 digest size. Also, eliminated fallback mechanism to the SHA-512 hash of report data and made the length requirement explicit, leading to immediate failure if not met. This harmonizes the error handling logic, reinforces data integrity checks, and simplifies debugging by removing ambiguous length correction behavior. Adjusted related validation functions to use a common utility, enhancing maintainability. Fix spelling errors in log messages for attestation validation commands to improve clarity in output. Signed-off-by: SammyOina <sammyoina@gmail.com> * Initialize attestation config with defaults Enhanced the attestation command initialization by setting default values for the `Config` struct to prevent nil pointer exceptions and potentially streamline configuration handling. Moreover, corrected a flag declaration by removing shorthand.P support for 'permit_provisional_software', aligning it with other flags for consistency. This change should improve stability and user experience while configuring attestation policy. Signed-off-by: SammyOina <sammyoina@gmail.com> * nil check prior Signed-off-by: SammyOina <sammyoina@gmail.com> * fix typos Signed-off-by: SammyOina <sammyoina@gmail.com> * use file for attestation report Signed-off-by: SammyOina <sammyoina@gmail.com> * Refine attestation validation with default policy values Adjusted default values and constants related to the attestation configuration for enhanced clarity and consistency. Changed the default minimum guest SVN value to align with new policy requirements and introduced a default guest policy constant. Additionally, standardized byte array placeholders for attestation validation fields to prevent potential issues with uninitialized bytes and ensure robust input validation checks. These changes promote more intuitive and secure attestation validation procedures. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refine attestation defaults and flag requirements Updated the default guest policy to use hexadecimal notation for clarity and added a default minimum version for the firmware API version flag. Made 'chip_id', 'measurement', 'report_id', and 'report_id_ma' flags mandatory to ensure critical attestation data is provided by the user, enhancing validation robustness. Added a check for 'measurement' field length for consistency in input validation. Signed-off-by: SammyOina <sammyoina@gmail.com> * Optimize attestation file format Standardized attestation file format across the codebase by moving the 'attestationFilePath' constant from the function scope to a higher scope within the package. Switched the file extension from .txt to .bin to reflect binary data storage, improving consistency and potential performance benefits from binary file handling. Signed-off-by: SammyOina <sammyoina@gmail.com> * Optimize default values for attestation validation Removed unnecessary empty initializers and replaced hardcoded empty values with nil for optional fields in attestation validation flags. Introduced a constant defaultReportIdMa for the REPORT_ID_MA field to provide a meaningful default instead of an empty one. Streamlined the command setup by removing the requirement flags for fields that are now optional and unchecked if unset, reducing burdens on the user for providing unnecessary input. The change simplifies the command's usage and eliminates the need for users to specify values for certain fields that can be optional, improving the command's flexibility and user experience. Additionally, setting a specific default for REPORT_ID_MA ensures clarity in cases where this is not provided by the user. This update enhances the maintainability and usability of the attestation validation functionality. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor hardcoded attestation report size Introduced a constant `size64` to replace the previously hardcoded array size of `64` used across various Attestation methods. This change enhances code maintainability and readability by centralizing the size definition, making it easier to update in the future if necessary. Signed-off-by: SammyOina <sammyoina@gmail.com> * Updated protocol buffer dependencies and cleaned CLI flags Protobuf compiler versions were updated to v4.25.2 across various RPC service definitions for agent and manager packages, ensuring compatibility with the latest features and fixes. In the CLI attestation code, unnecessary flags (`DisallowNetwork` and `PermitProvisionalSoftware`) were removed to streamline user options, reflecting a more secure and user-focused configuration interface. Added an input validation check in `attestation.go` for CA bundle presence when a product name is set, enhancing the reliability of attestation validation. Also rectified a bug in `grpc.go` by fixing an incorrect variable assignment for reading the key file content, thereby preventing potential TLS-related errors. Signed-off-by: SammyOina <sammyoina@gmail.com> * Updated protoc version and adjusted proto file paths Bumped the protocol compiler version to 25.2 to align with our dependency updates and ensure compatibility. Additionally, revised the proto file paths under the continuous integration setup, moving from 'manager/' to 'pkg/manager/' to reflect the recent directory restructuring. The changes ensure that proto file checks and comparisons are conducted in the correct file locations, preventing potential build and sync issues in future developments. Signed-off-by: SammyOina <sammyoina@gmail.com> * Enhance hash size consistency in gRPC attestation Aligned the hard-coded byte array size for `ReportData` with `sha512.Size` constant to ensure consistency and maintainability in gRPC attestation requests and responses. This change mitigates the risk of future errors if the SHA512 hash size standard is altered. Signed-off-by: SammyOina <sammyoina@gmail.com> * Optimize agent-config listener loop Removed the goroutine wrapping the listener loop in 'agent-config' manual test to streamline and simplify the connection handling logic. This change ensures the loop runs on the main goroutine, improving the readability and maintainability of the code. Refactoring assists in avoiding potential race conditions and makes the server's flow easier to understand for future debugging and development. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor report data size checks to use constant Centralize the report data size definition by replacing various size checks and array declarations with a constant `ReportDataSize`. This streamlines code maintenance and ensures consistency across all instances where report data size is validated or used. The changes remove direct references to the hash function output size, decoupling the report data size from the hash function's characteristics and allowing easier updates if the data size requirement changes. These modifications bolster the codebase's flexibility for potential adjustments in security protocols or data handling specifications. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor config JSON handling in attestation CLI Introduced an `exampleJSONConfig` constant containing a sample configuration to improve code readability and maintainability. Previously, the example JSON configuration for the attestation CLI was an inline string, making the code cluttered and less maintainable. With this change, the example configuration is now stored as a constant, resulting in cleaner command flag setup and enhanced clarity. This constant is used in the command flag description to guide users when providing their custom configuration. Additionally, refactored the command flags related to the configuration by aligning and sorting them for better code organization. Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
722b463b6a |
NOISSUE - Use a single listener for logs and events (#82)
* add handler Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor gRPC and Protobuf integration for manager service - Shifted Protobuf message definitions to a separate package `pkg/manager`. - Updated references throughout the codebase to import and use the new package for gRPC service definitions. - Enhanced AgentLog message with additional fields `level` and `timestamp`. - Removed direct dependencies on old Protobuf-generated types in favor of the new package. - Deleted obsolete Protobuf-generated files as they are now superseded by the new `pkg/manager`. - Streamlined event publishing and gRPC handling in the manager service to use the updated Protobuf messages. This refactoring improves modularity by centralizing Protobuf message definitions and decouples internal representation from the gRPC interface, aligning with best practices for microservice architecture. Additionally, the enriched logging structure paves the way for more detailed and fine-grained log analysis. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor vsock event/log handling and config Streamlined event and log services in the manager by moving vsock listening functions out of `managerService` initialization and into dedicated `RetrieveAgentEventsLogs` methods. This change decouples the manager service creation from the actual start of log listening, adding clarity and flexibility in service management. Also moved logging middleware invocation outside of network handling loops to avoid unnecessary overhead. Additionally, the agent's vsock port configuration is now dynamically passed to the `New` function in the `events` package instead of relying on a hardcoded constant, allowing for greater configurability and testability. Finally, updated message structures for event and log sending to conform with the `ClientStreamMessage` definitions. These modifications should improve parsing and handling consistency and prepare our system for future enhancements related to inter-process communication. Signed-off-by: SammyOina <sammyoina@gmail.com> * fix linting errors Signed-off-by: SammyOina <sammyoina@gmail.com> * correct path to generated files Signed-off-by: SammyOina <sammyoina@gmail.com> * fix comments Signed-off-by: SammyOina <sammyoina@gmail.com> * remove uneccessary comments Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
8975e28437 |
COCOS-77 - Stream agent logs and events over gRPC (#78)
* Refactor GRPC manager service and client The manager service and client have been restructured for stream communication, facilitating real-time agent events, logs, and run responses. The `Run` RPC is replaced by the `Process` stream RPC, enabling bidirectional streaming between clients and the manager service. This allows continuous interchange of different message types including `WhoAmIRequest`, `AgentLog`, `AgentEvent`, and `RunResponse`. Several message types have been adjusted and new fields introduced, like `AgentPort` in `RunResponse` and various agent-config attributes including CA files and instance IDs, to support TLS client authentication and distinguish between agent instances. We've also incorporated `google.protobuf.Timestamp` in `AgentEvent` for precise event logging. The client code reflects these modifications with updated method calls and stream handling logic for ongoing communication. Moreover, the updates necessitate corresponding changes throughout service, grpc, and sdk layers to interoperate with the new streaming approach. The transition to streaming paves the way for a more interactive, flexible communication system that can accommodate future expansion and real-time monitoring features. Signed-off-by: SammyOina <sammyoina@gmail.com> * fix lint Signed-off-by: SammyOina <sammyoina@gmail.com> * Update GitHub Actions to Latest Versions Upgraded GitHub Actions 'checkout' to version 4 and 'setup-go' to version 5 across various workflow files to leverage the latest features and improvements for better performance and reliability. This also ensures compatibility with Go version 1.21.x which is specified in the workflows. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor event handling and logging Reworked event and log processing to use channels instead of direct HTTP calls. Removed obsolete events package and consolidated event structures, leading to cleaner and more maintainable code. Updated agent events to use channels, enhanced error handling in log forwarding, and simplified manager `New` function signature to accept an event channel directly. - Removed `events` and `agentevents` packages to reduce complexity. - Replaced direct event server communication with internal channel usage. - Introduced `AgentEvent` struct in events.go for standardized event objects. - Adapted `managerService` to dispatch events and logs through channels. - Streamlined manager construction by removing the now-unnecessary event service and host IP parameters. This change results in a more robust and easier to extend event and log management system within the agent-manager interaction. Signed-off-by: SammyOina <sammyoina@gmail.com> * fix ci Signed-off-by: SammyOina <sammyoina@gmail.com> * remove unused code Signed-off-by: SammyOina <sammyoina@gmail.com> * add comments Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
c29ef354fa |
COCOS-24 - Add events service via http (#27)
* add status endpoint Signed-off-by: sammy <sammyoina@gmail.com> * feat: Update code generation tools to latest versions Update the code generation tools, including protoc-gen-go and protoc-gen-go-grpc, to their latest versions (v1.31.0 and v1.3.0, respectively). This ensures compatibility with the latest features and improvements. The updated tools also require gRPC-Go v1.32.0 or later. The new versions bring important updates and bug fixes, enhancing the performance and stability of the generated code. By staying up-to-date with the latest tooling, we can take advantage of the latest functionality and ensure a smooth development experience. No code changes are included in this commit. These updates only impact the code generation process. Signed-off-by: sammy <sammyoina@gmail.com> * Update Go version to 1.21.x Signed-off-by: sammy <sammyoina@gmail.com> * Refactor agent and manager services to publish event notifications The refactoring includes changes to the agent and manager services to incorporate event notifications. By publishing events, the services can inform subscribers about the current state of the computation or any updates. Specifically, the `agentService` now includes a `cmpHash` field to store the SHA-256 hash of the computation, which is subsequently used when publishing events. The `agentService` and `managerService` now use the `publisher` interface to publish events to the topic "manager". Notably, the removed `pubsub.go` file is no longer necessary. This commit improves the service architecture by allowing subscribers to receive relevant updates and monitor the progress of computations. It enhances the overall system by providing more transparency and enabling better coordination between the agent and manager services. Signed-off-by: sammy <sammyoina@gmail.com> * Improve generated Go file comparison in checkproto workflow Refactor the file comparison logic in the checkproto workflow to use the `-p` flag instead of `-s` for improved accuracy. This change ensures that the generated Go files are thoroughly compared with the original ones, detecting any discrepancies and preventing out-of-sync files from passing the validation. By using the `-p` flag, we now check both the contents and the metadata of the files, providing more robust synchronization checks. This update enhances the reliability of the checkproto workflow and helps maintain consistency between the proto files and their corresponding generated Go files. Signed-off-by: sammy <sammyoina@gmail.com> * Update file comparison command to detect differences line by line The code change updates the file comparison command used in the CI workflow to detect differences line by line instead of only reporting the first difference encountered. This change improves the accuracy of detecting inconsistencies between the original protobuf files and the generated Go files. Previously, only the first difference was reported, leading to potential missed issues. By comparing the files line by line, we can now detect and report all differences accurately. This change enhances the reliability of our CI pipeline and ensures that the generated Go files stay in sync with the protobuf files. Signed-off-by: sammy <sammyoina@gmail.com> * add event exporting to external server Signed-off-by: sammy <sammyoina@gmail.com> * feat: Add support for notification server URL The commit adds a new environment variable, `COCOS_NOTIFICATION_SERVER_URL`, which allows specifying the server to receive notification events from the agent. This addition provides flexibility to configure the notification server URL based on the deployment environment. This change enables seamless integration with different notification server instances and enhances the extensibility of the system. It resolves the need to modify the code directly when changing the server URL. Signed-off-by: sammy <sammyoina@gmail.com> * Refactor gRPC client and server, remove unused handlers The commit refactors the gRPC client and server code by removing the unused `nopDecoder` and `status` handlers from the client and server, respectively. This cleanup reduces code clutter and improves maintenance. No significant consequences are expected. Signed-off-by: sammy <sammyoina@gmail.com> * Ensure generated Go files stay in sync with proto files during the CI workflow Fixes an issue in the CI workflow where proto files and their corresponding generated Go files were not being properly compared for synchronization. Previously, the `cmp -l` command was used, which only printed differing byte positions, leading to false negatives. This has been corrected by using `cmp -s` instead, which outputs nothing if the files are identical. This change ensures that any differences between the proto files and their generated Go files will be detected, helping to maintain consistency and accuracy in the codebase. Signed-off-by: sammy <sammyoina@gmail.com> * Enhance notification payload and endpoint Extended the notification system to include 'status' and 'details' in the payload, improving traceability and debugging. Adapted the serialized JSON structure for clarity and added an 'originator' field to track the source service. Transitioned to a generalized event endpoint, facilitating a more streamlined event handling process. Refactors POST request to a more appropriate endpoint and updates the notification service interface to reflect new payload requirements. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor event notification logic Removed the legacy notifications package and consolidated event notification functionality using the new internal events service. Modified agent, manager, and main application code to use this service for consistent event reporting and error handling workflows across services. This change simplifies event management, improves error visibility, and allows for more maintainable code by centralizing event-related logic. The substitution of verbose state-specific publishEvent calls with generic status reporting aligns with the new service's capabilities. Signed-off-by: SammyOina <sammyoina@gmail.com> * Expand agent protobuf message types and improve error handling The protobuf definition for agent messages has been updated to include an additional message type, facilitating future data structure expansions. Additionally, error handling for event sending in the main agent execution has been enhanced to log errors when sending 'init' events fail, ensuring issues are properly tracked. The unused `notificationTopic` constant in the manager service has been removed for cleaner code maintenance. Signed-off-by: SammyOina <sammyoina@gmail.com> * Optimize JSON parsing and fix header omission Removed unnecessary unquoting of a JSON string before unmarshaling, streamlining the computation value extraction process. Also corrected a missing Content-Type header in the event sending function, ensuring proper handling of JSON requests by recipients. These changes improve performance and communication reliability. Signed-off-by: SammyOina <sammyoina@gmail.com> * align vars Signed-off-by: SammyOina <sammyoina@gmail.com> * Remove computation request timeout feature The timeout feature for computation requests has been removed to simplify the computation execution flow. This involved changes across multiple files, including protobuf definitions, HTTP endpoint handling, and the internal computation logic. We eliminated the timeout field, associated logic, and error handling to ensure the system no longer supports timeouts for computations, mitigating any unintended timeout impacts on long-running processes. Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: sammy <sammyoina@gmail.com> Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
a3c5c765b8 |
NOISSUE - Connect to agent when vm is created (#33)
* Add gRPC TLS config and update protoc versions Enhanced manager service with TLS configuration options and timeout settings for gRPC communication with agent services. Updated corresponding protobuf definitions to include new fields for CA certificates, TLS status, and timeouts. The added TLS support ensures secure inter-service communication while flexible timeout configurations improve the robustness of network interactions. The following adjustments and additions were made: - Included new gRPC configuration fields (ca_certs, client_tls, timeout) for agent communication in protobuf definitions. - Injected agent gRPC configuration into service endpoints, ensuring secure TLS setup and compliance with provided settings. - Revised main service function signatures to accept the gRPC configuration object. - Incremented port forwarding counters post-computation to avoid port conflicts. - Conducted compatibility update of protobuf version comments to reflect minor version bump (v4.25.0 to v4.25.1). This change impacts service deployment that requires proper configuration of TLS credentials and mindful determination of timeouts for efficient network use. Signed-off-by: SammyOina <sammyoina@gmail.com> * update proto Signed-off-by: SammyOina <sammyoina@gmail.com> * Set default timeout for agent configuration Introduced a default timeout of 60 seconds in both gRPC and HTTP endpoints for agent configuration when none is specified. This change ensures that operations do not hang indefinitely and provides a reasonable default for client interactions. Additionally, the instantiation of a new agent client is now outside the retry loop to avoid repeated setup on transient failures. Refactors service logic to optimize client connection handling by moving the agent client setup to occur before attempting retries, which should reduce overhead and improve clarity in error situations. Signed-off-by: SammyOina <sammyoina@gmail.com> * Subject: Move agent client creation into retry loop Body: Refactored the service manager's Run method to initialize the agent gRPC client inside the exponential backoff retry loop. This change addresses intermittent connection issues by reattempting client creation on temporary network failures, ensuring a robust setup before calling the Run method on the client. Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
5adf0349a3 |
COCOS-9 - Dockerize manager (#18)
* Fix bug in agent service and state The commit fixes a bug in the agent service and state files. Previously, the condition to check the state in the agent service was incorrect. It was checking the state directly instead of using the GetState() method. This has been fixed by using the GetState() method to check the state. Additionally, a new GetState() method has been added to the StateMachine struct in the state file. This method retrieves the current state by acquiring a lock and returning the state value. The changes have been tested and verified to resolve the bug and improve the accuracy of state checking in the agent service. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix bug in agent state machine The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix race condition in state machine The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix race condition in StateMachine.Start() The StateMachine.Start() method was experiencing a race condition when multiple events were being processed concurrently. This was caused by not properly locking and unlocking the state machine before and after updating the state. This commit fixes the issue by adding proper locking and unlocking around the state update operation. Additionally, the logging statement has been updated to include the previous and next states for better debugging. Signed-off-by: SammyOina <sammyoina@gmail.com> * add magistrala dep Signed-off-by: SammyOina <sammyoina@gmail.com> * remove mainflux Signed-off-by: SammyOina <sammyoina@gmail.com> * Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger. The commit message should be: "Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger" Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE The Makefile has been updated to fix an issue with setting the DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE variables. The USER_REPO variable is now used to generate the DOCKER_PROJECT name following the Docker Compose guidelines. Additionally, the COCOS_MESSAGE_BROKER_TYPE variable is now properly set to "nats" if it is empty. This ensures that the correct values are used when compiling and installing the service. Summary: Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE Details: - Update USER_REPO variable to generate DOCKER_PROJECT name - Set COCOS_MESSAGE_BROKER_TYPE to "nats" if empty Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix notification topic in agent service and update NATS ports in Docker environment variables The agent service's notification topic was incorrectly set to "channels.manager" instead of "agent". This commit fixes the issue by updating the notification topic. Additionally, the NATS ports in the Docker environment variables were incorrect. The COCOS_NATS_PORT and COCOS_NATS_HTTP_PORT have been updated to the correct values. These changes ensure that the agent service uses the correct notification topic and the NATS ports are properly configured. Signed-off-by: SammyOina <sammyoina@gmail.com> * Add Dockerfiles for production and development environments This commit adds two new Dockerfiles, one for the production environment and one for the development environment. The production Dockerfile sets up the necessary dependencies and builds the application, while the development Dockerfile simply copies the built application. Both Dockerfiles include the necessary SSL certificates for the application to work properly. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor Makefile and add new targets for building Docker images The Makefile has been refactored to include new targets for building Docker images. The `make_docker` and `make_docker_dev` functions have been defined to handle the Docker build process. The `dockers` and `dockers_dev` targets have been added to build the Docker images for all services and development environments respectively. This commit introduces changes to the Makefile to improve the build process and provide support for Docker images. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix makefile to generate Docker images for services - Added DOCKERS and DOCKERS_DEV variables to generate Docker images for services - Updated the dockers target to include DOCKERS - Updated the dockers_dev target to include DOCKERS_DEV - Updated the $(DOCKERS) and $(DOCKERS_DEV) targets to call the respective make_docker and make_docker_dev functions Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor Makefile and Dockerfile The Makefile has been refactored to remove unnecessary sed commands and improve code readability. The changes include: - Removed sed commands related to changing the broker in docker-compose.yml for nats and rabbitmq profiles. - Removed the eer target from the Makefile. The Dockerfile has been modified to include the WORKDIR directive. These changes improve the maintainability and readability of the codebase. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix NATS URL in .env file and add Manager configuration in docker-compose.yml The NATS URL in the .env file was updated to use the correct hostname for the broker. Additionally, the Manager configuration was added to the docker-compose.yml file. This commit fixes the NATS URL in the .env file and adds the necessary Manager configuration to the docker-compose.yml file. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix incorrect host and port values in Makefile and .env The commit updates the host and port values in the Makefile and .env files to fix incorrect values. The host and port values are updated to "cocos-manager" and "7003" respectively. This ensures that the correct host and port are used for the Manager HTTP and gRPC services. This commit resolves the issue with the incorrect host and port values and ensures that the Manager services are configured correctly. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix Dockerfile and docker-compose.yml configurations The Dockerfile and docker-compose.yml configurations have been fixed to address the following issues: - Removed unnecessary COPY commands in the Dockerfile. - Removed the "privileged" flag in the docker-compose.yml file. These changes ensure that the Docker image is built correctly and the container is launched with the appropriate configurations. Signed-off-by: SammyOina <sammyoina@gmail.com> * remove manual assignment Signed-off-by: SammyOina <sammyoina@gmail.com> * update protoc CI Signed-off-by: SammyOina <sammyoina@gmail.com> * add paths Signed-off-by: SammyOina <sammyoina@gmail.com> * remove profiles Signed-off-by: SammyOina <sammyoina@gmail.com> * restore default config Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix Docker project name in Makefile Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
|
Sammy Kerata Oina
|
4ea9ff6531 |
NOISSUE - Add message broker on agent and manager (#17)
* Fix bug in agent state machine The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix race condition in state machine The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly. Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix race condition in StateMachine.Start() The StateMachine.Start() method was experiencing a race condition when multiple events were being processed concurrently. This was caused by not properly locking and unlocking the state machine before and after updating the state. This commit fixes the issue by adding proper locking and unlocking around the state update operation. Additionally, the logging statement has been updated to include the previous and next states for better debugging. Signed-off-by: SammyOina <sammyoina@gmail.com> * add magistrala dep Signed-off-by: SammyOina <sammyoina@gmail.com> * remove mainflux Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix agentService New function to include messaging.Publisher parameter The agentService New function has been updated to include a messaging.Publisher parameter. This change allows the agent service to publish messages to a messaging system. The messaging.Publisher parameter has been added to the agentService struct and the New function signature has been updated accordingly. This change ensures that the agent service can communicate with other components using the messaging system. Signed-off-by: SammyOina <sammyoina@gmail.com> * Refactor service.go state functions The commit refactors the state functions in the service.go file. The functions for each state have been modified to use the svc.publishEvent method to publish events with appropriate messages. - Refactor state functions in service.go - Use svc.publishEvent to publish events with messages for each state Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix computation run event publishing and add pubsub functionality The computation run event publishing in the agent service was fixed to correctly call the publishEvent function. Additionally, the pubsub functionality was added to the manager package. - Fixed computation run event publishing in agent service - Added pubsub functionality to manager package Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix license header in pubsub.go file The commit fixes the license header in the pubsub.go file. The copyright and SPDX-License-Identifier have been added to comply with the Apache-2.0 license. Signed-off-by: SammyOina <sammyoina@gmail.com> * Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger. The commit message should be: "Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger" Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE The Makefile has been updated to fix an issue with setting the DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE variables. The USER_REPO variable is now used to generate the DOCKER_PROJECT name following the Docker Compose guidelines. Additionally, the COCOS_MESSAGE_BROKER_TYPE variable is now properly set to "nats" if it is empty. This ensures that the correct values are used when compiling and installing the service. Summary: Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE Details: - Update USER_REPO variable to generate DOCKER_PROJECT name - Set COCOS_MESSAGE_BROKER_TYPE to "nats" if empty Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix Makefile Docker profile assignment and build flags The Makefile was updated to fix the assignment of the Docker profile and build flags. The Docker profile is now assigned based on the value of COCOS_MESSAGE_BROKER_TYPE, and if it is not provided, the default value is set to "nats". The build flags were also updated to include the COCOS_MESSAGE_BROKER_TYPE value as a tag for the Go build process. This commit addresses the issue with the Docker profile assignment and ensures that the correct build flags are used during the build process. Signed-off-by: SammyOina <sammyoina@gmail.com> * fix makefile Signed-off-by: SammyOina <sammyoina@gmail.com> * Fix notification topic in agent service and update NATS ports in Docker environment variables The agent service's notification topic was incorrectly set to "channels.manager" instead of "agent". This commit fixes the issue by updating the notification topic. Additionally, the NATS ports in the Docker environment variables were incorrect. The COCOS_NATS_PORT and COCOS_NATS_HTTP_PORT have been updated to the correct values. These changes ensure that the agent service uses the correct notification topic and the NATS ports are properly configured. Signed-off-by: SammyOina <sammyoina@gmail.com> * add pubsub Signed-off-by: SammyOina <sammyoina@gmail.com> * update protoc Signed-off-by: SammyOina <sammyoina@gmail.com> --------- Signed-off-by: SammyOina <sammyoina@gmail.com> |
||
 Darko Draskovic
|
eb230b5aa6 |
Merge pull request #40 from ultravioletrs/checkproto
Bump proto version in github actions |
||
|
Darko Draskovic
|
e54d2731cd |
Bump proto version in github actions
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com> |
||
 rodneyosodo
|
f73c829d27 |
Update ProtoC workflow
Signed-off-by: rodneyosodo <blackd0t@protonmail.com> |