* feat: Implement per-resource KBS configuration, allowing algorithms and datasets to specify individual KBS URLs.
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* refactor: Encapsulate CLI error handling and CVM certificate paths within the CLI struct, and add algorithm type to agent's algorithm structure.
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* style: Remove blank lines and fix indentation in CLI commands.
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* refactor: Update downloadAndDecryptGenericResource to accept KBS URL as a parameter and adjust related tests
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* refactor: group CLI configuration into structured types and simplify skopeo decryption key handling
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
---------
Signed-off-by: Sammy Oina <sammyoina@gmail.com>
* feat(algorithm): Add wasm as an algo type
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* feat(algorithm): Use filesystem to store results
Move from unix socket for results storage to filesystem
* test: test new filesystem changes
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* refactor(files): rename resultFile to resultsFilePath
* feat(wasm-runtime): change from wasmtime to wasmedge
Wasmedge enables easier directory mapping to get results
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* feat(algorithm): send results as zipped directory
Create a new function to zip the results directory and send it back to the user
* fix(wasm): runtime argument
Fix the directory mapping for wasm runtime arguments
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* fix(errors): provide useful error message
* chore(gitignore): add results zip to gitignore
* feat(filesystem): Enable storing results on filesystem for python algos
* refactor: revert to upstream cocos repo
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* fix: remove AddDataset from algorithm interface
* fix: agent to handle results zipping
* test: test zipping directories
* refactor(agent): Handle file operations from agent
* test: run test inside eos
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* refactor(test): Document and test algos are running
Document steps on running the 2 python exampls and ensure they are running on eos
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* fix: remove witheDataset option
* test: test without dataset argument
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
---------
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* Update Go to 1.22 and enhance security features
- Upgraded the Go version in GitHub Actions workflows to 1.22.x for latest features and security patches.
- Added RSA public key field `UserKey` in `Dataset` and `Algorithm` to reinforce data integrity and encryption.
- Refactored `Result` method in `agentService` to use `containsID` for improved readability and potential performance benefits.
- Updated `grpcserver.New` and `internal/server/grpc` invocations to pass `agent.Service` by value in line with recommended Go practices.
- Introduced `grpc.StreamInterceptor` with no args in `Server.Start` which seems to be an initial step for future stream interceptor configuration.
These changes prepare for stronger data security measures, maintain compatibility with the latest Go features, and improve code quality regarding service struct usage. Potential follow-up is needed to configure the stream interceptor and to ensure the new RSA key field is appropriately utilized in data handling.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor auth system and protocol buffers
Enhanced the authentication system by adding context support and an improved user-role model. Implemented robust RSA public key verification for users and a restructured interceptor logic specific to stream types, streamlining the auth process. Updated protocol buffers and associated structures to accommodate user keys as byte slices, aligning with standard cryptographic practice. CLI commands for algorithms and datasets now require a private key file path argument for signing, strengthening security during interactions.
This comprehensive overhaul addresses security and efficiency considerations in the RPC framework and aligns with best practices for key handling. By streamlining and securing the user authentication process, the agent service's reliability is greatly improved, directly impacting the robustness of the entire computation pipeline.
- Refactored auth: added role-based user validation, context handling
- Reworked interceptors: separated stream types, fortified signature checks
- Updated protocol buffers: user public keys as byte slices for standard compatibility
- Enhanced CLI: introduced private key argument, ensuring secure algorithm and dataset submission
- Improved server and SDK contracts to align with auth changes
Related issues:
- Implements user roles and auth context [#103]
- CLI security enhancement for private key management
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Updated PEM decoding for key parsing in CLI and tests
Added `encoding/pem` to decode PEM blocks when parsing private and public keys across CLI commands and test computation scenarios, ensuring compatibility with key files. This enhances robustness in key handling by supporting PEM encoded keys. The update also includes registration of a new Keys command in the CLI.
Refactored code is now compliant with common key formats, addressing potential parsing issues.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix auth signature encoding and improve CLI usage example
The authentication system now decodes base64 strings before verifying signatures to align with the expected format. Additionally, the signature generation now encodes the output in base64, ensuring consistency across the auth process.
The CLI help message for the `result` command is enhanced by providing a usage example, making it more user-friendly and informative.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor containsID to handle dynamic fields
Updated the `containsID` function to accept a field name parameter, enabling dynamic field lookup within the reflection logic. This change facilitates the use of the function for various struct fields, improving code reusability and flexibility.
CLI command 'data' now requires an additional argument for the private key file path, outlined in the usage example update, reinforcing command clarity and user guidance.
Resolves issues with hardcoded field lookups and enhances CLI usability.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Remove extraneous newline in key generation log output
A redundant newline after the success message in the key generation command was removed to clean up log output formatting. This change ensures a more consistent and professional appearance of the CLI tool's messages.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Implemented auth service in gRPC startup
Added authentication services to the gRPC server initialization to enforce security measures. The gRPC server's New function now includes an `authSvc` parameter, requiring instantiation of the auth service before starting the server. Failure to create the auth service results in a fatal error, halting the process to avoid running without protection. Tests have been updated to include `nil` values for the auth service parameter to maintain their functionality without authentication.
Refactored `grpcserver.New` to accept the new auth service, and updated the main agent startup logic to create and inject the auth service. Added the auth middleware interceptors to the server options, which ensures that each gRPC call will undergo authentication.
This change is a step towards secure communication, and affected components should now consider the authentication requirement.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor config read logic and update agent setup
Improved the configuration reading in `cmd/agent/main.go` to handle larger payloads by reading data in chunks and checking for EOF, ensuring that all config data is captured even if it exceeds the initial buffer size. Enhanced the `test/manual/agent-config/main.go` to require additional command-line arguments, improving the setup process by explicitly requiring paths for data, algorithm, and public key as well as a boolean for attested TLS. Also updated the hashing method to SHA3 for the algorithm and data files, and included the hash and public keys as part of the agent, dataset, and result consumer configurations. These changes will make the agent setup more robust and provide better integrity checks for the involved files.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor run method to agentService
Moved the run function into agentService for better encapsulation and maintainability. This refactoring includes capturing both stdout and stderr during algorithm execution, enabling more informative debugging through enhanced logging. Consequentially, the run method now references members through the service instance, aligning with object-oriented best practices and improving code coherence.
Resolves issue with insufficient execution details when computations fail.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor computation data handling to use filepaths
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor error logging and ensure consistency
Replaced usage of the standard log package with a custom logger for error reporting to standardize error logging throughout the application. Additionally, introduced graceful shutdown by returning from the main function rather than forcing exit when failing to create auth service, aligning the application's error handling strategy.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor auth initialization and key file handling
Improved the readability and maintainability of the authentication service initialization by adding line breaks for logical separation. Also, standardized key filenames in the CLI key generation by introducing constants, enhancing code clarity and reducing the likelihood of file-naming errors.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor auth verification logic for improved security
Removed an extraneous line in the `verifySignature` function that was not necessary for the signature verification process. This change simplifies the code and improves readability.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor payload structures to simplify API
Removed the 'provider', 'id', 'consumer' fields from protocol buffers, gRPC services, and related functions across various files to streamline the data model and align with the new authentication system based on cryptographic verification rather than string identifiers. This results in more efficient data handling and a reduction in unnecessary payload data, while enhancing security by making entity validation strictly cryptographic.
The changes affect agent-SDK interactions, CLI tools, and related services, ensuring only the necessary data (algorithm/data bytes, user keys, and hashes) is transmitted and processed. Consequently, the core computation algorithm and dataset handlers now rely on indexes derived from context to associate data with respective manifest entries, thus maintaining the ability to link to specific computation manifests without relying on explicit IDs in the payload. Additionally, refactored authentication methods now enforce role-based security seamlessly through metadata.
This approach enhances privacy by avoiding transmission of potentially sensitive strings over the network and by ensuring that only internal indices, not globally interpretable identifiers, are used to process computations.
Aligned with the broader architectural goal of simplifying and securing the platform's core services, this change paves the way for upcoming revisions to the authentication scheme that will further consolidate role-based security and improve system integrity.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enhance CLI security with key paths
Removed the section on running computations from the CLI README as it may no longer be necessary or the functionality has been moved elsewhere. Required private key file paths for algorithm, dataset upload, and result retrieval commands to enhance security. This change associates each action with a specific identity, ensuring secure and traceable operations. Additionally, updated the manual test commands to reflect this new requirement.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix ci
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix fmt
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor CLI commands to use CLI struct
Introduced a CLI struct to encapsulate dependencies such as agentSDK and managerSDK previously passed to command constructors. This shift towards a more object-oriented approach simplifies command creation and enhances code readability. CLI-related commands are now methods on the CLI struct, accessing shared services directly, which streamlines the setup of CLI command hierarchies and reduces the need to pass dependencies around.
Removed the OpenAPI specification file as it was likely deemed obsolete or no longer necessary to be included with the CLI binary, suggesting a possible shift in how the API is managed or documented.
Lastly, the main application now sets up two distinct command groups for 'agent' and 'manager', with respective commands organized under them, making the CLI tool's structure more intuitive for users.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix ci
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add stringer
Signed-off-by: SammyOina <sammyoina@gmail.com>
* rename module to cocos
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix fmt.Stringer implementation in computations.go
The fmt.Stringer implementation for Datasets and Algorithms in computations.go was fixed to correctly use pointers.
This commit addresses the issue where the String() method for Datasets and Algorithms in computations.go was not correctly implemented. The fix ensures that the String() method now correctly marshals the data to JSON and returns the string representation.
The changes made in this commit will improve the functionality and accuracy of the String() method for Datasets and Algorithms.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add state machine
Signed-off-by: SammyOina <sammyoina@gmail.com>
* This commit addresses the following changes:
- In agent/service.go:
- Added a goroutine to start the StateMachine
- Sent the start event to the StateMachine
- In agent/state.go:
- Added a copyright notice and license identifier
These changes ensure that the StateMachine starts correctly and that the agent/state.go file includes the necessary copyright and license information.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix race condition in agent service initialization
The commit fixes a race condition in the agent service initialization.
Previously, the `New` function in `agent/service.go` would start the state machine
goroutine before initializing the `sm` field. This could lead to a race condition
where the state machine would access uninitialized fields.
To fix this, the `New` function now takes a `context.Context` argument and passes it
to the `Start` method of the state machine. Additionally, a `sync.Mutex` has been
added to the `StateMachine` struct to ensure thread safety.
This commit addresses the race condition and ensures proper initialization of the agent service.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix sending events when algorithms and datasets are received
This commit fixes a bug where events were not being sent when algorithms and datasets were received in the agent service. The bug was causing the events to not be triggered, leading to incorrect behavior. This commit adds the necessary code to send the events when algorithms and datasets are received, ensuring that the events are properly triggered.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix bug: computation results not ready
The computation results were not being marked as ready,
resulting in an error when trying to access them.
This commit fixes the issue by adding a check for
the computation results before returning them.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix bug in agent service and state
The commit fixes a bug in the agent service and state files.
Previously, the condition to check the state in the agent service
was incorrect. It was checking the state directly instead of
using the GetState() method. This has been fixed by using the
GetState() method to check the state.
Additionally, a new GetState() method has been added to the
StateMachine struct in the state file. This method retrieves
the current state by acquiring a lock and returning the state
value.
The changes have been tested and verified to resolve the bug
and improve the accuracy of state checking in the agent service.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* check for time outs
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix bug in agent state machine
The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix race condition in state machine
The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix race condition in StateMachine.Start()
The StateMachine.Start() method was experiencing a race condition
when multiple events were being processed concurrently. This was
caused by not properly locking and unlocking the state machine
before and after updating the state. This commit fixes the issue
by adding proper locking and unlocking around the state update
operation. Additionally, the logging statement has been updated
to include the previous and next states for better debugging.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
Sammy Kerata Oina
Smith Jilks
b1ackd0t
Washington Kigani Kamadi
Darko Draskovic