* Add agent service tests
Signed-off-by: Jilks Smith <smithjilks@gmail.com>
* Update agent service tests
* Fix agent service tests
* Improve agent service test coverage
* Improve agent service test coverage
Signed-off-by: Jilks Smith <smithjilks@gmail.com>
* Fix tests
Signed-off-by: Jilks Smith <smithjilks@gmail.com>
* Refactor and improve coverage
Signed-off-by: Jilks Smith <smithjilks@gmail.com>
---------
Signed-off-by: Jilks Smith <smithjilks@gmail.com>
* feat(algorithm): Add wasm as an algo type
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* feat(algorithm): Use filesystem to store results
Move from unix socket for results storage to filesystem
* test: test new filesystem changes
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* refactor(files): rename resultFile to resultsFilePath
* feat(wasm-runtime): change from wasmtime to wasmedge
Wasmedge enables easier directory mapping to get results
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* feat(algorithm): send results as zipped directory
Create a new function to zip the results directory and send it back to the user
* fix(wasm): runtime argument
Fix the directory mapping for wasm runtime arguments
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* fix(errors): provide useful error message
* chore(gitignore): add results zip to gitignore
* feat(filesystem): Enable storing results on filesystem for python algos
* refactor: revert to upstream cocos repo
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* fix: remove AddDataset from algorithm interface
* fix: agent to handle results zipping
* test: test zipping directories
* refactor(agent): Handle file operations from agent
* test: run test inside eos
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* refactor(test): Document and test algos are running
Document steps on running the 2 python exampls and ensure they are running on eos
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* fix: remove witheDataset option
* test: test without dataset argument
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
---------
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* test(events): failing tests
* fix: Add datapath as optional parameter
If datapath is not provided if should work as assume dataset is empty
If an invalid i.e non existent path is given it should return an error
* feat(agent): Allow empty dataset
Allow running of algorithm with empty dataset since not all algorithms require datasets.
Allow state-machine transition from algo-received state to running state incase of no dataset provided
Fixes https://github.com/ultravioletrs/cocos/issues/168
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* chore(gitignore): Remove build artefacts
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* feat(algorithms): Add test algorithm for addition
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* refactor(addition): Modify addition algo to one file
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* fix(agent): move state transition to callback func
Move state transition from `receivingAlgorithm` to `running` to state call back function
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* feat(agent-event): Add `algoReceivedNoData` event
`algoReceivedNoData` is an event that is sent if we receive an algorithm and it should not have a dataset hence changes the state from `receivingAlgorithm` to `running`
* fix(agent-state): Change state depending on manifest
Change state from `receivingAlgorithm` to either `receivingData` if there is a dataset or `running` if there is no dataset provided
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
---------
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* refactor(env): remove internal env package
No need for this package since the parent package github.com/caarlos0/env does everything we need
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* refactor(jaeger): remove internal jaeger package
No need for this package since we can use magistrala exported package
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* refactor(metrics): remove internal metrics pkg
Use exported magistrala prometheus package
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* chore(dep): Update grpc and other dependencies
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* style(linter): remove enabled by default linters
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
---------
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* * fix(interceptors.go): change role from auth.DataProviderRole to auth.ConsumerRole in AuthenticateUser call
Signed-off-by: SammyOina <sammyoina@gmail.com>
* * test(interceptors_test.go): update role variable values in TestAuthUnaryInterceptor
* - Change role value from auth.DataProviderRole to auth.ConsumerRole in the "authorized result method" test case
* - Change role value from auth.DataProviderRole to auth.ConsumerRole in the "unauthorized result method" test case
* - Change role value from auth.DataProviderRole to auth.ConsumerRole in the "other method" test case
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Implement gRPC auth interceptors and update dependencies
Introduced gRPC authentication interceptors for unary and stream calls in the agent's API to enhance security. These interceptors verify signatures based on roles before handling requests, rejecting unauthorized ones. The authentication logic was refactored, adding relevant error handling.
Added mocks for authentication interfaces to facilitate testing and replaced direct errors with wrapped ones to provide additional context.
The Makefile now includes a version pin for the Mockery tool, ensuring consistent mock generation across environments.
Updated the gRPC server setup in `grpc.go` to utilize the new interceptors. This change aims to reinforce the API's access control by authenticating users according to their roles and is expected to prevent unauthorized access to sensitive endpoints.
Lastly, dependencies in go.mod and go.sum were updated, most notably the inclusion of testify for improved testing capabilities.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix linting errors
Signed-off-by: SammyOina <sammyoina@gmail.com>
* * chore(Makefile): remove unused variables and update 'mocks' target
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Update Go to 1.22 and enhance security features
- Upgraded the Go version in GitHub Actions workflows to 1.22.x for latest features and security patches.
- Added RSA public key field `UserKey` in `Dataset` and `Algorithm` to reinforce data integrity and encryption.
- Refactored `Result` method in `agentService` to use `containsID` for improved readability and potential performance benefits.
- Updated `grpcserver.New` and `internal/server/grpc` invocations to pass `agent.Service` by value in line with recommended Go practices.
- Introduced `grpc.StreamInterceptor` with no args in `Server.Start` which seems to be an initial step for future stream interceptor configuration.
These changes prepare for stronger data security measures, maintain compatibility with the latest Go features, and improve code quality regarding service struct usage. Potential follow-up is needed to configure the stream interceptor and to ensure the new RSA key field is appropriately utilized in data handling.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor auth system and protocol buffers
Enhanced the authentication system by adding context support and an improved user-role model. Implemented robust RSA public key verification for users and a restructured interceptor logic specific to stream types, streamlining the auth process. Updated protocol buffers and associated structures to accommodate user keys as byte slices, aligning with standard cryptographic practice. CLI commands for algorithms and datasets now require a private key file path argument for signing, strengthening security during interactions.
This comprehensive overhaul addresses security and efficiency considerations in the RPC framework and aligns with best practices for key handling. By streamlining and securing the user authentication process, the agent service's reliability is greatly improved, directly impacting the robustness of the entire computation pipeline.
- Refactored auth: added role-based user validation, context handling
- Reworked interceptors: separated stream types, fortified signature checks
- Updated protocol buffers: user public keys as byte slices for standard compatibility
- Enhanced CLI: introduced private key argument, ensuring secure algorithm and dataset submission
- Improved server and SDK contracts to align with auth changes
Related issues:
- Implements user roles and auth context [#103]
- CLI security enhancement for private key management
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Updated PEM decoding for key parsing in CLI and tests
Added `encoding/pem` to decode PEM blocks when parsing private and public keys across CLI commands and test computation scenarios, ensuring compatibility with key files. This enhances robustness in key handling by supporting PEM encoded keys. The update also includes registration of a new Keys command in the CLI.
Refactored code is now compliant with common key formats, addressing potential parsing issues.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix auth signature encoding and improve CLI usage example
The authentication system now decodes base64 strings before verifying signatures to align with the expected format. Additionally, the signature generation now encodes the output in base64, ensuring consistency across the auth process.
The CLI help message for the `result` command is enhanced by providing a usage example, making it more user-friendly and informative.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor containsID to handle dynamic fields
Updated the `containsID` function to accept a field name parameter, enabling dynamic field lookup within the reflection logic. This change facilitates the use of the function for various struct fields, improving code reusability and flexibility.
CLI command 'data' now requires an additional argument for the private key file path, outlined in the usage example update, reinforcing command clarity and user guidance.
Resolves issues with hardcoded field lookups and enhances CLI usability.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Remove extraneous newline in key generation log output
A redundant newline after the success message in the key generation command was removed to clean up log output formatting. This change ensures a more consistent and professional appearance of the CLI tool's messages.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Implemented auth service in gRPC startup
Added authentication services to the gRPC server initialization to enforce security measures. The gRPC server's New function now includes an `authSvc` parameter, requiring instantiation of the auth service before starting the server. Failure to create the auth service results in a fatal error, halting the process to avoid running without protection. Tests have been updated to include `nil` values for the auth service parameter to maintain their functionality without authentication.
Refactored `grpcserver.New` to accept the new auth service, and updated the main agent startup logic to create and inject the auth service. Added the auth middleware interceptors to the server options, which ensures that each gRPC call will undergo authentication.
This change is a step towards secure communication, and affected components should now consider the authentication requirement.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor config read logic and update agent setup
Improved the configuration reading in `cmd/agent/main.go` to handle larger payloads by reading data in chunks and checking for EOF, ensuring that all config data is captured even if it exceeds the initial buffer size. Enhanced the `test/manual/agent-config/main.go` to require additional command-line arguments, improving the setup process by explicitly requiring paths for data, algorithm, and public key as well as a boolean for attested TLS. Also updated the hashing method to SHA3 for the algorithm and data files, and included the hash and public keys as part of the agent, dataset, and result consumer configurations. These changes will make the agent setup more robust and provide better integrity checks for the involved files.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor run method to agentService
Moved the run function into agentService for better encapsulation and maintainability. This refactoring includes capturing both stdout and stderr during algorithm execution, enabling more informative debugging through enhanced logging. Consequentially, the run method now references members through the service instance, aligning with object-oriented best practices and improving code coherence.
Resolves issue with insufficient execution details when computations fail.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor computation data handling to use filepaths
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor error logging and ensure consistency
Replaced usage of the standard log package with a custom logger for error reporting to standardize error logging throughout the application. Additionally, introduced graceful shutdown by returning from the main function rather than forcing exit when failing to create auth service, aligning the application's error handling strategy.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor auth initialization and key file handling
Improved the readability and maintainability of the authentication service initialization by adding line breaks for logical separation. Also, standardized key filenames in the CLI key generation by introducing constants, enhancing code clarity and reducing the likelihood of file-naming errors.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor auth verification logic for improved security
Removed an extraneous line in the `verifySignature` function that was not necessary for the signature verification process. This change simplifies the code and improves readability.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor payload structures to simplify API
Removed the 'provider', 'id', 'consumer' fields from protocol buffers, gRPC services, and related functions across various files to streamline the data model and align with the new authentication system based on cryptographic verification rather than string identifiers. This results in more efficient data handling and a reduction in unnecessary payload data, while enhancing security by making entity validation strictly cryptographic.
The changes affect agent-SDK interactions, CLI tools, and related services, ensuring only the necessary data (algorithm/data bytes, user keys, and hashes) is transmitted and processed. Consequently, the core computation algorithm and dataset handlers now rely on indexes derived from context to associate data with respective manifest entries, thus maintaining the ability to link to specific computation manifests without relying on explicit IDs in the payload. Additionally, refactored authentication methods now enforce role-based security seamlessly through metadata.
This approach enhances privacy by avoiding transmission of potentially sensitive strings over the network and by ensuring that only internal indices, not globally interpretable identifiers, are used to process computations.
Aligned with the broader architectural goal of simplifying and securing the platform's core services, this change paves the way for upcoming revisions to the authentication scheme that will further consolidate role-based security and improve system integrity.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enhance CLI security with key paths
Removed the section on running computations from the CLI README as it may no longer be necessary or the functionality has been moved elsewhere. Required private key file paths for algorithm, dataset upload, and result retrieval commands to enhance security. This change associates each action with a specific identity, ensuring secure and traceable operations. Additionally, updated the manual test commands to reflect this new requirement.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix ci
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix fmt
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
Changed the internal representation of algorithms and datasets within the service from byte slices to file paths, writing received data directly to temp files. This modification allows for handling potentially large data sets without the need to load them entirely into memory, improving the memory efficiency and scalability of the service. Additionally, it aligns the call signature of external algorithms with the new approach, updating documentation and examples accordingly. Updated the linear regression example for consistency with the new data handling process.
Resolves issues with memory bloat when processing large datasets.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enable streaming RPCs for Algo and Data services
Modified the gRPC service definitions for Algo and Data methods to use stream processing, enabling the handling of larger datasets and algorithms without being limited by memory restrictions. This allows client and server to send chunks of data sequentially rather than requiring the entire payload to be loaded into memory at once.
Updated server implementations to accumulate data from multiple chunks, allowing for more efficient processing and communication when dealing with large files. Client implementations have been adjusted to segment and send data in a streaming fashion.
Removed previously existing synchronous client code as it became redundant with the new streaming approach, streamlining the client's communication patterns with the gRPC backend.
This change allows for better resource management, especially in systems with constraints on memory, improving overall scalability and performance of the data and algorithm processing pipeline.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor algorithm ID check logic
Simplify the algorithm validation logic in the agent service by replacing the previous containment check with direct ID comparison. This change streamlines the error handling for undeclared algorithms and hash mismatches, while also ensuring clear and direct provider validation. The modifications enhance the readability and maintainability of the code without altering functionality.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Updated README to build single-file executable with PyInstaller
Modified the PyInstaller command in the manual testing README to bundle the linear regression script into a single executable file. This simplifies distribution and execution of the script by eliminating the need for multiple dependency files.
Ref: Optimization of deployment process
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Switch to self-contained algorithms as executables
Transitioned from using Python scripts to self-contained binary executables for running algorithms, improving modularity and reducing dependencies. This change removes the reliance on a Python environment, as evident by the removal of Python setup and packages from the build configuration. The service now creates temporary executable files for algorithm runs, handling all permissions and cleanup, enhancing security and maintaining clean execution states. A warning is logged if computation fails, aiding in debugging. Additionally, updated manual tests to reflect these changes in the agent's handling of algorithms.
Refactors:
- Removed Python runtime const since it's no longer needed.
- Updated documentation and test commands to reflect the change from .py to .bin for algorithm files.
Build config:
- Removed Python and pip packages to reduce the build size and complexity.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Update agent service.go file with new constants and file permission
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refine singular usage of 'algorithm' across modules
Standardized terminology throughout the project to refer to 'algorithm' in the singular form rather than plural. Streamlined various documentations, string constants, function names, and variable names to bring cohesiveness and eliminate ambiguity when handling algorithms across README files, CLI interfaces, and internal API representations.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix state names and indices in state_string.go
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor single algorithm processing
Simplified the agent service's algorithm handling logic to process a single algorithm instead of multiple. This change:
- Removed the `Algorithms` type and associated stringer implementation.
- Updated the state machine and service logic to expect a singular algorithm, aligning the agent's internal state transitions with the new model.
- Adjusted the manager service and computations test server to mirror these changes in their respective payload structures, ensuring API and test consistency.
- Altered README files to reflect the simplified interaction model and removed outdated descriptions.
- Reverted the protoc-gen-go version used for generating protobuf files to maintain compatibility with the rest of the codebase.
The single-algorithm approach streamlines the computation running process, reducing complexity and potential error conditions. It directly impacts how external services will construct and send computation requests.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Update protoc-gen-go version to v1.33.0
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor variable name in computations.go and grpc.go
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* added initial code for attested TLS
* added client validation and verification
* fixed bugs for attested TLS
* updated README for manual testing
* fixed CI errors
* removed SNP pollicy from agent config
* added attested TLS config param to AgentConfig
* generated manager.pb.go for protoc v25.2
* updated proto-gen-go version on CI
* generated agent.pb.go to match newest proto gen version
* define errors for error handling
* fixed comments
* Update protoc to v4.25.3 and streamline client identification
Protocol Buffer compiler `protoc` has been upgraded to version 4.25.3. This update unifies the version across multiple generated files to ensure compatibility and take advantage of any bug fixes and performance improvements in the new release.
Additionally, the client identification process has been refined. The redundant `WhoAmI` message and the corresponding checks have been removed from the gRPC server implementation. Clients are now identified via their address from the `Process` stream's context as soon as a connection is established, simplifying the code and potentially reducing handshake time.
This change sets the foundation for a leaner communication protocol between manager and agents, and could contribute to lower latencies in client-server interactions.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix goroutine bug in grpc server
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Update PROTOC_VERSION to 25.3
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor server reference in readme
Updated the deployment instructions in the README to point to the correct server location after consolidating test server documentation. Removed obsolete `manager-server` module and associated main entry point to align with the new architecture and streamline workflows.
Resolves issues with outdated links and cluttered repository structure.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enhanced state machine synchronization
Introduced a wait group to the state machine to manage go-routine synchronization effectively. The Start method now ensures that the state machine's execution is tracked, preventing potential early termination of go-routines during state transitions. The test suite is updated to wait for the go-routine to complete before proceeding, ensuring more reliable test execution.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Remove unnecessary code and fix test case in agent package
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Standardize attestation report data size to 64 bytes
Redefined the report data for attestation requests from a variable byte slice to a fixed 64-byte array across various critical components. This change enforces a standardized data length, simplifying validation logic and ensuring consistency when handling the attestation data for both the server and client end-points, logging functionalities, and the CLI tool. Updated attestation data handling in the SDK to accommodate the new fixed-length constraint.
By strictly adhering to the 64-byte requirement, the modifications promote robust input checking and prevent potential issues related to dynamic data length processing.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor attestation validation logic
Streamlined attestation validation in the CLI with a new configuration parsing approach. Introduced a retry mechanism with timeout and backoff for HTTPS getters that facilitate remote trust validation. Clarified expected data lengths in proto comments across attestation and manager services, ensuring consistency and correctness of the cryptographic data fields.
This modification enhances maintainability by replacing a verbose struct with a configurable object model, thus simplifying parameter handling. Robustness is improved through failure-retrieval strategies during remote validation, and additional in-code documentation specifies cryptographic constraints for critical data elements.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Add new attestation validation flags
Enhanced attestation validation in the CLI with additional flag support for FAMILY_ID, IMAGE_ID, REPORT_ID, REPORT_ID_MA, MEASUREMENT, CHIP_ID, and TCB-related fields. This update enables fine-grained control over attestation report criteria, ensuring stricter validation conforming to protocol requirements. Adjusted the default value for MinimumGuestSvn to align with the new policy specifications.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enhance attestation policy configuration
Introduced several new configuration flags to the CLI's attestation policy, including options for minimum AMD-SP firmware builds, revocation list checking, and network restrictions. Expanded trust criteria parameters with the support for trusted author and identity key hashes. The default VMPL setting is now explicitly initialized. These changes improve security controls and offer more detailed attestation verification settings.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Simplify attestation validation process
Refactored the attestation CLI command to improve usability by removing the need for explicitly provided report data as an argument. Default values for configuration parameters have been centralized into constants for maintainability. Additionally, integrated data parsing functions to streamline the validation checks with robust error handling for cases such as invalid hex strings and file I/O issues.
This enhancement reduces user error and the complexity of entering attestation data by parsing relevant information from within the environment, all while maintaining the same security standards.
The process of marking flags as required for CLI commands has been corrected to prevent runtime errors and improve command reliability.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refined attestation validation logic
Enhanced attestation command handling by removing the hardcoded report data length and replacing it with the standard SHA-512 hash size, ensuring dynamic compatibility with hash lengths. Introduced size constants to improve code readability and enforce explicit length checks on attestation input data, raising errors when requirements are unmet. This preventative measure aims to avert potential runtime errors stemming from unexpected data sizes. The change also simplifies the minimum guest SVN description for clarity.
References to removed unnecessary whitespace maintain code cleanliness. These adjustments culminate in a more robust and maintainable attestation process, in accordance with the best practices for secure handling of cryptographic data and compliance with AMD's ABI format specifications.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enhanced attestation documentation and command syntax
Updated CLI documentation to include detailed descriptions of the attestation retrieval and validation process, along with a comprehensive list of new flags for the validation command, providing users with extensive control over the attestation verification process. Updated command syntax with flags for specifying report data, improving command clarity and consistency. These changes ensure better user guidance and offer a more robust attestation handling experience.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enforce report data size validation uniformly
Refactored report data size validation to ensure it is performed consistently across the system. Removed the hard-coded length check from attestation request validation in favor of a centralized verification based on the sha512 digest size. Also, eliminated fallback mechanism to the SHA-512 hash of report data and made the length requirement explicit, leading to immediate failure if not met. This harmonizes the error handling logic, reinforces data integrity checks, and simplifies debugging by removing ambiguous length correction behavior. Adjusted related validation functions to use a common utility, enhancing maintainability.
Fix spelling errors in log messages for attestation validation commands to improve clarity in output.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Initialize attestation config with defaults
Enhanced the attestation command initialization by setting default values for the `Config` struct to prevent nil pointer exceptions and potentially streamline configuration handling. Moreover, corrected a flag declaration by removing shorthand.P support for 'permit_provisional_software', aligning it with other flags for consistency. This change should improve stability and user experience while configuring attestation policy.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* nil check prior
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix typos
Signed-off-by: SammyOina <sammyoina@gmail.com>
* use file for attestation report
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refine attestation validation with default policy values
Adjusted default values and constants related to the attestation configuration for enhanced clarity and consistency. Changed the default minimum guest SVN value to align with new policy requirements and introduced a default guest policy constant. Additionally, standardized byte array placeholders for attestation validation fields to prevent potential issues with uninitialized bytes and ensure robust input validation checks. These changes promote more intuitive and secure attestation validation procedures.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refine attestation defaults and flag requirements
Updated the default guest policy to use hexadecimal notation for clarity and added a default minimum version for the firmware API version flag. Made 'chip_id', 'measurement', 'report_id', and 'report_id_ma' flags mandatory to ensure critical attestation data is provided by the user, enhancing validation robustness. Added a check for 'measurement' field length for consistency in input validation.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Optimize attestation file format
Standardized attestation file format across the codebase by moving the 'attestationFilePath' constant from the function scope to a higher scope within the package. Switched the file extension from .txt to .bin to reflect binary data storage, improving consistency and potential performance benefits from binary file handling.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Optimize default values for attestation validation
Removed unnecessary empty initializers and replaced hardcoded empty values with nil for optional fields in attestation validation flags. Introduced a constant defaultReportIdMa for the REPORT_ID_MA field to provide a meaningful default instead of an empty one. Streamlined the command setup by removing the requirement flags for fields that are now optional and unchecked if unset, reducing burdens on the user for providing unnecessary input.
The change simplifies the command's usage and eliminates the need for users to specify values for certain fields that can be optional, improving the command's flexibility and user experience. Additionally, setting a specific default for REPORT_ID_MA ensures clarity in cases where this is not provided by the user.
This update enhances the maintainability and usability of the attestation validation functionality.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor hardcoded attestation report size
Introduced a constant `size64` to replace the previously hardcoded array size of `64` used across various Attestation methods. This change enhances code maintainability and readability by centralizing the size definition, making it easier to update in the future if necessary.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Updated protocol buffer dependencies and cleaned CLI flags
Protobuf compiler versions were updated to v4.25.2 across various RPC service definitions for agent and manager packages, ensuring compatibility with the latest features and fixes. In the CLI attestation code, unnecessary flags (`DisallowNetwork` and `PermitProvisionalSoftware`) were removed to streamline user options, reflecting a more secure and user-focused configuration interface. Added an input validation check in `attestation.go` for CA bundle presence when a product name is set, enhancing the reliability of attestation validation. Also rectified a bug in `grpc.go` by fixing an incorrect variable assignment for reading the key file content, thereby preventing potential TLS-related errors.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Updated protoc version and adjusted proto file paths
Bumped the protocol compiler version to 25.2 to align with our dependency updates and ensure compatibility. Additionally, revised the proto file paths under the continuous integration setup, moving from 'manager/' to 'pkg/manager/' to reflect the recent directory restructuring. The changes ensure that proto file checks and comparisons are conducted in the correct file locations, preventing potential build and sync issues in future developments.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enhance hash size consistency in gRPC attestation
Aligned the hard-coded byte array size for `ReportData` with `sha512.Size` constant to ensure consistency and maintainability in gRPC attestation requests and responses. This change mitigates the risk of future errors if the SHA512 hash size standard is altered.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Optimize agent-config listener loop
Removed the goroutine wrapping the listener loop in 'agent-config' manual test to streamline and simplify the connection handling logic. This change ensures the loop runs on the main goroutine, improving the readability and maintainability of the code.
Refactoring assists in avoiding potential race conditions and makes the server's flow easier to understand for future debugging and development.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor report data size checks to use constant
Centralize the report data size definition by replacing various size checks and array declarations with a constant `ReportDataSize`. This streamlines code maintenance and ensures consistency across all instances where report data size is validated or used. The changes remove direct references to the hash function output size, decoupling the report data size from the hash function's characteristics and allowing easier updates if the data size requirement changes.
These modifications bolster the codebase's flexibility for potential adjustments in security protocols or data handling specifications.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor config JSON handling in attestation CLI
Introduced an `exampleJSONConfig` constant containing a sample configuration to improve code readability and maintainability. Previously, the example JSON configuration for the attestation CLI was an inline string, making the code cluttered and less maintainable. With this change, the example configuration is now stored as a constant, resulting in cleaner command flag setup and enhanced clarity. This constant is used in the command flag description to guide users when providing their custom configuration. Additionally, refactored the command flags related to the configuration by aligning and sorting them for better code organization.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Add hash verification for datasets and algorithms
Enhanced data integrity checking by incorporating hash fields in Dataset and Algorithm structures, and modified the corresponding service logic to validate these hashes during processing. The update includes SHA-3 for hash computation, replacing the former SHA-256 usage, and ensures the provided data matches the expected hash from the manifest to prevent processing malformed or corrupted data.
- Introduce `Hash` field to both Dataset and Algorithm structs to store the expected hash value.
- Implement SHA-3 hashing within service methods that process the data, ensuring consistency with newly added `Hash` fields.
- Add error handling for hash mismatches, preventing further processing and alerting to potential data integrity issues.
- Update Protocol Buffers serialization to accommodate the new hash fields for gRPC communication.
- Modify manager service's Run method to pass the hash information when creating agent configurations.
Go module dependencies were updated to include the new SHA-3 package and upgrade Go version to 1.21.6 for improved stability and compatibility.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Remove identifiers from protobuf and related code
The protobuf definitions and related service handling code have been revised to drop specific identifier fields (`AlgorithmID` and `DatasetID`) to simplify API responses and internal function signatures. These removals streamline the overall data flow between components, reduce unnecessary data transmission, and lead to an aligned server-client expectation where identifiers are no longer a part of the response payload. Consequently, these changes simplify the logic within various functions and client commands, reinforcing encapsulation by ensuring that internal identifiers do not need to be managed or exposed unnecessarily.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix lint
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add handler
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor gRPC and Protobuf integration for manager service
- Shifted Protobuf message definitions to a separate package `pkg/manager`.
- Updated references throughout the codebase to import and use the new package for gRPC service definitions.
- Enhanced AgentLog message with additional fields `level` and `timestamp`.
- Removed direct dependencies on old Protobuf-generated types in favor of the new package.
- Deleted obsolete Protobuf-generated files as they are now superseded by the new `pkg/manager`.
- Streamlined event publishing and gRPC handling in the manager service to use the updated Protobuf messages.
This refactoring improves modularity by centralizing Protobuf message definitions and decouples internal representation from the gRPC interface, aligning with best practices for microservice architecture. Additionally, the enriched logging structure paves the way for more detailed and fine-grained log analysis.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor vsock event/log handling and config
Streamlined event and log services in the manager by moving vsock listening functions out of `managerService` initialization and into dedicated `RetrieveAgentEventsLogs` methods. This change decouples the manager service creation from the actual start of log listening, adding clarity and flexibility in service management. Also moved logging middleware invocation outside of network handling loops to avoid unnecessary overhead.
Additionally, the agent's vsock port configuration is now dynamically passed to the `New` function in the `events` package instead of relying on a hardcoded constant, allowing for greater configurability and testability.
Finally, updated message structures for event and log sending to conform with the `ClientStreamMessage` definitions. These modifications should improve parsing and handling consistency and prepare our system for future enhancements related to inter-process communication.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix linting errors
Signed-off-by: SammyOina <sammyoina@gmail.com>
* correct path to generated files
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix comments
Signed-off-by: SammyOina <sammyoina@gmail.com>
* remove uneccessary comments
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Implemented mTLS support across services
Extended gRPC configuration to support mutual TLS (mTLS) in agent and manager components for enhanced security. This includes the loading of Certificate Authority (CA) certificates, server, and client certificates, and keys. Updated README documentation to reflect the new environment variables required for mTLS configuration. Additionally, streamlined secure gRPC client connection setup and logging messages to indicate whether a service is running with TLS, mTLS, or without TLS.
The change ensures secure communication between services by verifying both client and server identities, thus addressing potential security concerns in network-level interactions.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enhance agent cert handling and update copyright
- Implement function to create certificate files for the agent configuration dynamically, ensuring file paths are updated to reflect newly created files. This improves the agent's setup process by automating the certificate handling.
- Update copyright clause to reflect the new owning entity, Ultraviolet, affirming correct attribution and compliance with legal requirements.
- Refactor gRPC client connection code to remove redundant package alias, streamlining the codebase and improving readability.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor cert loading with fallbacks
Removed redundant certificate file creation logic in the agent module and introduced a more robust loading mechanism in the gRPC server module to support direct byte content aside from file paths. This change simplifies the initial setup process for the agent by removing the need to create certificate files preemptively, thereby streamlining deployment in environments with varying filesystem access. It supports using certificate contents directly, enhancing compatibility with in-memory configurations or environments where file storage may not be ideal.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix lint
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor GRPC manager service and client
The manager service and client have been restructured for stream communication, facilitating real-time agent events, logs, and run responses. The `Run` RPC is replaced by the `Process` stream RPC, enabling bidirectional streaming between clients and the manager service. This allows continuous interchange of different message types including `WhoAmIRequest`, `AgentLog`, `AgentEvent`, and `RunResponse`.
Several message types have been adjusted and new fields introduced, like `AgentPort` in `RunResponse` and various agent-config attributes including CA files and instance IDs, to support TLS client authentication and distinguish between agent instances.
We've also incorporated `google.protobuf.Timestamp` in `AgentEvent` for precise event logging. The client code reflects these modifications with updated method calls and stream handling logic for ongoing communication. Moreover, the updates necessitate corresponding changes throughout service, grpc, and sdk layers to interoperate with the new streaming approach.
The transition to streaming paves the way for a more interactive, flexible communication system that can accommodate future expansion and real-time monitoring features.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix lint
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Update GitHub Actions to Latest Versions
Upgraded GitHub Actions 'checkout' to version 4 and 'setup-go' to version 5 across various workflow files to leverage the latest features and improvements for better performance and reliability. This also ensures compatibility with Go version 1.21.x which is specified in the workflows.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor event handling and logging
Reworked event and log processing to use channels instead of direct HTTP calls. Removed obsolete events package and consolidated event structures, leading to cleaner and more maintainable code. Updated agent events to use channels, enhanced error handling in log forwarding, and simplified manager `New` function signature to accept an event channel directly.
- Removed `events` and `agentevents` packages to reduce complexity.
- Replaced direct event server communication with internal channel usage.
- Introduced `AgentEvent` struct in events.go for standardized event objects.
- Adapted `managerService` to dispatch events and logs through channels.
- Streamlined manager construction by removing the now-unnecessary event service and host IP parameters.
This change results in a more robust and easier to extend event and log management system within the agent-manager interaction.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix ci
Signed-off-by: SammyOina <sammyoina@gmail.com>
* remove unused code
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add comments
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
The HTTP server-related code, documentation, and configurations have been removed as part of a shift towards prioritizing gRPC for service communication. This update includes deletions of HTTP host and port configs across various components, the manager HTTP API alongside its Swagger definition, and the removal of related scaffolding and utility code. This change simplifies the overall architecture and eliminates redundant HTTP support, focusing on optimizing gRPC performance and security features.
Signed-off-by: SammyOina <sammyoina@gmail.com>
Sammy Kerata Oina
Smith Jilks
Washington Kigani Kamadi
b1ackd0t
Danko Miladinovic