Compare commits

..

16 Commits

Author SHA1 Message Date
filip 7626cdda67 Covid19 added to test directory
Signed-off-by: filip <filipcivljak007@gmail.com>
2024-06-14 17:56:52 +02:00
Sammy Kerata Oina 4c4161ccfb Fix capitalized Vnc field and increment Vnc on run (#139)
Corrected the struct field 'vnc' to 'Vnc' to match Go's public field naming conventions and to make it exportable. Each QEMU virtual machine's VNC display number now increments upon instantiation to avoid display conflicts.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-06-06 16:34:13 +02:00
Sammy Kerata Oina ff64c10ac3 NOISSUE - Add new key management command to CLI (#136)
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-29 13:09:54 +02:00
Danko Miladinovic 0574abc228 NOISSUE - Add Rust script for fetching platform data (#133)
* add rust program for fetching platform data

* fix new line error

* add CLI options to add the measurement to platform_info.json file

* add documentation for platform info testing

* add explanation for sev-snp-measure

* delete excess space

* fix minor errors

* fix minor errors

* add file permision constant
2024-05-28 18:08:07 +02:00
Sammy Kerata Oina 2ce112cc1b COCOS-103 - User authN and AuthZ using digital signatures (#128)
* Update Go to 1.22 and enhance security features

- Upgraded the Go version in GitHub Actions workflows to 1.22.x for latest features and security patches.
- Added RSA public key field `UserKey` in `Dataset` and `Algorithm` to reinforce data integrity and encryption.
- Refactored `Result` method in `agentService` to use `containsID` for improved readability and potential performance benefits.
- Updated `grpcserver.New` and `internal/server/grpc` invocations to pass `agent.Service` by value in line with recommended Go practices.
- Introduced `grpc.StreamInterceptor` with no args in `Server.Start` which seems to be an initial step for future stream interceptor configuration.

These changes prepare for stronger data security measures, maintain compatibility with the latest Go features, and improve code quality regarding service struct usage. Potential follow-up is needed to configure the stream interceptor and to ensure the new RSA key field is appropriately utilized in data handling.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor auth system and protocol buffers

Enhanced the authentication system by adding context support and an improved user-role model. Implemented robust RSA public key verification for users and a restructured interceptor logic specific to stream types, streamlining the auth process. Updated protocol buffers and associated structures to accommodate user keys as byte slices, aligning with standard cryptographic practice. CLI commands for algorithms and datasets now require a private key file path argument for signing, strengthening security during interactions.

This comprehensive overhaul addresses security and efficiency considerations in the RPC framework and aligns with best practices for key handling. By streamlining and securing the user authentication process, the agent service's reliability is greatly improved, directly impacting the robustness of the entire computation pipeline.

- Refactored auth: added role-based user validation, context handling
- Reworked interceptors: separated stream types, fortified signature checks
- Updated protocol buffers: user public keys as byte slices for standard compatibility
- Enhanced CLI: introduced private key argument, ensuring secure algorithm and dataset submission
- Improved server and SDK contracts to align with auth changes

Related issues:
- Implements user roles and auth context [#103]
- CLI security enhancement for private key management

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Updated PEM decoding for key parsing in CLI and tests

Added `encoding/pem` to decode PEM blocks when parsing private and public keys across CLI commands and test computation scenarios, ensuring compatibility with key files. This enhances robustness in key handling by supporting PEM encoded keys. The update also includes registration of a new Keys command in the CLI.

Refactored code is now compliant with common key formats, addressing potential parsing issues.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix auth signature encoding and improve CLI usage example

The authentication system now decodes base64 strings before verifying signatures to align with the expected format. Additionally, the signature generation now encodes the output in base64, ensuring consistency across the auth process.

The CLI help message for the `result` command is enhanced by providing a usage example, making it more user-friendly and informative.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor containsID to handle dynamic fields

Updated the `containsID` function to accept a field name parameter, enabling dynamic field lookup within the reflection logic. This change facilitates the use of the function for various struct fields, improving code reusability and flexibility.

CLI command 'data' now requires an additional argument for the private key file path, outlined in the usage example update, reinforcing command clarity and user guidance.

Resolves issues with hardcoded field lookups and enhances CLI usability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Remove extraneous newline in key generation log output

A redundant newline after the success message in the key generation command was removed to clean up log output formatting. This change ensures a more consistent and professional appearance of the CLI tool's messages.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Implemented auth service in gRPC startup

Added authentication services to the gRPC server initialization to enforce security measures. The gRPC server's New function now includes an `authSvc` parameter, requiring instantiation of the auth service before starting the server. Failure to create the auth service results in a fatal error, halting the process to avoid running without protection. Tests have been updated to include `nil` values for the auth service parameter to maintain their functionality without authentication.

Refactored `grpcserver.New` to accept the new auth service, and updated the main agent startup logic to create and inject the auth service. Added the auth middleware interceptors to the server options, which ensures that each gRPC call will undergo authentication.

This change is a step towards secure communication, and affected components should now consider the authentication requirement.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor config read logic and update agent setup

Improved the configuration reading in `cmd/agent/main.go` to handle larger payloads by reading data in chunks and checking for EOF, ensuring that all config data is captured even if it exceeds the initial buffer size. Enhanced the `test/manual/agent-config/main.go` to require additional command-line arguments, improving the setup process by explicitly requiring paths for data, algorithm, and public key as well as a boolean for attested TLS. Also updated the hashing method to SHA3 for the algorithm and data files, and included the hash and public keys as part of the agent, dataset, and result consumer configurations. These changes will make the agent setup more robust and provide better integrity checks for the involved files.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor run method to agentService

Moved the run function into agentService for better encapsulation and maintainability. This refactoring includes capturing both stdout and stderr during algorithm execution, enabling more informative debugging through enhanced logging. Consequentially, the run method now references members through the service instance, aligning with object-oriented best practices and improving code coherence.

Resolves issue with insufficient execution details when computations fail.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor computation data handling to use filepaths

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor error logging and ensure consistency

Replaced usage of the standard log package with a custom logger for error reporting to standardize error logging throughout the application. Additionally, introduced graceful shutdown by returning from the main function rather than forcing exit when failing to create auth service, aligning the application's error handling strategy.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor auth initialization and key file handling

Improved the readability and maintainability of the authentication service initialization by adding line breaks for logical separation. Also, standardized key filenames in the CLI key generation by introducing constants, enhancing code clarity and reducing the likelihood of file-naming errors.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor auth verification logic for improved security

Removed an extraneous line in the `verifySignature` function that was not necessary for the signature verification process. This change simplifies the code and improves readability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor payload structures to simplify API

Removed the 'provider', 'id', 'consumer' fields from protocol buffers, gRPC services, and related functions across various files to streamline the data model and align with the new authentication system based on cryptographic verification rather than string identifiers. This results in more efficient data handling and a reduction in unnecessary payload data, while enhancing security by making entity validation strictly cryptographic.

The changes affect agent-SDK interactions, CLI tools, and related services, ensuring only the necessary data (algorithm/data bytes, user keys, and hashes) is transmitted and processed. Consequently, the core computation algorithm and dataset handlers now rely on indexes derived from context to associate data with respective manifest entries, thus maintaining the ability to link to specific computation manifests without relying on explicit IDs in the payload. Additionally, refactored authentication methods now enforce role-based security seamlessly through metadata.

This approach enhances privacy by avoiding transmission of potentially sensitive strings over the network and by ensuring that only internal indices, not globally interpretable identifiers, are used to process computations.

Aligned with the broader architectural goal of simplifying and securing the platform's core services, this change paves the way for upcoming revisions to the authentication scheme that will further consolidate role-based security and improve system integrity.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance CLI security with key paths

Removed the section on running computations from the CLI README as it may no longer be necessary or the functionality has been moved elsewhere. Required private key file paths for algorithm, dataset upload, and result retrieval commands to enhance security. This change associates each action with a specific identity, ensuring secure and traceable operations. Additionally, updated the manual test commands to reflect this new requirement.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix ci

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix fmt

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-28 14:10:13 +02:00
Sammy Kerata Oina aebe01a873 NOISSUE - Updated README with static binary instructions (#130)
Enhanced the manual algorithm testing documentation by including steps to install additional dependencies and guidance on creating static binaries. This ensures wider compatibility and ease of deployment for python programs tested with this framework.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-22 14:40:34 +02:00
Sammy Kerata Oina 4c681569f1 NOISSUE - Add sha3-256 file hashing command to CLI (#129)
Introduced a new command for the CLI to calculate the sha3-256 hash of a given file. The update enhances the utility's cryptography toolkit by allowing users to easily generate and verify file integrity directly through the command-line interface.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-20 16:56:42 +02:00
Sammy Kerata Oina c274521faf Refactor computation data handling to use filepaths (#126)
Changed the internal representation of algorithms and datasets within the service from byte slices to file paths, writing received data directly to temp files. This modification allows for handling potentially large data sets without the need to load them entirely into memory, improving the memory efficiency and scalability of the service. Additionally, it aligns the call signature of external algorithms with the new approach, updating documentation and examples accordingly. Updated the linear regression example for consistency with the new data handling process.

Resolves issues with memory bloat when processing large datasets.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-13 17:16:39 +02:00
Sammy Kerata Oina 226704cf0d COCOS-122 - Enable streaming RPCs for Algo and Data services (#123)
* Enable streaming RPCs for Algo and Data services

Modified the gRPC service definitions for Algo and Data methods to use stream processing, enabling the handling of larger datasets and algorithms without being limited by memory restrictions. This allows client and server to send chunks of data sequentially rather than requiring the entire payload to be loaded into memory at once.

Updated server implementations to accumulate data from multiple chunks, allowing for more efficient processing and communication when dealing with large files. Client implementations have been adjusted to segment and send data in a streaming fashion.

Removed previously existing synchronous client code as it became redundant with the new streaming approach, streamlining the client's communication patterns with the gRPC backend.

This change allows for better resource management, especially in systems with constraints on memory, improving overall scalability and performance of the data and algorithm processing pipeline.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor algorithm ID check logic

Simplify the algorithm validation logic in the agent service by replacing the previous containment check with direct ID comparison. This change streamlines the error handling for undeclared algorithms and hash mismatches, while also ensuring clear and direct provider validation. The modifications enhance the readability and maintainability of the code without altering functionality.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Updated README to build single-file executable with PyInstaller

Modified the PyInstaller command in the manual testing README to bundle the linear regression script into a single executable file. This simplifies distribution and execution of the script by eliminating the need for multiple dependency files.

Ref: Optimization of deployment process
Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-05-13 13:14:50 +02:00
Danko Miladinovic 479598a663 COCOS-116 -Separated networking setup from agent start up script (#118)
* added network setup script

* removed fixed name enp0s2 from code

* fixed small typo

* changed the capitalization of a letter
2024-05-01 17:03:28 +02:00
Sammy Kerata Oina 8d082567d7 COCOS-101 - Switch to self-contained algorithms as executables (#119)
* Switch to self-contained algorithms as executables

Transitioned from using Python scripts to self-contained binary executables for running algorithms, improving modularity and reducing dependencies. This change removes the reliance on a Python environment, as evident by the removal of Python setup and packages from the build configuration. The service now creates temporary executable files for algorithm runs, handling all permissions and cleanup, enhancing security and maintaining clean execution states. A warning is logged if computation fails, aiding in debugging. Additionally, updated manual tests to reflect these changes in the agent's handling of algorithms.

Refactors:
- Removed Python runtime const since it's no longer needed.
- Updated documentation and test commands to reflect the change from .py to .bin for algorithm files.

Build config:
- Removed Python and pip packages to reduce the build size and complexity.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update agent service.go file with new constants and file permission

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refine singular usage of 'algorithm' across modules

Standardized terminology throughout the project to refer to 'algorithm' in the singular form rather than plural. Streamlined various documentations, string constants, function names, and variable names to bring cohesiveness and eliminate ambiguity when handling algorithms across README files, CLI interfaces, and internal API representations.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix state names and indices in state_string.go

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-04-29 15:48:17 +02:00
Sammy Kerata Oina 4b5000d107 NOISSUE - Allow termination of manager (#95)
* Implement manager client heartbeat mechanism

Introduced a heartbeat system in the manager service to maintain an active link with client agents. The updates involve sending periodic heartbeat signals controlled via the new environment variable `MANAGER_HEARTBEAT_INTERVAL`, defaulting to 1 second. The protobuf definitions, service interfaces, and server logic have been updated accordingly to handle the incoming signals. This change allows better tracking of active client connections and could be used for features like auto-reconnect or resource cleanup for lost connections in the future.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance gRPC server to handle client termination

Introduce the capability for the gRPC server in the manager module to process termination requests from clients. Server and client proto buffers have been updated to include a Termination message, allowing clients to signal the server to stop processing. Handling for communication and termination flows involving context cancellation have been added across server implementation, manager client, and main execution logic to properly respond to termination signals. This ensures clean shutdowns and resource deallocation when a client requests termination, improving the robustness of the system.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor gRPC server context handling

Removed the redundant storage of context in grpcServer struct and modified the NewServer constructor to exclude the context parameter. Updated the Process method to derive context from the stream directly, ensuring better context scoping and adherence to proper gRPC patterns. This change simplifies the server's context management and ensures contexts are more accurately associated with their corresponding streams, which can improve debugging and request cancellation behavior.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update protoc to v4.25.3 and remove heartbeat system

Upgraded protoc version to 4.25.3, ensuring compatibility with the latest protocol buffer features and potential performance improvements. Additionally, removed the entire heartbeat system, involving both its gRPC server implementation and associated client-side logic. This elimination suggests a shift in the service health-check strategy, potentially to a more modern or efficient model.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Removed heartbeat functionality for manager service

The heartbeat feature and its related configuration have been removed across the application. This includes the removal of the Heartbeat method from the Service interface, the associated interval setting in the config structure, and related environment variable documentation. The change implies an architectural shift towards alternative methods of service availability checking or potentially a move to stateless service design. The cleanup removes unnecessary code, thus simplifying the overall service logic.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Introduce Terminate message and streamline gRPC server logic

- Added a new `Terminate` protobuf message to handle termination requests more explicitly.
- Removed redundant context cancels in gRPC server stream processing to rely on context propagation.
- Updated channel usage to match the altered stream message types; this aligns message handling with the updated protobuf definitions.
- Passed client authentication info to service run function to enhance the security during service execution.
- Utilized stream's context directly in error groups for improved clarity and error handling scope.
- Refactored message descriptors in protobuf to maintain correct ordering after introducing new message types.

These changes enhance the codebase's maintainability, simplify the logic, and align with the updated messaging protocol.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update agent configuration in main.go

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-04-29 14:08:42 +02:00
Sammy Kerata Oina 64f7e7f7fd NOISSUE - Refactor single algorithm processing (#117)
* Refactor single algorithm processing

Simplified the agent service's algorithm handling logic to process a single algorithm instead of multiple. This change:
- Removed the `Algorithms` type and associated stringer implementation.
- Updated the state machine and service logic to expect a singular algorithm, aligning the agent's internal state transitions with the new model.
- Adjusted the manager service and computations test server to mirror these changes in their respective payload structures, ensuring API and test consistency.
- Altered README files to reflect the simplified interaction model and removed outdated descriptions.
- Reverted the protoc-gen-go version used for generating protobuf files to maintain compatibility with the rest of the codebase.

The single-algorithm approach streamlines the computation running process, reducing complexity and potential error conditions. It directly impacts how external services will construct and send computation requests.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Update protoc-gen-go version to v1.33.0

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor variable name in computations.go and grpc.go

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-04-17 13:34:54 +02:00
Danko Miladinovic 3a14896555 NOISSUE - Attested TLS (#99)
* added initial code for attested TLS

* added client validation and verification

* fixed bugs for attested TLS

* updated README for manual testing

* fixed CI errors

* removed SNP pollicy from agent config

* added attested TLS config param to AgentConfig

* generated manager.pb.go for protoc v25.2

* updated proto-gen-go version on CI

* generated agent.pb.go to match newest proto gen version

* define errors for error handling

* fixed comments
2024-04-16 15:19:19 +02:00
Drasko DRASKOVIC b57bafd310 Fix usage (#114)
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2024-04-16 00:42:47 +02:00
Drasko DRASKOVIC 9e6fb31d90 Fix links and add features (#113)
Signed-off-by: Drasko Draskovic <drasko.draskovic@gmail.com>
2024-04-15 22:26:41 +02:00
69 changed files with 2557 additions and 1012 deletions
+2 -2
View File
@@ -29,12 +29,12 @@ jobs:
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
go-version: 1.22.x
- name: Set up protoc
run: |
PROTOC_VERSION=25.3
PROTOC_GEN_VERSION=v1.31.0
PROTOC_GEN_VERSION=v1.33.0
PROTOC_GRPC_VERSION=v1.3.0
# Download and install protoc
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
go-version: 1.22.x
cache-dependency-path: "go.sum"
- name: Checkout cocos
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
go-version: 1.22.x
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
+2
View File
@@ -6,3 +6,5 @@ cmd/manager/iso
cmd/manager/tmp
.cov
*.pem
+5 -1
View File
@@ -1,5 +1,6 @@
BUILD_DIR = build
SERVICES = manager agent cli
PLATFORM_INFO = platform_info
CGO_ENABLED ?= 0
GOARCH ?= amd64
VERSION ?= $(shell git describe --abbrev=0 --tags --always)
@@ -17,13 +18,16 @@ define compile_service
-o ${BUILD_DIR}/cocos-$(1) cmd/$(1)/main.go
endef
.PHONY: all $(SERVICES)
.PHONY: all $(SERVICES) $(PLATFORM_INFO)
all: $(SERVICES)
$(SERVICES):
$(call compile_service,$(@))
$(PLATFORM_INFO):
$(MAKE) -C ./scripts/platform_info
protoc:
protoc -I. --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative agent/agent.proto
protoc -I. --go_out=./pkg --go_opt=paths=source_relative --go-grpc_out=./pkg --go-grpc_opt=paths=source_relative manager/manager.proto
+17 -8
View File
@@ -7,7 +7,7 @@ based on the [Confidential Computing][cc] and [Trusted Execution Environments (T
</p>
With Cocos AI it becomes possible to run AI/ML workloads on combined datasets from multiple organizations
while guaranteeing the privacy and security of the data and the algorithms.
while guaranteeing the privacy and security of the data and the algorithm.
Data is always encrypted, protected by hardware secure enclaves (Trusted Execution Environments),
attested via secure remote attestation protocols, and invisible to cloud processors or any other
3rd party to which computation is offloaded.
@@ -26,20 +26,28 @@ Cocos AI is implementing the following features:
## Usage
The quickest way to start using Cocos is via the CLI. The latest version can be downloaded from the [official releases page][rel].
It can also be built and used from the project's root directory:
Clone the repo and create binaries:
```bash
make cli
./build/cocos-cli version
git clone git@github.com:ultravioletrs/cocos.git
make
```
Additional details on using the CLI can be found in the [CLI documentation][cli].
This will create 3 binaries:
```bash
ls build/
# cocos-agent cocos-cli cocos-manager
```
- Manager can be deployed on the AMD SEV-SNP host
- Agent can be built into [EOS][eos]-based HAL
- CLI can be used to communicate to remote Agent.
## Documentation
Official documentation is hosted at [Cocos official docs page][docs]. Documentation is auto-generated, checkout the instructions on [official docs repository](https://github.com/ultravioletrs/docs).
Project documentation is hosted at [Cocos AI official docs page][docs].
Documentation is generated from the [docs repository](https://github.com/ultravioletrs/docs).
## License
Cocos AI is published under permissive open-source [Apache-2.0](LICENSE) license.
@@ -50,3 +58,4 @@ Cocos AI is published under permissive open-source [Apache-2.0](LICENSE) license
[tee]: https://en.wikipedia.org/wiki/Trusted_execution_environment
[docs]: https://docs.cocos.ultraviolet.rs
[cli]: https://docs.cocos.ultraviolet.rs/cli
[eos]: https://github.com/ultravioletrs/eos
-1
View File
@@ -15,7 +15,6 @@ The service is configured using the environment variables from the following tab
| AGENT_GRPC_SERVER_KEY | Path to gRPC server key in pem format | "" |
| AGENT_GRPC_SERVER_CA_CERTS | Path to gRPC server CA certificate | "" |
| AGENT_GRPC_CLIENT_CA_CERTS | Path to gRPC client CA certificate | "" |
| COCOS_NOTIFICATION_SERVER_URL | Server to receive notification events from agent. | http:/localhost:9000 |
## Deployment
+26 -74
View File
@@ -3,7 +3,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.31.0
// protoc-gen-go v1.33.0
// protoc v4.25.3
// source: agent/agent.proto
@@ -29,8 +29,6 @@ type AlgoRequest struct {
unknownFields protoimpl.UnknownFields
Algorithm []byte `protobuf:"bytes,1,opt,name=algorithm,proto3" json:"algorithm,omitempty"`
Provider string `protobuf:"bytes,2,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,3,opt,name=id,proto3" json:"id,omitempty"`
}
func (x *AlgoRequest) Reset() {
@@ -72,20 +70,6 @@ func (x *AlgoRequest) GetAlgorithm() []byte {
return nil
}
func (x *AlgoRequest) GetProvider() string {
if x != nil {
return x.Provider
}
return ""
}
func (x *AlgoRequest) GetId() string {
if x != nil {
return x.Id
}
return ""
}
type AlgoResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -129,9 +113,7 @@ type DataRequest struct {
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Dataset []byte `protobuf:"bytes,1,opt,name=dataset,proto3" json:"dataset,omitempty"`
Provider string `protobuf:"bytes,2,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,3,opt,name=id,proto3" json:"id,omitempty"`
Dataset []byte `protobuf:"bytes,1,opt,name=dataset,proto3" json:"dataset,omitempty"`
}
func (x *DataRequest) Reset() {
@@ -173,20 +155,6 @@ func (x *DataRequest) GetDataset() []byte {
return nil
}
func (x *DataRequest) GetProvider() string {
if x != nil {
return x.Provider
}
return ""
}
func (x *DataRequest) GetId() string {
if x != nil {
return x.Id
}
return ""
}
type DataResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -229,8 +197,6 @@ type ResultRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Consumer string `protobuf:"bytes,1,opt,name=consumer,proto3" json:"consumer,omitempty"`
}
func (x *ResultRequest) Reset() {
@@ -265,13 +231,6 @@ func (*ResultRequest) Descriptor() ([]byte, []int) {
return file_agent_agent_proto_rawDescGZIP(), []int{4}
}
func (x *ResultRequest) GetConsumer() string {
if x != nil {
return x.Consumer
}
return ""
}
type ResultResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -417,39 +376,32 @@ var File_agent_agent_proto protoreflect.FileDescriptor
var file_agent_agent_proto_rawDesc = []byte{
0x0a, 0x11, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2f, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x12, 0x05, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x22, 0x57, 0x0a, 0x0b, 0x41, 0x6c,
0x6f, 0x74, 0x6f, 0x12, 0x05, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x22, 0x2b, 0x0a, 0x0b, 0x41, 0x6c,
0x67, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1c, 0x0a, 0x09, 0x61, 0x6c, 0x67,
0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x09, 0x61, 0x6c,
0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x1a, 0x0a, 0x08, 0x70, 0x72, 0x6f, 0x76, 0x69,
0x64, 0x65, 0x72, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x70, 0x72, 0x6f, 0x76, 0x69,
0x64, 0x65, 0x72, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52,
0x02, 0x69, 0x64, 0x22, 0x0e, 0x0a, 0x0c, 0x41, 0x6c, 0x67, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f,
0x6e, 0x73, 0x65, 0x22, 0x53, 0x0a, 0x0b, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65,
0x73, 0x74, 0x12, 0x18, 0x0a, 0x07, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x18, 0x01, 0x20,
0x01, 0x28, 0x0c, 0x52, 0x07, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x12, 0x1a, 0x0a, 0x08,
0x70, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08,
0x70, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x03,
0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x22, 0x0e, 0x0a, 0x0c, 0x44, 0x61, 0x74, 0x61,
0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x2b, 0x0a, 0x0d, 0x52, 0x65, 0x73, 0x75,
0x6c, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1a, 0x0a, 0x08, 0x63, 0x6f, 0x6e,
0x73, 0x75, 0x6d, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x63, 0x6f, 0x6e,
0x73, 0x75, 0x6d, 0x65, 0x72, 0x22, 0x24, 0x0a, 0x0e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52,
0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x18,
0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x22, 0x35, 0x0a, 0x12, 0x41,
0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x12, 0x1f, 0x0a, 0x0b, 0x72, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x5f, 0x64, 0x61, 0x74, 0x61,
0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x0a, 0x72, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x44, 0x61,
0x74, 0x61, 0x22, 0x29, 0x0a, 0x13, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f,
0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c,
0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x32, 0xf5, 0x01,
0x0a, 0x0c, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x31,
0x0a, 0x04, 0x41, 0x6c, 0x67, 0x6f, 0x12, 0x12, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41,
0x6c, 0x67, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x13, 0x2e, 0x61, 0x67, 0x65,
0x6e, 0x74, 0x2e, 0x41, 0x6c, 0x67, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22,
0x00, 0x12, 0x31, 0x0a, 0x04, 0x44, 0x61, 0x74, 0x61, 0x12, 0x12, 0x2e, 0x61, 0x67, 0x65, 0x6e,
0x74, 0x2e, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x13, 0x2e,
0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e,
0x73, 0x65, 0x22, 0x00, 0x12, 0x37, 0x0a, 0x06, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x12, 0x14,
0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x22, 0x0e, 0x0a, 0x0c, 0x41, 0x6c, 0x67, 0x6f, 0x52,
0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x27, 0x0a, 0x0b, 0x44, 0x61, 0x74, 0x61, 0x52,
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x18, 0x0a, 0x07, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65,
0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74,
0x22, 0x0e, 0x0a, 0x0c, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x22, 0x0f, 0x0a, 0x0d, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x22, 0x24, 0x0a, 0x0e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f,
0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28,
0x0c, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x22, 0x35, 0x0a, 0x12, 0x41, 0x74, 0x74, 0x65, 0x73,
0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1f, 0x0a,
0x0b, 0x72, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x5f, 0x64, 0x61, 0x74, 0x61, 0x18, 0x01, 0x20, 0x01,
0x28, 0x0c, 0x52, 0x0a, 0x72, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x44, 0x61, 0x74, 0x61, 0x22, 0x29,
0x0a, 0x13, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x73,
0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x01, 0x20,
0x01, 0x28, 0x0c, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x32, 0xf9, 0x01, 0x0a, 0x0c, 0x41, 0x67,
0x65, 0x6e, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x33, 0x0a, 0x04, 0x41, 0x6c,
0x67, 0x6f, 0x12, 0x12, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41, 0x6c, 0x67, 0x6f, 0x52,
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x13, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41,
0x6c, 0x67, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x28, 0x01, 0x12,
0x33, 0x0a, 0x04, 0x44, 0x61, 0x74, 0x61, 0x12, 0x12, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e,
0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x13, 0x2e, 0x61, 0x67,
0x65, 0x6e, 0x74, 0x2e, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x22, 0x00, 0x28, 0x01, 0x12, 0x37, 0x0a, 0x06, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x12, 0x14,
0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52, 0x65, 0x71,
0x75, 0x65, 0x73, 0x74, 0x1a, 0x15, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x52, 0x65, 0x73,
0x75, 0x6c, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x46, 0x0a,
+2 -7
View File
@@ -8,30 +8,25 @@ package agent;
option go_package = "./agent";
service AgentService {
rpc Algo(AlgoRequest) returns (AlgoResponse) {}
rpc Data(DataRequest) returns (DataResponse) {}
rpc Algo(stream AlgoRequest) returns (AlgoResponse) {}
rpc Data(stream DataRequest) returns (DataResponse) {}
rpc Result(ResultRequest) returns (ResultResponse) {}
rpc Attestation(AttestationRequest) returns (AttestationResponse) {}
}
message AlgoRequest {
bytes algorithm = 1;
string provider = 2;
string id = 3;
}
message AlgoResponse {}
message DataRequest {
bytes dataset = 1;
string provider = 2;
string id = 3;
}
message DataResponse {}
message ResultRequest {
string consumer = 1;
}
message ResultResponse {
+123 -54
View File
@@ -32,8 +32,8 @@ const (
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type AgentServiceClient interface {
Algo(ctx context.Context, in *AlgoRequest, opts ...grpc.CallOption) (*AlgoResponse, error)
Data(ctx context.Context, in *DataRequest, opts ...grpc.CallOption) (*DataResponse, error)
Algo(ctx context.Context, opts ...grpc.CallOption) (AgentService_AlgoClient, error)
Data(ctx context.Context, opts ...grpc.CallOption) (AgentService_DataClient, error)
Result(ctx context.Context, in *ResultRequest, opts ...grpc.CallOption) (*ResultResponse, error)
Attestation(ctx context.Context, in *AttestationRequest, opts ...grpc.CallOption) (*AttestationResponse, error)
}
@@ -46,22 +46,72 @@ func NewAgentServiceClient(cc grpc.ClientConnInterface) AgentServiceClient {
return &agentServiceClient{cc}
}
func (c *agentServiceClient) Algo(ctx context.Context, in *AlgoRequest, opts ...grpc.CallOption) (*AlgoResponse, error) {
out := new(AlgoResponse)
err := c.cc.Invoke(ctx, AgentService_Algo_FullMethodName, in, out, opts...)
func (c *agentServiceClient) Algo(ctx context.Context, opts ...grpc.CallOption) (AgentService_AlgoClient, error) {
stream, err := c.cc.NewStream(ctx, &AgentService_ServiceDesc.Streams[0], AgentService_Algo_FullMethodName, opts...)
if err != nil {
return nil, err
}
return out, nil
x := &agentServiceAlgoClient{stream}
return x, nil
}
func (c *agentServiceClient) Data(ctx context.Context, in *DataRequest, opts ...grpc.CallOption) (*DataResponse, error) {
out := new(DataResponse)
err := c.cc.Invoke(ctx, AgentService_Data_FullMethodName, in, out, opts...)
type AgentService_AlgoClient interface {
Send(*AlgoRequest) error
CloseAndRecv() (*AlgoResponse, error)
grpc.ClientStream
}
type agentServiceAlgoClient struct {
grpc.ClientStream
}
func (x *agentServiceAlgoClient) Send(m *AlgoRequest) error {
return x.ClientStream.SendMsg(m)
}
func (x *agentServiceAlgoClient) CloseAndRecv() (*AlgoResponse, error) {
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
m := new(AlgoResponse)
if err := x.ClientStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
func (c *agentServiceClient) Data(ctx context.Context, opts ...grpc.CallOption) (AgentService_DataClient, error) {
stream, err := c.cc.NewStream(ctx, &AgentService_ServiceDesc.Streams[1], AgentService_Data_FullMethodName, opts...)
if err != nil {
return nil, err
}
return out, nil
x := &agentServiceDataClient{stream}
return x, nil
}
type AgentService_DataClient interface {
Send(*DataRequest) error
CloseAndRecv() (*DataResponse, error)
grpc.ClientStream
}
type agentServiceDataClient struct {
grpc.ClientStream
}
func (x *agentServiceDataClient) Send(m *DataRequest) error {
return x.ClientStream.SendMsg(m)
}
func (x *agentServiceDataClient) CloseAndRecv() (*DataResponse, error) {
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
m := new(DataResponse)
if err := x.ClientStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
func (c *agentServiceClient) Result(ctx context.Context, in *ResultRequest, opts ...grpc.CallOption) (*ResultResponse, error) {
@@ -86,8 +136,8 @@ func (c *agentServiceClient) Attestation(ctx context.Context, in *AttestationReq
// All implementations must embed UnimplementedAgentServiceServer
// for forward compatibility
type AgentServiceServer interface {
Algo(context.Context, *AlgoRequest) (*AlgoResponse, error)
Data(context.Context, *DataRequest) (*DataResponse, error)
Algo(AgentService_AlgoServer) error
Data(AgentService_DataServer) error
Result(context.Context, *ResultRequest) (*ResultResponse, error)
Attestation(context.Context, *AttestationRequest) (*AttestationResponse, error)
mustEmbedUnimplementedAgentServiceServer()
@@ -97,11 +147,11 @@ type AgentServiceServer interface {
type UnimplementedAgentServiceServer struct {
}
func (UnimplementedAgentServiceServer) Algo(context.Context, *AlgoRequest) (*AlgoResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Algo not implemented")
func (UnimplementedAgentServiceServer) Algo(AgentService_AlgoServer) error {
return status.Errorf(codes.Unimplemented, "method Algo not implemented")
}
func (UnimplementedAgentServiceServer) Data(context.Context, *DataRequest) (*DataResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Data not implemented")
func (UnimplementedAgentServiceServer) Data(AgentService_DataServer) error {
return status.Errorf(codes.Unimplemented, "method Data not implemented")
}
func (UnimplementedAgentServiceServer) Result(context.Context, *ResultRequest) (*ResultResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Result not implemented")
@@ -122,40 +172,56 @@ func RegisterAgentServiceServer(s grpc.ServiceRegistrar, srv AgentServiceServer)
s.RegisterService(&AgentService_ServiceDesc, srv)
}
func _AgentService_Algo_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(AlgoRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(AgentServiceServer).Algo(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: AgentService_Algo_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(AgentServiceServer).Algo(ctx, req.(*AlgoRequest))
}
return interceptor(ctx, in, info, handler)
func _AgentService_Algo_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(AgentServiceServer).Algo(&agentServiceAlgoServer{stream})
}
func _AgentService_Data_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(DataRequest)
if err := dec(in); err != nil {
type AgentService_AlgoServer interface {
SendAndClose(*AlgoResponse) error
Recv() (*AlgoRequest, error)
grpc.ServerStream
}
type agentServiceAlgoServer struct {
grpc.ServerStream
}
func (x *agentServiceAlgoServer) SendAndClose(m *AlgoResponse) error {
return x.ServerStream.SendMsg(m)
}
func (x *agentServiceAlgoServer) Recv() (*AlgoRequest, error) {
m := new(AlgoRequest)
if err := x.ServerStream.RecvMsg(m); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(AgentServiceServer).Data(ctx, in)
return m, nil
}
func _AgentService_Data_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(AgentServiceServer).Data(&agentServiceDataServer{stream})
}
type AgentService_DataServer interface {
SendAndClose(*DataResponse) error
Recv() (*DataRequest, error)
grpc.ServerStream
}
type agentServiceDataServer struct {
grpc.ServerStream
}
func (x *agentServiceDataServer) SendAndClose(m *DataResponse) error {
return x.ServerStream.SendMsg(m)
}
func (x *agentServiceDataServer) Recv() (*DataRequest, error) {
m := new(DataRequest)
if err := x.ServerStream.RecvMsg(m); err != nil {
return nil, err
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: AgentService_Data_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(AgentServiceServer).Data(ctx, req.(*DataRequest))
}
return interceptor(ctx, in, info, handler)
return m, nil
}
func _AgentService_Result_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
@@ -201,14 +267,6 @@ var AgentService_ServiceDesc = grpc.ServiceDesc{
ServiceName: "agent.AgentService",
HandlerType: (*AgentServiceServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "Algo",
Handler: _AgentService_Algo_Handler,
},
{
MethodName: "Data",
Handler: _AgentService_Data_Handler,
},
{
MethodName: "Result",
Handler: _AgentService_Result_Handler,
@@ -218,6 +276,17 @@ var AgentService_ServiceDesc = grpc.ServiceDesc{
Handler: _AgentService_Attestation_Handler,
},
},
Streams: []grpc.StreamDesc{},
Streams: []grpc.StreamDesc{
{
StreamName: "Algo",
Handler: _AgentService_Algo_Handler,
ClientStreams: true,
},
{
StreamName: "Data",
Handler: _AgentService_Data_Handler,
ClientStreams: true,
},
},
Metadata: "agent/agent.proto",
}
-218
View File
@@ -1,218 +0,0 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package grpc
import (
"context"
"fmt"
"time"
"github.com/go-kit/kit/endpoint"
kitgrpc "github.com/go-kit/kit/transport/grpc"
"github.com/ultravioletrs/cocos/agent"
"google.golang.org/grpc"
)
const svcName = "agent.AgentService"
type grpcClient struct {
algo endpoint.Endpoint
data endpoint.Endpoint
result endpoint.Endpoint
attestation endpoint.Endpoint
timeout time.Duration
}
// NewClient returns new gRPC client instance.
func NewClient(conn *grpc.ClientConn, timeout time.Duration) agent.AgentServiceClient {
return &grpcClient{
algo: kitgrpc.NewClient(
conn,
svcName,
"Algo",
encodeAlgoRequest,
decodeAlgoResponse,
agent.AlgoResponse{},
).Endpoint(),
data: kitgrpc.NewClient(
conn,
svcName,
"Data",
encodeDataRequest,
decodeDataResponse,
agent.DataResponse{},
).Endpoint(),
result: kitgrpc.NewClient(
conn,
svcName,
"Result",
encodeResultRequest,
decodeResultResponse,
agent.ResultResponse{},
).Endpoint(),
attestation: kitgrpc.NewClient(
conn,
svcName,
"Attestation",
encodeAttestationRequest,
decodeAttestationResponse,
agent.AttestationResponse{},
).Endpoint(),
timeout: timeout,
}
}
// encodeAlgoRequest is a transport/grpc.EncodeRequestFunc that
// converts a user-domain algoReq to a gRPC request.
func encodeAlgoRequest(_ context.Context, request interface{}) (interface{}, error) {
req, ok := request.(*algoReq)
if !ok {
return nil, fmt.Errorf("invalid request type: %T", request)
}
return &agent.AlgoRequest{
Algorithm: req.Algorithm,
Provider: req.Provider,
Id: req.Id,
}, nil
}
// decodeAlgoResponse is a transport/grpc.DecodeResponseFunc that
// converts a gRPC AlgoResponse to a user-domain response.
func decodeAlgoResponse(_ context.Context, grpcResponse interface{}) (interface{}, error) {
_, ok := grpcResponse.(*agent.AlgoResponse)
if !ok {
return nil, fmt.Errorf("invalid response type: %T", grpcResponse)
}
return algoRes{}, nil
}
// encodeDataRequest is a transport/grpc.EncodeRequestFunc that
// converts a user-domain dataReq to a gRPC request.
func encodeDataRequest(_ context.Context, request interface{}) (interface{}, error) {
req, ok := request.(*dataReq)
if !ok {
return nil, fmt.Errorf("invalid request type: %T", request)
}
return &agent.DataRequest{
Dataset: req.Dataset,
Provider: req.Provider,
Id: req.Id,
}, nil
}
// decodeDataResponse is a transport/grpc.DecodeResponseFunc that
// converts a gRPC DataResponse to a user-domain response.
func decodeDataResponse(_ context.Context, grpcResponse interface{}) (interface{}, error) {
_, ok := grpcResponse.(*agent.DataResponse)
if !ok {
return nil, fmt.Errorf("invalid response type: %T", grpcResponse)
}
return dataRes{}, nil
}
// encodeResultRequest is a transport/grpc.EncodeRequestFunc that
// converts a user-domain resultReq to a gRPC request.
func encodeResultRequest(_ context.Context, request interface{}) (interface{}, error) {
req, ok := request.(*resultReq)
if !ok {
return nil, fmt.Errorf("invalid request type: %T", request)
}
return &agent.ResultRequest{
Consumer: req.Consumer,
}, nil
}
// decodeResultResponse is a transport/grpc.DecodeResponseFunc that
// converts a gRPC ResultResponse to a user-domain response.
func decodeResultResponse(_ context.Context, grpcResponse interface{}) (interface{}, error) {
response, ok := grpcResponse.(*agent.ResultResponse)
if !ok {
return nil, fmt.Errorf("invalid response type: %T", grpcResponse)
}
return resultRes{
File: response.File,
}, nil
}
// encodeAttestationRequest is a transport/grpc.EncodeRequestFunc that
// converts a user-domain attestationReq to a gRPC request.
func encodeAttestationRequest(_ context.Context, request interface{}) (interface{}, error) {
req, ok := request.(*attestationReq)
if !ok {
return nil, fmt.Errorf("invalid request type: %T", request)
}
return &agent.AttestationRequest{ReportData: req.ReportData[:]}, nil
}
// decodeAttestationResponse is a transport/grpc.DecodeResponseFunc that
// converts a gRPC AttestationResponse to a user-domain response.
func decodeAttestationResponse(_ context.Context, grpcResponse interface{}) (interface{}, error) {
response, ok := grpcResponse.(*agent.AttestationResponse)
if !ok {
return nil, fmt.Errorf("invalid response type: %T", grpcResponse)
}
return attestationRes{
File: response.File,
}, nil
}
// Algo implements the Algo method of the agent.AgentServiceClient interface.
func (c grpcClient) Algo(ctx context.Context, request *agent.AlgoRequest, _ ...grpc.CallOption) (*agent.AlgoResponse, error) {
ctx, cancel := context.WithTimeout(ctx, c.timeout)
defer cancel()
_, err := c.algo(ctx, &algoReq{Algorithm: request.Algorithm, Provider: request.Provider, Id: request.Id})
if err != nil {
return nil, err
}
return &agent.AlgoResponse{}, nil
}
// Data implements the Data method of the agent.AgentServiceClient interface.
func (c grpcClient) Data(ctx context.Context, request *agent.DataRequest, _ ...grpc.CallOption) (*agent.DataResponse, error) {
ctx, cancel := context.WithTimeout(ctx, c.timeout)
defer cancel()
_, err := c.data(ctx, &dataReq{Dataset: request.Dataset, Provider: request.Provider, Id: request.Id})
if err != nil {
return nil, err
}
return &agent.DataResponse{}, nil
}
// Result implements the Result method of the agent.AgentServiceClient interface.
func (c grpcClient) Result(ctx context.Context, request *agent.ResultRequest, _ ...grpc.CallOption) (*agent.ResultResponse, error) {
ctx, cancel := context.WithTimeout(ctx, c.timeout)
defer cancel()
res, err := c.result(ctx, &resultReq{Consumer: request.Consumer})
if err != nil {
return nil, err
}
resultRes := res.(resultRes)
return &agent.ResultResponse{File: resultRes.File}, nil
}
// Result implements the Result method of the agent.AgentServiceClient interface.
func (c grpcClient) Attestation(ctx context.Context, request *agent.AttestationRequest, _ ...grpc.CallOption) (*agent.AttestationResponse, error) {
ctx, cancel := context.WithTimeout(ctx, c.timeout)
defer cancel()
res, err := c.attestation(ctx, &attestationReq{ReportData: [agent.ReportDataSize]byte(request.ReportData)})
if err != nil {
return nil, err
}
attestationRes := res.(attestationRes)
return &agent.AttestationResponse{File: attestationRes.File}, nil
}
+3 -3
View File
@@ -17,7 +17,7 @@ func algoEndpoint(svc agent.Service) endpoint.Endpoint {
return algoRes{}, err
}
algo := agent.Algorithm{Algorithm: req.Algorithm, Provider: req.Provider, ID: req.Id}
algo := agent.Algorithm{Algorithm: req.Algorithm}
err := svc.Algo(ctx, algo)
if err != nil {
@@ -36,7 +36,7 @@ func dataEndpoint(svc agent.Service) endpoint.Endpoint {
return dataRes{}, err
}
dataset := agent.Dataset{Dataset: req.Dataset, Provider: req.Provider, ID: req.Id}
dataset := agent.Dataset{Dataset: req.Dataset}
err := svc.Data(ctx, dataset)
if err != nil {
@@ -54,7 +54,7 @@ func resultEndpoint(svc agent.Service) endpoint.Endpoint {
if err := req.validate(); err != nil {
return resultRes{}, err
}
file, err := svc.Result(ctx, req.Consumer)
file, err := svc.Result(ctx)
if err != nil {
return resultRes{}, err
}
+2 -20
View File
@@ -8,45 +8,27 @@ import (
type algoReq struct {
Algorithm []byte `protobuf:"bytes,1,opt,name=algorithm,proto3" json:"algorithm,omitempty"`
Provider string `protobuf:"bytes,2,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,3,opt,name=id,proto3" json:"id,omitempty"`
}
func (req algoReq) validate() error {
if len(req.Algorithm) == 0 {
return errors.New("algorithm binary is required")
}
if req.Id == "" {
return errors.New("malformed entity")
}
if req.Provider == "" {
return errors.New("malformed entity")
}
return nil
}
type dataReq struct {
Dataset []byte `protobuf:"bytes,1,opt,name=dataset,proto3" json:"dataset,omitempty"`
Provider string `protobuf:"bytes,2,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,3,opt,name=id,proto3" json:"id,omitempty"`
Dataset []byte `protobuf:"bytes,1,opt,name=dataset,proto3" json:"dataset,omitempty"`
}
func (req dataReq) validate() error {
if len(req.Dataset) == 0 {
return errors.New("dataset CSV file is required")
}
if req.Id == "" {
return errors.New("malformed entity")
}
if req.Provider == "" {
return errors.New("malformed entity")
}
return nil
}
type resultReq struct {
Consumer string `protobuf:"bytes,1,opt,name=consumer,proto3" json:"consumer,omitempty"`
}
type resultReq struct{}
func (req resultReq) validate() error {
// No request parameters to validate, so no validation logic needed
+41 -17
View File
@@ -5,11 +5,16 @@ package grpc
import (
"context"
"errors"
"io"
"github.com/go-kit/kit/transport/grpc"
"github.com/ultravioletrs/cocos/agent"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
var _ agent.AgentServiceServer = (*grpcServer)(nil)
type grpcServer struct {
algo grpc.Handler
data grpc.Handler
@@ -49,8 +54,6 @@ func decodeAlgoRequest(_ context.Context, grpcReq interface{}) (interface{}, err
return algoReq{
Algorithm: req.Algorithm,
Provider: req.Provider,
Id: req.Id,
}, nil
}
@@ -62,9 +65,7 @@ func decodeDataRequest(_ context.Context, grpcReq interface{}) (interface{}, err
req := grpcReq.(*agent.DataRequest)
return dataReq{
Dataset: req.Dataset,
Provider: req.Provider,
Id: req.Id,
Dataset: req.Dataset,
}, nil
}
@@ -73,8 +74,7 @@ func encodeDataResponse(_ context.Context, response interface{}) (interface{}, e
}
func decodeResultRequest(_ context.Context, grpcReq interface{}) (interface{}, error) {
req := grpcReq.(*agent.ResultRequest)
return resultReq{Consumer: req.Consumer}, nil
return resultReq{}, nil
}
func encodeResultResponse(_ context.Context, response interface{}) (interface{}, error) {
@@ -99,22 +99,46 @@ func encodeAttestationResponse(_ context.Context, response interface{}) (interfa
}, nil
}
func (s *grpcServer) Algo(ctx context.Context, req *agent.AlgoRequest) (*agent.AlgoResponse, error) {
_, res, err := s.algo.ServeGRPC(ctx, req)
// Algo implements agent.AgentServiceServer.
func (s *grpcServer) Algo(stream agent.AgentService_AlgoServer) error {
var algoFile []byte
for {
algoChunk, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
return status.Error(codes.Internal, err.Error())
}
algoFile = append(algoFile, algoChunk.Algorithm...)
}
_, res, err := s.algo.ServeGRPC(stream.Context(), &agent.AlgoRequest{Algorithm: algoFile})
if err != nil {
return nil, err
return err
}
ar := res.(*agent.AlgoResponse)
return ar, nil
return stream.SendAndClose(ar)
}
func (s *grpcServer) Data(ctx context.Context, req *agent.DataRequest) (*agent.DataResponse, error) {
_, res, err := s.data.ServeGRPC(ctx, req)
if err != nil {
return nil, err
// Data implements agent.AgentServiceServer.
func (s *grpcServer) Data(stream agent.AgentService_DataServer) error {
var dataFile []byte
for {
dataChunk, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
return status.Error(codes.Internal, err.Error())
}
dataFile = append(dataFile, dataChunk.Dataset...)
}
dr := res.(*agent.DataResponse)
return dr, nil
_, res, err := s.data.ServeGRPC(stream.Context(), &agent.DataRequest{Dataset: dataFile})
if err != nil {
return err
}
ar := res.(*agent.DataResponse)
return stream.SendAndClose(ar)
}
func (s *grpcServer) Result(ctx context.Context, req *agent.ResultRequest) (*agent.ResultResponse, error) {
+2 -2
View File
@@ -53,7 +53,7 @@ func (lm *loggingMiddleware) Data(ctx context.Context, dataset agent.Dataset) (e
return lm.svc.Data(ctx, dataset)
}
func (lm *loggingMiddleware) Result(ctx context.Context, consumer string) (response []byte, err error) {
func (lm *loggingMiddleware) Result(ctx context.Context) (response []byte, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method Result took %s to complete", time.Since(begin))
if err != nil {
@@ -63,7 +63,7 @@ func (lm *loggingMiddleware) Result(ctx context.Context, consumer string) (respo
lm.logger.Info(fmt.Sprintf("%s without errors", message))
}(time.Now())
return lm.svc.Result(ctx, consumer)
return lm.svc.Result(ctx)
}
func (lm *loggingMiddleware) Attestation(ctx context.Context, reportData [agent.ReportDataSize]byte) (response []byte, err error) {
+2 -2
View File
@@ -50,13 +50,13 @@ func (ms *metricsMiddleware) Data(ctx context.Context, dataset agent.Dataset) er
return ms.svc.Data(ctx, dataset)
}
func (ms *metricsMiddleware) Result(ctx context.Context, consumer string) ([]byte, error) {
func (ms *metricsMiddleware) Result(ctx context.Context) ([]byte, error) {
defer func(begin time.Time) {
ms.counter.With("method", "result").Add(1)
ms.latency.With("method", "result").Observe(time.Since(begin).Seconds())
}(time.Now())
return ms.svc.Result(ctx, consumer)
return ms.svc.Result(ctx)
}
func (ms *metricsMiddleware) Attestation(ctx context.Context, reportData [agent.ReportDataSize]byte) ([]byte, error) {
+180
View File
@@ -0,0 +1,180 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package auth
import (
"context"
"crypto"
"crypto/rsa"
"crypto/sha256"
"crypto/x509"
"encoding/base64"
"errors"
"github.com/ultravioletrs/cocos/agent"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
)
type UserRole string
const (
UserMetadataKey = "user-id"
SignatureMetadataKey = "signature"
ConsumerRole UserRole = "consumer"
DataProviderRole UserRole = "data-provider"
AlgorithmProviderRole UserRole = "algorithm-provider"
)
var errNotRSAPublicKey = errors.New("not an RSA public key")
type wrappedServerStream struct {
grpc.ServerStream
ctx context.Context
}
func (s *wrappedServerStream) Context() context.Context {
return s.ctx
}
type Service struct {
resultConsumers []*rsa.PublicKey
datasetProviders []*rsa.PublicKey
algorithmProvider *rsa.PublicKey
}
func New(manifest agent.Computation) (*Service, error) {
s := &Service{}
for _, rc := range manifest.ResultConsumers {
pubKey, err := x509.ParsePKIXPublicKey(rc.UserKey)
if err != nil {
return nil, err
}
rsaPubKey, ok := pubKey.(*rsa.PublicKey)
if !ok {
return nil, errNotRSAPublicKey
}
s.resultConsumers = append(s.resultConsumers, rsaPubKey)
}
for _, dp := range manifest.Datasets {
pubKey, err := x509.ParsePKIXPublicKey(dp.UserKey)
if err != nil {
return nil, err
}
rsaPubKey, ok := pubKey.(*rsa.PublicKey)
if !ok {
return nil, errNotRSAPublicKey
}
s.datasetProviders = append(s.datasetProviders, rsaPubKey)
}
pubKey, err := x509.ParsePKIXPublicKey(manifest.Algorithm.UserKey)
if err != nil {
return nil, err
}
rsaPubKey, ok := pubKey.(*rsa.PublicKey)
if !ok {
return nil, errNotRSAPublicKey
}
s.algorithmProvider = rsaPubKey
return s, nil
}
func (s *Service) AuthStreamInterceptor() grpc.StreamServerInterceptor {
return func(srv interface{}, stream grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
switch info.FullMethod {
case agent.AgentService_Algo_FullMethodName:
md, ok := metadata.FromIncomingContext(stream.Context())
if !ok {
return status.Errorf(codes.Unauthenticated, "missing metadata")
}
signature, err := extractSignature(md)
if err != nil {
return status.Errorf(codes.Unauthenticated, "invalid metadata")
}
isValid, err := verifySignature(AlgorithmProviderRole, signature, s.algorithmProvider)
if err != nil || !isValid {
return status.Errorf(codes.Unauthenticated, "signature verification failed")
}
case agent.AgentService_Data_FullMethodName:
md, ok := metadata.FromIncomingContext(stream.Context())
if !ok {
return status.Errorf(codes.Unauthenticated, "missing metadata")
}
signature, err := extractSignature(md)
if err != nil {
return status.Errorf(codes.Unauthenticated, "invalid metadata")
}
for index, dp := range s.datasetProviders {
isValid, err := verifySignature(DataProviderRole, signature, dp)
if err == nil || isValid {
ctx := agent.IndexToContext(stream.Context(), index)
wrapped := &wrappedServerStream{ServerStream: stream, ctx: ctx}
return handler(srv, wrapped)
}
}
return status.Errorf(codes.Unauthenticated, "signature verification failed")
default:
return handler(srv, stream)
}
return handler(srv, stream)
}
}
func (s *Service) AuthUnaryInterceptor() grpc.UnaryServerInterceptor {
return func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
switch info.FullMethod {
case agent.AgentService_Result_FullMethodName:
md, ok := metadata.FromIncomingContext(ctx)
if !ok {
return nil, status.Errorf(codes.Unauthenticated, "missing metadata")
}
signature, err := extractSignature(md)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "invalid metadata")
}
for index, rc := range s.resultConsumers {
isValid, err := verifySignature(ConsumerRole, signature, rc)
if err == nil || isValid {
ctx := agent.IndexToContext(ctx, index)
return handler(ctx, req)
}
}
return nil, status.Errorf(codes.Unauthenticated, "signature verification failed")
default:
return handler(ctx, req)
}
}
}
func extractSignature(md metadata.MD) (string, error) {
signature := md.Get(SignatureMetadataKey)
if len(signature) != 1 {
return "", status.Errorf(codes.Unauthenticated, "invalid metadata")
}
return signature[0], nil
}
func verifySignature(role UserRole, signature string, publicKey *rsa.PublicKey) (bool, error) {
hash := sha256.Sum256([]byte(role))
sigByte, err := base64.StdEncoding.DecodeString(signature)
if err != nil {
return false, err
}
if err := rsa.VerifyPKCS1v15(publicKey, crypto.SHA256, hash[:], sigByte); err != nil {
return false, err
}
return true, nil
}
+26 -39
View File
@@ -3,15 +3,12 @@
package agent
import (
"context"
"encoding/json"
"fmt"
"reflect"
)
var (
_ fmt.Stringer = (*Datasets)(nil)
_ fmt.Stringer = (*Algorithms)(nil)
)
var _ fmt.Stringer = (*Datasets)(nil)
type AgentConfig struct {
LogLevel string `json:"log_level"`
@@ -21,16 +18,21 @@ type AgentConfig struct {
KeyFile string `json:"server_key"`
ServerCAFile string `json:"server_ca_file"`
ClientCAFile string `json:"client_ca_file"`
AttestedTls bool `json:"attested_tls"`
}
type Computation struct {
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Datasets Datasets `json:"datasets,omitempty"`
Algorithms Algorithms `json:"algorithms,omitempty"`
ResultConsumers []string `json:"result_consumers,omitempty"`
AgentConfig AgentConfig `json:"agent_config,omitempty"`
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Datasets Datasets `json:"datasets,omitempty"`
Algorithm Algorithm `json:"algorithm,omitempty"`
ResultConsumers []ResultConsumer `json:"result_consumers,omitempty"`
AgentConfig AgentConfig `json:"agent_config,omitempty"`
}
type ResultConsumer struct {
UserKey []byte `json:"user_key,omitempty"`
}
func (d *Datasets) String() string {
@@ -41,19 +43,10 @@ func (d *Datasets) String() string {
return string(dat)
}
func (a *Algorithms) String() string {
dat, err := json.Marshal(a)
if err != nil {
return ""
}
return string(dat)
}
type Dataset struct {
Dataset []byte `json:"-"`
Hash [32]byte `json:"hash,omitempty"`
Provider string `json:"provider,omitempty"`
ID string `json:"id,omitempty"`
Dataset []byte `json:"-"`
Hash [32]byte `json:"hash,omitempty"`
UserKey []byte `json:"user_key,omitempty"`
}
type Datasets []Dataset
@@ -61,22 +54,16 @@ type Datasets []Dataset
type Algorithm struct {
Algorithm []byte `json:"-"`
Hash [32]byte `json:"hash,omitempty"`
Provider string `json:"provider,omitempty"`
ID string `json:"id,omitempty"`
UserKey []byte `json:"user_key,omitempty"`
}
type Algorithms []Algorithm
type ManifestIndexKey struct{}
func containsID(slice interface{}, id string) int {
rangeOnMe := reflect.ValueOf(slice)
for i := 0; i < rangeOnMe.Len(); i++ {
s := rangeOnMe.Index(i)
f := s.FieldByName("ID")
if f.IsValid() {
if f.Interface() == id {
return i
}
}
}
return -1
func IndexToContext(ctx context.Context, index int) context.Context {
return context.WithValue(ctx, ManifestIndexKey{}, index)
}
func IndexFromContext(ctx context.Context) (int, bool) {
index, ok := ctx.Value(ManifestIndexKey{}).(int)
return index, ok
}
+82 -60
View File
@@ -4,11 +4,13 @@
package agent
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"log/slog"
"os"
"os/exec"
"slices"
@@ -20,8 +22,12 @@ import (
var _ Service = (*agentService)(nil)
// ReportDataSize is the size of the report data expected by the attestation service.
const ReportDataSize = 64
const (
// ReportDataSize is the size of the report data expected by the attestation service.
ReportDataSize = 64
socketPath = "unix_socket"
algoFilePermission = 0o700
)
var (
// ErrMalformedEntity indicates malformed entity specification (e.g.
@@ -31,11 +37,7 @@ var (
// when accessing a protected resource.
ErrUnauthorizedAccess = errors.New("missing or invalid credentials provided")
// errUndeclaredAlgorithm indicates algorithm was not declared in computation manifest.
errUndeclaredAlgorithm = errors.New("algorithm not declared in computation manifest")
// errUndeclaredAlgorithm indicates algorithm was not declared in computation manifest.
errUndeclaredDataset = errors.New("dataset not declared in computation manifest")
// errProviderMissmatch algorithm/dataset provider does not match computation manifest.
errProviderMissmatch = errors.New("provider does not match declaration on manifest")
// errAllManifestItemsReceived indicates no new computation manifest items expected.
errAllManifestItemsReceived = errors.New("all expected manifest Items have been received")
// errUndeclaredConsumer indicates the consumer requesting results in not declared in computation manifest.
@@ -53,25 +55,20 @@ var (
type Service interface {
Algo(ctx context.Context, algorithm Algorithm) error
Data(ctx context.Context, dataset Dataset) error
Result(ctx context.Context, consumer string) ([]byte, error)
Result(ctx context.Context) ([]byte, error)
Attestation(ctx context.Context, reportData [ReportDataSize]byte) ([]byte, error)
}
type agentService struct {
computation Computation // Holds the current computation request details.
algorithms [][]byte // Stores the algorithms received for the computation.
datasets [][]byte // Stores the datasets received for the computation.
algorithm string // Filepath to the algorithm received for the computation.
datasets []string // Filepath to the datasets received for the computation.
result []byte // Stores the result of the computation.
sm *StateMachine // Manages the state transitions of the agent service.
runError error // Stores any error encountered during the computation run.
eventSvc events.Service // Service for publishing events related to computation.
}
const (
socketPath = "unix_socket"
pyRuntime = "python3"
)
var _ Service = (*agentService)(nil)
// New instantiates the agent service implementation.
@@ -80,49 +77,57 @@ func New(ctx context.Context, logger *slog.Logger, eventSvc events.Service, cmp
sm: NewStateMachine(logger),
eventSvc: eventSvc,
}
go svc.sm.Start(ctx)
svc.sm.SendEvent(start)
svc.sm.StateFunctions[idle] = svc.publishEvent("in-progress", json.RawMessage{})
svc.sm.StateFunctions[receivingManifests] = svc.publishEvent("in-progress", json.RawMessage{})
svc.sm.StateFunctions[receivingAlgorithms] = svc.publishEvent("in-progress", json.RawMessage{})
svc.sm.StateFunctions[receivingManifest] = svc.publishEvent("in-progress", json.RawMessage{})
svc.sm.StateFunctions[receivingAlgorithm] = svc.publishEvent("in-progress", json.RawMessage{})
svc.sm.StateFunctions[receivingData] = svc.publishEvent("in-progress", json.RawMessage{})
svc.sm.StateFunctions[resultsReady] = svc.publishEvent("in-progress", json.RawMessage{})
svc.sm.StateFunctions[complete] = svc.publishEvent("in-progress", json.RawMessage{})
svc.sm.StateFunctions[running] = svc.runComputation
svc.computation = cmp
svc.sm.SendEvent(manifestsReceived)
svc.sm.SendEvent(manifestReceived)
return svc
}
func (as *agentService) Algo(ctx context.Context, algorithm Algorithm) error {
if as.sm.GetState() != receivingAlgorithms {
if as.sm.GetState() != receivingAlgorithm {
return errStateNotReady
}
if len(as.computation.Algorithms) == 0 {
if as.algorithm != "" {
return errAllManifestItemsReceived
}
hash := sha3.Sum256(algorithm.Algorithm)
index := containsID(as.computation.Algorithms, algorithm.ID)
switch index {
case -1:
return errUndeclaredAlgorithm
default:
if as.computation.Algorithms[index].Provider != algorithm.Provider {
return errProviderMissmatch
}
if hash != as.computation.Algorithms[index].Hash {
return errHashMismatch
}
as.computation.Algorithms = slices.Delete(as.computation.Algorithms, index, index+1)
if hash != as.computation.Algorithm.Hash {
return errHashMismatch
}
as.algorithms = append(as.algorithms, algorithm.Algorithm)
f, err := os.CreateTemp("", "algorithm")
if err != nil {
return fmt.Errorf("error creating algorithm file: %v", err)
}
if len(as.computation.Algorithms) == 0 {
as.sm.SendEvent(algorithmsReceived)
if _, err := f.Write(algorithm.Algorithm); err != nil {
return fmt.Errorf("error writing algorithm to file: %v", err)
}
if err := os.Chmod(f.Name(), algoFilePermission); err != nil {
return fmt.Errorf("error changing file permissions: %v", err)
}
if err := f.Close(); err != nil {
return fmt.Errorf("error closing file: %v", err)
}
as.algorithm = f.Name()
if as.algorithm != "" {
as.sm.SendEvent(algorithmReceived)
}
return nil
@@ -138,21 +143,29 @@ func (as *agentService) Data(ctx context.Context, dataset Dataset) error {
hash := sha3.Sum256(dataset.Dataset)
index := containsID(as.computation.Datasets, dataset.ID)
switch index {
case -1:
index, ok := IndexFromContext(ctx)
if !ok {
return errUndeclaredDataset
default:
if as.computation.Datasets[index].Provider != dataset.Provider {
return errProviderMissmatch
}
if hash != as.computation.Datasets[index].Hash {
return errHashMismatch
}
as.computation.Datasets = slices.Delete(as.computation.Datasets, index, index+1)
}
as.datasets = append(as.datasets, dataset.Dataset)
if hash != as.computation.Datasets[index].Hash {
return errHashMismatch
}
as.computation.Datasets = slices.Delete(as.computation.Datasets, index, index+1)
f, err := os.CreateTemp("", fmt.Sprintf("dataset-%d", index))
if err != nil {
return fmt.Errorf("error creating dataset file: %v", err)
}
if _, err := f.Write(dataset.Dataset); err != nil {
return fmt.Errorf("error writing dataset to file: %v", err)
}
if err := f.Close(); err != nil {
return fmt.Errorf("error closing file: %v", err)
}
as.datasets = append(as.datasets, f.Name())
if len(as.computation.Datasets) == 0 {
as.sm.SendEvent(dataReceived)
@@ -161,20 +174,18 @@ func (as *agentService) Data(ctx context.Context, dataset Dataset) error {
return nil
}
func (as *agentService) Result(ctx context.Context, consumer string) ([]byte, error) {
func (as *agentService) Result(ctx context.Context) ([]byte, error) {
if as.sm.GetState() != resultsReady {
return []byte{}, errResultsNotReady
}
if len(as.computation.ResultConsumers) == 0 {
return []byte{}, errAllManifestItemsReceived
}
index := slices.Index(as.computation.ResultConsumers, consumer)
switch index {
case -1:
index, ok := IndexFromContext(ctx)
if !ok {
return []byte{}, errUndeclaredConsumer
default:
as.computation.ResultConsumers = slices.Delete(as.computation.ResultConsumers, index, index+1)
}
as.computation.ResultConsumers = slices.Delete(as.computation.ResultConsumers, index, index+1)
if len(as.computation.ResultConsumers) == 0 {
as.sm.SendEvent(resultsConsumed)
@@ -201,9 +212,10 @@ func (as *agentService) runComputation() {
as.sm.logger.Debug("computation run started")
defer as.sm.SendEvent(runComplete)
as.publishEvent("in-progress", json.RawMessage{})()
result, err := run(as.algorithms[0], as.datasets[0])
result, err := as.run(as.algorithm, as.datasets)
if err != nil {
as.runError = err
as.sm.logger.Warn(fmt.Sprintf("computation failed with error: %s", err.Error()))
as.publishEvent("failed", json.RawMessage{})()
return
}
@@ -219,7 +231,13 @@ func (as *agentService) publishEvent(status string, details json.RawMessage) fun
}
}
func run(algoContent, dataContent []byte) ([]byte, error) {
func (as *agentService) run(algoFile string, dataFiles []string) ([]byte, error) {
defer os.Remove(algoFile)
defer func() {
for _, file := range dataFiles {
os.Remove(file)
}
}()
listener, err := socket.StartUnixSocketServer(socketPath)
if err != nil {
return nil, fmt.Errorf("error creating stdout pipe: %v", err)
@@ -232,23 +250,27 @@ func run(algoContent, dataContent []byte) ([]byte, error) {
var result []byte
var outStd, outErr bytes.Buffer
go socket.AcceptConnection(listener, dataChannel, errorChannel)
// Construct the Python script content with CSV data as a command-line argument
script := string(algoContent)
data := string(dataContent)
cmd := exec.Command(pyRuntime, "-c", script, data, socketPath)
args := append([]string{socketPath}, dataFiles...)
cmd := exec.Command(algoFile, args...)
cmd.Stderr = &outErr
cmd.Stdout = &outStd
if err := cmd.Start(); err != nil {
return nil, fmt.Errorf("error starting Python script: %v", err)
return nil, fmt.Errorf("error starting algorithm: %v", err)
}
if err := cmd.Wait(); err != nil {
return nil, fmt.Errorf("python script execution error: %v", err)
as.sm.logger.Debug(outErr.String())
return nil, fmt.Errorf("algorithm execution error: %v", err)
}
select {
case result = <-dataChannel:
as.sm.logger.Debug(outStd.String())
return result, nil
case err = <-errorChannel:
return nil, fmt.Errorf("error receiving data: %v", err)
+9 -9
View File
@@ -14,8 +14,8 @@ type state int
const (
idle state = iota
receivingManifests
receivingAlgorithms
receivingManifest
receivingAlgorithm
receivingData
running
resultsReady
@@ -26,8 +26,8 @@ type event int
const (
start event = iota
manifestsReceived
algorithmsReceived
manifestReceived
algorithmReceived
dataReceived
runComplete
resultsConsumed
@@ -56,13 +56,13 @@ func NewStateMachine(logger *slog.Logger) *StateMachine {
}
sm.Transitions[idle] = make(map[event]state)
sm.Transitions[idle][start] = receivingManifests
sm.Transitions[idle][start] = receivingManifest
sm.Transitions[receivingManifests] = make(map[event]state)
sm.Transitions[receivingManifests][manifestsReceived] = receivingAlgorithms
sm.Transitions[receivingManifest] = make(map[event]state)
sm.Transitions[receivingManifest][manifestReceived] = receivingAlgorithm
sm.Transitions[receivingAlgorithms] = make(map[event]state)
sm.Transitions[receivingAlgorithms][algorithmsReceived] = receivingData
sm.Transitions[receivingAlgorithm] = make(map[event]state)
sm.Transitions[receivingAlgorithm][algorithmReceived] = receivingData
sm.Transitions[receivingData] = make(map[event]state)
sm.Transitions[receivingData][dataReceived] = running
+4 -4
View File
@@ -9,17 +9,17 @@ func _() {
// Re-run the stringer command to generate them again.
var x [1]struct{}
_ = x[idle-0]
_ = x[receivingManifests-1]
_ = x[receivingAlgorithms-2]
_ = x[receivingManifest-1]
_ = x[receivingAlgorithm-2]
_ = x[receivingData-3]
_ = x[running-4]
_ = x[resultsReady-5]
_ = x[complete-6]
}
const _state_name = "idlereceivingManifestsreceivingAlgorithmsreceivingDatarunningresultsReadycomplete"
const _state_name = "idlereceivingManifestreceivingAlgorithmreceivingDatarunningresultsReadycomplete"
var _state_index = [...]uint8{0, 4, 22, 41, 54, 61, 73, 81}
var _state_index = [...]uint8{0, 4, 21, 39, 52, 59, 71, 79}
func (i state) String() string {
if i < 0 || i >= state(len(_state_index)-1) {
+3 -3
View File
@@ -16,9 +16,9 @@ func TestStateMachineTransitions(t *testing.T) {
event event
expected state
}{
{idle, start, receivingManifests},
{receivingManifests, manifestsReceived, receivingAlgorithms},
{receivingAlgorithms, algorithmsReceived, receivingData},
{idle, start, receivingManifest},
{receivingManifest, manifestReceived, receivingAlgorithm},
{receivingAlgorithm, algorithmReceived, receivingData},
{receivingData, dataReceived, running},
{running, runComplete, resultsReady},
{resultsReady, resultsConsumed, complete},
+4 -12
View File
@@ -1,6 +1,6 @@
# Agent CLI
This repository contains the command-line interface (CLI) tool for interacting with the Agent and manager service. The CLI allows you to perform various tasks such as running computations, uploading algorithms and datasets, and retrieving results.
This repository contains the command-line interface (CLI) tool for interacting with the Agent and manager service. The CLI allows you to perform various tasks such as running computations, uploading algorithm and datasets, and retrieving results.
## Build
@@ -12,14 +12,6 @@ make cli
## Usage
#### Run Computation
To run a computation, use the following command:
```bash
./build/cocos-cli manager run --computation '{"name": "my-computation"}'
```
#### Get attestation
Retrieves attestation information from the SEV guest and saves it to a file.
To retrieve attestation from agent, use the following command:
@@ -70,7 +62,7 @@ To validate and verify attestation from agent, use the following command:
To upload an algorithm, use the following command:
```bash
./build/cocos-cli agent algo /path/to/algorithm
./build/cocos-cli agent algo /path/to/algorithm <private_key_file_path>
```
#### Upload Dataset
@@ -78,7 +70,7 @@ To upload an algorithm, use the following command:
To upload a dataset, use the following command:
```bash
./build/cocos-cli agent data /path/to/dataset.csv
./build/cocos-cli agent data /path/to/dataset.csv <private_key_file_path>
```
#### Retrieve result
@@ -86,5 +78,5 @@ To upload a dataset, use the following command:
To retrieve the computation result, use the following command:
```bash
./build/cocos-cli agent result
./build/cocos-cli agent result <private_key_file_path>
```
+19 -7
View File
@@ -3,6 +3,8 @@
package cli
import (
"crypto/x509"
"encoding/pem"
"log"
"os"
@@ -10,12 +12,12 @@ import (
"github.com/ultravioletrs/cocos/agent"
)
func (cli *CLI) NewAlgorithmsCmd() *cobra.Command {
func (cli *CLI) NewAlgorithmCmd() *cobra.Command {
return &cobra.Command{
Use: "algo",
Short: "Upload an algorithm binary",
Example: "algo <algo_file> <id> <provider>",
Args: cobra.ExactArgs(3),
Example: "algo <algo_file> <private_key_file_path>",
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
algorithmFile := args[0]
@@ -28,12 +30,22 @@ func (cli *CLI) NewAlgorithmsCmd() *cobra.Command {
algoReq := agent.Algorithm{
Algorithm: algorithm,
ID: args[1],
Provider: args[2],
}
if err := cli.agentSDK.Algo(cmd.Context(), algoReq); err != nil {
log.Fatalf("Error uploading algorithm with ID %s and provider %s: %v", algoReq.ID, algoReq.Provider, err)
privKeyFile, err := os.ReadFile(args[1])
if err != nil {
log.Fatalf("Error reading private key file: %v", err)
}
pemBlock, _ := pem.Decode(privKeyFile)
privKey, err := x509.ParsePKCS1PrivateKey(pemBlock.Bytes)
if err != nil {
log.Fatalf("Error parsing private key: %v", err)
}
if err := cli.agentSDK.Algo(cmd.Context(), algoReq, privKey); err != nil {
log.Fatalf("Error uploading algorithm with error: %v", err)
}
log.Println("Successfully uploaded algorithm")
+18 -6
View File
@@ -3,6 +3,8 @@
package cli
import (
"crypto/x509"
"encoding/pem"
"log"
"os"
@@ -14,8 +16,8 @@ func (cli *CLI) NewDatasetsCmd() *cobra.Command {
return &cobra.Command{
Use: "data",
Short: "Upload a dataset CSV file",
Example: "data <dataset.csv> <id> <provider>",
Args: cobra.ExactArgs(3),
Example: "data <dataset.csv> <private_key_file_path>",
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
datasetFile := args[0]
@@ -27,12 +29,22 @@ func (cli *CLI) NewDatasetsCmd() *cobra.Command {
}
dataReq := agent.Dataset{
Dataset: dataset,
ID: args[1],
Provider: args[2],
Dataset: dataset,
}
if err := cli.agentSDK.Data(cmd.Context(), dataReq); err != nil {
privKeyFile, err := os.ReadFile(args[1])
if err != nil {
log.Fatalf("Error reading private key file: %v", err)
}
pemBlock, _ := pem.Decode(privKeyFile)
privKey, err := x509.ParsePKCS1PrivateKey(pemBlock.Bytes)
if err != nil {
log.Fatalf("Error parsing private key: %v", err)
}
if err := cli.agentSDK.Data(cmd.Context(), dataReq, privKey); err != nil {
log.Fatalf("Error uploading dataset: %v", err)
}
+35
View File
@@ -0,0 +1,35 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"encoding/hex"
"log"
"os"
"github.com/spf13/cobra"
"golang.org/x/crypto/sha3"
)
func (cli *CLI) NewFileHashCmd() *cobra.Command {
return &cobra.Command{
Use: "file-hash",
Short: "Compute the sha3-256 hash of a file",
Example: "file-hash <file>",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
fileName := args[0]
file, err := os.ReadFile(fileName)
if err != nil {
log.Fatalf("Error reading dataset file: %v", err)
}
hashBytes := sha3.Sum256(file)
hash := hex.EncodeToString(hashBytes[:])
log.Println("Hash of file:", hash)
},
}
}
+68
View File
@@ -0,0 +1,68 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"log"
"os"
"github.com/spf13/cobra"
)
const (
keyBitSize = 4096
privateKeyType = "RSA PRIVATE KEY"
publicKeyType = "PUBLIC KEY"
publicKeyFile = "public.pem"
privateKeyFile = "private.pem"
)
func (cli *CLI) NewKeysCmd() *cobra.Command {
return &cobra.Command{
Use: "keys",
Short: "Generate a new public/private key pair",
Run: func(cmd *cobra.Command, args []string) {
privKey, err := rsa.GenerateKey(rand.Reader, keyBitSize)
if err != nil {
log.Fatalf("Error generating public key: %v", err)
}
pubKey, err := x509.MarshalPKIXPublicKey(&privKey.PublicKey)
if err != nil {
log.Fatalf("Error marshalling public key: %v", err)
}
privFile, err := os.Create(privateKeyFile)
if err != nil {
log.Fatalf("Error creating private key file: %v", err)
}
defer privFile.Close()
if err := pem.Encode(privFile, &pem.Block{
Type: privateKeyType,
Bytes: x509.MarshalPKCS1PrivateKey(privKey),
}); err != nil {
log.Fatalf("Error encoding private key: %v", err)
}
pubFile, err := os.Create(publicKeyFile)
if err != nil {
log.Fatalf("Error creating public key file: %v", err)
}
defer pubFile.Close()
if err := pem.Encode(pubFile, &pem.Block{
Type: publicKeyType,
Bytes: pubKey,
}); err != nil {
log.Fatalf("Error encoding public key: %v", err)
}
log.Println("Successfully generated public/private key pair")
},
}
}
+62
View File
@@ -0,0 +1,62 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"encoding/base64"
"encoding/json"
"log"
"os"
"github.com/google/go-sev-guest/proto/check"
"github.com/spf13/cobra"
)
const filePermision = 0o755
type AttestationConfiguration struct {
SNPPolicy *check.Policy `json:"snp_policy,omitempty"`
RootOFTrust *check.RootOfTrust `json:"root_of_trust,omitempty"`
}
func (cli *CLI) NewAddMeasurementCmd() *cobra.Command {
return &cobra.Command{
Use: "measurement",
Short: "Add measurement to the platform info file. The value should be in base64. The second parameter is platform_info.json file",
Example: "measurement <measurement> <platform_info.json>",
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
measurement, err := base64.StdEncoding.DecodeString(args[0])
if err != nil {
log.Fatalf("Error could not decode base64: %v", err)
}
attestationConfiguration := AttestationConfiguration{}
manifest, err := os.OpenFile(args[1], os.O_RDWR, filePermision)
if err != nil {
log.Fatalf("Error opening the platform information file: %v", err)
}
defer manifest.Close()
decoder := json.NewDecoder(manifest)
err = decoder.Decode(&attestationConfiguration)
if err != nil {
log.Fatalf("Error decoding the platform information file: %v", err)
}
attestationConfiguration.SNPPolicy.Measurement = measurement
if err = manifest.Truncate(0); err != nil {
log.Fatalf("Error could not truncate platform information JSON file: %v", err)
}
fileJson, err := json.MarshalIndent(attestationConfiguration, "", " ")
if err != nil {
log.Fatalf("Error marshaling the platform information JSON: %v", err)
}
if err = os.WriteFile(manifest.Name(), fileJson, filePermision); err != nil {
log.Fatalf("Error writing into platform information JSON file: %v", err)
}
},
}
}
+19 -4
View File
@@ -3,6 +3,8 @@
package cli
import (
"crypto/x509"
"encoding/pem"
"log"
"os"
@@ -13,13 +15,26 @@ const resultFilePath = "result.bin"
func (cli *CLI) NewResultsCmd() *cobra.Command {
return &cobra.Command{
Use: "result",
Short: "Retrieve computation result file",
Args: cobra.ExactArgs(1),
Use: "result",
Short: "Retrieve computation result file",
Example: "result <private_key_file_path>",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
log.Println("Retrieving computation result file")
result, err := cli.agentSDK.Result(cmd.Context(), args[0])
privKeyFile, err := os.ReadFile(args[0])
if err != nil {
log.Fatalf("Error reading private key file: %v", err)
}
pemBlock, _ := pem.Decode(privKeyFile)
privKey, err := x509.ParsePKCS1PrivateKey(pemBlock.Bytes)
if err != nil {
log.Fatalf("Error parsing private key: %v", err)
}
result, err := cli.agentSDK.Result(cmd.Context(), privKey)
if err != nil {
log.Fatalf("Error retrieving computation result: %v", err)
}
+3 -5
View File
@@ -2,15 +2,13 @@
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"github.com/ultravioletrs/cocos/agent"
)
import "github.com/ultravioletrs/cocos/pkg/sdk"
type CLI struct {
agentSDK agent.Service
agentSDK sdk.SDK
}
func New(agentSDK agent.Service) *CLI {
func New(agentSDK sdk.SDK) *CLI {
return &CLI{
agentSDK: agentSDK,
}
+25 -7
View File
@@ -6,6 +6,7 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"log"
"log/slog"
@@ -13,6 +14,7 @@ import (
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/api"
agentgrpc "github.com/ultravioletrs/cocos/agent/api/grpc"
"github.com/ultravioletrs/cocos/agent/auth"
"github.com/ultravioletrs/cocos/agent/events"
"github.com/ultravioletrs/cocos/internal"
agentlogger "github.com/ultravioletrs/cocos/internal/logger"
@@ -53,7 +55,7 @@ func main() {
eventSvc, err := events.New(svcName, cfg.ID, manager.ManagerVsockPort)
if err != nil {
log.Printf("failed to create events service %s", err.Error())
logger.Error(fmt.Sprintf("failed to create events service %s", err.Error()))
return
}
defer eventSvc.Close()
@@ -66,13 +68,20 @@ func main() {
KeyFile: cfg.AgentConfig.KeyFile,
ServerCAFile: cfg.AgentConfig.ServerCAFile,
ClientCAFile: cfg.AgentConfig.ClientCAFile,
AttestedTLS: cfg.AgentConfig.AttestedTls,
}
registerAgentServiceServer := func(srv *grpc.Server) {
reflection.Register(srv)
agent.RegisterAgentServiceServer(srv, agentgrpc.NewServer(svc))
}
gs := grpcserver.New(ctx, cancel, svcName, grpcServerConfig, registerAgentServiceServer, logger)
authSvc, err := auth.New(cfg)
if err != nil {
logger.Error(fmt.Sprintf("failed to create auth service %s", err.Error()))
return
}
gs := grpcserver.New(ctx, cancel, svcName, grpcServerConfig, registerAgentServiceServer, logger, svc, authSvc)
g.Go(func() error {
return gs.Start()
@@ -108,15 +117,24 @@ func readConfig() (agent.Computation, error) {
return agent.Computation{}, err
}
defer conn.Close()
b := make([]byte, 1024)
n, err := conn.Read(b)
if err != nil {
return agent.Computation{}, err
var buffer []byte
for {
chunk := make([]byte, 1024)
n, err := conn.Read(chunk)
if err != nil {
if err == io.EOF {
break
}
return agent.Computation{}, err
}
buffer = append(buffer, chunk[:n]...)
}
ac := agent.Computation{
AgentConfig: agent.AgentConfig{},
}
if err := json.Unmarshal(b[:n], &ac); err != nil {
if err := json.Unmarshal(buffer, &ac); err != nil {
return agent.Computation{}, err
}
if ac.AgentConfig.LogLevel == "" {
+5 -2
View File
@@ -24,7 +24,7 @@ const (
)
type config struct {
LogLevel string `env:"AGENT_LOG_LEVEL" envDefault:"info"`
LogLevel string `env:"AGENT_LOG_LEVEL" envDefault:"info"`
}
func main() {
@@ -84,11 +84,14 @@ func main() {
}
// Agent Commands
rootCmd.AddCommand(cliSVC.NewAlgorithmsCmd())
rootCmd.AddCommand(cliSVC.NewAlgorithmCmd())
rootCmd.AddCommand(cliSVC.NewDatasetsCmd())
rootCmd.AddCommand(cliSVC.NewResultsCmd())
attestaionCmd := cliSVC.NewAttestationCmd()
rootCmd.AddCommand(attestaionCmd)
rootCmd.AddCommand(cliSVC.NewFileHashCmd())
rootCmd.AddCommand(cliSVC.NewAddMeasurementCmd())
rootCmd.AddCommand(cliSVC.NewKeysCmd())
// Attestation commands
attestaionCmd.AddCommand(cliSVC.NewGetAttestationCmd())
+5 -5
View File
@@ -35,13 +35,13 @@ const (
)
type config struct {
LogLevel string `env:"MANAGER_LOG_LEVEL" envDefault:"info"`
JaegerURL string `env:"COCOS_JAEGER_URL" envDefault:"http://localhost:14268/api/traces"`
InstanceID string `env:"MANAGER_INSTANCE_ID" envDefault:""`
LogLevel string `env:"MANAGER_LOG_LEVEL" envDefault:"info"`
JaegerURL string `env:"COCOS_JAEGER_URL" envDefault:"http://localhost:14268/api/traces"`
InstanceID string `env:"MANAGER_INSTANCE_ID" envDefault:""`
}
func main() {
ctx := context.Background()
ctx, cancel := context.WithCancel(context.Background())
g, ctx := errgroup.WithContext(ctx)
var cfg config
@@ -110,7 +110,7 @@ func main() {
mc := managerapi.NewClient(pc, svc, eventsChan)
g.Go(func() error {
return mc.Process(ctx)
return mc.Process(ctx, cancel)
})
if err := g.Wait(); err != nil {
+1 -1
View File
@@ -1,6 +1,6 @@
module github.com/ultravioletrs/cocos
go 1.21.6
go 1.22.0
require (
github.com/absmach/magistrala v0.0.0-20240119191055-d95283d31472
+1 -4
View File
@@ -7,6 +7,7 @@ BR2_TARGET_GENERIC_ISSUE="Welcome to Cocos"
BR2_PACKAGE_DHCP=y
BR2_PACKAGE_DHCP_CLIENT=y
BR2_INIT_SYSTEMD=y
BR2_SYSTEM_BIN_SH_BASH=y
# Filesystem
# BR2_TARGET_ROOTFS_TAR is not set
@@ -42,7 +43,3 @@ BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF=y
# host-qemu for gitlab testing
BR2_PACKAGE_HOST_QEMU=y
BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
# Python
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON_PIP=y
+1 -1
View File
@@ -3,6 +3,6 @@ config BR2_PACKAGE_AGENT
default y
help
Confidential Computing Agent is a state machine capable of
receiving data and algorithms, running computations, and
receiving datasets and algorithm, running computations, and
fetching the attestation report from within the
Confidential VM.
+4 -3
View File
@@ -12,14 +12,15 @@ define AGENT_BUILD_CMDS
endef
define AGENT_INSTALL_TARGET_CMDS
$(INSTALL) -D -m 0755 $(@D)/build/cocos-agent $(TARGET_DIR)/bin
$(INSTALL) -D -m 0750 $(@D)/build/cocos-agent $(TARGET_DIR)/bin
mkdir -p $(TARGET_DIR)/var/log/cocos
mkdir -p $(TARGET_DIR)/cocos/
endef
define AGENT_INSTALL_INIT_SYSTEMD
$(INSTALL) -D -m 0644 $(@D)/init/systemd/cocos-agent.service $(TARGET_DIR)/usr/lib/systemd/system/cocos-agent.service
$(INSTALL) -D -m 0755 $(@D)/init/systemd/agent_start_script.sh $(TARGET_DIR)/cocos/agent_start_script.sh
$(INSTALL) -D -m 0640 $(@D)/init/systemd/cocos-agent.service $(TARGET_DIR)/usr/lib/systemd/system/cocos-agent.service
$(INSTALL) -D -m 0750 $(@D)/init/systemd/cocos_network_setup.sh $(TARGET_DIR)/cocos/cocos_network_setup.sh
$(INSTALL) -D -m 0750 $(@D)/init/systemd/agent_start_script.sh $(TARGET_DIR)/cocos/agent_start_script.sh
endef
$(eval $(golang-package))
+9 -14
View File
@@ -1,21 +1,16 @@
#!/bin/sh
# The variable ETH_IFACE contains the name the systemd gave to the network interface.
# The systemd configures the name based on the QEMU parameters.
# The parts of the name enp0s2 mean:
# et - ethernet card. It means this is the ethernet interface.
# p - means that the interface is connected to a PCI bus.
# 0 - the interface is connected to bus 0.
# s2 -the interface is connected to slot 2.
NUM_OF_PERMITED_IFACE=1
# The variable ETH_IFACE value must match the name configured through QEMU parameters for the network device.
# The bus number and slot number are configured through QEMU device parameters, parameters
# addr (for slot number), and bus (for bus number).
ETH_IFACE=enp0s2
NUM_OF_IFACE=$(ip route | grep -Eo 'dev [a-z0-9]+' | awk '{ print $2 }' | sort | uniq | wc -l)
ip link set dev $ETH_IFACE up
dhclient $ETH_IFACE
AGENT_GRPC_HOST=$(ip -4 addr show $ETH_IFACE | grep inet | awk '{print $2}' | cut -d/ -f1)
if [ $NUM_OF_IFACE -gt $NUM_OF_PERMITED_IFACE ]; then
echo "More then one network interface in the VM"
exit 1
fi
DEFAULT_IFACE=$(route | grep '^default' | grep -o '[^ ]*$')
AGENT_GRPC_HOST=$(ip -4 addr show $DEFAULT_IFACE | grep inet | awk '{print $2}' | cut -d/ -f1)
export AGENT_GRPC_HOST
+1
View File
@@ -9,6 +9,7 @@ StandardError=file:/var/log/cocos/agent.stderr
Environment=AGENT_GRPC_PORT=7002
Environment=AGENT_LOG_LEVEL=info
ExecStartPre=/cocos/cocos_network_setup.sh
ExecStart=/cocos/agent_start_script.sh
[Install]
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
# IFACES are all network interfaces excluding lo (LOOPBACK) and sit interfaces
IFACES=$(ip link show | grep -vE 'LOOPBACK|sit*' | awk -F': ' '{print $2}')
# This for loop brings up all network interfaces in IFACES and dhclient obtains an IP address for the every interface
for IFACE in $IFACES; do
STATE=$(ip link show $IFACE | grep DOWN)
if [ -n "$STATE" ]; then
ip link set $IFACE up
fi
IP_ADDR=$(ip addr show $IFACE | grep 'inet ')
if [ -z "$IP_ADDR" ]; then
dhclient $IFACE
fi
done
+113 -2
View File
@@ -5,36 +5,57 @@ package grpc
import (
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/asn1"
"encoding/pem"
"fmt"
"log/slog"
"math/big"
"net"
"os"
"time"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/auth"
"github.com/ultravioletrs/cocos/internal/server"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"golang.org/x/crypto/sha3"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/credentials/insecure"
)
const (
stopWaitTime = 5 * time.Second
stopWaitTime = 5 * time.Second
organization = "Ultraviolet"
country = "Serbia"
province = ""
locality = "Belgrade"
streetAddress = "Bulevar Arsenija Carnojevica 103"
postalCode = "11000"
notAfterYear = 1
notAfterMonth = 0
notAfterDay = 0
)
type Server struct {
server.BaseServer
server *grpc.Server
registerService serviceRegister
agent agent.Service
authSvc *auth.Service
}
type serviceRegister func(srv *grpc.Server)
var _ server.Server = (*Server)(nil)
func New(ctx context.Context, cancel context.CancelFunc, name string, config server.Config, registerService serviceRegister, logger *slog.Logger) server.Server {
func New(ctx context.Context, cancel context.CancelFunc, name string, config server.Config, registerService serviceRegister, logger *slog.Logger, agentSvc agent.Service, authSvc *auth.Service) server.Server {
listenFullAddress := fmt.Sprintf("%s:%s", config.Host, config.Port)
return &Server{
BaseServer: server.BaseServer{
@@ -46,6 +67,8 @@ func New(ctx context.Context, cancel context.CancelFunc, name string, config ser
Logger: logger,
},
registerService: registerService,
agent: agentSvc,
authSvc: authSvc,
}
}
@@ -55,6 +78,11 @@ func (s *Server) Start() error {
grpc.StatsHandler(otelgrpc.NewServerHandler()),
}
if s.authSvc != nil {
grpcServerOptions = append(grpcServerOptions, grpc.UnaryInterceptor(s.authSvc.AuthUnaryInterceptor()))
grpcServerOptions = append(grpcServerOptions, grpc.StreamInterceptor(s.authSvc.AuthStreamInterceptor()))
}
listener, err := net.Listen("tcp", s.Address)
if err != nil {
return fmt.Errorf("failed to listen on port %s: %w", s.Address, err)
@@ -62,6 +90,24 @@ func (s *Server) Start() error {
creds := grpc.Creds(insecure.NewCredentials())
switch {
case s.Config.AttestedTLS:
certificateBytes, privateKeyBytes, err := generateCertificatesForATLS(s.agent)
if err != nil {
return fmt.Errorf("failed to create certificate: %w", err)
}
certificate, err := tls.X509KeyPair(certificateBytes, privateKeyBytes)
if err != nil {
return fmt.Errorf("falied due to invalid key pair: %w", err)
}
tlsConfig := &tls.Config{
ClientAuth: tls.NoClientCert,
Certificates: []tls.Certificate{certificate},
}
creds = grpc.Creds(credentials.NewTLS(tlsConfig))
s.Logger.Info(fmt.Sprintf("%s service gRPC server listening at %s with Attested TLS", s.Name, s.Address))
case s.Config.CertFile != "" || s.Config.KeyFile != "":
certificate, err := loadX509KeyPair(s.Config.CertFile, s.Config.KeyFile)
if err != nil {
@@ -179,3 +225,68 @@ func loadX509KeyPair(certfile, keyfile string) (tls.Certificate, error) {
}
return tls.X509KeyPair(cert, key)
}
func generateCertificatesForATLS(svc agent.Service) ([]byte, []byte, error) {
curve := elliptic.P256()
privateKey, err := ecdsa.GenerateKey(curve, rand.Reader)
if err != nil {
return nil, nil, fmt.Errorf("failed to generate private/public key: %w", err)
}
publicKeyBytes, err := x509.MarshalPKIXPublicKey(&privateKey.PublicKey)
if err != nil {
return nil, nil, fmt.Errorf("failed to marshal the public key: %w", err)
}
// The Attestation Report will be added as an X.509 certificate extension
attestationReport, err := svc.Attestation(context.Background(), sha3.Sum512(publicKeyBytes))
if err != nil {
return nil, nil, fmt.Errorf("failed to fetch the attestation report: %w", err)
}
certTemplate := &x509.Certificate{
SerialNumber: big.NewInt(202403311),
Subject: pkix.Name{
Organization: []string{organization},
Country: []string{country},
Province: []string{province},
Locality: []string{locality},
StreetAddress: []string{streetAddress},
PostalCode: []string{postalCode},
},
NotBefore: time.Now(),
NotAfter: time.Now().AddDate(notAfterYear, notAfterMonth, notAfterDay),
KeyUsage: x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
BasicConstraintsValid: true,
ExtraExtensions: []pkix.Extension{
{
Id: asn1.ObjectIdentifier{1, 2, 3, 4, 5, 6},
Critical: false,
Value: attestationReport,
},
},
}
certDERBytes, err := x509.CreateCertificate(rand.Reader, certTemplate, certTemplate, &privateKey.PublicKey, privateKey)
if err != nil {
return nil, nil, fmt.Errorf("failed to create certificate: %w", err)
}
certBytes := pem.EncodeToMemory(&pem.Block{
Type: "CERTIFICATE",
Bytes: certDERBytes,
})
privateKeyBytes, err := x509.MarshalPKCS8PrivateKey(privateKey)
if err != nil {
return nil, nil, fmt.Errorf("failed to marshal the private key: %w", err)
}
keyBytes := pem.EncodeToMemory(&pem.Block{
Type: "PRIVATE KEY",
Bytes: privateKeyBytes,
})
return certBytes, keyBytes, nil
}
+7 -6
View File
@@ -17,12 +17,13 @@ type Server interface {
}
type Config struct {
Host string `env:"HOST" envDefault:""`
Port string `env:"PORT" envDefault:""`
CertFile string `env:"SERVER_CERT" envDefault:""`
KeyFile string `env:"SERVER_KEY" envDefault:""`
ServerCAFile string `env:"SERVER_CA_CERTS" envDefault:""`
ClientCAFile string `env:"CLIENT_CA_CERTS" envDefault:""`
Host string `env:"HOST" envDefault:""`
Port string `env:"PORT" envDefault:""`
CertFile string `env:"SERVER_CERT" envDefault:""`
KeyFile string `env:"SERVER_KEY" envDefault:""`
ServerCAFile string `env:"SERVER_CA_CERTS" envDefault:""`
ClientCAFile string `env:"CLIENT_CA_CERTS" envDefault:""`
AttestedTLS bool `env:"ATTESTED_TLS" envDefault:"false"`
}
type BaseServer struct {
+6 -4
View File
@@ -176,6 +176,8 @@ MANAGER_QEMU_SEV_CBITPOS=51 \
The kernel hash feature might not work with the current build of OVMF and QEMU. If so, build the host kernel, QEMU, and OVMF from the [AMD SEV GitHub](https://github.com/AMDESE/AMDSEV/tree/snp-latest) repository.
To build the OVMF with the kernel hash capability, we must build the AmdSev package of OVMF. The result of the build should be a single `OVMF.fd` file (unlike the regular two OVFM files). The OVMF package is located at `OvmfPkg/AmdSev/AmdSevX64.dsc`.
To enable [AMD SEV-SNP](https://www.amd.com/en/developer/sev.html) support, start manager like this
```sh
@@ -185,7 +187,7 @@ MANAGER_QEMU_ENABLE_SEV=false \
MANAGER_QEMU_ENABLE_SEV_SNP=true \
MANAGER_QEMU_SEV_CBITPOS=51 \
MANAGER_QEMU_BIN_PATH=<path to QEMU binary> \
MANAGER_QEMU_QEMU_OVMF_CODE_FILE=<path to OVMF Amd Sev built package> \
MANAGER_QEMU_QEMU_OVMF_CODE_FILE=<path to OVMF.fd Amd Sev built package> \
./build/cocos-manager
```
@@ -203,10 +205,10 @@ MANAGER_QEMU_KERNEL_HASH=true \
### Verifying VM launch
NB: To verify that the manager successfully launched the VM, you need to open three terminals on the same machine. In one terminal, you need to launch the Manager test server by executing (with the environment variables of choice):
NB: To verify that the manager successfully launched the VM, you need to open three terminals on the same machine. In one terminal, you need to launch the computations server by executing (with the environment variables of choice):
```bash
go run ./test/manager-server/main.go
go run ./test/computations/main.go <dataset path> <algo path>
```
and in the second the manager by executing (with the environment variables of choice):
@@ -215,7 +217,7 @@ and in the second the manager by executing (with the environment variables of ch
go run ./cmd/manager/main.go
```
Ensure that the Manager can connect to the Manager test server by setting the MANAGER_GRPC_PORT with the port value of the Manager test server. The Manager test server is listening on the default value of the MANAGER_GRPC_PORT. In the last one, you can run the verification commands.
Ensure that the Manager can connect to the Manager test server by setting the MANAGER_GRPC_PORT with the port value of the Manager test server. In the last terminal, you can run the verification commands.
To verify that the manager launched the VM successfully, run the following command:
+25 -12
View File
@@ -4,12 +4,15 @@ package grpc
import (
"context"
"errors"
"github.com/ultravioletrs/cocos/manager"
pkgmanager "github.com/ultravioletrs/cocos/pkg/manager"
"golang.org/x/sync/errgroup"
)
var errTerminationFromServer = errors.New("server requested client termination")
type ManagerClient struct {
stream pkgmanager.ManagerService_ProcessClient
svc manager.Service
@@ -25,7 +28,7 @@ func NewClient(stream pkgmanager.ManagerService_ProcessClient, svc manager.Servi
}
}
func (client ManagerClient) Process(ctx context.Context) error {
func (client ManagerClient) Process(ctx context.Context, cancel context.CancelFunc) error {
eg, ctx := errgroup.WithContext(ctx)
eg.Go(func() error {
@@ -34,24 +37,34 @@ func (client ManagerClient) Process(ctx context.Context) error {
if err != nil {
return err
}
port, err := client.svc.Run(ctx, req)
if err != nil {
return err
}
runRes := &pkgmanager.ClientStreamMessage_RunRes{RunRes: &pkgmanager.RunResponse{AgentPort: port, ComputationId: req.Id}}
if err := client.stream.Send(&pkgmanager.ClientStreamMessage{Message: runRes}); err != nil {
return err
switch mes := req.Message.(type) {
case *pkgmanager.ServerStreamMessage_RunReq:
port, err := client.svc.Run(ctx, mes.RunReq)
if err != nil {
return err
}
runRes := &pkgmanager.ClientStreamMessage_RunRes{RunRes: &pkgmanager.RunResponse{AgentPort: port, ComputationId: mes.RunReq.Id}}
if err := client.stream.Send(&pkgmanager.ClientStreamMessage{Message: runRes}); err != nil {
return err
}
case *pkgmanager.ServerStreamMessage_TerminateReq:
cancel()
return errors.Join(errTerminationFromServer, errors.New(mes.TerminateReq.Message))
}
}
})
eg.Go(func() error {
for mes := range client.responses {
if err := client.stream.Send(mes); err != nil {
return err
for {
select {
case <-ctx.Done():
return nil
case mes := <-client.responses:
if err := client.stream.Send(mes); err != nil {
return err
}
}
}
return nil
})
return eg.Wait()
+16 -13
View File
@@ -3,40 +3,39 @@
package grpc
import (
"context"
"github.com/ultravioletrs/cocos/pkg/manager"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/peer"
)
var _ manager.ManagerServiceServer = (*grpcServer)(nil)
type grpcServer struct {
manager.UnimplementedManagerServiceServer
incoming chan *manager.ClientStreamMessage
svc Service
ctx context.Context
}
type Service interface {
Run(ipAddress string, runReqChan chan *manager.ComputationRunReq)
Run(ipAddress string, runReqChan chan *manager.ServerStreamMessage, authInfo credentials.AuthInfo)
}
// NewServer returns new AuthServiceServer instance.
func NewServer(ctx context.Context, incoming chan *manager.ClientStreamMessage, svc Service) manager.ManagerServiceServer {
func NewServer(incoming chan *manager.ClientStreamMessage, svc Service) manager.ManagerServiceServer {
return &grpcServer{
incoming: incoming,
svc: svc,
ctx: ctx,
}
}
func (s *grpcServer) Process(stream manager.ManagerService_ProcessServer) error {
runReqChan := make(chan *manager.ComputationRunReq)
runReqChan := make(chan *manager.ServerStreamMessage)
client, ok := peer.FromContext(stream.Context())
if ok {
go s.svc.Run(client.Addr.String(), runReqChan)
go s.svc.Run(client.Addr.String(), runReqChan, client.AuthInfo)
}
eg, _ := errgroup.WithContext(s.ctx)
eg, ctx := errgroup.WithContext(stream.Context())
eg.Go(func() error {
for {
@@ -50,12 +49,16 @@ func (s *grpcServer) Process(stream manager.ManagerService_ProcessServer) error
})
eg.Go(func() error {
for runReq := range runReqChan {
if err := stream.Send(runReq); err != nil {
return err
for {
select {
case <-ctx.Done():
return nil
case req := <-runReqChan:
if err := stream.Send(req); err != nil {
return err
}
}
}
return nil
})
return eg.Wait()
}
+24 -9
View File
@@ -10,7 +10,11 @@ package manager;
option go_package = "./manager";
service ManagerService {
rpc Process(stream ClientStreamMessage) returns (stream ComputationRunReq) {}
rpc Process(stream ClientStreamMessage) returns (stream ServerStreamMessage) {}
}
message Terminate {
string message = 1;
}
message RunResponse{
@@ -42,26 +46,35 @@ message ClientStreamMessage {
}
}
message ServerStreamMessage {
oneof message {
ComputationRunReq runReq = 1;
Terminate terminateReq = 2;
}
}
message ComputationRunReq {
string id = 1;
string name = 2;
string description = 3;
repeated Dataset datasets = 4;
repeated Algorithm algorithms = 5;
repeated string result_consumers = 6;
Algorithm algorithm = 5;
repeated ResultConsumer result_consumers = 6;
AgentConfig agent_config = 7;
}
message ResultConsumer {
bytes userKey = 1;
}
message Dataset {
string provider = 1;
string id = 2;
bytes hash = 3; // should be sha3.Sum256, 32 byte length.
bytes hash = 1; // should be sha3.Sum256, 32 byte length.
bytes userKey = 2;
}
message Algorithm {
string provider = 1;
string id = 2;
bytes hash = 3; // should be sha3.Sum256, 32 byte length.
bytes hash = 1; // should be sha3.Sum256, 32 byte length.
bytes userKey = 2;
}
message AgentConfig {
@@ -72,4 +85,6 @@ message AgentConfig {
string client_ca_file = 5;
string server_ca_file = 6;
string log_level = 7;
bool attested_tls = 8;
}
+2 -2
View File
@@ -55,7 +55,7 @@ type SevConfig struct {
type VSockConfig struct {
ID string `env:"VSOCK_ID" envDefault:"vhost-vsock-pci0"`
GuestCID int `env:"VSOCK_GUEST_CID" envDefault:"3"`
vnc int `env:"VSOCK_VNC" envDefault:"0"`
Vnc int `env:"VSOCK_VNC" envDefault:"0"`
}
type Config struct {
@@ -157,7 +157,7 @@ func constructQemuArgs(config Config) []string {
config.VirtioNetPciConfig.ROMFile))
args = append(args, "-device", fmt.Sprintf("vhost-vsock-pci,id=%s,guest-cid=%d", config.VSockConfig.ID, config.VSockConfig.GuestCID))
args = append(args, "-vnc", fmt.Sprintf(":%d", config.vnc))
args = append(args, "-vnc", fmt.Sprintf(":%d", config.Vnc))
if config.EnableSEVSNP {
args = append(args, "-object",
+13 -11
View File
@@ -18,7 +18,7 @@ const (
)
func CreateVM(ctx context.Context, cfg Config) (*exec.Cmd, error) {
// Create unique emu device identifiers.
// Create unique emu device identifiers
id, err := uuid.NewV4()
if err != nil {
return &exec.Cmd{}, err
@@ -27,18 +27,20 @@ func CreateVM(ctx context.Context, cfg Config) (*exec.Cmd, error) {
qemuCfg.NetDevConfig.ID = fmt.Sprintf("%s-%s", qemuCfg.NetDevConfig.ID, id)
qemuCfg.SevConfig.ID = fmt.Sprintf("%s-%s", qemuCfg.SevConfig.ID, id)
// Copy firmware vars file.
srcFile := qemuCfg.OVMFVarsConfig.File
dstFile := fmt.Sprintf("%s/%s-%s.fd", cfg.TmpFileLoc, firmwareVars, id)
err = internal.CopyFile(srcFile, dstFile)
if err != nil {
return &exec.Cmd{}, err
if !cfg.KernelHash {
// Copy firmware vars file
srcFile := qemuCfg.OVMFVarsConfig.File
dstFile := fmt.Sprintf("%s/%s-%s.fd", cfg.TmpFileLoc, firmwareVars, id)
err = internal.CopyFile(srcFile, dstFile)
if err != nil {
return &exec.Cmd{}, err
}
qemuCfg.OVMFVarsConfig.File = dstFile
}
qemuCfg.OVMFVarsConfig.File = dstFile
// Copy img files.
srcFile = qemuCfg.DiskImgConfig.KernelFile
dstFile = fmt.Sprintf("%s/%s-%s", cfg.TmpFileLoc, KernelFile, id)
// Copy img files
srcFile := qemuCfg.DiskImgConfig.KernelFile
dstFile := fmt.Sprintf("%s/%s-%s", cfg.TmpFileLoc, KernelFile, id)
err = internal.CopyFile(srcFile, dstFile)
if err != nil {
return &exec.Cmd{}, err
+11 -12
View File
@@ -70,10 +70,9 @@ func New(qemuCfg qemu.Config, logger *slog.Logger, eventsChan chan *manager.Clie
func (ms *managerService) Run(ctx context.Context, c *manager.ComputationRunReq) (string, error) {
ms.publishEvent("vm-provision", c.Id, "starting", json.RawMessage{})
ac := agent.Computation{
ID: c.Id,
Name: c.Name,
Description: c.Description,
ResultConsumers: c.ResultConsumers,
ID: c.Id,
Name: c.Name,
Description: c.Description,
AgentConfig: agent.AgentConfig{
Port: c.AgentConfig.Port,
Host: c.AgentConfig.Host,
@@ -84,19 +83,18 @@ func (ms *managerService) Run(ctx context.Context, c *manager.ComputationRunReq)
LogLevel: c.AgentConfig.LogLevel,
},
}
for _, algo := range c.Algorithms {
if len(algo.Hash) != hashLength {
ms.publishEvent("vm-provision", c.Id, "failed", json.RawMessage{})
return "", errInvalidHashLength
}
ac.Algorithms = append(ac.Algorithms, agent.Algorithm{ID: algo.Id, Provider: algo.Provider, Hash: [hashLength]byte(algo.Hash)})
}
ac.Algorithm = agent.Algorithm{Hash: [hashLength]byte(c.Algorithm.Hash), UserKey: c.Algorithm.UserKey}
for _, data := range c.Datasets {
if len(data.Hash) != hashLength {
ms.publishEvent("vm-provision", c.Id, "failed", json.RawMessage{})
return "", errInvalidHashLength
}
ac.Datasets = append(ac.Datasets, agent.Dataset{ID: data.Id, Provider: data.Provider, Hash: [hashLength]byte(data.Hash)})
ac.Datasets = append(ac.Datasets, agent.Dataset{Hash: [hashLength]byte(data.Hash), UserKey: data.UserKey})
}
for _, rc := range c.ResultConsumers {
ac.ResultConsumers = append(ac.ResultConsumers, agent.ResultConsumer{UserKey: rc.UserKey})
}
agentPort, err := getFreePort()
@@ -121,6 +119,7 @@ func (ms *managerService) Run(ctx context.Context, c *manager.ComputationRunReq)
return "", err
}
ms.qemuCfg.VSockConfig.GuestCID++
ms.qemuCfg.VSockConfig.Vnc++
ms.publishEvent("vm-provision", c.Id, "complete", json.RawMessage{})
return fmt.Sprint(ms.qemuCfg.HostFwdAgent), nil
+1 -2
View File
@@ -4,7 +4,6 @@ package agent
import (
"github.com/ultravioletrs/cocos/agent"
agentapi "github.com/ultravioletrs/cocos/agent/api/grpc"
"github.com/ultravioletrs/cocos/pkg/clients/grpc"
)
@@ -15,5 +14,5 @@ func NewAgentClient(cfg grpc.Config) (grpc.Client, agent.AgentServiceClient, err
return nil, nil, err
}
return client, agentapi.NewClient(client.Connection(), cfg.Timeout), nil
return client, agent.NewAgentServiceClient(client.Connection()), nil
}
+163 -25
View File
@@ -5,12 +5,20 @@ package grpc
import (
"crypto/tls"
"crypto/x509"
"encoding/asn1"
"encoding/json"
"fmt"
"os"
"time"
"github.com/absmach/magistrala/pkg/errors"
"github.com/google/go-sev-guest/abi"
"github.com/google/go-sev-guest/proto/check"
"github.com/google/go-sev-guest/validate"
"github.com/google/go-sev-guest/verify"
"github.com/google/go-sev-guest/verify/trust"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"golang.org/x/crypto/sha3"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/credentials/insecure"
@@ -25,8 +33,22 @@ const (
)
var (
errGrpcConnect = errors.New("failed to connect to grpc server")
errGrpcClose = errors.New("failed to close grpc connection")
errGrpcConnect = errors.New("failed to connect to grpc server")
errGrpcClose = errors.New("failed to close grpc connection")
errManifestOpen = errors.New("failed to open Manifest")
errManifestMissing = errors.New("failed due to missing Manifest")
errManifestDecode = errors.New("failed to decode Manifest json")
errCertificateParse = errors.New("failed to parse x509 certificate")
errAttVerification = errors.New("attestation verification failed")
errAttValidation = errors.New("attestation validation failed")
errCustomExtension = errors.New("failed due to missing custom extension")
)
var (
customSEVSNPExtensionOID = asn1.ObjectIdentifier{1, 2, 3, 4, 5, 6}
attestationConfiguration = AttestationConfiguration{}
timeout = time.Minute * 2
maxTryDelay = time.Second * 30
)
type Config struct {
@@ -35,6 +57,13 @@ type Config struct {
ServerCAFile string `env:"SERVER_CA_CERTS" envDefault:""`
URL string `env:"URL" envDefault:"localhost:7001"`
Timeout time.Duration `env:"TIMEOUT" envDefault:"60s"`
AttestedTLS bool `env:"ATTESTED_TLS" envDefault:"false"`
Manifest string `env:"MANIFEST" envDefault:""`
}
type AttestationConfiguration struct {
SNPPolicy *check.Policy `json:"snp_policy,omitempty"`
RootOFTrust *check.RootOfTrust `json:"root_of_trust,omitempty"`
}
type Client interface {
@@ -102,34 +131,47 @@ func connect(cfg Config) (*grpc.ClientConn, security, error) {
secure := withoutTLS
tc := insecure.NewCredentials()
if cfg.ServerCAFile != "" {
tlsConfig := &tls.Config{}
// Loading root ca certificates file
rootCA, err := os.ReadFile(cfg.ServerCAFile)
if cfg.AttestedTLS {
err := readManifest(cfg)
if err != nil {
return nil, secure, fmt.Errorf("failed to load root ca file: %w", err)
}
if len(rootCA) > 0 {
capool := x509.NewCertPool()
if !capool.AppendCertsFromPEM(rootCA) {
return nil, secure, fmt.Errorf("failed to append root ca to tls.Config")
}
tlsConfig.RootCAs = capool
secure = withTLS
return nil, secure, fmt.Errorf("failed to read Manifest %w", err)
}
// Loading mtls certificates file
if cfg.ClientCert != "" || cfg.ClientKey != "" {
certificate, err := tls.LoadX509KeyPair(cfg.ClientCert, cfg.ClientKey)
if err != nil {
return nil, secure, fmt.Errorf("failed to client certificate and key %w", err)
}
tlsConfig.Certificates = []tls.Certificate{certificate}
secure = withmTLS
tlsConfig := &tls.Config{
InsecureSkipVerify: true,
VerifyPeerCertificate: verifyAttestationReportTLS,
}
tc = credentials.NewTLS(tlsConfig)
} else {
if cfg.ServerCAFile != "" {
tlsConfig := &tls.Config{}
// Loading root ca certificates file
rootCA, err := os.ReadFile(cfg.ServerCAFile)
if err != nil {
return nil, secure, fmt.Errorf("failed to load root ca file: %w", err)
}
if len(rootCA) > 0 {
capool := x509.NewCertPool()
if !capool.AppendCertsFromPEM(rootCA) {
return nil, secure, fmt.Errorf("failed to append root ca to tls.Config")
}
tlsConfig.RootCAs = capool
secure = withTLS
}
// Loading mTLS certificates file
if cfg.ClientCert != "" || cfg.ClientKey != "" {
certificate, err := tls.LoadX509KeyPair(cfg.ClientCert, cfg.ClientKey)
if err != nil {
return nil, secure, fmt.Errorf("failed to client certificate and key %w", err)
}
tlsConfig.Certificates = []tls.Certificate{certificate}
secure = withmTLS
}
tc = credentials.NewTLS(tlsConfig)
}
}
opts = append(opts, grpc.WithTransportCredentials(tc))
@@ -140,3 +182,99 @@ func connect(cfg Config) (*grpc.ClientConn, security, error) {
}
return conn, secure, nil
}
func readManifest(cfg Config) error {
if cfg.Manifest != "" {
manifest, err := os.Open(cfg.Manifest)
if err != nil {
return errors.Wrap(errManifestOpen, err)
}
defer manifest.Close()
decoder := json.NewDecoder(manifest)
err = decoder.Decode(&attestationConfiguration)
if err != nil {
return errors.Wrap(errManifestDecode, err)
}
return nil
}
return errManifestMissing
}
func verifyAttestationReportTLS(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
cert, err := x509.ParseCertificate(rawCerts[0])
if err != nil {
return errors.Wrap(errCertificateParse, err)
}
for _, ext := range cert.Extensions {
if ext.Id.Equal(customSEVSNPExtensionOID) {
// Check if the certificate is self-signed
err := checkIfCertificateSelfSigned(cert)
if err != nil {
return errors.Wrap(errAttVerification, err)
}
publicKeyBytes, err := x509.MarshalPKIXPublicKey(cert.PublicKey)
if err != nil {
return errors.Wrap(errAttVerification, err)
}
expectedReportData := sha3.Sum512(publicKeyBytes)
attestationConfiguration.SNPPolicy.ReportData = expectedReportData[:]
// Attestation verification and validation
sopts, err := verify.RootOfTrustToOptions(attestationConfiguration.RootOFTrust)
if err != nil {
return errors.Wrap(errAttVerification, err)
}
sopts.Product = attestationConfiguration.SNPPolicy.Product
sopts.Getter = &trust.RetryHTTPSGetter{
Timeout: timeout,
MaxRetryDelay: maxTryDelay,
Getter: &trust.SimpleHTTPSGetter{},
}
attestationPB, err := abi.ReportCertsToProto(ext.Value)
if err != nil {
return errors.Wrap(errAttVerification, err)
}
if err = verify.SnpAttestation(attestationPB, sopts); err != nil {
return errors.Wrap(errAttVerification, err)
}
opts, err := validate.PolicyToOptions(attestationConfiguration.SNPPolicy)
if err != nil {
return errors.Wrap(errAttVerification, err)
}
if err = validate.SnpAttestation(attestationPB, opts); err != nil {
return errors.Wrap(errAttValidation, err)
}
return nil
}
}
return errCustomExtension
}
func checkIfCertificateSelfSigned(cert *x509.Certificate) error {
certPool := x509.NewCertPool()
certPool.AddCert(cert)
opts := x509.VerifyOptions{
Roots: certPool,
CurrentTime: time.Now(),
}
if _, err := cert.Verify(opts); err != nil {
return err
}
return nil
}
+420 -191
View File
@@ -3,7 +3,7 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.31.0
// protoc-gen-go v1.33.0
// protoc v4.25.3
// source: manager/manager.proto
@@ -24,6 +24,53 @@ const (
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type Terminate struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Message string `protobuf:"bytes,1,opt,name=message,proto3" json:"message,omitempty"`
}
func (x *Terminate) Reset() {
*x = Terminate{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *Terminate) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Terminate) ProtoMessage() {}
func (x *Terminate) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Terminate.ProtoReflect.Descriptor instead.
func (*Terminate) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{0}
}
func (x *Terminate) GetMessage() string {
if x != nil {
return x.Message
}
return ""
}
type RunResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -36,7 +83,7 @@ type RunResponse struct {
func (x *RunResponse) Reset() {
*x = RunResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[0]
mi := &file_manager_manager_proto_msgTypes[1]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -49,7 +96,7 @@ func (x *RunResponse) String() string {
func (*RunResponse) ProtoMessage() {}
func (x *RunResponse) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[0]
mi := &file_manager_manager_proto_msgTypes[1]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -62,7 +109,7 @@ func (x *RunResponse) ProtoReflect() protoreflect.Message {
// Deprecated: Use RunResponse.ProtoReflect.Descriptor instead.
func (*RunResponse) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{0}
return file_manager_manager_proto_rawDescGZIP(), []int{1}
}
func (x *RunResponse) GetAgentPort() string {
@@ -95,7 +142,7 @@ type AgentEvent struct {
func (x *AgentEvent) Reset() {
*x = AgentEvent{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[1]
mi := &file_manager_manager_proto_msgTypes[2]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -108,7 +155,7 @@ func (x *AgentEvent) String() string {
func (*AgentEvent) ProtoMessage() {}
func (x *AgentEvent) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[1]
mi := &file_manager_manager_proto_msgTypes[2]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -121,7 +168,7 @@ func (x *AgentEvent) ProtoReflect() protoreflect.Message {
// Deprecated: Use AgentEvent.ProtoReflect.Descriptor instead.
func (*AgentEvent) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{1}
return file_manager_manager_proto_rawDescGZIP(), []int{2}
}
func (x *AgentEvent) GetEventType() string {
@@ -180,7 +227,7 @@ type AgentLog struct {
func (x *AgentLog) Reset() {
*x = AgentLog{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[2]
mi := &file_manager_manager_proto_msgTypes[3]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -193,7 +240,7 @@ func (x *AgentLog) String() string {
func (*AgentLog) ProtoMessage() {}
func (x *AgentLog) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[2]
mi := &file_manager_manager_proto_msgTypes[3]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -206,7 +253,7 @@ func (x *AgentLog) ProtoReflect() protoreflect.Message {
// Deprecated: Use AgentLog.ProtoReflect.Descriptor instead.
func (*AgentLog) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{2}
return file_manager_manager_proto_rawDescGZIP(), []int{3}
}
func (x *AgentLog) GetMessage() string {
@@ -253,7 +300,7 @@ type ClientStreamMessage struct {
func (x *ClientStreamMessage) Reset() {
*x = ClientStreamMessage{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[3]
mi := &file_manager_manager_proto_msgTypes[4]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -266,7 +313,7 @@ func (x *ClientStreamMessage) String() string {
func (*ClientStreamMessage) ProtoMessage() {}
func (x *ClientStreamMessage) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[3]
mi := &file_manager_manager_proto_msgTypes[4]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -279,7 +326,7 @@ func (x *ClientStreamMessage) ProtoReflect() protoreflect.Message {
// Deprecated: Use ClientStreamMessage.ProtoReflect.Descriptor instead.
func (*ClientStreamMessage) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{3}
return file_manager_manager_proto_rawDescGZIP(), []int{4}
}
func (m *ClientStreamMessage) GetMessage() isClientStreamMessage_Message {
@@ -332,24 +379,105 @@ func (*ClientStreamMessage_AgentEvent) isClientStreamMessage_Message() {}
func (*ClientStreamMessage_RunRes) isClientStreamMessage_Message() {}
type ServerStreamMessage struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// Types that are assignable to Message:
//
// *ServerStreamMessage_RunReq
// *ServerStreamMessage_TerminateReq
Message isServerStreamMessage_Message `protobuf_oneof:"message"`
}
func (x *ServerStreamMessage) Reset() {
*x = ServerStreamMessage{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[5]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *ServerStreamMessage) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*ServerStreamMessage) ProtoMessage() {}
func (x *ServerStreamMessage) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[5]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use ServerStreamMessage.ProtoReflect.Descriptor instead.
func (*ServerStreamMessage) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{5}
}
func (m *ServerStreamMessage) GetMessage() isServerStreamMessage_Message {
if m != nil {
return m.Message
}
return nil
}
func (x *ServerStreamMessage) GetRunReq() *ComputationRunReq {
if x, ok := x.GetMessage().(*ServerStreamMessage_RunReq); ok {
return x.RunReq
}
return nil
}
func (x *ServerStreamMessage) GetTerminateReq() *Terminate {
if x, ok := x.GetMessage().(*ServerStreamMessage_TerminateReq); ok {
return x.TerminateReq
}
return nil
}
type isServerStreamMessage_Message interface {
isServerStreamMessage_Message()
}
type ServerStreamMessage_RunReq struct {
RunReq *ComputationRunReq `protobuf:"bytes,1,opt,name=runReq,proto3,oneof"`
}
type ServerStreamMessage_TerminateReq struct {
TerminateReq *Terminate `protobuf:"bytes,2,opt,name=terminateReq,proto3,oneof"`
}
func (*ServerStreamMessage_RunReq) isServerStreamMessage_Message() {}
func (*ServerStreamMessage_TerminateReq) isServerStreamMessage_Message() {}
type ComputationRunReq struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
Description string `protobuf:"bytes,3,opt,name=description,proto3" json:"description,omitempty"`
Datasets []*Dataset `protobuf:"bytes,4,rep,name=datasets,proto3" json:"datasets,omitempty"`
Algorithms []*Algorithm `protobuf:"bytes,5,rep,name=algorithms,proto3" json:"algorithms,omitempty"`
ResultConsumers []string `protobuf:"bytes,6,rep,name=result_consumers,json=resultConsumers,proto3" json:"result_consumers,omitempty"`
AgentConfig *AgentConfig `protobuf:"bytes,7,opt,name=agent_config,json=agentConfig,proto3" json:"agent_config,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
Description string `protobuf:"bytes,3,opt,name=description,proto3" json:"description,omitempty"`
Datasets []*Dataset `protobuf:"bytes,4,rep,name=datasets,proto3" json:"datasets,omitempty"`
Algorithm *Algorithm `protobuf:"bytes,5,opt,name=algorithm,proto3" json:"algorithm,omitempty"`
ResultConsumers []*ResultConsumer `protobuf:"bytes,6,rep,name=result_consumers,json=resultConsumers,proto3" json:"result_consumers,omitempty"`
AgentConfig *AgentConfig `protobuf:"bytes,7,opt,name=agent_config,json=agentConfig,proto3" json:"agent_config,omitempty"`
}
func (x *ComputationRunReq) Reset() {
*x = ComputationRunReq{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[4]
mi := &file_manager_manager_proto_msgTypes[6]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -362,7 +490,7 @@ func (x *ComputationRunReq) String() string {
func (*ComputationRunReq) ProtoMessage() {}
func (x *ComputationRunReq) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[4]
mi := &file_manager_manager_proto_msgTypes[6]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -375,7 +503,7 @@ func (x *ComputationRunReq) ProtoReflect() protoreflect.Message {
// Deprecated: Use ComputationRunReq.ProtoReflect.Descriptor instead.
func (*ComputationRunReq) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{4}
return file_manager_manager_proto_rawDescGZIP(), []int{6}
}
func (x *ComputationRunReq) GetId() string {
@@ -406,14 +534,14 @@ func (x *ComputationRunReq) GetDatasets() []*Dataset {
return nil
}
func (x *ComputationRunReq) GetAlgorithms() []*Algorithm {
func (x *ComputationRunReq) GetAlgorithm() *Algorithm {
if x != nil {
return x.Algorithms
return x.Algorithm
}
return nil
}
func (x *ComputationRunReq) GetResultConsumers() []string {
func (x *ComputationRunReq) GetResultConsumers() []*ResultConsumer {
if x != nil {
return x.ResultConsumers
}
@@ -427,20 +555,66 @@ func (x *ComputationRunReq) GetAgentConfig() *AgentConfig {
return nil
}
type ResultConsumer struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
UserKey []byte `protobuf:"bytes,1,opt,name=userKey,proto3" json:"userKey,omitempty"`
}
func (x *ResultConsumer) Reset() {
*x = ResultConsumer{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[7]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *ResultConsumer) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*ResultConsumer) ProtoMessage() {}
func (x *ResultConsumer) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[7]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use ResultConsumer.ProtoReflect.Descriptor instead.
func (*ResultConsumer) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{7}
}
func (x *ResultConsumer) GetUserKey() []byte {
if x != nil {
return x.UserKey
}
return nil
}
type Dataset struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Provider string `protobuf:"bytes,1,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,2,opt,name=id,proto3" json:"id,omitempty"`
Hash []byte `protobuf:"bytes,3,opt,name=hash,proto3" json:"hash,omitempty"` // should be sha3.Sum256, 32 byte length.
Hash []byte `protobuf:"bytes,1,opt,name=hash,proto3" json:"hash,omitempty"` // should be sha3.Sum256, 32 byte length.
UserKey []byte `protobuf:"bytes,2,opt,name=userKey,proto3" json:"userKey,omitempty"`
}
func (x *Dataset) Reset() {
*x = Dataset{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[5]
mi := &file_manager_manager_proto_msgTypes[8]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -453,7 +627,7 @@ func (x *Dataset) String() string {
func (*Dataset) ProtoMessage() {}
func (x *Dataset) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[5]
mi := &file_manager_manager_proto_msgTypes[8]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -466,21 +640,7 @@ func (x *Dataset) ProtoReflect() protoreflect.Message {
// Deprecated: Use Dataset.ProtoReflect.Descriptor instead.
func (*Dataset) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{5}
}
func (x *Dataset) GetProvider() string {
if x != nil {
return x.Provider
}
return ""
}
func (x *Dataset) GetId() string {
if x != nil {
return x.Id
}
return ""
return file_manager_manager_proto_rawDescGZIP(), []int{8}
}
func (x *Dataset) GetHash() []byte {
@@ -490,20 +650,26 @@ func (x *Dataset) GetHash() []byte {
return nil
}
func (x *Dataset) GetUserKey() []byte {
if x != nil {
return x.UserKey
}
return nil
}
type Algorithm struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Provider string `protobuf:"bytes,1,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,2,opt,name=id,proto3" json:"id,omitempty"`
Hash []byte `protobuf:"bytes,3,opt,name=hash,proto3" json:"hash,omitempty"` // should be sha3.Sum256, 32 byte length.
Hash []byte `protobuf:"bytes,1,opt,name=hash,proto3" json:"hash,omitempty"` // should be sha3.Sum256, 32 byte length.
UserKey []byte `protobuf:"bytes,2,opt,name=userKey,proto3" json:"userKey,omitempty"`
}
func (x *Algorithm) Reset() {
*x = Algorithm{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[6]
mi := &file_manager_manager_proto_msgTypes[9]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -516,7 +682,7 @@ func (x *Algorithm) String() string {
func (*Algorithm) ProtoMessage() {}
func (x *Algorithm) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[6]
mi := &file_manager_manager_proto_msgTypes[9]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -529,21 +695,7 @@ func (x *Algorithm) ProtoReflect() protoreflect.Message {
// Deprecated: Use Algorithm.ProtoReflect.Descriptor instead.
func (*Algorithm) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{6}
}
func (x *Algorithm) GetProvider() string {
if x != nil {
return x.Provider
}
return ""
}
func (x *Algorithm) GetId() string {
if x != nil {
return x.Id
}
return ""
return file_manager_manager_proto_rawDescGZIP(), []int{9}
}
func (x *Algorithm) GetHash() []byte {
@@ -553,6 +705,13 @@ func (x *Algorithm) GetHash() []byte {
return nil
}
func (x *Algorithm) GetUserKey() []byte {
if x != nil {
return x.UserKey
}
return nil
}
type AgentConfig struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -565,12 +724,13 @@ type AgentConfig struct {
ClientCaFile string `protobuf:"bytes,5,opt,name=client_ca_file,json=clientCaFile,proto3" json:"client_ca_file,omitempty"`
ServerCaFile string `protobuf:"bytes,6,opt,name=server_ca_file,json=serverCaFile,proto3" json:"server_ca_file,omitempty"`
LogLevel string `protobuf:"bytes,7,opt,name=log_level,json=logLevel,proto3" json:"log_level,omitempty"`
AttestedTls bool `protobuf:"varint,8,opt,name=attested_tls,json=attestedTls,proto3" json:"attested_tls,omitempty"`
}
func (x *AgentConfig) Reset() {
*x = AgentConfig{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[7]
mi := &file_manager_manager_proto_msgTypes[10]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -583,7 +743,7 @@ func (x *AgentConfig) String() string {
func (*AgentConfig) ProtoMessage() {}
func (x *AgentConfig) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[7]
mi := &file_manager_manager_proto_msgTypes[10]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -596,7 +756,7 @@ func (x *AgentConfig) ProtoReflect() protoreflect.Message {
// Deprecated: Use AgentConfig.ProtoReflect.Descriptor instead.
func (*AgentConfig) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{7}
return file_manager_manager_proto_rawDescGZIP(), []int{10}
}
func (x *AgentConfig) GetPort() string {
@@ -648,6 +808,13 @@ func (x *AgentConfig) GetLogLevel() string {
return ""
}
func (x *AgentConfig) GetAttestedTls() bool {
if x != nil {
return x.AttestedTls
}
return false
}
var File_manager_manager_proto protoreflect.FileDescriptor
var file_manager_manager_proto_rawDesc = []byte{
@@ -655,96 +822,112 @@ var file_manager_manager_proto_rawDesc = []byte{
0x72, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x07, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72,
0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75,
0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x2e, 0x70, 0x72, 0x6f, 0x74,
0x6f, 0x22, 0x53, 0x0a, 0x0b, 0x52, 0x75, 0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x12, 0x1d, 0x0a, 0x0a, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f, 0x70, 0x6f, 0x72, 0x74, 0x18, 0x01,
0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x50, 0x6f, 0x72, 0x74, 0x12,
0x25, 0x0a, 0x0e, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x69,
0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61,
0x74, 0x69, 0x6f, 0x6e, 0x49, 0x64, 0x22, 0xde, 0x01, 0x0a, 0x0a, 0x41, 0x67, 0x65, 0x6e, 0x74,
0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x1d, 0x0a, 0x0a, 0x65, 0x76, 0x65, 0x6e, 0x74, 0x5f, 0x74,
0x79, 0x70, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x65, 0x76, 0x65, 0x6e, 0x74,
0x54, 0x79, 0x70, 0x65, 0x12, 0x38, 0x0a, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d,
0x70, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65,
0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74,
0x61, 0x6d, 0x70, 0x52, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x12, 0x25,
0x0a, 0x0e, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x69, 0x64,
0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74,
0x69, 0x6f, 0x6e, 0x49, 0x64, 0x12, 0x18, 0x0a, 0x07, 0x64, 0x65, 0x74, 0x61, 0x69, 0x6c, 0x73,
0x18, 0x04, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x64, 0x65, 0x74, 0x61, 0x69, 0x6c, 0x73, 0x12,
0x1e, 0x0a, 0x0a, 0x6f, 0x72, 0x69, 0x67, 0x69, 0x6e, 0x61, 0x74, 0x6f, 0x72, 0x18, 0x05, 0x20,
0x01, 0x28, 0x09, 0x52, 0x0a, 0x6f, 0x72, 0x69, 0x67, 0x69, 0x6e, 0x61, 0x74, 0x6f, 0x72, 0x12,
0x16, 0x0a, 0x06, 0x73, 0x74, 0x61, 0x74, 0x75, 0x73, 0x18, 0x06, 0x20, 0x01, 0x28, 0x09, 0x52,
0x06, 0x73, 0x74, 0x61, 0x74, 0x75, 0x73, 0x22, 0x9b, 0x01, 0x0a, 0x08, 0x41, 0x67, 0x65, 0x6e,
0x74, 0x4c, 0x6f, 0x67, 0x12, 0x18, 0x0a, 0x07, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x18,
0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x12, 0x25,
0x0a, 0x0e, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x69, 0x64,
0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74,
0x69, 0x6f, 0x6e, 0x49, 0x64, 0x12, 0x14, 0x0a, 0x05, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x18, 0x03,
0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x12, 0x38, 0x0a, 0x09, 0x74,
0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a,
0x6f, 0x22, 0x25, 0x0a, 0x09, 0x54, 0x65, 0x72, 0x6d, 0x69, 0x6e, 0x61, 0x74, 0x65, 0x12, 0x18,
0x0a, 0x07, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,
0x07, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x22, 0x53, 0x0a, 0x0b, 0x52, 0x75, 0x6e, 0x52,
0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x1d, 0x0a, 0x0a, 0x61, 0x67, 0x65, 0x6e, 0x74,
0x5f, 0x70, 0x6f, 0x72, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x61, 0x67, 0x65,
0x6e, 0x74, 0x50, 0x6f, 0x72, 0x74, 0x12, 0x25, 0x0a, 0x0e, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74,
0x61, 0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0d,
0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x64, 0x22, 0xde, 0x01,
0x0a, 0x0a, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x1d, 0x0a, 0x0a,
0x65, 0x76, 0x65, 0x6e, 0x74, 0x5f, 0x74, 0x79, 0x70, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09,
0x52, 0x09, 0x65, 0x76, 0x65, 0x6e, 0x74, 0x54, 0x79, 0x70, 0x65, 0x12, 0x38, 0x0a, 0x09, 0x74,
0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a,
0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66,
0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x52, 0x09, 0x74, 0x69, 0x6d, 0x65,
0x73, 0x74, 0x61, 0x6d, 0x70, 0x22, 0xbb, 0x01, 0x0a, 0x13, 0x43, 0x6c, 0x69, 0x65, 0x6e, 0x74,
0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x12, 0x30, 0x0a,
0x09, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f, 0x6c, 0x6f, 0x67, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b,
0x32, 0x11, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x41, 0x67, 0x65, 0x6e, 0x74,
0x4c, 0x6f, 0x67, 0x48, 0x00, 0x52, 0x08, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x4c, 0x6f, 0x67, 0x12,
0x36, 0x0a, 0x0b, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f, 0x65, 0x76, 0x65, 0x6e, 0x74, 0x18, 0x02,
0x20, 0x01, 0x28, 0x0b, 0x32, 0x13, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x41,
0x67, 0x65, 0x6e, 0x74, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x48, 0x00, 0x52, 0x0a, 0x61, 0x67, 0x65,
0x6e, 0x74, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x2f, 0x0a, 0x07, 0x72, 0x75, 0x6e, 0x5f, 0x72,
0x65, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x14, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67,
0x65, 0x72, 0x2e, 0x52, 0x75, 0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x48, 0x00,
0x52, 0x06, 0x72, 0x75, 0x6e, 0x52, 0x65, 0x73, 0x42, 0x09, 0x0a, 0x07, 0x6d, 0x65, 0x73, 0x73,
0x61, 0x67, 0x65, 0x22, 0x9f, 0x02, 0x0a, 0x11, 0x43, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74,
0x69, 0x6f, 0x6e, 0x52, 0x75, 0x6e, 0x52, 0x65, 0x71, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18,
0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d,
0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x20, 0x0a,
0x0b, 0x64, 0x65, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x18, 0x03, 0x20, 0x01,
0x28, 0x09, 0x52, 0x0b, 0x64, 0x65, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12,
0x2c, 0x0a, 0x08, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x73, 0x18, 0x04, 0x20, 0x03, 0x28,
0x0b, 0x32, 0x10, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x44, 0x61, 0x74, 0x61,
0x73, 0x65, 0x74, 0x52, 0x08, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x73, 0x12, 0x32, 0x0a,
0x0a, 0x61, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x73, 0x18, 0x05, 0x20, 0x03, 0x28,
0x0b, 0x32, 0x12, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x41, 0x6c, 0x67, 0x6f,
0x72, 0x69, 0x74, 0x68, 0x6d, 0x52, 0x0a, 0x61, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d,
0x73, 0x12, 0x29, 0x0a, 0x10, 0x72, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x5f, 0x63, 0x6f, 0x6e, 0x73,
0x75, 0x6d, 0x65, 0x72, 0x73, 0x18, 0x06, 0x20, 0x03, 0x28, 0x09, 0x52, 0x0f, 0x72, 0x65, 0x73,
0x75, 0x6c, 0x74, 0x43, 0x6f, 0x6e, 0x73, 0x75, 0x6d, 0x65, 0x72, 0x73, 0x12, 0x37, 0x0a, 0x0c,
0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x18, 0x07, 0x20, 0x01,
0x28, 0x0b, 0x32, 0x14, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x41, 0x67, 0x65,
0x6e, 0x74, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x52, 0x0b, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x43,
0x6f, 0x6e, 0x66, 0x69, 0x67, 0x22, 0x49, 0x0a, 0x07, 0x44, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74,
0x12, 0x1a, 0x0a, 0x08, 0x70, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01,
0x28, 0x09, 0x52, 0x08, 0x70, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x12, 0x0e, 0x0a, 0x02,
0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x12, 0x12, 0x0a, 0x04,
0x68, 0x61, 0x73, 0x68, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x68, 0x61, 0x73, 0x68,
0x22, 0x4b, 0x0a, 0x09, 0x41, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x1a, 0x0a,
0x08, 0x70, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,
0x08, 0x70, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18,
0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x68, 0x61, 0x73,
0x68, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x68, 0x61, 0x73, 0x68, 0x22, 0xd6, 0x01,
0x0a, 0x0b, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x12, 0x0a,
0x04, 0x70, 0x6f, 0x72, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x70, 0x6f, 0x72,
0x74, 0x12, 0x12, 0x0a, 0x04, 0x68, 0x6f, 0x73, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52,
0x04, 0x68, 0x6f, 0x73, 0x74, 0x12, 0x1b, 0x0a, 0x09, 0x63, 0x65, 0x72, 0x74, 0x5f, 0x66, 0x69,
0x6c, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x63, 0x65, 0x72, 0x74, 0x46, 0x69,
0x6c, 0x65, 0x12, 0x19, 0x0a, 0x08, 0x6b, 0x65, 0x79, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x04,
0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x6b, 0x65, 0x79, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x24, 0x0a,
0x0e, 0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x5f, 0x63, 0x61, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18,
0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0c, 0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x43, 0x61, 0x46,
0x69, 0x6c, 0x65, 0x12, 0x24, 0x0a, 0x0e, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x5f, 0x63, 0x61,
0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x06, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0c, 0x73, 0x65, 0x72,
0x76, 0x65, 0x72, 0x43, 0x61, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x1b, 0x0a, 0x09, 0x6c, 0x6f, 0x67,
0x5f, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x18, 0x07, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x6c, 0x6f,
0x67, 0x4c, 0x65, 0x76, 0x65, 0x6c, 0x32, 0x5b, 0x0a, 0x0e, 0x4d, 0x61, 0x6e, 0x61, 0x67, 0x65,
0x72, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x49, 0x0a, 0x07, 0x50, 0x72, 0x6f, 0x63,
0x65, 0x73, 0x73, 0x12, 0x1c, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x43, 0x6c,
0x69, 0x65, 0x6e, 0x74, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67,
0x65, 0x1a, 0x1a, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x43, 0x6f, 0x6d, 0x70,
0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x75, 0x6e, 0x52, 0x65, 0x71, 0x22, 0x00, 0x28,
0x01, 0x30, 0x01, 0x42, 0x0b, 0x5a, 0x09, 0x2e, 0x2f, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72,
0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
0x73, 0x74, 0x61, 0x6d, 0x70, 0x12, 0x25, 0x0a, 0x0e, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61,
0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x69, 0x64, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x63,
0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x64, 0x12, 0x18, 0x0a, 0x07,
0x64, 0x65, 0x74, 0x61, 0x69, 0x6c, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x64,
0x65, 0x74, 0x61, 0x69, 0x6c, 0x73, 0x12, 0x1e, 0x0a, 0x0a, 0x6f, 0x72, 0x69, 0x67, 0x69, 0x6e,
0x61, 0x74, 0x6f, 0x72, 0x18, 0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0a, 0x6f, 0x72, 0x69, 0x67,
0x69, 0x6e, 0x61, 0x74, 0x6f, 0x72, 0x12, 0x16, 0x0a, 0x06, 0x73, 0x74, 0x61, 0x74, 0x75, 0x73,
0x18, 0x06, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x73, 0x74, 0x61, 0x74, 0x75, 0x73, 0x22, 0x9b,
0x01, 0x0a, 0x08, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x4c, 0x6f, 0x67, 0x12, 0x18, 0x0a, 0x07, 0x6d,
0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x6d, 0x65,
0x73, 0x73, 0x61, 0x67, 0x65, 0x12, 0x25, 0x0a, 0x0e, 0x63, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61,
0x74, 0x69, 0x6f, 0x6e, 0x5f, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x63,
0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x49, 0x64, 0x12, 0x14, 0x0a, 0x05,
0x6c, 0x65, 0x76, 0x65, 0x6c, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x6c, 0x65, 0x76,
0x65, 0x6c, 0x12, 0x38, 0x0a, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x18,
0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d,
0x70, 0x52, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x22, 0xbb, 0x01, 0x0a,
0x13, 0x43, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x73,
0x73, 0x61, 0x67, 0x65, 0x12, 0x30, 0x0a, 0x09, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f, 0x6c, 0x6f,
0x67, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x11, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65,
0x72, 0x2e, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x4c, 0x6f, 0x67, 0x48, 0x00, 0x52, 0x08, 0x61, 0x67,
0x65, 0x6e, 0x74, 0x4c, 0x6f, 0x67, 0x12, 0x36, 0x0a, 0x0b, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f,
0x65, 0x76, 0x65, 0x6e, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x13, 0x2e, 0x6d, 0x61,
0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x45, 0x76, 0x65, 0x6e, 0x74,
0x48, 0x00, 0x52, 0x0a, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x45, 0x76, 0x65, 0x6e, 0x74, 0x12, 0x2f,
0x0a, 0x07, 0x72, 0x75, 0x6e, 0x5f, 0x72, 0x65, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32,
0x14, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x52, 0x75, 0x6e, 0x52, 0x65, 0x73,
0x70, 0x6f, 0x6e, 0x73, 0x65, 0x48, 0x00, 0x52, 0x06, 0x72, 0x75, 0x6e, 0x52, 0x65, 0x73, 0x42,
0x09, 0x0a, 0x07, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x22, 0x90, 0x01, 0x0a, 0x13, 0x53,
0x65, 0x72, 0x76, 0x65, 0x72, 0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x73, 0x73, 0x61,
0x67, 0x65, 0x12, 0x34, 0x0a, 0x06, 0x72, 0x75, 0x6e, 0x52, 0x65, 0x71, 0x18, 0x01, 0x20, 0x01,
0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x43, 0x6f, 0x6d,
0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x75, 0x6e, 0x52, 0x65, 0x71, 0x48, 0x00,
0x52, 0x06, 0x72, 0x75, 0x6e, 0x52, 0x65, 0x71, 0x12, 0x38, 0x0a, 0x0c, 0x74, 0x65, 0x72, 0x6d,
0x69, 0x6e, 0x61, 0x74, 0x65, 0x52, 0x65, 0x71, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x12,
0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x54, 0x65, 0x72, 0x6d, 0x69, 0x6e, 0x61,
0x74, 0x65, 0x48, 0x00, 0x52, 0x0c, 0x74, 0x65, 0x72, 0x6d, 0x69, 0x6e, 0x61, 0x74, 0x65, 0x52,
0x65, 0x71, 0x42, 0x09, 0x0a, 0x07, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x22, 0xb6, 0x02,
0x0a, 0x11, 0x43, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x75, 0x6e,
0x52, 0x65, 0x71, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,
0x02, 0x69, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28,
0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x20, 0x0a, 0x0b, 0x64, 0x65, 0x73, 0x63, 0x72,
0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x64, 0x65,
0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x2c, 0x0a, 0x08, 0x64, 0x61, 0x74,
0x61, 0x73, 0x65, 0x74, 0x73, 0x18, 0x04, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x6d, 0x61,
0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x44, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x52, 0x08, 0x64,
0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x73, 0x12, 0x30, 0x0a, 0x09, 0x61, 0x6c, 0x67, 0x6f, 0x72,
0x69, 0x74, 0x68, 0x6d, 0x18, 0x05, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x6d, 0x61, 0x6e,
0x61, 0x67, 0x65, 0x72, 0x2e, 0x41, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x52, 0x09,
0x61, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x42, 0x0a, 0x10, 0x72, 0x65, 0x73,
0x75, 0x6c, 0x74, 0x5f, 0x63, 0x6f, 0x6e, 0x73, 0x75, 0x6d, 0x65, 0x72, 0x73, 0x18, 0x06, 0x20,
0x03, 0x28, 0x0b, 0x32, 0x17, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x52, 0x65,
0x73, 0x75, 0x6c, 0x74, 0x43, 0x6f, 0x6e, 0x73, 0x75, 0x6d, 0x65, 0x72, 0x52, 0x0f, 0x72, 0x65,
0x73, 0x75, 0x6c, 0x74, 0x43, 0x6f, 0x6e, 0x73, 0x75, 0x6d, 0x65, 0x72, 0x73, 0x12, 0x37, 0x0a,
0x0c, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x18, 0x07, 0x20,
0x01, 0x28, 0x0b, 0x32, 0x14, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x41, 0x67,
0x65, 0x6e, 0x74, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x52, 0x0b, 0x61, 0x67, 0x65, 0x6e, 0x74,
0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x22, 0x2a, 0x0a, 0x0e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74,
0x43, 0x6f, 0x6e, 0x73, 0x75, 0x6d, 0x65, 0x72, 0x12, 0x18, 0x0a, 0x07, 0x75, 0x73, 0x65, 0x72,
0x4b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x75, 0x73, 0x65, 0x72, 0x4b,
0x65, 0x79, 0x22, 0x37, 0x0a, 0x07, 0x44, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x12, 0x12, 0x0a,
0x04, 0x68, 0x61, 0x73, 0x68, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x68, 0x61, 0x73,
0x68, 0x12, 0x18, 0x0a, 0x07, 0x75, 0x73, 0x65, 0x72, 0x4b, 0x65, 0x79, 0x18, 0x02, 0x20, 0x01,
0x28, 0x0c, 0x52, 0x07, 0x75, 0x73, 0x65, 0x72, 0x4b, 0x65, 0x79, 0x22, 0x39, 0x0a, 0x09, 0x41,
0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x12, 0x0a, 0x04, 0x68, 0x61, 0x73, 0x68,
0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x68, 0x61, 0x73, 0x68, 0x12, 0x18, 0x0a, 0x07,
0x75, 0x73, 0x65, 0x72, 0x4b, 0x65, 0x79, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x75,
0x73, 0x65, 0x72, 0x4b, 0x65, 0x79, 0x22, 0xf9, 0x01, 0x0a, 0x0b, 0x41, 0x67, 0x65, 0x6e, 0x74,
0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x12, 0x0a, 0x04, 0x70, 0x6f, 0x72, 0x74, 0x18, 0x01,
0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x70, 0x6f, 0x72, 0x74, 0x12, 0x12, 0x0a, 0x04, 0x68, 0x6f,
0x73, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x68, 0x6f, 0x73, 0x74, 0x12, 0x1b,
0x0a, 0x09, 0x63, 0x65, 0x72, 0x74, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28,
0x09, 0x52, 0x08, 0x63, 0x65, 0x72, 0x74, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x19, 0x0a, 0x08, 0x6b,
0x65, 0x79, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x6b,
0x65, 0x79, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x24, 0x0a, 0x0e, 0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74,
0x5f, 0x63, 0x61, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0c,
0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x43, 0x61, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x24, 0x0a, 0x0e,
0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x5f, 0x63, 0x61, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x06,
0x20, 0x01, 0x28, 0x09, 0x52, 0x0c, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x43, 0x61, 0x46, 0x69,
0x6c, 0x65, 0x12, 0x1b, 0x0a, 0x09, 0x6c, 0x6f, 0x67, 0x5f, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x18,
0x07, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x6c, 0x6f, 0x67, 0x4c, 0x65, 0x76, 0x65, 0x6c, 0x12,
0x21, 0x0a, 0x0c, 0x61, 0x74, 0x74, 0x65, 0x73, 0x74, 0x65, 0x64, 0x5f, 0x74, 0x6c, 0x73, 0x18,
0x08, 0x20, 0x01, 0x28, 0x08, 0x52, 0x0b, 0x61, 0x74, 0x74, 0x65, 0x73, 0x74, 0x65, 0x64, 0x54,
0x6c, 0x73, 0x32, 0x5d, 0x0a, 0x0e, 0x4d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x53, 0x65, 0x72,
0x76, 0x69, 0x63, 0x65, 0x12, 0x4b, 0x0a, 0x07, 0x50, 0x72, 0x6f, 0x63, 0x65, 0x73, 0x73, 0x12,
0x1c, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x43, 0x6c, 0x69, 0x65, 0x6e, 0x74,
0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x1a, 0x1c, 0x2e,
0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x53, 0x74,
0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x22, 0x00, 0x28, 0x01, 0x30,
0x01, 0x42, 0x0b, 0x5a, 0x09, 0x2e, 0x2f, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x62, 0x06,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (
@@ -759,34 +942,40 @@ func file_manager_manager_proto_rawDescGZIP() []byte {
return file_manager_manager_proto_rawDescData
}
var file_manager_manager_proto_msgTypes = make([]protoimpl.MessageInfo, 8)
var file_manager_manager_proto_msgTypes = make([]protoimpl.MessageInfo, 11)
var file_manager_manager_proto_goTypes = []interface{}{
(*RunResponse)(nil), // 0: manager.RunResponse
(*AgentEvent)(nil), // 1: manager.AgentEvent
(*AgentLog)(nil), // 2: manager.AgentLog
(*ClientStreamMessage)(nil), // 3: manager.ClientStreamMessage
(*ComputationRunReq)(nil), // 4: manager.ComputationRunReq
(*Dataset)(nil), // 5: manager.Dataset
(*Algorithm)(nil), // 6: manager.Algorithm
(*AgentConfig)(nil), // 7: manager.AgentConfig
(*timestamppb.Timestamp)(nil), // 8: google.protobuf.Timestamp
(*Terminate)(nil), // 0: manager.Terminate
(*RunResponse)(nil), // 1: manager.RunResponse
(*AgentEvent)(nil), // 2: manager.AgentEvent
(*AgentLog)(nil), // 3: manager.AgentLog
(*ClientStreamMessage)(nil), // 4: manager.ClientStreamMessage
(*ServerStreamMessage)(nil), // 5: manager.ServerStreamMessage
(*ComputationRunReq)(nil), // 6: manager.ComputationRunReq
(*ResultConsumer)(nil), // 7: manager.ResultConsumer
(*Dataset)(nil), // 8: manager.Dataset
(*Algorithm)(nil), // 9: manager.Algorithm
(*AgentConfig)(nil), // 10: manager.AgentConfig
(*timestamppb.Timestamp)(nil), // 11: google.protobuf.Timestamp
}
var file_manager_manager_proto_depIdxs = []int32{
8, // 0: manager.AgentEvent.timestamp:type_name -> google.protobuf.Timestamp
8, // 1: manager.AgentLog.timestamp:type_name -> google.protobuf.Timestamp
2, // 2: manager.ClientStreamMessage.agent_log:type_name -> manager.AgentLog
1, // 3: manager.ClientStreamMessage.agent_event:type_name -> manager.AgentEvent
0, // 4: manager.ClientStreamMessage.run_res:type_name -> manager.RunResponse
5, // 5: manager.ComputationRunReq.datasets:type_name -> manager.Dataset
6, // 6: manager.ComputationRunReq.algorithms:type_name -> manager.Algorithm
7, // 7: manager.ComputationRunReq.agent_config:type_name -> manager.AgentConfig
3, // 8: manager.ManagerService.Process:input_type -> manager.ClientStreamMessage
4, // 9: manager.ManagerService.Process:output_type -> manager.ComputationRunReq
9, // [9:10] is the sub-list for method output_type
8, // [8:9] is the sub-list for method input_type
8, // [8:8] is the sub-list for extension type_name
8, // [8:8] is the sub-list for extension extendee
0, // [0:8] is the sub-list for field type_name
11, // 0: manager.AgentEvent.timestamp:type_name -> google.protobuf.Timestamp
11, // 1: manager.AgentLog.timestamp:type_name -> google.protobuf.Timestamp
3, // 2: manager.ClientStreamMessage.agent_log:type_name -> manager.AgentLog
2, // 3: manager.ClientStreamMessage.agent_event:type_name -> manager.AgentEvent
1, // 4: manager.ClientStreamMessage.run_res:type_name -> manager.RunResponse
6, // 5: manager.ServerStreamMessage.runReq:type_name -> manager.ComputationRunReq
0, // 6: manager.ServerStreamMessage.terminateReq:type_name -> manager.Terminate
8, // 7: manager.ComputationRunReq.datasets:type_name -> manager.Dataset
9, // 8: manager.ComputationRunReq.algorithm:type_name -> manager.Algorithm
7, // 9: manager.ComputationRunReq.result_consumers:type_name -> manager.ResultConsumer
10, // 10: manager.ComputationRunReq.agent_config:type_name -> manager.AgentConfig
4, // 11: manager.ManagerService.Process:input_type -> manager.ClientStreamMessage
5, // 12: manager.ManagerService.Process:output_type -> manager.ServerStreamMessage
12, // [12:13] is the sub-list for method output_type
11, // [11:12] is the sub-list for method input_type
11, // [11:11] is the sub-list for extension type_name
11, // [11:11] is the sub-list for extension extendee
0, // [0:11] is the sub-list for field type_name
}
func init() { file_manager_manager_proto_init() }
@@ -796,7 +985,7 @@ func file_manager_manager_proto_init() {
}
if !protoimpl.UnsafeEnabled {
file_manager_manager_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*RunResponse); i {
switch v := v.(*Terminate); i {
case 0:
return &v.state
case 1:
@@ -808,7 +997,7 @@ func file_manager_manager_proto_init() {
}
}
file_manager_manager_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*AgentEvent); i {
switch v := v.(*RunResponse); i {
case 0:
return &v.state
case 1:
@@ -820,7 +1009,7 @@ func file_manager_manager_proto_init() {
}
}
file_manager_manager_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*AgentLog); i {
switch v := v.(*AgentEvent); i {
case 0:
return &v.state
case 1:
@@ -832,7 +1021,7 @@ func file_manager_manager_proto_init() {
}
}
file_manager_manager_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*ClientStreamMessage); i {
switch v := v.(*AgentLog); i {
case 0:
return &v.state
case 1:
@@ -844,7 +1033,7 @@ func file_manager_manager_proto_init() {
}
}
file_manager_manager_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*ComputationRunReq); i {
switch v := v.(*ClientStreamMessage); i {
case 0:
return &v.state
case 1:
@@ -856,7 +1045,7 @@ func file_manager_manager_proto_init() {
}
}
file_manager_manager_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Dataset); i {
switch v := v.(*ServerStreamMessage); i {
case 0:
return &v.state
case 1:
@@ -868,7 +1057,7 @@ func file_manager_manager_proto_init() {
}
}
file_manager_manager_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Algorithm); i {
switch v := v.(*ComputationRunReq); i {
case 0:
return &v.state
case 1:
@@ -880,6 +1069,42 @@ func file_manager_manager_proto_init() {
}
}
file_manager_manager_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*ResultConsumer); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_manager_manager_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Dataset); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_manager_manager_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Algorithm); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_manager_manager_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*AgentConfig); i {
case 0:
return &v.state
@@ -892,18 +1117,22 @@ func file_manager_manager_proto_init() {
}
}
}
file_manager_manager_proto_msgTypes[3].OneofWrappers = []interface{}{
file_manager_manager_proto_msgTypes[4].OneofWrappers = []interface{}{
(*ClientStreamMessage_AgentLog)(nil),
(*ClientStreamMessage_AgentEvent)(nil),
(*ClientStreamMessage_RunRes)(nil),
}
file_manager_manager_proto_msgTypes[5].OneofWrappers = []interface{}{
(*ServerStreamMessage_RunReq)(nil),
(*ServerStreamMessage_TerminateReq)(nil),
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_manager_manager_proto_rawDesc,
NumEnums: 0,
NumMessages: 8,
NumMessages: 11,
NumExtensions: 0,
NumServices: 1,
},
+5 -5
View File
@@ -51,7 +51,7 @@ func (c *managerServiceClient) Process(ctx context.Context, opts ...grpc.CallOpt
type ManagerService_ProcessClient interface {
Send(*ClientStreamMessage) error
Recv() (*ComputationRunReq, error)
Recv() (*ServerStreamMessage, error)
grpc.ClientStream
}
@@ -63,8 +63,8 @@ func (x *managerServiceProcessClient) Send(m *ClientStreamMessage) error {
return x.ClientStream.SendMsg(m)
}
func (x *managerServiceProcessClient) Recv() (*ComputationRunReq, error) {
m := new(ComputationRunReq)
func (x *managerServiceProcessClient) Recv() (*ServerStreamMessage, error) {
m := new(ServerStreamMessage)
if err := x.ClientStream.RecvMsg(m); err != nil {
return nil, err
}
@@ -104,7 +104,7 @@ func _ManagerService_Process_Handler(srv interface{}, stream grpc.ServerStream)
}
type ManagerService_ProcessServer interface {
Send(*ComputationRunReq) error
Send(*ServerStreamMessage) error
Recv() (*ClientStreamMessage, error)
grpc.ServerStream
}
@@ -113,7 +113,7 @@ type managerServiceProcessServer struct {
grpc.ServerStream
}
func (x *managerServiceProcessServer) Send(m *ComputationRunReq) error {
func (x *managerServiceProcessServer) Send(m *ServerStreamMessage) error {
return x.ServerStream.SendMsg(m)
}
+111 -21
View File
@@ -3,63 +3,131 @@
package sdk
import (
"bytes"
"context"
"crypto"
"crypto/rand"
"crypto/rsa"
"crypto/sha256"
"encoding/base64"
"io"
"log/slog"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/auth"
"google.golang.org/grpc/metadata"
)
var _ agent.Service = (*agentSDK)(nil)
type SDK interface {
Algo(ctx context.Context, algorithm agent.Algorithm, privKey *rsa.PrivateKey) error
Data(ctx context.Context, dataset agent.Dataset, privKey *rsa.PrivateKey) error
Result(ctx context.Context, privKey *rsa.PrivateKey) ([]byte, error)
Attestation(ctx context.Context, reportData [size64]byte) ([]byte, error)
}
const size64 = 64
const (
size64 = 64
bufferSize = 1024 * 1024
)
type agentSDK struct {
client agent.AgentServiceClient
logger *slog.Logger
}
func NewAgentSDK(log *slog.Logger, agentClient agent.AgentServiceClient) *agentSDK {
func NewAgentSDK(log *slog.Logger, agentClient agent.AgentServiceClient) SDK {
return &agentSDK{
client: agentClient,
logger: log,
}
}
func (sdk *agentSDK) Algo(ctx context.Context, algorithm agent.Algorithm) error {
request := &agent.AlgoRequest{
Algorithm: algorithm.Algorithm,
Provider: algorithm.Provider,
Id: algorithm.ID,
func (sdk *agentSDK) Algo(ctx context.Context, algorithm agent.Algorithm, privKey *rsa.PrivateKey) error {
md, err := generateMetadata(string(auth.AlgorithmProviderRole), privKey)
if err != nil {
sdk.logger.Error("Failed to generate metadata")
return err
}
if _, err := sdk.client.Algo(ctx, request); err != nil {
ctx = metadata.NewOutgoingContext(ctx, md)
stream, err := sdk.client.Algo(ctx)
if err != nil {
sdk.logger.Error("Failed to call Algo RPC")
return err
}
algoBuffer := bytes.NewBuffer(algorithm.Algorithm)
return nil
}
buf := make([]byte, bufferSize)
for {
n, err := algoBuffer.Read(buf)
if err == io.EOF {
break
}
if err != nil {
return err
}
func (sdk *agentSDK) Data(ctx context.Context, dataset agent.Dataset) error {
request := &agent.DataRequest{
Dataset: dataset.Dataset,
Provider: dataset.Provider,
Id: dataset.ID,
err = stream.Send(&agent.AlgoRequest{Algorithm: buf[:n]})
if err != nil {
return err
}
}
if _, err := sdk.client.Data(ctx, request); err != nil {
sdk.logger.Error("Failed to call Data RPC")
if _, err := stream.CloseAndRecv(); err != nil {
return err
}
return nil
}
func (sdk *agentSDK) Result(ctx context.Context, consumer string) ([]byte, error) {
request := &agent.ResultRequest{
Consumer: consumer,
func (sdk *agentSDK) Data(ctx context.Context, dataset agent.Dataset, privKey *rsa.PrivateKey) error {
md, err := generateMetadata(string(auth.DataProviderRole), privKey)
if err != nil {
sdk.logger.Error("Failed to generate metadata")
return err
}
ctx = metadata.NewOutgoingContext(ctx, md)
stream, err := sdk.client.Data(ctx)
if err != nil {
sdk.logger.Error("Failed to call Algo RPC")
return err
}
dataBuffer := bytes.NewBuffer(dataset.Dataset)
buf := make([]byte, bufferSize)
for {
n, err := dataBuffer.Read(buf)
if err == io.EOF {
break
}
if err != nil {
return err
}
err = stream.Send(&agent.DataRequest{Dataset: buf[:n]})
if err != nil {
return err
}
}
if _, err := stream.CloseAndRecv(); err != nil {
return err
}
return nil
}
func (sdk *agentSDK) Result(ctx context.Context, privKey *rsa.PrivateKey) ([]byte, error) {
request := &agent.ResultRequest{}
md, err := generateMetadata(string(auth.ConsumerRole), privKey)
if err != nil {
sdk.logger.Error("Failed to generate metadata")
return nil, err
}
ctx = metadata.NewOutgoingContext(ctx, md)
response, err := sdk.client.Result(ctx, request)
if err != nil {
sdk.logger.Error("Failed to call Result RPC")
@@ -82,3 +150,25 @@ func (sdk *agentSDK) Attestation(ctx context.Context, reportData [size64]byte) (
return response.File, nil
}
func signData(userID string, privKey *rsa.PrivateKey) ([]byte, error) {
hash := sha256.Sum256([]byte(userID))
signature, err := rsa.SignPKCS1v15(rand.Reader, privKey, crypto.SHA256, hash[:])
if err != nil {
return nil, err
}
return signature, nil
}
func generateMetadata(userID string, privateKey *rsa.PrivateKey) (metadata.MD, error) {
signature, err := signData(userID, privateKey)
if err != nil {
return nil, err
}
kv := make(map[string]string)
kv[auth.UserMetadataKey] = userID
kv[auth.SignatureMetadataKey] = base64.StdEncoding.EncodeToString(signature)
return metadata.New(kv), nil
}
+14
View File
@@ -0,0 +1,14 @@
[package]
name = "platform_info"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
clap = { version = "4.0", features = ["derive"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
sev = "3.1.1"
sysinfo = "0.30.12"
regex = "1.10.4"
+14
View File
@@ -0,0 +1,14 @@
CARGO = cargo
TARGET = target
BUILD_DIR = $(TARGET)/release
BIN_NAME = platform_info
all: build
build:
$(CARGO) build --release
clean:
$(CARGO) clean
.PHONY: all build clean
+19
View File
@@ -0,0 +1,19 @@
# Rust project for fetching platform info
This rust project fetches information from the host system needed for validation of the attestation report. It outputs a JSON file that contains the said information.
The JSON file is in a format that can be used with the [go-sev-guest](https://github.com/google/go-sev-guest) library.
## Usage
Clone `cocos` repository:
```bash
git clone git@github.com:ultravioletrs/cocos.git
cd ./cocos/scripts/platform_info
make
```
Then run the binary. Keep in mind that you have to specify the policy of the Guest VM:
```bash
cd ./target/releas
# Run with option --policy (policy is 64 bit number)
./platform_info --policy 196608
```
+137
View File
@@ -0,0 +1,137 @@
use clap::{Arg, Command, value_parser};
use serde::Serialize;
use std::fs::File;
use std::io::Write;
use sysinfo::System;
use regex::Regex;
use sev::firmware::host::*;
const PLATFORM_INFO_JSON : &str = "platform_info.json";
#[derive(Serialize)]
struct Vmpl {
value : u32,
}
#[derive(Serialize)]
struct SnpPolicy {
policy: u64,
family_id: Vec<u8>,
image_id: Vec<u8>,
vmpl: Vmpl,
minimum_tcb: u64,
minimum_launch_tcb: u64,
require_author_key: bool,
measurement: Vec<u8>,
host_data: Vec<u8>,
report_id_ma: Vec<u8>,
chip_id: Vec<u8>,
minimum_build: u32,
minimum_version: String,
permit_provisional_firmware: bool,
require_id_block: bool,
}
#[derive(Serialize)]
struct RootOfTrust {
product: String,
check_crl : bool,
disallow_network : bool,
}
#[derive(Serialize)]
struct Computation {
snp_policy: SnpPolicy,
root_of_trust: RootOfTrust,
}
fn get_product_name() -> String {
let mut sys = System::new_all();
sys.refresh_all();
let re = Regex::new(r"EPYC.*7..3.*").unwrap();
for cpu in sys.cpus() {
if re.is_match(cpu.brand()) {
return "Milan".to_string()
}
}
"Unknown".to_string()
}
fn get_uint64_from_tcb(tcb_version : &TcbVersion) -> u64 {
let microcode = (tcb_version.microcode as u64) << 56;
let snp = (tcb_version.snp as u64) << 48;
let tee = (tcb_version.tee as u64) << 8;
let bootloader = tcb_version.bootloader as u64;
microcode | snp | tee | bootloader
}
fn main() {
let matches = Command::new("Platform info")
.about("Processes command line options and outputs a JSON file for Attestation verification")
.arg(Arg::new("policy")
.long("policy")
.value_name("INT")
.help("Sets the policy integer")
.required(true)
.value_parser(value_parser!(u64)))
.get_matches();
let mut firmware: Firmware = Firmware::open().unwrap();
let status: SnpPlatformStatus = firmware.snp_platform_status().unwrap();
let policy: u64 = *matches.get_one::<u64>("policy").unwrap();
let family_id = vec![0];
let image_id = vec![0];
let vmpl = Vmpl { value: 0};
let minimum_tcb = get_uint64_from_tcb(&status.platform_tcb_version);
let minimum_launch_tcb = get_uint64_from_tcb(&status.platform_tcb_version);
let require_author_key = false;
let measurement = vec![0];
let host_data = vec![0];
let report_id_ma = vec![0];
let cpu_id: Identifier = firmware.get_identifier().unwrap();
let chip_id: Vec<u8> = cpu_id.0;
let minimum_build = status.build_id;
let minimum_version = status.version.to_string();
let permit_provisional_firmware = false;
let require_id_block = false;
let snp_policy = SnpPolicy {
policy,
family_id,
image_id,
vmpl,
minimum_tcb,
minimum_launch_tcb,
require_author_key,
measurement,
host_data,
report_id_ma,
chip_id,
minimum_build,
minimum_version,
permit_provisional_firmware,
require_id_block,
};
let root_of_trust = RootOfTrust {
product : get_product_name(),
check_crl : true,
disallow_network : false,
};
let computation = Computation {
snp_policy,
root_of_trust,
};
let json = serde_json::to_string_pretty(&computation).expect("Failed to serialize to JSON");
let mut file = File::create(PLATFORM_INFO_JSON).expect("Failed to create file");
file.write_all(json.as_bytes()).expect("Failed to write to file");
println!("Computation JSON has been written to {}", PLATFORM_INFO_JSON);
}
+40 -17
View File
@@ -4,10 +4,12 @@ package main
import (
"context"
"encoding/pem"
"fmt"
"log"
"log/slog"
"os"
"strconv"
mglog "github.com/absmach/magistrala/logger"
"github.com/ultravioletrs/cocos/internal/env"
@@ -18,6 +20,7 @@ import (
"golang.org/x/crypto/sha3"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/reflection"
)
@@ -29,15 +32,17 @@ const (
)
var (
algoPath = "./test/manual/algo/lin_reg.py"
dataPath = "./test/manual/data/iris.csv"
algoPath = "./test/manual/algo/lin_reg.py"
dataPath = "./test/manual/data/iris.csv"
attestedTLS = false
pubKeyFile string
)
type svc struct {
logger *slog.Logger
}
func (s *svc) Run(ipAdress string, reqChan chan *manager.ComputationRunReq) {
func (s *svc) Run(ipAdress string, reqChan chan *manager.ServerStreamMessage, auth credentials.AuthInfo) {
s.logger.Debug(fmt.Sprintf("received who am on ip address %s", ipAdress))
algo, err := os.ReadFile(algoPath)
if err != nil {
@@ -49,28 +54,46 @@ func (s *svc) Run(ipAdress string, reqChan chan *manager.ComputationRunReq) {
s.logger.Error(fmt.Sprintf("failed to read data file: %s", err))
return
}
pubKey, err := os.ReadFile(pubKeyFile)
if err != nil {
s.logger.Error(fmt.Sprintf("failed to read public key file: %s", err))
return
}
pubPem, _ := pem.Decode(pubKey)
algoHash := sha3.Sum256(algo)
dataHash := sha3.Sum256(data)
reqChan <- &manager.ComputationRunReq{
Id: "1",
Name: "sample computation",
Description: "sample descrption",
Datasets: []*manager.Dataset{{Id: "1", Provider: "provider1", Hash: dataHash[:]}},
Algorithms: []*manager.Algorithm{{Id: "1", Provider: "provider1", Hash: algoHash[:]}},
ResultConsumers: []string{"consumer1"},
AgentConfig: &manager.AgentConfig{
Port: "7002",
LogLevel: "debug",
reqChan <- &manager.ServerStreamMessage{
Message: &manager.ServerStreamMessage_RunReq{
RunReq: &manager.ComputationRunReq{
Id: "1",
Name: "sample computation",
Description: "sample descrption",
Datasets: []*manager.Dataset{{Hash: dataHash[:], UserKey: pubPem.Bytes}},
Algorithm: &manager.Algorithm{Hash: algoHash[:], UserKey: pubPem.Bytes},
ResultConsumers: []*manager.ResultConsumer{{UserKey: pubPem.Bytes}},
AgentConfig: &manager.AgentConfig{
Port: "7002",
LogLevel: "debug",
AttestedTls: attestedTLS,
},
},
},
}
}
func main() {
if len(os.Args) < 3 {
log.Fatalf("usage: %s <data-path> <algo-path>", os.Args[0])
if len(os.Args) < 5 {
log.Fatalf("usage: %s <data-path> <algo-path> <public-key-path> <attested-tls-bool>", os.Args[0])
}
dataPath = os.Args[1]
algoPath = os.Args[2]
pubKeyFile = os.Args[3]
attestedTLSParam, err := strconv.ParseBool(os.Args[4])
if err != nil {
log.Fatalf("usage: %s <data-path> <algo-path> <attested-tls-bool>, <attested-tls-bool> must be a bool value", os.Args[0])
}
attestedTLS = attestedTLSParam
ctx, cancel := context.WithCancel(context.Background())
g, ctx := errgroup.WithContext(ctx)
incomingChan := make(chan *manager.ClientStreamMessage)
@@ -96,7 +119,7 @@ func main() {
registerAgentServiceServer := func(srv *grpc.Server) {
reflection.Register(srv)
manager.RegisterManagerServiceServer(srv, managergrpc.NewServer(ctx, incomingChan, &svc{logger: logger}))
manager.RegisterManagerServiceServer(srv, managergrpc.NewServer(incomingChan, &svc{logger: logger}))
}
grpcServerConfig := server.Config{Port: defaultPort}
if err := env.Parse(&grpcServerConfig, env.Options{}); err != nil {
@@ -104,7 +127,7 @@ func main() {
return
}
gs := grpcserver.New(ctx, cancel, svcName, grpcServerConfig, registerAgentServiceServer, logger)
gs := grpcserver.New(ctx, cancel, svcName, grpcServerConfig, registerAgentServiceServer, logger, nil, nil)
g.Go(func() error {
return gs.Start()
+41 -15
View File
@@ -2,27 +2,53 @@
## CLI
Throughout the tests, we assume that our current working directory is the root of the `agent` repository, both on the host machine and in the VM.
Throughout the tests, we assume that our current working directory is the root of the `cocos` repository, both on the host machine and in the VM.
### Python requirements
### Algorithm requirements
Do this both on the host machine and in the VM.
```sh
apt update
apt install python3-pip
pip3 install pandas scikit-learn
Agent accepts the algorithm as a binary that take in two command line arguments.
```shell
algorithm-file <unix socket path> <dataset file paths>
```
The algorithm program should return the results to a socket and an example can be seen in this [file](./algo/lin_reg.py).
### Agent-CLI interaction
Agent is started automatically in the VM when launched but requires configuration and manifest to be passed by manager. Alternatively you can pass configuration using this [simplified script](./agent-config/main.go)
Open console on the host, and run
For attested TLS, you will have to calculate the VM's measurement, which can be done using a tool [sev-snp-measure](https://pypi.org/project/sev-snp-measure/).
```bash
# Define the path to the OVMF, KERNEL, INITRD and CMD Kernel line arguments.
OVMF_CODE="/home/cocosai/ovmf/Build/AmdSev/DEBUG_GCC5/FV/OVMF.fd"
INITRD="/home/cocosai/initramfs.cpio.gz"
KERNEL="/home/cocosai/bzImage"
LINE="earlyprintk=serial console=ttyS0"
# Call sev-snp-measure
sev-snp-measure --mode snp --vcpus 4 --vcpu-type EPYC-v4 --ovmf $OVMF_CODE --kernel $KERNEL --initrd $INITRD --append "$LINE" --output-format base64
```
```sh
export AGENT_GRPC_URL=localhost:7002
export MANAGER_GRPC_URL=localhost:7001
# For attested TLS, the CLI should also be aware of the VM measurement. To
# add the measurement to the .json file that contains the information about
# the platform, run CLI with the measurement in base64 format and the path
# of the platform_info.json file.:
go run cmd/cli/main.go measurement '<measurement>' '<platform_info.json>'
# The platform_info.json file can be generated using Rust by running:
cd scripts/platform_info
make
sudo ./target/release/platform_info --policy 196608 # Default value of the policy should be 196608
# The output file platform_info.json will be generated in the directory from which the executable has been called.
cd ../..
# For attested TLS, also define the path to the platform_info.json that contains reference values for the fields of the attestation report
export AGENT_GRPC_MANIFEST=./scripts/platform_info/platform_info.json
export AGENT_GRPC_ATTESTED_TLS=true
# Retieve Attestation
go run cmd/cli/main.go attestation get '<report_data>'
@@ -31,16 +57,16 @@ go run cmd/cli/main.go attestation get '<report_data>'
go run cmd/cli/main.go attestation validate '<attesation>' --report_data '<report_data>'
# Run the CLI program with algorithm input
go run cmd/cli/main.go algo test/manual/algo/lin_reg.py Algorithm1 AlgorithmProvider1
# 2023/09/21 10:43:53 Uploading algorithm binary: test/manual/algo/lin_reg.py
go run cmd/cli/main.go algo test/manual/algo/lin_reg.bin <private_key_file_path>
# 2023/09/21 10:43:53 Uploading algorithm binary: test/manual/algo/lin_reg.bin
# Run the CLI program with dataset input
go run cmd/cli/main.go data test/manual/data/iris.csv Dataset1 Provider1
go run cmd/cli/main.go data test/manual/data/iris.csv Dataset2 Provider2
go run cmd/cli/main.go data test/manual/data/iris.csv <private_key_file_path>
go run cmd/cli/main.go data test/manual/data/iris.csv <private_key_file_path>
# 2023/09/21 10:45:25 Uploading dataset CSV: test/manual/data/iris.csv
# Run the CLI program to fetch computation result
go run cmd/cli/main.go result Consumer1
go run cmd/cli/main.go result <private_key_file_path>
# 2023/09/21 10:45:39 Retrieving computation result file
# 2023/09/21 10:45:40 Computation result retrieved and saved successfully!
```
+52 -65
View File
@@ -7,80 +7,62 @@ package main
import (
"encoding/json"
"encoding/pem"
"fmt"
"log"
"os"
"strconv"
"github.com/mdlayher/vsock"
"github.com/ultravioletrs/cocos/pkg/manager"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/manager"
pkgmanager "github.com/ultravioletrs/cocos/pkg/manager"
"golang.org/x/crypto/sha3"
"google.golang.org/protobuf/proto"
)
const VsockConfigPort uint32 = 9999
type AgentConfig struct {
LogLevel string `json:"log_level"`
InstanceID string `json:"instance_id"`
Host string `json:"host"`
Port string `json:"port"`
CertFile string `json:"cert_file"`
KeyFile string `json:"server_key"`
}
type Computation struct {
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Datasets Datasets `json:"datasets,omitempty"`
Algorithms Algorithms `json:"algorithms,omitempty"`
ResultConsumers []string `json:"result_consumers,omitempty"`
AgentConfig AgentConfig `json:"agent_config,omitempty"`
}
func (d *Datasets) String() string {
dat, err := json.Marshal(d)
if err != nil {
return ""
}
return string(dat)
}
func (a *Algorithms) String() string {
dat, err := json.Marshal(a)
if err != nil {
return ""
}
return string(dat)
}
type Dataset struct {
Dataset []byte `json:"-"`
Provider string `json:"provider,omitempty"`
ID string `json:"id,omitempty"`
}
type Datasets []Dataset
type Algorithm struct {
Algorithm []byte `json:"-"`
Provider string `json:"provider,omitempty"`
ID string `json:"id,omitempty"`
}
type Algorithms []Algorithm
func main() {
l, err := vsock.Listen(9997, nil)
if len(os.Args) < 5 {
log.Fatalf("usage: %s <data-path> <algo-path> <public-key-path> <attested-tls-bool>", os.Args[0])
}
dataPath := os.Args[1]
algoPath := os.Args[2]
pubKeyFile := os.Args[3]
attestedTLSParam, err := strconv.ParseBool(os.Args[4])
if err != nil {
log.Fatalf("usage: %s <data-path> <algo-path> <attested-tls-bool>, <attested-tls-bool> must be a bool value", os.Args[0])
}
attestedTLS := attestedTLSParam
algo, err := os.ReadFile(algoPath)
if err != nil {
log.Fatalf(fmt.Sprintf("failed to read algorithm file: %s", err))
}
data, err := os.ReadFile(dataPath)
if err != nil {
log.Fatalf(fmt.Sprintf("failed to read data file: %s", err))
}
pubKey, err := os.ReadFile(pubKeyFile)
if err != nil {
log.Fatalf(fmt.Sprintf("failed to read public key file: %s", err))
}
pubPem, _ := pem.Decode(pubKey)
algoHash := sha3.Sum256(algo)
dataHash := sha3.Sum256(data)
l, err := vsock.Listen(manager.ManagerVsockPort, nil)
if err != nil {
log.Fatal(err)
}
ac := Computation{
ac := agent.Computation{
ID: "123",
Datasets: Datasets{Dataset{ID: "1", Provider: "pr1"}},
Algorithms: Algorithms{Algorithm{ID: "1", Provider: "pr1"}},
ResultConsumers: []string{"1"},
AgentConfig: AgentConfig{
LogLevel: "debug",
Port: "7002",
Datasets: agent.Datasets{agent.Dataset{Hash: dataHash, UserKey: pubPem.Bytes}},
Algorithm: agent.Algorithm{Hash: algoHash, UserKey: pubPem.Bytes},
ResultConsumers: []agent.ResultConsumer{{UserKey: pubPem.Bytes}},
AgentConfig: agent.AgentConfig{
LogLevel: "debug",
Port: "7002",
AttestedTls: attestedTLS,
},
}
fmt.Println(SendAgentConfig(3, ac))
@@ -98,7 +80,7 @@ func main() {
continue
}
conn.Close()
var mes manager.ClientStreamMessage
var mes pkgmanager.ClientStreamMessage
if err := proto.Unmarshal(b[:n], &mes); err != nil {
log.Println(err)
}
@@ -106,8 +88,8 @@ func main() {
}
}
func SendAgentConfig(cid uint32, ac Computation) error {
conn, err := vsock.Dial(cid, VsockConfigPort, nil)
func SendAgentConfig(cid uint32, ac agent.Computation) error {
conn, err := vsock.Dial(cid, manager.VsockConfigPort, nil)
if err != nil {
return err
}
@@ -116,6 +98,11 @@ func SendAgentConfig(cid uint32, ac Computation) error {
if err != nil {
return err
}
var ac2 agent.Computation
if err := json.Unmarshal(payload, &ac2); err != nil {
return err
}
if _, err := conn.Write(payload); err != nil {
return err
}
+19
View File
@@ -0,0 +1,19 @@
## Algorihm covid19.py
To download dataset first you have to sign in Keggle. https://www.kaggle.com/
Then follow this documentation. https://github.com/Kaggle/kaggle-api
Go to settings and download API key. Copy it into keggle folder.
Run divide_save_data.py
You should have now 3 datasets, named Hospital_1, Hospital_2, Hospital_3
`covid19.py` trains the the model and produces model.
run :
`python covid19.py Hospital_1 Hospital_2 Hospital_3 --model model.pth`
in this example dataset is Hospital_1, Hospital_2, and Hospital_3 and it produces model.pth
`covid.19.py` produced model, and now you should be able to use that model in `predict.py`
run :
`python predict.py --model model.pth --image Hospital_2/COVID/images/COVID-3.png`
in this example you have trained model (model.pth) that runs tests on images (this example is image from Hospital_2 in COVID/images/COVID-3.png)
+190
View File
@@ -0,0 +1,190 @@
import os
import shutil
import random
import torch
import torchvision
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image
import argparse
import zipfile
import socket
import sys
import ssl
def prepare_test_set(root_dir, class_names):
test_dir = os.path.join(root_dir, 'test')
if os.path.isdir(test_dir):
shutil.rmtree(test_dir)
os.mkdir(test_dir)
for class_name in class_names:
os.mkdir(os.path.join(test_dir, class_name))
for class_name in class_names:
images = [x for x in os.listdir(os.path.join(root_dir, class_name, "images")) if x.lower().endswith('png')]
selected_images = random.sample(images, 30)
for image in selected_images:
source_path = os.path.join(root_dir, class_name, "images", image)
target_path = os.path.join(test_dir, class_name, image)
shutil.copy(source_path, target_path)
return test_dir
class ChestXRayDataset(torch.utils.data.Dataset):
def __init__(self, image_dirs, transform):
def get_images(class_name):
images = []
for dir_path in image_dirs[class_name]:
images += [os.path.join(dir_path, x) for x in os.listdir(dir_path) if x.lower().endswith('png')]
print(f'Found {len(images)} {class_name} examples')
return images
self.images = {}
self.class_names = ['Normal', 'Viral Pneumonia', 'COVID']
for class_name in self.class_names:
self.images[class_name] = get_images(class_name)
self.transform = transform
def __len__(self):
return sum([len(self.images[class_name]) for class_name in self.class_names])
def __getitem__(self, index):
class_name = random.choice(self.class_names)
index = index % len(self.images[class_name])
image_path = self.images[class_name][index]
image = Image.open(image_path).convert('RGB')
return self.transform(image), self.class_names.index(class_name)
def train_and_evaluate_model(dl_train, dl_test, class_names, model_file_name, epochs=1):
resnet18 = torchvision.models.resnet18(pretrained=True)
resnet18.fc = torch.nn.Linear(in_features=512, out_features=len(class_names))
loss_fn = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(resnet18.parameters(), lr=3e-5)
print(resnet18)
print('Starting training..')
for e in range(0, epochs):
print('=' * 20)
print(f'Starting epoch {e + 1}/{epochs}')
print('=' * 20)
train_loss = 0.
val_loss = 0.
resnet18.train() # set model to training phase
for train_step, (images, labels) in enumerate(dl_train):
optimizer.zero_grad()
outputs = resnet18(images)
loss = loss_fn(outputs, labels)
loss.backward()
optimizer.step()
train_loss += loss.item()
if train_step % 20 == 0:
print('Evaluating at step', train_step)
accuracy = 0
resnet18.eval() # set model to eval phase
for val_step, (images, labels) in enumerate(dl_test):
outputs = resnet18(images)
loss = loss_fn(outputs, labels)
val_loss += loss.item()
_, preds = torch.max(outputs, 1)
accuracy += sum((preds == labels).numpy())
val_loss /= (val_step + 1)
accuracy = accuracy / len(dl_test.dataset)
print(f'Validation Loss: {val_loss:.4f}, Accuracy: {accuracy:.4f}')
#show_preds()
resnet18.train()
if accuracy >= 0.95:
print('Performance condition satisfied, stopping..')
torch.save(resnet18.state_dict(), model_file_name)
return
train_loss /= (train_step + 1)
print(f'Training Loss: {train_loss:.4f}')
print('Training complete..')
torch.save(resnet18.state_dict(), model_file_name)
def main():
hospitals_zip = []
for i, arg in enumerate(sys.argv[2:]):
hospitals_zip.append(arg)
model_file_name = "model.pth"
class_names = ['Normal', 'Viral Pneumonia', 'COVID']
# Combine datasets from multiple hospitals
train_transform = torchvision.transforms.Compose([
torchvision.transforms.Resize(size=(224, 224)),
torchvision.transforms.RandomHorizontalFlip(),
torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
test_transform = torchvision.transforms.Compose([
torchvision.transforms.Resize(size=(224, 224)),
torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
hospital_path = '/tmp/work'
if not os.path.isdir(hospital_path):
os.mkdir(hospital_path, mode = 0o777)
for hospital in hospitals_zip:
with zipfile.ZipFile(hospital, 'r') as zip_ref:
zip_ref.extractall(hospital_path)
hospitals = []
for hospital in os.listdir(hospital_path):
hospitals.append(os.path.join(hospital_path, hospital))
# Prepare combined training dataset
train_dirs = {class_name: [] for class_name in class_names}
for hospital in hospitals:
for class_name in class_names:
train_dirs[class_name].append(os.path.join(hospital, class_name, 'images'))
train_image_dirs = {class_name: train_dirs[class_name] for class_name in class_names}
train_dataset = ChestXRayDataset(train_image_dirs, train_transform)
print(f'Total number of training images: {len(train_dataset)}')
# Prepare test dataset
test_dirs = {class_name: [] for class_name in class_names}
for hospital in hospitals:
test_dir = prepare_test_set(hospital, class_names)
for class_name in class_names:
test_dirs[class_name].append(os.path.join(test_dir, class_name))
test_image_dirs = {class_name: test_dirs[class_name] for class_name in class_names}
test_dataset = ChestXRayDataset(test_image_dirs, test_transform)
batch_size = 6
dl_train = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
dl_test = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size, shuffle=True)
print('Number of training batches:', len(dl_train))
print('Number of test batches:', len(dl_test))
train_and_evaluate_model(dl_train, dl_test, class_names, model_file_name, epochs=1)
# Define the path for the Unix domain socket
socket_path = sys.argv[1]
# Create a Unix domain socket client
client = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
# Connect to the server
client.connect(socket_path)
# Send the serialized model over the socket
with open(model_file_name, 'rb') as f:
data = f.read()
client.sendall(data)
finally:
# Close the socket
client.close()
if __name__ == '__main__':
ssl._create_default_https_context = ssl._create_unverified_context
main()
@@ -0,0 +1,190 @@
import os
import shutil
import random
import torch
import torchvision
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image
import argparse
import zipfile
import socket
import sys
def prepare_test_set(root_dir, class_names):
test_dir = os.path.join(root_dir, 'test')
if os.path.isdir(test_dir):
shutil.rmtree(test_dir)
os.mkdir(test_dir)
for class_name in class_names:
os.mkdir(os.path.join(test_dir, class_name))
for class_name in class_names:
images = [x for x in os.listdir(os.path.join(root_dir, class_name, "images")) if x.lower().endswith('png')]
selected_images = random.sample(images, 30)
for image in selected_images:
source_path = os.path.join(root_dir, class_name, "images", image)
target_path = os.path.join(test_dir, class_name, image)
shutil.copy(source_path, target_path)
return test_dir
class ChestXRayDataset(torch.utils.data.Dataset):
def __init__(self, image_dirs, transform):
def get_images(class_name):
images = []
for dir_path in image_dirs[class_name]:
images += [os.path.join(dir_path, x) for x in os.listdir(dir_path) if x.lower().endswith('png')]
print(f'Found {len(images)} {class_name} examples')
return images
self.images = {}
self.class_names = ['Normal', 'Viral Pneumonia', 'COVID']
for class_name in self.class_names:
self.images[class_name] = get_images(class_name)
self.transform = transform
def __len__(self):
return sum([len(self.images[class_name]) for class_name in self.class_names])
def __getitem__(self, index):
class_name = random.choice(self.class_names)
index = index % len(self.images[class_name])
image_path = self.images[class_name][index]
image = Image.open(image_path).convert('RGB')
return self.transform(image), self.class_names.index(class_name)
def train_and_evaluate_model(dl_train, dl_test, class_names, model_file_name, epochs=1):
resnet18 = torchvision.models.resnet18(pretrained=True)
resnet18.fc = torch.nn.Linear(in_features=512, out_features=len(class_names))
loss_fn = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(resnet18.parameters(), lr=3e-5)
print(resnet18)
print('Starting training..')
for e in range(0, epochs):
print('=' * 20)
print(f'Starting epoch {e + 1}/{epochs}')
print('=' * 20)
train_loss = 0.
val_loss = 0.
resnet18.train() # set model to training phase
for train_step, (images, labels) in enumerate(dl_train):
optimizer.zero_grad()
outputs = resnet18(images)
loss = loss_fn(outputs, labels)
loss.backward()
optimizer.step()
train_loss += loss.item()
if train_step % 20 == 0:
print('Evaluating at step', train_step)
accuracy = 0
resnet18.eval() # set model to eval phase
for val_step, (images, labels) in enumerate(dl_test):
outputs = resnet18(images)
loss = loss_fn(outputs, labels)
val_loss += loss.item()
_, preds = torch.max(outputs, 1)
accuracy += sum((preds == labels).numpy())
val_loss /= (val_step + 1)
accuracy = accuracy / len(dl_test.dataset)
print(f'Validation Loss: {val_loss:.4f}, Accuracy: {accuracy:.4f}')
#show_preds()
resnet18.train()
if accuracy >= 0.95:
print('Performance condition satisfied, stopping..')
torch.save(resnet18.state_dict(), model_file_name)
return
train_loss /= (train_step + 1)
print(f'Training Loss: {train_loss:.4f}')
print('Training complete..')
torch.save(resnet18.state_dict(), model_file_name)
def main():
# parser = argparse.ArgumentParser(description='Process hospital datasets and save the model.')
# parser.add_argument('hospitals', metavar='-H', type=str, nargs='+',
# help='paths to hospital datasets')
# parser.add_argument('--model', type=str, required=True,
# help='name of the output model file')
# args = parser.parse_args()
hospitals_zip = []
for i, arg in enumerate(sys.argv[2:]):
hospitals_zip.append(arg)
# hospitals_zip = args.hospitals
model_file_name = "model.pth"
class_names = ['Normal', 'Viral Pneumonia', 'COVID']
# Combine datasets from multiple hospitals
train_transform = torchvision.transforms.Compose([
torchvision.transforms.Resize(size=(224, 224)),
torchvision.transforms.RandomHorizontalFlip(),
torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
test_transform = torchvision.transforms.Compose([
torchvision.transforms.Resize(size=(224, 224)),
torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
hospitals = []
for hospital in hospitals_zip:
hospitals.append(os.path.basename(os.path.splitext(hospital)[0]))
with zipfile.ZipFile(hospital, 'r') as zip_ref:
zip_ref.extractall('./')
# Prepare combined training dataset
train_dirs = {class_name: [] for class_name in class_names}
for hospital in hospitals:
for class_name in class_names:
train_dirs[class_name].append(os.path.join(hospital, class_name, 'images'))
train_image_dirs = {class_name: train_dirs[class_name] for class_name in class_names}
train_dataset = ChestXRayDataset(train_image_dirs, train_transform)
print(f'Total number of training images: {len(train_dataset)}')
# Prepare test dataset
test_dirs = {class_name: [] for class_name in class_names}
for hospital in hospitals:
test_dir = prepare_test_set(hospital, class_names)
for class_name in class_names:
test_dirs[class_name].append(os.path.join(test_dir, class_name))
test_image_dirs = {class_name: test_dirs[class_name] for class_name in class_names}
test_dataset = ChestXRayDataset(test_image_dirs, test_transform)
batch_size = 6
dl_train = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
dl_test = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size, shuffle=True)
print('Number of training batches:', len(dl_train))
print('Number of test batches:', len(dl_test))
train_and_evaluate_model(dl_train, dl_test, class_names, model_file_name, epochs=1)
# Define the path for the Unix domain socket
socket_path = sys.argv[1]
# Create a Unix domain socket client
client = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
# Connect to the server
client.connect(socket_path)
# Send the serialized model over the socket
with open(model_file_name, 'rb') as f:
data = f.read()
client.sendall(data)
finally:
# Close the socket
client.close()
if __name__ == '__main__':
main()
+59
View File
@@ -0,0 +1,59 @@
import torch
import torchvision
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image
import argparse
def load_model(model_path, class_names):
model = torchvision.models.resnet18(weights=None)
model.fc = torch.nn.Linear(in_features=512, out_features=len(class_names))
model.load_state_dict(torch.load(model_path))
model.eval() # Set the model to evaluation mode
return model
def preprocess_image(image_path):
transform = torchvision.transforms.Compose([
torchvision.transforms.Resize(size=(224, 224)),
torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
image = Image.open(image_path).convert('RGB')
image = transform(image)
image = image.unsqueeze(0) # Add batch dimension
return image
def predict(model, image_path, class_names):
image = preprocess_image(image_path)
outputs = model(image)
_, preds = torch.max(outputs, 1)
predicted_class = class_names[preds.item()]
return predicted_class
def show_image_with_prediction(image_path, predicted_class):
image = Image.open(image_path)
plt.imshow(image)
plt.title(f'Predicted: {predicted_class}')
plt.axis('off')
plt.show()
def main():
parser = argparse.ArgumentParser(description='Load model and make a prediction.')
parser.add_argument('--model', type=str, required=True,
help='path to the model file')
parser.add_argument('--image', type=str, required=True,
help='path to the image for prediction')
args = parser.parse_args()
model_path = args.model
image_path = args.image
class_names = ['Normal', 'Viral Pneumonia', 'COVID']
model = load_model(model_path, class_names)
predicted_class = predict(model, image_path, class_names)
print(f'The predicted class for the image is: {predicted_class}')
show_image_with_prediction(image_path, predicted_class)
if __name__ == '__main__':
main()
+17
View File
@@ -0,0 +1,17 @@
# Algorithm
Agent accepts binaries programs. To use the python program you need to bundle or compile it.
In this example we'll use [pyinstaller](https://pypi.org/project/pyinstaller/)
```shell
pip install pandas scikit-learn
pip install -U pyinstaller
pyinstaller --onefile lin_reg.py
```
Make the binary static:
```shell
pip install staticx
staticx <dynamic_binary_file_path> <output_file_path>
```
@@ -6,8 +6,8 @@ import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
dataset = sys.argv[1]
iris = pd.read_csv(io.StringIO(dataset))
csv_file_path = sys.argv[2]
iris = pd.read_csv(csv_file_path)
# Droping the Species since we only need the measurements
X = iris.drop(['Species'], axis=1)
@@ -30,7 +30,7 @@ joblib.dump(log_reg, model_buffer)
model_bytes = model_buffer.getvalue()
# Define the path for the Unix domain socket
socket_path = sys.argv[2]
socket_path = sys.argv[1]
# Create a Unix domain socket client
client = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
+15
View File
@@ -0,0 +1,15 @@
{
"snp_policy": {
"minimum_guest_svn": 0,
"policy": 196608,
"minimum_tcb": 1,
"minimum_version": "1.0",
"minimum_launch_tcb": 1,
"measurement": [232, 141, 188, 114, 162, 221, 214, 6, 150, 248, 3, 173, 230, 39, 48, 120, 105, 243, 15, 242, 79, 67, 112, 128, 44, 119, 216, 226, 170, 255, 212, 154, 58, 68, 231, 30, 20, 235, 228, 42, 43, 1, 95, 191, 51, 113, 19, 72],
"minimum_build": 1
},
"root_of_trust": {
"product": "Milan",
"check_crl": true
}
}