COCOS-103 - User authN and AuthZ using digital signatures (#128)

* Update Go to 1.22 and enhance security features

- Upgraded the Go version in GitHub Actions workflows to 1.22.x for latest features and security patches.
- Added RSA public key field `UserKey` in `Dataset` and `Algorithm` to reinforce data integrity and encryption.
- Refactored `Result` method in `agentService` to use `containsID` for improved readability and potential performance benefits.
- Updated `grpcserver.New` and `internal/server/grpc` invocations to pass `agent.Service` by value in line with recommended Go practices.
- Introduced `grpc.StreamInterceptor` with no args in `Server.Start` which seems to be an initial step for future stream interceptor configuration.

These changes prepare for stronger data security measures, maintain compatibility with the latest Go features, and improve code quality regarding service struct usage. Potential follow-up is needed to configure the stream interceptor and to ensure the new RSA key field is appropriately utilized in data handling.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor auth system and protocol buffers

Enhanced the authentication system by adding context support and an improved user-role model. Implemented robust RSA public key verification for users and a restructured interceptor logic specific to stream types, streamlining the auth process. Updated protocol buffers and associated structures to accommodate user keys as byte slices, aligning with standard cryptographic practice. CLI commands for algorithms and datasets now require a private key file path argument for signing, strengthening security during interactions.

This comprehensive overhaul addresses security and efficiency considerations in the RPC framework and aligns with best practices for key handling. By streamlining and securing the user authentication process, the agent service's reliability is greatly improved, directly impacting the robustness of the entire computation pipeline.

- Refactored auth: added role-based user validation, context handling
- Reworked interceptors: separated stream types, fortified signature checks
- Updated protocol buffers: user public keys as byte slices for standard compatibility
- Enhanced CLI: introduced private key argument, ensuring secure algorithm and dataset submission
- Improved server and SDK contracts to align with auth changes

Related issues:
- Implements user roles and auth context [#103]
- CLI security enhancement for private key management

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Updated PEM decoding for key parsing in CLI and tests

Added `encoding/pem` to decode PEM blocks when parsing private and public keys across CLI commands and test computation scenarios, ensuring compatibility with key files. This enhances robustness in key handling by supporting PEM encoded keys. The update also includes registration of a new Keys command in the CLI.

Refactored code is now compliant with common key formats, addressing potential parsing issues.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix auth signature encoding and improve CLI usage example

The authentication system now decodes base64 strings before verifying signatures to align with the expected format. Additionally, the signature generation now encodes the output in base64, ensuring consistency across the auth process.

The CLI help message for the `result` command is enhanced by providing a usage example, making it more user-friendly and informative.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor containsID to handle dynamic fields

Updated the `containsID` function to accept a field name parameter, enabling dynamic field lookup within the reflection logic. This change facilitates the use of the function for various struct fields, improving code reusability and flexibility.

CLI command 'data' now requires an additional argument for the private key file path, outlined in the usage example update, reinforcing command clarity and user guidance.

Resolves issues with hardcoded field lookups and enhances CLI usability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Remove extraneous newline in key generation log output

A redundant newline after the success message in the key generation command was removed to clean up log output formatting. This change ensures a more consistent and professional appearance of the CLI tool's messages.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Implemented auth service in gRPC startup

Added authentication services to the gRPC server initialization to enforce security measures. The gRPC server's New function now includes an `authSvc` parameter, requiring instantiation of the auth service before starting the server. Failure to create the auth service results in a fatal error, halting the process to avoid running without protection. Tests have been updated to include `nil` values for the auth service parameter to maintain their functionality without authentication.

Refactored `grpcserver.New` to accept the new auth service, and updated the main agent startup logic to create and inject the auth service. Added the auth middleware interceptors to the server options, which ensures that each gRPC call will undergo authentication.

This change is a step towards secure communication, and affected components should now consider the authentication requirement.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor config read logic and update agent setup

Improved the configuration reading in `cmd/agent/main.go` to handle larger payloads by reading data in chunks and checking for EOF, ensuring that all config data is captured even if it exceeds the initial buffer size. Enhanced the `test/manual/agent-config/main.go` to require additional command-line arguments, improving the setup process by explicitly requiring paths for data, algorithm, and public key as well as a boolean for attested TLS. Also updated the hashing method to SHA3 for the algorithm and data files, and included the hash and public keys as part of the agent, dataset, and result consumer configurations. These changes will make the agent setup more robust and provide better integrity checks for the involved files.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor run method to agentService

Moved the run function into agentService for better encapsulation and maintainability. This refactoring includes capturing both stdout and stderr during algorithm execution, enabling more informative debugging through enhanced logging. Consequentially, the run method now references members through the service instance, aligning with object-oriented best practices and improving code coherence.

Resolves issue with insufficient execution details when computations fail.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor computation data handling to use filepaths

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor error logging and ensure consistency

Replaced usage of the standard log package with a custom logger for error reporting to standardize error logging throughout the application. Additionally, introduced graceful shutdown by returning from the main function rather than forcing exit when failing to create auth service, aligning the application's error handling strategy.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor auth initialization and key file handling

Improved the readability and maintainability of the authentication service initialization by adding line breaks for logical separation. Also, standardized key filenames in the CLI key generation by introducing constants, enhancing code clarity and reducing the likelihood of file-naming errors.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor auth verification logic for improved security

Removed an extraneous line in the `verifySignature` function that was not necessary for the signature verification process. This change simplifies the code and improves readability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor payload structures to simplify API

Removed the 'provider', 'id', 'consumer' fields from protocol buffers, gRPC services, and related functions across various files to streamline the data model and align with the new authentication system based on cryptographic verification rather than string identifiers. This results in more efficient data handling and a reduction in unnecessary payload data, while enhancing security by making entity validation strictly cryptographic.

The changes affect agent-SDK interactions, CLI tools, and related services, ensuring only the necessary data (algorithm/data bytes, user keys, and hashes) is transmitted and processed. Consequently, the core computation algorithm and dataset handlers now rely on indexes derived from context to associate data with respective manifest entries, thus maintaining the ability to link to specific computation manifests without relying on explicit IDs in the payload. Additionally, refactored authentication methods now enforce role-based security seamlessly through metadata.

This approach enhances privacy by avoiding transmission of potentially sensitive strings over the network and by ensuring that only internal indices, not globally interpretable identifiers, are used to process computations.

Aligned with the broader architectural goal of simplifying and securing the platform's core services, this change paves the way for upcoming revisions to the authentication scheme that will further consolidate role-based security and improve system integrity.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance CLI security with key paths

Removed the section on running computations from the CLI README as it may no longer be necessary or the functionality has been moved elsewhere. Required private key file paths for algorithm, dataset upload, and result retrieval commands to enhance security. This change associates each action with a specific identity, ensuring secure and traceable operations. Additionally, updated the manual test commands to reflect this new requirement.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix ci

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix fmt

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
This commit is contained in:
Sammy Kerata Oina
2024-05-28 15:10:13 +03:00
committed by GitHub
parent aebe01a873
commit 2ce112cc1b
32 changed files with 741 additions and 394 deletions
+1 -1
View File
@@ -29,7 +29,7 @@ jobs:
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
go-version: 1.22.x
- name: Set up protoc
run: |
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
go-version: 1.22.x
cache-dependency-path: "go.sum"
- name: Checkout cocos
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
- name: Install Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
go-version: 1.22.x
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
+2
View File
@@ -6,3 +6,5 @@ cmd/manager/iso
cmd/manager/tmp
.cov
*.pem
+34 -82
View File
@@ -29,8 +29,6 @@ type AlgoRequest struct {
unknownFields protoimpl.UnknownFields
Algorithm []byte `protobuf:"bytes,1,opt,name=algorithm,proto3" json:"algorithm,omitempty"`
Provider string `protobuf:"bytes,2,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,3,opt,name=id,proto3" json:"id,omitempty"`
}
func (x *AlgoRequest) Reset() {
@@ -72,20 +70,6 @@ func (x *AlgoRequest) GetAlgorithm() []byte {
return nil
}
func (x *AlgoRequest) GetProvider() string {
if x != nil {
return x.Provider
}
return ""
}
func (x *AlgoRequest) GetId() string {
if x != nil {
return x.Id
}
return ""
}
type AlgoResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -129,9 +113,7 @@ type DataRequest struct {
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Dataset []byte `protobuf:"bytes,1,opt,name=dataset,proto3" json:"dataset,omitempty"`
Provider string `protobuf:"bytes,2,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,3,opt,name=id,proto3" json:"id,omitempty"`
Dataset []byte `protobuf:"bytes,1,opt,name=dataset,proto3" json:"dataset,omitempty"`
}
func (x *DataRequest) Reset() {
@@ -173,20 +155,6 @@ func (x *DataRequest) GetDataset() []byte {
return nil
}
func (x *DataRequest) GetProvider() string {
if x != nil {
return x.Provider
}
return ""
}
func (x *DataRequest) GetId() string {
if x != nil {
return x.Id
}
return ""
}
type DataResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -229,8 +197,6 @@ type ResultRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Consumer string `protobuf:"bytes,1,opt,name=consumer,proto3" json:"consumer,omitempty"`
}
func (x *ResultRequest) Reset() {
@@ -265,13 +231,6 @@ func (*ResultRequest) Descriptor() ([]byte, []int) {
return file_agent_agent_proto_rawDescGZIP(), []int{4}
}
func (x *ResultRequest) GetConsumer() string {
if x != nil {
return x.Consumer
}
return ""
}
type ResultResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -417,48 +376,41 @@ var File_agent_agent_proto protoreflect.FileDescriptor
var file_agent_agent_proto_rawDesc = []byte{
0x0a, 0x11, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2f, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x12, 0x05, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x22, 0x57, 0x0a, 0x0b, 0x41, 0x6c,
0x6f, 0x74, 0x6f, 0x12, 0x05, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x22, 0x2b, 0x0a, 0x0b, 0x41, 0x6c,
0x67, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1c, 0x0a, 0x09, 0x61, 0x6c, 0x67,
0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x09, 0x61, 0x6c,
0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x1a, 0x0a, 0x08, 0x70, 0x72, 0x6f, 0x76, 0x69,
0x64, 0x65, 0x72, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x70, 0x72, 0x6f, 0x76, 0x69,
0x64, 0x65, 0x72, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09, 0x52,
0x02, 0x69, 0x64, 0x22, 0x0e, 0x0a, 0x0c, 0x41, 0x6c, 0x67, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f,
0x6e, 0x73, 0x65, 0x22, 0x53, 0x0a, 0x0b, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65,
0x73, 0x74, 0x12, 0x18, 0x0a, 0x07, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x18, 0x01, 0x20,
0x01, 0x28, 0x0c, 0x52, 0x07, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x12, 0x1a, 0x0a, 0x08,
0x70, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08,
0x70, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65, 0x72, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x03,
0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x22, 0x0e, 0x0a, 0x0c, 0x44, 0x61, 0x74, 0x61,
0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x2b, 0x0a, 0x0d, 0x52, 0x65, 0x73, 0x75,
0x6c, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1a, 0x0a, 0x08, 0x63, 0x6f, 0x6e,
0x73, 0x75, 0x6d, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x63, 0x6f, 0x6e,
0x73, 0x75, 0x6d, 0x65, 0x72, 0x22, 0x24, 0x0a, 0x0e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52,
0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x18,
0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x22, 0x35, 0x0a, 0x12, 0x41,
0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x12, 0x1f, 0x0a, 0x0b, 0x72, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x5f, 0x64, 0x61, 0x74, 0x61,
0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x0a, 0x72, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x44, 0x61,
0x74, 0x61, 0x22, 0x29, 0x0a, 0x13, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f,
0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c,
0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x32, 0xf9, 0x01,
0x0a, 0x0c, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x33,
0x0a, 0x04, 0x41, 0x6c, 0x67, 0x6f, 0x12, 0x12, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41,
0x6c, 0x67, 0x6f, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x13, 0x2e, 0x61, 0x67, 0x65,
0x6e, 0x74, 0x2e, 0x41, 0x6c, 0x67, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22,
0x00, 0x28, 0x01, 0x12, 0x33, 0x0a, 0x04, 0x44, 0x61, 0x74, 0x61, 0x12, 0x12, 0x2e, 0x61, 0x67,
0x65, 0x6e, 0x74, 0x2e, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a,
0x13, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70,
0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x28, 0x01, 0x12, 0x37, 0x0a, 0x06, 0x52, 0x65, 0x73, 0x75,
0x6c, 0x74, 0x12, 0x14, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x52, 0x65, 0x73, 0x75, 0x6c,
0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x15, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74,
0x2e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22,
0x00, 0x12, 0x46, 0x0a, 0x0b, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e,
0x12, 0x19, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61,
0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1a, 0x2e, 0x61, 0x67,
0x65, 0x6e, 0x74, 0x2e, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52,
0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x42, 0x09, 0x5a, 0x07, 0x2e, 0x2f, 0x61,
0x67, 0x65, 0x6e, 0x74, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x22, 0x0e, 0x0a, 0x0c, 0x41, 0x6c, 0x67, 0x6f, 0x52,
0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x27, 0x0a, 0x0b, 0x44, 0x61, 0x74, 0x61, 0x52,
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x18, 0x0a, 0x07, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65,
0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x64, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74,
0x22, 0x0e, 0x0a, 0x0c, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x22, 0x0f, 0x0a, 0x0d, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x22, 0x24, 0x0a, 0x0e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f,
0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28,
0x0c, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x22, 0x35, 0x0a, 0x12, 0x41, 0x74, 0x74, 0x65, 0x73,
0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1f, 0x0a,
0x0b, 0x72, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x5f, 0x64, 0x61, 0x74, 0x61, 0x18, 0x01, 0x20, 0x01,
0x28, 0x0c, 0x52, 0x0a, 0x72, 0x65, 0x70, 0x6f, 0x72, 0x74, 0x44, 0x61, 0x74, 0x61, 0x22, 0x29,
0x0a, 0x13, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x73,
0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x01, 0x20,
0x01, 0x28, 0x0c, 0x52, 0x04, 0x66, 0x69, 0x6c, 0x65, 0x32, 0xf9, 0x01, 0x0a, 0x0c, 0x41, 0x67,
0x65, 0x6e, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x33, 0x0a, 0x04, 0x41, 0x6c,
0x67, 0x6f, 0x12, 0x12, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41, 0x6c, 0x67, 0x6f, 0x52,
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x13, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41,
0x6c, 0x67, 0x6f, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x28, 0x01, 0x12,
0x33, 0x0a, 0x04, 0x44, 0x61, 0x74, 0x61, 0x12, 0x12, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e,
0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x13, 0x2e, 0x61, 0x67,
0x65, 0x6e, 0x74, 0x2e, 0x44, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x22, 0x00, 0x28, 0x01, 0x12, 0x37, 0x0a, 0x06, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x12, 0x14,
0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x52, 0x65, 0x71,
0x75, 0x65, 0x73, 0x74, 0x1a, 0x15, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e, 0x52, 0x65, 0x73,
0x75, 0x6c, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x46, 0x0a,
0x0b, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x19, 0x2e, 0x61,
0x67, 0x65, 0x6e, 0x74, 0x2e, 0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e,
0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1a, 0x2e, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x2e,
0x41, 0x74, 0x74, 0x65, 0x73, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f,
0x6e, 0x73, 0x65, 0x22, 0x00, 0x42, 0x09, 0x5a, 0x07, 0x2e, 0x2f, 0x61, 0x67, 0x65, 0x6e, 0x74,
0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (
-5
View File
@@ -16,22 +16,17 @@ service AgentService {
message AlgoRequest {
bytes algorithm = 1;
string provider = 2;
string id = 3;
}
message AlgoResponse {}
message DataRequest {
bytes dataset = 1;
string provider = 2;
string id = 3;
}
message DataResponse {}
message ResultRequest {
string consumer = 1;
}
message ResultResponse {
+3 -3
View File
@@ -17,7 +17,7 @@ func algoEndpoint(svc agent.Service) endpoint.Endpoint {
return algoRes{}, err
}
algo := agent.Algorithm{Algorithm: req.Algorithm, Provider: req.Provider, ID: req.Id}
algo := agent.Algorithm{Algorithm: req.Algorithm}
err := svc.Algo(ctx, algo)
if err != nil {
@@ -36,7 +36,7 @@ func dataEndpoint(svc agent.Service) endpoint.Endpoint {
return dataRes{}, err
}
dataset := agent.Dataset{Dataset: req.Dataset, Provider: req.Provider, ID: req.Id}
dataset := agent.Dataset{Dataset: req.Dataset}
err := svc.Data(ctx, dataset)
if err != nil {
@@ -54,7 +54,7 @@ func resultEndpoint(svc agent.Service) endpoint.Endpoint {
if err := req.validate(); err != nil {
return resultRes{}, err
}
file, err := svc.Result(ctx, req.Consumer)
file, err := svc.Result(ctx)
if err != nil {
return resultRes{}, err
}
+2 -20
View File
@@ -8,45 +8,27 @@ import (
type algoReq struct {
Algorithm []byte `protobuf:"bytes,1,opt,name=algorithm,proto3" json:"algorithm,omitempty"`
Provider string `protobuf:"bytes,2,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,3,opt,name=id,proto3" json:"id,omitempty"`
}
func (req algoReq) validate() error {
if len(req.Algorithm) == 0 {
return errors.New("algorithm binary is required")
}
if req.Id == "" {
return errors.New("malformed entity")
}
if req.Provider == "" {
return errors.New("malformed entity")
}
return nil
}
type dataReq struct {
Dataset []byte `protobuf:"bytes,1,opt,name=dataset,proto3" json:"dataset,omitempty"`
Provider string `protobuf:"bytes,2,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,3,opt,name=id,proto3" json:"id,omitempty"`
Dataset []byte `protobuf:"bytes,1,opt,name=dataset,proto3" json:"dataset,omitempty"`
}
func (req dataReq) validate() error {
if len(req.Dataset) == 0 {
return errors.New("dataset CSV file is required")
}
if req.Id == "" {
return errors.New("malformed entity")
}
if req.Provider == "" {
return errors.New("malformed entity")
}
return nil
}
type resultReq struct {
Consumer string `protobuf:"bytes,1,opt,name=consumer,proto3" json:"consumer,omitempty"`
}
type resultReq struct{}
func (req resultReq) validate() error {
// No request parameters to validate, so no validation logic needed
+4 -15
View File
@@ -54,8 +54,6 @@ func decodeAlgoRequest(_ context.Context, grpcReq interface{}) (interface{}, err
return algoReq{
Algorithm: req.Algorithm,
Provider: req.Provider,
Id: req.Id,
}, nil
}
@@ -67,9 +65,7 @@ func decodeDataRequest(_ context.Context, grpcReq interface{}) (interface{}, err
req := grpcReq.(*agent.DataRequest)
return dataReq{
Dataset: req.Dataset,
Provider: req.Provider,
Id: req.Id,
Dataset: req.Dataset,
}, nil
}
@@ -78,8 +74,7 @@ func encodeDataResponse(_ context.Context, response interface{}) (interface{}, e
}
func decodeResultRequest(_ context.Context, grpcReq interface{}) (interface{}, error) {
req := grpcReq.(*agent.ResultRequest)
return resultReq{Consumer: req.Consumer}, nil
return resultReq{}, nil
}
func encodeResultResponse(_ context.Context, response interface{}) (interface{}, error) {
@@ -107,7 +102,6 @@ func encodeAttestationResponse(_ context.Context, response interface{}) (interfa
// Algo implements agent.AgentServiceServer.
func (s *grpcServer) Algo(stream agent.AgentService_AlgoServer) error {
var algoFile []byte
var provider, id string
for {
algoChunk, err := stream.Recv()
if err == io.EOF {
@@ -116,11 +110,9 @@ func (s *grpcServer) Algo(stream agent.AgentService_AlgoServer) error {
if err != nil {
return status.Error(codes.Internal, err.Error())
}
provider = algoChunk.Provider
id = algoChunk.Id
algoFile = append(algoFile, algoChunk.Algorithm...)
}
_, res, err := s.algo.ServeGRPC(stream.Context(), &agent.AlgoRequest{Algorithm: algoFile, Provider: provider, Id: id})
_, res, err := s.algo.ServeGRPC(stream.Context(), &agent.AlgoRequest{Algorithm: algoFile})
if err != nil {
return err
}
@@ -131,7 +123,6 @@ func (s *grpcServer) Algo(stream agent.AgentService_AlgoServer) error {
// Data implements agent.AgentServiceServer.
func (s *grpcServer) Data(stream agent.AgentService_DataServer) error {
var dataFile []byte
var provider, id string
for {
dataChunk, err := stream.Recv()
if err == io.EOF {
@@ -140,11 +131,9 @@ func (s *grpcServer) Data(stream agent.AgentService_DataServer) error {
if err != nil {
return status.Error(codes.Internal, err.Error())
}
provider = dataChunk.Provider
id = dataChunk.Id
dataFile = append(dataFile, dataChunk.Dataset...)
}
_, res, err := s.data.ServeGRPC(stream.Context(), &agent.DataRequest{Dataset: dataFile, Provider: provider, Id: id})
_, res, err := s.data.ServeGRPC(stream.Context(), &agent.DataRequest{Dataset: dataFile})
if err != nil {
return err
}
+2 -2
View File
@@ -53,7 +53,7 @@ func (lm *loggingMiddleware) Data(ctx context.Context, dataset agent.Dataset) (e
return lm.svc.Data(ctx, dataset)
}
func (lm *loggingMiddleware) Result(ctx context.Context, consumer string) (response []byte, err error) {
func (lm *loggingMiddleware) Result(ctx context.Context) (response []byte, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method Result took %s to complete", time.Since(begin))
if err != nil {
@@ -63,7 +63,7 @@ func (lm *loggingMiddleware) Result(ctx context.Context, consumer string) (respo
lm.logger.Info(fmt.Sprintf("%s without errors", message))
}(time.Now())
return lm.svc.Result(ctx, consumer)
return lm.svc.Result(ctx)
}
func (lm *loggingMiddleware) Attestation(ctx context.Context, reportData [agent.ReportDataSize]byte) (response []byte, err error) {
+2 -2
View File
@@ -50,13 +50,13 @@ func (ms *metricsMiddleware) Data(ctx context.Context, dataset agent.Dataset) er
return ms.svc.Data(ctx, dataset)
}
func (ms *metricsMiddleware) Result(ctx context.Context, consumer string) ([]byte, error) {
func (ms *metricsMiddleware) Result(ctx context.Context) ([]byte, error) {
defer func(begin time.Time) {
ms.counter.With("method", "result").Add(1)
ms.latency.With("method", "result").Observe(time.Since(begin).Seconds())
}(time.Now())
return ms.svc.Result(ctx, consumer)
return ms.svc.Result(ctx)
}
func (ms *metricsMiddleware) Attestation(ctx context.Context, reportData [agent.ReportDataSize]byte) ([]byte, error) {
+180
View File
@@ -0,0 +1,180 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package auth
import (
"context"
"crypto"
"crypto/rsa"
"crypto/sha256"
"crypto/x509"
"encoding/base64"
"errors"
"github.com/ultravioletrs/cocos/agent"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata"
"google.golang.org/grpc/status"
)
type UserRole string
const (
UserMetadataKey = "user-id"
SignatureMetadataKey = "signature"
ConsumerRole UserRole = "consumer"
DataProviderRole UserRole = "data-provider"
AlgorithmProviderRole UserRole = "algorithm-provider"
)
var errNotRSAPublicKey = errors.New("not an RSA public key")
type wrappedServerStream struct {
grpc.ServerStream
ctx context.Context
}
func (s *wrappedServerStream) Context() context.Context {
return s.ctx
}
type Service struct {
resultConsumers []*rsa.PublicKey
datasetProviders []*rsa.PublicKey
algorithmProvider *rsa.PublicKey
}
func New(manifest agent.Computation) (*Service, error) {
s := &Service{}
for _, rc := range manifest.ResultConsumers {
pubKey, err := x509.ParsePKIXPublicKey(rc.UserKey)
if err != nil {
return nil, err
}
rsaPubKey, ok := pubKey.(*rsa.PublicKey)
if !ok {
return nil, errNotRSAPublicKey
}
s.resultConsumers = append(s.resultConsumers, rsaPubKey)
}
for _, dp := range manifest.Datasets {
pubKey, err := x509.ParsePKIXPublicKey(dp.UserKey)
if err != nil {
return nil, err
}
rsaPubKey, ok := pubKey.(*rsa.PublicKey)
if !ok {
return nil, errNotRSAPublicKey
}
s.datasetProviders = append(s.datasetProviders, rsaPubKey)
}
pubKey, err := x509.ParsePKIXPublicKey(manifest.Algorithm.UserKey)
if err != nil {
return nil, err
}
rsaPubKey, ok := pubKey.(*rsa.PublicKey)
if !ok {
return nil, errNotRSAPublicKey
}
s.algorithmProvider = rsaPubKey
return s, nil
}
func (s *Service) AuthStreamInterceptor() grpc.StreamServerInterceptor {
return func(srv interface{}, stream grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
switch info.FullMethod {
case agent.AgentService_Algo_FullMethodName:
md, ok := metadata.FromIncomingContext(stream.Context())
if !ok {
return status.Errorf(codes.Unauthenticated, "missing metadata")
}
signature, err := extractSignature(md)
if err != nil {
return status.Errorf(codes.Unauthenticated, "invalid metadata")
}
isValid, err := verifySignature(AlgorithmProviderRole, signature, s.algorithmProvider)
if err != nil || !isValid {
return status.Errorf(codes.Unauthenticated, "signature verification failed")
}
case agent.AgentService_Data_FullMethodName:
md, ok := metadata.FromIncomingContext(stream.Context())
if !ok {
return status.Errorf(codes.Unauthenticated, "missing metadata")
}
signature, err := extractSignature(md)
if err != nil {
return status.Errorf(codes.Unauthenticated, "invalid metadata")
}
for index, dp := range s.datasetProviders {
isValid, err := verifySignature(DataProviderRole, signature, dp)
if err == nil || isValid {
ctx := agent.IndexToContext(stream.Context(), index)
wrapped := &wrappedServerStream{ServerStream: stream, ctx: ctx}
return handler(srv, wrapped)
}
}
return status.Errorf(codes.Unauthenticated, "signature verification failed")
default:
return handler(srv, stream)
}
return handler(srv, stream)
}
}
func (s *Service) AuthUnaryInterceptor() grpc.UnaryServerInterceptor {
return func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
switch info.FullMethod {
case agent.AgentService_Result_FullMethodName:
md, ok := metadata.FromIncomingContext(ctx)
if !ok {
return nil, status.Errorf(codes.Unauthenticated, "missing metadata")
}
signature, err := extractSignature(md)
if err != nil {
return nil, status.Errorf(codes.Unauthenticated, "invalid metadata")
}
for index, rc := range s.resultConsumers {
isValid, err := verifySignature(ConsumerRole, signature, rc)
if err == nil || isValid {
ctx := agent.IndexToContext(ctx, index)
return handler(ctx, req)
}
}
return nil, status.Errorf(codes.Unauthenticated, "signature verification failed")
default:
return handler(ctx, req)
}
}
}
func extractSignature(md metadata.MD) (string, error) {
signature := md.Get(SignatureMetadataKey)
if len(signature) != 1 {
return "", status.Errorf(codes.Unauthenticated, "invalid metadata")
}
return signature[0], nil
}
func verifySignature(role UserRole, signature string, publicKey *rsa.PublicKey) (bool, error) {
hash := sha256.Sum256([]byte(role))
sigByte, err := base64.StdEncoding.DecodeString(signature)
if err != nil {
return false, err
}
if err := rsa.VerifyPKCS1v15(publicKey, crypto.SHA256, hash[:], sigByte); err != nil {
return false, err
}
return true, nil
}
+25 -26
View File
@@ -3,9 +3,9 @@
package agent
import (
"context"
"encoding/json"
"fmt"
"reflect"
)
var _ fmt.Stringer = (*Datasets)(nil)
@@ -22,13 +22,17 @@ type AgentConfig struct {
}
type Computation struct {
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Datasets Datasets `json:"datasets,omitempty"`
Algorithm Algorithm `json:"algorithm,omitempty"`
ResultConsumers []string `json:"result_consumers,omitempty"`
AgentConfig AgentConfig `json:"agent_config,omitempty"`
ID string `json:"id,omitempty"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Datasets Datasets `json:"datasets,omitempty"`
Algorithm Algorithm `json:"algorithm,omitempty"`
ResultConsumers []ResultConsumer `json:"result_consumers,omitempty"`
AgentConfig AgentConfig `json:"agent_config,omitempty"`
}
type ResultConsumer struct {
UserKey []byte `json:"user_key,omitempty"`
}
func (d *Datasets) String() string {
@@ -40,10 +44,9 @@ func (d *Datasets) String() string {
}
type Dataset struct {
Dataset []byte `json:"-"`
Hash [32]byte `json:"hash,omitempty"`
Provider string `json:"provider,omitempty"`
ID string `json:"id,omitempty"`
Dataset []byte `json:"-"`
Hash [32]byte `json:"hash,omitempty"`
UserKey []byte `json:"user_key,omitempty"`
}
type Datasets []Dataset
@@ -51,20 +54,16 @@ type Datasets []Dataset
type Algorithm struct {
Algorithm []byte `json:"-"`
Hash [32]byte `json:"hash,omitempty"`
Provider string `json:"provider,omitempty"`
ID string `json:"id,omitempty"`
UserKey []byte `json:"user_key,omitempty"`
}
func containsID(slice interface{}, id string) int {
rangeOnMe := reflect.ValueOf(slice)
for i := 0; i < rangeOnMe.Len(); i++ {
s := rangeOnMe.Index(i)
f := s.FieldByName("ID")
if f.IsValid() {
if f.Interface() == id {
return i
}
}
}
return -1
type ManifestIndexKey struct{}
func IndexToContext(ctx context.Context, index int) context.Context {
return context.WithValue(ctx, ManifestIndexKey{}, index)
}
func IndexFromContext(ctx context.Context) (int, bool) {
index, ok := ctx.Value(ManifestIndexKey{}).(int)
return index, ok
}
+22 -33
View File
@@ -4,6 +4,7 @@
package agent
import (
"bytes"
"context"
"encoding/json"
"errors"
@@ -36,11 +37,7 @@ var (
// when accessing a protected resource.
ErrUnauthorizedAccess = errors.New("missing or invalid credentials provided")
// errUndeclaredAlgorithm indicates algorithm was not declared in computation manifest.
errUndeclaredAlgorithm = errors.New("algorithm not declared in computation manifest")
// errUndeclaredAlgorithm indicates algorithm was not declared in computation manifest.
errUndeclaredDataset = errors.New("dataset not declared in computation manifest")
// errProviderMissmatch algorithm/dataset provider does not match computation manifest.
errProviderMissmatch = errors.New("provider does not match declaration on manifest")
// errAllManifestItemsReceived indicates no new computation manifest items expected.
errAllManifestItemsReceived = errors.New("all expected manifest Items have been received")
// errUndeclaredConsumer indicates the consumer requesting results in not declared in computation manifest.
@@ -58,7 +55,7 @@ var (
type Service interface {
Algo(ctx context.Context, algorithm Algorithm) error
Data(ctx context.Context, dataset Dataset) error
Result(ctx context.Context, consumer string) ([]byte, error)
Result(ctx context.Context) ([]byte, error)
Attestation(ctx context.Context, reportData [ReportDataSize]byte) ([]byte, error)
}
@@ -106,14 +103,6 @@ func (as *agentService) Algo(ctx context.Context, algorithm Algorithm) error {
hash := sha3.Sum256(algorithm.Algorithm)
if as.computation.Algorithm.ID != algorithm.ID {
return errUndeclaredAlgorithm
}
if as.computation.Algorithm.Provider != algorithm.Provider {
return errProviderMissmatch
}
if hash != as.computation.Algorithm.Hash {
return errHashMismatch
}
@@ -154,21 +143,17 @@ func (as *agentService) Data(ctx context.Context, dataset Dataset) error {
hash := sha3.Sum256(dataset.Dataset)
index := containsID(as.computation.Datasets, dataset.ID)
switch index {
case -1:
index, ok := IndexFromContext(ctx)
if !ok {
return errUndeclaredDataset
default:
if as.computation.Datasets[index].Provider != dataset.Provider {
return errProviderMissmatch
}
if hash != as.computation.Datasets[index].Hash {
return errHashMismatch
}
as.computation.Datasets = slices.Delete(as.computation.Datasets, index, index+1)
}
f, err := os.CreateTemp("", fmt.Sprintf("dataset-%s", dataset.ID))
if hash != as.computation.Datasets[index].Hash {
return errHashMismatch
}
as.computation.Datasets = slices.Delete(as.computation.Datasets, index, index+1)
f, err := os.CreateTemp("", fmt.Sprintf("dataset-%d", index))
if err != nil {
return fmt.Errorf("error creating dataset file: %v", err)
}
@@ -189,20 +174,18 @@ func (as *agentService) Data(ctx context.Context, dataset Dataset) error {
return nil
}
func (as *agentService) Result(ctx context.Context, consumer string) ([]byte, error) {
func (as *agentService) Result(ctx context.Context) ([]byte, error) {
if as.sm.GetState() != resultsReady {
return []byte{}, errResultsNotReady
}
if len(as.computation.ResultConsumers) == 0 {
return []byte{}, errAllManifestItemsReceived
}
index := slices.Index(as.computation.ResultConsumers, consumer)
switch index {
case -1:
index, ok := IndexFromContext(ctx)
if !ok {
return []byte{}, errUndeclaredConsumer
default:
as.computation.ResultConsumers = slices.Delete(as.computation.ResultConsumers, index, index+1)
}
as.computation.ResultConsumers = slices.Delete(as.computation.ResultConsumers, index, index+1)
if len(as.computation.ResultConsumers) == 0 {
as.sm.SendEvent(resultsConsumed)
@@ -229,7 +212,7 @@ func (as *agentService) runComputation() {
as.sm.logger.Debug("computation run started")
defer as.sm.SendEvent(runComplete)
as.publishEvent("in-progress", json.RawMessage{})()
result, err := run(as.algorithm, as.datasets)
result, err := as.run(as.algorithm, as.datasets)
if err != nil {
as.runError = err
as.sm.logger.Warn(fmt.Sprintf("computation failed with error: %s", err.Error()))
@@ -248,7 +231,7 @@ func (as *agentService) publishEvent(status string, details json.RawMessage) fun
}
}
func run(algoFile string, dataFiles []string) ([]byte, error) {
func (as *agentService) run(algoFile string, dataFiles []string) ([]byte, error) {
defer os.Remove(algoFile)
defer func() {
for _, file := range dataFiles {
@@ -267,21 +250,27 @@ func run(algoFile string, dataFiles []string) ([]byte, error) {
var result []byte
var outStd, outErr bytes.Buffer
go socket.AcceptConnection(listener, dataChannel, errorChannel)
args := append([]string{socketPath}, dataFiles...)
cmd := exec.Command(algoFile, args...)
cmd.Stderr = &outErr
cmd.Stdout = &outStd
if err := cmd.Start(); err != nil {
return nil, fmt.Errorf("error starting algorithm: %v", err)
}
if err := cmd.Wait(); err != nil {
as.sm.logger.Debug(outErr.String())
return nil, fmt.Errorf("algorithm execution error: %v", err)
}
select {
case result = <-dataChannel:
as.sm.logger.Debug(outStd.String())
return result, nil
case err = <-errorChannel:
return nil, fmt.Errorf("error receiving data: %v", err)
+3 -11
View File
@@ -12,14 +12,6 @@ make cli
## Usage
#### Run Computation
To run a computation, use the following command:
```bash
./build/cocos-cli manager run --computation '{"name": "my-computation"}'
```
#### Get attestation
Retrieves attestation information from the SEV guest and saves it to a file.
To retrieve attestation from agent, use the following command:
@@ -70,7 +62,7 @@ To validate and verify attestation from agent, use the following command:
To upload an algorithm, use the following command:
```bash
./build/cocos-cli agent algo /path/to/algorithm
./build/cocos-cli agent algo /path/to/algorithm <private_key_file_path>
```
#### Upload Dataset
@@ -78,7 +70,7 @@ To upload an algorithm, use the following command:
To upload a dataset, use the following command:
```bash
./build/cocos-cli agent data /path/to/dataset.csv
./build/cocos-cli agent data /path/to/dataset.csv <private_key_file_path>
```
#### Retrieve result
@@ -86,5 +78,5 @@ To upload a dataset, use the following command:
To retrieve the computation result, use the following command:
```bash
./build/cocos-cli agent result
./build/cocos-cli agent result <private_key_file_path>
```
+18 -6
View File
@@ -3,6 +3,8 @@
package cli
import (
"crypto/x509"
"encoding/pem"
"log"
"os"
@@ -14,8 +16,8 @@ func (cli *CLI) NewAlgorithmCmd() *cobra.Command {
return &cobra.Command{
Use: "algo",
Short: "Upload an algorithm binary",
Example: "algo <algo_file> <id> <provider>",
Args: cobra.ExactArgs(3),
Example: "algo <algo_file> <private_key_file_path>",
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
algorithmFile := args[0]
@@ -28,12 +30,22 @@ func (cli *CLI) NewAlgorithmCmd() *cobra.Command {
algoReq := agent.Algorithm{
Algorithm: algorithm,
ID: args[1],
Provider: args[2],
}
if err := cli.agentSDK.Algo(cmd.Context(), algoReq); err != nil {
log.Fatalf("Error uploading algorithm with ID %s and provider %s: %v", algoReq.ID, algoReq.Provider, err)
privKeyFile, err := os.ReadFile(args[1])
if err != nil {
log.Fatalf("Error reading private key file: %v", err)
}
pemBlock, _ := pem.Decode(privKeyFile)
privKey, err := x509.ParsePKCS1PrivateKey(pemBlock.Bytes)
if err != nil {
log.Fatalf("Error parsing private key: %v", err)
}
if err := cli.agentSDK.Algo(cmd.Context(), algoReq, privKey); err != nil {
log.Fatalf("Error uploading algorithm with error: %v", err)
}
log.Println("Successfully uploaded algorithm")
+18 -6
View File
@@ -3,6 +3,8 @@
package cli
import (
"crypto/x509"
"encoding/pem"
"log"
"os"
@@ -14,8 +16,8 @@ func (cli *CLI) NewDatasetsCmd() *cobra.Command {
return &cobra.Command{
Use: "data",
Short: "Upload a dataset CSV file",
Example: "data <dataset.csv> <id> <provider>",
Args: cobra.ExactArgs(3),
Example: "data <dataset.csv> <private_key_file_path>",
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
datasetFile := args[0]
@@ -27,12 +29,22 @@ func (cli *CLI) NewDatasetsCmd() *cobra.Command {
}
dataReq := agent.Dataset{
Dataset: dataset,
ID: args[1],
Provider: args[2],
Dataset: dataset,
}
if err := cli.agentSDK.Data(cmd.Context(), dataReq); err != nil {
privKeyFile, err := os.ReadFile(args[1])
if err != nil {
log.Fatalf("Error reading private key file: %v", err)
}
pemBlock, _ := pem.Decode(privKeyFile)
privKey, err := x509.ParsePKCS1PrivateKey(pemBlock.Bytes)
if err != nil {
log.Fatalf("Error parsing private key: %v", err)
}
if err := cli.agentSDK.Data(cmd.Context(), dataReq, privKey); err != nil {
log.Fatalf("Error uploading dataset: %v", err)
}
+68
View File
@@ -0,0 +1,68 @@
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"encoding/pem"
"log"
"os"
"github.com/spf13/cobra"
)
const (
keyBitSize = 4096
privateKeyType = "RSA PRIVATE KEY"
publicKeyType = "PUBLIC KEY"
publicKeyFile = "public.pem"
privateKeyFile = "private.pem"
)
func (cli *CLI) NewKeysCmd() *cobra.Command {
return &cobra.Command{
Use: "keys",
Short: "Generate a new public/private key pair",
Run: func(cmd *cobra.Command, args []string) {
privKey, err := rsa.GenerateKey(rand.Reader, keyBitSize)
if err != nil {
log.Fatalf("Error generating public key: %v", err)
}
pubKey, err := x509.MarshalPKIXPublicKey(&privKey.PublicKey)
if err != nil {
log.Fatalf("Error marshalling public key: %v", err)
}
privFile, err := os.Create(privateKeyFile)
if err != nil {
log.Fatalf("Error creating private key file: %v", err)
}
defer privFile.Close()
if err := pem.Encode(privFile, &pem.Block{
Type: privateKeyType,
Bytes: x509.MarshalPKCS1PrivateKey(privKey),
}); err != nil {
log.Fatalf("Error encoding private key: %v", err)
}
pubFile, err := os.Create(publicKeyFile)
if err != nil {
log.Fatalf("Error creating public key file: %v", err)
}
defer pubFile.Close()
if err := pem.Encode(pubFile, &pem.Block{
Type: publicKeyType,
Bytes: pubKey,
}); err != nil {
log.Fatalf("Error encoding public key: %v", err)
}
log.Println("Successfully generated public/private key pair")
},
}
}
+19 -4
View File
@@ -3,6 +3,8 @@
package cli
import (
"crypto/x509"
"encoding/pem"
"log"
"os"
@@ -13,13 +15,26 @@ const resultFilePath = "result.bin"
func (cli *CLI) NewResultsCmd() *cobra.Command {
return &cobra.Command{
Use: "result",
Short: "Retrieve computation result file",
Args: cobra.ExactArgs(1),
Use: "result",
Short: "Retrieve computation result file",
Example: "result <private_key_file_path>",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
log.Println("Retrieving computation result file")
result, err := cli.agentSDK.Result(cmd.Context(), args[0])
privKeyFile, err := os.ReadFile(args[0])
if err != nil {
log.Fatalf("Error reading private key file: %v", err)
}
pemBlock, _ := pem.Decode(privKeyFile)
privKey, err := x509.ParsePKCS1PrivateKey(pemBlock.Bytes)
if err != nil {
log.Fatalf("Error parsing private key: %v", err)
}
result, err := cli.agentSDK.Result(cmd.Context(), privKey)
if err != nil {
log.Fatalf("Error retrieving computation result: %v", err)
}
+3 -5
View File
@@ -2,15 +2,13 @@
// SPDX-License-Identifier: Apache-2.0
package cli
import (
"github.com/ultravioletrs/cocos/agent"
)
import "github.com/ultravioletrs/cocos/pkg/sdk"
type CLI struct {
agentSDK agent.Service
agentSDK sdk.SDK
}
func New(agentSDK agent.Service) *CLI {
func New(agentSDK sdk.SDK) *CLI {
return &CLI{
agentSDK: agentSDK,
}
+24 -7
View File
@@ -6,6 +6,7 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"log"
"log/slog"
@@ -13,6 +14,7 @@ import (
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/api"
agentgrpc "github.com/ultravioletrs/cocos/agent/api/grpc"
"github.com/ultravioletrs/cocos/agent/auth"
"github.com/ultravioletrs/cocos/agent/events"
"github.com/ultravioletrs/cocos/internal"
agentlogger "github.com/ultravioletrs/cocos/internal/logger"
@@ -53,7 +55,7 @@ func main() {
eventSvc, err := events.New(svcName, cfg.ID, manager.ManagerVsockPort)
if err != nil {
log.Printf("failed to create events service %s", err.Error())
logger.Error(fmt.Sprintf("failed to create events service %s", err.Error()))
return
}
defer eventSvc.Close()
@@ -73,7 +75,13 @@ func main() {
reflection.Register(srv)
agent.RegisterAgentServiceServer(srv, agentgrpc.NewServer(svc))
}
gs := grpcserver.New(ctx, cancel, svcName, grpcServerConfig, registerAgentServiceServer, logger, &svc)
authSvc, err := auth.New(cfg)
if err != nil {
logger.Error(fmt.Sprintf("failed to create auth service %s", err.Error()))
return
}
gs := grpcserver.New(ctx, cancel, svcName, grpcServerConfig, registerAgentServiceServer, logger, svc, authSvc)
g.Go(func() error {
return gs.Start()
@@ -109,15 +117,24 @@ func readConfig() (agent.Computation, error) {
return agent.Computation{}, err
}
defer conn.Close()
b := make([]byte, 1024)
n, err := conn.Read(b)
if err != nil {
return agent.Computation{}, err
var buffer []byte
for {
chunk := make([]byte, 1024)
n, err := conn.Read(chunk)
if err != nil {
if err == io.EOF {
break
}
return agent.Computation{}, err
}
buffer = append(buffer, chunk[:n]...)
}
ac := agent.Computation{
AgentConfig: agent.AgentConfig{},
}
if err := json.Unmarshal(b[:n], &ac); err != nil {
if err := json.Unmarshal(buffer, &ac); err != nil {
return agent.Computation{}, err
}
if ac.AgentConfig.LogLevel == "" {
+1
View File
@@ -90,6 +90,7 @@ func main() {
attestaionCmd := cliSVC.NewAttestationCmd()
rootCmd.AddCommand(attestaionCmd)
rootCmd.AddCommand(cliSVC.NewFileHashCmd())
rootCmd.AddCommand(cliSVC.NewKeysCmd())
// Attestation commands
attestaionCmd.AddCommand(cliSVC.NewGetAttestationCmd())
+1 -1
View File
@@ -1,6 +1,6 @@
module github.com/ultravioletrs/cocos
go 1.21.6
go 1.22.0
require (
github.com/absmach/magistrala v0.0.0-20240119191055-d95283d31472
+12 -4
View File
@@ -21,6 +21,7 @@ import (
"time"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/auth"
"github.com/ultravioletrs/cocos/internal/server"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"golang.org/x/crypto/sha3"
@@ -46,14 +47,15 @@ type Server struct {
server.BaseServer
server *grpc.Server
registerService serviceRegister
agent *agent.Service
agent agent.Service
authSvc *auth.Service
}
type serviceRegister func(srv *grpc.Server)
var _ server.Server = (*Server)(nil)
func New(ctx context.Context, cancel context.CancelFunc, name string, config server.Config, registerService serviceRegister, logger *slog.Logger, agentSvc *agent.Service) server.Server {
func New(ctx context.Context, cancel context.CancelFunc, name string, config server.Config, registerService serviceRegister, logger *slog.Logger, agentSvc agent.Service, authSvc *auth.Service) server.Server {
listenFullAddress := fmt.Sprintf("%s:%s", config.Host, config.Port)
return &Server{
BaseServer: server.BaseServer{
@@ -66,6 +68,7 @@ func New(ctx context.Context, cancel context.CancelFunc, name string, config ser
},
registerService: registerService,
agent: agentSvc,
authSvc: authSvc,
}
}
@@ -75,6 +78,11 @@ func (s *Server) Start() error {
grpc.StatsHandler(otelgrpc.NewServerHandler()),
}
if s.authSvc != nil {
grpcServerOptions = append(grpcServerOptions, grpc.UnaryInterceptor(s.authSvc.AuthUnaryInterceptor()))
grpcServerOptions = append(grpcServerOptions, grpc.StreamInterceptor(s.authSvc.AuthStreamInterceptor()))
}
listener, err := net.Listen("tcp", s.Address)
if err != nil {
return fmt.Errorf("failed to listen on port %s: %w", s.Address, err)
@@ -218,7 +226,7 @@ func loadX509KeyPair(certfile, keyfile string) (tls.Certificate, error) {
return tls.X509KeyPair(cert, key)
}
func generateCertificatesForATLS(svc *agent.Service) ([]byte, []byte, error) {
func generateCertificatesForATLS(svc agent.Service) ([]byte, []byte, error) {
curve := elliptic.P256()
privateKey, err := ecdsa.GenerateKey(curve, rand.Reader)
if err != nil {
@@ -231,7 +239,7 @@ func generateCertificatesForATLS(svc *agent.Service) ([]byte, []byte, error) {
}
// The Attestation Report will be added as an X.509 certificate extension
attestationReport, err := (*svc).Attestation(context.Background(), sha3.Sum512(publicKeyBytes))
attestationReport, err := svc.Attestation(context.Background(), sha3.Sum512(publicKeyBytes))
if err != nil {
return nil, nil, fmt.Errorf("failed to fetch the attestation report: %w", err)
}
+9 -7
View File
@@ -59,20 +59,22 @@ message ComputationRunReq {
string description = 3;
repeated Dataset datasets = 4;
Algorithm algorithm = 5;
repeated string result_consumers = 6;
repeated ResultConsumer result_consumers = 6;
AgentConfig agent_config = 7;
}
message ResultConsumer {
bytes userKey = 1;
}
message Dataset {
string provider = 1;
string id = 2;
bytes hash = 3; // should be sha3.Sum256, 32 byte length.
bytes hash = 1; // should be sha3.Sum256, 32 byte length.
bytes userKey = 2;
}
message Algorithm {
string provider = 1;
string id = 2;
bytes hash = 3; // should be sha3.Sum256, 32 byte length.
bytes hash = 1; // should be sha3.Sum256, 32 byte length.
bytes userKey = 2;
}
message AgentConfig {
+9 -6
View File
@@ -70,10 +70,9 @@ func New(qemuCfg qemu.Config, logger *slog.Logger, eventsChan chan *manager.Clie
func (ms *managerService) Run(ctx context.Context, c *manager.ComputationRunReq) (string, error) {
ms.publishEvent("vm-provision", c.Id, "starting", json.RawMessage{})
ac := agent.Computation{
ID: c.Id,
Name: c.Name,
Description: c.Description,
ResultConsumers: c.ResultConsumers,
ID: c.Id,
Name: c.Name,
Description: c.Description,
AgentConfig: agent.AgentConfig{
Port: c.AgentConfig.Port,
Host: c.AgentConfig.Host,
@@ -84,14 +83,18 @@ func (ms *managerService) Run(ctx context.Context, c *manager.ComputationRunReq)
LogLevel: c.AgentConfig.LogLevel,
},
}
ac.Algorithm = agent.Algorithm{ID: c.Algorithm.Id, Provider: c.Algorithm.Provider, Hash: [hashLength]byte(c.Algorithm.Hash)}
ac.Algorithm = agent.Algorithm{Hash: [hashLength]byte(c.Algorithm.Hash), UserKey: c.Algorithm.UserKey}
for _, data := range c.Datasets {
if len(data.Hash) != hashLength {
ms.publishEvent("vm-provision", c.Id, "failed", json.RawMessage{})
return "", errInvalidHashLength
}
ac.Datasets = append(ac.Datasets, agent.Dataset{ID: data.Id, Provider: data.Provider, Hash: [hashLength]byte(data.Hash)})
ac.Datasets = append(ac.Datasets, agent.Dataset{Hash: [hashLength]byte(data.Hash), UserKey: data.UserKey})
}
for _, rc := range c.ResultConsumers {
ac.ResultConsumers = append(ac.ResultConsumers, agent.ResultConsumer{UserKey: rc.UserKey})
}
agentPort, err := getFreePort()
+158 -111
View File
@@ -465,13 +465,13 @@ type ComputationRunReq struct {
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
Description string `protobuf:"bytes,3,opt,name=description,proto3" json:"description,omitempty"`
Datasets []*Dataset `protobuf:"bytes,4,rep,name=datasets,proto3" json:"datasets,omitempty"`
Algorithm *Algorithm `protobuf:"bytes,5,opt,name=algorithm,proto3" json:"algorithm,omitempty"`
ResultConsumers []string `protobuf:"bytes,6,rep,name=result_consumers,json=resultConsumers,proto3" json:"result_consumers,omitempty"`
AgentConfig *AgentConfig `protobuf:"bytes,7,opt,name=agent_config,json=agentConfig,proto3" json:"agent_config,omitempty"`
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
Description string `protobuf:"bytes,3,opt,name=description,proto3" json:"description,omitempty"`
Datasets []*Dataset `protobuf:"bytes,4,rep,name=datasets,proto3" json:"datasets,omitempty"`
Algorithm *Algorithm `protobuf:"bytes,5,opt,name=algorithm,proto3" json:"algorithm,omitempty"`
ResultConsumers []*ResultConsumer `protobuf:"bytes,6,rep,name=result_consumers,json=resultConsumers,proto3" json:"result_consumers,omitempty"`
AgentConfig *AgentConfig `protobuf:"bytes,7,opt,name=agent_config,json=agentConfig,proto3" json:"agent_config,omitempty"`
}
func (x *ComputationRunReq) Reset() {
@@ -541,7 +541,7 @@ func (x *ComputationRunReq) GetAlgorithm() *Algorithm {
return nil
}
func (x *ComputationRunReq) GetResultConsumers() []string {
func (x *ComputationRunReq) GetResultConsumers() []*ResultConsumer {
if x != nil {
return x.ResultConsumers
}
@@ -555,20 +555,66 @@ func (x *ComputationRunReq) GetAgentConfig() *AgentConfig {
return nil
}
type ResultConsumer struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
UserKey []byte `protobuf:"bytes,1,opt,name=userKey,proto3" json:"userKey,omitempty"`
}
func (x *ResultConsumer) Reset() {
*x = ResultConsumer{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[7]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *ResultConsumer) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*ResultConsumer) ProtoMessage() {}
func (x *ResultConsumer) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[7]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use ResultConsumer.ProtoReflect.Descriptor instead.
func (*ResultConsumer) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{7}
}
func (x *ResultConsumer) GetUserKey() []byte {
if x != nil {
return x.UserKey
}
return nil
}
type Dataset struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Provider string `protobuf:"bytes,1,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,2,opt,name=id,proto3" json:"id,omitempty"`
Hash []byte `protobuf:"bytes,3,opt,name=hash,proto3" json:"hash,omitempty"` // should be sha3.Sum256, 32 byte length.
Hash []byte `protobuf:"bytes,1,opt,name=hash,proto3" json:"hash,omitempty"` // should be sha3.Sum256, 32 byte length.
UserKey []byte `protobuf:"bytes,2,opt,name=userKey,proto3" json:"userKey,omitempty"`
}
func (x *Dataset) Reset() {
*x = Dataset{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[7]
mi := &file_manager_manager_proto_msgTypes[8]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -581,7 +627,7 @@ func (x *Dataset) String() string {
func (*Dataset) ProtoMessage() {}
func (x *Dataset) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[7]
mi := &file_manager_manager_proto_msgTypes[8]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -594,21 +640,7 @@ func (x *Dataset) ProtoReflect() protoreflect.Message {
// Deprecated: Use Dataset.ProtoReflect.Descriptor instead.
func (*Dataset) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{7}
}
func (x *Dataset) GetProvider() string {
if x != nil {
return x.Provider
}
return ""
}
func (x *Dataset) GetId() string {
if x != nil {
return x.Id
}
return ""
return file_manager_manager_proto_rawDescGZIP(), []int{8}
}
func (x *Dataset) GetHash() []byte {
@@ -618,20 +650,26 @@ func (x *Dataset) GetHash() []byte {
return nil
}
func (x *Dataset) GetUserKey() []byte {
if x != nil {
return x.UserKey
}
return nil
}
type Algorithm struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Provider string `protobuf:"bytes,1,opt,name=provider,proto3" json:"provider,omitempty"`
Id string `protobuf:"bytes,2,opt,name=id,proto3" json:"id,omitempty"`
Hash []byte `protobuf:"bytes,3,opt,name=hash,proto3" json:"hash,omitempty"` // should be sha3.Sum256, 32 byte length.
Hash []byte `protobuf:"bytes,1,opt,name=hash,proto3" json:"hash,omitempty"` // should be sha3.Sum256, 32 byte length.
UserKey []byte `protobuf:"bytes,2,opt,name=userKey,proto3" json:"userKey,omitempty"`
}
func (x *Algorithm) Reset() {
*x = Algorithm{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[8]
mi := &file_manager_manager_proto_msgTypes[9]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -644,7 +682,7 @@ func (x *Algorithm) String() string {
func (*Algorithm) ProtoMessage() {}
func (x *Algorithm) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[8]
mi := &file_manager_manager_proto_msgTypes[9]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -657,21 +695,7 @@ func (x *Algorithm) ProtoReflect() protoreflect.Message {
// Deprecated: Use Algorithm.ProtoReflect.Descriptor instead.
func (*Algorithm) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{8}
}
func (x *Algorithm) GetProvider() string {
if x != nil {
return x.Provider
}
return ""
}
func (x *Algorithm) GetId() string {
if x != nil {
return x.Id
}
return ""
return file_manager_manager_proto_rawDescGZIP(), []int{9}
}
func (x *Algorithm) GetHash() []byte {
@@ -681,6 +705,13 @@ func (x *Algorithm) GetHash() []byte {
return nil
}
func (x *Algorithm) GetUserKey() []byte {
if x != nil {
return x.UserKey
}
return nil
}
type AgentConfig struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
@@ -699,7 +730,7 @@ type AgentConfig struct {
func (x *AgentConfig) Reset() {
*x = AgentConfig{}
if protoimpl.UnsafeEnabled {
mi := &file_manager_manager_proto_msgTypes[9]
mi := &file_manager_manager_proto_msgTypes[10]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
@@ -712,7 +743,7 @@ func (x *AgentConfig) String() string {
func (*AgentConfig) ProtoMessage() {}
func (x *AgentConfig) ProtoReflect() protoreflect.Message {
mi := &file_manager_manager_proto_msgTypes[9]
mi := &file_manager_manager_proto_msgTypes[10]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
@@ -725,7 +756,7 @@ func (x *AgentConfig) ProtoReflect() protoreflect.Message {
// Deprecated: Use AgentConfig.ProtoReflect.Descriptor instead.
func (*AgentConfig) Descriptor() ([]byte, []int) {
return file_manager_manager_proto_rawDescGZIP(), []int{9}
return file_manager_manager_proto_rawDescGZIP(), []int{10}
}
func (x *AgentConfig) GetPort() string {
@@ -843,7 +874,7 @@ var file_manager_manager_proto_rawDesc = []byte{
0x69, 0x6e, 0x61, 0x74, 0x65, 0x52, 0x65, 0x71, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x12,
0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x54, 0x65, 0x72, 0x6d, 0x69, 0x6e, 0x61,
0x74, 0x65, 0x48, 0x00, 0x52, 0x0c, 0x74, 0x65, 0x72, 0x6d, 0x69, 0x6e, 0x61, 0x74, 0x65, 0x52,
0x65, 0x71, 0x42, 0x09, 0x0a, 0x07, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x22, 0x9d, 0x02,
0x65, 0x71, 0x42, 0x09, 0x0a, 0x07, 0x6d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x22, 0xb6, 0x02,
0x0a, 0x11, 0x43, 0x6f, 0x6d, 0x70, 0x75, 0x74, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x75, 0x6e,
0x52, 0x65, 0x71, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,
0x02, 0x69, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28,
@@ -855,46 +886,48 @@ var file_manager_manager_proto_rawDesc = []byte{
0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x73, 0x12, 0x30, 0x0a, 0x09, 0x61, 0x6c, 0x67, 0x6f, 0x72,
0x69, 0x74, 0x68, 0x6d, 0x18, 0x05, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x6d, 0x61, 0x6e,
0x61, 0x67, 0x65, 0x72, 0x2e, 0x41, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x52, 0x09,
0x61, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x29, 0x0a, 0x10, 0x72, 0x65, 0x73,
0x61, 0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x42, 0x0a, 0x10, 0x72, 0x65, 0x73,
0x75, 0x6c, 0x74, 0x5f, 0x63, 0x6f, 0x6e, 0x73, 0x75, 0x6d, 0x65, 0x72, 0x73, 0x18, 0x06, 0x20,
0x03, 0x28, 0x09, 0x52, 0x0f, 0x72, 0x65, 0x73, 0x75, 0x6c, 0x74, 0x43, 0x6f, 0x6e, 0x73, 0x75,
0x6d, 0x65, 0x72, 0x73, 0x12, 0x37, 0x0a, 0x0c, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f, 0x63, 0x6f,
0x6e, 0x66, 0x69, 0x67, 0x18, 0x07, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x14, 0x2e, 0x6d, 0x61, 0x6e,
0x61, 0x67, 0x65, 0x72, 0x2e, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67,
0x52, 0x0b, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x22, 0x49, 0x0a,
0x07, 0x44, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x12, 0x1a, 0x0a, 0x08, 0x70, 0x72, 0x6f, 0x76,
0x69, 0x64, 0x65, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x70, 0x72, 0x6f, 0x76,
0x69, 0x64, 0x65, 0x72, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09,
0x52, 0x02, 0x69, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x68, 0x61, 0x73, 0x68, 0x18, 0x03, 0x20, 0x01,
0x28, 0x0c, 0x52, 0x04, 0x68, 0x61, 0x73, 0x68, 0x22, 0x4b, 0x0a, 0x09, 0x41, 0x6c, 0x67, 0x6f,
0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x1a, 0x0a, 0x08, 0x70, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65,
0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x70, 0x72, 0x6f, 0x76, 0x69, 0x64, 0x65,
0x72, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69,
0x64, 0x12, 0x12, 0x0a, 0x04, 0x68, 0x61, 0x73, 0x68, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52,
0x04, 0x68, 0x61, 0x73, 0x68, 0x22, 0xf9, 0x01, 0x0a, 0x0b, 0x41, 0x67, 0x65, 0x6e, 0x74, 0x43,
0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x12, 0x0a, 0x04, 0x70, 0x6f, 0x72, 0x74, 0x18, 0x01, 0x20,
0x01, 0x28, 0x09, 0x52, 0x04, 0x70, 0x6f, 0x72, 0x74, 0x12, 0x12, 0x0a, 0x04, 0x68, 0x6f, 0x73,
0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x68, 0x6f, 0x73, 0x74, 0x12, 0x1b, 0x0a,
0x09, 0x63, 0x65, 0x72, 0x74, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x09,
0x52, 0x08, 0x63, 0x65, 0x72, 0x74, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x19, 0x0a, 0x08, 0x6b, 0x65,
0x79, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x6b, 0x65,
0x79, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x24, 0x0a, 0x0e, 0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x5f,
0x63, 0x61, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0c, 0x63,
0x6c, 0x69, 0x65, 0x6e, 0x74, 0x43, 0x61, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x24, 0x0a, 0x0e, 0x73,
0x65, 0x72, 0x76, 0x65, 0x72, 0x5f, 0x63, 0x61, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x06, 0x20,
0x01, 0x28, 0x09, 0x52, 0x0c, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x43, 0x61, 0x46, 0x69, 0x6c,
0x65, 0x12, 0x1b, 0x0a, 0x09, 0x6c, 0x6f, 0x67, 0x5f, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x18, 0x07,
0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x6c, 0x6f, 0x67, 0x4c, 0x65, 0x76, 0x65, 0x6c, 0x12, 0x21,
0x0a, 0x0c, 0x61, 0x74, 0x74, 0x65, 0x73, 0x74, 0x65, 0x64, 0x5f, 0x74, 0x6c, 0x73, 0x18, 0x08,
0x20, 0x01, 0x28, 0x08, 0x52, 0x0b, 0x61, 0x74, 0x74, 0x65, 0x73, 0x74, 0x65, 0x64, 0x54, 0x6c,
0x73, 0x32, 0x5d, 0x0a, 0x0e, 0x4d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x53, 0x65, 0x72, 0x76,
0x69, 0x63, 0x65, 0x12, 0x4b, 0x0a, 0x07, 0x50, 0x72, 0x6f, 0x63, 0x65, 0x73, 0x73, 0x12, 0x1c,
0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x43, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x53,
0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x1a, 0x1c, 0x2e, 0x6d,
0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x53, 0x74, 0x72,
0x65, 0x61, 0x6d, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x22, 0x00, 0x28, 0x01, 0x30, 0x01,
0x42, 0x0b, 0x5a, 0x09, 0x2e, 0x2f, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x62, 0x06, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x33,
0x03, 0x28, 0x0b, 0x32, 0x17, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x52, 0x65,
0x73, 0x75, 0x6c, 0x74, 0x43, 0x6f, 0x6e, 0x73, 0x75, 0x6d, 0x65, 0x72, 0x52, 0x0f, 0x72, 0x65,
0x73, 0x75, 0x6c, 0x74, 0x43, 0x6f, 0x6e, 0x73, 0x75, 0x6d, 0x65, 0x72, 0x73, 0x12, 0x37, 0x0a,
0x0c, 0x61, 0x67, 0x65, 0x6e, 0x74, 0x5f, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x18, 0x07, 0x20,
0x01, 0x28, 0x0b, 0x32, 0x14, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x41, 0x67,
0x65, 0x6e, 0x74, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x52, 0x0b, 0x61, 0x67, 0x65, 0x6e, 0x74,
0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x22, 0x2a, 0x0a, 0x0e, 0x52, 0x65, 0x73, 0x75, 0x6c, 0x74,
0x43, 0x6f, 0x6e, 0x73, 0x75, 0x6d, 0x65, 0x72, 0x12, 0x18, 0x0a, 0x07, 0x75, 0x73, 0x65, 0x72,
0x4b, 0x65, 0x79, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x75, 0x73, 0x65, 0x72, 0x4b,
0x65, 0x79, 0x22, 0x37, 0x0a, 0x07, 0x44, 0x61, 0x74, 0x61, 0x73, 0x65, 0x74, 0x12, 0x12, 0x0a,
0x04, 0x68, 0x61, 0x73, 0x68, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x68, 0x61, 0x73,
0x68, 0x12, 0x18, 0x0a, 0x07, 0x75, 0x73, 0x65, 0x72, 0x4b, 0x65, 0x79, 0x18, 0x02, 0x20, 0x01,
0x28, 0x0c, 0x52, 0x07, 0x75, 0x73, 0x65, 0x72, 0x4b, 0x65, 0x79, 0x22, 0x39, 0x0a, 0x09, 0x41,
0x6c, 0x67, 0x6f, 0x72, 0x69, 0x74, 0x68, 0x6d, 0x12, 0x12, 0x0a, 0x04, 0x68, 0x61, 0x73, 0x68,
0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x04, 0x68, 0x61, 0x73, 0x68, 0x12, 0x18, 0x0a, 0x07,
0x75, 0x73, 0x65, 0x72, 0x4b, 0x65, 0x79, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, 0x75,
0x73, 0x65, 0x72, 0x4b, 0x65, 0x79, 0x22, 0xf9, 0x01, 0x0a, 0x0b, 0x41, 0x67, 0x65, 0x6e, 0x74,
0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x12, 0x0a, 0x04, 0x70, 0x6f, 0x72, 0x74, 0x18, 0x01,
0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x70, 0x6f, 0x72, 0x74, 0x12, 0x12, 0x0a, 0x04, 0x68, 0x6f,
0x73, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x68, 0x6f, 0x73, 0x74, 0x12, 0x1b,
0x0a, 0x09, 0x63, 0x65, 0x72, 0x74, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x03, 0x20, 0x01, 0x28,
0x09, 0x52, 0x08, 0x63, 0x65, 0x72, 0x74, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x19, 0x0a, 0x08, 0x6b,
0x65, 0x79, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x6b,
0x65, 0x79, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x24, 0x0a, 0x0e, 0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74,
0x5f, 0x63, 0x61, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x05, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0c,
0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x43, 0x61, 0x46, 0x69, 0x6c, 0x65, 0x12, 0x24, 0x0a, 0x0e,
0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x5f, 0x63, 0x61, 0x5f, 0x66, 0x69, 0x6c, 0x65, 0x18, 0x06,
0x20, 0x01, 0x28, 0x09, 0x52, 0x0c, 0x73, 0x65, 0x72, 0x76, 0x65, 0x72, 0x43, 0x61, 0x46, 0x69,
0x6c, 0x65, 0x12, 0x1b, 0x0a, 0x09, 0x6c, 0x6f, 0x67, 0x5f, 0x6c, 0x65, 0x76, 0x65, 0x6c, 0x18,
0x07, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x6c, 0x6f, 0x67, 0x4c, 0x65, 0x76, 0x65, 0x6c, 0x12,
0x21, 0x0a, 0x0c, 0x61, 0x74, 0x74, 0x65, 0x73, 0x74, 0x65, 0x64, 0x5f, 0x74, 0x6c, 0x73, 0x18,
0x08, 0x20, 0x01, 0x28, 0x08, 0x52, 0x0b, 0x61, 0x74, 0x74, 0x65, 0x73, 0x74, 0x65, 0x64, 0x54,
0x6c, 0x73, 0x32, 0x5d, 0x0a, 0x0e, 0x4d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x53, 0x65, 0x72,
0x76, 0x69, 0x63, 0x65, 0x12, 0x4b, 0x0a, 0x07, 0x50, 0x72, 0x6f, 0x63, 0x65, 0x73, 0x73, 0x12,
0x1c, 0x2e, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x43, 0x6c, 0x69, 0x65, 0x6e, 0x74,
0x53, 0x74, 0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x1a, 0x1c, 0x2e,
0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x2e, 0x53, 0x65, 0x72, 0x76, 0x65, 0x72, 0x53, 0x74,
0x72, 0x65, 0x61, 0x6d, 0x4d, 0x65, 0x73, 0x73, 0x61, 0x67, 0x65, 0x22, 0x00, 0x28, 0x01, 0x30,
0x01, 0x42, 0x0b, 0x5a, 0x09, 0x2e, 0x2f, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x72, 0x62, 0x06,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (
@@ -909,7 +942,7 @@ func file_manager_manager_proto_rawDescGZIP() []byte {
return file_manager_manager_proto_rawDescData
}
var file_manager_manager_proto_msgTypes = make([]protoimpl.MessageInfo, 10)
var file_manager_manager_proto_msgTypes = make([]protoimpl.MessageInfo, 11)
var file_manager_manager_proto_goTypes = []interface{}{
(*Terminate)(nil), // 0: manager.Terminate
(*RunResponse)(nil), // 1: manager.RunResponse
@@ -918,29 +951,31 @@ var file_manager_manager_proto_goTypes = []interface{}{
(*ClientStreamMessage)(nil), // 4: manager.ClientStreamMessage
(*ServerStreamMessage)(nil), // 5: manager.ServerStreamMessage
(*ComputationRunReq)(nil), // 6: manager.ComputationRunReq
(*Dataset)(nil), // 7: manager.Dataset
(*Algorithm)(nil), // 8: manager.Algorithm
(*AgentConfig)(nil), // 9: manager.AgentConfig
(*timestamppb.Timestamp)(nil), // 10: google.protobuf.Timestamp
(*ResultConsumer)(nil), // 7: manager.ResultConsumer
(*Dataset)(nil), // 8: manager.Dataset
(*Algorithm)(nil), // 9: manager.Algorithm
(*AgentConfig)(nil), // 10: manager.AgentConfig
(*timestamppb.Timestamp)(nil), // 11: google.protobuf.Timestamp
}
var file_manager_manager_proto_depIdxs = []int32{
10, // 0: manager.AgentEvent.timestamp:type_name -> google.protobuf.Timestamp
10, // 1: manager.AgentLog.timestamp:type_name -> google.protobuf.Timestamp
11, // 0: manager.AgentEvent.timestamp:type_name -> google.protobuf.Timestamp
11, // 1: manager.AgentLog.timestamp:type_name -> google.protobuf.Timestamp
3, // 2: manager.ClientStreamMessage.agent_log:type_name -> manager.AgentLog
2, // 3: manager.ClientStreamMessage.agent_event:type_name -> manager.AgentEvent
1, // 4: manager.ClientStreamMessage.run_res:type_name -> manager.RunResponse
6, // 5: manager.ServerStreamMessage.runReq:type_name -> manager.ComputationRunReq
0, // 6: manager.ServerStreamMessage.terminateReq:type_name -> manager.Terminate
7, // 7: manager.ComputationRunReq.datasets:type_name -> manager.Dataset
8, // 8: manager.ComputationRunReq.algorithm:type_name -> manager.Algorithm
9, // 9: manager.ComputationRunReq.agent_config:type_name -> manager.AgentConfig
4, // 10: manager.ManagerService.Process:input_type -> manager.ClientStreamMessage
5, // 11: manager.ManagerService.Process:output_type -> manager.ServerStreamMessage
11, // [11:12] is the sub-list for method output_type
10, // [10:11] is the sub-list for method input_type
10, // [10:10] is the sub-list for extension type_name
10, // [10:10] is the sub-list for extension extendee
0, // [0:10] is the sub-list for field type_name
8, // 7: manager.ComputationRunReq.datasets:type_name -> manager.Dataset
9, // 8: manager.ComputationRunReq.algorithm:type_name -> manager.Algorithm
7, // 9: manager.ComputationRunReq.result_consumers:type_name -> manager.ResultConsumer
10, // 10: manager.ComputationRunReq.agent_config:type_name -> manager.AgentConfig
4, // 11: manager.ManagerService.Process:input_type -> manager.ClientStreamMessage
5, // 12: manager.ManagerService.Process:output_type -> manager.ServerStreamMessage
12, // [12:13] is the sub-list for method output_type
11, // [11:12] is the sub-list for method input_type
11, // [11:11] is the sub-list for extension type_name
11, // [11:11] is the sub-list for extension extendee
0, // [0:11] is the sub-list for field type_name
}
func init() { file_manager_manager_proto_init() }
@@ -1034,7 +1069,7 @@ func file_manager_manager_proto_init() {
}
}
file_manager_manager_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Dataset); i {
switch v := v.(*ResultConsumer); i {
case 0:
return &v.state
case 1:
@@ -1046,7 +1081,7 @@ func file_manager_manager_proto_init() {
}
}
file_manager_manager_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Algorithm); i {
switch v := v.(*Dataset); i {
case 0:
return &v.state
case 1:
@@ -1058,6 +1093,18 @@ func file_manager_manager_proto_init() {
}
}
file_manager_manager_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Algorithm); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_manager_manager_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*AgentConfig); i {
case 0:
return &v.state
@@ -1085,7 +1132,7 @@ func file_manager_manager_proto_init() {
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_manager_manager_proto_rawDesc,
NumEnums: 0,
NumMessages: 10,
NumMessages: 11,
NumExtensions: 0,
NumServices: 1,
},
+62 -9
View File
@@ -5,13 +5,25 @@ package sdk
import (
"bytes"
"context"
"crypto"
"crypto/rand"
"crypto/rsa"
"crypto/sha256"
"encoding/base64"
"io"
"log/slog"
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/agent/auth"
"google.golang.org/grpc/metadata"
)
var _ agent.Service = (*agentSDK)(nil)
type SDK interface {
Algo(ctx context.Context, algorithm agent.Algorithm, privKey *rsa.PrivateKey) error
Data(ctx context.Context, dataset agent.Dataset, privKey *rsa.PrivateKey) error
Result(ctx context.Context, privKey *rsa.PrivateKey) ([]byte, error)
Attestation(ctx context.Context, reportData [size64]byte) ([]byte, error)
}
const (
size64 = 64
@@ -23,14 +35,21 @@ type agentSDK struct {
logger *slog.Logger
}
func NewAgentSDK(log *slog.Logger, agentClient agent.AgentServiceClient) *agentSDK {
func NewAgentSDK(log *slog.Logger, agentClient agent.AgentServiceClient) SDK {
return &agentSDK{
client: agentClient,
logger: log,
}
}
func (sdk *agentSDK) Algo(ctx context.Context, algorithm agent.Algorithm) error {
func (sdk *agentSDK) Algo(ctx context.Context, algorithm agent.Algorithm, privKey *rsa.PrivateKey) error {
md, err := generateMetadata(string(auth.AlgorithmProviderRole), privKey)
if err != nil {
sdk.logger.Error("Failed to generate metadata")
return err
}
ctx = metadata.NewOutgoingContext(ctx, md)
stream, err := sdk.client.Algo(ctx)
if err != nil {
sdk.logger.Error("Failed to call Algo RPC")
@@ -48,7 +67,7 @@ func (sdk *agentSDK) Algo(ctx context.Context, algorithm agent.Algorithm) error
return err
}
err = stream.Send(&agent.AlgoRequest{Id: algorithm.ID, Provider: algorithm.Provider, Algorithm: buf[:n]})
err = stream.Send(&agent.AlgoRequest{Algorithm: buf[:n]})
if err != nil {
return err
}
@@ -61,7 +80,14 @@ func (sdk *agentSDK) Algo(ctx context.Context, algorithm agent.Algorithm) error
return nil
}
func (sdk *agentSDK) Data(ctx context.Context, dataset agent.Dataset) error {
func (sdk *agentSDK) Data(ctx context.Context, dataset agent.Dataset, privKey *rsa.PrivateKey) error {
md, err := generateMetadata(string(auth.DataProviderRole), privKey)
if err != nil {
sdk.logger.Error("Failed to generate metadata")
return err
}
ctx = metadata.NewOutgoingContext(ctx, md)
stream, err := sdk.client.Data(ctx)
if err != nil {
sdk.logger.Error("Failed to call Algo RPC")
@@ -79,7 +105,7 @@ func (sdk *agentSDK) Data(ctx context.Context, dataset agent.Dataset) error {
return err
}
err = stream.Send(&agent.DataRequest{Id: dataset.ID, Provider: dataset.Provider, Dataset: buf[:n]})
err = stream.Send(&agent.DataRequest{Dataset: buf[:n]})
if err != nil {
return err
}
@@ -92,11 +118,16 @@ func (sdk *agentSDK) Data(ctx context.Context, dataset agent.Dataset) error {
return nil
}
func (sdk *agentSDK) Result(ctx context.Context, consumer string) ([]byte, error) {
request := &agent.ResultRequest{
Consumer: consumer,
func (sdk *agentSDK) Result(ctx context.Context, privKey *rsa.PrivateKey) ([]byte, error) {
request := &agent.ResultRequest{}
md, err := generateMetadata(string(auth.ConsumerRole), privKey)
if err != nil {
sdk.logger.Error("Failed to generate metadata")
return nil, err
}
ctx = metadata.NewOutgoingContext(ctx, md)
response, err := sdk.client.Result(ctx, request)
if err != nil {
sdk.logger.Error("Failed to call Result RPC")
@@ -119,3 +150,25 @@ func (sdk *agentSDK) Attestation(ctx context.Context, reportData [size64]byte) (
return response.File, nil
}
func signData(userID string, privKey *rsa.PrivateKey) ([]byte, error) {
hash := sha256.Sum256([]byte(userID))
signature, err := rsa.SignPKCS1v15(rand.Reader, privKey, crypto.SHA256, hash[:])
if err != nil {
return nil, err
}
return signature, nil
}
func generateMetadata(userID string, privateKey *rsa.PrivateKey) (metadata.MD, error) {
signature, err := signData(userID, privateKey)
if err != nil {
return nil, err
}
kv := make(map[string]string)
kv[auth.UserMetadataKey] = userID
kv[auth.SignatureMetadataKey] = base64.StdEncoding.EncodeToString(signature)
return metadata.New(kv), nil
}
+16 -7
View File
@@ -4,6 +4,7 @@ package main
import (
"context"
"encoding/pem"
"fmt"
"log"
"log/slog"
@@ -34,6 +35,7 @@ var (
algoPath = "./test/manual/algo/lin_reg.py"
dataPath = "./test/manual/data/iris.csv"
attestedTLS = false
pubKeyFile string
)
type svc struct {
@@ -52,6 +54,12 @@ func (s *svc) Run(ipAdress string, reqChan chan *manager.ServerStreamMessage, au
s.logger.Error(fmt.Sprintf("failed to read data file: %s", err))
return
}
pubKey, err := os.ReadFile(pubKeyFile)
if err != nil {
s.logger.Error(fmt.Sprintf("failed to read public key file: %s", err))
return
}
pubPem, _ := pem.Decode(pubKey)
algoHash := sha3.Sum256(algo)
dataHash := sha3.Sum256(data)
reqChan <- &manager.ServerStreamMessage{
@@ -60,9 +68,9 @@ func (s *svc) Run(ipAdress string, reqChan chan *manager.ServerStreamMessage, au
Id: "1",
Name: "sample computation",
Description: "sample descrption",
Datasets: []*manager.Dataset{{Id: "1", Provider: "provider1", Hash: dataHash[:]}},
Algorithm: &manager.Algorithm{Id: "1", Provider: "provider1", Hash: algoHash[:]},
ResultConsumers: []string{"consumer1"},
Datasets: []*manager.Dataset{{Hash: dataHash[:], UserKey: pubPem.Bytes}},
Algorithm: &manager.Algorithm{Hash: algoHash[:], UserKey: pubPem.Bytes},
ResultConsumers: []*manager.ResultConsumer{{UserKey: pubPem.Bytes}},
AgentConfig: &manager.AgentConfig{
Port: "7002",
LogLevel: "debug",
@@ -74,12 +82,13 @@ func (s *svc) Run(ipAdress string, reqChan chan *manager.ServerStreamMessage, au
}
func main() {
if len(os.Args) < 4 {
log.Fatalf("usage: %s <data-path> <algo-path> <attested-tls-bool>", os.Args[0])
if len(os.Args) < 5 {
log.Fatalf("usage: %s <data-path> <algo-path> <public-key-path> <attested-tls-bool>", os.Args[0])
}
dataPath = os.Args[1]
algoPath = os.Args[2]
attestedTLSParam, err := strconv.ParseBool(os.Args[3])
pubKeyFile = os.Args[3]
attestedTLSParam, err := strconv.ParseBool(os.Args[4])
if err != nil {
log.Fatalf("usage: %s <data-path> <algo-path> <attested-tls-bool>, <attested-tls-bool> must be a bool value", os.Args[0])
}
@@ -118,7 +127,7 @@ func main() {
return
}
gs := grpcserver.New(ctx, cancel, svcName, grpcServerConfig, registerAgentServiceServer, logger, nil)
gs := grpcserver.New(ctx, cancel, svcName, grpcServerConfig, registerAgentServiceServer, logger, nil, nil)
g.Go(func() error {
return gs.Start()
+4 -4
View File
@@ -33,16 +33,16 @@ go run cmd/cli/main.go attestation get '<report_data>'
go run cmd/cli/main.go attestation validate '<attesation>' --report_data '<report_data>'
# Run the CLI program with algorithm input
go run cmd/cli/main.go algo test/manual/algo/lin_reg.bin Algorithm1 AlgorithmProvider1
go run cmd/cli/main.go algo test/manual/algo/lin_reg.bin <private_key_file_path>
# 2023/09/21 10:43:53 Uploading algorithm binary: test/manual/algo/lin_reg.bin
# Run the CLI program with dataset input
go run cmd/cli/main.go data test/manual/data/iris.csv Dataset1 Provider1
go run cmd/cli/main.go data test/manual/data/iris.csv Dataset2 Provider2
go run cmd/cli/main.go data test/manual/data/iris.csv <private_key_file_path>
go run cmd/cli/main.go data test/manual/data/iris.csv <private_key_file_path>
# 2023/09/21 10:45:25 Uploading dataset CSV: test/manual/data/iris.csv
# Run the CLI program to fetch computation result
go run cmd/cli/main.go result Consumer1
go run cmd/cli/main.go result <private_key_file_path>
# 2023/09/21 10:45:39 Retrieving computation result file
# 2023/09/21 10:45:40 Computation result retrieved and saved successfully!
```
+36 -14
View File
@@ -7,6 +7,7 @@ package main
import (
"encoding/json"
"encoding/pem"
"fmt"
"log"
"os"
@@ -16,22 +17,38 @@ import (
"github.com/ultravioletrs/cocos/agent"
"github.com/ultravioletrs/cocos/manager"
pkgmanager "github.com/ultravioletrs/cocos/pkg/manager"
"golang.org/x/crypto/sha3"
"google.golang.org/protobuf/proto"
)
func main() {
attestedTLS := false
if len(os.Args) == 2 {
attestedTLSParam, err := strconv.ParseBool(os.Args[1])
if err != nil {
log.Fatalf("usage: %s <attested-tls> - <attested-tls> must be true or false", os.Args[0])
}
attestedTLS = attestedTLSParam
} else if len(os.Args) > 2 {
log.Fatalf("usage: %s <attested-tls>", os.Args[0])
if len(os.Args) < 5 {
log.Fatalf("usage: %s <data-path> <algo-path> <public-key-path> <attested-tls-bool>", os.Args[0])
}
dataPath := os.Args[1]
algoPath := os.Args[2]
pubKeyFile := os.Args[3]
attestedTLSParam, err := strconv.ParseBool(os.Args[4])
if err != nil {
log.Fatalf("usage: %s <data-path> <algo-path> <attested-tls-bool>, <attested-tls-bool> must be a bool value", os.Args[0])
}
attestedTLS := attestedTLSParam
algo, err := os.ReadFile(algoPath)
if err != nil {
log.Fatalf(fmt.Sprintf("failed to read algorithm file: %s", err))
}
data, err := os.ReadFile(dataPath)
if err != nil {
log.Fatalf(fmt.Sprintf("failed to read data file: %s", err))
}
pubKey, err := os.ReadFile(pubKeyFile)
if err != nil {
log.Fatalf(fmt.Sprintf("failed to read public key file: %s", err))
}
pubPem, _ := pem.Decode(pubKey)
algoHash := sha3.Sum256(algo)
dataHash := sha3.Sum256(data)
l, err := vsock.Listen(manager.ManagerVsockPort, nil)
if err != nil {
@@ -39,9 +56,9 @@ func main() {
}
ac := agent.Computation{
ID: "123",
Datasets: agent.Datasets{agent.Dataset{ID: "1", Provider: "pr1"}},
Algorithm: agent.Algorithm{ID: "1", Provider: "pr1"},
ResultConsumers: []string{"1"},
Datasets: agent.Datasets{agent.Dataset{Hash: dataHash, UserKey: pubPem.Bytes}},
Algorithm: agent.Algorithm{Hash: algoHash, UserKey: pubPem.Bytes},
ResultConsumers: []agent.ResultConsumer{{UserKey: pubPem.Bytes}},
AgentConfig: agent.AgentConfig{
LogLevel: "debug",
Port: "7002",
@@ -81,6 +98,11 @@ func SendAgentConfig(cid uint32, ac agent.Computation) error {
if err != nil {
return err
}
var ac2 agent.Computation
if err := json.Unmarshal(payload, &ac2); err != nil {
return err
}
if _, err := conn.Write(payload); err != nil {
return err
}
+1 -1
View File
@@ -4,7 +4,7 @@ Agent accepts binaries programs. To use the python program you need to bundle or
In this example we'll use [pyinstaller](https://pypi.org/project/pyinstaller/)
```shell
pip3 install pandas scikit-learn
pip install pandas scikit-learn
pip install -U pyinstaller
pyinstaller --onefile lin_reg.py
```