Files
cocos/agent/service_test.go
T
Sammy Kerata Oina da31d76c94
CI / checkproto (push) Has been cancelled
CI / lint (push) Has been cancelled
Rust CI Pipeline / rust-check (push) Has been cancelled
CI / test (agent) (push) Has been cancelled
CI / test (cli) (push) Has been cancelled
CI / test (cmd) (push) Has been cancelled
CI / test (internal) (push) Has been cancelled
CI / test (manager, true) (push) Has been cancelled
CI / test (pkg) (push) Has been cancelled
CI / upload-coverage (push) Has been cancelled
NOISSUE - Agent Pull mode for remote resources (#575)
* feat(kbs): implement KBS client for attestation and resource retrieval

- Added KBS client implementation in pkg/kbs/client.go with methods for attestation and resource retrieval.
- Introduced necessary data structures for requests and responses.
- Implemented error handling for various scenarios.

test(kbs): add unit tests for KBS client

- Created comprehensive tests for the KBS client in pkg/kbs/client_test.go.
- Included tests for attestation success and failure cases, as well as resource retrieval.

feat(registry): introduce HTTP and S3 registry implementations

- Added HTTPRegistry for downloading resources over HTTP/HTTPS with retry logic in pkg/registry/http.go.
- Implemented S3Registry for downloading resources from AWS S3 and S3-compatible services in pkg/registry/s3.go.
- Included error handling and configuration options for both registries.

chore(registry): define registry interface and configuration

- Created registry interface and configuration struct in pkg/registry/registry.go.
- Added default configuration settings for registry clients.

docs(cvms): update README for CVMS server configuration and usage

- Enhanced documentation for CVMS server with detailed command-line flags and usage examples.
- Clarified direct upload and remote resource modes, including KBS integration.

fix(cvms): integrate KBS for remote resource handling in main.go

- Updated main.go to support remote datasets and algorithms using KBS.
- Added validation for command-line flags to ensure proper configuration.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: Move ifeq conditional outside define block in attestation-service.mk

Make conditionals cannot be evaluated inside define...endef blocks
when used as recipe bodies. Restructured to define the
ATTESTATION_SERVICE_INSTALL_INIT_SYSTEMD block conditionally based
on BR2_PACKAGE_CC_ATTESTATION_AGENT configuration.

* feat: Implement remote resource downloading for algorithms and datasets using AWS S3/MinIO credentials.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add comprehensive documentation and agent support for testing remote resource download with KBS attestation.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Improve agent logging for remote resource configuration and KBS status, and add a testing guide for remote resource downloads with KBS attestation.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add a comprehensive guide for testing remote resource download with KBS attestation and update multiple package versions to a specific commit.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add failure transitions for resource reception states and a comprehensive guide for testing remote resource downloads with KBS attestation.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Implement remote resource download with KBS attestation in the agent and add a comprehensive testing guide.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* test: Add comprehensive guide for testing remote resource download with KBS attestation and include a debug log in the attestation client.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Delegate KBS attestation and token retrieval to a new attestation-agent service and document remote resource testing.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* client fixes

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* raw evidence

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: Build all Go files in cmd directories, not just main.go

This fixes the issue where fetch_raw_evidence.go wasn't being included
in the attestation-service build.

* fix: Wrap binary evidence in JSON for KBS compatibility

Fixes 'invalid character' error by wrapping raw binary evidence
in a JSON structure with base64 encoding, as expected by KBS.

* chore: Update buildroot packages to c28cefae

Includes fixes for:
1. attestation-service build (including fetch_raw_evidence.go)
2. Agent KBS evidence format (wrapping binary in JSON)

* fix: Implement KBS RCAR handshake with cookies

Fixes 'cookie not found' error (401) from KBS by:
1. Adding CookieJar support to KBS client
2. Implementing GetChallenge() to perform /auth handshake and capture session cookie
3. Updating Agent to get challenge, decode nonce, and use it for evidence generation
4. Regenerating mocks

* chore: Update buildroot packages to f6981ac5

Includes KBS RCAR handshake fix (cookie support + GetChallenge loop)

* fix: Update KBS client JSON tags to kebab-case

Fixes deserialization error (401) from KBS by:
1. Using kebab-case (e.g. extra-params) for JSON tags as per protocol.
2. Initializing ExtraParams as empty object {} instead of null/omitted.

* fix: Wrap attestation evidence in primary_evidence format

Updates Agent to construct 'tee-evidence' payload with:
- primary_evidence: containing the actual quote/data
- additional_evidence: empty JSON object

This matches the Confidential Containers KBS Attestation Protocol requirements.

* fix: Update KBS protocol version to 0.4.0

KBS rejected 0.1.0 with a version mismatch error. Bumping to 0.4.0 to match server expectation.

* fix: Generate ephemeral key for KBS RuntimeData

Updates RuntimeData to include a valid ephemeral EC P-256 public key in JWK format, as required by the KBS RCAR protocol.
Also fixes the KBS client struct to support TEEPubKey as an object.

* fix: Update sample attestation quote to valid JSON

The default attestation.bin was binary, but the KBS Sample Verifier expects a valid JSON quote containing 'svn' and 'report_data'.
Updated the embedded bin file to contain this JSON structure.

* fix: Generate dynamic JSON quote for Sample TEE in FetchRawEvidence

The KBS Sample Verifier expects a JSON object with 'svn' and 'report_data'.
Previously, we were returning raw binary data (reportData+nonce).
This commit updates FetchRawEvidence to return a marshaled JSON structure with:
- svn: "1"
- report_data: base64(req.ReportData)

* refactor: Delegate Sample Attestation to Provider

Refactored sample attestation logic:
- Moved JSON Quote generation into EmptyProvider (standalone mode).
- Updated FetchRawEvidence to call provider.TeeAttestation instead of manual generation.
This enables using the real CC Attestation Agent for UNSPECIFIED platform if configured.

* feat: Add comprehensive debug logging and enforce CC AA usage

Changes:
- Updated EmptyProvider to return error instead of generating mock data
  This forces proper use of CC Attestation Agent's sample attester
- Added detailed logging to attestation-service FetchRawEvidence:
  * Hex dump of evidence (first 200 bytes)
  * String preview of evidence
  * Total evidence length
- Added detailed logging to agent service:
  * Raw evidence hex and string previews
  * KBS evidence JSON preview (first 500 bytes)
  * Evidence lengths at each transformation step

This logging will help diagnose why KBS Sample Verifier is rejecting evidence.

* fix: Enable CC AA by default and add attestation-service log forwarding

Changes:
- Set USE_CC_ATTESTATION_AGENT=true by default in systemd service
- Added StandardOutput/StandardError to forward logs to /var/log/cocos/
- Updated HAL makefile to handle new default value
- This ensures attestation-service uses CC AA's sample attester
- Logs will now be visible in CVMS output for debugging

* feat: Add gRPC log forwarding to attestation-service

Implemented the same log forwarding mechanism used by the agent:
- Added ProtoHandler to write logs to both stdout and logQueue
- Connected to log client (/run/cocos/log.sock) for gRPC forwarding
- Added goroutine to forward logs to CVMS via log client
- Logs will now appear in CVMS output during computation runs

This enables visibility into attestation-service debug output including:
- CC AA connection status
- Evidence generation details (hex dumps, string previews)
- Any errors from providers

* fix: Parse sample evidence JSON instead of base64-encoding it

The attestation-service returns sample evidence as JSON:
{"svn":"1","report_data":"base64..."}

The agent was incorrectly base64-encoding this JSON string again.
KBS Sample Verifier expects the parsed JSON object directly.

Fixed by:
- Parsing the JSON evidence from attestation-service
- Passing the parsed object directly in primary_evidence.evidence
- This matches what KBS Sample Verifier expects

* debug: Increase KBS evidence logging preview to 1000 bytes

Show the complete JSON structure being sent to KBS to debug
the attestation failure.

* debug: Add comprehensive CC AA configuration logging

Added debug logs to show:
- Whether CC AA is enabled in config
- CC AA address being used
- Connection success/failure
- Which provider is ultimately selected
- Warning when falling back to EmptyProvider

This will help diagnose why EmptyProvider is being used
instead of CC Attestation Agent.

* debug: Add startup logging for log client connection

Added log message to show if log client connection succeeds
at attestation-service startup. This will help diagnose why
logs aren't appearing in CVMS output.

* feat: Add retry logic with exponential backoff to log client

Added simple retry mechanism to handle concurrent log requests:
- 3 retry attempts with exponential backoff (10ms, 20ms, 40ms)
- Applies to both SendLog and SendEvent methods
- Centralized in log client so all services benefit
- Should eliminate 'failed to send log' errors from concurrent requests

This fixes the issue where attestation-service logs weren't
appearing in CVMS output due to dropped messages.

* fix: Flatten sample evidence fields in primary_evidence for KBS

KBS Sample Verifier expects svn and report_data at the top level
of primary_evidence, not nested under an 'evidence' key.

Changed structure from:
{"primary_evidence": {"tee": "sample", "evidence": {"svn": "1", ...}}}

To:
{"primary_evidence": {"tee": "sample", "svn": "1", "report_data": "...", ...}}

This matches what KBS expects when deserializing the Quote structure.

* fix: Use sample quote directly as primary_evidence per KBS protocol

According to KBS attestation protocol spec, for sample TEE type,
primary_evidence should be the sample quote JSON directly:
{"svn": "1", "report_data": "..."}

Removed extra 'tee' and 'platform' fields that were causing KBS
to fail deserializing the Quote structure. The 'tee' field is
already sent in the Request payload during RCAR handshake.

Refs:
- https://github.com/confidential-containers/trustee/blob/main/kbs/docs/kbs_attestation_protocol.md
- https://github.com/confidential-containers/guest-components/blob/main/attestation-agent/attester/src/sample/mod.rs

* fix: Make CC AA required for sample attestation when configured

When USE_CC_ATTESTATION_AGENT=true, attestation-service now
requires AA to be available for NoCC/sample platform. This ensures
sample evidence always comes from AA with the correct KBS format.

Changes:
- Error out if AA connection fails for NoCC platform when AA is configured
- Only use EmptyProvider if AA is explicitly NOT configured
- Prevents incorrect sample evidence format from EmptyProvider

This ensures attestation-service delegates to AA for sample evidence
generation instead of creating it itself.

* fix: Implement proper RCAR protocol with tee-pubkey and runtime-data hash

Fixed KBS attestation error 'REPORT_DATA is different from that in Sample Quote'

Changes:
1. Generate ephemeral EC key pair BEFORE getting evidence from AA
2. Create runtime-data with nonce + tee-pubkey (JWK format)
3. Hash runtime-data (SHA-256) and use as report_data for AA
4. This binds the tee-pubkey to the TEE evidence per RCAR protocol

The report_data in the evidence now matches what KBS expects:
hash(runtime-data) instead of computation ID.

This completes the full RCAR protocol implementation:
- Request → Challenge → Attestation (with bound tee-pubkey) → Response

* fix(agent): use simple nonce for Sample attestation report_data

For Sample/NoCC attestation, use the raw nonce bytes directly as
report_data instead of hashing runtime-data. This avoids JSON
serialization mismatches with the KBS Sample verifier.

Real TEEs (TDX/SNP) still use runtime-data hash binding to
cryptographically bind the ephemeral tee-pubkey to the evidence.

* fix(agent): use RFC 8785 canonical JSON for runtime-data hashing

The KBS Sample attestation verifier (and likely others) expects the
report_data to be the SHA-256 hash of the *canonical* JSON serialization
(RFC 8785) of the runtime-data. Standard Go JSON marshaling does not
guarantee key ordering, leading to hash mismatches.

This change uses github.com/gowebpki/jcs to canonicalize the runtime-data
before hashing, ensuring compatibility with the KBS RCAR implementation.
Also reverted the temporary 'simple nonce' workaround.

* feat(hal): add CoCo Keyprovider and Skopeo packages

- Add coco-keyprovider buildroot package with systemd service
- Add skopeo buildroot package for OCI image handling
- Add ocicrypt_keyprovider.conf for encrypted image decryption
- Update Config.in to include new packages

This enables standard CoCo ecosystem integration for encrypted
OCI images instead of custom S3/HTTP registry clients.

* feat(oci): add OCI image handling package with Skopeo integration

- Add pkg/oci/types.go with ResourceSource and ImageManifest types
- Add pkg/oci/skopeo.go with Skopeo wrapper for pull/decrypt
- Add pkg/oci/extract.go for extracting algorithms and datasets from layers

This package provides OCI image handling using Skopeo and CoCo
Keyprovider for encrypted image decryption, replacing custom
S3/HTTP registry clients.

* chore: regenerate protobuf files for updated cvms.proto

* refactor(agent): replace S3/HTTP/KBS with OCI package

- Remove pkg/kbs and pkg/registry imports
- Add pkg/oci import for OCI image handling
- Replace downloadAndDecryptResource with OCI-based implementation
- Use Skopeo + CoCo Keyprovider for automatic decryption
- Reduce code from ~240 lines to ~70 lines

This eliminates custom KBS RCAR handshake, S3/HTTP registry clients,
and manual decryption logic. CoCo Keyprovider handles all decryption
automatically via ocicrypt protocol.

* chore: remove obsolete pkg/kbs and pkg/registry packages

- Delete pkg/kbs/ (custom KBS client, ~300 lines)
- Delete pkg/registry/ (S3/HTTP registry clients, ~400 lines)
- Remove unused imports from agent/service.go
- Run go mod tidy to clean up dependencies

These packages have been replaced by pkg/oci with Skopeo and
CoCo Keyprovider for standard CoCo ecosystem integration.

* fix(agent): update ResourceSource struct to include type and encryption fields

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix(hal): update CoCo Keyprovider to v0.16.0 and fix build path

- Update version from v0.11.0 to v0.16.0 (matches attestation agent)
- Fix install path: target is at repo root, not in coco_keyprovider subdir
- This fixes the build error where coco_keyprovider binary wasn't found

The cargo workspace in guest-components builds to a shared target/
directory at the repository root, not within each crate's subdirectory.

* feat: Update remote resources testing guide to use kbs-client and coco-keyprovider for key management and encryption, enable insecure TLS for Skopeo, and enhance CVMS with

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Update component versions, revise image encryption documentation, and sanitize OCI image paths for Skopeo compatibility.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add `decompress` option to Dataset and `algo_type`/`algo_args` to Algorithm protobuf messages, updating client, test, and build configurations.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* Update multiple package versions and enhance OCI image extraction error reporting for missing algorithm files.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* chore: Bump package versions, improve OCI image extraction debugging by returning seen files, and remove unused dataset type parsing from test code.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* refactor: Migrate OCI extraction to use structured logging with `slog` and `context`, and update package versions.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Bump multiple component versions, add encrypted status for computation inputs and algorithms, and refine OCI layer extraction warnings.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* logging

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add `Encrypted` field to algorithm and dataset resource sources and update all component versions.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: update component versions, integrate coco-keyprovider service, and configure ocicrypt key provider.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: add support for KBS parameters and dataset/algorithm hash calculations in CVMS

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: update resource download and extraction logic to support requirements.txt and improve hash verification

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* chore: Update dependencies, improve code style, and add GetRawEvidence to attestation client mocks.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* Refactor code structure for improved readability and maintainability

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: update golangci configuration to include errcheck for build path and remove unnecessary exclusions

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: streamline kernel command line handling in QEMU args construction

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: add attestation binary and update checksum tests and policy structure

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* Add unit tests for attestation agent, attestation, log, crypto, OCI, and Skopeo clients

- Implement tests for the attestation agent client including Unix socket and TCP address handling, token retrieval, and error scenarios.
- Enhance attestation client tests to cover fetching raw evidence for various platforms (SNP, TDX, VTPM, SNPvTPM) and validate error handling.
- Introduce log client tests to verify retry behavior for sending logs and events.
- Create comprehensive tests for crypto package focusing on AES-GCM decryption, encrypted resource parsing, and key unwrapping.
- Add tests for OCI package to validate algorithm and dataset extraction, including JSON serialization of OCILayout.
- Implement Skopeo client tests to ensure proper functionality for image pulling, inspecting, and resource source handling.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: handle JSON marshal errors in test cases for decrypt and extract functions

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* test: add comprehensive tests for algorithm and dataset extraction with various scenarios

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* refactor: replace hardcoded Python script content with constant variable

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: remove redundant mock expectation for SendAgentConfig in TestCreateVMWithAaKbsParams

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* test: add tests for event sending failure, dataset extraction with path traversal, and Skopeo client behavior

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* test: add tests for download and decryption of resources with various URL formats

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* refactor: Introduce OCIClient interface for agent service to improve testability of OCI image operations and enhance related tests.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* refactor: Change `get_uint64_from_tcb` to accept `TcbVersion` by value and use `u64::from` for type conversions.

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
2026-03-16 14:48:55 +01:00

1237 lines
35 KiB
Go
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package agent
import (
"archive/tar"
"compress/gzip"
"context"
"crypto/rand"
"encoding/json"
"fmt"
"log"
"log/slog"
"os"
"path/filepath"
"testing"
"time"
mglog "github.com/absmach/supermq/logger"
"github.com/absmach/supermq/pkg/errors"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/ultravioletrs/cocos/agent/algorithm"
"github.com/ultravioletrs/cocos/agent/algorithm/python"
agentevents "github.com/ultravioletrs/cocos/agent/events"
"github.com/ultravioletrs/cocos/agent/events/mocks"
runnerpb "github.com/ultravioletrs/cocos/agent/runner"
"github.com/ultravioletrs/cocos/agent/statemachine"
smmocks "github.com/ultravioletrs/cocos/agent/statemachine/mocks"
"github.com/ultravioletrs/cocos/pkg/attestation"
"github.com/ultravioletrs/cocos/pkg/attestation/vtpm"
runnermocks "github.com/ultravioletrs/cocos/pkg/clients/grpc/runner/mocks"
"github.com/ultravioletrs/cocos/pkg/oci"
"golang.org/x/crypto/sha3"
"google.golang.org/grpc/metadata"
"google.golang.org/protobuf/types/known/emptypb"
)
type MockOCIClient struct {
mock.Mock
}
func (m *MockOCIClient) PullAndDecrypt(ctx context.Context, source oci.ResourceSource, destDir string) error {
args := m.Called(ctx, source, destDir)
return args.Error(0)
}
var (
algoPath = "../test/manual/algo/lin_reg.py"
reqPath = "../test/manual/algo/requirements.txt"
dataPath = "../test/manual/data/iris.csv"
)
const datasetFile = "iris.csv"
func TestAlgo(t *testing.T) {
algo, err := os.ReadFile(algoPath)
require.NoError(t, err)
algoHash := sha3.Sum256(algo)
vtpm.ExternalTPM = &vtpm.DummyRWC{}
reqFile, err := os.ReadFile(reqPath)
require.NoError(t, err)
testCases := []struct {
name string
err error
algo Algorithm
algoType string
}{
{
name: "Test Algo successfully",
algo: Algorithm{
Algorithm: algo,
Hash: algoHash,
},
algoType: "python",
err: nil,
},
{
name: "Test Algo successfully with requirements file",
algo: Algorithm{
Algorithm: algo,
Hash: algoHash,
Requirements: reqFile,
},
algoType: "python",
err: nil,
},
{
name: "Test Algo type binary successfully",
algo: Algorithm{
Algorithm: algo,
Hash: algoHash,
},
algoType: "bin",
err: nil,
},
{
name: "Test Algo type wasm successfully",
algo: Algorithm{
Algorithm: algo,
Hash: algoHash,
},
algoType: "wasm",
err: nil,
},
{
name: "Test Algo type docker successfully",
algo: Algorithm{
Algorithm: algo,
Hash: algoHash,
},
algoType: "docker",
err: nil,
},
{
name: "Test algo hash mismatch",
algo: Algorithm{},
algoType: "python",
err: ErrHashMismatch,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
err = os.RemoveAll("datasets")
require.NoError(t, err)
ctx := metadata.NewIncomingContext(context.Background(),
metadata.Pairs(algorithm.AlgoTypeKey, tc.algoType, python.PyRuntimeKey, python.PyRuntime),
)
events := new(mocks.Service)
events.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return()
ctx, cancel := context.WithCancel(ctx)
defer cancel()
client := new(MockAttestationClient)
runnerCli := new(runnermocks.Client)
runnerCli.On("Run", mock.Anything, mock.Anything).Return(&runnerpb.RunResponse{}, nil)
svc := New(ctx, mglog.NewMock(), events, client, runnerCli, 0)
err := svc.InitComputation(ctx, testComputation(t))
require.NoError(t, err)
time.Sleep(300 * time.Millisecond)
err = svc.Algo(ctx, tc.algo)
assert.True(t, errors.Contains(err, tc.err), "expected %v, got %v", tc.err, err)
t.Cleanup(func() {
err = os.RemoveAll("venv")
err = os.RemoveAll("algo")
err = os.RemoveAll("datasets")
})
})
}
}
func TestData(t *testing.T) {
algo, err := os.ReadFile(algoPath)
require.NoError(t, err)
algoHash := sha3.Sum256(algo)
vtpm.ExternalTPM = &vtpm.DummyRWC{}
alg := Algorithm{
Hash: algoHash,
Algorithm: algo,
}
data, err := os.ReadFile(dataPath)
require.NoError(t, err)
dataHash := sha3.Sum256(data)
cases := []struct {
name string
data Dataset
err error
}{
{
name: "Test data successfully",
data: Dataset{
Hash: dataHash,
Dataset: data,
Filename: datasetFile,
},
},
{
name: "Test State not ready",
data: Dataset{
Dataset: data,
Hash: dataHash,
Filename: datasetFile,
},
err: ErrStateNotReady,
},
{
name: "Test File name does not match manifest",
data: Dataset{
Dataset: data,
Hash: dataHash,
Filename: "invalid",
},
err: ErrFileNameMismatch,
},
{
name: "Test dataset not declared in manifest",
data: Dataset{
Filename: datasetFile,
},
err: ErrUndeclaredDataset,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
ctx := metadata.NewIncomingContext(context.Background(),
metadata.Pairs(
algorithm.AlgoTypeKey, "python",
python.PyRuntimeKey, python.PyRuntime),
)
events := new(mocks.Service)
events.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return()
if tc.err != ErrUndeclaredDataset {
ctx = IndexToContext(ctx, 0)
}
ctx, cancel := context.WithCancel(ctx)
defer cancel()
client := new(MockAttestationClient)
runnerCli := new(runnermocks.Client)
runnerCli.On("Run", mock.Anything, mock.Anything).Return(&runnerpb.RunResponse{}, nil)
svc := New(ctx, mglog.NewMock(), events, client, runnerCli, 0)
err := svc.InitComputation(ctx, testComputation(t))
require.NoError(t, err)
time.Sleep(300 * time.Millisecond)
if tc.err != ErrStateNotReady {
err = svc.Algo(ctx, alg)
require.NoError(t, err)
time.Sleep(300 * time.Millisecond)
}
err = svc.Data(ctx, tc.data)
assert.True(t, errors.Contains(err, tc.err), "expected %v, got %v", tc.err, err)
t.Cleanup(func() {
_ = os.RemoveAll("datasets")
_ = os.RemoveAll("results")
err = os.RemoveAll("venv")
err = os.RemoveAll("algo")
})
})
}
}
func TestResult(t *testing.T) {
cases := []struct {
name string
err error
setup func(svc *agentService)
ctxSetup func(ctx context.Context) context.Context
state statemachine.State
}{
{
name: "Test results not ready",
err: ErrResultsNotReady,
setup: func(svc *agentService) {
},
state: Running,
},
{
name: "Test undeclared consumer",
err: ErrUndeclaredConsumer,
setup: func(svc *agentService) {
svc.computation.ResultConsumers = []ResultConsumer{{UserKey: []byte("user")}}
},
ctxSetup: func(ctx context.Context) context.Context {
return ctx
},
state: ConsumingResults,
},
{
name: "Test results consumed and event sent",
err: nil,
setup: func(svc *agentService) {
svc.computation.ResultConsumers = []ResultConsumer{{UserKey: []byte("key")}}
},
ctxSetup: func(ctx context.Context) context.Context {
return IndexToContext(ctx, 0)
},
state: ConsumingResults,
},
}
for _, tc := range cases {
events := new(mocks.Service)
events.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return()
t.Run(tc.name, func(t *testing.T) {
ctx := metadata.NewIncomingContext(context.Background(),
metadata.Pairs(algorithm.AlgoTypeKey, "python", python.PyRuntimeKey, python.PyRuntime),
)
if tc.ctxSetup != nil {
ctx = tc.ctxSetup(ctx)
}
client := new(MockAttestationClient)
runnerCli := new(runnermocks.Client)
sm := new(smmocks.StateMachine)
sm.On("Start", ctx).Return(nil)
sm.On("GetState").Return(tc.state)
sm.On("SendEvent", mock.Anything).Return()
svc := &agentService{
sm: sm,
eventSvc: events,
attestationClient: client,
runnerClient: runnerCli,
computation: testComputation(t),
}
go func() {
if err := svc.sm.Start(ctx); err != nil {
t.Errorf("Error starting state machine: %v", err)
}
}()
tc.setup(svc)
_, err := svc.Result(ctx)
t.Cleanup(func() {
_ = os.RemoveAll("datasets")
_ = os.RemoveAll("results")
})
assert.ErrorIs(t, err, tc.err, "expected %v, got %v", tc.err, err)
})
}
}
func TestAttestation(t *testing.T) {
client := new(MockAttestationClient)
cases := []struct {
name string
reportData [vtpm.SEVNonce]byte
nonce [vtpm.Nonce]byte
rawQuote []uint8
platform attestation.PlatformType
err error
}{
{
name: "Test SNP attestation successful",
reportData: generateReportData(),
nonce: [32]byte{},
rawQuote: make([]uint8, 0),
platform: attestation.SNP,
err: nil,
},
{
name: "Test SNP attestation failed",
reportData: generateReportData(),
nonce: [32]byte{},
rawQuote: nil,
platform: attestation.SNP,
err: ErrAttestationFailed,
},
{
name: "Test vTPM attestation successful",
reportData: generateReportData(),
nonce: [32]byte{},
rawQuote: make([]uint8, 0),
platform: attestation.VTPM,
err: nil,
},
{
name: "Test vTPM attestation failed",
reportData: generateReportData(),
nonce: [32]byte{},
rawQuote: nil,
platform: attestation.VTPM,
err: ErrAttestationVTpmFailed,
},
{
name: "Test SNP-vTPM attestation successful",
reportData: generateReportData(),
nonce: [32]byte{},
rawQuote: make([]uint8, 0),
platform: attestation.SNPvTPM,
err: nil,
},
{
name: "Test SNP-vTPM attestation failed",
reportData: generateReportData(),
nonce: [32]byte{},
rawQuote: nil,
platform: attestation.SNPvTPM,
err: ErrAttestationVTpmFailed,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
events := new(mocks.Service)
events.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return()
ctx := metadata.NewIncomingContext(context.Background(),
metadata.Pairs(algorithm.AlgoTypeKey, "python", python.PyRuntimeKey, python.PyRuntime),
)
ctx, cancel := context.WithCancel(ctx)
defer cancel()
getQuote := client.On("GetAttestation", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(tc.rawQuote, tc.err)
if tc.err != ErrAttestationFailed && tc.err != ErrAttestationVTpmFailed {
getQuote = client.On("GetAttestation", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return(tc.nonce[:], nil)
}
defer getQuote.Unset()
runnerCli := new(runnermocks.Client)
svc := New(ctx, mglog.NewMock(), events, client, runnerCli, 0)
time.Sleep(300 * time.Millisecond)
_, err := svc.Attestation(ctx, tc.reportData, tc.nonce, tc.platform)
assert.True(t, errors.Contains(err, tc.err), "expected %v, got %v", tc.err, err)
})
}
}
func TestAzureAttestationToken(t *testing.T) {
client := new(MockAttestationClient)
cases := []struct {
name string
nonce [vtpm.Nonce]byte
token []byte
err error
}{
{
name: "Azure token fetch successful",
nonce: [32]byte{1, 2, 3}, // any test nonce
token: []byte("mockToken"),
err: nil,
},
{
name: "Azure token fetch failed",
nonce: [32]byte{4, 5, 6},
token: []byte{},
err: ErrAttestationType,
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
events := new(mocks.Service)
events.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return()
client.On("GetAzureToken", mock.Anything, tc.nonce).Return(tc.token, tc.err)
ctx := context.Background()
runnerCli := new(runnermocks.Client)
svc := New(ctx, mglog.NewMock(), events, client, runnerCli, 0)
_, err := svc.AzureAttestationToken(ctx, tc.nonce)
assert.True(t, errors.Contains(err, tc.err), "expected error %v, got %v", tc.err, err)
})
}
}
func generateReportData() [vtpm.SEVNonce]byte {
bytes := make([]byte, vtpm.SEVNonce)
_, err := rand.Read(bytes)
if err != nil {
log.Fatalf("Failed to generate random bytes: %v", err)
}
return [64]byte(bytes)
}
func testComputation(t *testing.T) Computation {
algo, err := os.ReadFile(algoPath)
require.NoError(t, err)
algoHash := sha3.Sum256(algo)
data, err := os.ReadFile(dataPath)
require.NoError(t, err)
dataHash := sha3.Sum256(data)
return Computation{
ID: "1",
Name: "sample computation",
Description: "sample description",
Datasets: []Dataset{{Hash: dataHash, UserKey: []byte("key"), Dataset: data, Filename: datasetFile}},
Algorithm: Algorithm{Hash: algoHash, UserKey: []byte("key"), Algorithm: algo},
ResultConsumers: []ResultConsumer{{UserKey: []byte("key")}},
}
}
func TestStopComputation(t *testing.T) {
cases := []struct {
name string
setupDirs bool
setupAlgo bool
algoStopErr error
expectedErr error
}{
{
name: "Stop computation successfully",
setupDirs: true,
setupAlgo: true,
algoStopErr: nil,
expectedErr: nil,
},
{
name: "Stop computation with algorithm stop error",
setupDirs: true,
setupAlgo: true,
algoStopErr: fmt.Errorf("algorithm stop failed"),
expectedErr: nil, // Warn only
},
// We log warnings but don't return error in StopComputation in new implementation for Stop failure.
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
events := new(mocks.Service)
events.On("SendEvent", mock.Anything, "Stopped", "Stopped", mock.Anything).Return()
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
defer cancel()
client := new(MockAttestationClient)
runnerCli := new(runnermocks.Client)
// Mock Stop call
var stopErr error
if tc.algoStopErr != nil {
stopErr = tc.algoStopErr
}
runnerCli.On("Stop", mock.Anything, mock.Anything).Return(&emptypb.Empty{}, stopErr)
svc := New(ctx, mglog.NewMock(), events, client, runnerCli, 0).(*agentService)
svc.computation = Computation{
ID: "test-computation",
Name: "test",
}
if tc.setupDirs {
err := os.MkdirAll(algorithm.DatasetsDir, 0o755)
require.NoError(t, err)
err = os.MkdirAll(algorithm.ResultsDir, 0o755)
require.NoError(t, err)
}
// Use real dirs for test
// algorithm.DatasetsDir refers to global var?
// "github.com/ultravioletrs/cocos/agent/algorithm"
// It uses hardcoded path "datasets" and "results" in current dir.
// Tests create them in current dir.
err := svc.StopComputation(ctx)
if tc.expectedErr != nil {
assert.Error(t, err)
assert.Contains(t, err.Error(), tc.expectedErr.Error())
} else {
assert.NoError(t, err)
}
assert.Equal(t, ReceivingManifest, svc.sm.GetState())
assert.Nil(t, svc.result)
assert.Nil(t, svc.runError)
assert.False(t, svc.resultsConsumed)
events.AssertExpectations(t)
_ = os.RemoveAll(algorithm.DatasetsDir)
_ = os.RemoveAll(algorithm.ResultsDir)
})
}
}
func TestStopComputationIntegration(t *testing.T) {
if testing.Short() {
t.Skip("Skipping integration test in short mode")
}
algo := []byte("#!/bin/bash\necho 'test algorithm'")
algoHash := sha3.Sum256(algo)
testDir := "test_integration"
err := os.MkdirAll(testDir, 0o755)
require.NoError(t, err)
defer os.RemoveAll(testDir)
algoFile := filepath.Join(testDir, "test_algo")
err = os.WriteFile(algoFile, algo, 0o755)
require.NoError(t, err)
events := new(mocks.Service)
events.On("SendEvent", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return()
ctx := metadata.NewIncomingContext(context.Background(),
metadata.Pairs(algorithm.AlgoTypeKey, "bin"),
)
ctx, cancel := context.WithCancel(ctx)
defer cancel()
client := new(MockAttestationClient)
runnerCli := new(runnermocks.Client)
runnerCli.On("Run", mock.Anything, mock.Anything).Return(&runnerpb.RunResponse{}, nil)
runnerCli.On("Stop", mock.Anything, mock.Anything).Return(&emptypb.Empty{}, nil)
svc := New(ctx, mglog.NewMock(), events, client, runnerCli, 0)
computation := Computation{
ID: "integration-test",
Name: "Integration Test",
Algorithm: Algorithm{
Hash: algoHash,
Algorithm: algo,
},
}
err = svc.InitComputation(ctx, computation)
require.NoError(t, err)
time.Sleep(100 * time.Millisecond)
err = svc.Algo(ctx, Algorithm{
Hash: algoHash,
Algorithm: algo,
})
require.NoError(t, err)
time.Sleep(100 * time.Millisecond)
err = svc.StopComputation(ctx)
assert.NoError(t, err)
assert.Equal(t, "ReceivingManifest", svc.State())
}
func TestStopComputationConcurrent(t *testing.T) {
events := new(mocks.Service)
events.On("SendEvent", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return()
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
defer cancel()
client := new(MockAttestationClient)
runnerCli := new(runnermocks.Client)
runnerCli.On("Stop", mock.Anything, mock.Anything).Return(&emptypb.Empty{}, nil)
svc := New(ctx, mglog.NewMock(), events, client, runnerCli, 0)
svc.(*agentService).computation = Computation{
ID: "concurrent-test",
Name: "Concurrent Test",
}
const numGoroutines = 10
errChan := make(chan error, numGoroutines)
for i := 0; i < numGoroutines; i++ {
go func() {
err := svc.StopComputation(ctx)
errChan <- err
}()
}
var errors []error
for i := 0; i < numGoroutines; i++ {
err := <-errChan
if err != nil {
errors = append(errors, err)
}
}
assert.True(t, len(errors) < numGoroutines, "All StopComputation calls failed")
}
// newTestAgentService creates a minimal agentService for direct method testing.
func newTestAgentService(sm statemachine.StateMachine, eventSvc agentevents.Service) *agentService {
return &agentService{
logger: slog.Default(),
eventSvc: eventSvc,
sm: sm,
}
}
func TestDownloadAndDecryptResource(t *testing.T) {
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
sm.On("SendEvent", mock.Anything).Return().Maybe()
svc := newTestAgentService(sm, eventsSvc)
ctx := context.Background()
t.Run("unsupported URL format no type", func(t *testing.T) {
source := &ResourceSource{URL: "http://unsupported-format"}
_, err := svc.downloadAndDecryptResource(ctx, source, "algorithm")
require.Error(t, err)
assert.Contains(t, err.Error(), "unsupported source URL format")
})
t.Run("ftp URL unsupported format", func(t *testing.T) {
source := &ResourceSource{URL: "ftp://some-server/file"}
_, err := svc.downloadAndDecryptResource(ctx, source, "algorithm")
require.Error(t, err)
assert.Contains(t, err.Error(), "unsupported source URL format")
})
t.Run("unsupported explicit source type", func(t *testing.T) {
source := &ResourceSource{Type: "s3-bucket", URL: "s3://mybucket/algo"}
_, err := svc.downloadAndDecryptResource(ctx, source, "algorithm")
require.Error(t, err)
assert.Contains(t, err.Error(), "unsupported source type: s3-bucket")
})
t.Run("docker:// URL inferred as oci-image routes to skopeo", func(t *testing.T) {
// This exercises the oci-image path; will fail at skopeo step
source := &ResourceSource{URL: "docker://invalid.example.com/algo:latest"}
_, err := svc.downloadAndDecryptResource(ctx, source, "algorithm")
require.Error(t, err)
// Should be a skopeo or OCI error, not an "unsupported" error
assert.NotContains(t, err.Error(), "unsupported source URL format")
})
t.Run("oci: URL inferred as oci-image routes to skopeo", func(t *testing.T) {
source := &ResourceSource{URL: "oci:some-local-dir"}
_, err := svc.downloadAndDecryptResource(ctx, source, "algorithm")
require.Error(t, err)
assert.NotContains(t, err.Error(), "unsupported source URL format")
})
t.Run("explicit oci-image type routes to skopeo", func(t *testing.T) {
source := &ResourceSource{Type: "oci-image", URL: "docker://invalid.example.com/algo:latest"}
_, err := svc.downloadAndDecryptResource(ctx, source, "algorithm")
require.Error(t, err)
assert.NotContains(t, err.Error(), "unsupported source type")
})
t.Run("dataset resource type with oci-image", func(t *testing.T) {
source := &ResourceSource{Type: "oci-image", URL: "docker://invalid.example.com/data:latest"}
_, err := svc.downloadAndDecryptResource(ctx, source, "dataset")
require.Error(t, err)
})
}
func TestDownloadAlgorithmIfRemote(t *testing.T) {
t.Run("no source configured - no-op, waits for direct upload", func(t *testing.T) {
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
// No SendEvent expected — just the no-op path
svc := newTestAgentService(sm, eventsSvc)
svc.computation = Computation{} // Algorithm.Source == nil
svc.downloadAlgorithmIfRemote(ReceivingAlgorithm)
assert.Nil(t, svc.runError)
sm.AssertExpectations(t)
})
t.Run("source set but KBS disabled - no-op", func(t *testing.T) {
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
svc := newTestAgentService(sm, eventsSvc)
svc.computation = Computation{
Algorithm: Algorithm{
Source: &ResourceSource{URL: "docker://registry/algo:latest"},
},
KBS: KBSConfig{Enabled: false},
}
svc.downloadAlgorithmIfRemote(ReceivingAlgorithm)
assert.Nil(t, svc.runError)
sm.AssertExpectations(t)
})
t.Run("source + KBS enabled - download fails, sends RunFailed", func(t *testing.T) {
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
sm.On("SendEvent", RunFailed).Return().Once()
svc := newTestAgentService(sm, eventsSvc)
svc.computation = Computation{
Algorithm: Algorithm{
Source: &ResourceSource{
Type: "oci-image",
URL: "docker://invalid.example.com/algo:latest",
},
},
KBS: KBSConfig{Enabled: true, URL: "https://kbs.example.com"},
}
svc.downloadAlgorithmIfRemote(ReceivingAlgorithm)
assert.NotNil(t, svc.runError)
assert.Contains(t, svc.runError.Error(), "failed to download and decrypt algorithm")
sm.AssertExpectations(t)
})
t.Run("unsupported URL format - download fails, sends RunFailed", func(t *testing.T) {
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
sm.On("SendEvent", RunFailed).Return().Once()
svc := newTestAgentService(sm, eventsSvc)
svc.computation = Computation{
Algorithm: Algorithm{
Source: &ResourceSource{
URL: "http://unsupported-format/algo",
},
},
KBS: KBSConfig{Enabled: true},
}
svc.downloadAlgorithmIfRemote(ReceivingAlgorithm)
assert.NotNil(t, svc.runError)
sm.AssertExpectations(t)
})
}
func TestDownloadDatasetsIfRemote(t *testing.T) {
t.Run("no datasets with remote sources - no-op", func(t *testing.T) {
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
svc := newTestAgentService(sm, eventsSvc)
// Dataset with no Source
dataHash := sha3.Sum256([]byte("testdata"))
svc.computation = Computation{
Datasets: []Dataset{
{Hash: dataHash, Filename: "data.csv"},
},
KBS: KBSConfig{Enabled: true},
}
svc.downloadDatasetsIfRemote(ReceivingData)
// No RunFailed event, no DataReceived event
sm.AssertExpectations(t)
})
t.Run("no datasets at all - no-op", func(t *testing.T) {
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
svc := newTestAgentService(sm, eventsSvc)
svc.computation = Computation{
Datasets: []Dataset{},
KBS: KBSConfig{Enabled: true},
}
svc.downloadDatasetsIfRemote(ReceivingData)
sm.AssertExpectations(t)
})
t.Run("KBS disabled even with source - no-op", func(t *testing.T) {
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
svc := newTestAgentService(sm, eventsSvc)
svc.computation = Computation{
Datasets: []Dataset{
{
Filename: "data.csv",
Source: &ResourceSource{URL: "docker://registry/data:latest"},
},
},
KBS: KBSConfig{Enabled: false},
}
svc.downloadDatasetsIfRemote(ReceivingData)
sm.AssertExpectations(t)
})
t.Run("remote dataset + KBS enabled - download fails, sends RunFailed", func(t *testing.T) {
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
sm.On("SendEvent", RunFailed).Return().Once()
svc := newTestAgentService(sm, eventsSvc)
svc.computation = Computation{
Datasets: []Dataset{
{
Filename: "data.csv",
Source: &ResourceSource{
Type: "oci-image",
URL: "docker://invalid.example.com/data:latest",
},
},
},
KBS: KBSConfig{Enabled: true, URL: "https://kbs.example.com"},
}
svc.downloadDatasetsIfRemote(ReceivingData)
sm.AssertExpectations(t)
})
t.Run("unsupported URL fails - sends RunFailed", func(t *testing.T) {
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
sm.On("SendEvent", RunFailed).Return().Once()
svc := newTestAgentService(sm, eventsSvc)
svc.computation = Computation{
Datasets: []Dataset{
{
Filename: "data.csv",
Source: &ResourceSource{
URL: "ftp://unsupported/data",
},
},
},
KBS: KBSConfig{Enabled: true},
}
svc.downloadDatasetsIfRemote(ReceivingData)
sm.AssertExpectations(t)
})
}
func TestRunComputation(t *testing.T) {
// Helper to set up a temp working directory and restore CWD afterwards.
withTempDir := func(t *testing.T) (tmpDir string, restore func()) {
t.Helper()
origDir, err := os.Getwd()
require.NoError(t, err)
tmpDir = t.TempDir()
require.NoError(t, os.Chdir(tmpDir))
return tmpDir, func() { _ = os.Chdir(origDir) }
}
t.Run("algo file not found sends RunFailed", func(t *testing.T) {
_, restore := withTempDir(t)
defer restore()
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
sm.On("SendEvent", RunFailed).Return().Once()
svc := newTestAgentService(sm, eventsSvc)
// No algo file exists runComputation should hit the ReadFile error path.
svc.runComputation(Running)
assert.Error(t, svc.runError)
assert.Contains(t, svc.runError.Error(), "failed to read algo file")
sm.AssertExpectations(t)
})
t.Run("runner client returns error sends RunFailed", func(t *testing.T) {
_, restore := withTempDir(t)
defer restore()
// Write a dummy algo file so ReadFile succeeds.
require.NoError(t, os.WriteFile("algo", []byte("#!/bin/sh\necho ok\n"), 0o755))
runnerCli := new(runnermocks.Client)
runnerCli.On("Run", mock.Anything, mock.Anything).Return((*runnerpb.RunResponse)(nil), fmt.Errorf("runner unavailable"))
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
sm.On("SendEvent", RunFailed).Return().Once()
svc := newTestAgentService(sm, eventsSvc)
svc.runnerClient = runnerCli
svc.runComputation(Running)
assert.Error(t, svc.runError)
assert.Contains(t, svc.runError.Error(), "runner unavailable")
sm.AssertExpectations(t)
})
t.Run("runner returns non-empty error field sends RunFailed", func(t *testing.T) {
_, restore := withTempDir(t)
defer restore()
require.NoError(t, os.WriteFile("algo", []byte("#!/bin/sh\necho ok\n"), 0o755))
runnerCli := new(runnermocks.Client)
runnerCli.On("Run", mock.Anything, mock.Anything).Return(&runnerpb.RunResponse{Error: "computation crashed"}, nil)
eventsSvc := new(mocks.Service)
eventsSvc.EXPECT().SendEvent(mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
sm.On("SendEvent", RunFailed).Return().Once()
svc := newTestAgentService(sm, eventsSvc)
svc.runnerClient = runnerCli
svc.runComputation(Running)
assert.Error(t, svc.runError)
assert.Contains(t, svc.runError.Error(), "computation crashed")
sm.AssertExpectations(t)
})
}
func TestIMAMeasurements(t *testing.T) {
t.Run("error when IMA measurements file does not exist in non-SGX environment", func(t *testing.T) {
// In a regular test environment (non-SGX), the IMA measurements file
// at /sys/kernel/security/integrity/ima/ascii_runtime_measurements won't exist.
// Verify our error handling works correctly.
origPath := ImaMeasurementsFilePath
ImaMeasurementsFilePath = "/non/existent/path"
defer func() { ImaMeasurementsFilePath = origPath }()
eventsSvc := new(mocks.Service)
eventsSvc.On("SendEvent", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
svc := newTestAgentService(sm, eventsSvc)
data, pcr10, err := svc.IMAMeasurements(context.Background())
assert.Error(t, err)
assert.Contains(t, err.Error(), "error reading Linux IMA measurements file")
assert.Nil(t, data)
assert.Nil(t, pcr10)
})
t.Run("successful reading of IMA measurements", func(t *testing.T) {
tempFile := filepath.Join(t.TempDir(), "ima_measurements")
content := []byte("10 sha1:0000000000000000000000000000000000000000 ima-ng sha256:0000000000000000000000000000000000000000000000000000000000000000 /usr/bin/python3\n")
err := os.WriteFile(tempFile, content, 0o644)
require.NoError(t, err)
vtpm.ExternalTPM = &vtpm.DummyRWC{}
origPath := ImaMeasurementsFilePath
ImaMeasurementsFilePath = tempFile
defer func() { ImaMeasurementsFilePath = origPath }()
eventsSvc := new(mocks.Service)
eventsSvc.On("SendEvent", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
svc := newTestAgentService(sm, eventsSvc)
data, pcr10, err := svc.IMAMeasurements(context.Background())
assert.NoError(t, err)
assert.Equal(t, content, data)
assert.NotEmpty(t, pcr10)
})
}
func TestDownloadAlgorithmIfRemote_Success(t *testing.T) {
// Skip this test in short mode as it might involve more setup if we were using real OCI
if testing.Short() {
t.Skip("skipping in short mode")
}
origDir, _ := os.Getwd()
tmpDir := t.TempDir()
require.NoError(t, os.Chdir(tmpDir))
defer func() { require.NoError(t, os.Chdir(origDir)) }()
eventsSvc := new(mocks.Service)
eventsSvc.On("SendEvent", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
sm.On("SendEvent", AlgorithmReceived).Return().Once()
mockOCI := new(MockOCIClient)
algoContent := []byte("print('hello')")
mockOCI.On("PullAndDecrypt", mock.Anything, mock.Anything, mock.Anything).Run(func(args mock.Arguments) {
destDir := args.String(2)
setupMinimalOCI(t, destDir, "main.py", string(algoContent))
}).Return(nil)
svc := newTestAgentService(sm, eventsSvc)
svc.ociClient = mockOCI
algoContent = []byte("print('hello')")
algoHash := sha3.Sum256(algoContent)
svc.computation = Computation{
Algorithm: Algorithm{
Hash: algoHash,
AlgoType: "python",
Source: &ResourceSource{
Type: "oci-image",
URL: "docker://test/image",
},
},
KBS: KBSConfig{Enabled: true},
}
// We need to bypass oci.ExtractAlgorithm by manually creating what it would create
// OR use a real-enough looking OCI layout.
// Since we can't easily mock oci.ExtractAlgorithm, we'll try to provide a minimal OCI layout
// so that oci.ExtractAlgorithm doesn't fail.
svc.downloadAlgorithmIfRemote(ReceivingAlgorithm)
assert.Nil(t, svc.runError)
assert.True(t, svc.algoReceived)
sm.AssertExpectations(t)
mockOCI.AssertExpectations(t)
}
func setupMinimalOCI(t *testing.T, ociDir, filename, content string) {
t.Helper()
blobsDir := filepath.Join(ociDir, "blobs", "sha256")
require.NoError(t, os.MkdirAll(blobsDir, 0o755))
layerPath := filepath.Join(blobsDir, "layer123")
layerFile, err := os.Create(layerPath)
require.NoError(t, err)
gw := gzip.NewWriter(layerFile)
tw := tar.NewWriter(gw)
hdr := &tar.Header{
Name: filename,
Mode: 0o755,
Size: int64(len(content)),
}
require.NoError(t, tw.WriteHeader(hdr))
_, err = tw.Write([]byte(content))
require.NoError(t, err)
require.NoError(t, tw.Close())
require.NoError(t, gw.Close())
require.NoError(t, layerFile.Close())
manifest := struct {
Layers []struct {
Digest string `json:"digest"`
} `json:"layers"`
}{
Layers: []struct {
Digest string `json:"digest"`
}{{Digest: "sha256:layer123"}},
}
manifestData, err := json.Marshal(manifest)
require.NoError(t, err)
require.NoError(t, os.WriteFile(filepath.Join(blobsDir, "manifest123"), manifestData, 0o644))
index := oci.OCIIndex{
SchemaVersion: 2,
Manifests: []struct {
MediaType string `json:"mediaType"`
Digest string `json:"digest"`
Size int `json:"size"`
}{{Digest: "sha256:manifest123", Size: len(manifestData)}},
}
indexData, err := json.Marshal(index)
require.NoError(t, err)
require.NoError(t, os.WriteFile(filepath.Join(ociDir, "index.json"), indexData, 0o644))
}
func TestDownloadDatasetsIfRemote_Success(t *testing.T) {
if testing.Short() {
t.Skip("skipping in short mode")
}
origDir, _ := os.Getwd()
tmpDir := t.TempDir()
require.NoError(t, os.Chdir(tmpDir))
defer func() { require.NoError(t, os.Chdir(origDir)) }()
eventsSvc := new(mocks.Service)
eventsSvc.On("SendEvent", mock.Anything, mock.Anything, mock.Anything, mock.Anything).Return().Maybe()
sm := &smmocks.StateMachine{}
sm.On("SendEvent", DataReceived).Return().Once()
mockOCI := new(MockOCIClient)
dataContent := []byte("a,b,c\n1,2,3")
mockOCI.On("PullAndDecrypt", mock.Anything, mock.Anything, mock.Anything).Run(func(args mock.Arguments) {
destDir := args.String(2)
setupMinimalOCI(t, destDir, "data.csv", string(dataContent))
}).Return(nil)
svc := newTestAgentService(sm, eventsSvc)
svc.ociClient = mockOCI
dataContent = []byte("a,b,c\n1,2,3")
dataHash := sha3.Sum256(dataContent)
svc.computation = Computation{
Datasets: []Dataset{
{
Filename: "data.csv",
Hash: dataHash,
Source: &ResourceSource{
Type: "oci-image",
URL: "docker://test/image",
},
},
},
KBS: KBSConfig{Enabled: true},
}
err := os.MkdirAll(algorithm.DatasetsDir, 0o755)
require.NoError(t, err)
svc.downloadDatasetsIfRemote(ReceivingData)
assert.Nil(t, svc.runError)
assert.Len(t, svc.computation.Datasets, 0)
sm.AssertExpectations(t)
mockOCI.AssertExpectations(t)
}