COCOS-432 - FDE support (#553)
CI / lint (push) Has been cancelled
CI / test (agent) (push) Has been cancelled
CI / test (cli) (push) Has been cancelled
CI / test (cmd) (push) Has been cancelled
CI / test (internal) (push) Has been cancelled
CI / test (manager, true) (push) Has been cancelled
CI / test (pkg) (push) Has been cancelled
CI / upload-coverage (push) Has been cancelled

* initial FDE setup

* add Manager support

* fix igvmmeasure build

* rebase on main

* add tests

* NOISSUE - Allow interoperability with CC Attestation Agent (#568)

* feat: Add Confidential Containers attestation agent as an alternative attestation backend with new proto definitions and build system integration.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: Update protoc-gen-go and protoc-gen-go-grpc versions in CI workflow

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add mock implementation for AttestationAgentServiceClient and corresponding tests

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: Add missing periods to test function comments in provider_test.go

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* NOISSUE - Agent Pull mode for remote resources (#575)

* feat(kbs): implement KBS client for attestation and resource retrieval

- Added KBS client implementation in pkg/kbs/client.go with methods for attestation and resource retrieval.
- Introduced necessary data structures for requests and responses.
- Implemented error handling for various scenarios.

test(kbs): add unit tests for KBS client

- Created comprehensive tests for the KBS client in pkg/kbs/client_test.go.
- Included tests for attestation success and failure cases, as well as resource retrieval.

feat(registry): introduce HTTP and S3 registry implementations

- Added HTTPRegistry for downloading resources over HTTP/HTTPS with retry logic in pkg/registry/http.go.
- Implemented S3Registry for downloading resources from AWS S3 and S3-compatible services in pkg/registry/s3.go.
- Included error handling and configuration options for both registries.

chore(registry): define registry interface and configuration

- Created registry interface and configuration struct in pkg/registry/registry.go.
- Added default configuration settings for registry clients.

docs(cvms): update README for CVMS server configuration and usage

- Enhanced documentation for CVMS server with detailed command-line flags and usage examples.
- Clarified direct upload and remote resource modes, including KBS integration.

fix(cvms): integrate KBS for remote resource handling in main.go

- Updated main.go to support remote datasets and algorithms using KBS.
- Added validation for command-line flags to ensure proper configuration.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: Move ifeq conditional outside define block in attestation-service.mk

Make conditionals cannot be evaluated inside define...endef blocks
when used as recipe bodies. Restructured to define the
ATTESTATION_SERVICE_INSTALL_INIT_SYSTEMD block conditionally based
on BR2_PACKAGE_CC_ATTESTATION_AGENT configuration.

* feat: Implement remote resource downloading for algorithms and datasets using AWS S3/MinIO credentials.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add comprehensive documentation and agent support for testing remote resource download with KBS attestation.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Improve agent logging for remote resource configuration and KBS status, and add a testing guide for remote resource downloads with KBS attestation.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add a comprehensive guide for testing remote resource download with KBS attestation and update multiple package versions to a specific commit.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add failure transitions for resource reception states and a comprehensive guide for testing remote resource downloads with KBS attestation.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Implement remote resource download with KBS attestation in the agent and add a comprehensive testing guide.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* test: Add comprehensive guide for testing remote resource download with KBS attestation and include a debug log in the attestation client.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Delegate KBS attestation and token retrieval to a new attestation-agent service and document remote resource testing.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* client fixes

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* raw evidence

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: Build all Go files in cmd directories, not just main.go

This fixes the issue where fetch_raw_evidence.go wasn't being included
in the attestation-service build.

* fix: Wrap binary evidence in JSON for KBS compatibility

Fixes 'invalid character' error by wrapping raw binary evidence
in a JSON structure with base64 encoding, as expected by KBS.

* chore: Update buildroot packages to c28cefae

Includes fixes for:
1. attestation-service build (including fetch_raw_evidence.go)
2. Agent KBS evidence format (wrapping binary in JSON)

* fix: Implement KBS RCAR handshake with cookies

Fixes 'cookie not found' error (401) from KBS by:
1. Adding CookieJar support to KBS client
2. Implementing GetChallenge() to perform /auth handshake and capture session cookie
3. Updating Agent to get challenge, decode nonce, and use it for evidence generation
4. Regenerating mocks

* chore: Update buildroot packages to f6981ac5

Includes KBS RCAR handshake fix (cookie support + GetChallenge loop)

* fix: Update KBS client JSON tags to kebab-case

Fixes deserialization error (401) from KBS by:
1. Using kebab-case (e.g. extra-params) for JSON tags as per protocol.
2. Initializing ExtraParams as empty object {} instead of null/omitted.

* fix: Wrap attestation evidence in primary_evidence format

Updates Agent to construct 'tee-evidence' payload with:
- primary_evidence: containing the actual quote/data
- additional_evidence: empty JSON object

This matches the Confidential Containers KBS Attestation Protocol requirements.

* fix: Update KBS protocol version to 0.4.0

KBS rejected 0.1.0 with a version mismatch error. Bumping to 0.4.0 to match server expectation.

* fix: Generate ephemeral key for KBS RuntimeData

Updates RuntimeData to include a valid ephemeral EC P-256 public key in JWK format, as required by the KBS RCAR protocol.
Also fixes the KBS client struct to support TEEPubKey as an object.

* fix: Update sample attestation quote to valid JSON

The default attestation.bin was binary, but the KBS Sample Verifier expects a valid JSON quote containing 'svn' and 'report_data'.
Updated the embedded bin file to contain this JSON structure.

* fix: Generate dynamic JSON quote for Sample TEE in FetchRawEvidence

The KBS Sample Verifier expects a JSON object with 'svn' and 'report_data'.
Previously, we were returning raw binary data (reportData+nonce).
This commit updates FetchRawEvidence to return a marshaled JSON structure with:
- svn: "1"
- report_data: base64(req.ReportData)

* refactor: Delegate Sample Attestation to Provider

Refactored sample attestation logic:
- Moved JSON Quote generation into EmptyProvider (standalone mode).
- Updated FetchRawEvidence to call provider.TeeAttestation instead of manual generation.
This enables using the real CC Attestation Agent for UNSPECIFIED platform if configured.

* feat: Add comprehensive debug logging and enforce CC AA usage

Changes:
- Updated EmptyProvider to return error instead of generating mock data
  This forces proper use of CC Attestation Agent's sample attester
- Added detailed logging to attestation-service FetchRawEvidence:
  * Hex dump of evidence (first 200 bytes)
  * String preview of evidence
  * Total evidence length
- Added detailed logging to agent service:
  * Raw evidence hex and string previews
  * KBS evidence JSON preview (first 500 bytes)
  * Evidence lengths at each transformation step

This logging will help diagnose why KBS Sample Verifier is rejecting evidence.

* fix: Enable CC AA by default and add attestation-service log forwarding

Changes:
- Set USE_CC_ATTESTATION_AGENT=true by default in systemd service
- Added StandardOutput/StandardError to forward logs to /var/log/cocos/
- Updated HAL makefile to handle new default value
- This ensures attestation-service uses CC AA's sample attester
- Logs will now be visible in CVMS output for debugging

* feat: Add gRPC log forwarding to attestation-service

Implemented the same log forwarding mechanism used by the agent:
- Added ProtoHandler to write logs to both stdout and logQueue
- Connected to log client (/run/cocos/log.sock) for gRPC forwarding
- Added goroutine to forward logs to CVMS via log client
- Logs will now appear in CVMS output during computation runs

This enables visibility into attestation-service debug output including:
- CC AA connection status
- Evidence generation details (hex dumps, string previews)
- Any errors from providers

* fix: Parse sample evidence JSON instead of base64-encoding it

The attestation-service returns sample evidence as JSON:
{"svn":"1","report_data":"base64..."}

The agent was incorrectly base64-encoding this JSON string again.
KBS Sample Verifier expects the parsed JSON object directly.

Fixed by:
- Parsing the JSON evidence from attestation-service
- Passing the parsed object directly in primary_evidence.evidence
- This matches what KBS Sample Verifier expects

* debug: Increase KBS evidence logging preview to 1000 bytes

Show the complete JSON structure being sent to KBS to debug
the attestation failure.

* debug: Add comprehensive CC AA configuration logging

Added debug logs to show:
- Whether CC AA is enabled in config
- CC AA address being used
- Connection success/failure
- Which provider is ultimately selected
- Warning when falling back to EmptyProvider

This will help diagnose why EmptyProvider is being used
instead of CC Attestation Agent.

* debug: Add startup logging for log client connection

Added log message to show if log client connection succeeds
at attestation-service startup. This will help diagnose why
logs aren't appearing in CVMS output.

* feat: Add retry logic with exponential backoff to log client

Added simple retry mechanism to handle concurrent log requests:
- 3 retry attempts with exponential backoff (10ms, 20ms, 40ms)
- Applies to both SendLog and SendEvent methods
- Centralized in log client so all services benefit
- Should eliminate 'failed to send log' errors from concurrent requests

This fixes the issue where attestation-service logs weren't
appearing in CVMS output due to dropped messages.

* fix: Flatten sample evidence fields in primary_evidence for KBS

KBS Sample Verifier expects svn and report_data at the top level
of primary_evidence, not nested under an 'evidence' key.

Changed structure from:
{"primary_evidence": {"tee": "sample", "evidence": {"svn": "1", ...}}}

To:
{"primary_evidence": {"tee": "sample", "svn": "1", "report_data": "...", ...}}

This matches what KBS expects when deserializing the Quote structure.

* fix: Use sample quote directly as primary_evidence per KBS protocol

According to KBS attestation protocol spec, for sample TEE type,
primary_evidence should be the sample quote JSON directly:
{"svn": "1", "report_data": "..."}

Removed extra 'tee' and 'platform' fields that were causing KBS
to fail deserializing the Quote structure. The 'tee' field is
already sent in the Request payload during RCAR handshake.

Refs:
- https://github.com/confidential-containers/trustee/blob/main/kbs/docs/kbs_attestation_protocol.md
- https://github.com/confidential-containers/guest-components/blob/main/attestation-agent/attester/src/sample/mod.rs

* fix: Make CC AA required for sample attestation when configured

When USE_CC_ATTESTATION_AGENT=true, attestation-service now
requires AA to be available for NoCC/sample platform. This ensures
sample evidence always comes from AA with the correct KBS format.

Changes:
- Error out if AA connection fails for NoCC platform when AA is configured
- Only use EmptyProvider if AA is explicitly NOT configured
- Prevents incorrect sample evidence format from EmptyProvider

This ensures attestation-service delegates to AA for sample evidence
generation instead of creating it itself.

* fix: Implement proper RCAR protocol with tee-pubkey and runtime-data hash

Fixed KBS attestation error 'REPORT_DATA is different from that in Sample Quote'

Changes:
1. Generate ephemeral EC key pair BEFORE getting evidence from AA
2. Create runtime-data with nonce + tee-pubkey (JWK format)
3. Hash runtime-data (SHA-256) and use as report_data for AA
4. This binds the tee-pubkey to the TEE evidence per RCAR protocol

The report_data in the evidence now matches what KBS expects:
hash(runtime-data) instead of computation ID.

This completes the full RCAR protocol implementation:
- Request → Challenge → Attestation (with bound tee-pubkey) → Response

* fix(agent): use simple nonce for Sample attestation report_data

For Sample/NoCC attestation, use the raw nonce bytes directly as
report_data instead of hashing runtime-data. This avoids JSON
serialization mismatches with the KBS Sample verifier.

Real TEEs (TDX/SNP) still use runtime-data hash binding to
cryptographically bind the ephemeral tee-pubkey to the evidence.

* fix(agent): use RFC 8785 canonical JSON for runtime-data hashing

The KBS Sample attestation verifier (and likely others) expects the
report_data to be the SHA-256 hash of the *canonical* JSON serialization
(RFC 8785) of the runtime-data. Standard Go JSON marshaling does not
guarantee key ordering, leading to hash mismatches.

This change uses github.com/gowebpki/jcs to canonicalize the runtime-data
before hashing, ensuring compatibility with the KBS RCAR implementation.
Also reverted the temporary 'simple nonce' workaround.

* feat(hal): add CoCo Keyprovider and Skopeo packages

- Add coco-keyprovider buildroot package with systemd service
- Add skopeo buildroot package for OCI image handling
- Add ocicrypt_keyprovider.conf for encrypted image decryption
- Update Config.in to include new packages

This enables standard CoCo ecosystem integration for encrypted
OCI images instead of custom S3/HTTP registry clients.

* feat(oci): add OCI image handling package with Skopeo integration

- Add pkg/oci/types.go with ResourceSource and ImageManifest types
- Add pkg/oci/skopeo.go with Skopeo wrapper for pull/decrypt
- Add pkg/oci/extract.go for extracting algorithms and datasets from layers

This package provides OCI image handling using Skopeo and CoCo
Keyprovider for encrypted image decryption, replacing custom
S3/HTTP registry clients.

* chore: regenerate protobuf files for updated cvms.proto

* refactor(agent): replace S3/HTTP/KBS with OCI package

- Remove pkg/kbs and pkg/registry imports
- Add pkg/oci import for OCI image handling
- Replace downloadAndDecryptResource with OCI-based implementation
- Use Skopeo + CoCo Keyprovider for automatic decryption
- Reduce code from ~240 lines to ~70 lines

This eliminates custom KBS RCAR handshake, S3/HTTP registry clients,
and manual decryption logic. CoCo Keyprovider handles all decryption
automatically via ocicrypt protocol.

* chore: remove obsolete pkg/kbs and pkg/registry packages

- Delete pkg/kbs/ (custom KBS client, ~300 lines)
- Delete pkg/registry/ (S3/HTTP registry clients, ~400 lines)
- Remove unused imports from agent/service.go
- Run go mod tidy to clean up dependencies

These packages have been replaced by pkg/oci with Skopeo and
CoCo Keyprovider for standard CoCo ecosystem integration.

* fix(agent): update ResourceSource struct to include type and encryption fields

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix(hal): update CoCo Keyprovider to v0.16.0 and fix build path

- Update version from v0.11.0 to v0.16.0 (matches attestation agent)
- Fix install path: target is at repo root, not in coco_keyprovider subdir
- This fixes the build error where coco_keyprovider binary wasn't found

The cargo workspace in guest-components builds to a shared target/
directory at the repository root, not within each crate's subdirectory.

* feat: Update remote resources testing guide to use kbs-client and coco-keyprovider for key management and encryption, enable insecure TLS for Skopeo, and enhance CVMS with

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Update component versions, revise image encryption documentation, and sanitize OCI image paths for Skopeo compatibility.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add `decompress` option to Dataset and `algo_type`/`algo_args` to Algorithm protobuf messages, updating client, test, and build configurations.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* Update multiple package versions and enhance OCI image extraction error reporting for missing algorithm files.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* chore: Bump package versions, improve OCI image extraction debugging by returning seen files, and remove unused dataset type parsing from test code.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* refactor: Migrate OCI extraction to use structured logging with `slog` and `context`, and update package versions.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Bump multiple component versions, add encrypted status for computation inputs and algorithms, and refine OCI layer extraction warnings.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* logging

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: Add `Encrypted` field to algorithm and dataset resource sources and update all component versions.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: update component versions, integrate coco-keyprovider service, and configure ocicrypt key provider.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: add support for KBS parameters and dataset/algorithm hash calculations in CVMS

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: update resource download and extraction logic to support requirements.txt and improve hash verification

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* chore: Update dependencies, improve code style, and add GetRawEvidence to attestation client mocks.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* Refactor code structure for improved readability and maintainability

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: update golangci configuration to include errcheck for build path and remove unnecessary exclusions

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: streamline kernel command line handling in QEMU args construction

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* feat: add attestation binary and update checksum tests and policy structure

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* Add unit tests for attestation agent, attestation, log, crypto, OCI, and Skopeo clients

- Implement tests for the attestation agent client including Unix socket and TCP address handling, token retrieval, and error scenarios.
- Enhance attestation client tests to cover fetching raw evidence for various platforms (SNP, TDX, VTPM, SNPvTPM) and validate error handling.
- Introduce log client tests to verify retry behavior for sending logs and events.
- Create comprehensive tests for crypto package focusing on AES-GCM decryption, encrypted resource parsing, and key unwrapping.
- Add tests for OCI package to validate algorithm and dataset extraction, including JSON serialization of OCILayout.
- Implement Skopeo client tests to ensure proper functionality for image pulling, inspecting, and resource source handling.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: handle JSON marshal errors in test cases for decrypt and extract functions

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* test: add comprehensive tests for algorithm and dataset extraction with various scenarios

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* refactor: replace hardcoded Python script content with constant variable

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* fix: remove redundant mock expectation for SendAgentConfig in TestCreateVMWithAaKbsParams

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* test: add tests for event sending failure, dataset extraction with path traversal, and Skopeo client behavior

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* test: add tests for download and decryption of resources with various URL formats

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* refactor: Introduce OCIClient interface for agent service to improve testability of OCI image operations and enhance related tests.

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* refactor: Change `get_uint64_from_tcb` to accept `TcbVersion` by value and use `u64::from` for type conversions.

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>

* initial FDE setup

* add Manager support

* add cloud-init script

* rebase onto main

* add blank lines

* add tdx rtmr support

* add FDE flow

* use DiskConfig.Format instead of fixed values

* add tests and expand Manager README.md

* add curl command

* add encrypted partition support

* remove nbd

* add dm-verity

* fix manager boot sequence

---------

Signed-off-by: Sammy Oina <sammyoina@gmail.com>
Co-authored-by: ultraviolet <cocosai@worker-52.local.pragmatic-it.com>
Co-authored-by: Sammy Kerata Oina <44265300+SammyOina@users.noreply.github.com>
This commit is contained in:
Danko Miladinovic
2026-05-08 14:59:13 +02:00
committed by GitHub
parent d5badba547
commit 81fe0b11b5
54 changed files with 2436 additions and 30 deletions
+27 -4
View File
@@ -85,6 +85,29 @@ var (
ImaPcrIndex = 10
)
func ensureDir(path string, mode os.FileMode) error {
info, err := os.Stat(path)
switch {
case err == nil:
if info.IsDir() {
return nil
}
if err := os.Remove(path); err != nil {
return fmt.Errorf("removing non-directory path %q: %w", path, err)
}
case os.IsNotExist(err):
// Continue and create it below.
default:
return fmt.Errorf("stating path %q: %w", path, err)
}
if err := os.MkdirAll(path, mode); err != nil {
return fmt.Errorf("creating directory %q: %w", path, err)
}
return nil
}
var (
// ErrMalformedEntity indicates malformed entity specification (e.g.
// invalid username or password).
@@ -478,8 +501,8 @@ func (as *agentService) downloadAlgorithmIfRemote(state statemachine.State) {
as.algoReceived = true
as.algoRequirements = res.Requirements // Store requirements for installation
// Create datasets directory
if err := os.Mkdir(algorithm.DatasetsDir, 0o755); err != nil {
// The initramfs may have already provisioned /cocos/datasets.
if err := ensureDir(algorithm.DatasetsDir, 0o755); err != nil {
as.runError = fmt.Errorf("error creating datasets directory: %w", err)
as.logger.Error(as.runError.Error())
as.sm.SendEvent(RunFailed)
@@ -976,7 +999,7 @@ func (as *agentService) Algo(ctx context.Context, algo Algorithm) error {
as.algoRequirements = algo.Requirements
as.algoReceived = true
if err := os.Mkdir(algorithm.DatasetsDir, 0o755); err != nil {
if err := ensureDir(algorithm.DatasetsDir, 0o755); err != nil {
return fmt.Errorf("error creating datasets directory: %v", err)
}
@@ -1145,7 +1168,7 @@ func (as *agentService) runComputation(state statemachine.State) {
}
}()
if err := os.Mkdir(algorithm.ResultsDir, 0o755); err != nil {
if err := ensureDir(algorithm.ResultsDir, 0o755); err != nil {
as.mu.Lock()
as.runError = fmt.Errorf("error creating results directory: %s", err.Error())
as.mu.Unlock()
+9
View File
@@ -0,0 +1,9 @@
source "$BR2_EXTERNAL_COCOS_PATH/package/agent/Config.in"
source "$BR2_EXTERNAL_COCOS_PATH/package/attestation-service/Config.in"
source "$BR2_EXTERNAL_COCOS_PATH/package/cc-attestation-agent/Config.in"
source "$BR2_EXTERNAL_COCOS_PATH/package/coco-keyprovider/Config.in"
source "$BR2_EXTERNAL_COCOS_PATH/package/wasmedge/Config.in"
source "$BR2_EXTERNAL_COCOS_PATH/package/log-forwarder/Config.in"
source "$BR2_EXTERNAL_COCOS_PATH/package/computation-runner/Config.in"
source "$BR2_EXTERNAL_COCOS_PATH/package/egress-proxy/Config.in"
source "$BR2_EXTERNAL_COCOS_PATH/package/ingress-proxy/Config.in"
+214
View File
@@ -0,0 +1,214 @@
# Disk Image Workflow
This directory is the Buildroot external tree for the current Cocos disk test
VM image and its runtime configuration.
## Layout
- [configs/cocos_defconfig](./configs/cocos_defconfig):
Buildroot configuration for the bootable image.
- [board/rootfs-overlay/init](./board/rootfs-overlay/init):
early initramfs script that provisions `/cocos`, mounts the real root, and
switches into the installed system.
- [board/cocos/genimage.cfg](./board/cocos/genimage.cfg):
GPT disk layout for the final `disk.img`.
- [board/cocos/post-image.sh](./board/cocos/post-image.sh):
builds the minimal initramfs, stages EFI files, signs boot artifacts, and
assembles `disk.img`.
- [external.desc](./external.desc): Buildroot external tree descriptor.
- [external.mk](./external.mk): includes package makefiles from `package/*`.
## Current Buildroot Image
The current Buildroot flow produces a bootable GPT disk image:
- `efi` partition: FAT EFI system partition with GRUB, kernel, and initramfs
- `root` partition: ext4 root filesystem protected by dm-verity
- `verity` partition: dm-verity hash tree for the root filesystem
- `cocos` partition: blank partition provisioned at boot as an encrypted ext4
filesystem mounted at `/cocos`
The final image is written to:
```bash
output/images/disk.img
```
The root filesystem image is also available separately as:
```bash
output/images/rootfs.ext4
```
## Current Boot Flow
At boot, GRUB loads:
- `bzImage`
- `initrd.cpio.gz`
The initramfs script in
[board/rootfs-overlay/init](./board/rootfs-overlay/init)
then:
1. mounts `/proc`, `/sys`, `devtmpfs`, and `devpts`
2. assumes the boot disk is `/dev/sda`
3. opens a dm-verity mapping for the root filesystem using:
- `/dev/sda2` as the data partition
- `/dev/sda3` as the verity hash partition
- `roothash=` from the kernel command line
4. mounts `/dev/mapper/root_verity` read-only at `/root`
5. generates a fresh ephemeral key
6. formats `/dev/sda4` as LUKS2
7. opens it as `/dev/mapper/cocos_crypt`
8. formats that mapper as ext4 and mounts it at `/root/cocos`
9. creates working directories on `/cocos`, including:
- `/cocos/.cache/oci`
- `/cocos/datasets`
- `/cocos/docker`
- `/cocos/cocos_init`
10. mounts `tmpfs` on `/tmp` and `/var` because the root filesystem is
intentionally read-only
11. bind-mounts `/cocos/docker` onto `/var/lib/docker`
12. bind-mounts `/cocos/cocos_init` onto `/cocos_init`
13. rewrites `/etc/fstab` in the mounted root to describe the live runtime
14. preserves or adds 9P mounts for:
- `certs_share` -> `/etc/certs`
- `env_share` -> `/etc/cocos`
15. securely wipes the temporary LUKS key file
16. runs `switch_root /root /sbin/init`
Important details:
- the root filesystem is verified through dm-verity before it is mounted
- `/cocos` is encrypted with an ephemeral per-boot key
- that key is not persisted, so `/cocos` is provisioned fresh on each boot
## Runtime Filesystem Model
The running system is split into:
- read-only root on `/`
- encrypted writable storage on `/cocos`
- `tmpfs` on `/tmp`
- `tmpfs` on `/var`
Service state that must survive within a boot session is redirected away from
the read-only root:
- Docker data lives on `/cocos/docker`
- agent setup scripts work through `/cocos_init`, which is backed by
`/cocos/cocos_init`
- algorithm datasets and results live under `/cocos`
This means services can use `/cocos` like a regular directory tree after boot,
even though it is backed by an encrypted mapper created in early userspace.
## systemd Runtime Expectations
Several services depend on files mounted from 9P shares under `/etc/certs` and
`/etc/cocos`. To avoid boot-order races, the rootfs overlay includes systemd
drop-ins under:
```bash
board/rootfs-overlay/usr/lib/systemd/system/*service.d/
```
These drop-ins require the relevant mount points before starting services such
as:
- `egress-proxy.service`
- `log-forwarder.service`
- `computation-runner.service`
- `cocos-agent.service`
The overlay also ships tmpfiles rules in
[board/rootfs-overlay/usr/lib/tmpfiles.d/cocos.conf](./board/rootfs-overlay/usr/lib/tmpfiles.d/cocos.conf)
to create:
- `/var/log/cocos`
- `/run/cocos`
## Agent Packaging In Buildroot
The Buildroot `agent` package is wired to build the binary from the local Cocos
checkout, not only from a downloaded release snapshot. The package definition is
in [package/agent/agent.mk](./package/agent/agent.mk).
That package currently:
- builds `cocos-agent` from the local source tree
- installs the local
[cocos-agent.service](../../init/systemd/cocos-agent.service)
- installs the local
[agent_setup.sh](../../init/systemd/agent_setup.sh)
- installs the local
[agent_start_script.sh](../../init/systemd/agent_start_script.sh)
So changes under:
- `cocos/agent/...`
- `cocos/init/systemd/...`
are intended to be picked up by the next Buildroot rebuild.
## Buildroot Packages And Tools
The current `cocos_defconfig` includes the components needed by the boot flow
and runtime image, including:
- systemd
- DHCP client
- `cryptsetup`
- `eudev`
- `e2fsprogs`
- Docker, containerd, and runc
- `skopeo`
- TPM2 tools
- 9P filesystem support
- GRUB2 EFI boot support
- host `genimage`
The initramfs built in `post-image.sh` is intentionally minimal and contains
only the binaries needed for early boot, dm-verity root verification, and
`/cocos` provisioning.
## Secure Boot Notes
During `post-image.sh`:
- GRUB is rebuilt with `--disable-shim-lock`
- `bootx64.efi` and `bzImage` are signed with the configured Secure Boot keys
when those keys are present
This flow is designed for booting directly through OVMF with your own enrolled
keys. It does not currently rely on booting through `shim`.
## Rebuilding
This directory is meant to be used as a Buildroot external tree. From this
directory, configure a Buildroot checkout with:
```bash
make -C /path/to/buildroot BR2_EXTERNAL=$PWD cocos_defconfig
```
Then build with:
```bash
make -C /path/to/buildroot BR2_EXTERNAL=$PWD -j$(nproc)
```
The resulting boot image is:
```bash
/path/to/buildroot/output/images/disk.img
```
Additional generated artifacts include:
```bash
/path/to/buildroot/output/images/rootfs.ext4
/path/to/buildroot/output/images/rootfs.verity
/path/to/buildroot/output/images/rootfs.roothash
```
+42
View File
@@ -0,0 +1,42 @@
image efi-part.vfat {
vfat {
file EFI {
image = "efi-part/EFI"
}
file bzImage {
image = "efi-part/bzImage"
}
file initrd.cpio.gz {
image = "efi-part/initrd.cpio.gz"
}
}
size = 256M
}
image disk.img {
hdimage {
partition-table-type = "gpt"
}
partition efi {
image = "efi-part.vfat"
partition-type-uuid = "C12A7328-F81F-11D2-BA4B-00A0C93EC93B"
offset = 1M
bootable = true
}
partition root {
image = "rootfs.ext4"
partition-type-uuid = "0FC63DAF-8483-4772-8E79-3D69D8477DE4"
}
partition verity {
image = "rootfs.verity"
partition-type-uuid = "0FC63DAF-8483-4772-8E79-3D69D8477DE4"
}
partition cocos {
size = "20480M"
partition-type-uuid = "0FC63DAF-8483-4772-8E79-3D69D8477DE4"
}
}
+279
View File
@@ -0,0 +1,279 @@
###
# Architecture / base
###
CONFIG_SYSVIPC=y
CONFIG_SMP=y
CONFIG_EXPERT=y
CONFIG_LOCALVERSION_AUTO=n
###
# Modules
###
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
###
# Virtualization
###
CONFIG_HYPERVISOR_GUEST=y
CONFIG_PARAVIRT=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
CONFIG_KVM_SW_PROTECTED_VM=y
CONFIG_KVM_INTEL=y
CONFIG_VIRT_DRIVERS=y
###
# Cgroups — base + Docker/container subsystems
###
CONFIG_CGROUPS=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_MISC=y
CONFIG_CGROUP_PIDS=y
CONFIG_CGROUP_BPF=y
CONFIG_CGROUP_NET_PRIO=y
CONFIG_CGROUP_NET_CLASSID=y
CONFIG_CPUSETS=y
CONFIG_MEMCG=y
CONFIG_BLK_CGROUP=y
###
# Namespaces — required by containerd / runc
###
CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y
###
# PCI
###
CONFIG_PCI=y
CONFIG_PCI_MSI=y
CONFIG_IRQ_REMAP=y
###
# Initramfs
###
CONFIG_BLK_DEV_INITRD=y
CONFIG_RD_GZIP=y
###
# Block devices
###
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_BLK_DEV_SD=y
CONFIG_SCSI_VIRTIO=y
CONFIG_ATA=y
CONFIG_ATA_PIIX=y
CONFIG_VIRTIO_BLK=y
# Loop device (used by containerd image mounts)
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_LOOP_MIN_COUNT=8
###
# Device mapper — FDE, dm-verity, dm-crypt, dm-integrity
# These must be built-in (y) because they are needed before the
# rootfs is mounted, during the initramfs FDE init stage.
###
CONFIG_MD=y
CONFIG_BLK_DEV_DM_BUILTIN=y
CONFIG_BLK_DEV_DM=y
CONFIG_DM_CRYPT=y
CONFIG_DM_VERITY=y
CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG=y
# CONFIG_DM_VERITY_FEC is not set
CONFIG_DM_INTEGRITY=y
CONFIG_DM_INIT=y
###
# Networking — base
###
CONFIG_NET=y
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_INET=y
# CONFIG_WIRELESS is not set
CONFIG_NETDEVICES=y
CONFIG_VIRTIO_NET=y
CONFIG_NE2K_PCI=y
CONFIG_8139CP=y
# CONFIG_WLAN is not set
CONFIG_VSOCKETS=y
CONFIG_VIRTIO_VSOCKETS=y
# Virtual Ethernet pairs and bridge (Docker networking)
CONFIG_VETH=m
CONFIG_BRIDGE=m
CONFIG_BRIDGE_NETFILTER=m
###
# Netfilter — Docker NAT, iptables, conntrack (modules, loaded on demand)
###
CONFIG_NETFILTER=y
CONFIG_NETFILTER_ADVANCED=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CONNTRACK_MARK=y
CONFIG_NF_NAT=m
CONFIG_NF_NAT_MASQUERADE=y
CONFIG_NF_TABLES=y
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
###
# BPF
###
CONFIG_BPF_SYSCALL=y
###
# Filesystems
###
CONFIG_EXT4_FS=y
CONFIG_OVERLAY_FS=y
CONFIG_AUTOFS4_FS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_PROC_FS=y
CONFIG_SYSFS=y
###
# 9P filesystem (virtio shares for certs and env)
###
CONFIG_NET_9P=y
CONFIG_NET_9P_VIRTIO=y
CONFIG_9P_FS=y
CONFIG_9P_FS_POSIX_ACL=y
CONFIG_9P_FS_SECURITY=y
###
# Virtio devices
###
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_VIRTIO_INPUT=y
CONFIG_VIRTIO_CONSOLE=y
CONFIG_VIRTIO_MMIO=y
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y
CONFIG_HW_RANDOM_VIRTIO=m
###
# Console / Input
###
CONFIG_INPUT_EVDEV=y
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
###
# Kernel features required by systemd
###
CONFIG_FHANDLE=y
CONFIG_INOTIFY_USER=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EPOLL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
CONFIG_UNWINDER_FRAME_POINTER=y
###
# Security
###
CONFIG_SECCOMP=y
CONFIG_SECCOMP_FILTER=y
CONFIG_SECURITY=y
CONFIG_SECURITYFS=y
###
# EFI
###
CONFIG_EFI=y
CONFIG_EFI_STUB=y
###
# AMD SEV-SNP
###
CONFIG_AMD_MEM_ENCRYPT=y
CONFIG_AMD_MEM_ENCRYPT_ACTIVE_BY_DEFAULT=n
CONFIG_SEV_GUEST=y
CONFIG_IOMMU_DEFAULT_PASSTHROUGH=n
###
# Intel TDX
###
CONFIG_X86_X2APIC=y
CONFIG_X86_CPUID=y
CONFIG_X86_SGX=y
CONFIG_X86_SGX_KVM=y
CONFIG_INTEL_TDX_GUEST=y
CONFIG_TDX_GUEST_DRIVER=y
###
# Preemption (disabled for VM performance)
###
CONFIG_PREEMPT_COUNT=n
CONFIG_PREEMPT=n
CONFIG_PREEMPT_DYNAMIC=n
CONFIG_DEBUG_PREEMPT=n
###
# Key/signature management
###
CONFIG_SYSTEM_TRUSTED_KEYS=n
CONFIG_SYSTEM_REVOCATION_KEYS=n
CONFIG_MODULE_SIG_KEY=n
CONFIG_KEYS=y
CONFIG_ENCRYPTED_KEYS=y
###
# Crypto — AES-GCM (LUKS2 cipher) + SHA-256 (dm-verity hash)
###
CONFIG_CRYPTO_AES=y
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_GCM=y
CONFIG_CRYPTO_GHASH=y
CONFIG_CRYPTO_SEQIV=y
CONFIG_CRYPTO_ECHAINIV=y
CONFIG_CRYPTO_XTS=y
CONFIG_CRYPTO_CBC=y
CONFIG_CRYPTO_AUTHENC=y
CONFIG_CRYPTO_ESSIV=y
CONFIG_CRYPTO_USER_API=y
CONFIG_CRYPTO_USER_API_HASH=y
CONFIG_CRYPTO_USER_API_SKCIPHER=y
CONFIG_CRYPTO_USER_API_AEAD=y
CONFIG_CRYPTO_AES_NI_INTEL=m
CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL=m
###
# TPM
###
CONFIG_TCG_TPM=y
CONFIG_TCG_TPM2_HMAC=y
CONFIG_TCG_PLATFORM=y
###
# IMA (Linux Integrity Measurement Architecture)
###
CONFIG_INTEGRITY=y
CONFIG_INTEGRITY_SIGNATURE=y
CONFIG_IMA=y
CONFIG_IMA_MEASURE_PCR_IDX=10
CONFIG_IMA_LSM_RULES=y
CONFIG_IMA_APPRAISE=y
CONFIG_IMA_DEFAULT_TEMPLATE="ima-ng"
CONFIG_IMA_DEFAULT_HASH="sha256"
###
# Disabled options
###
CONFIG_KSM=n
CONFIG_EISA=n
+11
View File
@@ -0,0 +1,11 @@
#!/bin/sh
set -u
set -e
# Add a console on tty1
if [ -e ${TARGET_DIR}/etc/inittab ]; then
grep -qE '^tty1::' ${TARGET_DIR}/etc/inittab || \
sed -i '/GENERIC_SERIAL/a\
tty1::respawn:/sbin/getty -L tty1 0 vt100 # QEMU graphical window' ${TARGET_DIR}/etc/inittab
fi
+273
View File
@@ -0,0 +1,273 @@
#!/bin/bash
COCOS_BOARD_DIR="$(dirname "$0")"
DEFCONFIG_NAME="$(basename "$2")"
README_FILES="${COCOS_BOARD_DIR}/readme.txt"
START_QEMU_SCRIPT="${BINARIES_DIR}/start-qemu.sh"
# ---------------------------------------------------------------------------
# Build a minimal FDE initramfs (rootfs.cpio.gz) containing only the tools
# needed to mount the root partition read-only, provision LUKS2, and switch_root.
# All other packages live
# on the ext4 disk image and are available after switch_root.
# ---------------------------------------------------------------------------
echo "[post-image] Building minimal FDE initramfs..."
INITRAMFS_STAGE="${BUILD_DIR}/initramfs-staging"
rm -rf "${INITRAMFS_STAGE}"
# Merged-usr layout: bin/sbin/lib/lib64 are symlinks into usr/, matching the
# Buildroot target layout so that hardcoded ELF interpreter paths (ld-linux)
# and the #!/bin/sh shebang both resolve correctly inside the initramfs.
mkdir -p "${INITRAMFS_STAGE}/usr/bin" \
"${INITRAMFS_STAGE}/usr/sbin" \
"${INITRAMFS_STAGE}/usr/lib" \
"${INITRAMFS_STAGE}/dev" \
"${INITRAMFS_STAGE}/proc" \
"${INITRAMFS_STAGE}/sys" \
"${INITRAMFS_STAGE}/tmp" \
"${INITRAMFS_STAGE}/run" \
"${INITRAMFS_STAGE}/root" \
"${INITRAMFS_STAGE}/etc/udev/rules.d"
ln -s usr/bin "${INITRAMFS_STAGE}/bin"
ln -s usr/sbin "${INITRAMFS_STAGE}/sbin"
ln -s usr/lib "${INITRAMFS_STAGE}/lib"
ln -s usr/lib "${INITRAMFS_STAGE}/lib64"
# init script (PID 1)
install -m 0755 "${BR2_EXTERNAL_COCOS_PATH}/board/rootfs-overlay/init" \
"${INITRAMFS_STAGE}/init"
# Binaries required by the init script
FDE_BINS="
bash
cryptsetup
veritysetup
mkfs.ext4
mount
umount
losetup
switch_root
dd
shred
tr
cut
grep
awk
cat
ls
cp
mkdir
readlink
dirname
lsblk
udevadm
blkid
rm
"
for BIN in ${FDE_BINS}; do
SRC="$(find "${TARGET_DIR}/usr/bin" "${TARGET_DIR}/usr/sbin" \
"${TARGET_DIR}/bin" "${TARGET_DIR}/sbin" \
-name "${BIN}" \( -type f -o -type l \) 2>/dev/null | head -1)"
if [ -n "${SRC}" ]; then
cp -P "${SRC}" "${INITRAMFS_STAGE}/usr/bin/${BIN}"
chmod 0755 "${INITRAMFS_STAGE}/usr/bin/${BIN}" 2>/dev/null || true
# If this is a symlink, also copy the resolved target binary (e.g. busybox, coreutils, mke2fs)
# so that other applet symlinks pointing to the same target also work at runtime.
if [ -L "${SRC}" ]; then
REAL_SRC="$(readlink -f "${SRC}")"
REAL_NAME="$(basename "${REAL_SRC}")"
if [ -f "${REAL_SRC}" ] && [ ! -e "${INITRAMFS_STAGE}/usr/bin/${REAL_NAME}" ]; then
cp "${REAL_SRC}" "${INITRAMFS_STAGE}/usr/bin/${REAL_NAME}"
chmod 0755 "${INITRAMFS_STAGE}/usr/bin/${REAL_NAME}" 2>/dev/null || true
fi
fi
else
echo "[post-image] WARNING: ${BIN} not found in target, skipping"
fi
done
# sh symlink so #!/bin/sh in the init script resolves correctly
ln -sf bash "${INITRAMFS_STAGE}/usr/bin/sh"
# Shared libraries from usr/lib (TARGET_DIR uses merged-usr so lib → usr/lib)
# Skip large runtimes that are only needed on the real root.
find "${TARGET_DIR}/usr/lib" \( \
-path "*/python3*" -o \
-path "*/gcc*" -o \
-path "*/wasmedge*" \
\) -prune -o \
\( -name "*.so" -o -name "*.so.*" \) -print | while read -r LIB; do
REL="${LIB#${TARGET_DIR}/usr/lib/}"
DEST="${INITRAMFS_STAGE}/usr/lib/${REL}"
mkdir -p "$(dirname "${DEST}")"
cp -P "${LIB}" "${DEST}"
done
# udev rules (needed for udevadm settle)
if [ -d "${TARGET_DIR}/etc/udev" ]; then
cp -a "${TARGET_DIR}/etc/udev/." "${INITRAMFS_STAGE}/etc/udev/"
fi
# /dev seed nodes
mknod -m 0600 "${INITRAMFS_STAGE}/dev/console" c 5 1 2>/dev/null || true
mknod -m 0666 "${INITRAMFS_STAGE}/dev/null" c 1 3 2>/dev/null || true
echo "[post-image] Packing initramfs..."
( cd "${INITRAMFS_STAGE}" && \
find . | cpio --quiet -o -H newc -R 0:0 | gzip -9 \
> "${BINARIES_DIR}/rootfs.cpio.gz" )
echo "[post-image] rootfs.cpio.gz: $(du -sh "${BINARIES_DIR}/rootfs.cpio.gz" | cut -f1)"
ROOTFS_IMAGE="${BINARIES_DIR}/rootfs.ext4"
VERITY_IMAGE="${BINARIES_DIR}/rootfs.verity"
ROOT_HASH_FILE="${BINARIES_DIR}/rootfs.roothash"
VERITYSETUP_BIN="${HOST_DIR}/bin/veritysetup"
if [ ! -x "${VERITYSETUP_BIN}" ]; then
VERITYSETUP_BIN="${HOST_DIR}/sbin/veritysetup"
fi
if [ ! -x "${VERITYSETUP_BIN}" ]; then
echo "[post-image] FATAL: host veritysetup not found at ${VERITYSETUP_BIN}"
exit 1
fi
echo "[post-image] Building dm-verity hash image..."
rm -f "${VERITY_IMAGE}" "${ROOT_HASH_FILE}"
truncate -s 256M "${VERITY_IMAGE}"
VERITY_FORMAT_OUTPUT="$("${VERITYSETUP_BIN}" format "${ROOTFS_IMAGE}" "${VERITY_IMAGE}")" || {
echo "[post-image] FATAL: veritysetup format failed"
exit 1
}
ROOT_HASH="$(printf '%s\n' "${VERITY_FORMAT_OUTPUT}" | awk -F': ' '/^Root hash:/ {print $2}' | tr -d '[:space:]')"
if [ -z "${ROOT_HASH}" ]; then
echo "[post-image] FATAL: failed to parse dm-verity root hash"
printf '%s\n' "${VERITY_FORMAT_OUTPUT}"
exit 1
fi
printf '%s\n' "${ROOT_HASH}" > "${ROOT_HASH_FILE}"
echo "[post-image] dm-verity root hash: ${ROOT_HASH}"
# Stage kernel and initramfs for the EFI partition.
# Buildroot's GRUB2 package has already placed bootx64.efi at
# ${BINARIES_DIR}/efi-part/EFI/BOOT/bootx64.efi; we add the kernel,
# initramfs, and overwrite the default grub.cfg with our boot entry.
echo "[post-image] Staging EFI partition files..."
mkdir -p "${BINARIES_DIR}/efi-part/EFI/BOOT"
cp "${BINARIES_DIR}/bzImage" "${BINARIES_DIR}/efi-part/bzImage"
cp "${BINARIES_DIR}/rootfs.cpio.gz" "${BINARIES_DIR}/efi-part/initrd.cpio.gz"
cat > "${BINARIES_DIR}/efi-part/EFI/BOOT/grub.cfg" << GRUBCFG
set default=0
set timeout=0
menuentry "Cocos" {
linux /bzImage console=ttyS0 roothash=${ROOT_HASH} systemd.verity=0 systemd.gpt_auto=0
initrd /initrd.cpio.gz
}
GRUBCFG
# Regenerate bootx64.efi with --disable-shim-lock so GRUB can load the kernel
# directly without requiring the shim bootloader (OVMF still verifies GRUB via
# Secure Boot; shim is not needed when booting from a custom OVMF with own DB key).
GRUB_CORE="$(ls -d "${BUILD_DIR}"/grub2-*/build-x86_64-efi/grub-core 2>/dev/null | head -1)"
if [ -n "${GRUB_CORE}" ]; then
echo "[post-image] Regenerating bootx64.efi with --disable-shim-lock..."
"${HOST_DIR}/bin/grub-mkimage" \
-d "${GRUB_CORE}" \
-O x86_64-efi \
-o "${BINARIES_DIR}/efi-part/EFI/BOOT/bootx64.efi" \
-p "/EFI/BOOT" \
--disable-shim-lock \
boot linux echo normal part_gpt fat ls search || {
echo "[post-image] FATAL: grub-mkimage failed"
exit 1
}
else
echo "[post-image] WARNING: GRUB core dir not found, skipping --disable-shim-lock rebuild"
fi
# Sign GRUB and kernel for UEFI Secure Boot.
# Keys are resolved in order: env var → board/secure-boot/ defaults.
SB_KEY="${SB_KEY:-${COCOS_BOARD_DIR}/secure-boot/db.key}"
SB_CERT="${SB_CERT:-${COCOS_BOARD_DIR}/secure-boot/db.crt}"
if [ -f "${SB_KEY}" ] && [ -f "${SB_CERT}" ]; then
echo "[post-image] Signing EFI binaries for Secure Boot..."
sbsign --key "${SB_KEY}" --cert "${SB_CERT}" \
--output "${BINARIES_DIR}/efi-part/EFI/BOOT/bootx64.efi" \
"${BINARIES_DIR}/efi-part/EFI/BOOT/bootx64.efi" || {
echo "[post-image] FATAL: Failed to sign bootx64.efi"
exit 1
}
sbsign --key "${SB_KEY}" --cert "${SB_CERT}" \
--output "${BINARIES_DIR}/efi-part/bzImage" \
"${BINARIES_DIR}/efi-part/bzImage" || {
echo "[post-image] FATAL: Failed to sign bzImage"
exit 1
}
echo "[post-image] Secure Boot signing complete"
else
echo "[post-image] WARNING: Secure Boot keys not found — EFI binaries are unsigned"
echo "[post-image] Default location: ${COCOS_BOARD_DIR}/secure-boot/db.key + db.crt"
echo "[post-image] Override: SB_KEY=/path/to/db.key SB_CERT=/path/to/db.crt make"
fi
GENIMAGE_CFG="${COCOS_BOARD_DIR}/genimage.cfg"
if [ -f "${GENIMAGE_CFG}" ]; then
GENIMAGE_TMP="${BUILD_DIR}/genimage.tmp"
rm -rf "${GENIMAGE_TMP}"
genimage \
--rootpath "${TARGET_DIR}" \
--tmppath "${GENIMAGE_TMP}" \
--inputpath "${BINARIES_DIR}" \
--outputpath "${BINARIES_DIR}" \
--config "${GENIMAGE_CFG}"
fi
if [[ "${DEFCONFIG_NAME}" =~ ^"cocos_*" ]]; then
# Not a Qemu defconfig, can't test.
exit 0
fi
# Search for "# qemu_*_defconfig" tag in all readme.txt files.
# Qemu command line on multilines using back slash are accepted.
# shellcheck disable=SC2086 # glob over each readme file
QEMU_CMD_LINE="$(sed -r ':a; /\\$/N; s/\\\n//; s/\t/ /; ta; /# '"${DEFCONFIG_NAME}"'$/!d; s/#.*//' ${README_FILES})"
if [ -z "${QEMU_CMD_LINE}" ]; then
# No Qemu cmd line found, can't test.
exit 0
fi
# Remove output/images path since the script will be in
# the same directory as the kernel and the rootfs images.
QEMU_CMD_LINE="${QEMU_CMD_LINE//output\/images\//}"
# Remove -serial stdio if present, keep it as default args
DEFAULT_ARGS="$(sed -r -e '/-serial stdio/!d; s/.*(-serial stdio).*/\1/' <<<"${QEMU_CMD_LINE}")"
QEMU_CMD_LINE="${QEMU_CMD_LINE//-serial stdio/}"
# Remove any string before qemu-system-*
QEMU_CMD_LINE="$(sed -r -e 's/^.*(qemu-system-)/\1/' <<<"${QEMU_CMD_LINE}")"
# Disable graphical output and redirect serial I/Os to console
case ${DEFCONFIG_NAME} in
(qemu_sh4eb_r2d_defconfig|qemu_sh4_r2d_defconfig)
# Special case for SH4
SERIAL_ARGS="-serial stdio -display none"
;;
(*)
SERIAL_ARGS="-nographic"
;;
esac
sed -e "s|@SERIAL_ARGS@|${SERIAL_ARGS}|g" \
-e "s|@DEFAULT_ARGS@|${DEFAULT_ARGS}|g" \
-e "s|@QEMU_CMD_LINE@|${QEMU_CMD_LINE}|g" \
-e "s|@HOST_DIR@|${HOST_DIR}|g" \
<"${COCOS_BOARD_DIR}/start-qemu.sh.in" \
>"${START_QEMU_SCRIPT}"
chmod +x "${START_QEMU_SCRIPT}"
+7
View File
@@ -0,0 +1,7 @@
Run the emulation with:
qemu-system-x86_64 -M pc -kernel output/images/bzImage -drive file=output/images/rootfs.ext2,if=virtio,format=raw -append "rootwait root=/dev/vda console=tty1 console=ttyS0" -serial stdio -net nic,model=virtio -net user # cocos_defconfig
Optionally add -smp N to emulate a SMP system with N CPUs.
The login prompt will appear in the graphical window.
@@ -0,0 +1,3 @@
# Private key must not be committed
db.key
db.crt
+28
View File
@@ -0,0 +1,28 @@
#!/bin/sh
BINARIES_DIR="${0%/*}/"
# shellcheck disable=SC2164
cd "${BINARIES_DIR}"
mode_serial=false
mode_sys_qemu=false
while [ "$1" ]; do
case "$1" in
--serial-only|serial-only) mode_serial=true; shift;;
--use-system-qemu) mode_sys_qemu=true; shift;;
--) shift; break;;
*) echo "unknown option: $1" >&2; exit 1;;
esac
done
if ${mode_serial}; then
EXTRA_ARGS='@SERIAL_ARGS@'
else
EXTRA_ARGS='@DEFAULT_ARGS@'
fi
if ! ${mode_sys_qemu}; then
export PATH="@HOST_DIR@/bin:${PATH}"
fi
exec @QEMU_CMD_LINE@ ${EXTRA_ARGS} "$@"
+7
View File
@@ -0,0 +1,7 @@
# Root is mounted read-only by the initramfs through dm-verity.
# /cocos, /var, /tmp, and bind mounts are set up by the initramfs init script.
/dev/mapper/root_verity / ext4 ro,defaults 0 0
# 9P virtio shares — provided by the hypervisor, optional (nofail)
certs_share /etc/certs 9p trans=virtio,version=9p2000.L,cache=mmap,nofail 0 0
env_share /etc/cocos 9p trans=virtio,version=9p2000.L,cache=mmap,nofail 0 0
@@ -0,0 +1,7 @@
{
"key-providers": {
"attestation-agent": {
"grpc": "127.0.0.1:50011"
}
}
}
+265
View File
@@ -0,0 +1,265 @@
#!/bin/sh
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
if (exec 0</dev/console) 2>/dev/null; then
exec 0</dev/console
exec 1>/dev/console
exec 2>/dev/console
fi
echo "Welcome to the Cocos FDE test VM initramfs!"
echo "This is a minimal initramfs environment used for testing the FDE provisioning flow."
echo "If you see this message, the initramfs was loaded and executed successfully."
echo "The initramfs will now attempt to provision the disk and mount the real root filesystem."
echo "If any step fails, it will drop to a shell for debugging."
[ -d /dev ] || mkdir -m 0755 /dev
[ -d /etc ] || mkdir -m 0755 /etc
[ -d /root ] || mkdir -m 0700 /root
[ -d /run ] || mkdir -m 0755 /run
[ -d /sys ] || mkdir /sys
[ -d /proc ] || mkdir /proc
[ -d /tmp ] || mkdir /tmp
if [ -L /etc/resolv.conf ]; then
RESOLV_TARGET="$(readlink /etc/resolv.conf)"
case "$RESOLV_TARGET" in
/*)
RESOLV_PATH="$RESOLV_TARGET"
;;
*)
RESOLV_PATH="/etc/$RESOLV_TARGET"
;;
esac
mkdir -p "$(dirname "$RESOLV_PATH")"
[ -e "$RESOLV_PATH" ] || : > "$RESOLV_PATH"
else
[ -e /etc/resolv.conf ] || : > /etc/resolv.conf
fi
mount -t sysfs -o nodev,noexec,nosuid sysfs /sys
mount -t proc -o nodev,noexec,nosuid proc /proc
mkdir -p /sys/kernel/config
if ! grep -q ' /sys/kernel/config ' /proc/mounts; then
mount -t configfs configfs /sys/kernel/config 2>/dev/null || true
fi
mount -t devtmpfs -o nosuid,mode=0755 udev /dev
mkdir /dev/pts
mount -t devpts -o noexec,nosuid,gid=5,mode=0620 devpts /dev/pts || true
MNT_DIR=/root
BASE=$(pwd)
DST=/dev/sda
ROOTFS_TYPE="ext4"
ROOT_VERITY_MAP=root_verity
ROOT_VERITY_MAPPER="/dev/mapper/$ROOT_VERITY_MAP"
COCOS_MOUNT=/cocos
COCOS_MAP=cocos_crypt
COCOS_MAPPER="/dev/mapper/$COCOS_MAP"
LUKS_PARAMS="--cipher aes-gcm-random --integrity aead"
settle_devices() {
echo "[init] Waiting for devices to settle..."
if command -v udevadm >/dev/null 2>&1; then
udevadm settle --timeout=10 || sleep 2
else
sleep 2
fi
}
wipe_file() {
file_path="$1"
if [ -z "$file_path" ] || [ ! -e "$file_path" ]; then
return 0
fi
shred -vfz -n 3 "$file_path" 2>/dev/null || dd if=/dev/zero of="$file_path" bs=64 count=1
rm -f "$file_path"
}
partition_path() {
disk="$1"
partition="$2"
case "$disk" in
*[0-9])
printf '%sp%s\n' "$disk" "$partition"
;;
*)
printf '%s%s\n' "$disk" "$partition"
;;
esac
}
append_9p_entry() {
pattern="$1"
default_entry="$2"
existing_entry=""
if [ -f "$FSTAB_BAK" ]; then
existing_entry="$(grep -E "$pattern" "$FSTAB_BAK" | head -n 1 || true)"
fi
if [ -n "$existing_entry" ]; then
printf '%s\n' "$existing_entry" >> "$FSTAB"
else
printf '%s\n' "$default_entry" >> "$FSTAB"
fi
}
cmdline_arg() {
key="$1"
for arg in $(cat /proc/cmdline); do
case "$arg" in
"$key="*)
printf '%s\n' "${arg#*=}"
return 0
;;
esac
done
return 1
}
echo "[init] Starting disk provisioning..."
ROOT_PART="$(partition_path "$DST" 2)"
VERITY_PART="$(partition_path "$DST" 3)"
COCOS_PART="$(partition_path "$DST" 4)"
ROOT_HASH="$(cmdline_arg roothash)"
if [ -z "$ROOT_HASH" ]; then
echo "[init] FATAL: Missing roothash= on kernel command line"
exec /bin/sh
fi
settle_devices
for part in "$ROOT_PART" "$VERITY_PART" "$COCOS_PART"; do
if [ ! -b "$part" ]; then
echo "[init] FATAL: Could not find partition $part"
echo "[init] Available block devices:"
lsblk || ls -la /dev/ || true
echo "[init] Dropping to shell."
exec /bin/sh
fi
done
echo "[init] Opening dm-verity root mapping..."
veritysetup open "$ROOT_PART" "$ROOT_VERITY_MAP" "$VERITY_PART" "$ROOT_HASH" || {
echo "[init] FATAL: Failed to open dm-verity mapping for root"
exec /bin/sh
}
echo "[init] Mounting root at $MNT_DIR (read-only)..."
mount -o ro -t "$ROOTFS_TYPE" "$ROOT_VERITY_MAPPER" "$MNT_DIR" || {
echo "[init] FATAL: Failed to mount verity root"
veritysetup close "$ROOT_VERITY_MAP" 2>/dev/null || true
exec /bin/sh
}
echo "[init] Generating ephemeral key for $COCOS_MOUNT..."
dd if=/dev/urandom of=kk.bin bs=64 count=1 || {
echo "[init] FATAL: Failed to generate encryption key"
umount "$MNT_DIR" 2>/dev/null || true
exec /bin/sh
}
KK_BIN=$BASE/kk.bin
cryptsetup luksFormat "$COCOS_PART" --type luks2 $LUKS_PARAMS --key-file="$KK_BIN" -q || {
echo "[init] FATAL: LUKS format failed"
umount "$MNT_DIR" 2>/dev/null || true
veritysetup close "$ROOT_VERITY_MAP" 2>/dev/null || true
wipe_file "$KK_BIN"
exec /bin/sh
}
cryptsetup open "$COCOS_PART" "$COCOS_MAP" --key-file="$KK_BIN" || {
echo "[init] FATAL: Failed to open LUKS container for $COCOS_MOUNT"
umount "$MNT_DIR" 2>/dev/null || true
veritysetup close "$ROOT_VERITY_MAP" 2>/dev/null || true
wipe_file "$KK_BIN"
exec /bin/sh
}
mkfs.ext4 -F -m 0 "$COCOS_MAPPER" >/dev/null || {
echo "[init] FATAL: Failed to create ext4 filesystem for $COCOS_MOUNT"
cryptsetup close "$COCOS_MAP" 2>/dev/null || true
umount "$MNT_DIR" 2>/dev/null || true
veritysetup close "$ROOT_VERITY_MAP" 2>/dev/null || true
wipe_file "$KK_BIN"
exec /bin/sh
}
echo "[init] Mounting encrypted $COCOS_MOUNT..."
mkdir -p "$MNT_DIR$COCOS_MOUNT"
mount -t ext4 "$COCOS_MAPPER" "$MNT_DIR$COCOS_MOUNT" || {
echo "[init] FATAL: Failed to mount encrypted $COCOS_MOUNT filesystem"
cryptsetup close "$COCOS_MAP" 2>/dev/null || true
umount "$MNT_DIR" 2>/dev/null || true
veritysetup close "$ROOT_VERITY_MAP" 2>/dev/null || true
wipe_file "$KK_BIN"
exec /bin/sh
}
mkdir -p \
"$MNT_DIR$COCOS_MOUNT/.cache/oci" \
"$MNT_DIR$COCOS_MOUNT/datasets" \
"$MNT_DIR$COCOS_MOUNT/docker" \
"$MNT_DIR$COCOS_MOUNT/cocos_init"
# The root is read-only; provide tmpfs for writable system directories.
mount -t tmpfs tmpfs "$MNT_DIR/tmp"
mount -t tmpfs -o mode=0755 tmpfs "$MNT_DIR/var"
# Bind Docker's data root onto /cocos so large images don't exhaust RAM.
mkdir -p "$MNT_DIR/var/lib/docker"
mount --bind "$MNT_DIR$COCOS_MOUNT/docker" "$MNT_DIR/var/lib/docker"
# /cocos_init is on the read-only root; shadow it with a writable
# copy on /cocos so agent setup scripts can write state alongside the scripts.
if [ -d "$MNT_DIR/cocos_init" ]; then
cp -a "$MNT_DIR/cocos_init/." "$MNT_DIR$COCOS_MOUNT/cocos_init/" 2>/dev/null || true
mount --bind "$MNT_DIR$COCOS_MOUNT/cocos_init" "$MNT_DIR/cocos_init" || true
fi
mount --move /proc $MNT_DIR/proc
mount --move /sys $MNT_DIR/sys
FSTAB="$MNT_DIR/etc/fstab"
FSTAB_BAK="$MNT_DIR/etc/fstab.bak"
mkdir -p "$MNT_DIR/etc/certs" "$MNT_DIR/etc/cocos" 2>/dev/null || true
if [ -f "$FSTAB" ]; then
mv "$FSTAB" "$FSTAB_BAK"
fi
cat > "$FSTAB" << EOF
# Generated by init script
$ROOT_VERITY_MAPPER / $ROOTFS_TYPE ro,defaults 0 0
EOF
append_9p_entry \
'^certs_share[[:space:]]+/etc/certs[[:space:]]+9p([[:space:]]|$)' \
'certs_share /etc/certs 9p trans=virtio,version=9p2000.L,cache=mmap,nofail 0 0'
append_9p_entry \
'^env_share[[:space:]]+/etc/cocos[[:space:]]+9p([[:space:]]|$)' \
'env_share /etc/cocos 9p trans=virtio,version=9p2000.L,cache=mmap,nofail 0 0'
printf '%s\n' '# /cocos is mounted by the FDE initramfs using an ephemeral LUKS key.' >> "$FSTAB"
# Securely wipe the encryption key before switching root.
echo "[init] Securely wiping the $COCOS_MOUNT encryption key..."
wipe_file "$KK_BIN"
echo "[init] Switching to real root..."
exec switch_root $MNT_DIR/ /sbin/init
# If switch_root somehow returns:
echo "[init] switch_root failed, dropping to shell"
exec /bin/sh
@@ -0,0 +1,3 @@
[Unit]
RequiresMountsFor=/etc/cocos /etc/certs
@@ -0,0 +1,3 @@
[Unit]
RequiresMountsFor=/etc/cocos
@@ -0,0 +1,3 @@
[Unit]
RequiresMountsFor=/etc/cocos
@@ -0,0 +1,3 @@
[Unit]
RequiresMountsFor=/etc/cocos
@@ -0,0 +1,2 @@
d /var/log/cocos 0755 root root -
d /run/cocos 0755 root root -
+117
View File
@@ -0,0 +1,117 @@
# Architecture
BR2_x86_64=y
# System
BR2_TARGET_GENERIC_HOSTNAME="cocos"
BR2_TARGET_GENERIC_ISSUE="Welcome to Cocos"
BR2_PACKAGE_DHCP=y
BR2_PACKAGE_DHCP_CLIENT=y
BR2_INIT_SYSTEMD=y
BR2_SYSTEM_BIN_SH_BASH=y
# Filesystem
# BR2_TARGET_ROOTFS_TAR is not set
# Initramfs (rootfs.cpio.gz) is built by post-image.sh from only the FDE tools,
# not from the full target rootfs. The full rootfs goes to rootfs.ext4 (disk image).
BR2_ROOTFS_OVERLAY="$(BR2_EXTERNAL_COCOS_PATH)/board/rootfs-overlay"
# Patches for existing Buildroot packages
BR2_GLOBAL_PATCH_DIR="$(BR2_EXTERNAL_COCOS_PATH)/patches"
# Bootloader
BR2_TARGET_GRUB2=y
BR2_TARGET_GRUB2_X86_64_EFI=y
BR2_TARGET_GRUB2_BUILTIN_MODULES_EFI="boot linux echo normal part_gpt fat ls search"
# Disk image
BR2_TARGET_ROOTFS_EXT2=y
BR2_TARGET_ROOTFS_EXT2_4=y
BR2_TARGET_ROOTFS_EXT2_SIZE="10G"
BR2_PACKAGE_HOST_GENIMAGE=y
BR2_PACKAGE_HOST_CRYPTSETUP=y
# Image
BR2_ROOTFS_POST_BUILD_SCRIPT="$(BR2_EXTERNAL_COCOS_PATH)/board/cocos/post-build.sh"
# Image
BR2_ROOTFS_POST_IMAGE_SCRIPT="$(BR2_EXTERNAL_COCOS_PATH)/board/cocos/post-image.sh"
BR2_ROOTFS_POST_SCRIPT_ARGS="$(BR2_DEFCONFIG)"
# Linux headers same as kernel
BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_6_11=y
BR2_TOOLCHAIN_HEADERS_LATEST=y
BR2_TOOLCHAIN_HEADERS_AT_LEAST="6.11-rc7"
# Kernel
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_GIT=y
BR2_LINUX_KERNEL_CUSTOM_REPO_URL="https://github.com/coconut-svsm/linux.git"
BR2_LINUX_KERNEL_CUSTOM_REPO_VERSION="svsm"
BR2_LINUX_KERNEL_VERSION="svsm"
BR2_LINUX_KERNEL_PATCH=""
BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="$(BR2_EXTERNAL_COCOS_PATH)/board/cocos/linux.config"
BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF=y
# host-qemu for gitlab testing
BR2_PACKAGE_HOST_QEMU=y
BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
# Python
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON_PIP=y
BR2_PACKAGE_BZIP2=y
BR2_PACKAGE_XZ=y
BR2_PACKAGE_ZIP=y
BR2_PACKAGE_PYTHON3_ZLIB=y
BR2_PACKAGE_PYTHON3_XZ=y
BR2_PACKAGE_PYTHON3_BZIP2=y
BR2_INSTALL_LIBSTDCPP=y
BR2_TOOLCHAIN_BUILDROOT_CXX=y
BR2_PACKAGE_HOST_GCC_TARGET=y
BR2_TOOLCHAIN_BUILDROOT_LIBSTDCPP=y
BR2_PACKAGE_GCC=y
BR2_PACKAGE_GCC_TARGET=y
BR2_PACKAGE_LIBSTDCPP=y
# FDE
BR2_PACKAGE_NBD=y
BR2_PACKAGE_NBD_CLIENT=y
BR2_PACKAGE_CRYPTSETUP=y
BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV=y
BR2_PACKAGE_EUDEV=y
BR2_PACKAGE_HAS_UDEV=y
BR2_PACKAGE_MULTIPATH_TOOLS=y
BR2_PACKAGE_UTIL_LINUX_BINARIES=y
BR2_PACKAGE_E2FSPROGS=y
BR2_LINUX_KERNEL_NEEDS_HOST_PAHOLE=y
# TPM2
BR2_PACKAGE_TPM2_TOOLS=y
BR2_PACKAGE_COREUTILS=y
# Docker
BR2_PACKAGE_LIBSECCOMP_ARCH_SUPPORTS=y
BR2_PACKAGE_LIBSECCOMP=y
BR2_PACKAGE_CA_CERTIFICATES=y
BR2_PACKAGE_DOCKER_CLI=y
BR2_PACKAGE_DOCKER_COMPOSE=y
BR2_PACKAGE_DOCKER_ENGINE=y
BR2_PACKAGE_CONTAINERD=y
BR2_PACKAGE_RUNC=y
BR2_PACKAGE_IPTABLES=y
# Skopeo for OCI image handling with CoCo Keyprovider
BR2_PACKAGE_SKOPEO=y
BR2_PACKAGE_GPGME=y
BR2_PACKAGE_LVM2=y
BR2_PACKAGE_LVM2_STANDARD_INSTALL=y
BR2_PACKAGE_9PFS=y
# Host tools
BR2_PACKAGE_HOST_RUSTC=y
BR2_PACKAGE_HOST_RUST_BIN=y
# Cocos AI Packages
BR2_PACKAGE_AGENT=y
# BR2_PACKAGE_CC_ATTESTATION_AGENT is not set
+2
View File
@@ -0,0 +1,2 @@
name: COCOS
desc: External buildroot tree for Cocos AI
+1
View File
@@ -0,0 +1 @@
include $(sort $(wildcard $(BR2_EXTERNAL_COCOS_PATH)/package/*/*.mk))
+13
View File
@@ -0,0 +1,13 @@
config BR2_PACKAGE_AGENT
bool "agent"
default y
select BR2_PACKAGE_ATTESTATION_SERVICE
select BR2_PACKAGE_LOG_FORWARDER
select BR2_PACKAGE_COMPUTATION_RUNNER
select BR2_PACKAGE_INGRESS_PROXY
select BR2_PACKAGE_EGRESS_PROXY
help
Confidential Computing Agent is a state machine capable of
receiving datasets and algorithm, running computations, and
fetching the attestation report from within the
Confidential VM.
+27
View File
@@ -0,0 +1,27 @@
################################################################################
#
# Cocos AI Agent
#
################################################################################
AGENT_VERSION = main
AGENT_SITE = $(call github,ultravioletrs,cocos,$(AGENT_VERSION))
define AGENT_BUILD_CMDS
$(MAKE) -C $(@D) agent EMBED_ENABLED=$(AGENT_EMBED_ENABLED)
endef
define AGENT_INSTALL_TARGET_CMDS
mkdir -p $(TARGET_DIR)/cocos/
mkdir -p $(TARGET_DIR)/var/log/cocos
mkdir -p $(TARGET_DIR)/cocos_init/
$(INSTALL) -D -m 0750 $(@D)/build/cocos-agent $(TARGET_DIR)/bin
endef
define AGENT_INSTALL_INIT_SYSTEMD
$(INSTALL) -D -m 0640 $(@D)/init/systemd/cocos-agent.service $(TARGET_DIR)/usr/lib/systemd/system/cocos-agent.service
$(INSTALL) -D -m 0750 $(@D)/init/systemd/agent_setup.sh $(TARGET_DIR)/cocos_init/agent_setup.sh
$(INSTALL) -D -m 0750 $(@D)/init/systemd/agent_start_script.sh $(TARGET_DIR)/cocos_init/agent_start_script.sh
endef
$(eval $(generic-package))
@@ -0,0 +1,11 @@
config BR2_PACKAGE_ATTESTATION_SERVICE
bool
default y
help
Cocos AI attestation service that generates EAT tokens
for TEE attestation (SNP, TDX, vTPM, Azure).
This service can optionally use the Confidential Containers
attestation-agent as a backend provider via gRPC.
https://github.com/ultravioletrs/cocos
@@ -0,0 +1,34 @@
################################################################################
#
# attestation-service
#
################################################################################
ATTESTATION_SERVICE_VERSION = main
ATTESTATION_SERVICE_SITE = $(call github,ultravioletrs,cocos,$(ATTESTATION_SERVICE_VERSION))
define ATTESTATION_SERVICE_BUILD_CMDS
$(MAKE) -C $(@D) attestation-service
endef
define ATTESTATION_SERVICE_INSTALL_TARGET_CMDS
$(INSTALL) -D -m 0755 $(@D)/build/cocos-attestation-service $(TARGET_DIR)/usr/bin/attestation-service
endef
ifeq ($(BR2_PACKAGE_CC_ATTESTATION_AGENT),y)
define ATTESTATION_SERVICE_INSTALL_INIT_SYSTEMD
$(INSTALL) -D -m 0640 $(@D)/init/systemd/attestation-service.service $(TARGET_DIR)/usr/lib/systemd/system/attestation-service.service
$(INSTALL) -D -m 0750 $(@D)/init/systemd/attestation_setup.sh $(TARGET_DIR)/cocos_init/attestation_setup.sh
# CC attestation agent is already enabled by default
endef
else
define ATTESTATION_SERVICE_INSTALL_INIT_SYSTEMD
$(INSTALL) -D -m 0640 $(@D)/init/systemd/attestation-service.service $(TARGET_DIR)/usr/lib/systemd/system/attestation-service.service
$(INSTALL) -D -m 0750 $(@D)/init/systemd/attestation_setup.sh $(TARGET_DIR)/cocos_init/attestation_setup.sh
# Disable CC attestation agent backend if not selected
sed -i 's/USE_CC_ATTESTATION_AGENT=true/USE_CC_ATTESTATION_AGENT=false/' $(TARGET_DIR)/usr/lib/systemd/system/attestation-service.service
sed -i '/Wants=attestation-agent.service/d' $(TARGET_DIR)/usr/lib/systemd/system/attestation-service.service
endef
endif
$(eval $(generic-package))
@@ -0,0 +1,28 @@
config BR2_PACKAGE_CC_ATTESTATION_AGENT
bool "cc-attestation-agent"
select BR2_PACKAGE_PROTOBUF
select BR2_PACKAGE_OPENSSL
select BR2_PACKAGE_TPM2_TSS
help
Confidential Containers attestation-agent for TEE attestation.
Optional backend for the Cocos AI attestation service that
provides KBS protocol support for remote attestation and
encrypted secret provisioning.
https://github.com/confidential-containers/guest-components
if BR2_PACKAGE_CC_ATTESTATION_AGENT
config BR2_PACKAGE_CC_ATTESTATION_AGENT_KBS_URL
string "Default KBS URL (optional)"
default ""
help
Optional default KBS (Key Broker Service) URL for remote
attestation and secret provisioning.
Leave empty to operate in local attestation mode only.
Example: https://kbs.example.com:8080
endif
@@ -0,0 +1,5 @@
#!/bin/bash
# Setup permissions for attestation socket directory
mkdir -p /run/cocos
chmod 755 /run/cocos
@@ -0,0 +1,37 @@
################################################################################
#
# cc-attestation-agent
#
################################################################################
CC_ATTESTATION_AGENT_VERSION = mvp-runner
CC_ATTESTATION_AGENT_SITE = $(call github,rodneyosodo,guest-components,$(CC_ATTESTATION_AGENT_VERSION))
CC_ATTESTATION_AGENT_LICENSE = Apache-2.0
CC_ATTESTATION_AGENT_LICENSE_FILES = LICENSE
CC_ATTESTATION_AGENT_DEPENDENCIES = host-rustc openssl protobuf tpm2-tss
# Build the attestation-agent from the guest-components repository with gRPC support
define CC_ATTESTATION_AGENT_BUILD_CMDS
cd $(@D)/attestation-agent && \
$(TARGET_MAKE_ENV) \
CARGO_HOME=$(@D)/.cargo \
make ATTESTER=all-attesters ttrpc=false
endef
define CC_ATTESTATION_AGENT_INSTALL_TARGET_CMDS
$(INSTALL) -D -m 0755 \
$(@D)/target/$(RUSTC_TARGET_NAME)/release/attestation-agent \
$(TARGET_DIR)/usr/bin/attestation-agent
endef
define CC_ATTESTATION_AGENT_INSTALL_INIT_SYSTEMD
$(INSTALL) -D -m 0644 \
$(BR2_EXTERNAL_COCOS_PATH)/package/cc-attestation-agent/cc-attestation-agent.service \
$(TARGET_DIR)/usr/lib/systemd/system/attestation-agent.service
$(INSTALL) -D -m 0750 \
$(BR2_EXTERNAL_COCOS_PATH)/package/cc-attestation-agent/cc-attestation-agent-setup.sh \
$(TARGET_DIR)/cocos_init/attestation_setup.sh
endef
$(eval $(generic-package))
@@ -0,0 +1,13 @@
[Unit]
Description=Confidential Containers Attestation Agent (gRPC)
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/attestation-agent --attestation_sock 127.0.0.1:50002
Restart=always
RestartSec=5
Environment=RUST_LOG=info
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,11 @@
config BR2_PACKAGE_COCO_KEYPROVIDER
bool "coco-keyprovider"
depends on BR2_PACKAGE_HOST_RUSTC_ARCH_SUPPORTS
select BR2_PACKAGE_HOST_RUSTC
help
CoCo Keyprovider is a keyprovider tool for generating and
decrypting CoCo-compatible encrypted images. It implements
the ocicrypt keyprovider protocol to decrypt OCI image layers
using the Key Broker Service (KBS).
https://github.com/confidential-containers/guest-components
@@ -0,0 +1,28 @@
#!/bin/sh
set -e
# Read kernel command line
CMDLINE=$(cat /proc/cmdline)
# Extract agent.aa_kbc_params value
# Format: agent.aa_kbc_params=cc_kbc::URL
PARAMS=$(echo "$CMDLINE" | tr ' ' '\n' | grep '^agent.aa_kbc_params=' | cut -d= -f2-)
if [ -n "$PARAMS" ]; then
# Extract URL part (after ::)
KBS_URL="${PARAMS#*::}"
if [ -n "$KBS_URL" ]; then
echo "[coco-keyprovider-setup] Detected KBS URL from kernel cmdline: $KBS_URL"
KBS_ARG="--kbs $KBS_URL"
fi
else
echo "[coco-keyprovider-setup] No agent.aa_kbc_params found in kernel cmdline. Starting without --kbs."
fi
# COCO_KP_SOCKET is set by EnvironmentFile in .service
if [ -z "$COCO_KP_SOCKET" ]; then
COCO_KP_SOCKET="127.0.0.1:50011"
fi
echo "[coco-keyprovider-setup] Starting coco_keyprovider listening on $COCO_KP_SOCKET $KBS_ARG"
exec /usr/local/bin/coco_keyprovider --socket "$COCO_KP_SOCKET" $KBS_ARG
@@ -0,0 +1,3 @@
# CoCo Keyprovider Environment Variables
COCO_KP_SOCKET=127.0.0.1:50011
RUST_LOG=info
@@ -0,0 +1,34 @@
################################################################################
#
# coco-keyprovider
#
################################################################################
COCO_KEYPROVIDER_VERSION = mvp-runner
COCO_KEYPROVIDER_SITE = $(call github,rodneyosodo,guest-components,$(COCO_KEYPROVIDER_VERSION))
COCO_KEYPROVIDER_LICENSE = Apache-2.0
COCO_KEYPROVIDER_LICENSE_FILES = LICENSE
COCO_KEYPROVIDER_DEPENDENCIES = host-rustc
define COCO_KEYPROVIDER_BUILD_CMDS
cd $(@D)/attestation-agent/coco_keyprovider && \
$(TARGET_MAKE_ENV) $(TARGET_CONFIGURE_OPTS) \
CARGO_HOME=$(HOST_DIR)/share/cargo \
cargo build --release --target=$(RUSTC_TARGET_NAME)
endef
define COCO_KEYPROVIDER_INSTALL_TARGET_CMDS
$(INSTALL) -D -m 0755 $(@D)/target/$(RUSTC_TARGET_NAME)/release/coco_keyprovider \
$(TARGET_DIR)/usr/local/bin/coco_keyprovider
$(INSTALL) -D -m 0755 $(BR2_EXTERNAL_COCOS_PATH)/package/coco-keyprovider/coco-keyprovider-setup.sh \
$(TARGET_DIR)/usr/local/bin/coco-keyprovider-setup.sh
$(INSTALL) -D -m 0644 $(BR2_EXTERNAL_COCOS_PATH)/package/coco-keyprovider/coco-keyprovider.service \
$(TARGET_DIR)/etc/systemd/system/coco-keyprovider.service
$(INSTALL) -D -m 0644 $(BR2_EXTERNAL_COCOS_PATH)/package/coco-keyprovider/coco-keyprovider.default \
$(TARGET_DIR)/etc/default/coco-keyprovider
mkdir -p $(TARGET_DIR)/etc
echo '{"key-providers": {"attestation-agent": {"grpc": "127.0.0.1:50011"}}}' > $(TARGET_DIR)/etc/ocicrypt_keyprovider.conf
endef
$(eval $(generic-package))
@@ -0,0 +1,25 @@
[Unit]
Description=CoCo Keyprovider for Confidential Containers
Documentation=https://github.com/confidential-containers/guest-components
After=network-online.target attestation-agent.service
Wants=network-online.target
Requires=attestation-agent.service
[Service]
Type=simple
EnvironmentFile=/etc/default/coco-keyprovider
RuntimeDirectory=coco-keyprovider
ExecStart=/usr/local/bin/coco-keyprovider-setup.sh
Restart=on-failure
RestartSec=5s
StandardOutput=journal
StandardError=journal
# Security hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,5 @@
config BR2_PACKAGE_COMPUTATION_RUNNER
bool "computation-runner"
select BR2_PACKAGE_LOG_FORWARDER
help
Cocos AI Computation Runner service.
@@ -0,0 +1,22 @@
################################################################################
#
# computation-runner
#
################################################################################
COMPUTATION_RUNNER_VERSION = main
COMPUTATION_RUNNER_SITE = $(call github,ultravioletrs,cocos,$(COMPUTATION_RUNNER_VERSION))
define COMPUTATION_RUNNER_BUILD_CMDS
$(MAKE) -C $(@D) computation-runner
endef
define COMPUTATION_RUNNER_INSTALL_TARGET_CMDS
$(INSTALL) -D -m 0750 $(@D)/build/cocos-computation-runner $(TARGET_DIR)/usr/bin/computation-runner
endef
define COMPUTATION_RUNNER_INSTALL_INIT_SYSTEMD
$(INSTALL) -D -m 0640 $(@D)/init/systemd/computation-runner.service $(TARGET_DIR)/usr/lib/systemd/system/computation-runner.service
endef
$(eval $(generic-package))
+6
View File
@@ -0,0 +1,6 @@
config BR2_PACKAGE_EGRESS_PROXY
bool "egress-proxy"
help
Cocos AI Egress Proxy Service.
https://github.com/ultravioletrs/cocos
@@ -0,0 +1,22 @@
################################################################################
#
# Cocos AI Egress Proxy
#
################################################################################
EGRESS_PROXY_VERSION = main
EGRESS_PROXY_SITE = $(call github,ultravioletrs,cocos,$(EGRESS_PROXY_VERSION))
define EGRESS_PROXY_BUILD_CMDS
$(MAKE) -C $(@D) egress-proxy
endef
define EGRESS_PROXY_INSTALL_TARGET_CMDS
$(INSTALL) -D -m 0755 $(@D)/build/cocos-egress-proxy $(TARGET_DIR)/usr/bin/egress-proxy
endef
define EGRESS_PROXY_INSTALL_INIT_SYSTEMD
$(INSTALL) -D -m 0644 $(@D)/init/systemd/egress-proxy.service $(TARGET_DIR)/usr/lib/systemd/system/egress-proxy.service
endef
$(eval $(generic-package))
+4
View File
@@ -0,0 +1,4 @@
config BR2_PACKAGE_INGRESS_PROXY
bool "ingress-proxy"
help
Cocos Ingress Proxy service.
@@ -0,0 +1,22 @@
################################################################################
#
# ingress-proxy
#
################################################################################
INGRESS_PROXY_VERSION = main
INGRESS_PROXY_SITE = $(call github,ultravioletrs,cocos,$(INGRESS_PROXY_VERSION))
define INGRESS_PROXY_BUILD_CMDS
$(MAKE) -C $(@D) ingress-proxy
endef
define INGRESS_PROXY_INSTALL_TARGET_CMDS
$(INSTALL) -D -m 0750 $(@D)/build/cocos-ingress-proxy $(TARGET_DIR)/usr/bin/ingress-proxy
endef
# NOTE: The ingress-proxy is managed per-computation by the agent, not as a standalone
# systemd service. The binary is installed for use by the agent, but no systemd service
# is created.
$(eval $(generic-package))
+4
View File
@@ -0,0 +1,4 @@
config BR2_PACKAGE_LOG_FORWARDER
bool "log-forwarder"
help
Cocos AI Log Forwarder service.
@@ -0,0 +1,22 @@
################################################################################
#
# log-forwarder
#
################################################################################
LOG_FORWARDER_VERSION = main
LOG_FORWARDER_SITE = $(call github,ultravioletrs,cocos,$(LOG_FORWARDER_VERSION))
define LOG_FORWARDER_BUILD_CMDS
$(MAKE) -C $(@D) log-forwarder
endef
define LOG_FORWARDER_INSTALL_TARGET_CMDS
$(INSTALL) -D -m 0750 $(@D)/build/cocos-log-forwarder $(TARGET_DIR)/usr/bin/log-forwarder
endef
define LOG_FORWARDER_INSTALL_INIT_SYSTEMD
$(INSTALL) -D -m 0640 $(@D)/init/systemd/log-forwarder.service $(TARGET_DIR)/usr/lib/systemd/system/log-forwarder.service
endef
$(eval $(generic-package))
+6
View File
@@ -0,0 +1,6 @@
config BR2_PACKAGE_WASMEDGE
bool "wasmedge"
default y
help
Wasmedge is a standalone runtime for WebAssembly.
https://wasmedge.org/docs/
+8
View File
@@ -0,0 +1,8 @@
WASMEDGE_DOWNLOAD_URL = https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh
define WASMEDGE_INSTALL_TARGET_CMDS
curl -sSf $(WASMEDGE_DOWNLOAD_URL) | bash -s -- -p $(TARGET_DIR)/usr -v 0.14.1
echo "source /usr/env" >> $(TARGET_DIR)/etc/profile
endef
$(eval $(generic-package))
@@ -0,0 +1,34 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
Subject: [PATCH] efi: skip lockdown when built with --disable-shim-lock
When GRUB is built with --disable-shim-lock, grub_shim_lock_verifier_setup()
returns early without registering the shim_lock verifier. However
grub_lockdown() was called unconditionally before that, registering the
lockdown_verifier which marks kernel files as DEFER_AUTH. With no verifier
present to approve them, every kernel load fails with "verification requested
but nobody cares".
Fix by calling grub_shim_lock_verifier_setup() first and only calling
grub_lockdown() if shim_lock is actually active. This preserves full
lockdown behaviour in shim-based chains while allowing direct
OVMF->GRUB->kernel boot with a custom DB key and no shim.
Signed-off-by: Cocos AI <build@cocos.ai>
---
grub-core/kern/efi/init.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/grub-core/kern/efi/init.c b/grub-core/kern/efi/init.c
--- a/grub-core/kern/efi/init.c
+++ b/grub-core/kern/efi/init.c
@@ -122,8 +122,9 @@
*/
if (grub_efi_get_secureboot () == GRUB_EFI_SECUREBOOT_MODE_ENABLED)
{
- grub_lockdown ();
grub_shim_lock_verifier_setup ();
+ if (grub_is_shim_lock_enabled ())
+ grub_lockdown ();
}
grub_efi_system_table->boot_services->set_watchdog_timer (0, 0, 0, NULL);
+1 -1
View File
@@ -10,7 +10,7 @@ HAL uses [Buildroot](https://buildroot.org/)'s [_External Tree_ mechanism](https
git clone git@github.com:ultravioletrs/cocos.git
git clone git@github.com:buildroot/buildroot.git
cd buildroot
git checkout 2025.08-rc3
git checkout 2025.11
make BR2_EXTERNAL=../cocos/hal/linux cocos_defconfig
# Execute 'make menuconfig' only if you want to make additional configuration changes to Buildroot.
make menuconfig
+5 -2
View File
@@ -22,5 +22,8 @@ if [ ! -d "$WORK_DIR" ]; then
mkdir -p $WORK_DIR
fi
# Resize the root file system to 100%
mount -o remount,size=100% /
# RAM-only agent images use tmpfs as the root filesystem
ROOT_FSTYPE=$(awk '$2 == "/" { print $3; exit }' /proc/mounts)
if [ "$ROOT_FSTYPE" = "tmpfs" ]; then
mount -o remount,size=100% /
fi
+2 -2
View File
@@ -1,7 +1,7 @@
[Unit]
Description=Cocos AI agent
After=network.target attestation-service.service log-forwarder.service computation-runner.service egress-proxy.service coco-keyprovider.service
Requires=log-forwarder.service computation-runner.service egress-proxy.service coco-keyprovider.service
After=network.target attestation-service.service log-forwarder.service computation-runner.service egress-proxy.service
Requires=log-forwarder.service computation-runner.service egress-proxy.service
Before=docker.service
[Service]
@@ -8,6 +8,7 @@ package attestation
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
+36 -4
View File
@@ -49,6 +49,12 @@ The service is configured using the environment variables from the following tab
| MANAGER_QEMU_VIRTIO_NET_PCI_ROMFILE | The file path for the ROM image for the virtio-net PCI device. | |
| MANAGER_QEMU_DISK_IMG_KERNEL_FILE | The file path for the kernel image. | img/bzImage |
| MANAGER_QEMU_DISK_IMG_ROOTFS_FILE | The file path for the root filesystem image. | img/rootfs.cpio.gz |
| MANAGER_QEMU_ENABLE_DISK | Whether to attach a writable qcow2 disk to the CVM. | false |
| MANAGER_QEMU_SRC_DISK_FILE | Path to a qcow2 image whose virtual size is used to size the per-VM writable disk. | img/enc_os.qcow2 |
| MANAGER_QEMU_DST_DISK_FILE | Runtime path of the per-VM writable disk created by the manager. | |
| MANAGER_QEMU_DISK_ID | The QEMU drive identifier for the attached disk. | disk0 |
| MANAGER_QEMU_DISK_FORMAT | The format of the attached disk image. | qcow2 |
| MANAGER_QEMU_DISK_SCSI_ID | The SCSI controller identifier used for the attached disk. | scsi0 |
| MANAGER_QEMU_SEV_SNP_ID | The ID for the Secure Encrypted Virtualization (SEV-SNP) device. | sev0 |
| MANAGER_QEMU_SEV_SNP_CBITPOS | The position of the C-bit in the physical address. | 51 |
| MANAGER_QEMU_SEV_SNP_REDUCED_PHYS_BITS | The number of reduced physical address bits for SEV-SNP. | 1 |
@@ -112,6 +118,20 @@ Once the image is built copy the kernel and rootfs image to `cmd/manager/img` fr
Another option is to use release versions of EOS that can be downloaded from the [Cocos GitHub repository](https://github.com/ultravioletrs/cocos/releases).
#### Optional writable disk
If you want the manager to attach a writable disk to each CVM, place a qcow2 reference image at `cmd/manager/img/enc_os.qcow2`, or point `MANAGER_QEMU_SRC_DISK_FILE` to another qcow2 file.
When `MANAGER_QEMU_ENABLE_DISK=true`, the manager:
- reads the virtual size of `MANAGER_QEMU_SRC_DISK_FILE` with `qemu-img info`
- creates a per-VM qcow2 disk under `/tmp/cvmDisk-<uuid>.qcow2`
- sizes the disk to the source image size plus 1 GiB, leaving room for the LUKS header
- attaches the disk through a virtio-scsi controller
- removes the temporary disk again when the VM stops
`MANAGER_QEMU_DST_DISK_FILE` is primarily a runtime value. In the normal manager flow it is populated automatically and usually does not need to be set manually.
#### Test VM creation
```sh
@@ -207,7 +227,7 @@ nc -zv localhost 7020
#### Conclusion
Now you are able to use `Manager` with `Agent`. Namely, `Manager` will create a VM with a separate OVMF variables file on manager `/run` request.
Now you are able to use `Manager` with `Agent`. On each manager `/run` request, the manager creates a VM with a separate OVMF variables file and, when enabled, a per-VM writable qcow2 disk.
### OVMF
@@ -284,6 +304,18 @@ MANAGER_QEMU_OVMF_FILE=<path to OVMF file> \
./build/cocos-manager
```
To enable writable disk support, start manager like this
```sh
MANAGER_GRPC_URL=localhost:7001 \
MANAGER_LOG_LEVEL=debug \
MANAGER_QEMU_ENABLE_DISK=true \
MANAGER_QEMU_SRC_DISK_FILE=<path to reference qcow2 image> \
./build/cocos-manager
```
The reference qcow2 image is used to determine the disk size. The manager creates a fresh writable qcow2 disk for each VM under `/tmp` and deletes it on shutdown.
### Troubleshooting
If the `ps aux | grep qemu-system-x86_64` give you something like this
@@ -294,16 +326,16 @@ darko 13913 0.0 0.0 0 0 pts/2 Z+ 20:17 0:00 [qemu-system-
means that the a QEMU virtual machine that is currently defunct, meaning that it is no longer running. More precisely, the defunct process in the output is also known as a ["zombie" process](https://en.wikipedia.org/wiki/Zombie_process).
You can troubleshoot the VM launch procedure by running directly `qemu-system-x86_64` command. When you run `manager` with `MANAGER_LOG_LEVEL=info` env var set, it prints out the entire command used to launch a VM. The relevant part of the log might look like this
You can troubleshoot the VM launch procedure by running directly `qemu-system-x86_64` command. When you run `manager` with `MANAGER_LOG_LEVEL=info` env var set, it prints out the entire command used to launch a VM. When writable disk support is enabled, the relevant part of the log might look like this
```
{"level":"info","message":"/usr/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -nographic -monitor pty","ts":"2023-08-14T18:29:19.2653908Z"}
{"level":"info","message":"/usr/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=/tmp/OVMF_VARS-<uuid>.fd -netdev user,id=vmnic-<uuid>,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic-<uuid>,addr=0x2,romfile= -drive file=/tmp/cvmDisk-<uuid>.qcow2,if=none,id=disk0,format=qcow2 -device virtio-scsi-pci,id=scsi0,disable-legacy=on,iommu_platform=true -device scsi-hd,drive=disk0,bus=scsi0.0 -kernel img/bzImage -append quiet console=null -initrd img/rootfs.cpio.gz -nographic -monitor pty","ts":"2026-04-27T00:00:00Z"}
```
You can run the command - the value of the `"message"` key - directly in the terminal:
```sh
/usr/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=img/OVMF_VARS.fd -device virtio-scsi-pci,id=scsi,disable-legacy=on,iommu_platform=true -drive file=img/focal-server-cloudimg-amd64.img,if=none,id=disk0,format=qcow2 -device scsi-hd,drive=disk0 -netdev user,id=vmnic,hostfwd=tcp::2222-:22,hostfwd=tcp::9301-:9031,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,romfile= -nographic -monitor pty
/usr/bin/qemu-system-x86_64 -enable-kvm -machine q35 -cpu EPYC -smp 4,maxcpus=64 -m 4096M,slots=5,maxmem=30G -drive if=pflash,format=raw,unit=0,file=/usr/share/OVMF/OVMF_CODE.fd,readonly=on -drive if=pflash,format=raw,unit=1,file=/tmp/OVMF_VARS-<uuid>.fd -netdev user,id=vmnic-<uuid>,hostfwd=tcp::7020-:7002 -device virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic-<uuid>,addr=0x2,romfile= -drive file=/tmp/cvmDisk-<uuid>.qcow2,if=none,id=disk0,format=qcow2 -device virtio-scsi-pci,id=scsi0,disable-legacy=on,iommu_platform=true -device scsi-hd,drive=disk0,bus=scsi0.0 -kernel img/bzImage -append "quiet console=null" -initrd img/rootfs.cpio.gz -nographic -monitor pty
```
and look for the possible problems. This problems can usually be solved by using the adequate env var assignments. Look in the `manager/qemu/config.go` file to see the recognized env vars. Don't forget to prepend `MANAGER_QEMU_` to the name of the env vars.
+56 -6
View File
@@ -4,6 +4,7 @@ package qemu
import (
"fmt"
"strings"
"github.com/caarlos0/env/v10"
)
@@ -48,7 +49,7 @@ type VirtioNetPciConfig struct {
ROMFile string `env:"VIRTIO_NET_PCI_ROMFILE"`
}
type DiskImgConfig struct {
type KernelConfig struct {
KernelFile string `env:"DISK_IMG_KERNEL_FILE" envDefault:"img/bzImage"`
RootFsFile string `env:"DISK_IMG_ROOTFS_FILE" envDefault:"img/rootfs.cpio.gz"`
}
@@ -72,9 +73,18 @@ type IGVMConfig struct {
File string `env:"IGVM_FILE" envDefault:"/root/coconut-qemu.igvm"`
}
type DiskConfig struct {
SrcFile string `env:"SRC_DISK_FILE" envDefault:"img/enc_os.qcow2"`
DstFile string `env:"DST_DISK_FILE" envDefault:""`
ID string `env:"DISK_ID" envDefault:"disk0"`
Format string `env:"DISK_FORMAT" envDefault:"qcow2"`
SCSIID string `env:"DISK_SCSI_ID" envDefault:"scsi0"`
}
type Config struct {
EnableSEVSNP bool
EnableTDX bool
EnableDisk bool `env:"ENABLE_DISK" envDefault:"false"`
QemuBinPath string `env:"BIN_PATH" envDefault:"qemu-system-x86_64"`
UseSudo bool `env:"USE_SUDO" envDefault:"false"`
@@ -96,8 +106,11 @@ type Config struct {
NetDevConfig
VirtioNetPciConfig
// disk
DiskImgConfig
// disk config
DiskConfig
// kernel and initramfs
KernelConfig
// SEV-SNP
SEVSNPConfig
@@ -123,6 +136,25 @@ type Config struct {
EnvMount string `env:"ENV_MOUNT" envDefault:""`
}
func (config Config) ValidateBootConfig() error {
if config.EnableDisk {
if strings.TrimSpace(config.DiskConfig.DstFile) == "" {
return fmt.Errorf("disk boot enabled but destination disk image is not set")
}
return nil
}
if strings.TrimSpace(config.KernelConfig.KernelFile) == "" {
return fmt.Errorf("kernel boot enabled but kernel image is not set")
}
if strings.TrimSpace(config.KernelConfig.RootFsFile) == "" {
return fmt.Errorf("kernel boot enabled but initramfs image is not set")
}
return nil
}
func (config Config) ConstructQemuArgs() []string {
args := []string{}
@@ -179,6 +211,22 @@ func (config Config) ConstructQemuArgs() []string {
config.VirtioNetPciConfig.Addr,
config.VirtioNetPciConfig.ROMFile))
if config.EnableDisk {
// disk image
args = append(args, "-drive",
fmt.Sprintf("file=%s,if=none,id=%s,format=%s",
config.DiskConfig.DstFile,
config.DiskConfig.ID,
config.DiskConfig.Format))
args = append(args, "-device",
fmt.Sprintf("virtio-scsi-pci,id=%s,disable-legacy=on,iommu_platform=true",
config.DiskConfig.SCSIID))
args = append(args, "-device",
fmt.Sprintf("scsi-hd,drive=%s,bus=%s.0",
config.DiskConfig.ID,
config.DiskConfig.SCSIID))
}
// SEV-SNP
if config.EnableSEVSNP {
sevSnpType := "sev-snp-guest"
@@ -233,9 +281,11 @@ func (config Config) ConstructQemuArgs() []string {
args = append(args, "-nodefaults")
}
args = append(args, "-kernel", config.DiskImgConfig.KernelFile)
args = append(args, "-append", config.KernelCommandLine)
args = append(args, "-initrd", config.DiskImgConfig.RootFsFile)
if !config.EnableDisk {
args = append(args, "-kernel", config.KernelConfig.KernelFile)
args = append(args, "-append", config.KernelCommandLine)
args = append(args, "-initrd", config.KernelConfig.RootFsFile)
}
// display
if config.NoGraphic {
+153 -2
View File
@@ -51,7 +51,7 @@ func TestConstructQemuArgs(t *testing.T) {
IOMMUPlatform: true,
Addr: "0x2",
},
DiskImgConfig: DiskImgConfig{
KernelConfig: KernelConfig{
KernelFile: "img/bzImage",
RootFsFile: "img/rootfs.cpio.gz",
},
@@ -115,7 +115,7 @@ func TestConstructQemuArgs(t *testing.T) {
IOMMUPlatform: true,
Addr: "0x2",
},
DiskImgConfig: DiskImgConfig{
KernelConfig: KernelConfig{
KernelFile: "img/bzImage",
RootFsFile: "img/rootfs.cpio.gz",
},
@@ -194,3 +194,154 @@ func TestConstructQemuArgs_HostData(t *testing.T) {
t.Errorf("ConstructQemuArgs() did not contain expected SEV-SNP configuration with host data")
}
}
func TestConstructQemuArgs_TDX(t *testing.T) {
config := Config{
EnableKVM: true,
EnableTDX: true,
Machine: "q35",
CPU: "EPYC",
SMPCount: 4,
MaxCPUs: 64,
MemID: "ram1",
MemoryConfig: MemoryConfig{
Size: "4096M",
Slots: 8,
Max: "64G",
},
NetDevConfig: NetDevConfig{
ID: "vmnic",
HostFwdAgent: 7020,
GuestFwdAgent: 7002,
},
VirtioNetPciConfig: VirtioNetPciConfig{
DisableLegacy: "on",
IOMMUPlatform: true,
Addr: "0x2",
},
TDXConfig: TDXConfig{
ID: "tdx0",
QuoteGenerationPort: 4050,
OVMF: "/usr/share/ovmf/OVMF.fd",
},
KernelConfig: KernelConfig{
KernelFile: "img/bzImage",
RootFsFile: "img/rootfs.cpio.gz",
},
KernelCommandLine: "quiet console=null",
NoGraphic: true,
Monitor: "pty",
}
expected := []string{
"-enable-kvm",
"-machine", "q35",
"-cpu", "EPYC",
"-smp", "4,maxcpus=64",
"-m", "4096M,slots=8,maxmem=64G",
"-netdev", "user,id=vmnic,hostfwd=tcp::7020-:7002",
"-device", "virtio-net-pci,disable-legacy=on,iommu_platform=true,netdev=vmnic,addr=0x2,romfile=",
"-object", "{\"qom-type\":\"tdx-guest\",\"id\":\"tdx0\",\"quote-generation-socket\":{\"type\": \"vsock\", \"cid\":\"2\",\"port\":\"4050\"}}",
"-machine", "confidential-guest-support=tdx0,memory-backend=ram1,hpet=off",
"-object", "memory-backend-memfd,id=ram1,size=4096M,share=true,prealloc=false",
"-bios", "/usr/share/ovmf/OVMF.fd",
"-nodefaults",
"-kernel", "img/bzImage",
"-append", "quiet console=null",
"-initrd", "img/rootfs.cpio.gz",
"-nographic",
"-monitor", "pty",
}
result := config.ConstructQemuArgs()
if !reflect.DeepEqual(result, expected) {
t.Errorf("ConstructQemuArgs() = %v, want %v", result, expected)
}
}
func TestConstructQemuArgs_DiskBootSkipsKernelAndInitrd(t *testing.T) {
config := Config{
EnableKVM: true,
EnableDisk: true,
Machine: "q35",
CPU: "EPYC",
SMPCount: 4,
MaxCPUs: 64,
MemID: "ram1",
MemoryConfig: MemoryConfig{
Size: "2048M",
Slots: 5,
Max: "30G",
},
NetDevConfig: NetDevConfig{
ID: "vmnic",
HostFwdAgent: 7020,
GuestFwdAgent: 7002,
},
VirtioNetPciConfig: VirtioNetPciConfig{
DisableLegacy: "on",
IOMMUPlatform: true,
Addr: "0x2",
},
DiskConfig: DiskConfig{
DstFile: "img/disk.img",
ID: "disk0",
Format: "qcow2",
SCSIID: "scsi0",
},
KernelConfig: KernelConfig{
KernelFile: "img/bzImage",
RootFsFile: "img/rootfs.cpio.gz",
},
NoGraphic: true,
Monitor: "pty",
}
result := config.ConstructQemuArgs()
for _, forbidden := range []string{"-kernel", "-append", "-initrd"} {
for _, arg := range result {
if arg == forbidden {
t.Fatalf("ConstructQemuArgs() unexpectedly contained %s during disk boot: %v", forbidden, result)
}
}
}
}
func TestConstructQemuArgs_EnableDisk(t *testing.T) {
config := Config{
EnableDisk: true,
DiskConfig: DiskConfig{
SrcFile: "img/enc_os.qcow2",
DstFile: "img/enc_os_dst.qcow2",
ID: "disk0",
Format: "qcow2",
SCSIID: "scsi0",
},
}
result := config.ConstructQemuArgs()
expected := []string{
"-drive", "file=img/enc_os_dst.qcow2,if=none,id=disk0,format=qcow2",
"-device", "virtio-scsi-pci,id=scsi0,disable-legacy=on,iommu_platform=true",
"-device", "scsi-hd,drive=disk0,bus=scsi0.0",
}
var found []bool = make([]bool, len(expected))
for i, arg := range result {
for j := 0; j < len(expected); j += 2 {
if arg == expected[j] && i+1 < len(result) && result[i+1] == expected[j+1] {
found[j] = true
found[j+1] = true
break
}
}
}
for j, f := range found {
if !f {
t.Errorf("ConstructQemuArgs() did not contain expected disk configuration: %s", expected[j])
}
}
}
+94 -6
View File
@@ -3,10 +3,12 @@
package qemu
import (
"encoding/json"
"fmt"
"log/slog"
"os"
"os/exec"
"path/filepath"
"strings"
"syscall"
"time"
@@ -18,12 +20,15 @@ import (
)
const (
firmwareVars = "OVMF_VARS"
KernelFile = "bzImage"
rootfsFile = "rootfs.cpio"
tmpDir = "/tmp"
interval = 5 * time.Second
shutdownTimeout = 30 * time.Second
firmwareVars = "OVMF_VARS"
KernelFile = "bzImage"
rootfsFile = "rootfs.cpio"
tmpDir = "/tmp"
diskDstName = "cvmDisk"
interval = 5 * time.Second
shutdownTimeout = 30 * time.Second
encryptedPartitionSizeDeltaGB = 1
sourceDiskFormat = "qcow2"
)
type VMInfo struct {
@@ -39,6 +44,10 @@ type qemuVM struct {
vm.StateMachine
}
type qemuInfo struct {
VirtualSize int64 `json:"virtual-size"`
}
func NewVM(config any, cvmId string, logger *slog.Logger) vm.VM {
return &qemuVM{
vmi: config.(VMInfo),
@@ -75,6 +84,44 @@ func (v *qemuVM) Start() (err error) {
v.vmi.Config.OVMFVarsConfig.File = dstFile
}
if v.vmi.Config.EnableDisk {
srcDiskFile, err := filepath.Abs(v.vmi.Config.SrcFile)
if err != nil {
return err
}
sizeGB, err := GetVirtualSizeGB(srcDiskFile)
if err != nil {
return err
}
dstDiskFile := fmt.Sprintf("%s/%s-%s.%s", tmpDir, diskDstName, id, v.vmi.Config.DiskConfig.Format)
sizeArg := fmt.Sprintf("%dG", sizeGB+encryptedPartitionSizeDeltaGB)
cmd := exec.Command(
"qemu-img",
"convert",
"-f", sourceDiskFormat,
"-O", v.vmi.Config.DiskConfig.Format,
srcDiskFile,
dstDiskFile,
)
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("qemu-img convert failed: %w: %s", err, string(out))
}
cmd = exec.Command(
"qemu-img",
"resize",
dstDiskFile,
sizeArg,
)
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("qemu-img resize failed: %w: %s", err, string(out))
}
v.vmi.Config.DstFile = dstDiskFile
}
exe, args, err := v.executableAndArgs()
if err != nil {
return err
@@ -111,6 +158,14 @@ func (v *qemuVM) Stop() error {
}
}
if v.vmi.Config.EnableDisk {
if v.vmi.Config.DstFile != "" {
if err := os.RemoveAll(v.vmi.Config.DstFile); err != nil {
return fmt.Errorf("failed to remove disk file: %v", err)
}
}
}
done := make(chan error, 1)
go func() {
_, err := v.cmd.Process.Wait()
@@ -156,6 +211,10 @@ func (v *qemuVM) executableAndArgs() (string, []string, error) {
return "", nil, err
}
if err := v.vmi.Config.ValidateBootConfig(); err != nil {
return "", nil, err
}
args := v.vmi.Config.ConstructQemuArgs()
if v.vmi.Config.UseSudo {
@@ -231,3 +290,32 @@ func TDXEnabledOnHost() bool {
return TDXEnabled(string(cpuinfo), string(kernelParam))
}
func GetVirtualSizeBytes(path string) (int64, error) {
cmd := exec.Command("qemu-img", "info", "--output=json", path)
out, err := cmd.Output()
if err != nil {
return 0, fmt.Errorf("qemu-img info failed: %w", err)
}
var info qemuInfo
if err := json.Unmarshal(out, &info); err != nil {
return 0, fmt.Errorf("failed to parse qemu-img JSON: %w", err)
}
if info.VirtualSize <= 0 {
return 0, fmt.Errorf("invalid virtual size: %d", info.VirtualSize)
}
return info.VirtualSize, nil
}
func GetVirtualSizeGB(path string) (int, error) {
bytes, err := GetVirtualSizeBytes(path)
if err != nil {
return 0, err
}
gb := (bytes + (1<<30 - 1)) >> 30
return int(gb), nil
}
+358 -3
View File
@@ -3,9 +3,12 @@
package qemu
import (
"fmt"
"log/slog"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
@@ -15,6 +18,31 @@ import (
const testComputationID = "test-computation"
func cleanupStrayQcow2(t *testing.T) {
t.Helper()
wd, err := os.Getwd()
if err != nil {
t.Fatalf("failed to get working directory: %v", err)
}
t.Cleanup(func() {
_ = os.Remove(filepath.Join(wd, "qcow2"))
})
}
func requireTempFile(t *testing.T, path string) {
t.Helper()
f, err := os.Create(path)
if err != nil {
t.Fatalf("failed to create temp file %s: %v", path, err)
}
if err := f.Close(); err != nil {
t.Fatalf("failed to close temp file %s: %v", path, err)
}
}
func TestNewVM(t *testing.T) {
config := VMInfo{Config: Config{}}
@@ -35,6 +63,10 @@ func TestStart(t *testing.T) {
File: tmpFile.Name(),
},
QemuBinPath: "echo",
KernelConfig: KernelConfig{
KernelFile: "img/bzImage",
RootFsFile: "img/rootfs.cpio.gz",
},
}}
vm := NewVM(config, testComputationID, slog.Default()).(*qemuVM)
@@ -58,6 +90,10 @@ func TestStartSudo(t *testing.T) {
},
QemuBinPath: "echo",
UseSudo: true,
KernelConfig: KernelConfig{
KernelFile: "img/bzImage",
RootFsFile: "img/rootfs.cpio.gz",
},
}}
vm := NewVM(config, testComputationID, slog.Default()).(*qemuVM)
@@ -69,6 +105,136 @@ func TestStartSudo(t *testing.T) {
_ = vm.Stop()
}
func TestStart_EnableDisk(t *testing.T) {
cleanupStrayQcow2(t)
toolsDir := t.TempDir()
convertLogFile := filepath.Join(toolsDir, "qemu-img-convert.log")
resizeLogFile := filepath.Join(toolsDir, "qemu-img-resize.log")
srcDiskFile := filepath.Join(toolsDir, "enc_os.qcow2")
requireTempFile(t, srcDiskFile)
writeFakeExecutable(t, toolsDir, "qemu-img", fmt.Sprintf(`#!/bin/sh
case "$1" in
info)
printf '%%s' '{"virtual-size":2147483648}'
;;
convert)
printf '%%s\n' "$@" > %q
dst="$7"
: > "$dst"
;;
resize)
printf '%%s\n' "$@" > %q
;;
*)
echo "unexpected subcommand: $1" >&2
exit 2
;;
esac
`, convertLogFile, resizeLogFile))
writeFakeExecutable(t, toolsDir, "fake-qemu", `#!/bin/sh
trap 'exit 0' TERM INT
while :; do
sleep 1
done
`)
prependPath(t, toolsDir)
config := VMInfo{Config: Config{
EnableTDX: true,
EnableDisk: true,
QemuBinPath: "fake-qemu",
DiskConfig: DiskConfig{
SrcFile: srcDiskFile,
ID: "disk0",
Format: "qcow2",
SCSIID: "scsi0",
},
}}
vm := NewVM(config, testComputationID, slog.Default()).(*qemuVM)
err := vm.Start()
assert.NoError(t, err)
assert.NotNil(t, vm.cmd)
assert.Contains(t, vm.vmi.Config.DstFile, filepath.Join(tmpDir, diskDstName))
_, err = os.Stat(vm.vmi.Config.DstFile)
assert.NoError(t, err)
loggedArgs, err := os.ReadFile(convertLogFile)
assert.NoError(t, err)
assert.Equal(t, []string{
"convert",
"-f",
"qcow2",
"-O",
"qcow2",
srcDiskFile,
vm.vmi.Config.DstFile,
}, strings.Fields(string(loggedArgs)))
loggedArgs, err = os.ReadFile(resizeLogFile)
assert.NoError(t, err)
assert.Equal(t, []string{
"resize",
vm.vmi.Config.DstFile,
"3G",
}, strings.Fields(string(loggedArgs)))
err = vm.Stop()
assert.NoError(t, err)
_, err = os.Stat(vm.vmi.Config.DstFile)
assert.Error(t, err)
assert.True(t, os.IsNotExist(err))
}
func TestStart_EnableDiskCreateError(t *testing.T) {
cleanupStrayQcow2(t)
toolsDir := t.TempDir()
srcDiskFile := filepath.Join(toolsDir, "enc_os.qcow2")
requireTempFile(t, srcDiskFile)
writeFakeExecutable(t, toolsDir, "qemu-img", `#!/bin/sh
case "$1" in
info)
printf '%s' '{"virtual-size":2147483648}'
;;
convert)
echo 'disk create failed' >&2
exit 1
;;
resize)
exit 0
;;
*)
echo "unexpected subcommand: $1" >&2
exit 2
;;
esac
`)
prependPath(t, toolsDir)
config := VMInfo{Config: Config{
EnableTDX: true,
EnableDisk: true,
QemuBinPath: "fake-qemu",
DiskConfig: DiskConfig{
SrcFile: srcDiskFile,
},
}}
vm := NewVM(config, testComputationID, slog.Default()).(*qemuVM)
err := vm.Start()
assert.Error(t, err)
assert.ErrorContains(t, err, "qemu-img convert failed")
assert.ErrorContains(t, err, "disk create failed")
assert.Nil(t, vm.cmd)
}
func TestStop(t *testing.T) {
t.Run("success", func(t *testing.T) {
cmd := exec.Command("echo", "test")
@@ -102,6 +268,42 @@ func TestStop(t *testing.T) {
StateMachine: sm,
}
err = vm.Stop()
assert.NoError(t, err)
})
t.Run("disk enable", func(t *testing.T) {
dir := t.TempDir()
dst := filepath.Join(dir, "disk.qcow2")
f, err := os.Create(dst)
if err != nil {
t.Fatal(err)
}
if err := f.Close(); err != nil {
t.Fatal(err)
}
cmd := exec.Command("echo", "test")
err = cmd.Start()
assert.NoError(t, err)
sm := new(mocks.StateMachine)
sm.On("Transition", pkgmanager.StopComputationRun).Return(nil)
vm := &qemuVM{
vmi: VMInfo{
Config: Config{
EnableDisk: true,
DiskConfig: DiskConfig{
DstFile: dst,
},
},
},
cmd: &exec.Cmd{
Process: cmd.Process,
},
StateMachine: sm,
}
err = vm.Stop()
assert.NoError(t, err)
})
@@ -110,7 +312,13 @@ func TestStop(t *testing.T) {
func TestSetProcess(t *testing.T) {
vm := &qemuVM{
vmi: VMInfo{
Config: Config{QemuBinPath: "echo"}, // Use 'echo' as a dummy QEMU binary
Config: Config{
QemuBinPath: "echo", // Use 'echo' as a dummy QEMU binary
KernelConfig: KernelConfig{
KernelFile: "img/bzImage",
RootFsFile: "img/rootfs.cpio.gz",
},
},
},
}
@@ -175,9 +383,156 @@ func TestTDXEnabled(t *testing.T) {
}
func TestSEVSNPEnabledOnHost(t *testing.T) {
assert.False(t, SEVSNPEnabledOnHost())
cpuinfo, cpuErr := os.ReadFile("/proc/cpuinfo")
kernelParam, kernelErr := os.ReadFile("/sys/module/kvm_amd/parameters/sev_snp")
expected := false
if cpuErr == nil && kernelErr == nil {
expected = SEVSNPEnabled(string(cpuinfo), string(kernelParam))
}
assert.Equal(t, expected, SEVSNPEnabledOnHost())
}
func TestTDXEnabledOnHost(t *testing.T) {
assert.False(t, TDXEnabledOnHost())
cpuinfo, cpuErr := os.ReadFile("/proc/cpuinfo")
kernelParam, kernelErr := os.ReadFile("/sys/module/kvm_intel/parameters/tdx")
expected := false
if cpuErr == nil && kernelErr == nil {
expected = TDXEnabled(string(cpuinfo), string(kernelParam))
}
assert.Equal(t, expected, TDXEnabledOnHost())
}
func TestGetVirtualSizeBytes_Success(t *testing.T) {
cleanup := writeFakeQemuImg(t, `{"virtual-size":2147483648}`, 0) // 2 GiB
defer cleanup()
got, err := GetVirtualSizeBytes("whatever.qcow2")
if err != nil {
t.Fatalf("expected nil error, got %v", err)
}
if got != 2147483648 {
t.Fatalf("expected 2147483648, got %d", got)
}
}
func TestGetVirtualSizeBytes_CommandFailure(t *testing.T) {
cleanup := writeFakeQemuImg(t, `{"virtual-size":2147483648}`, 1) // non-zero exit
defer cleanup()
_, err := GetVirtualSizeBytes("whatever.qcow2")
if err == nil {
t.Fatalf("expected error, got nil")
}
if !strings.Contains(err.Error(), "qemu-img info failed") {
t.Fatalf("expected wrapped error to contain %q, got %q", "qemu-img info failed", err.Error())
}
}
func TestGetVirtualSizeBytes_InvalidJSON(t *testing.T) {
cleanup := writeFakeQemuImg(t, `not-json`, 0)
defer cleanup()
_, err := GetVirtualSizeBytes("whatever.qcow2")
if err == nil {
t.Fatalf("expected error, got nil")
}
if !strings.Contains(err.Error(), "failed to parse qemu-img JSON") {
t.Fatalf("expected error to contain %q, got %q", "failed to parse qemu-img JSON", err.Error())
}
}
func TestGetVirtualSizeBytes_InvalidVirtualSize(t *testing.T) {
cleanup := writeFakeQemuImg(t, `{"virtual-size":0}`, 0)
defer cleanup()
_, err := GetVirtualSizeBytes("whatever.qcow2")
if err == nil {
t.Fatalf("expected error, got nil")
}
if !strings.Contains(err.Error(), "invalid virtual size") {
t.Fatalf("expected error to contain %q, got %q", "invalid virtual size", err.Error())
}
}
func TestGetVirtualSizeGB_RoundsUp(t *testing.T) {
tests := []struct {
name string
virtualSz int64
wantGB int
}{
{"exact_1GiB", 1 << 30, 1},
{"one_byte_over", (1 << 30) + 1, 2},
{"just_under_2GiB", (2 << 30) - 1, 2},
{"exact_2GiB", 2 << 30, 2},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
cleanup := writeFakeQemuImg(t, fmt.Sprintf(`{"virtual-size":%d}`, tc.virtualSz), 0)
defer cleanup()
got, err := GetVirtualSizeGB("whatever.qcow2")
if err != nil {
t.Fatalf("expected nil error, got %v", err)
}
if got != tc.wantGB {
t.Fatalf("expected %d, got %d", tc.wantGB, got)
}
})
}
}
func writeFakeQemuImg(t *testing.T, stdout string, exitCode int) func() {
dir := t.TempDir()
fake := filepath.Join(dir, "qemu-img")
script := fmt.Sprintf(`#!/bin/sh
# Minimal fake for: qemu-img info --output=json <path>
if [ "$1" != "info" ]; then
echo "unexpected subcommand: $1" >&2
exit 2
fi
# always print provided stdout, even if empty
printf '%s' %q
exit %d
`, stdout, stdout, exitCode)
if err := os.WriteFile(fake, []byte(script), 0o755); err != nil {
t.Fatalf("failed to write fake qemu-img: %v", err)
}
oldPath := os.Getenv("PATH")
if err := os.Setenv("PATH", dir+string(os.PathListSeparator)+oldPath); err != nil {
t.Fatalf("failed to set PATH: %v", err)
}
return func() {
_ = os.Setenv("PATH", oldPath)
}
}
func writeFakeExecutable(t *testing.T, dir, name, script string) {
t.Helper()
path := filepath.Join(dir, name)
if err := os.WriteFile(path, []byte(script), 0o755); err != nil {
t.Fatalf("failed to write fake executable %q: %v", name, err)
}
}
func prependPath(t *testing.T, dir string) {
t.Helper()
oldPath := os.Getenv("PATH")
if err := os.Setenv("PATH", dir+string(os.PathListSeparator)+oldPath); err != nil {
t.Fatalf("failed to set PATH: %v", err)
}
t.Cleanup(func() {
_ = os.Setenv("PATH", oldPath)
})
}