Files
cocos/manager/qemu/qemu.go
T
Sammy Kerata Oina 55afe4c038 COCOS-49 - Pass agent configuration and computation via vsock (#57)
* Optimize QEMU launch and add V-sock support

Refactored QEMU argument construction and launching logic by removing the dependency on 'agent.Computation'. This simplification makes the VM creation process more streamlined. Additionally, introduced V-sock capabilities in the QEMU configuration to facilitate improved guest-host communication. Updated the README to include kernel module setup instructions for the new V-sock feature.

The V-sock implementation enables VMs to use a consistent communication channel that is not affected by network configuration changes, enhancing reliability and potential interoperability with host services. It's important to ensure that the necessary kernel modules are loaded as part of the setup process, as documented.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add vsock-based communication to manager

Introduced virtual socket (vsock) communication abilities in the manager package by implementing a new socket service. This includes establishing a vsock listener and stub methods for sending computation results and cleaning up resources. The addition provides the groundwork for interprocess communication between guest and host in virtualized environments.

- Integrated the `mdlayher/vsock` library for handling virtual socket operations.
- Created a new `sockService` struct to encapsulate vsock listener handling.
- Implemented `NewVsock` constructor to initialize the listener with domain value `3`.
- Added placeholder methods for future computation sending and service closing logic.

This enhancement targets scenarios where efficient VM-to-host communication is required.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove env

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor agent config and use vsock

Introduce `AgentConfig` struct to group agent-related configurations, and update `Computation` struct to include the new `AgentConfig` field. Replace command-line computation extraction with vsock-based config retrieval for robustness and decoupling. The agent configuration is now read from a vsock connection during runtime, allowing for more dynamic and flexible deployments. Adjusted the main agent application logic to support these configuration changes, and corresponding changes have been made in the manager to facilitate vsock communication.

This approach aligns with modern practices for microservices by streamlining configuration management and reducing reliance on static command-line parameters. Moreover, it enhances the scalability of the agent service by allowing configuration to be managed externally.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor agent config and remove deprecated code

Consolidated agent configuration management into a single `AgentConfig` message and pruned deprecated Protobuf `ComputationReq`, `DatasetReq`, and `AlgorithmReq` messages. Adapted corresponding manager service logic to the new configuration structure. These modifications align with updated manager API schema, facilitate clearer configuration handling, and improve maintainability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* send configuration

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Switch agent to listen mode for manager connections

Previously, the agent established a connection to the manager using a direct dial. This change shifts the setup to where the agent listens on a specified port and accepts incoming connections. It ensures that the agent properly handles incoming requests by initiating a listening socket and waiting for the manager to connect, enhancing the system's flexibility in connection management. This adjustment also includes graceful closure of the listening socket.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-19 17:24:51 +01:00

94 lines
2.1 KiB
Go

// Copyright (c) Ultraviolet
// SPDX-License-Identifier: Apache-2.0
package qemu
import (
"context"
"fmt"
"os/exec"
"github.com/gofrs/uuid"
"github.com/ultravioletrs/cocos/internal"
)
const (
qemuRelPath = "qemu-system-x86_64"
firmwareVars = "OVMF_VARS"
KernelFile = "bzImage"
rootfsFile = "rootfs.cpio"
)
func CreateVM(ctx context.Context, cfg Config) (*exec.Cmd, error) {
// Create unique emu device identifiers.
id, err := uuid.NewV4()
if err != nil {
return &exec.Cmd{}, err
}
qemuCfg := cfg
qemuCfg.NetDevConfig.ID = fmt.Sprintf("%s-%s", qemuCfg.NetDevConfig.ID, id)
qemuCfg.VirtioScsiPciConfig.ID = fmt.Sprintf("%s-%s", qemuCfg.VirtioScsiPciConfig.ID, id)
qemuCfg.SevConfig.ID = fmt.Sprintf("%s-%s", qemuCfg.SevConfig.ID, id)
// Copy firmware vars file.
srcFile := qemuCfg.OVMFVarsConfig.File
dstFile := fmt.Sprintf("%s/%s-%s.fd", cfg.TmpFileLoc, firmwareVars, id)
err = internal.CopyFile(srcFile, dstFile)
if err != nil {
return &exec.Cmd{}, err
}
qemuCfg.OVMFVarsConfig.File = dstFile
// Copy img files.
srcFile = qemuCfg.DiskImgConfig.KernelFile
dstFile = fmt.Sprintf("%s/%s-%s", cfg.TmpFileLoc, KernelFile, id)
err = internal.CopyFile(srcFile, dstFile)
if err != nil {
return &exec.Cmd{}, err
}
qemuCfg.DiskImgConfig.KernelFile = dstFile
srcFile = qemuCfg.DiskImgConfig.RootFsFile
dstFile = fmt.Sprintf("%s/%s-%s.gz", cfg.TmpFileLoc, rootfsFile, id)
err = internal.CopyFile(srcFile, dstFile)
if err != nil {
return &exec.Cmd{}, err
}
qemuCfg.DiskImgConfig.RootFsFile = dstFile
exe, args, err := ExecutableAndArgs(qemuCfg)
if err != nil {
return &exec.Cmd{}, err
}
cmd, err := runQemuVM(exe, args)
if err != nil {
return cmd, err
}
return cmd, nil
}
func ExecutableAndArgs(cfg Config) (string, []string, error) {
exe, err := exec.LookPath(qemuRelPath)
if err != nil {
return "", nil, err
}
args := constructQemuArgs(cfg)
if cfg.UseSudo {
args = append([]string{exe}, args...)
exe = "sudo"
}
return exe, args, nil
}
func runQemuVM(exe string, args []string) (*exec.Cmd, error) {
cmd, err := internal.RunCmdStart(exe, args...)
if err != nil {
return nil, err
}
return cmd, nil
}