* Update protoc to v4.25.3 and streamline client identification
Protocol Buffer compiler `protoc` has been upgraded to version 4.25.3. This update unifies the version across multiple generated files to ensure compatibility and take advantage of any bug fixes and performance improvements in the new release.
Additionally, the client identification process has been refined. The redundant `WhoAmI` message and the corresponding checks have been removed from the gRPC server implementation. Clients are now identified via their address from the `Process` stream's context as soon as a connection is established, simplifying the code and potentially reducing handshake time.
This change sets the foundation for a leaner communication protocol between manager and agents, and could contribute to lower latencies in client-server interactions.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix goroutine bug in grpc server
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Update PROTOC_VERSION to 25.3
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor server reference in readme
Updated the deployment instructions in the README to point to the correct server location after consolidating test server documentation. Removed obsolete `manager-server` module and associated main entry point to align with the new architecture and streamline workflows.
Resolves issues with outdated links and cluttered repository structure.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add handler
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor gRPC and Protobuf integration for manager service
- Shifted Protobuf message definitions to a separate package `pkg/manager`.
- Updated references throughout the codebase to import and use the new package for gRPC service definitions.
- Enhanced AgentLog message with additional fields `level` and `timestamp`.
- Removed direct dependencies on old Protobuf-generated types in favor of the new package.
- Deleted obsolete Protobuf-generated files as they are now superseded by the new `pkg/manager`.
- Streamlined event publishing and gRPC handling in the manager service to use the updated Protobuf messages.
This refactoring improves modularity by centralizing Protobuf message definitions and decouples internal representation from the gRPC interface, aligning with best practices for microservice architecture. Additionally, the enriched logging structure paves the way for more detailed and fine-grained log analysis.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor vsock event/log handling and config
Streamlined event and log services in the manager by moving vsock listening functions out of `managerService` initialization and into dedicated `RetrieveAgentEventsLogs` methods. This change decouples the manager service creation from the actual start of log listening, adding clarity and flexibility in service management. Also moved logging middleware invocation outside of network handling loops to avoid unnecessary overhead.
Additionally, the agent's vsock port configuration is now dynamically passed to the `New` function in the `events` package instead of relying on a hardcoded constant, allowing for greater configurability and testability.
Finally, updated message structures for event and log sending to conform with the `ClientStreamMessage` definitions. These modifications should improve parsing and handling consistency and prepare our system for future enhancements related to inter-process communication.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix linting errors
Signed-off-by: SammyOina <sammyoina@gmail.com>
* correct path to generated files
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix comments
Signed-off-by: SammyOina <sammyoina@gmail.com>
* remove uneccessary comments
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enhanced gRPC server concurrency with errgroup
Refactored the gRPC server's `Process` method to handle incoming messages and service execution concurrently using an `errgroup.Group`. Added a context to the `grpcServer` struct and propagated it to support graceful shutdowns and better request handling. Changed the `Service` interface's `Run` method to use a channel for transmitting `ComputationRunReq`, facilitating non-blocking operation and streamlining the request-response cycle.
Asynchronous handling of stream requests with `errgroup` allows simultaneous processing and error management, improving the server's responsiveness and fault tolerance. This update paves the way for more robust service implementations that require concurrent operations.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* remove trailing space
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor GRPC manager service and client
The manager service and client have been restructured for stream communication, facilitating real-time agent events, logs, and run responses. The `Run` RPC is replaced by the `Process` stream RPC, enabling bidirectional streaming between clients and the manager service. This allows continuous interchange of different message types including `WhoAmIRequest`, `AgentLog`, `AgentEvent`, and `RunResponse`.
Several message types have been adjusted and new fields introduced, like `AgentPort` in `RunResponse` and various agent-config attributes including CA files and instance IDs, to support TLS client authentication and distinguish between agent instances.
We've also incorporated `google.protobuf.Timestamp` in `AgentEvent` for precise event logging. The client code reflects these modifications with updated method calls and stream handling logic for ongoing communication. Moreover, the updates necessitate corresponding changes throughout service, grpc, and sdk layers to interoperate with the new streaming approach.
The transition to streaming paves the way for a more interactive, flexible communication system that can accommodate future expansion and real-time monitoring features.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add computation id to run response
Signed-off-by: SammyOina <sammyoina@gmail.com>
* rename request
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor GRPC manager service and client
The manager service and client have been restructured for stream communication, facilitating real-time agent events, logs, and run responses. The `Run` RPC is replaced by the `Process` stream RPC, enabling bidirectional streaming between clients and the manager service. This allows continuous interchange of different message types including `WhoAmIRequest`, `AgentLog`, `AgentEvent`, and `RunResponse`.
Several message types have been adjusted and new fields introduced, like `AgentPort` in `RunResponse` and various agent-config attributes including CA files and instance IDs, to support TLS client authentication and distinguish between agent instances.
We've also incorporated `google.protobuf.Timestamp` in `AgentEvent` for precise event logging. The client code reflects these modifications with updated method calls and stream handling logic for ongoing communication. Moreover, the updates necessitate corresponding changes throughout service, grpc, and sdk layers to interoperate with the new streaming approach.
The transition to streaming paves the way for a more interactive, flexible communication system that can accommodate future expansion and real-time monitoring features.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix lint
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Update GitHub Actions to Latest Versions
Upgraded GitHub Actions 'checkout' to version 4 and 'setup-go' to version 5 across various workflow files to leverage the latest features and improvements for better performance and reliability. This also ensures compatibility with Go version 1.21.x which is specified in the workflows.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor event handling and logging
Reworked event and log processing to use channels instead of direct HTTP calls. Removed obsolete events package and consolidated event structures, leading to cleaner and more maintainable code. Updated agent events to use channels, enhanced error handling in log forwarding, and simplified manager `New` function signature to accept an event channel directly.
- Removed `events` and `agentevents` packages to reduce complexity.
- Replaced direct event server communication with internal channel usage.
- Introduced `AgentEvent` struct in events.go for standardized event objects.
- Adapted `managerService` to dispatch events and logs through channels.
- Streamlined manager construction by removing the now-unnecessary event service and host IP parameters.
This change results in a more robust and easier to extend event and log management system within the agent-manager interaction.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix ci
Signed-off-by: SammyOina <sammyoina@gmail.com>
* remove unused code
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add comments
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
The HTTP server-related code, documentation, and configurations have been removed as part of a shift towards prioritizing gRPC for service communication. This update includes deletions of HTTP host and port configs across various components, the manager HTTP API alongside its Swagger definition, and the removal of related scaffolding and utility code. This change simplifies the overall architecture and eliminates redundant HTTP support, focusing on optimizing gRPC performance and security features.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Simplify event handling and config
Streamlined event service interface by consolidating `SendEvent` and introducing `SendRaw`. Removed `notification_server_url` and `instance_id` parameters from several event publication calls to leverage centralized event construction. This change not only cleans up redundancy in event-related code but also simplifies the configuration data flow across the system, making it easier to manage and less error-prone. Uniform event generation now improves consistency and maintainability.
Refactored configuration management in the agent and manager services. Removed notifications URL from the agent configuration, relying on a simplification that assumes a single source of events. Updated Manager Port to VsockConfigPort for clarity and consistency across vsock communication.
These modifications should facilitate easier integration and extension of event and configuration systems in the future.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix lint
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor error handling in agent event forwarding
Introduced context and error channel handling to the agent event forwarding process. The logger now warns on errors during forward operations asynchronously, allowing for non-blocking error reporting. Additionally, reliance on the global logger was removed in favor of passing error information via channels, improving modularity and error flow control.
Resolves issue with silent forwarding failures by providing a means to alert system operators without halting the service. This enhancement makes the error reporting more robust and reactive while maintaining service continuity.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* remove unused field
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enhance agent logging via vsock connection
Redirected agent logging to use a vsock connection instead of standard output, improving the process isolation and enabling centralized log management. The change involved dialing to the specified vsock log port and initializing the logger with the vsock connection rather than stdout.
Additionally, the manager service now maintains a map of agent vsock cids to computation IDs, providing better tracking of computation resources. A routine to retrieve logs from agents was also initiated during the service setup to facilitate log collection.
Consequential to these changes is the removal of a redundant os package import in the agent's main.go, further cleaning up the dependencies.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fail gracefully
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Updated backoff strategy and VM configurations
- Added `github.com/cenkalti/backoff` to direct dependencies for robust retry logic in agent configuration sending.
- Modified the vsock logs port to align with the updated port range standards.
- Enclosed kernel console arguments in quotes to ensure proper parsing in QEMU configurations.
- Implemented exponential backoff when sending agent configurations to handle transient failures.
Refactors:
- Streamlined creation of `AgentConfig` within the computation setup to avoid unnecessary initializations when `c.AgentConfig` is not nil.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor command execution and improve argument construction
Consolidated the error handling in the command execution function for better readability. In the QEMU configuration, the argument assembly process is enhanced for clarity and correctness; the VNC parameter is now separate, and string quoting is handled properly for kernel parameters. These changes result in more maintainable code and prevent potential formatting issues during QEMU argument parsing.
Resolves issues with argument construction in QEMU config module.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refine default config handling and unpacking
Improved the agent configuration by dynamically setting default values for the log level and port if they are not specified in the incoming configuration. Also streamlined configuration unpacking in the endpoint and service layers, reducing redundancy and ensuring all required fields are correctly copied over to the Manager's configuration structure. This change ensures better fault tolerance and more maintainable code by handling edge cases where configuration values might be missing.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* rename dir
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix lint
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Ensure runRes.Empty() reflects non-empty state
Changed the always-true return value of the `runRes.Empty()` method to `false` to accurately indicate the presence of a response body. This adjustment ensures downstream handling of API responses aligns with actual content state.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Replace mglog with slog across codebase
Updated various components to replace the `mglog` logger implementation with the `slog` logger. This change affects logging initialization and calls throughout the codebase including the agent, manager, and internal server components. Transitioning to `slog` is part of a broader shift to standardize the logging mechanism to improve maintainability and consistency.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix bug in agent service and state
The commit fixes a bug in the agent service and state files.
Previously, the condition to check the state in the agent service
was incorrect. It was checking the state directly instead of
using the GetState() method. This has been fixed by using the
GetState() method to check the state.
Additionally, a new GetState() method has been added to the
StateMachine struct in the state file. This method retrieves
the current state by acquiring a lock and returning the state
value.
The changes have been tested and verified to resolve the bug
and improve the accuracy of state checking in the agent service.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix bug in agent state machine
The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix race condition in state machine
The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix race condition in StateMachine.Start()
The StateMachine.Start() method was experiencing a race condition
when multiple events were being processed concurrently. This was
caused by not properly locking and unlocking the state machine
before and after updating the state. This commit fixes the issue
by adding proper locking and unlocking around the state update
operation. Additionally, the logging statement has been updated
to include the previous and next states for better debugging.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add magistrala dep
Signed-off-by: SammyOina <sammyoina@gmail.com>
* remove mainflux
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger.
The commit message should be:
"Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger"
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix notification topic in agent service and update NATS ports in Docker environment variables
The agent service's notification topic was incorrectly set to "channels.manager" instead of "agent". This commit fixes the issue by updating the notification topic.
Additionally, the NATS ports in the Docker environment variables were incorrect. The COCOS_NATS_PORT and COCOS_NATS_HTTP_PORT have been updated to the correct values.
These changes ensure that the agent service uses the correct notification topic and the NATS ports are properly configured.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor Makefile and add new targets for building Docker images
The Makefile has been refactored to include new targets for building Docker images. The `make_docker` and `make_docker_dev` functions have been defined to handle the Docker build process. The `dockers` and `dockers_dev` targets have been added to build the Docker images for all services and development environments respectively.
This commit introduces changes to the Makefile to improve the build process and provide support for Docker images.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* update readme
Signed-off-by: SammyOina <sammyoina@gmail.com>
* wrap env vars
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor main.go in cmd/cli
The main.go file in cmd/cli has been refactored to improve code readability and maintainability. The defURL constant has been removed as it is no longer needed. Additionally, unnecessary whitespace has been removed.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix linting
Signed-off-by: sammy <sammyoina@gmail.com>
* rename cocos-ai to cocos
Signed-off-by: sammy <sammyoina@gmail.com>
* Updated README with NATS setup instructions and correct systemd path
Expanded the project's README to include setup instructions for the NATS server, which is now a necessary component for agent and manager communication. Additionally, the systemd service file path has been corrected from 'systemd' to 'init/systemd' ensuring the agent's proper installation and configuration as a daemon. This facilitates a smoother setup experience and reflects the dependency on NATS for push notifications.
Ref: Agent and manager communication enhancement
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Added message broker URL to agent service config
Included the environment variable for the message broker URL in the systemd service configuration to facilitate agent communication with the messaging system. This ensures the agent can connect to the designated message broker for event publishing and subscription.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* update docs to hal
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Remove TLS and timeout config from gRPC client
Refactored gRPC client and associated API code by removing unnecessary TLS configuration and timeout settings. Simplified the communication setup by trusting the environment to enforce security policies and handle operation durations, instead of hardcoding these within the application.
This brings about a cleaner, more maintainable codebase and shifts responsibility for security configurations out of the code, aligning with infrastructure-as-code practices and enabling easier scalability and environment-specific adjustments.
This change also affects the API documentation and example commands, which have been updated accordingly to reflect the simplification and to guide users with the streamlined setup process.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
Signed-off-by: sammy <sammyoina@gmail.com>
* Add agent address to run responses
The manager service's Run method now retrieves the agent address upon successful computation execution, providing more informative responses across gRPC, HTTP, and logging endpoints. This change improves service transparency by returning the agent's address to be used by client services, making the manager service's external communication more comprehensive. Updated the `RunResponse` structure in the corresponding protocol buffers definition and response handling in gRPC and HTTP APIs, ensuring that agent address information is serialized appropriately. It also necessitates a slight adjustment in the QEMU configuration to manage port forwarding rules more dynamically, simplifying the process as only agent-relevant ports are incremented with each new computation. This extra detail in responses aids in debugging and offers better integration capabilities for clients.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Allocate dynamic ports for VM guests and expand error handling
Refactored service initialization to accept host IP and incorporated dynamic port allocation for VM guests, replacing the prior static increment method. Introduced a new error type 'ErrFailedToAllocatePort' to capture instances where the system is unable to find a free port. Integrated a third-party error package for improved error wrapping and context. These changes prevent port conflicts between VM guests and enhance error diagnostics for service operations.
Resolves issue with static port allocation leading to conflicts.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Add HOST_IP to service configuration
Extend service configuration to include the host machine's IP address, allowing instances to be aware of their deployment environment. This update passes the new HostIP field to the service constructor, ensuring the service can now operate with host-specific logic.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Populate AgentAddress in gRPC Responses
Enhanced the gRPC encode/decode functions to properly populate the 'AgentAddress' field in 'RunResponse' objects. This ensures that consumers of the gRPC interface receive complete response data, which previously omitted the important 'AgentAddress' information. The change impacts both server-side response encoding and client-side response decoding, aligning the implementation with the expected interface contract.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix ci
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add status endpoint
Signed-off-by: sammy <sammyoina@gmail.com>
* feat: Update code generation tools to latest versions
Update the code generation tools, including protoc-gen-go and
protoc-gen-go-grpc, to their latest versions (v1.31.0 and v1.3.0,
respectively). This ensures compatibility with the latest features and
improvements. The updated tools also require gRPC-Go v1.32.0 or later.
The new versions bring important updates and bug fixes, enhancing the
performance and stability of the generated code. By staying up-to-date
with the latest tooling, we can take advantage of the latest
functionality and ensure a smooth development experience.
No code changes are included in this commit. These updates only impact
the code generation process.
Signed-off-by: sammy <sammyoina@gmail.com>
* Update Go version to 1.21.x
Signed-off-by: sammy <sammyoina@gmail.com>
* Refactor agent and manager services to publish event notifications
The refactoring includes changes to the agent and manager services to incorporate event notifications. By publishing events, the services can inform subscribers about the current state of the computation or any updates. Specifically, the `agentService` now includes a `cmpHash` field to store the SHA-256 hash of the computation, which is subsequently used when publishing events. The `agentService` and `managerService` now use the `publisher` interface to publish events to the topic "manager". Notably, the removed `pubsub.go` file is no longer necessary.
This commit improves the service architecture by allowing subscribers to receive relevant updates and monitor the progress of computations. It enhances the overall system by providing more transparency and enabling better coordination between the agent and manager services.
Signed-off-by: sammy <sammyoina@gmail.com>
* Improve generated Go file comparison in checkproto workflow
Refactor the file comparison logic in the checkproto workflow to use the `-p` flag instead of `-s` for improved accuracy. This change ensures that the generated Go files are thoroughly compared with the original ones, detecting any discrepancies and preventing out-of-sync files from passing the validation. By using the `-p` flag, we now check both the contents and the metadata of the files, providing more robust synchronization checks. This update enhances the reliability of the checkproto workflow and helps maintain consistency between the proto files and their corresponding generated Go files.
Signed-off-by: sammy <sammyoina@gmail.com>
* Update file comparison command to detect differences line by line
The code change updates the file comparison command used in the CI workflow to detect differences line by line instead of only reporting the first difference encountered. This change improves the accuracy of detecting inconsistencies between the original protobuf files and the generated Go files. Previously, only the first difference was reported, leading to potential missed issues. By comparing the files line by line, we can now detect and report all differences accurately. This change enhances the reliability of our CI pipeline and ensures that the generated Go files stay in sync with the protobuf files.
Signed-off-by: sammy <sammyoina@gmail.com>
* add event exporting to external server
Signed-off-by: sammy <sammyoina@gmail.com>
* feat: Add support for notification server URL
The commit adds a new environment variable, `COCOS_NOTIFICATION_SERVER_URL`, which allows specifying the server to receive notification events from the agent. This addition provides flexibility to configure the notification server URL based on the deployment environment. This change enables seamless integration with different notification server instances and enhances the extensibility of the system. It resolves the need to modify the code directly when changing the server URL.
Signed-off-by: sammy <sammyoina@gmail.com>
* Refactor gRPC client and server, remove unused handlers
The commit refactors the gRPC client and server code by removing the unused `nopDecoder` and `status` handlers from the client and server, respectively. This cleanup reduces code clutter and improves maintenance. No significant consequences are expected.
Signed-off-by: sammy <sammyoina@gmail.com>
* Ensure generated Go files stay in sync with proto files during the CI workflow
Fixes an issue in the CI workflow where proto files and their corresponding generated Go files were not being properly compared for synchronization. Previously, the `cmp -l` command was used, which only printed differing byte positions, leading to false negatives. This has been corrected by using `cmp -s` instead, which outputs nothing if the files are identical. This change ensures that any differences between the proto files and their generated Go files will be detected, helping to maintain consistency and accuracy in the codebase.
Signed-off-by: sammy <sammyoina@gmail.com>
* Enhance notification payload and endpoint
Extended the notification system to include 'status' and 'details' in the payload, improving traceability and debugging. Adapted the serialized JSON structure for clarity and added an 'originator' field to track the source service. Transitioned to a generalized event endpoint, facilitating a more streamlined event handling process.
Refactors POST request to a more appropriate endpoint and updates the notification service interface to reflect new payload requirements.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor event notification logic
Removed the legacy notifications package and consolidated event notification functionality using the new internal events service. Modified agent, manager, and main application code to use this service for consistent event reporting and error handling workflows across services. This change simplifies event management, improves error visibility, and allows for more maintainable code by centralizing event-related logic. The substitution of verbose state-specific publishEvent calls with generic status reporting aligns with the new service's capabilities.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Expand agent protobuf message types and improve error handling
The protobuf definition for agent messages has been updated to include an additional message type, facilitating future data structure expansions. Additionally, error handling for event sending in the main agent execution has been enhanced to log errors when sending 'init' events fail, ensuring issues are properly tracked. The unused `notificationTopic` constant in the manager service has been removed for cleaner code maintenance.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Optimize JSON parsing and fix header omission
Removed unnecessary unquoting of a JSON string before unmarshaling, streamlining the computation value extraction process. Also corrected a missing Content-Type header in the event sending function, ensuring proper handling of JSON requests by recipients. These changes improve performance and communication reliability.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* align vars
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Remove computation request timeout feature
The timeout feature for computation requests has been removed to simplify the computation execution flow. This involved changes across multiple files, including protobuf definitions, HTTP endpoint handling, and the internal computation logic. We eliminated the timeout field, associated logic, and error handling to ensure the system no longer supports timeouts for computations, mitigating any unintended timeout impacts on long-running processes.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: sammy <sammyoina@gmail.com>
Signed-off-by: SammyOina <sammyoina@gmail.com>
Standardized the variable name for computation parameter across various service and middleware implementations to improve code consistency. This enhances readability and maintainability.
Additionally, tracing functionality has been removed completely, signifying a potential shift in the telemetry strategy or removal of an unused feature. The run command associated with cli package has also been deleted, suggesting a change in the CLI tool's capabilities or its command handling strategy.
The service interface's Run method definition and its implementations now align with the updated parameter naming, ensuring coherence throughout the codebase.
Removed files:
- The entire `tracing` package has been deleted.
- Removed the `cli/run.go` file, indicating the removal of the 'run' command from the CLI.
Please note that these changes may affect any existing workflows or systems that rely on the removed tracing capabilities or the CLI run command, and appropriate adjustments will need to be made to accommodate these updates.
Refer to internal documentation or release notes for migration paths or additional context.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enable passing computation details to QEMU args
Refactored the QEMU argument construction to accept computation details as JSON. The changes include updating function signatures to pass an `agent.Computation` object and serialize it into the QEMU boot parameters. These modifications allow for custom computations to be specified per VM, facilitating more dynamic and configurable virtual machine creation. This update also integrates timing and data transformation necessary for the agent computation setup.
Resolves computation configurability for virtual machines.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Enable extraction of computation value from cmdline
Introduced a feature to parse the computation parameter directly from the machine's command line, thus enabling dynamic configuration of agent computations without the need for a preset environment. This change enhances the agent's ability to adapt to different computation requirements on startup. A fallback warning informs of cases where the computation parameter was not found, ensuring traceability of operational state during service initialization.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Remove RunRequest and RunResponse from Agent gRPC
Refactor the Agent service by deleting the RunRequest and RunResponse messages along with the corresponding method implementations in the gRPC interface. This simplifies the gRPC protocol by removing an obsolete RPC method, streamlining communication between services. The deletion also impacts the manager service, which no longer uses the removed RPC method. Consequently, the service definitions across related files like endpoints, servers, and clients have been adjusted. This adjustment enforces a more direct approach for starting computations, reflecting a shift toward manager-initiated computation execution.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor RunRequest to use structured Computation
The protobuf and associated service implementations for the RunRequest message were refactored to replace the raw Computation byte slice with a structured ComputationReq object. This allows clearer and more type-safe manipulation of computation requests. The grpc, http, and agent service layers were updated to build and parse ComputationReq accordingly. The ComputationReq structure includes details like IDs, names, time stamps, and metadata, forming a well-defined contract for computation tasks.
This change aligns with efforts to standardize request formats and improve clarity in inter-service communication. It impacts all systems interfacing with the RunRequest service and thus requires coordinated updates to the entire stack.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Initialize metadata maps and handle nil values
Improved the robustness of metadata handling in gRPC endpoints and SDK by initializing metadata maps and explicitly checking for nil values before converting them. This ensures that both the agent's gRPC endpoint and the SDK properly handle cases where metadata fields may be uninitialized or contain nil values, preventing potential null pointer exceptions.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor computation request handling
Refactored the endpoint to construct Computation object from gRPC request, incorporating structpb for metadata handling and timestamppb for StartTime and EndTime fields. The management service and API requests are also updated to align with these changes, improving type safety and ensuring data is correctly marshalled when making service calls.
Resolves data marshalling issues for computation requests.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* use singular
Signed-off-by: SammyOina <sammyoina@gmail.com>
* remove unuse fields
Signed-off-by: SammyOina <sammyoina@gmail.com>
* remove unused fields
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Add gRPC TLS config and update protoc versions
Enhanced manager service with TLS configuration options and timeout settings for gRPC communication with agent services. Updated corresponding protobuf definitions to include new fields for CA certificates, TLS status, and timeouts. The added TLS support ensures secure inter-service communication while flexible timeout configurations improve the robustness of network interactions.
The following adjustments and additions were made:
- Included new gRPC configuration fields (ca_certs, client_tls, timeout) for agent communication in protobuf definitions.
- Injected agent gRPC configuration into service endpoints, ensuring secure TLS setup and compliance with provided settings.
- Revised main service function signatures to accept the gRPC configuration object.
- Incremented port forwarding counters post-computation to avoid port conflicts.
- Conducted compatibility update of protobuf version comments to reflect minor version bump (v4.25.0 to v4.25.1).
This change impacts service deployment that requires proper configuration of TLS credentials and mindful determination of timeouts for efficient network use.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* update proto
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Set default timeout for agent configuration
Introduced a default timeout of 60 seconds in both gRPC and HTTP endpoints for agent configuration when none is specified. This change ensures that operations do not hang indefinitely and provides a reasonable default for client interactions. Additionally, the instantiation of a new agent client is now outside the retry loop to avoid repeated setup on transient failures.
Refactors service logic to optimize client connection handling by moving the agent client setup to occur before attempting retries, which should reduce overhead and improve clarity in error situations.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Subject: Move agent client creation into retry loop
Body:
Refactored the service manager's Run method to initialize the agent gRPC client inside the exponential backoff retry loop. This change addresses intermittent connection issues by reattempting client creation on temporary network failures, ensuring a robust setup before calling the Run method on the client.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add stringer
Signed-off-by: SammyOina <sammyoina@gmail.com>
* rename module to cocos
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix fmt.Stringer implementation in computations.go
The fmt.Stringer implementation for Datasets and Algorithms in computations.go was fixed to correctly use pointers.
This commit addresses the issue where the String() method for Datasets and Algorithms in computations.go was not correctly implemented. The fix ensures that the String() method now correctly marshals the data to JSON and returns the string representation.
The changes made in this commit will improve the functionality and accuracy of the String() method for Datasets and Algorithms.
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix bug in agent state machine
The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix race condition in state machine
The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix race condition in StateMachine.Start()
The StateMachine.Start() method was experiencing a race condition
when multiple events were being processed concurrently. This was
caused by not properly locking and unlocking the state machine
before and after updating the state. This commit fixes the issue
by adding proper locking and unlocking around the state update
operation. Additionally, the logging statement has been updated
to include the previous and next states for better debugging.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add magistrala dep
Signed-off-by: SammyOina <sammyoina@gmail.com>
* remove mainflux
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix agentService New function to include messaging.Publisher parameter
The agentService New function has been updated to include a messaging.Publisher parameter. This change allows the agent service to publish messages to a messaging system. The messaging.Publisher parameter has been added to the agentService struct and the New function signature has been updated accordingly. This change ensures that the agent service can communicate with other components using the messaging system.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Refactor service.go state functions
The commit refactors the state functions in the service.go file.
The functions for each state have been modified to use the svc.publishEvent
method to publish events with appropriate messages.
- Refactor state functions in service.go
- Use svc.publishEvent to publish events with messages for each state
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix computation run event publishing and add pubsub functionality
The computation run event publishing in the agent service was fixed to correctly call the publishEvent function. Additionally, the pubsub functionality was added to the manager package.
- Fixed computation run event publishing in agent service
- Added pubsub functionality to manager package
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix license header in pubsub.go file
The commit fixes the license header in the pubsub.go file.
The copyright and SPDX-License-Identifier have been added
to comply with the Apache-2.0 license.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger.
The commit message should be:
"Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger"
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE
The Makefile has been updated to fix an issue with setting the DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE variables. The USER_REPO variable is now used to generate the DOCKER_PROJECT name following the Docker Compose guidelines. Additionally, the COCOS_MESSAGE_BROKER_TYPE variable is now properly set to "nats" if it is empty. This ensures that the correct values are used when compiling and installing the service.
Summary:
Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE
Details:
- Update USER_REPO variable to generate DOCKER_PROJECT name
- Set COCOS_MESSAGE_BROKER_TYPE to "nats" if empty
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix Makefile Docker profile assignment and build flags
The Makefile was updated to fix the assignment of the Docker profile and build flags. The Docker profile is now assigned based on the value of COCOS_MESSAGE_BROKER_TYPE, and if it is not provided, the default value is set to "nats". The build flags were also updated to include the COCOS_MESSAGE_BROKER_TYPE value as a tag for the Go build process.
This commit addresses the issue with the Docker profile assignment and ensures that the correct build flags are used during the build process.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* fix makefile
Signed-off-by: SammyOina <sammyoina@gmail.com>
* Fix notification topic in agent service and update NATS ports in Docker environment variables
The agent service's notification topic was incorrectly set to "channels.manager" instead of "agent". This commit fixes the issue by updating the notification topic.
Additionally, the NATS ports in the Docker environment variables were incorrect. The COCOS_NATS_PORT and COCOS_NATS_HTTP_PORT have been updated to the correct values.
These changes ensure that the agent service uses the correct notification topic and the NATS ports are properly configured.
Signed-off-by: SammyOina <sammyoina@gmail.com>
* add pubsub
Signed-off-by: SammyOina <sammyoina@gmail.com>
* update protoc
Signed-off-by: SammyOina <sammyoina@gmail.com>
---------
Signed-off-by: SammyOina <sammyoina@gmail.com>
Sammy Kerata Oina
Darko Draskovic
rodneyosodo