Commit Graph

85 Commits

Author SHA1 Message Date
Sammy Kerata Oina 34c3bbdbd8 COCOS-55 - Pass agent logs to manager via vsock (#64)
* Simplify event handling and config

Streamlined event service interface by consolidating `SendEvent` and introducing `SendRaw`. Removed `notification_server_url` and `instance_id` parameters from several event publication calls to leverage centralized event construction. This change not only cleans up redundancy in event-related code but also simplifies the configuration data flow across the system, making it easier to manage and less error-prone. Uniform event generation now improves consistency and maintainability.

Refactored configuration management in the agent and manager services. Removed notifications URL from the agent configuration, relying on a simplification that assumes a single source of events. Updated Manager Port to VsockConfigPort for clarity and consistency across vsock communication.

These modifications should facilitate easier integration and extension of event and configuration systems in the future.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor error handling in agent event forwarding

Introduced context and error channel handling to the agent event forwarding process. The logger now warns on errors during forward operations asynchronously, allowing for non-blocking error reporting. Additionally, reliance on the global logger was removed in favor of passing error information via channels, improving modularity and error flow control.

Resolves issue with silent forwarding failures by providing a means to alert system operators without halting the service. This enhancement makes the error reporting more robust and reactive while maintaining service continuity.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove unused field

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enhance agent logging via vsock connection

Redirected agent logging to use a vsock connection instead of standard output, improving the process isolation and enabling centralized log management. The change involved dialing to the specified vsock log port and initializing the logger with the vsock connection rather than stdout.

Additionally, the manager service now maintains a map of agent vsock cids to computation IDs, providing better tracking of computation resources. A routine to retrieve logs from agents was also initiated during the service setup to facilitate log collection.

Consequential to these changes is the removal of a redundant os package import in the agent's main.go, further cleaning up the dependencies.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fail gracefully

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Updated backoff strategy and VM configurations

- Added `github.com/cenkalti/backoff` to direct dependencies for robust retry logic in agent configuration sending.
- Modified the vsock logs port to align with the updated port range standards.
- Enclosed kernel console arguments in quotes to ensure proper parsing in QEMU configurations.
- Implemented exponential backoff when sending agent configurations to handle transient failures.

Refactors:
- Streamlined creation of `AgentConfig` within the computation setup to avoid unnecessary initializations when `c.AgentConfig` is not nil.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor command execution and improve argument construction

Consolidated the error handling in the command execution function for better readability. In the QEMU configuration, the argument assembly process is enhanced for clarity and correctness; the VNC parameter is now separate, and string quoting is handled properly for kernel parameters. These changes result in more maintainable code and prevent potential formatting issues during QEMU argument parsing.

Resolves issues with argument construction in QEMU config module.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refine default config handling and unpacking

Improved the agent configuration by dynamically setting default values for the log level and port if they are not specified in the incoming configuration. Also streamlined configuration unpacking in the endpoint and service layers, reducing redundancy and ensuring all required fields are correctly copied over to the Manager's configuration structure. This change ensures better fault tolerance and more maintainable code by handling edge cases where configuration values might be missing.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* rename dir

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix lint

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Ensure runRes.Empty() reflects non-empty state

Changed the always-true return value of the `runRes.Empty()` method to `false` to accurately indicate the presence of a response body. This adjustment ensures downstream handling of API responses aligns with actual content state.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Replace mglog with slog across codebase

Updated various components to replace the `mglog` logger implementation with the `slog` logger. This change affects logging initialization and calls throughout the codebase including the agent, manager, and internal server components. Transitioning to `slog` is part of a broader shift to standardize the logging mechanism to improve maintainability and consistency.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-22 14:13:47 +01:00
Sammy Kerata Oina 55afe4c038 COCOS-49 - Pass agent configuration and computation via vsock (#57)
* Optimize QEMU launch and add V-sock support

Refactored QEMU argument construction and launching logic by removing the dependency on 'agent.Computation'. This simplification makes the VM creation process more streamlined. Additionally, introduced V-sock capabilities in the QEMU configuration to facilitate improved guest-host communication. Updated the README to include kernel module setup instructions for the new V-sock feature.

The V-sock implementation enables VMs to use a consistent communication channel that is not affected by network configuration changes, enhancing reliability and potential interoperability with host services. It's important to ensure that the necessary kernel modules are loaded as part of the setup process, as documented.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add vsock-based communication to manager

Introduced virtual socket (vsock) communication abilities in the manager package by implementing a new socket service. This includes establishing a vsock listener and stub methods for sending computation results and cleaning up resources. The addition provides the groundwork for interprocess communication between guest and host in virtualized environments.

- Integrated the `mdlayher/vsock` library for handling virtual socket operations.
- Created a new `sockService` struct to encapsulate vsock listener handling.
- Implemented `NewVsock` constructor to initialize the listener with domain value `3`.
- Added placeholder methods for future computation sending and service closing logic.

This enhancement targets scenarios where efficient VM-to-host communication is required.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove env

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor agent config and use vsock

Introduce `AgentConfig` struct to group agent-related configurations, and update `Computation` struct to include the new `AgentConfig` field. Replace command-line computation extraction with vsock-based config retrieval for robustness and decoupling. The agent configuration is now read from a vsock connection during runtime, allowing for more dynamic and flexible deployments. Adjusted the main agent application logic to support these configuration changes, and corresponding changes have been made in the manager to facilitate vsock communication.

This approach aligns with modern practices for microservices by streamlining configuration management and reducing reliance on static command-line parameters. Moreover, it enhances the scalability of the agent service by allowing configuration to be managed externally.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor agent config and remove deprecated code

Consolidated agent configuration management into a single `AgentConfig` message and pruned deprecated Protobuf `ComputationReq`, `DatasetReq`, and `AlgorithmReq` messages. Adapted corresponding manager service logic to the new configuration structure. These modifications align with updated manager API schema, facilitate clearer configuration handling, and improve maintainability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* send configuration

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Switch agent to listen mode for manager connections

Previously, the agent established a connection to the manager using a direct dial. This change shifts the setup to where the agent listens on a specified port and accepts incoming connections. It ensures that the agent properly handles incoming requests by initiating a listening socket and waiting for the manager to connect, enhancing the system's flexibility in connection management. This adjustment also includes graceful closure of the listening socket.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-19 17:24:51 +01:00
Sammy Kerata Oina c29ef354fa COCOS-24 - Add events service via http (#27)
* add status endpoint

Signed-off-by: sammy <sammyoina@gmail.com>

* feat: Update code generation tools to latest versions

Update the code generation tools, including protoc-gen-go and
protoc-gen-go-grpc, to their latest versions (v1.31.0 and v1.3.0,
respectively). This ensures compatibility with the latest features and
improvements. The updated tools also require gRPC-Go v1.32.0 or later.

The new versions bring important updates and bug fixes, enhancing the
performance and stability of the generated code. By staying up-to-date
with the latest tooling, we can take advantage of the latest
functionality and ensure a smooth development experience.

No code changes are included in this commit. These updates only impact
the code generation process.

Signed-off-by: sammy <sammyoina@gmail.com>

* Update Go version to 1.21.x

Signed-off-by: sammy <sammyoina@gmail.com>

* Refactor agent and manager services to publish event notifications

The refactoring includes changes to the agent and manager services to incorporate event notifications. By publishing events, the services can inform subscribers about the current state of the computation or any updates. Specifically, the `agentService` now includes a `cmpHash` field to store the SHA-256 hash of the computation, which is subsequently used when publishing events. The `agentService` and `managerService` now use the `publisher` interface to publish events to the topic "manager". Notably, the removed `pubsub.go` file is no longer necessary.

This commit improves the service architecture by allowing subscribers to receive relevant updates and monitor the progress of computations. It enhances the overall system by providing more transparency and enabling better coordination between the agent and manager services.

Signed-off-by: sammy <sammyoina@gmail.com>

* Improve generated Go file comparison in checkproto workflow

Refactor the file comparison logic in the checkproto workflow to use the `-p` flag instead of `-s` for improved accuracy. This change ensures that the generated Go files are thoroughly compared with the original ones, detecting any discrepancies and preventing out-of-sync files from passing the validation. By using the `-p` flag, we now check both the contents and the metadata of the files, providing more robust synchronization checks. This update enhances the reliability of the checkproto workflow and helps maintain consistency between the proto files and their corresponding generated Go files.

Signed-off-by: sammy <sammyoina@gmail.com>

* Update file comparison command to detect differences line by line

The code change updates the file comparison command used in the CI workflow to detect differences line by line instead of only reporting the first difference encountered. This change improves the accuracy of detecting inconsistencies between the original protobuf files and the generated Go files. Previously, only the first difference was reported, leading to potential missed issues. By comparing the files line by line, we can now detect and report all differences accurately. This change enhances the reliability of our CI pipeline and ensures that the generated Go files stay in sync with the protobuf files.

Signed-off-by: sammy <sammyoina@gmail.com>

* add event exporting to external server

Signed-off-by: sammy <sammyoina@gmail.com>

* feat: Add support for notification server URL

The commit adds a new environment variable, `COCOS_NOTIFICATION_SERVER_URL`, which allows specifying the server to receive notification events from the agent. This addition provides flexibility to configure the notification server URL based on the deployment environment. This change enables seamless integration with different notification server instances and enhances the extensibility of the system. It resolves the need to modify the code directly when changing the server URL.

Signed-off-by: sammy <sammyoina@gmail.com>

* Refactor gRPC client and server, remove unused handlers

The commit refactors the gRPC client and server code by removing the unused `nopDecoder` and `status` handlers from the client and server, respectively. This cleanup reduces code clutter and improves maintenance. No significant consequences are expected.

Signed-off-by: sammy <sammyoina@gmail.com>

* Ensure generated Go files stay in sync with proto files during the CI workflow

Fixes an issue in the CI workflow where proto files and their corresponding generated Go files were not being properly compared for synchronization. Previously, the `cmp -l` command was used, which only printed differing byte positions, leading to false negatives. This has been corrected by using `cmp -s` instead, which outputs nothing if the files are identical. This change ensures that any differences between the proto files and their generated Go files will be detected, helping to maintain consistency and accuracy in the codebase.

Signed-off-by: sammy <sammyoina@gmail.com>

* Enhance notification payload and endpoint

Extended the notification system to include 'status' and 'details' in the payload, improving traceability and debugging. Adapted the serialized JSON structure for clarity and added an 'originator' field to track the source service. Transitioned to a generalized event endpoint, facilitating a more streamlined event handling process.

Refactors POST request to a more appropriate endpoint and updates the notification service interface to reflect new payload requirements.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor event notification logic

Removed the legacy notifications package and consolidated event notification functionality using the new internal events service. Modified agent, manager, and main application code to use this service for consistent event reporting and error handling workflows across services. This change simplifies event management, improves error visibility, and allows for more maintainable code by centralizing event-related logic. The substitution of verbose state-specific publishEvent calls with generic status reporting aligns with the new service's capabilities.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Expand agent protobuf message types and improve error handling

The protobuf definition for agent messages has been updated to include an additional message type, facilitating future data structure expansions. Additionally, error handling for event sending in the main agent execution has been enhanced to log errors when sending 'init' events fail, ensuring issues are properly tracked. The unused `notificationTopic` constant in the manager service has been removed for cleaner code maintenance.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Optimize JSON parsing and fix header omission

Removed unnecessary unquoting of a JSON string before unmarshaling, streamlining the computation value extraction process. Also corrected a missing Content-Type header in the event sending function, ensuring proper handling of JSON requests by recipients. These changes improve performance and communication reliability.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* align vars

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Remove computation request timeout feature

The timeout feature for computation requests has been removed to simplify the computation execution flow. This involved changes across multiple files, including protobuf definitions, HTTP endpoint handling, and the internal computation logic. We eliminated the timeout field, associated logic, and error handling to ensure the system no longer supports timeouts for computations, mitigating any unintended timeout impacts on long-running processes.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: sammy <sammyoina@gmail.com>
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-11 14:37:14 +01:00
Sammy Kerata Oina b55b6c9734 NOISSUE - Refactor computation parameter naming and remove tracing (#42)
Standardized the variable name for computation parameter across various service and middleware implementations to improve code consistency. This enhances readability and maintainability.

Additionally, tracing functionality has been removed completely, signifying a potential shift in the telemetry strategy or removal of an unused feature. The run command associated with cli package has also been deleted, suggesting a change in the CLI tool's capabilities or its command handling strategy.

The service interface's Run method definition and its implementations now align with the updated parameter naming, ensuring coherence throughout the codebase.

Removed files:
- The entire `tracing` package has been deleted.
- Removed the `cli/run.go` file, indicating the removal of the 'run' command from the CLI.

Please note that these changes may affect any existing workflows or systems that rely on the removed tracing capabilities or the CLI run command, and appropriate adjustments will need to be made to accommodate these updates.

Refer to internal documentation or release notes for migration paths or additional context.

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-10 11:52:37 +01:00
Sammy Kerata Oina 4d9f7b1a45 COCOS-25 - Remove networking between manager and agent (#43)
* Enable passing computation details to QEMU args

Refactored the QEMU argument construction to accept computation details as JSON. The changes include updating function signatures to pass an `agent.Computation` object and serialize it into the QEMU boot parameters. These modifications allow for custom computations to be specified per VM, facilitating more dynamic and configurable virtual machine creation. This update also integrates timing and data transformation necessary for the agent computation setup.

Resolves computation configurability for virtual machines.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Enable extraction of computation value from cmdline

Introduced a feature to parse the computation parameter directly from the machine's command line, thus enabling dynamic configuration of agent computations without the need for a preset environment. This change enhances the agent's ability to adapt to different computation requirements on startup. A fallback warning informs of cases where the computation parameter was not found, ensuring traceability of operational state during service initialization.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Remove RunRequest and RunResponse from Agent gRPC

Refactor the Agent service by deleting the RunRequest and RunResponse messages along with the corresponding method implementations in the gRPC interface. This simplifies the gRPC protocol by removing an obsolete RPC method, streamlining communication between services. The deletion also impacts the manager service, which no longer uses the removed RPC method. Consequently, the service definitions across related files like endpoints, servers, and clients have been adjusted. This adjustment enforces a more direct approach for starting computations, reflecting a shift toward manager-initiated computation execution.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-09 20:07:45 +01:00
Sammy Kerata Oina 46a91ccbc3 remove nats (#41)
Signed-off-by: SammyOina <sammyoina@gmail.com>
2024-01-09 19:56:31 +01:00
Sammy Kerata Oina 18508796c1 add linters (#23)
- switchTrue
      - importShadow
      - httpNoBody
      - paramTypeCombine
      - emptyStringTest
      - builtinShadow
      - exposedSyncMutex
      - importas

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-24 18:59:42 +01:00
Sammy Kerata Oina 5adf0349a3 COCOS-9 - Dockerize manager (#18)
* Fix bug in agent service and state

The commit fixes a bug in the agent service and state files.
Previously, the condition to check the state in the agent service
was incorrect. It was checking the state directly instead of
using the GetState() method. This has been fixed by using the
GetState() method to check the state.

Additionally, a new GetState() method has been added to the
StateMachine struct in the state file. This method retrieves
the current state by acquiring a lock and returning the state
value.

The changes have been tested and verified to resolve the bug
and improve the accuracy of state checking in the agent service.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix bug in agent state machine

The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in state machine

The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in StateMachine.Start()

The StateMachine.Start() method was experiencing a race condition
when multiple events were being processed concurrently. This was
caused by not properly locking and unlocking the state machine
before and after updating the state. This commit fixes the issue
by adding proper locking and unlocking around the state update
operation. Additionally, the logging statement has been updated
to include the previous and next states for better debugging.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add magistrala dep

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove mainflux

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger.

The commit message should be:

"Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger"

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE

The Makefile has been updated to fix an issue with setting the DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE variables. The USER_REPO variable is now used to generate the DOCKER_PROJECT name following the Docker Compose guidelines. Additionally, the COCOS_MESSAGE_BROKER_TYPE variable is now properly set to "nats" if it is empty. This ensures that the correct values are used when compiling and installing the service.

Summary:
Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE

Details:
- Update USER_REPO variable to generate DOCKER_PROJECT name
- Set COCOS_MESSAGE_BROKER_TYPE to "nats" if empty

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix notification topic in agent service and update NATS ports in Docker environment variables

The agent service's notification topic was incorrectly set to "channels.manager" instead of "agent". This commit fixes the issue by updating the notification topic.

Additionally, the NATS ports in the Docker environment variables were incorrect. The COCOS_NATS_PORT and COCOS_NATS_HTTP_PORT have been updated to the correct values.

These changes ensure that the agent service uses the correct notification topic and the NATS ports are properly configured.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add Dockerfiles for production and development environments

This commit adds two new Dockerfiles, one for the production environment and one for the development environment. The production Dockerfile sets up the necessary dependencies and builds the application, while the development Dockerfile simply copies the built application. Both Dockerfiles include the necessary SSL certificates for the application to work properly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor Makefile and add new targets for building Docker images

The Makefile has been refactored to include new targets for building Docker images. The `make_docker` and `make_docker_dev` functions have been defined to handle the Docker build process. The `dockers` and `dockers_dev` targets have been added to build the Docker images for all services and development environments respectively.

This commit introduces changes to the Makefile to improve the build process and provide support for Docker images.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix makefile to generate Docker images for services

- Added DOCKERS and DOCKERS_DEV variables to generate Docker images for services
- Updated the dockers target to include DOCKERS
- Updated the dockers_dev target to include DOCKERS_DEV
- Updated the $(DOCKERS) and $(DOCKERS_DEV) targets to call the respective make_docker and make_docker_dev functions

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor Makefile and Dockerfile

The Makefile has been refactored to remove unnecessary sed commands and improve code readability. The changes include:

- Removed sed commands related to changing the broker in docker-compose.yml for nats and rabbitmq profiles.
- Removed the eer target from the Makefile.

The Dockerfile has been modified to include the WORKDIR directive.

These changes improve the maintainability and readability of the codebase.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix NATS URL in .env file and add Manager configuration in docker-compose.yml

The NATS URL in the .env file was updated to use the correct hostname for the broker. Additionally, the Manager configuration was added to the docker-compose.yml file.

This commit fixes the NATS URL in the .env file and adds the necessary Manager configuration to the docker-compose.yml file.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix incorrect host and port values in Makefile and .env

The commit updates the host and port values in the Makefile and .env files to fix incorrect values. The host and port values are updated to "cocos-manager" and "7003" respectively. This ensures that the correct host and port are used for the Manager HTTP and gRPC services.

This commit resolves the issue with the incorrect host and port values and ensures that the Manager services are configured correctly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Dockerfile and docker-compose.yml configurations

The Dockerfile and docker-compose.yml configurations have been fixed to address the following issues:

- Removed unnecessary COPY commands in the Dockerfile.
- Removed the "privileged" flag in the docker-compose.yml file.

These changes ensure that the Docker image is built correctly and the container is launched with the appropriate configurations.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove manual assignment

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update protoc CI

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add paths

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove profiles

Signed-off-by: SammyOina <sammyoina@gmail.com>

* restore default config

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Docker project name in Makefile

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-24 18:57:31 +01:00
Sammy Kerata Oina 0f831706f9 NOISSUE - Add CoCos-AI Manager API specification (#19)
* Add CoCos-AI Manager API specification

This commit adds the CoCos-AI Manager API specification
in the form of a YAML file. The specification includes
information about the title, description, contact,
license, and version of the API. It also defines the
servers where the API is hosted and the paths and
operations available, such as running computation on
a virtual machine.

The API specification is based on OpenAPI 3.0.1 and
provides a clear and concise overview of the CoCos-AI
Manager service.

The commit also includes a link to the CoCos-AI repository
and the license information.

This commit is necessary to provide a clear and documented
API specification for the CoCos-AI Manager service.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor go.mod and go.sum files

The go.mod and go.sum files have been refactored to remove the go.opentelemetry.io/contrib/propagators/jaeger package, which is no longer needed. This package was causing compatibility issues with the current version of the project. The refactoring ensures that the project is using the latest compatible versions of the required dependencies.

This commit removes the go.opentelemetry.io/contrib/propagators/jaeger package from the go.mod file and updates the go.sum file accordingly.

Note: The go.mod file now uses go.opentelemetry.io/otel v1.19.0 and go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.19.0.

Please review the changes to ensure compatibility and functionality.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix typo

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-24 18:54:54 +01:00
Sammy Kerata Oina 298878c96d NOISSUE - Rename module to cocos (#22)
* add stringer

Signed-off-by: SammyOina <sammyoina@gmail.com>

* rename module to cocos

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix fmt.Stringer implementation in computations.go

The fmt.Stringer implementation for Datasets and Algorithms in computations.go was fixed to correctly use pointers.

This commit addresses the issue where the String() method for Datasets and Algorithms in computations.go was not correctly implemented. The fix ensures that the String() method now correctly marshals the data to JSON and returns the string representation.

The changes made in this commit will improve the functionality and accuracy of the String() method for Datasets and Algorithms.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-22 13:15:19 +01:00
Sammy Kerata Oina 4ea9ff6531 NOISSUE - Add message broker on agent and manager (#17)
* Fix bug in agent state machine

The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in state machine

The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in StateMachine.Start()

The StateMachine.Start() method was experiencing a race condition
when multiple events were being processed concurrently. This was
caused by not properly locking and unlocking the state machine
before and after updating the state. This commit fixes the issue
by adding proper locking and unlocking around the state update
operation. Additionally, the logging statement has been updated
to include the previous and next states for better debugging.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add magistrala dep

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove mainflux

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix agentService New function to include messaging.Publisher parameter

The agentService New function has been updated to include a messaging.Publisher parameter. This change allows the agent service to publish messages to a messaging system. The messaging.Publisher parameter has been added to the agentService struct and the New function signature has been updated accordingly. This change ensures that the agent service can communicate with other components using the messaging system.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Refactor service.go state functions

The commit refactors the state functions in the service.go file.
The functions for each state have been modified to use the svc.publishEvent
method to publish events with appropriate messages.

- Refactor state functions in service.go
- Use svc.publishEvent to publish events with messages for each state

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix computation run event publishing and add pubsub functionality

The computation run event publishing in the agent service was fixed to correctly call the publishEvent function. Additionally, the pubsub functionality was added to the manager package.

- Fixed computation run event publishing in agent service
- Added pubsub functionality to manager package

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix license header in pubsub.go file

The commit fixes the license header in the pubsub.go file.
The copyright and SPDX-License-Identifier have been added
to comply with the Apache-2.0 license.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger.

The commit message should be:

"Add Docker environment variables for Nats, RabbitMQ, Message Broker, and Jaeger"

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE

The Makefile has been updated to fix an issue with setting the DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE variables. The USER_REPO variable is now used to generate the DOCKER_PROJECT name following the Docker Compose guidelines. Additionally, the COCOS_MESSAGE_BROKER_TYPE variable is now properly set to "nats" if it is empty. This ensures that the correct values are used when compiling and installing the service.

Summary:
Fix Makefile to properly set DOCKER_PROJECT and COCOS_MESSAGE_BROKER_TYPE

Details:
- Update USER_REPO variable to generate DOCKER_PROJECT name
- Set COCOS_MESSAGE_BROKER_TYPE to "nats" if empty

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Makefile Docker profile assignment and build flags

The Makefile was updated to fix the assignment of the Docker profile and build flags. The Docker profile is now assigned based on the value of COCOS_MESSAGE_BROKER_TYPE, and if it is not provided, the default value is set to "nats". The build flags were also updated to include the COCOS_MESSAGE_BROKER_TYPE value as a tag for the Go build process.

This commit addresses the issue with the Docker profile assignment and ensures that the correct build flags are used during the build process.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix makefile

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix notification topic in agent service and update NATS ports in Docker environment variables

The agent service's notification topic was incorrectly set to "channels.manager" instead of "agent". This commit fixes the issue by updating the notification topic.

Additionally, the NATS ports in the Docker environment variables were incorrect. The COCOS_NATS_PORT and COCOS_NATS_HTTP_PORT have been updated to the correct values.

These changes ensure that the agent service uses the correct notification topic and the NATS ports are properly configured.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* add pubsub

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update protoc

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-10 16:12:20 +01:00
Sammy Kerata Oina 4958f9468f NOISSUE - Implement State-machine (#16)
* add state machine

Signed-off-by: SammyOina <sammyoina@gmail.com>

* This commit addresses the following changes:

- In agent/service.go:
  - Added a goroutine to start the StateMachine
  - Sent the start event to the StateMachine

- In agent/state.go:
  - Added a copyright notice and license identifier

These changes ensure that the StateMachine starts correctly and that the agent/state.go file includes the necessary copyright and license information.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in agent service initialization

The commit fixes a race condition in the agent service initialization.
Previously, the `New` function in `agent/service.go` would start the state machine
goroutine before initializing the `sm` field. This could lead to a race condition
where the state machine would access uninitialized fields.

To fix this, the `New` function now takes a `context.Context` argument and passes it
to the `Start` method of the state machine. Additionally, a `sync.Mutex` has been
added to the `StateMachine` struct to ensure thread safety.

This commit addresses the race condition and ensures proper initialization of the agent service.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix sending events when algorithms and datasets are received

This commit fixes a bug where events were not being sent when algorithms and datasets were received in the agent service. The bug was causing the events to not be triggered, leading to incorrect behavior. This commit adds the necessary code to send the events when algorithms and datasets are received, ensuring that the events are properly triggered.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix bug: computation results not ready

The computation results were not being marked as ready,
resulting in an error when trying to access them.
This commit fixes the issue by adding a check for
the computation results before returning them.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix bug in agent service and state

The commit fixes a bug in the agent service and state files.
Previously, the condition to check the state in the agent service
was incorrect. It was checking the state directly instead of
using the GetState() method. This has been fixed by using the
GetState() method to check the state.

Additionally, a new GetState() method has been added to the
StateMachine struct in the state file. This method retrieves
the current state by acquiring a lock and returning the state
value.

The changes have been tested and verified to resolve the bug
and improve the accuracy of state checking in the agent service.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* check for time outs

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix bug in agent state machine

The bug in the agent state machine caused an error when attempting an invalid transition. This commit fixes the bug by properly locking and unlocking the state machine before and after transitioning to the next state. Additionally, the logger now correctly logs the current and next state during a valid transition.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in state machine

The commit fixes a race condition in the state machine implementation in the `Start` method. The race condition occurs when multiple goroutines try to access and modify the state concurrently. To fix this, a mutex lock and unlock are added around the critical sections of code to ensure exclusive access to the state variable. This prevents race conditions and ensures the state transitions are executed correctly.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix race condition in StateMachine.Start()

The StateMachine.Start() method was experiencing a race condition
when multiple events were being processed concurrently. This was
caused by not properly locking and unlocking the state machine
before and after updating the state. This commit fixes the issue
by adding proper locking and unlocking around the state update
operation. Additionally, the logging statement has been updated
to include the previous and next states for better debugging.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-10 14:41:05 +01:00
Sammy Kerata Oina 8c7d4a76ec NOISSUE - Remove HTTP server from agent and update agent service (#15)
* Update agent.pb.go to include a new field "Provider" in the AlgoRequest struct.

Signed-off-by: SammyOina <sammyoina@gmail.com>

* fix build

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix undeclared consumer error in agent service

The commit fixes an error in the agent service where an undeclared consumer was requesting results in the computation manifest. The error is resolved by adding an error message for the undeclared consumer case.

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-11-10 13:52:25 +01:00
Sammy Kerata Oina f1f8f95653 NOISSUE - Fix dependencies (#13)
* vendor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Return agent changes

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Add missing import

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* remove vendor

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Fix formatting

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Formatting errors

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>

* Update agent/api/grpc/client.go

Signed off: WashingtonKK

washingtonkigan@gmail.com

Co-authored-by: Sammy Kerata Oina <44265300+SammyOina@users.noreply.github.com>

* add linters and fix

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update ci

Signed-off-by: SammyOina <sammyoina@gmail.com>

* remove deprecated dependencies and
 use local agent and manager

Signed-off-by: SammyOina <sammyoina@gmail.com>

* update mainflux

Signed-off-by: SammyOina <sammyoina@gmail.com>

* Fix Jaeger URL in agent and manager main.go files

The Jaeger URL in the agent and manager main.go files was incorrect. This commit fixes the Jaeger URL by updating it to "http://localhost::4318/v1/traces".

Signed-off-by: SammyOina <sammyoina@gmail.com>

---------

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
Signed-off-by: SammyOina <sammyoina@gmail.com>
Co-authored-by: WashingtonKK <washingtonkigan@gmail.com>
2023-10-27 22:12:53 +02:00
SammyOina 7657f8bd7a fix build
Signed-off-by: SammyOina <sammyoina@gmail.com>
2023-10-19 12:22:52 +03:00
rodneyosodo d1b40f9a06 Remove MFXKIT Dep
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-26 18:01:14 +03:00
rodneyosodo e46a2f137b Move From OpenTracing To OpenTelemetry
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 17:03:29 +03:00
rodneyosodo fb596ab1ee Add Service Layer Tracing
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-25 16:07:18 +03:00
rodneyosodo 7ac8372720 Rename StopHandler
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 18:57:44 +03:00
rodneyosodo 76b2bed007 Fix Remarks
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 15:52:00 +03:00
rodneyosodo 885f4c9be1 Move HTTP and GRPC Server Implementations To Internal
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 14:57:05 +03:00
rodneyosodo 8505deedc6 Update Mainflux Module
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 12:16:30 +03:00
rodneyosodo c09383ac72 Add Metric To Internal Package
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-24 12:00:28 +03:00
rodneyosodo 2c354c3977 Remove MFXKIT Env Vars
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-21 19:18:26 +03:00
rodneyosodo d59b9293cb Initialixe Epty Vars More Idomatic
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-21 19:02:54 +03:00
rodneyosodo b5e64528b1 Make config items Public
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-21 18:55:30 +03:00
rodneyosodo f18e82efa7 Change How To Load Env Variables
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-21 18:20:40 +03:00
rodneyosodo f4e71e74b2 Initial Commit: Add CI
Signed-off-by: rodneyosodo <blackd0t@protonmail.com>
2023-07-10 15:06:45 +03:00
fbugarski 93b47a5897 Add initial structs for CLI commands
Signed-off-by: fbugarski <filipbugarski@gmail.com>
2023-06-26 11:53:02 +02:00
Darko Draskovic 5030a20af6 Remove redundant agent dir
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-06-12 15:18:18 +02:00
Darko Draskovic 90cea0d143 Add copy files and run computation instructions
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-18 18:42:51 +02:00
Darko Draskovic f02a500c2d Fix agent rc script to use grpc addr visible from host
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-17 14:03:53 +02:00
Darko Draskovic c1f9fa6328 Add agent grpc svc and client and Run endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-15 16:24:58 +02:00
Darko Draskovic 750bed0d76 Add run svc endpoint
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-05-12 12:52:00 +02:00
Darko Draskovic ad60e95736 Rename mfxkit to agent
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2023-03-27 16:37:52 +02:00