* feat(algorithm): Add wasm as an algo type
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* feat(algorithm): Use filesystem to store results
Move from unix socket for results storage to filesystem
* test: test new filesystem changes
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* refactor(files): rename resultFile to resultsFilePath
* feat(wasm-runtime): change from wasmtime to wasmedge
Wasmedge enables easier directory mapping to get results
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* feat(algorithm): send results as zipped directory
Create a new function to zip the results directory and send it back to the user
* fix(wasm): runtime argument
Fix the directory mapping for wasm runtime arguments
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* fix(errors): provide useful error message
* chore(gitignore): add results zip to gitignore
* feat(filesystem): Enable storing results on filesystem for python algos
* refactor: revert to upstream cocos repo
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* fix: remove AddDataset from algorithm interface
* fix: agent to handle results zipping
* test: test zipping directories
* refactor(agent): Handle file operations from agent
* test: run test inside eos
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* refactor(test): Document and test algos are running
Document steps on running the 2 python exampls and ensure they are running on eos
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
* fix: remove witheDataset option
* test: test without dataset argument
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
---------
Signed-off-by: Rodney Osodo <socials@rodneyosodo.com>
Changed the internal representation of algorithms and datasets within the service from byte slices to file paths, writing received data directly to temp files. This modification allows for handling potentially large data sets without the need to load them entirely into memory, improving the memory efficiency and scalability of the service. Additionally, it aligns the call signature of external algorithms with the new approach, updating documentation and examples accordingly. Updated the linear regression example for consistency with the new data handling process.
Resolves issues with memory bloat when processing large datasets.
Signed-off-by: SammyOina <sammyoina@gmail.com>
Sammy Kerata Oina
b1ackd0t
Darko Draskovic