Compare commits

..

78 Commits

Author SHA1 Message Date
Nikola Marčetić 6af65ea159 Added changelog for 0.9.0 release (#799)
Signed-off-by: nmarcetic <n.marcetic86@gmail.com>
2019-07-19 17:56:23 +02:00
Aleksandar Novaković b95bb82e0b Fix users service to work in offline mode (#795)
Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-07-19 12:28:45 +02:00
Ivan Milošević 8bda799ef9 Create and push docker manifest for new release from Makefile (#794)
Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-07-18 17:31:37 +02:00
Aleksandar Novaković 61f1c2e379 MF-399 - Add open tracing support (#782)
* Add open tracing dependencies

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add open tracing to users service

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add open tracing to the things service

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add open tracing to the http adapter

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add open tracing to the ws adapter

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add open tracing to the CoAP adapter

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update LoRa adapter in accordance with changes

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update SDK tests in accordance with changes

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update bootstrap service in accordance with changes

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update reader services with accordance with changes

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update .env and docker-compose file

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add jaeger and timeout env vars

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Fix broken test for can access by id endpoint

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update deps with proto empty package

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-07-18 15:01:09 +02:00
Ivan Milošević 1ad14e7fd3 fix mainflux_id parameter in bootstrap swagger (#789)
Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-07-16 16:21:37 +02:00
Aleksandar Novaković a6a8648e4f MF-783 - Allow access checking by a thing ID (#784)
* Add can access by things ID endpoint to things service

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add new auth endpoint to the swagger docs

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add test for the new endpoint of the things service

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-07-15 18:28:15 +02:00
Dušan Borovčanin 4a7af7c0b3 NOISSUE - Fix CoAP adapter (#779)
* Fix CoAP mux

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Remove prints and commented code

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-07-15 17:49:02 +02:00
Ivan Milošević b0a79436b8 Fix MF_THINGS_AUTH_GRPC_PORT in addons' docker-compose files (#781)
Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-07-11 23:19:08 +02:00
Aleksandar Novaković 4ab2e396c2 NOISSUE - Add authorization HTTP API to things service (#772)
* Add authorization HTTP API to things service

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add new tests and update existing ones

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update swagger documentation

Update swagger documentation for auth endpoints.

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update README docs for things service

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update docker-compose and fix endpoint typo

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Remove commented code

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-07-04 17:06:55 +02:00
Manuel Imperiale 22a3b881fd NOISSUE - Typo fix (#777)
Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2019-07-04 09:57:03 +02:00
Aleksandar Novaković aeebf06799 Remove cli executable from repo (#776)
Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-07-04 05:19:49 +08:00
Mirko Teodorovic c396e6ded8 fix docker-compose env (#775)
Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
2019-07-03 20:50:28 +02:00
Manuel Imperiale f5b6c63cca NOISSUE - Use .env vars in docker-compose (#770)
* NOISSUE - Use .enr vars in docker-compose

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add README and dupllicated .env in docker repo

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Use .env for Bootstrap Docker Composition

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Use .env for Cassandra (Reader-Writer) Docker Composition

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Use .env for InfluxDB (Reader-Writer) Docker Composition

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Use .env for MongoDB (Reader-Writer) Docker Composition

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Use .env for Postgres (Reader-Writer) Docker Composition

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Use .env for LoRa adapter Docker Composition

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix Grafana port in .env file

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Update README.md

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>

* remove docker/.env and update README.md

Signed-off-by: Mirko Teodorovic <mirko.teodorovic@gmail.com>
2019-07-02 14:22:12 +02:00
Dušan Borovčanin 81cd73e277 Use errors and null packets in authorized pub/sub (#773)
Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-07-01 11:22:31 +02:00
Mirko Teodorovic 204aec6171 MF-663 - enable nginx port conf from docker env (#769)
* MF-722 - Change UUID lib (#746)

* Update uuid package and update things serivce

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update bootstrap service tests

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update existing postgres writer tests

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* MF-732 - Add Postgres reader (#740)

* NOISSUE - Fix Readers logs

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* MF-732 - Add Postgres reader

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix total count

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Rm commented code

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add Postgres reader tests

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix editor format

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Change UUID lib

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* MF-742 - Things to support single user scenario (#749)

* Add single user mode to things service

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add tests for things/users package

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update init order in main

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* Update grpc and protobuf deps in mqtt adapter (#751)

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* Fix MQTT raw message deserialization (#753)

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* NOISSUE - Add certificate fields to the Bootstrap service (#752)

* Add cert fields to the BS

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add cert fields when creating a config

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add update cert endpoint

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix key column name

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add cert fields to db converters

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Secure cert update endpoint

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Authroize cert update methods

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix Bootstrap service tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add cert update service tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update endpoit tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update API docs

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update request tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix request tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update repository tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix typo in repo tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* NOISSUE - Add searchable Things name (#750)

* NOISSUE - Add searchable Things name

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix typo

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add postgres schema validation and tests

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add namme tests in requests_test

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Typo fix

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Rm requests_test

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add name in ListThings loggins

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add invalidName var for tests

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Set maxNameSize to 1024

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix postgres test

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix total when filtering things by name

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix review

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* Add missing Websocket.js into docker ui image (#755)

Signed-off-by: Ivan Milošević <iva@blokovi.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* MF-466 - ARM docker deployment (#756)

* Add arm Dockerfiles
Change version of postgres images in docker-compose to use versions with  multiarch

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* docker compose file

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove bin qemu file

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove unnecesarry comments

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Add make dockers_arm32v7 in Makefile

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Remove docker/Dockerfile.arm

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Add build for arm32v7 in ui Makefile

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Add arm32v7 tags

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* add docker-compose for arm
remove thing-db in docker compose (use same db as user)
fix dockerfile and makefile

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix thing database env variable

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Switch back to separate database for things and users

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* rename arm32v7 to arm

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove arm32v7 tag
rename arm32v7 image names

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Rename command for making mqtt and ui individual images, to match other miscroservices

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Push arm docker images

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix fucntion call in ci.sh

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* mergiing ui/Dockerfile

Signed-off-by: Ivan Milošević <iva@blokovi.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

*  NOISSUE - Add searchable Channels name  (#754)

* NOISSUE - Add searchable Things name

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix typo

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add postgres schema validation and tests

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add namme tests in requests_test

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* NOISSUE - Add searchable Channels name

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix test description

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix bootstrap mocks

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* NOISSUE - Remove installing non-existent package in ci (#758)

* Remove installing non-existent  package from ci

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove branch master condition in ci.sh to test it

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix syntax error

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* update apt

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* build arm images for latest release
push ui and mqtt arm images
install only qemu-user-static

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* clean docker after pushing amd64 images

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* installing all qemu app

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove docker images before building ui and mqtt arm-images

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* prune dockers and test only arm build

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix syntax error

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix moving qemu-arm-static

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* installing qemu with apt-get

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* add another apt-get update

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* apt install one liner

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* testing ci script

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* After installing qemu, get appropriate version

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Test ci without amd64 dockers

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove tests from ci for test purpose

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Uncomment commands

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Remove whitespace and change order of building images

Signed-off-by: Ivan Milošević <iva@blokovi.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* nginx

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* change to openresty

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* change to openresty and accept env from docker compose

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* revert to master

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* revert to master

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* revert to master

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* NOISSUE - Fix Docker for ARM (#760)

* NOISSUE - Fix Docker for ARM

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* Correct ARGs

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* Fix docker-compose for ARM

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* Add docker manifest

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* cp not mv qemu from mf root

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix amd64 docker names for ui and mqtt images

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* edit ci.sh for testing purposes

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove bachslash from make manifest
edits in ci for testing purposes

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix manifest call

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* add manifest on version realese
delete space before latest argument
fix for loop in manifest creation

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* include tests

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* docker system prune
remove tests for testin purposes

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Add variant in manifest file for armv7

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Remove white space

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* paralelise the compilation

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix place of -j$NPROC

Signed-off-by: Ivan Milošević <iva@blokovi.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* adding env variable

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* fix variant option for manifest annotate (#765)

Signed-off-by: Ivan Milošević <iva@blokovi.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* enable port configure from env var in docker-compose

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* enable port configure from env var in docker-compose

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* enable port configure from env var in docker-compose

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* use docker env to set port

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* add env to conf port in nginx.conf

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* replace string with docker env

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* replace string with docker env

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* Update docs (#766)

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* remove not needed comment

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* adding .env file for default UI_PORT - if no enviroment UI_PORT is set value from .env file is used

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* get default UI_PORT variable value from .env file

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>

* set default port value to 3000

Signed-off-by: mteodor <mirko.teodorovic@gmail.com>
2019-06-24 20:13:02 +02:00
Dušan Borovčanin aa219b6f99 NOISSUE - Fix count when search by name is performed (#767)
* Fix total field when search by name

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix tests to check total count

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-06-24 14:03:48 +02:00
Dušan Borovčanin f27cb1d019 Update docs (#766)
Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-06-21 14:07:24 +02:00
Ivan Milošević 41897cee11 fix variant option for manifest annotate (#765)
Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-06-20 15:31:56 +02:00
Drasko DRASKOVIC 91274aca9b NOISSUE - Fix Docker for ARM (#760)
* NOISSUE - Fix Docker for ARM

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* Correct ARGs

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* Fix docker-compose for ARM

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* Add docker manifest

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* cp not mv qemu from mf root

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix amd64 docker names for ui and mqtt images

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* edit ci.sh for testing purposes

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove bachslash from make manifest
edits in ci for testing purposes

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix manifest call

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* add manifest on version realese
delete space before latest argument
fix for loop in manifest creation

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* include tests

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* docker system prune
remove tests for testin purposes

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Add variant in manifest file for armv7

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Remove white space

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* paralelise the compilation

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix place of -j$NPROC

Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-06-20 14:14:06 +02:00
Ivan Milošević b53979104c NOISSUE - Remove installing non-existent package in ci (#758)
* Remove installing non-existent  package from ci

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove branch master condition in ci.sh to test it

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix syntax error

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* update apt

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* build arm images for latest release
push ui and mqtt arm images
install only qemu-user-static

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* clean docker after pushing amd64 images

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* installing all qemu app

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove docker images before building ui and mqtt arm-images

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* prune dockers and test only arm build

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix syntax error

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix moving qemu-arm-static

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* installing qemu with apt-get

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* add another apt-get update

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* apt install one liner

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* testing ci script

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* After installing qemu, get appropriate version

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Test ci without amd64 dockers

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove tests from ci for test purpose

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Uncomment commands

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Remove whitespace and change order of building images

Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-06-12 21:11:45 +02:00
Manuel Imperiale 63de955a7c NOISSUE - Add searchable Channels name (#754)
* NOISSUE - Add searchable Things name

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix typo

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add postgres schema validation and tests

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add namme tests in requests_test

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* NOISSUE - Add searchable Channels name

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix test description

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix bootstrap mocks

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2019-06-11 10:37:25 +02:00
Ivan Milošević 44cc20b9ca MF-466 - ARM docker deployment (#756)
* Add arm Dockerfiles
Change version of postgres images in docker-compose to use versions with  multiarch

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* docker compose file

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove bin qemu file

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove unnecesarry comments

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Add make dockers_arm32v7 in Makefile

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Remove docker/Dockerfile.arm

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Add build for arm32v7 in ui Makefile

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Add arm32v7 tags

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* add docker-compose for arm
remove thing-db in docker compose (use same db as user)
fix dockerfile and makefile

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix thing database env variable

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Switch back to separate database for things and users

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* rename arm32v7 to arm

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* remove arm32v7 tag
rename arm32v7 image names

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Rename command for making mqtt and ui individual images, to match other miscroservices

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Push arm docker images

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* fix fucntion call in ci.sh

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* mergiing ui/Dockerfile

Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-06-11 02:14:05 +02:00
Ivan Milošević 62e7d69f7b Add missing Websocket.js into docker ui image (#755)
Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-06-10 00:15:12 +02:00
Manuel Imperiale c8979ac297 NOISSUE - Add searchable Things name (#750)
* NOISSUE - Add searchable Things name

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix typo

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add postgres schema validation and tests

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add namme tests in requests_test

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Typo fix

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Rm requests_test

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add name in ListThings loggins

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add invalidName var for tests

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Set maxNameSize to 1024

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix postgres test

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix total when filtering things by name

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix review

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2019-05-30 15:33:49 +02:00
Dušan Borovčanin cac1e35b58 NOISSUE - Add certificate fields to the Bootstrap service (#752)
* Add cert fields to the BS

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add cert fields when creating a config

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add update cert endpoint

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix key column name

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add cert fields to db converters

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Secure cert update endpoint

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Authroize cert update methods

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix Bootstrap service tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add cert update service tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update endpoit tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update API docs

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update request tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix request tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update repository tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix typo in repo tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-05-22 23:22:19 +02:00
Aleksandar Novaković 6d6f3cc858 Fix MQTT raw message deserialization (#753)
Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-05-21 11:27:26 +02:00
Aleksandar Novaković 066057d914 Update grpc and protobuf deps in mqtt adapter (#751)
Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-05-17 16:51:22 +02:00
Aleksandar Novaković f9093413ee MF-742 - Things to support single user scenario (#749)
* Add single user mode to things service

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Add tests for things/users package

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update init order in main

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-05-17 16:06:21 +02:00
Manuel Imperiale 5fec0a9e92 MF-732 - Add Postgres reader (#740)
* NOISSUE - Fix Readers logs

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* MF-732 - Add Postgres reader

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix total count

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Rm commented code

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add Postgres reader tests

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix editor format

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Change UUID lib

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2019-05-16 18:00:19 +02:00
Aleksandar Novaković 3125f0bbc2 MF-722 - Change UUID lib (#746)
* Update uuid package and update things serivce

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update bootstrap service tests

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update existing postgres writer tests

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-05-16 13:35:13 +02:00
Aleksandar Novaković 0dc4d12e73 Add performance improvement to writer filtering (#744)
Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-05-16 13:19:05 +02:00
Dušan Borovčanin 9d30006d31 NOISSUE - Update nginx version (#748)
* Update version of the Nginx image

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add undefined check for HTTP and WS

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-05-16 11:59:23 +02:00
Aleksandar Novaković 5202d734c9 MF-574 - Add missing environment variables to Cassandra writer (#745)
* Add username, password and port env var

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>

* Update existing readme files with new env vars

Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-05-15 23:31:40 +02:00
Aleksandar Novaković cc5c66bfba Fix CI with fixed plugin versions (#747)
Signed-off-by: Aleksandar Novakovic <anovakovic01@gmail.com>
2019-05-15 17:41:18 +02:00
Dušan Borovčanin f5bb65d0e4 NOISSUE - Add compile test to CI (#743)
* Add compile tests to CI script

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix parallelization flag

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-05-13 14:48:54 +02:00
Mirko Teodorovic e46164f93f fix building problems (#741)
Signed-off-by: mteodor <mirko.teodorovic@gmail.com>
2019-05-11 17:16:25 +02:00
Manuel Imperiale d2fc379e3f NOISSUE - Fix Postgres logs in Things service (#734)
* NOISSUE - Fix Postgres logs in Things service

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix uuid package

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2019-05-10 18:55:13 +02:00
Aleksandar Novaković 5e5fe88927 MF-708 - Assign Writer(s) to a channel (#737)
* Add support for channel filtering using config

Add support for channel filtering using yaml configuration files
for writers.

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Update writers documentation with new env var

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Add info comment to configuration file

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Fix configuration and update yaml dep

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Update config from yaml to toml

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>
2019-05-10 14:36:27 +02:00
Manuel Imperiale 886c8b2467 MF-732 - Add PostgreSQL writer (#733)
* MF-732 - Add PostgreSQL writer

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Add README

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Unify writers api

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix host and port

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2019-05-07 19:09:28 +02:00
Manuel Imperiale 138dae9277 NOISSUE - Fix Readers logs (#735)
* NOISSUE - Fix Readers logs

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2019-05-07 15:59:18 +02:00
Manuel Imperiale 9f1151b8dc NOISSUE - Add readers pagination in SDK (#736)
* NOISSUE - Add readers pagination in SDK

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2019-05-07 15:10:02 +02:00
Darko Draskovic 700cfdf6f3 Add UI websocket open/close and send/receive (#728)
* Add websocket open/close and send/receive

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Add websocket feedback/log to Elm

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Align WS open/close buttons

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Add websocket query for channel list

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Add websocket related encoders/decoders for port interops

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Fix .js indentation and simplify ws (dis)connect method

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Use single quotes consistently

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Add HTTP Send and WS Listen

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2019-04-26 19:17:04 +02:00
Aleksandar Novaković dc9333237f MF-707 - Allow custom Thing key (#726)
* Add support for setting up thing key manually

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Fix existing tests and add new ones

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Update SQL schema for things entity

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Add update thing key endpoint to swagger docs

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Fix response code when handling conflicting key

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>
2019-04-25 14:37:51 +02:00
Aleksandar Novaković aba7d909ed MF-525 - Add pagination response to the readers (#729)
* Add pagination response to the readers

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Remove println from influx reader

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>
2019-04-25 00:18:43 +02:00
Manuel Imperiale 04281f2388 NOISSUE - Rm Things type from lora-adapter (#727)
Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2019-04-22 21:31:08 +02:00
Milan Andric 08f2545df9 fix to makefile for OSX/Darwin (#724)
Signed-off-by: Milan Andric <mandric@gmail.com>
2019-04-22 21:03:05 +02:00
Darko Draskovic 64f154dcc9 Fix .dockerignore file by removing index.html (#725)
* Fix .dockerignore file by removing index.html

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Add autogen main.js to .dockerignore

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2019-04-21 20:10:24 +02:00
Darko Draskovic 9688fc316a Fix things and channels metadata create and edit & remove thing type (#721)
* Add basic websocket support

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Add separate cards for base and regular attribs

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Add arbitrary json metadata enc/dec

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Add JsonMF module for JSON val handling

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Rename json related func for naming consistency

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Remove Thing type from user interface

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Add arbitrary metadata json to channels

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Change json string identation

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2019-04-21 02:40:14 +02:00
Jovan Kostovski 5b66b6e169 skip deleting of persistent volumes by default (#723)
This commit, intruduces changes to the current behavior of the "make cleandocker" command.
From now on, an extra parameter must be given in order to delete the persistent volumes,  which up until now was done by default. The new command which represents the old behavior (deleting all artifacts created from the docker composition) is "make pv=true cleandocker".

Signed-off-by: Jovan Kostovski <chombium@gmail.com>
2019-04-20 17:56:51 +02:00
Manuel Imperiale b97deb50b2 MF-488 - Remove Thing type (app or device) (#718)
* MF-488 - Remove Thing type (app or device)

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Typo fix

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>

* Fix reviews

Signed-off-by: Manuel Imperiale <manuel.imperiale@gmail.com>
2019-04-20 14:09:11 +02:00
Dušan Borovčanin 44615c5ff0 Remove empty channels check (#720)
Remove empty list of channels check from the default Bootstrap
reader implementation.

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-18 18:02:20 +02:00
Jovan Kostovski a30666d999 MF-655 Proper usage of docker volumes (#657)
* added filter and a switch for volume deletion

Signed-off-by: Jovan Kostovski <chombium@gmail.com>

* added persistent volume configuration

Signed-off-by: Jovan Kostovski <chombium@gmail.com>
2019-04-18 17:31:24 +02:00
Drasko DRASKOVIC baebe8abc1 NOISSUE - Improve UI styling (#719)
* Add styling

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* Fix errors

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* Fix env

Signed-off-by: drasko <drasko.draskovic@gmail.com>
2019-04-18 13:29:38 +02:00
Dušan Borovčanin d9d9da49a8 MF-715 - Conflict on updating connection with a valid list of channels (#716)
* Add check if Channels already exist

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update streams tests to use time offset

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-17 19:41:37 +02:00
Aleksandar Novaković 2f17a05b3e MF-711 - Create separate Redis instance for ES (#717)
* Fix Redis connection error handling in MQTT adapter

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Add separate env vars for ES in MQTT adapter

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Create dedicated redis service for ES

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Update docs according to changes

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>
2019-04-17 12:39:47 +02:00
Dušan Borovčanin fe9f4dca8e NOISSUE - Update event fields naming (#713)
* Update event fields naming

Replace `id` with `thing_id`.
Fix typo in `success` field.

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update docs

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-16 18:32:05 +02:00
Aleksandar Novaković de736a5c47 MF-698 - Add missing info and docs about sys event sourcing (#712)
* Add event sourcing section in dev guide

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Fix spelling in the docs

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Fix notice formatting in docs

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Fix typos in docs

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Add highlight for event type

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>
2019-04-16 17:28:43 +02:00
Aleksandar Novaković 5799356b14 MF-549 - Change metadata format from JSON string to JSON object (#706)
* Update metadata type in things service

Update things service so that metadata has map type. Update repo
implementation by adding sqlx lib.

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Add sqlx lib to bootstrap service

Add sqlx lib to bootstrap service and update metadata field type.

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Update metadata in redis streams consumer

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Update tests for bootstrap service

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Fix mongo reader logging and driver version

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Fix mongo reader and writer

Fix mongo reader and writer by updating driver version.

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Update SDK with new metadata format

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Update LoRa adapter with new metadata format

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Update users service in order to use sqlx

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Replace anonymous struct with map

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Update docs for LoRa adapter

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Fix LoRa application metadata format

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Fix metadata format in LoRa docs

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>

* Add metadata2 var to SDK things test

Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>
2019-04-16 14:58:56 +02:00
Dušan Borovčanin 0bbc4f54ab Fix Bootstrap service event map keys (#705)
Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-11 13:55:21 +02:00
Darko Draskovic 0b89975947 NOISSUE - Replace repeating code by card gen func (#697)
* Replace repeating code by card gen func

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Remove absolute URL from Env.elm

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Fix md links in README.md

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>

* Refactor appendIF helper func

Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2019-04-09 16:21:26 +02:00
Dušan Borovčanin df11a20a32 Update Bootstrap service docker-compose.yml (#700)
Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-09 02:44:50 -07:00
Ivan Milošević 77ce0f9fab Remove Debug function (#699)
Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-04-09 01:50:23 -07:00
Dušan Borovčanin 6ebc758f89 MF-687 - Add event sourcing to Bootstrap service (#695)
* Add initial Bootstrap event producer

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add operation name to events

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add ES path for BS events

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add BS event sourcing tests

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add connections update events

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Extract event XADD to separeate method

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add header comments

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix service docs typo

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update BS service README.md

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update Service method tests

Tests for handler methods are not added.

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-08 08:46:07 -07:00
Ivan Milošević 98d588e8ef NOISSUE - Remove debugging message from response of handle error function (#696)
* Remove debugging message from response of handle error function

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Print error in debug console in case of bad body HTTP response

Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-04-08 13:44:11 +02:00
Aleksandar Novaković 12a85f565c Fix logging in publish event callback (#694)
Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>
2019-04-05 17:29:43 +02:00
Aleksandar Novaković 5fcf0aa229 Add event stream to MQTT adapter for conn status (#692)
Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>
2019-04-05 11:25:39 +02:00
Drasko DRASKOVIC 9b41c2db3a NOISSUE - Improve UI style (#691)
* Improve UI style

Signed-off-by: drasko <drasko.draskovic@gmail.com>

* Remove unused font

Signed-off-by: drasko <drasko.draskovic@gmail.com>
2019-04-04 20:28:56 +02:00
Dušan Borovčanin e4d3b1d8c3 Fix InfluxDB time bug (#689)
Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-04 18:10:01 +02:00
Dušan Borovčanin 4f62f89b87 Update docs structure (#686)
Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-04 12:10:46 +02:00
Dušan Borovčanin 06f4c6e059 Use images instead of carousel (#685)
Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-04 11:19:43 +02:00
Dušan Borovčanin 25d9783514 NOISSUE - Update docs (#683)
* Add authorization section to the docs

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Slow down Bootstrap gif

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Use heading 4 instead of italic

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Replace gif with carousel

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-04 10:22:09 +02:00
Ivan Milošević 56a50962b6 MF-662 - Change menu style (#678)
* env variable for local dev

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Change menu style

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Revert "env variable for local dev"

This reverts commit 205e13be6cdde930f02e7bda2019bb615267ed14.

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Move font awesome class names to record in Helper

Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-04-02 19:17:28 +02:00
Dušan Borovčanin f9b17d5f24 MF-651 - X509 Mutual TLS authentication (#676)
* Use NginX njs module for mutual authentication

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add Makefile for cert management

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Move certificates make context to scripts dir

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Move nginx.conf to separate directory

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Choose between two NginX configurations

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Move certs Makefile to docker/ssl/

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Use default key-based authentication

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add mTLS docs

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update Makefile

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add check if Authorization is present

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add check if Will Flag is 1

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Return MQTT over WS

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Fix docker-compose.yml volume mapping

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Rename security section in docs

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add message type check before message parsing

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Remove double comments

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Remove s.AGAIN in return

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Update Makefile

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Remove CSR and key from the root

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Drop TLS version below 1.2

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>

* Add comments for cert and key paths

Signed-off-by: Dušan Borovčanin <dusan.borovcanin@mainflux.com>
2019-04-02 17:54:24 +02:00
Ivan Milošević bc49f6a543 Fix offset calculation after deleting thing/channel, not to go to negative offset after deleting last thing/channel (#679)
Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-04-01 17:49:10 +02:00
Aleksandar Novaković 83cafeb6be Update Aedes version for MQTT adapter (#677)
Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>
2019-03-29 14:04:02 +01:00
Ivan Milošević c634bf8c7d MF-661 - Bootstrap pagination in UI (#672)
* Add bootstrap pagination

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Fix enviroment variable in readme

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Change style
Remove pagination from channels and things lists if total less then limit

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Hide pagination when not needed on connection and message page
Code refacotoring

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Remove debug module and local dev variable.

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Remove debug logging

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* Add trailing slash note in README

Signed-off-by: Ivan Milošević <iva@blokovi.com>

* proto

Signed-off-by: Ivan Milošević <iva@blokovi.com>
2019-03-28 10:20:15 +01:00
Aleksandar Novaković 86a3115804 Update subtopics section in documentation (#670)
Signed-off-by: Aleksandar Novakovic <aleksandar.novakovic@mainflux.com>
2019-03-22 16:47:30 +01:00
Darko Draskovic 4a31f11615 Remove default base URL value (#671)
Signed-off-by: Darko Draskovic <darko.draskovic@gmail.com>
2019-03-22 14:53:03 +01:00
885 changed files with 76824 additions and 36951 deletions
+153
View File
@@ -0,0 +1,153 @@
# Docker: Environment variables in Compose
## NGINX
MF_NGINX_HTTP_PORT=80
MF_NGINX_SSL_PORT=443
MF_NGINX_MQTT_PORT=8883
## NATS
MF_NATS_URL=nats://nats:4222
## REDIS
MF_REDIS_TCP_PORT=6379
## UI
MF_UI_PORT=3000
## Grafana
MF_GRAFANA_PORT=3000
## Jaeger
MF_JAEGER_PORT=6831
MF_JAEGER_FRONTEND=16686
MF_JAEGER_COLLECTOR=14268
MF_JAEGER_CONFIGS=5778
MF_JAEGER_URL=jaeger:6831
## Core Services
### Users
MF_USERS_LOG_LEVEL=debug
MF_USERS_HTTP_PORT=8180
MF_USERS_GRPC_PORT=8181
MF_USERS_DB_PORT=5432
MF_USERS_DB_USER=mainflux
MF_USERS_DB_PASS=mainflux
MF_USERS_DB=users
MF_USERS_SECRET=secret
### Things
MF_THINGS_LOG_LEVEL=debug
MF_THINGS_HTTP_PORT=8182
MF_THINGS_AUTH_HTTP_PORT=8989
MF_THINGS_AUTH_GRPC_PORT=8183
MF_THINGS_DB_PORT=5432
MF_THINGS_DB_USER=mainflux
MF_THINGS_DB_PASS=mainflux
MF_THINGS_DB=things
MF_THINGS_SECRET=secret
### Normalizer
MF_NORMALIZER_LOG_LEVEL=debug
MF_NORMALIZER_PORT=8184
### WS
MF_WS_ADAPTER_LOG_LEVEL=debug
MF_WS_ADAPTER_PORT=8186
### HTTP
MF_HTTP_ADAPTER_PORT=8185
### MQTT
MF_MQTT_ADAPTER_LOG_LEVEL=debug
MF_MQTT_ADAPTER_PORT=1883
MF_MQTT_ADAPTER_WS_PORT=8880
### COAP
MF_COAP_ADAPTER_LOG_LEVEL=debug
MF_COAP_ADAPTER_PORT=5683
## Addons Services
### Bootstrap
MF_BOOTSTRAP_LOG_LEVEL=debug
MF_BOOTSTRAP_PORT=8200
MF_BOOTSTRAP_DB_PORT=5432
MF_BOOTSTRAP_DB_USER=mainflux
MF_BOOTSTRAP_DB_PASS=mainflux
MF_BOOTSTRAP_DB=bootstrap
MF_BOOTSTRAP_DB_SSL_MODE=disable
### LoRa
MF_LORA_ADAPTER_LOG_LEVEL=debug
MF_LORA_ADAPTER_MESSAGES_URL=tcp://lora.mqtt.mainflux.io:1883
MF_LORA_ADAPTER_HTTP_PORT=8187
### Cassandra Writer
MF_CASSANDRA_WRITER_LOG_LEVEL=debug
MF_CASSANDRA_WRITER_PORT=8902
MF_CASSANDRA_WRITER_DB_PORT=9042
MF_CASSANDRA_WRITER_DB_CLUSTER=mainflux-cassandra
MF_CASSANDRA_WRITER_DB_KEYSPACE=mainflux
### Cassandra Reader
MF_CASSANDRA_READER_LOG_LEVEL=debug
MF_CASSANDRA_READER_PORT=8903
MF_CASSANDRA_READER_DB_PORT=9042
MF_CASSANDRA_READER_DB_CLUSTER=mainflux-cassandra
MF_CASSANDRA_READER_DB_KEYSPACE=mainflux
### InfluxDB Writer
MF_INFLUX_WRITER_LOG_LEVEL=debug
MF_INFLUX_WRITER_PORT=8900
MF_INFLUX_WRITER_BATCH_SIZE=5000
MF_INFLUX_WRITER_BATCH_TIMEOUT=5
MF_INFLUX_WRITER_DB_PORT=8086
MF_INFLUX_WRITER_DB_NAME=mainflux
MF_INFLUX_WRITER_DB_USER=mainflux
MF_INFLUX_WRITER_DB_PASS=mainflux
MF_INFLUX_WRITER_GRAFANA_PORT=3001
### InfluxDB Reader
MF_INFLUX_READER_LOG_LEVEL=debug
MF_INFLUX_READER_PORT=8905
MF_INFLUX_READER_DB_NAME=mainflux
MF_INFLUX_READER_DB_PORT=8086
MF_INFLUX_READER_DB_USER=mainflux
MF_INFLUX_READER_DB_PASS=mainflux
### MongoDB Writer
MF_MONGO_WRITER_LOG_LEVEL=debug
MF_MONGO_WRITER_PORT=8901
MF_MONGO_WRITER_DB_NAME=mainflux
MF_MONGO_WRITER_DB_PORT=27017
### MongoDB Reader
MF_MONGO_READER_LOG_LEVEL=debug
MF_MONGO_READER_PORT=8904
MF_MONGO_READER_DB_NAME=mainflux
MF_MONGO_READER_DB_PORT=27017
### Postgres Writer
MF_POSTGRES_WRITER_LOG_LEVEL=debug
MF_POSTGRES_WRITER_PORT=9104
MF_POSTGRES_WRITER_DB_PORT=5432
MF_POSTGRES_WRITER_DB_USER=mainflux
MF_POSTGRES_WRITER_DB_PASS=mainflux
MF_POSTGRES_WRITER_DB_NAME=messages
MF_POSTGRES_WRITER_DB_SSL_MODE=disable
MF_POSTGRES_WRITER_DB_SSL_CERT=""
MF_POSTGRES_WRITER_DB_SSL_KEY=""
MF_POSTGRES_WRITER_DB_SSL_ROOT_CERT=""
### Postgres Reader
MF_POSTGRES_READER_LOG_LEVEL=debug
MF_POSTGRES_READER_PORT=9204
MF_POSTGRES_READER_CLIENT_TLS=false
MF_POSTGRES_READER_CA_CERTS=""
MF_POSTGRES_READER_DB_PORT=5432
MF_POSTGRES_READER_DB_USER=mainflux
MF_POSTGRES_READER_DB_PASS=mainflux
MF_POSTGRES_READER_DB_NAME=messages
MF_POSTGRES_READER_DB_SSL_MODE=disable
MF_POSTGRES_READER_DB_SSL_CERT=""
MF_POSTGRES_READER_DB_SSL_KEY=""
MF_POSTGRES_READER_DB_SSL_ROOT_CERT=""
+86
View File
@@ -11,6 +11,92 @@ Otherwise, whole log in a similar format can be observed via:
git log --pretty=oneline --abbrev-commit
```
## 0.9.0 - 19. JUL 2019.
### Features
- Create and push docker manifest for new release from Makefile (#794)
- MF-399 - Add open tracing support (#782)
- MF-783 - Allow access checking by a thing ID (#784)
- NOISSUE - Add authorization HTTP API to things service (#772)
- Remove cli executable from repo (#776)
- NOISSUE - Use .env vars in docker-compose (#770)
- MF-663 - enable nginx port conf from docker env (#769)
- Update docs (#766)
- NOISSUE - Remove installing non-existent package in ci (#758)
- NOISSUE - Add searchable Channels name (#754)
- MF-466 - ARM docker deployment (#756)
- Add missing Websocket.js into docker ui image (#755)
- NOISSUE - Add searchable Things name (#750)
- NOISSUE - Add certificate fields to the Bootstrap service (#752)
- Update grpc and protobuf deps in mqtt adapter (#751)
- MF-742 - Things to support single user scenario (#749)
- MF-732 - Add Postgres reader (#740)
- MF-722 - Change UUID lib (#746)
- Add performance improvement to writer filtering (#744)
- NOISSUE - Update nginx version (#748)
- MF-574 - Add missing environment variables to Cassandra writer (#745)
- NOISSUE - Add compile test to CI (#743)
- MF-708 - Assign Writer(s) to a channel (#737)
- MF-732 - Add PostgreSQL writer (#733)
- NOISSUE - Add readers pagination in SDK (#736)
- Add UI websocket open/close and send/receive (#728)
- MF-707 - Allow custom Thing key (#726)
- MF-525 - Add pagination response to the readers (#729)
- NOISSUE - Rm Things type from lora-adapter (#727)
- skip deleting of persistent volumes by default (#723)
- MF-488 - Remove Thing type (app or device) (#718)
- Remove empty channels check (#720)
- MF-655 Proper usage of docker volumes (#657)
- NOISSUE - Improve UI styling (#719)
- MF-715 - Conflict on updating connection with a valid list of channels (#716)
- MF-711 - Create separate Redis instance for ES (#717)
- NOISSUE - Update event fields naming (#713)
- MF-698 - Add missing info and docs about sys event sourcing (#712)
- MF-549 - Change metadata format from JSON string to JSON object (#706)
- NOISSUE - Replace repeating code by card gen func (#697)
- Update Bootstrap service docker-compose.yml (#700)
- Remove Debug function (#699)
- MF-687 - Add event sourcing to Bootstrap service (#695)
- NOISSUE - Remove debugging message from response of handle error function (#696)
- Add event stream to MQTT adapter for conn status (#692)
- NOISSUE - Improve UI style (#691)
- Update docs structure (#686)
- Use images instead of carousel (#685)
- NOISSUE - Update docs (#683)
- MF-662 - Change menu style (#678)
- MF-651 - X509 Mutual TLS authentication (#676)
- Update Aedes version for MQTT adapter (#677)
- MF-661 - Bootstrap pagination in UI (#672)
- Update subtopics section in documentation (#670)
- Remove default base URL value (#671)
### Bugfixes
- NOISSUE - Fix Readers logs (#735)
- NOISSUE - Fix Docker for ARM (#760)
- NOISSUE - Fix count when search by name is performed (#767)
- NOISSUE - Typo fix (#777)
- NOISSUE - Fix Postgres logs in Things service (#734)
- Fix CI with fixed plugin versions (#747)
- fix building problems (#741)
- fix docker-compose env (#775)
- Fix MF_THINGS_AUTH_GRPC_PORT in addons' docker-compose files (#781)
- Fix MQTT raw message deserialization (#753)
- fix variant option for manifest annotate (#765)
- fix to makefile for OSX/Darwin (#724)
- Fix .dockerignore file by removing index.html (#725)
- Fix things and channels metadata create and edit & remove thing type (#721)
- Fix Bootstrap service event map keys (#705)
- Fix logging in publish event callback (#694)
- Fix InfluxDB time bug (#689)
- Fix users service to work in offline mode (#795)
- fix mainflux_id parameter in bootstrap swagger (#789)
- Fix offset calculation after deleting thing/channel, not to go to negative offset after deleting last thing/channel (#679)
- Use errors and null packets in authorized pub/sub (#773)
- NOISSUE - Fix CoAP adapter (#779)
### Summary
https://github.com/mainflux/mainflux/milestone/10?closed=1
## 0.8.0 - 20. MAR 2019.
### Features
- MF-571 - Add Env.elm to set custom base URL (#654)
Generated
+128 -69
View File
@@ -12,6 +12,14 @@
pruneopts = "UT"
revision = "d6e3b3328b783f23731bc4d058875b0371ff8109"
[[projects]]
digest = "1:9f3b30d9f8e0d7040f729b82dcbc8f0dead820a133b3147ce355fc451f32d761"
name = "github.com/BurntSushi/toml"
packages = ["."]
pruneopts = "UT"
revision = "3012a1dbe2e4bd1391d42b32f0577cb7bbc7f005"
version = "v0.3.1"
[[projects]]
digest = "1:bf42be3cb1519bf8018dfd99720b1005ee028d947124cab3ccf965da59381df6"
name = "github.com/Microsoft/go-winio"
@@ -28,14 +36,6 @@
pruneopts = "UT"
revision = "cd527374f1e5bff4938207604a14f2e38a9cf512"
[[projects]]
digest = "1:320e7ead93de9fd2b0e59b50fd92a4d50c1f8ab455d96bc2eb083267453a9709"
name = "github.com/asaskevich/govalidator"
packages = ["."]
pruneopts = "UT"
revision = "ccb8e960c48f04d6935e72476ae4a51028f9e22f"
version = "v9"
[[projects]]
branch = "master"
digest = "1:d6afaeed1502aa28e80a4ed0981d570ad91b2579193404256ce672ed0a609e0d"
@@ -76,14 +76,6 @@
revision = "346938d642f2ec3594ed81d874461961cd0faa76"
version = "v1.1.0"
[[projects]]
branch = "master"
digest = "1:ca71cdedcb487a361fe2588afeb3d373c0dfe21feaac229270fd6de234bd3c5e"
name = "github.com/dereulenspiegel/coap-mux"
packages = ["."]
pruneopts = "UT"
revision = "e137e310a99955be01d60b2237da89153ea40987"
[[projects]]
digest = "1:76dc72490af7174349349838f2fe118996381b31ea83243812a97e5a0fd5ed55"
name = "github.com/dgrijalva/jwt-go"
@@ -147,7 +139,7 @@
version = "v1.7.0"
[[projects]]
digest = "1:90aaa664c659918dd0055e29eb5fd6b96a17a779d423db3dd3f5764c5dbbbd4f"
digest = "1:80abda6d164916248f3297f714a17b1a33e54bb5694ff283409b4d0320357b3a"
name = "github.com/go-kit/kit"
packages = [
"endpoint",
@@ -155,6 +147,7 @@
"metrics",
"metrics/internal/lv",
"metrics/prometheus",
"tracing/opentracing",
"transport/grpc",
"transport/http",
]
@@ -216,6 +209,14 @@
pruneopts = "UT"
revision = "68ae1e384be4d7cd7df0f5b77f331759d806308e"
[[projects]]
digest = "1:bed9d72d596f94e65fff37f4d6c01398074a6bb1c3f3ceff963516bd01db6ff5"
name = "github.com/gofrs/uuid"
packages = ["."]
pruneopts = "UT"
revision = "6b08a5c5172ba18946672b49749cde22873dd7c2"
version = "v3.2.0"
[[projects]]
digest = "1:bbadccf3d3317ea03c0dac0b45b673b4b397c8f91a1d2eff550a3c51c4ad770e"
name = "github.com/gogo/protobuf"
@@ -225,13 +226,14 @@
version = "v1.1.1"
[[projects]]
digest = "1:4c0989ca0bcd10799064318923b9bc2db6b4d6338dd75f3f2d86c3511aaaf5cf"
digest = "1:5ac24c0ad5dc117fb2b6d90d205c876ec599c09cccb702a36586680e09cb20b5"
name = "github.com/golang/protobuf"
packages = [
"proto",
"ptypes",
"ptypes/any",
"ptypes/duration",
"ptypes/empty",
"ptypes/timestamp",
]
pruneopts = "UT"
@@ -289,6 +291,17 @@
revision = "a2ba6e7654fb078f8a3f5add1f8d935df38161bd"
version = "v1.6.4"
[[projects]]
branch = "master"
digest = "1:965881afde38cce93606988d7ef9cd187b394a8a73b11a7dde379206f5e2fba3"
name = "github.com/jmoiron/sqlx"
packages = [
".",
"reflectx",
]
pruneopts = "UT"
revision = "cdf62fdf55f66fb2484fe214f89c059f0bd3f567"
[[projects]]
branch = "master"
digest = "1:a64e323dc06b73892e5bb5d040ced475c4645d456038333883f58934abbf6f72"
@@ -332,45 +345,6 @@
revision = "3247c84500bff8d9fb6d579d800f20b3e091582c"
version = "v1.0.0"
[[projects]]
digest = "1:27fa1b385644151ee1fc0c8b04f1f86963d9701335491b94488fd3912e370354"
name = "github.com/mongodb/mongo-go-driver"
packages = [
"bson",
"bson/bsoncodec",
"bson/bsonrw",
"bson/bsontype",
"bson/primitive",
"event",
"internal",
"mongo",
"mongo/options",
"mongo/readconcern",
"mongo/readpref",
"mongo/writeconcern",
"tag",
"version",
"x/bsonx",
"x/bsonx/bsoncore",
"x/mongo/driver",
"x/mongo/driver/auth",
"x/mongo/driver/auth/internal/gssapi",
"x/mongo/driver/session",
"x/mongo/driver/topology",
"x/mongo/driver/uuid",
"x/network/address",
"x/network/command",
"x/network/compressor",
"x/network/connection",
"x/network/connstring",
"x/network/description",
"x/network/result",
"x/network/wiremessage",
]
pruneopts = "UT"
revision = "3cfdc9ffcf780eede57d27ec26a3edad4c9ab6a9"
version = "v0.2.0"
[[projects]]
digest = "1:4ce94f3d2074662c04677db471456feb56078244ebcfe3789e6e95901c085bc5"
name = "github.com/nats-io/go-nats"
@@ -421,6 +395,19 @@
revision = "baf6536d6259209c3edfa2b22237af82942d3dfa"
version = "v0.1.1"
[[projects]]
digest = "1:c2024d8533dc702a6ef5c92d5cf1494858eff4484d8abbf794f21acd29e7ef70"
name = "github.com/opentracing/opentracing-go"
packages = [
".",
"ext",
"log",
"mocktracer",
]
pruneopts = "UT"
revision = "659c90643e714681897ec2521c60567dd21da733"
version = "v1.1.0"
[[projects]]
digest = "1:3b830e06971be8448e7c9e57c9cc15ce53ab7e8846e7b7c39110a177c7fae466"
name = "github.com/ory/dockertest"
@@ -526,14 +513,6 @@
pruneopts = "UT"
revision = "ba2c6a7295c59448dbc195cef2f41df5163b3892"
[[projects]]
digest = "1:274f67cb6fed9588ea2521ecdac05a6d62a8c51c074c1fccc6a49a40ba80e925"
name = "github.com/satori/go.uuid"
packages = ["."]
pruneopts = "UT"
revision = "f58768cc1a7a7e77a3bd49e98cdd21419399b6a3"
version = "v1.2.0"
[[projects]]
digest = "1:9e9193aa51197513b3abcb108970d831fbcf40ef96aa845c4f03276e1fa316d2"
name = "github.com/sirupsen/logrus"
@@ -577,6 +556,40 @@
revision = "f35b8ab0b5a2cef36673838d662e249dd9c94686"
version = "v1.2.2"
[[projects]]
digest = "1:3b0ac1ac6f703e6d43f4e29c97da2c9e34465c66e2e19f9828e9841d1c9279a4"
name = "github.com/uber/jaeger-client-go"
packages = [
".",
"config",
"internal/baggage",
"internal/baggage/remote",
"internal/spanlog",
"internal/throttler",
"internal/throttler/remote",
"log",
"rpcmetrics",
"thrift",
"thrift-gen/agent",
"thrift-gen/baggage",
"thrift-gen/jaeger",
"thrift-gen/sampling",
"thrift-gen/zipkincore",
"transport",
"utils",
]
pruneopts = "UT"
revision = "2f47546e3facd43297739439600bcf43f44cce5d"
version = "v2.16.0"
[[projects]]
digest = "1:c9d69a04f7fa171f50360bbcc32196b4de8ab8837ef772f6302d0140a1e3e7f6"
name = "github.com/uber/jaeger-lib"
packages = ["metrics"]
pruneopts = "UT"
revision = "0e30338a695636fe5bcf7301e8030ce8dd2a8530"
version = "v2.0.0"
[[projects]]
digest = "1:03aa6e485e528acb119fb32901cf99582c380225fc7d5a02758e08b180cb56c3"
name = "github.com/ugorji/go"
@@ -601,6 +614,45 @@
pruneopts = "UT"
revision = "73f8eece6fdcd902c185bf651de50f3828bed5ed"
[[projects]]
digest = "1:21f9cb6f1337c4776ed1d2d7b7ed6ffba3269ca12274c065364a5c0edee1f28b"
name = "go.mongodb.org/mongo-driver"
packages = [
"bson",
"bson/bsoncodec",
"bson/bsonrw",
"bson/bsontype",
"bson/primitive",
"event",
"internal",
"mongo",
"mongo/options",
"mongo/readconcern",
"mongo/readpref",
"mongo/writeconcern",
"tag",
"version",
"x/bsonx",
"x/bsonx/bsoncore",
"x/mongo/driver",
"x/mongo/driver/auth",
"x/mongo/driver/auth/internal/gssapi",
"x/mongo/driver/session",
"x/mongo/driver/topology",
"x/mongo/driver/uuid",
"x/network/address",
"x/network/command",
"x/network/compressor",
"x/network/connection",
"x/network/connstring",
"x/network/description",
"x/network/result",
"x/network/wiremessage",
]
pruneopts = "UT"
revision = "1c3b9b9a41eecdf056560e07b68e6407b8d598c3"
version = "v1.0.0"
[[projects]]
branch = "master"
digest = "1:7310f5459b88177d24e26c5ec7deb100b78ec03c0437d7a7aaec3c7eac333a55"
@@ -746,9 +798,8 @@
analyzer-name = "dep"
analyzer-version = 1
input-imports = [
"github.com/asaskevich/govalidator",
"github.com/BurntSushi/toml",
"github.com/cisco/senml",
"github.com/dereulenspiegel/coap-mux",
"github.com/dgrijalva/jwt-go",
"github.com/docker/docker/pkg/namesgenerator",
"github.com/dustin/go-coap",
@@ -758,27 +809,35 @@
"github.com/go-kit/kit/log",
"github.com/go-kit/kit/metrics",
"github.com/go-kit/kit/metrics/prometheus",
"github.com/go-kit/kit/tracing/opentracing",
"github.com/go-kit/kit/transport/grpc",
"github.com/go-kit/kit/transport/http",
"github.com/go-redis/redis",
"github.com/go-zoo/bone",
"github.com/gocql/gocql",
"github.com/gofrs/uuid",
"github.com/gogo/protobuf/proto",
"github.com/golang/protobuf/proto",
"github.com/golang/protobuf/ptypes/empty",
"github.com/gorilla/websocket",
"github.com/hokaccha/go-prettyjson",
"github.com/influxdata/influxdb/client/v2",
"github.com/jmoiron/sqlx",
"github.com/lib/pq",
"github.com/mongodb/mongo-go-driver/mongo",
"github.com/nats-io/go-nats",
"github.com/opentracing/opentracing-go",
"github.com/opentracing/opentracing-go/mocktracer",
"github.com/prometheus/client_golang/prometheus",
"github.com/prometheus/client_golang/prometheus/promhttp",
"github.com/rubenv/sql-migrate",
"github.com/satori/go.uuid",
"github.com/sony/gobreaker",
"github.com/spf13/cobra",
"github.com/stretchr/testify/assert",
"github.com/stretchr/testify/require",
"github.com/uber/jaeger-client-go/config",
"go.mongodb.org/mongo-driver/bson",
"go.mongodb.org/mongo-driver/mongo",
"go.mongodb.org/mongo-driver/mongo/options",
"golang.org/x/crypto/bcrypt",
"golang.org/x/net/context",
"google.golang.org/grpc",
+23 -18
View File
@@ -1,7 +1,3 @@
[[constraint]]
name = "github.com/asaskevich/govalidator"
version = "9.0.0"
[[constraint]]
name = "github.com/cisco/senml"
branch = "master"
@@ -46,10 +42,6 @@
name = "github.com/prometheus/client_golang"
version = "0.9.1"
[[constraint]]
name = "github.com/satori/go.uuid"
version = "1.2.0"
[[constraint]]
name = "github.com/sony/gobreaker"
version = "0.4.0"
@@ -79,19 +71,16 @@
branch = "master"
[[constraint]]
name = "github.com/influxdata/influxdb"
version = "1.6.4"
[[constraint]]
name = "github.com/mongodb/mongo-go-driver"
version = "0.0.5"
[[constraint]]
name = "github.com/gocql/gocql"
name = "github.com/jmoiron/sqlx"
branch = "master"
[[constraint]]
name = "github.com/dereulenspiegel/coap-mux"
name = "github.com/influxdata/influxdb"
version = "1.6.4"
[[constraint]]
name = "github.com/gocql/gocql"
branch = "master"
[[constraint]]
@@ -110,6 +99,22 @@
name = "github.com/go-redis/redis"
version = "v6.15.0"
[[constraint]]
name = "go.mongodb.org/mongo-driver"
version = "~1.0.0"
[[constraint]]
name = "github.com/BurntSushi/toml"
version = "~v0.3.1"
[[constraint]]
name = "github.com/gofrs/uuid"
version = "~v3.2.0"
[[constraint]]
name = "github.com/opentracing/opentracing-go"
version = "~v1.1.0"
[prune]
go-tests = true
unused-packages = true
+62 -28
View File
@@ -4,49 +4,61 @@
## SPDX-License-Identifier: Apache-2.0
BUILD_DIR = build
SERVICES = users things http normalizer ws coap lora influxdb-writer influxdb-reader mongodb-writer mongodb-reader cassandra-writer cassandra-reader cli bootstrap
SERVICES = users things http normalizer ws coap lora influxdb-writer influxdb-reader mongodb-writer mongodb-reader cassandra-writer cassandra-reader postgres-writer postgres-reader cli bootstrap
DOCKERS = $(addprefix docker_,$(SERVICES))
DOCKERS_DEV = $(addprefix docker_dev_,$(SERVICES))
CGO_ENABLED ?= 0
GOOS ?= linux
GOARCH ?= amd64
define compile_service
CGO_ENABLED=$(CGO_ENABLED) GOOS=$(GOOS) GOARCH=$(GOARCH) GOARM=$(GOARM) go build -ldflags "-s -w" -o ${BUILD_DIR}/mainflux-$(1) cmd/$(1)/main.go
endef
define make_docker
docker build --no-cache --build-arg SVC_NAME=$(subst docker_,,$(1)) --tag=mainflux/$(subst docker_,,$(1)) -f docker/Dockerfile .
docker build \
--no-cache \
--build-arg SVC=$(subst docker_,,$(1)) \
--build-arg GOARCH=$(GOARCH) \
--build-arg GOARM=$(GOARM) \
--tag=mainflux/$(subst docker_,,$(1))-$(2) \
-f docker/Dockerfile .
endef
define make_docker_dev
docker build --build-arg SVC_NAME=$(subst docker_dev_,,$(1)) --tag=mainflux/$(subst docker_dev_,,$(1)) -f docker/Dockerfile.dev ./build
docker build \
--no-cache \
--build-arg SVC=$(subst docker_dev_,,$(1)) \
--tag=mainflux/$(subst docker_dev_,,$(1)) \
-f docker/Dockerfile.dev ./build
endef
all: $(SERVICES) mqtt
.PHONY: all $(SERVICES) dockers dockers_dev latest release mqtt ui
.PHONY: all $(SERVICES) dockers dockers_dev latest release mqtt ui latest_manifest
clean:
rm -rf ${BUILD_DIR}
rm -rf mqtt/node_modules
cleandocker: cleanghost
cleandocker:
# Stop all containers (if running)
docker-compose -f docker/docker-compose.yml stop
# Remove mainflux containers
docker ps -f name=mainflux -aq | xargs -r docker rm
# Remove exited containers
docker ps -f name=mainflux -f status=dead -f status=exited -aq | xargs -r docker rm -v
# Remove unused images
docker images "mainflux\/*" -f dangling=true -q | xargs -r docker rmi
# Remove old mainflux images
docker images -q mainflux\/* | xargs -r docker rmi
# Clean ghost docker images
cleanghost:
# Remove exited containers
docker ps -f status=dead -f status=exited -aq | xargs -r docker rm -v
# Remove unused images
docker images -f dangling=true -q | xargs -r docker rmi
ifdef pv
# Remove unused volumes
docker volume ls -f dangling=true -q | xargs -r docker volume rm
docker volume ls -f name=mainflux -f dangling=true -q | xargs -r docker volume rm
endif
install:
cp ${BUILD_DIR}/* $(GOBIN)
@@ -60,20 +72,24 @@ $(SERVICES):
$(call compile_service,$(@))
$(DOCKERS):
$(call make_docker,$(@))
$(call make_docker,$(@),$(GOARCH))
$(DOCKERS_DEV):
$(call make_docker_dev,$(@))
docker_ui:
$(MAKE) -C ui docker
docker_mqtt:
# MQTT Docker build must be done from root dir because it copies .proto files
docker build --tag=mainflux/mqtt -f mqtt/Dockerfile .
ifeq ($(GOARCH), arm)
docker build --tag=mainflux/mqtt-arm -f mqtt/Dockerfile.arm .
else
docker build --tag=mainflux/mqtt-amd64 -f mqtt/Dockerfile .
endif
dockers: $(DOCKERS) docker_ui docker_mqtt
$(DOCKERS_DEV):
$(call make_docker_dev,$(@))
dockers_dev: $(DOCKERS_DEV)
ui:
@@ -84,28 +100,46 @@ mqtt:
define docker_push
for svc in $(SERVICES); do \
docker push mainflux/$$svc:$(1); \
docker push mainflux/$$svc-$(1):$(2); \
done
docker push mainflux/ui:$(1)
docker push mainflux/mqtt:$(1)
docker push mainflux/ui-$(1):$(2)
docker push mainflux/mqtt-$(1):$(2)
endef
changelog:
git log $(shell git describe --tags --abbrev=0)..HEAD --pretty=format:"- %s"
define docker_manifest
for svc in $(SERVICES); do \
docker manifest create mainflux/$$svc:$(1) mainflux/$$svc-amd64:$(1) mainflux/$$svc-arm:$(1); \
docker manifest annotate mainflux/$$svc:$(1) mainflux/$$svc-arm:$(1) --arch arm --variant v7; \
docker manifest push mainflux/$$svc:$(1); \
done
docker manifest create mainflux/ui:$(1) mainflux/ui-amd64:$(1) mainflux/ui-arm:$(1)
docker manifest annotate mainflux/ui:$(1) mainflux/ui-arm:$(1) --arch arm --variant v7
docker manifest push mainflux/ui:$(1)
docker manifest create mainflux/mqtt:$(1) mainflux/mqtt-amd64:$(1) mainflux/mqtt-arm:$(1)
docker manifest annotate mainflux/mqtt:$(1) mainflux/mqtt-arm:$(1) --arch arm --variant v7
docker manifest push mainflux/mqtt:$(1)
endef
latest: dockers
$(call docker_push,latest)
$(call docker_push,$(GOARCH),latest)
latest_manifest:
$(call docker_manifest,latest)
release:
$(eval version = $(shell git describe --abbrev=0 --tags))
git checkout $(version)
$(MAKE) dockers
GOARCH=$(GOARCH) GOARM=$(GOARM) $(MAKE) dockers
for svc in $(SERVICES); do \
docker tag mainflux/$$svc mainflux/$$svc:$(version); \
docker tag mainflux/$$svc-$(GOARCH) mainflux/$$svc-$(GOARCH):$(version); \
done
docker tag mainflux/ui mainflux/ui:$(version)
docker tag mainflux/mqtt mainflux/mqtt:$(version)
$(call docker_push,$(version))
docker tag mainflux/ui mainflux/ui-$(GOARCH):$(version)
docker tag mainflux/mqtt mainflux/mqtt-$(GOARCH):$(version)
$(call docker_push,$(GOARCH),$(version))
$(call docker_manifest,$(version))
rundev:
cd scripts && ./run.sh
+1 -1
View File
@@ -10,7 +10,7 @@
Mainflux is modern, scalable, secure open source and patent-free IoT cloud platform written in Go.
It accepts user, device, and application connections over various network protocols (i.e. HTTP,
It accepts user and thing connections over various network protocols (i.e. HTTP,
MQTT, WebSocket, CoAP), thus making a seamless bridge between them. It is used as the IoT middleware
for building complex IoT solutions.
+18 -4
View File
@@ -1,6 +1,7 @@
# BOOTSTRAP SERVICE
New devices need to be configured properly and connected to the Mainflux. Bootstrap service is used in order to accomplish that. This service provides the following features:
1) Creating new Mainflux Things
2) Providing basic configuration for the newly created Things
3) Enabling/disabling Things
@@ -9,9 +10,10 @@ Pre-provisioning a new Thing is as simple as sending Configuration data to the B
In order to bootstrap successfully, the Thing needs to send bootstrapping request to the specific URL, as well as a secret key. This key and URL are pre-provisioned during the manufacturing process. If the Thing is provisioned on the Bootstrap service side, the corresponding configuration will be sent as a response. Otherwise, the Thing will be saved so that it can be provisioned later.
***Thing Configuration***
## Thing Configuration Entity
Thing Configuration consists of two logical parts: the custom configuration that can be interpreted by the Thing itself and Mainflux-related configuration. Mainflux config contains:
1) corresponding Mainflux Thing ID
2) corresponding Mainflux Thing key
3) list of the Mainflux channels the Thing is connected to
@@ -27,6 +29,8 @@ Enabling and disabling Thing (adding Thing to/from whitelist) is as simple as co
Switching between states `Active` and `Inactive` enables and disables Thing, respectively.
Thing configuration also contains the so-called `external ID` and `external key`. An external ID is a unique identifier of corresponding Thing. For example, a device MAC address is a good choice for external ID. External key is a secret key that is used for authentication during the bootstrapping procedure.
## Configuration
The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.
@@ -54,7 +58,12 @@ The service is configured using the environment variables presented in the follo
| MF_THINGS_ES_URL | Things service event source URL | localhost:6379 |
| MF_THINGS_ES_PASS | Things service event source password | |
| MF_THINGS_ES_DB | Things service event source database | 0 |
| MF_BOOTSTRAP_ES_URL | Bootstrap service event source URL | localhost:6379 |
| MF_BOOTSTRAP_ES_PASS | Bootstrap service event source password | |
| MF_BOOTSTRAP_ES_DB | Bootstrap service event source database | 0 |
| MF_BOOTSTRAP_INSTANCE_NAME | Bootstrap service instance name | bootstrap |
| MF_JAEGER_URL | Jaeger server URL | localhost:6831 |
| MF_BOOTSTRAP_THINGS_TIMEOUT | Things gRPC request timeout in seconds | 1 |
## Deployment
@@ -91,11 +100,16 @@ version: "2"
MF_SDK_BASE_URL: [Base SDK URL for the Mainflux services]
MF_SDK_THINGS_PREFIX: [SDK prefix for Things service]
MF_USERS_URL: [Users service URL]
MF_THINGS_ES_URL: [Things service event source URL]
MF_THINGS_ES_URL: [Things service event source URL]
MF_THINGS_ES_PASS: [Things service event source password]
MF_THINGS_ES_DB: [Things service event source database]
MF_BOOTSTRAP_ES_URL: [Bootstrap service event source URL]
MF_BOOTSTRAP_ES_PASS: [Bootstrap service event source password]
MF_BOOTSTRAP_ES_DB: [Bootstrap service event source database]
MF_BOOTSTRAP_INSTANCE_NAME: [Bootstrap service instance name]
```
MF_JAEGER_URL: [Jaeger server URL]
MF_BOOTSTRAP_THINGS_TIMEOUT: [Things gRPC request timeout in seconds]
```
To start the service outside of the container, execute the following shell script:
@@ -112,7 +126,7 @@ make bootstrap
make install
# set the environment variables and run the service
MF_BOOTSTRAP_LOG_LEVEL=[Bootstrap log level] MF_BOOTSTRAP_DB_HOST=[Database host address] MF_BOOTSTRAP_DB_PORT=[Database host port] MF_BOOTSTRAP_DB_USER=[Database user] MF_BOOTSTRAP_DB_PASS=[Database password] MF_BOOTSTRAP_DB=[Name of the database used by the service] MF_BOOTSTRAP_DB_SSL_MODE=[SSL mode to connect to the database with] MF_BOOTSTRAP_DB_SSL_CERT=[Path to the PEM encoded certificate file] MF_BOOTSTRAP_DB_SSL_KEY=[Path to the PEM encoded key file] MF_BOOTSTRAP_DB_SSL_ROOT_CERT=[Path to the PEM encoded root certificate file] MF_BOOTSTRAP_CLIENT_TLS=[Boolean value to enable/disable client TLS] MF_BOOTSTRAP_CA_CERTS=[Path to trusted CAs in PEM format] MF_BOOTSTRAP_PORT=[Service HTTP port] MF_BOOTSTRAP_SERVER_CERT=[Path to server certificate] MF_BOOTSTRAP_SERVER_KEY=[Path to server key] MF_SDK_BASE_URL=[Base SDK URL for the Mainflux services] MF_SDK_THINGS_PREFIX=[SDK prefix for Things service] MF_USERS_URL=[Users service URL] $GOBIN/mainflux-bootstrap
MF_BOOTSTRAP_LOG_LEVEL=[Bootstrap log level] MF_BOOTSTRAP_DB_HOST=[Database host address] MF_BOOTSTRAP_DB_PORT=[Database host port] MF_BOOTSTRAP_DB_USER=[Database user] MF_BOOTSTRAP_DB_PASS=[Database password] MF_BOOTSTRAP_DB=[Name of the database used by the service] MF_BOOTSTRAP_DB_SSL_MODE=[SSL mode to connect to the database with] MF_BOOTSTRAP_DB_SSL_CERT=[Path to the PEM encoded certificate file] MF_BOOTSTRAP_DB_SSL_KEY=[Path to the PEM encoded key file] MF_BOOTSTRAP_DB_SSL_ROOT_CERT=[Path to the PEM encoded root certificate file] MF_BOOTSTRAP_CLIENT_TLS=[Boolean value to enable/disable client TLS] MF_BOOTSTRAP_CA_CERTS=[Path to trusted CAs in PEM format] MF_BOOTSTRAP_PORT=[Service HTTP port] MF_BOOTSTRAP_SERVER_CERT=[Path to server certificate] MF_BOOTSTRAP_SERVER_KEY=[Path to server key] MF_SDK_BASE_URL=[Base SDK URL for the Mainflux services] MF_SDK_THINGS_PREFIX=[SDK prefix for Things service] MF_USERS_URL=[Users service URL] MF_JAEGER_URL=[Jaeger server URL] MF_BOOTSTRAP_THINGS_TIMEOUT=[Things gRPC request timeout in seconds] $GOBIN/mainflux-bootstrap
```
Setting `MF_BOOTSTRAP_CA_CERTS` expects a file in PEM format of trusted CAs. This will enable TLS against the Users gRPC endpoint trusting only those CAs that are provided.
+1 -1
View File
@@ -5,5 +5,5 @@
// SPDX-License-Identifier: Apache-2.0
//
// Package api contains implementation of things service HTTP API.
// Package api contains implementation of bootstrap service HTTP API.
package api
+20
View File
@@ -32,6 +32,9 @@ func addEndpoint(svc bootstrap.Service) endpoint.Endpoint {
ExternalKey: req.ExternalKey,
MFChannels: channels,
Name: req.Name,
ClientCert: req.ClientCert,
ClientKey: req.ClientKey,
CACert: req.CACert,
Content: req.Content,
}
@@ -49,6 +52,23 @@ func addEndpoint(svc bootstrap.Service) endpoint.Endpoint {
}
}
func updateCertEndpoint(svc bootstrap.Service) endpoint.Endpoint {
return func(_ context.Context, request interface{}) (interface{}, error) {
req := request.(updateCertReq)
if err := req.validate(); err != nil {
return nil, err
}
if err := svc.UpdateCert(req.key, req.thingKey, req.ClientCert, req.ClientKey, req.CACert); err != nil {
return nil, err
}
res := configRes{}
return res, nil
}
}
func viewEndpoint(svc bootstrap.Service) endpoint.Endpoint {
return func(_ context.Context, request interface{}) (interface{}, error) {
req := request.(entityReq)
+111 -10
View File
@@ -24,7 +24,8 @@ import (
"github.com/mainflux/mainflux/bootstrap/mocks"
mfsdk "github.com/mainflux/mainflux/sdk/go"
"github.com/mainflux/mainflux/things"
thingsapi "github.com/mainflux/mainflux/things/api/http"
thingsapi "github.com/mainflux/mainflux/things/api/things/http"
"github.com/opentracing/opentracing-go/mocktracer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -37,7 +38,6 @@ const (
channelsNum = 3
contentType = "application/json"
wrongID = "wrong_id"
metadata = `{"meta": "data"}`
addExternalID = "external-id"
addExternalKey = "external-key"
addName = "name"
@@ -46,6 +46,7 @@ const (
var (
addChannels = []string{"1"}
metadata = map[string]interface{}{"meta": "data"}
addReq = struct {
ThingID string `json:"thing_id"`
ExternalID string `json:"external_id"`
@@ -62,13 +63,19 @@ var (
}
updateReq = struct {
Channels []string `json:"channels"`
Content string `json:"content"`
State bootstrap.State `json:"state"`
Channels []string `json:"channels,omitempty"`
Content string `json:"content,omitempty"`
State bootstrap.State `json:"state,omitempty"`
ClientCert string `json:"client_cert,omitempty"`
ClientKey string `json:"client_key,omitempty"`
CACert string `json:"ca_cert,omitempty"`
}{
Channels: []string{"2", "3"},
Content: "config update",
State: 1,
Channels: []string{"2", "3"},
Content: "config update",
State: 1,
ClientCert: "newcert",
ClientKey: "newkey",
CACert: "newca",
}
)
@@ -137,7 +144,7 @@ func newThingsService(users mainflux.UsersServiceClient) things.Service {
}
func newThingsServer(svc things.Service) *httptest.Server {
mux := thingsapi.MakeHandler(svc)
mux := thingsapi.MakeHandler(mocktracer.New(), svc)
return httptest.NewServer(mux)
}
@@ -288,7 +295,7 @@ func TestView(t *testing.T) {
c.MFChannels = append(c.MFChannels, bootstrap.Channel{
ID: ch.ID,
Name: fmt.Sprintf("%s%s", "name ", id),
Metadata: fmt.Sprintf("{\"type\":\"some type %s\"}", id),
Metadata: map[string]interface{}{"type": fmt.Sprintf("some type %s", id)},
})
}
@@ -466,6 +473,100 @@ func TestUpdate(t *testing.T) {
assert.Equal(t, tc.status, res.StatusCode, fmt.Sprintf("%s: expected status code %d got %d", tc.desc, tc.status, res.StatusCode))
}
}
func TestUpdateCert(t *testing.T) {
users := mocks.NewUsersService(map[string]string{validToken: email})
ts := newThingsServer(newThingsService(users))
svc := newService(users, nil, ts.URL)
bs := newBootstrapServer(svc)
c := newConfig([]bootstrap.Channel{bootstrap.Channel{ID: "1"}})
saved, err := svc.Add(validToken, c)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
data := toJSON(updateReq)
cases := []struct {
desc string
req string
key string
auth string
contentType string
status int
}{
{
desc: "update unauthorized",
req: data,
key: saved.MFKey,
auth: invalidToken,
contentType: contentType,
status: http.StatusForbidden,
},
{
desc: "update with an empty token",
req: data,
key: saved.MFKey,
auth: "",
contentType: contentType,
status: http.StatusForbidden,
},
{
desc: "update a valid config",
req: data,
key: saved.MFKey,
auth: validToken,
contentType: contentType,
status: http.StatusOK,
},
{
desc: "update a config with wrong content type",
req: data,
key: saved.MFKey,
auth: validToken,
contentType: "",
status: http.StatusUnsupportedMediaType,
},
{
desc: "update a non-existing config",
req: data,
key: wrongID,
auth: validToken,
contentType: contentType,
status: http.StatusNotFound,
},
{
desc: "update a config with invalid request format",
req: "}",
key: saved.MFKey,
auth: validToken,
contentType: contentType,
status: http.StatusBadRequest,
},
{
desc: "update a config with an empty request",
key: saved.MFKey,
req: "",
auth: validToken,
contentType: contentType,
status: http.StatusBadRequest,
},
}
for _, tc := range cases {
req := testRequest{
client: bs.Client(),
method: http.MethodPut,
url: fmt.Sprintf("%s/things/configs/certs/%s", bs.URL, tc.key),
contentType: tc.contentType,
token: tc.auth,
body: strings.NewReader(tc.req),
}
res, err := req.make()
assert.Nil(t, err, fmt.Sprintf("%s: unexpected error %s", tc.desc, err))
assert.Equal(t, tc.status, res.StatusCode, fmt.Sprintf("%s: expected status code %d got %d", tc.desc, tc.status, res.StatusCode))
}
}
func TestUpdateConnections(t *testing.T) {
users := mocks.NewUsersService(map[string]string{validToken: email})
+18 -5
View File
@@ -29,7 +29,7 @@ func NewLoggingMiddleware(svc bootstrap.Service, logger log.Logger) bootstrap.Se
return &loggingMiddleware{logger, svc}
}
func (lm *loggingMiddleware) Add(key string, thing bootstrap.Config) (saved bootstrap.Config, err error) {
func (lm *loggingMiddleware) Add(key string, cfg bootstrap.Config) (saved bootstrap.Config, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method add for key %s and thing %s took %s to complete", key, saved.MFThing, time.Since(begin))
if err != nil {
@@ -39,7 +39,7 @@ func (lm *loggingMiddleware) Add(key string, thing bootstrap.Config) (saved boot
lm.logger.Info(fmt.Sprintf("%s without errors.", message))
}(time.Now())
return lm.svc.Add(key, thing)
return lm.svc.Add(key, cfg)
}
func (lm *loggingMiddleware) View(key, id string) (saved bootstrap.Config, err error) {
@@ -55,9 +55,9 @@ func (lm *loggingMiddleware) View(key, id string) (saved bootstrap.Config, err e
return lm.svc.View(key, id)
}
func (lm *loggingMiddleware) Update(key string, thing bootstrap.Config) (err error) {
func (lm *loggingMiddleware) Update(key string, cfg bootstrap.Config) (err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method update for key %s and thing %s took %s to complete", key, thing.MFThing, time.Since(begin))
message := fmt.Sprintf("Method update for key %s and thing %s took %s to complete", key, cfg.MFThing, time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
return
@@ -65,7 +65,20 @@ func (lm *loggingMiddleware) Update(key string, thing bootstrap.Config) (err err
lm.logger.Info(fmt.Sprintf("%s without errors.", message))
}(time.Now())
return lm.svc.Update(key, thing)
return lm.svc.Update(key, cfg)
}
func (lm *loggingMiddleware) UpdateCert(key, thingKey, clientCert, clientKey, caCert string) (err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method update_cert for thing with key %s took %s to complete", thingKey, time.Since(begin))
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
return
}
lm.logger.Info(fmt.Sprintf("%s without errors.", message))
}(time.Now())
return lm.svc.UpdateCert(key, thingKey, clientCert, clientKey, caCert)
}
func (lm *loggingMiddleware) UpdateConnections(key, id string, connections []string) (err error) {
+13 -4
View File
@@ -34,13 +34,13 @@ func MetricsMiddleware(svc bootstrap.Service, counter metrics.Counter, latency m
}
}
func (mm *metricsMiddleware) Add(key string, thing bootstrap.Config) (saved bootstrap.Config, err error) {
func (mm *metricsMiddleware) Add(key string, cfg bootstrap.Config) (saved bootstrap.Config, err error) {
defer func(begin time.Time) {
mm.counter.With("method", "add").Add(1)
mm.latency.With("method", "add").Observe(time.Since(begin).Seconds())
}(time.Now())
return mm.svc.Add(key, thing)
return mm.svc.Add(key, cfg)
}
func (mm *metricsMiddleware) View(id, key string) (saved bootstrap.Config, err error) {
@@ -52,13 +52,22 @@ func (mm *metricsMiddleware) View(id, key string) (saved bootstrap.Config, err e
return mm.svc.View(id, key)
}
func (mm *metricsMiddleware) Update(key string, thing bootstrap.Config) (err error) {
func (mm *metricsMiddleware) Update(key string, cfg bootstrap.Config) (err error) {
defer func(begin time.Time) {
mm.counter.With("method", "update").Add(1)
mm.latency.With("method", "update").Observe(time.Since(begin).Seconds())
}(time.Now())
return mm.svc.Update(key, thing)
return mm.svc.Update(key, cfg)
}
func (mm *metricsMiddleware) UpdateCert(key, thingKey, clientCert, clientKey, caCert string) (err error) {
defer func(begin time.Time) {
mm.counter.With("method", "update_cert").Add(1)
mm.latency.With("method", "update_cert").Observe(time.Since(begin).Seconds())
}(time.Now())
return mm.svc.UpdateCert(key, thingKey, clientCert, clientKey, caCert)
}
func (mm *metricsMiddleware) UpdateConnections(key, id string, connections []string) (err error) {
+23
View File
@@ -21,6 +21,9 @@ type addReq struct {
Channels []string `json:"channels"`
Name string `json:"name"`
Content string `json:"content"`
ClientCert string `json:"client_cert"`
ClientKey string `json:"client_key"`
CACert string `json:"ca_cert"`
}
func (req addReq) validate() error {
@@ -71,6 +74,26 @@ func (req updateReq) validate() error {
return nil
}
type updateCertReq struct {
key string
thingKey string
ClientCert string `json:"client_cert"`
ClientKey string `json:"client_key"`
CACert string `json:"ca_cert"`
}
func (req updateCertReq) validate() error {
if req.key == "" {
return bootstrap.ErrUnauthorizedAccess
}
if req.thingKey == "" {
return bootstrap.ErrNotFound
}
return nil
}
type updateConnReq struct {
key string
id string
+32
View File
@@ -114,6 +114,38 @@ func TestUpdateReqValidation(t *testing.T) {
}
}
func TestUpdateCertReqValidation(t *testing.T) {
cases := []struct {
desc string
key string
thingKey string
err error
}{
{
desc: "empty key",
key: "",
thingKey: "thingKey",
err: bootstrap.ErrUnauthorizedAccess,
},
{
desc: "empty thing key",
key: "key",
thingKey: "",
err: bootstrap.ErrNotFound,
},
}
for _, tc := range cases {
req := updateCertReq{
key: tc.key,
thingKey: tc.thingKey,
}
err := req.validate()
assert.Equal(t, tc.err, err, fmt.Sprintf("%s: expected %s got %s\n", tc.desc, tc.err, err))
}
}
func TestUpdateConnReqValidation(t *testing.T) {
cases := []struct {
desc string
+20
View File
@@ -62,6 +62,12 @@ func MakeHandler(svc bootstrap.Service, reader bootstrap.ConfigReader) http.Hand
encodeResponse,
opts...))
r.Put("/things/configs/certs/:key", kithttp.NewServer(
updateCertEndpoint(svc),
decodeUpdateCertRequest,
encodeResponse,
opts...))
r.Put("/things/configs/connections/:id", kithttp.NewServer(
updateConnEndpoint(svc),
decodeUpdateConnRequest,
@@ -131,6 +137,20 @@ func decodeUpdateRequest(_ context.Context, r *http.Request) (interface{}, error
return req, nil
}
func decodeUpdateCertRequest(_ context.Context, r *http.Request) (interface{}, error) {
if !strings.Contains(r.Header.Get("Content-Type"), contentType) {
return nil, errUnsupportedContentType
}
req := updateCertReq{key: r.Header.Get("Authorization")}
req.thingKey = bone.GetValue(r, "key")
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
return nil, err
}
return req, nil
}
func decodeUpdateConnRequest(_ context.Context, r *http.Request) (interface{}, error) {
if !strings.Contains(r.Header.Get("Content-Type"), contentType) {
return nil, errUnsupportedContentType
+9 -2
View File
@@ -16,6 +16,9 @@ type Config struct {
MFThing string
Owner string
Name string
ClientCert string
ClientKey string
CACert string
MFKey string
MFChannels []Channel
ExternalID string
@@ -28,7 +31,7 @@ type Config struct {
type Channel struct {
ID string
Name string
Metadata interface{}
Metadata map[string]interface{}
}
// Filter is used for the search filters.
@@ -64,10 +67,14 @@ type ConfigRepository interface {
// RetrieveByExternalID returns Config for given external ID.
RetrieveByExternalID(string, string) (Config, error)
// Update performs and update to an existing Config. A non-nil error is returned
// Update updates an existing Config. A non-nil error is returned
// to indicate operation failure.
Update(Config) error
// UpdateCerts updates an existing Config certificate and key.
// A non-nil error is returned to indicate operation failure.
UpdateCert(string, string, string, string, string) error
// UpdateConnections updates a list of Channels the Config is connected to
// adding new Channels if needed.
UpdateConnections(string, string, []Channel, []string) error
+26 -4
View File
@@ -31,7 +31,7 @@ type configRepositoryMock struct {
unknown map[string]string
}
// NewConfigsRepository creates in-memory thing repository.
// NewConfigsRepository creates in-memory config repository.
func NewConfigsRepository(unknown map[string]string) bootstrap.ConfigRepository {
return &configRepositoryMock{
configs: make(map[string]bootstrap.Config),
@@ -138,9 +138,9 @@ func (crm *configRepositoryMock) RetrieveByExternalID(externalKey, externalID st
crm.mu.Lock()
defer crm.mu.Unlock()
for _, thing := range crm.configs {
if thing.ExternalID == externalID && thing.ExternalKey == externalKey {
return thing, nil
for _, cfg := range crm.configs {
if cfg.ExternalID == externalID && cfg.ExternalKey == externalKey {
return cfg, nil
}
}
@@ -163,6 +163,27 @@ func (crm *configRepositoryMock) Update(config bootstrap.Config) error {
return nil
}
func (crm *configRepositoryMock) UpdateCert(owner, thingKey, clientCert, clientKey, caCert string) error {
crm.mu.Lock()
defer crm.mu.Unlock()
var forUpdate bootstrap.Config
for _, v := range crm.configs {
if v.MFKey == thingKey && v.Owner == owner {
forUpdate = v
break
}
}
if _, ok := crm.configs[forUpdate.MFThing]; !ok {
return bootstrap.ErrNotFound
}
forUpdate.ClientCert = clientCert
forUpdate.ClientKey = clientKey
forUpdate.CACert = caCert
crm.configs[forUpdate.MFThing] = forUpdate
return nil
}
func (crm *configRepositoryMock) UpdateConnections(key, id string, channels []bootstrap.Channel, connections []string) error {
crm.mu.Lock()
defer crm.mu.Unlock()
@@ -171,6 +192,7 @@ func (crm *configRepositoryMock) UpdateConnections(key, id string, channels []bo
if !ok {
return bootstrap.ErrNotFound
}
for _, ch := range channels {
crm.channels[ch.ID] = ch
}
+24 -16
View File
@@ -38,7 +38,7 @@ func NewThingsService(things map[string]things.Thing, channels map[string]things
}
}
func (svc *mainfluxThings) AddThing(owner string, thing things.Thing) (things.Thing, error) {
func (svc *mainfluxThings) AddThing(_ context.Context, owner string, thing things.Thing) (things.Thing, error) {
svc.mu.Lock()
defer svc.mu.Unlock()
@@ -55,7 +55,7 @@ func (svc *mainfluxThings) AddThing(owner string, thing things.Thing) (things.Th
return thing, nil
}
func (svc *mainfluxThings) ViewThing(owner, id string) (things.Thing, error) {
func (svc *mainfluxThings) ViewThing(_ context.Context, owner, id string) (things.Thing, error) {
svc.mu.Lock()
defer svc.mu.Unlock()
@@ -72,7 +72,7 @@ func (svc *mainfluxThings) ViewThing(owner, id string) (things.Thing, error) {
return things.Thing{}, things.ErrNotFound
}
func (svc *mainfluxThings) Connect(owner, chanID, thingID string) error {
func (svc *mainfluxThings) Connect(_ context.Context, owner, chanID, thingID string) error {
svc.mu.Lock()
defer svc.mu.Unlock()
@@ -89,7 +89,7 @@ func (svc *mainfluxThings) Connect(owner, chanID, thingID string) error {
return nil
}
func (svc *mainfluxThings) Disconnect(owner, chanID, thingID string) error {
func (svc *mainfluxThings) Disconnect(_ context.Context, owner, chanID, thingID string) error {
svc.mu.Lock()
defer svc.mu.Unlock()
@@ -121,7 +121,7 @@ func (svc *mainfluxThings) Disconnect(owner, chanID, thingID string) error {
return nil
}
func (svc *mainfluxThings) RemoveThing(owner, id string) error {
func (svc *mainfluxThings) RemoveThing(_ context.Context, owner, id string) error {
svc.mu.Lock()
defer svc.mu.Unlock()
@@ -151,50 +151,58 @@ func (svc *mainfluxThings) RemoveThing(owner, id string) error {
return nil
}
func (svc *mainfluxThings) ViewChannel(owner, id string) (things.Channel, error) {
func (svc *mainfluxThings) ViewChannel(_ context.Context, owner, id string) (things.Channel, error) {
if c, ok := svc.channels[id]; ok {
return c, nil
}
return things.Channel{}, things.ErrNotFound
}
func (svc *mainfluxThings) UpdateThing(string, things.Thing) error {
func (svc *mainfluxThings) UpdateThing(context.Context, string, things.Thing) error {
panic("not implemented")
}
func (svc *mainfluxThings) ListThings(string, uint64, uint64) (things.ThingsPage, error) {
func (svc *mainfluxThings) UpdateKey(context.Context, string, string, string) error {
panic("not implemented")
}
func (svc *mainfluxThings) ListChannelsByThing(string, string, uint64, uint64) (things.ChannelsPage, error) {
func (svc *mainfluxThings) ListThings(context.Context, string, uint64, uint64, string) (things.ThingsPage, error) {
panic("not implemented")
}
func (svc *mainfluxThings) ListThingsByChannel(string, string, uint64, uint64) (things.ThingsPage, error) {
func (svc *mainfluxThings) ListChannelsByThing(context.Context, string, string, uint64, uint64) (things.ChannelsPage, error) {
panic("not implemented")
}
func (svc *mainfluxThings) CreateChannel(string, things.Channel) (things.Channel, error) {
func (svc *mainfluxThings) ListThingsByChannel(context.Context, string, string, uint64, uint64) (things.ThingsPage, error) {
panic("not implemented")
}
func (svc *mainfluxThings) UpdateChannel(string, things.Channel) error {
func (svc *mainfluxThings) CreateChannel(context.Context, string, things.Channel) (things.Channel, error) {
panic("not implemented")
}
func (svc *mainfluxThings) ListChannels(string, uint64, uint64) (things.ChannelsPage, error) {
func (svc *mainfluxThings) UpdateChannel(context.Context, string, things.Channel) error {
panic("not implemented")
}
func (svc *mainfluxThings) RemoveChannel(string, string) error {
func (svc *mainfluxThings) ListChannels(context.Context, string, uint64, uint64, string) (things.ChannelsPage, error) {
panic("not implemented")
}
func (svc *mainfluxThings) CanAccess(string, string) (string, error) {
func (svc *mainfluxThings) RemoveChannel(context.Context, string, string) error {
panic("not implemented")
}
func (svc *mainfluxThings) Identify(string) (string, error) {
func (svc *mainfluxThings) CanAccess(context.Context, string, string) (string, error) {
panic("not implemented")
}
func (svc *mainfluxThings) CanAccessByID(context.Context, string, string) error {
panic("not implemented")
}
func (svc *mainfluxThings) Identify(context.Context, string) (string, error) {
panic("not implemented")
}
+254 -98
View File
@@ -9,9 +9,11 @@ package postgres
import (
"database/sql"
"encoding/json"
"fmt"
"strings"
"github.com/jmoiron/sqlx"
"github.com/lib/pq"
"github.com/mainflux/mainflux/bootstrap"
"github.com/mainflux/mainflux/logger"
@@ -32,29 +34,28 @@ const (
var _ bootstrap.ConfigRepository = (*configRepository)(nil)
type configRepository struct {
db *sql.DB
db *sqlx.DB
log logger.Logger
}
// NewConfigRepository instantiates a PostgreSQL implementation of thing
// NewConfigRepository instantiates a PostgreSQL implementation of config
// repository.
func NewConfigRepository(db *sql.DB, log logger.Logger) bootstrap.ConfigRepository {
func NewConfigRepository(db *sqlx.DB, log logger.Logger) bootstrap.ConfigRepository {
return &configRepository{db: db, log: log}
}
func (cr configRepository) Save(cfg bootstrap.Config, connections []string) (string, error) {
q := `INSERT INTO configs (mainflux_thing, owner, name, mainflux_key, external_id, external_key, content, state)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)`
content := nullString(cfg.Content)
name := nullString(cfg.Name)
tx, err := cr.db.Begin()
q := `INSERT INTO configs (mainflux_thing, owner, name, client_cert, client_key, ca_cert, mainflux_key, external_id, external_key, content, state)
VALUES (:mainflux_thing, :owner, :name, :client_cert, :client_key, :ca_cert, :mainflux_key, :external_id, :external_key, :content, :state)`
tx, err := cr.db.Beginx()
if err != nil {
return "", err
}
if _, err := tx.Exec(q, cfg.MFThing, cfg.Owner, name, cfg.MFKey, cfg.ExternalID, cfg.ExternalKey, content, cfg.State); err != nil {
dbcfg := toDBConfig(cfg)
if _, err := tx.NamedExec(q, dbcfg); err != nil {
e := err
if pqErr, ok := err.(*pq.Error); ok && pqErr.Code.Name() == duplicateErr {
e = bootstrap.ErrConflict
@@ -77,9 +78,9 @@ func (cr configRepository) Save(cfg bootstrap.Config, connections []string) (str
return "", err
}
q = "DELETE FROM unknown_configs WHERE external_id = $1 AND external_key = $2"
q = "DELETE FROM unknown_configs WHERE external_id = :external_id AND external_key = :external_key"
if _, err := tx.Exec(q, cfg.ExternalID, cfg.ExternalKey); err != nil {
if _, err := tx.NamedExec(q, dbcfg); err != nil {
cr.rollback("Failed to remove from unknown", tx, err)
return "", err
@@ -93,11 +94,16 @@ func (cr configRepository) Save(cfg bootstrap.Config, connections []string) (str
}
func (cr configRepository) RetrieveByID(key, id string) (bootstrap.Config, error) {
q := `SELECT mainflux_thing, mainflux_key, external_id, external_key, name, content, state FROM configs WHERE mainflux_thing = $1 AND owner = $2`
cfg := bootstrap.Config{MFThing: id, Owner: key, MFChannels: []bootstrap.Channel{}}
var name, content sql.NullString
if err := cr.db.QueryRow(q, id, key).
Scan(&cfg.MFThing, &cfg.MFKey, &cfg.ExternalID, &cfg.ExternalKey, &name, &content, &cfg.State); err != nil {
q := `SELECT mainflux_thing, mainflux_key, external_id, external_key, name, content, state
FROM configs
WHERE mainflux_thing = $1 AND owner = $2`
dbcfg := dbConfig{
MFThing: id,
Owner: key,
}
if err := cr.db.QueryRowx(q, id, key).StructScan(&dbcfg); err != nil {
empty := bootstrap.Config{}
if err == sql.ErrNoRows {
return empty, bootstrap.ErrNotFound
@@ -107,29 +113,35 @@ func (cr configRepository) RetrieveByID(key, id string) (bootstrap.Config, error
}
q = `SELECT mainflux_channel, name, metadata FROM channels ch
INNER JOIN connections conn
ON ch.mainflux_channel = conn.channel_id AND ch.owner = conn.config_owner
WHERE conn.config_id = $1 AND conn.config_owner = $2`
INNER JOIN connections conn
ON ch.mainflux_channel = conn.channel_id AND ch.owner = conn.config_owner
WHERE conn.config_id = :mainflux_thing AND conn.config_owner = :owner`
rows, err := cr.db.Query(q, cfg.MFThing, cfg.Owner)
rows, err := cr.db.NamedQuery(q, dbcfg)
if err != nil {
cr.log.Error(fmt.Sprintf("Failed to retrieve connected due to %s", err))
return bootstrap.Config{}, err
}
defer rows.Close()
chans := []bootstrap.Channel{}
for rows.Next() {
c := bootstrap.Channel{}
if err := rows.Scan(&c.ID, &c.Name, &c.Metadata); err != nil {
dbch := dbChannel{}
if err := rows.StructScan(&dbch); err != nil {
cr.log.Error(fmt.Sprintf("Failed to read connected thing due to %s", err))
return bootstrap.Config{}, err
}
dbch.Owner = nullString(dbcfg.Owner)
cfg.MFChannels = append(cfg.MFChannels, c)
ch, err := toChannel(dbch)
if err != nil {
return bootstrap.Config{}, err
}
chans = append(chans, ch)
}
cfg.Content = content.String
cfg.Name = name.String
cfg := toConfig(dbcfg)
cfg.MFChannels = chans
return cfg, nil
}
@@ -139,7 +151,7 @@ func (cr configRepository) RetrieveAll(key string, filter bootstrap.Filter, offs
n := len(params)
q := `SELECT mainflux_thing, mainflux_key, external_id, external_key, name, content, state
FROM configs %s ORDER BY mainflux_thing LIMIT $%d OFFSET $%d`
FROM configs %s ORDER BY mainflux_thing LIMIT $%d OFFSET $%d`
q = fmt.Sprintf(q, search, n+1, n+2)
rows, err := cr.db.Query(q, append(params, limit, offset)...)
@@ -181,12 +193,15 @@ func (cr configRepository) RetrieveAll(key string, filter bootstrap.Filter, offs
}
func (cr configRepository) RetrieveByExternalID(externalKey, externalID string) (bootstrap.Config, error) {
q := `SELECT mainflux_thing, mainflux_key, owner, name, content, state FROM configs WHERE external_key = $1 AND external_id = $2`
cfg := bootstrap.Config{ExternalID: externalID, ExternalKey: externalKey, MFChannels: []bootstrap.Channel{}}
var name, content sql.NullString
q := `SELECT mainflux_thing, mainflux_key, owner, name, client_cert, client_key, ca_cert, content, state
FROM configs
WHERE external_key = $1 AND external_id = $2`
dbcfg := dbConfig{
ExternalID: externalID,
ExternalKey: externalKey,
}
if err := cr.db.QueryRow(q, externalKey, externalID).
Scan(&cfg.MFThing, &cfg.MFKey, &cfg.Owner, &name, &content, &cfg.State); err != nil {
if err := cr.db.QueryRowx(q, externalKey, externalID).StructScan(&dbcfg); err != nil {
empty := bootstrap.Config{}
if err == sql.ErrNoRows {
return empty, bootstrap.ErrNotFound
@@ -195,29 +210,36 @@ func (cr configRepository) RetrieveByExternalID(externalKey, externalID string)
}
q = `SELECT mainflux_channel, name, metadata FROM channels ch
INNER JOIN connections conn
ON ch.mainflux_channel = conn.channel_id AND ch.owner = conn.config_owner
WHERE conn.config_id = $1 AND conn.config_owner = $2`
INNER JOIN connections conn
ON ch.mainflux_channel = conn.channel_id AND ch.owner = conn.config_owner
WHERE conn.config_id = :mainflux_thing AND conn.config_owner = :owner`
rows, err := cr.db.Query(q, cfg.MFThing, cfg.Owner)
rows, err := cr.db.NamedQuery(q, dbcfg)
if err != nil {
cr.log.Error(fmt.Sprintf("Failed to retrieve connected due to %s", err))
return bootstrap.Config{}, err
}
defer rows.Close()
channels := []bootstrap.Channel{}
for rows.Next() {
c := bootstrap.Channel{}
if err := rows.Scan(&c.ID, &c.Name, &c.Metadata); err != nil {
dbch := dbChannel{}
if err := rows.StructScan(&dbch); err != nil {
cr.log.Error(fmt.Sprintf("Failed to read connected thing due to %s", err))
return bootstrap.Config{}, err
}
cfg.MFChannels = append(cfg.MFChannels, c)
ch, err := toChannel(dbch)
if err != nil {
cr.log.Error(fmt.Sprintf("Failed to deserialize channel due to %s", err))
return bootstrap.Config{}, err
}
channels = append(channels, ch)
}
cfg.Content = content.String
cfg.Name = name.String
cfg := toConfig(dbcfg)
cfg.MFChannels = channels
return cfg, nil
}
@@ -245,8 +267,28 @@ func (cr configRepository) Update(cfg bootstrap.Config) error {
return nil
}
func (cr configRepository) UpdateCert(key, thingKey, clientCert, clientKey, caCert string) error {
q := `UPDATE configs SET client_cert = $1, client_key = $2, ca_cert = $3 WHERE mainflux_key = $4 AND owner = $5`
res, err := cr.db.Exec(q, clientCert, clientKey, caCert, thingKey, key)
if err != nil {
return err
}
cnt, err := res.RowsAffected()
if err != nil {
return err
}
if cnt == 0 {
return bootstrap.ErrNotFound
}
return nil
}
func (cr configRepository) UpdateConnections(key, id string, channels []bootstrap.Channel, connections []string) error {
tx, err := cr.db.Begin()
tx, err := cr.db.Beginx()
if err != nil {
return err
}
@@ -309,19 +351,28 @@ func (cr configRepository) ChangeState(key, id string, state bootstrap.State) er
}
func (cr configRepository) ListExisting(key string, ids []string) ([]bootstrap.Channel, error) {
q := "SELECT mainflux_channel, name, metadata FROM channels WHERE owner = $1 AND mainflux_channel = ANY ($2)"
var channels []bootstrap.Channel
if len(ids) == 0 {
return channels, nil
}
rows, err := cr.db.Query(q, key, pq.Array(ids))
q := "SELECT mainflux_channel, name, metadata FROM channels WHERE owner = $1 AND mainflux_channel = ANY ($2)"
rows, err := cr.db.Queryx(q, key, pq.Array(ids))
if err != nil {
return []bootstrap.Channel{}, err
}
var channels []bootstrap.Channel
for rows.Next() {
var ch bootstrap.Channel
if err := rows.Scan(&ch.ID, &ch.Name, &ch.Metadata); err != nil {
var dbch dbChannel
if err := rows.StructScan(&dbch); err != nil {
cr.log.Error(fmt.Sprintf("Failed to read retrieved channels due to %s", err))
return []bootstrap.Channel{}, nil
return []bootstrap.Channel{}, err
}
ch, err := toChannel(dbch)
if err != nil {
cr.log.Error(fmt.Sprintf("Failed to deserialize channel due to %s", err))
return []bootstrap.Channel{}, err
}
channels = append(channels, ch)
@@ -391,8 +442,13 @@ func (cr configRepository) RemoveThing(id string) error {
}
func (cr configRepository) UpdateChannel(channel bootstrap.Channel) error {
q := `UPDATE channels SET name = $1, metadata = $2 WHERE mainflux_channel = $3`
_, err := cr.db.Exec(q, channel.Name, channel.Metadata, channel.ID)
dbch, err := toDBChannel("", channel)
if err != nil {
return err
}
q := `UPDATE channels SET name = :name, metadata = :metadata WHERE mainflux_channel = :mainflux_channel`
_, err = cr.db.NamedExec(q, dbch)
return err
}
@@ -435,7 +491,7 @@ func (cr configRepository) retrieveAll(key string, filter bootstrap.Filter) (str
return fmt.Sprintf(template, f), params
}
func (cr configRepository) rollback(content string, tx *sql.Tx, err error) {
func (cr configRepository) rollback(content string, tx *sqlx.Tx, err error) {
cr.log.Error(fmt.Sprintf("%s %s", content, err))
if err := tx.Rollback(); err != nil {
@@ -443,24 +499,23 @@ func (cr configRepository) rollback(content string, tx *sql.Tx, err error) {
}
}
func insertChannels(key string, channels []bootstrap.Channel, tx *sql.Tx) error {
func insertChannels(key string, channels []bootstrap.Channel, tx *sqlx.Tx) error {
if len(channels) == 0 {
return nil
}
q := `INSERT INTO channels (mainflux_channel, owner, name, metadata) VALUES `
v := []interface{}{key}
var vals []string
// Since the first value is owner, start with the second one.
count := 2
var chans []dbChannel
for _, ch := range channels {
vals = append(vals, fmt.Sprintf("($%d, $1, $%d, $%d)", count, count+1, count+2))
v = append(v, ch.ID, ch.Name, ch.Metadata)
count += chanFieldsNum
dbch, err := toDBChannel(key, ch)
if err != nil {
return err
}
chans = append(chans, dbch)
}
q = fmt.Sprintf("%s%s", q, strings.Join(vals, ","))
if _, err := tx.Exec(q, v...); err != nil {
q := `INSERT INTO channels (mainflux_channel, owner, name, metadata)
VALUES (:mainflux_channel, :owner, :name, :metadata)`
if _, err := tx.NamedExec(q, chans); err != nil {
e := err
if pqErr, ok := err.(*pq.Error); ok && pqErr.Code.Name() == duplicateErr {
e = bootstrap.ErrConflict
@@ -471,44 +526,38 @@ func insertChannels(key string, channels []bootstrap.Channel, tx *sql.Tx) error
return nil
}
func insertConnections(cfg bootstrap.Config, connections []string, tx *sql.Tx) error {
func insertConnections(cfg bootstrap.Config, connections []string, tx *sqlx.Tx) error {
if len(connections) == 0 {
return nil
}
q := `INSERT INTO connections (config_id, channel_id, config_owner, channel_owner) VALUES`
v := []interface{}{cfg.MFThing, cfg.Owner}
var vals []string
// Since the first value is Config ID and the second and third
// are Config owner, start with the second one.
count := 3
for _, id := range connections {
vals = append(vals, fmt.Sprintf("($1, $%d, $2, $2)", count))
v = append(v, id)
count++
q := `INSERT INTO connections (config_id, channel_id, config_owner, channel_owner)
VALUES (:config_id, :channel_id, :config_owner, :channel_owner)`
conns := []dbConnection{}
for _, conn := range connections {
dbconn := dbConnection{
Config: cfg.MFThing,
Channel: conn,
ConfigOwner: cfg.Owner,
ChannelOwner: cfg.Owner,
}
conns = append(conns, dbconn)
}
q = fmt.Sprintf("%s%s", q, strings.Join(vals, ","))
_, err := tx.Exec(q, v...)
_, err := tx.NamedExec(q, conns)
return err
}
func updateConnections(key, id string, connections []string, tx *sql.Tx) error {
func updateConnections(key, id string, connections []string, tx *sqlx.Tx) error {
if len(connections) == 0 {
return nil
}
q := `DELETE FROM connections
WHERE config_id = $1 AND config_owner = $2 AND channel_owner = $2
AND channel_id NOT IN ($3)`
v := []interface{}{id, key}
v = append(v, pq.Array(connections))
res, err := tx.Exec(q, v...)
WHERE config_id = $1 AND config_owner = $2 AND channel_owner = $2
AND channel_id NOT IN ($3)`
res, err := tx.Exec(q, id, key, pq.Array(connections))
if err != nil {
return err
}
@@ -518,22 +567,21 @@ func updateConnections(key, id string, connections []string, tx *sql.Tx) error {
return err
}
q = `INSERT INTO connections (config_id, channel_id, config_owner, channel_owner) VALUES`
v = []interface{}{id, key}
var vals []string
q = `INSERT INTO connections (config_id, channel_id, config_owner, channel_owner)
VALUES (:config_id, :channel_id, :config_owner, :channel_owner)`
// Since the first value is Config ID and the second is Config
// owner, start with the third one.
count := 3
for _, chID := range connections {
vals = append(vals, fmt.Sprintf("($1, $%d, $2, $2)", count))
v = append(v, chID)
count++
conns := []dbConnection{}
for _, conn := range connections {
dbconn := dbConnection{
Config: id,
Channel: conn,
ConfigOwner: key,
ChannelOwner: key,
}
conns = append(conns, dbconn)
}
// Add connections for current list of channels. Ignore if already exists.
q = fmt.Sprintf("%s%s%s", q, strings.Join(vals, ","), "ON CONFLICT (config_id, config_owner, channel_id, channel_owner) DO NOTHING")
if _, err := tx.Exec(q, v...); err != nil {
if _, err := tx.NamedExec(q, conns); err != nil {
return err
}
@@ -556,3 +604,111 @@ func nullString(s string) sql.NullString {
Valid: true,
}
}
type dbConfig struct {
MFThing string `db:"mainflux_thing"`
Owner string `db:"owner"`
Name sql.NullString `db:"name"`
ClientCert sql.NullString `db:"client_cert"`
ClientKey sql.NullString `db:"client_key"`
CaCert sql.NullString `db:"ca_cert"`
MFKey string `db:"mainflux_key"`
ExternalID string `db:"external_id"`
ExternalKey string `db:"external_key"`
Content sql.NullString `db:"content"`
State bootstrap.State `db:"state"`
}
func toDBConfig(cfg bootstrap.Config) dbConfig {
return dbConfig{
MFThing: cfg.MFThing,
Owner: cfg.Owner,
Name: nullString(cfg.Name),
ClientCert: nullString(cfg.ClientCert),
ClientKey: nullString(cfg.ClientKey),
CaCert: nullString(cfg.CACert),
MFKey: cfg.MFKey,
ExternalID: cfg.ExternalID,
ExternalKey: cfg.ExternalKey,
Content: nullString(cfg.Content),
State: cfg.State,
}
}
func toConfig(dbcfg dbConfig) bootstrap.Config {
cfg := bootstrap.Config{
MFThing: dbcfg.MFThing,
Owner: dbcfg.Owner,
MFKey: dbcfg.MFKey,
ExternalID: dbcfg.ExternalID,
ExternalKey: dbcfg.ExternalKey,
State: dbcfg.State,
}
if dbcfg.Name.Valid {
cfg.Name = dbcfg.Name.String
}
if dbcfg.Content.Valid {
cfg.Content = dbcfg.Content.String
}
if dbcfg.ClientCert.Valid {
cfg.ClientCert = dbcfg.ClientCert.String
}
if dbcfg.ClientKey.Valid {
cfg.ClientKey = dbcfg.ClientKey.String
}
if dbcfg.CaCert.Valid {
cfg.CACert = dbcfg.CaCert.String
}
return cfg
}
type dbChannel struct {
ID string `db:"mainflux_channel"`
Name sql.NullString `db:"name"`
Owner sql.NullString `db:"owner"`
Metadata string `db:"metadata"`
}
func toDBChannel(owner string, ch bootstrap.Channel) (dbChannel, error) {
dbch := dbChannel{
ID: ch.ID,
Name: nullString(ch.Name),
Owner: nullString(owner),
}
metadata, err := json.Marshal(ch.Metadata)
if err != nil {
return dbChannel{}, err
}
dbch.Metadata = string(metadata)
return dbch, nil
}
func toChannel(dbch dbChannel) (bootstrap.Channel, error) {
ch := bootstrap.Channel{
ID: dbch.ID,
}
if dbch.Name.Valid {
ch.Name = dbch.Name.String
}
if err := json.Unmarshal([]byte(dbch.Metadata), &ch.Metadata); err != nil {
return bootstrap.Channel{}, err
}
return ch, nil
}
type dbConnection struct {
Config string `db:"config_id"`
Channel string `db:"channel_id"`
ConfigOwner string `db:"config_owner"`
ChannelOwner string `db:"channel_owner"`
}
+163 -86
View File
@@ -12,9 +12,9 @@ import (
"strconv"
"testing"
"github.com/gofrs/uuid"
"github.com/mainflux/mainflux/bootstrap"
"github.com/mainflux/mainflux/bootstrap/postgres"
uuid "github.com/satori/go.uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -29,8 +29,8 @@ var (
ExternalKey: "external-key",
Owner: "user@email.com",
MFChannels: []bootstrap.Channel{
bootstrap.Channel{ID: "1", Name: "name 1", Metadata: "{\"meta\":1}"},
bootstrap.Channel{ID: "2", Name: "name 2", Metadata: "{\"meta\":2}"},
bootstrap.Channel{ID: "1", Name: "name 1", Metadata: map[string]interface{}{"meta": 1.0}},
bootstrap.Channel{ID: "2", Name: "name 2", Metadata: map[string]interface{}{"meta": 2.0}},
},
Content: "content",
State: bootstrap.Inactive,
@@ -105,13 +105,18 @@ func TestRetrieveByID(t *testing.T) {
c := config
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
id, err = repo.Save(c, channels)
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
id, err := repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
nonexistentConfID, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
cases := []struct {
desc string
owner string
@@ -133,7 +138,7 @@ func TestRetrieveByID(t *testing.T) {
{
desc: "retrieve a non-existing config",
owner: c.Owner,
id: uuid.NewV4().String(),
id: nonexistentConfID.String(),
err: bootstrap.ErrNotFound,
},
{
@@ -157,11 +162,13 @@ func TestRetrieveAll(t *testing.T) {
for i := 0; i < numConfigs; i++ {
c := config
// Use UUID to prevent conflict errors.
id := uuid.NewV4().String()
c.ExternalID = id
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.ExternalID = uid.String()
c.Name = fmt.Sprintf("name %d", i)
c.MFThing = id
c.MFKey = id
c.MFThing = uid.String()
c.MFKey = uid.String()
if i%2 == 0 {
c.State = bootstrap.Active
@@ -171,7 +178,7 @@ func TestRetrieveAll(t *testing.T) {
c.MFChannels = nil
}
_, err := repo.Save(c, channels)
_, err = repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
}
@@ -235,11 +242,12 @@ func TestRetrieveByExternalID(t *testing.T) {
c := config
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
_, err = repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
@@ -281,11 +289,12 @@ func TestUpdate(t *testing.T) {
c := config
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
_, err = repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
@@ -318,6 +327,62 @@ func TestUpdate(t *testing.T) {
}
}
func TestUpdateCert(t *testing.T) {
repo := postgres.NewConfigRepository(db, testLog)
err := deleteChannels(repo)
require.Nil(t, err, "Channels cleanup expected to succeed.")
c := config
// Use UUID to prevent conflicts.
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
_, err = repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
c.Content = "new content"
c.Name = "new name"
wrongOwner := c
wrongOwner.Owner = "3"
cases := []struct {
desc string
key string
owner string
cert string
certKey string
ca string
err error
}{
{
desc: "update with wrong owner",
key: "",
cert: "cert",
certKey: "certKey",
ca: "",
owner: "wrong",
err: bootstrap.ErrNotFound,
},
{
desc: "update a config",
key: c.MFKey,
cert: "cert",
certKey: "certKey",
ca: "ca",
owner: c.Owner,
err: nil,
},
}
for _, tc := range cases {
err := repo.UpdateCert(tc.owner, tc.key, tc.cert, tc.key, tc.ca)
assert.Equal(t, tc.err, err, fmt.Sprintf("%s: expected %s got %s\n", tc.desc, tc.err, err))
}
}
func TestUpdateConnections(t *testing.T) {
repo := postgres.NewConfigRepository(db, testLog)
err := deleteChannels(repo)
@@ -325,21 +390,23 @@ func TestUpdateConnections(t *testing.T) {
c := config
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
_, err = repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
// Use UUID to prevent conflicts.
id = uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
uid, err = uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
c.MFChannels = []bootstrap.Channel{}
_, err = repo.Save(c, []string{channels[0]})
c2, err := repo.Save(c, []string{channels[0]})
require.Nil(t, err, fmt.Sprintf("Saving a config expected to succeed: %s.\n", err))
cases := []struct {
@@ -366,6 +433,14 @@ func TestUpdateConnections(t *testing.T) {
connections: []string{channels[1]},
err: nil,
},
{
desc: "update connections with existing channels",
key: config.Owner,
id: c2,
channels: nil,
connections: channels,
err: nil,
},
{
desc: "update connections no channels",
key: config.Owner,
@@ -388,12 +463,13 @@ func TestRemove(t *testing.T) {
c := config
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
id, err = repo.Save(c, channels)
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
id, err := repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
// Removal works the same for both existing and non-existing
@@ -414,11 +490,12 @@ func TestChangeState(t *testing.T) {
c := config
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
saved, err := repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
@@ -437,7 +514,7 @@ func TestChangeState(t *testing.T) {
},
{
desc: "change state with wrong id",
id: uuid.NewV4().String(),
id: "wrong",
owner: c.Owner,
err: bootstrap.ErrNotFound,
},
@@ -469,17 +546,17 @@ func TestListExisting(t *testing.T) {
c := config
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
_, err = repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
var chs []bootstrap.Channel
for _, ch := range config.MFChannels {
ch.Metadata = []byte(ch.Metadata.(string))
chs = append(chs, ch)
}
@@ -526,13 +603,13 @@ func TestSaveUnknown(t *testing.T) {
}{
{
desc: "save unknown",
externalID: uuid.NewV4().String(),
externalKey: uuid.NewV4().String(),
externalID: "unknown",
externalKey: "unknown",
err: nil,
},
{
desc: "save invalid unknown",
externalID: uuid.NewV4().String(),
externalID: "unknown",
externalKey: "",
err: nil,
},
@@ -547,8 +624,9 @@ func TestRetrieveUnknown(t *testing.T) {
repo := postgres.NewConfigRepository(db, testLog)
for i := 0; i < numConfigs; i++ {
id := uuid.NewV4().String()
repo.SaveUnknown(id, id)
id, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
repo.SaveUnknown(id.String(), id.String())
}
cases := []struct {
@@ -584,11 +662,12 @@ func TestRemoveThing(t *testing.T) {
c := config
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
saved, err := repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
for i := 0; i < 2; i++ {
@@ -604,19 +683,20 @@ func TestUpdateChannel(t *testing.T) {
c := config
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
_, err = repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
id = c.MFChannels[0].ID
id := c.MFChannels[0].ID
update := bootstrap.Channel{
ID: id,
Name: "update name",
Metadata: []byte("{\"update\":\"metadata update\"}"),
Metadata: map[string]interface{}{"update": "metadata update"},
}
err = repo.UpdateChannel(update)
assert.Nil(t, err, fmt.Sprintf("updating config expected to succeed: %s.\n", err))
@@ -640,16 +720,12 @@ func TestRemoveChannel(t *testing.T) {
require.Nil(t, err, "Channels cleanup expected to succeed.")
c := config
for i, ch := range c.MFChannels {
c.MFChannels[i].Metadata = []byte(ch.Metadata.(string))
}
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
_, err = repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
@@ -668,11 +744,12 @@ func TestDisconnectThing(t *testing.T) {
c := config
// Use UUID to prevent conflicts.
id := uuid.NewV4().String()
c.MFKey = id
c.MFThing = id
c.ExternalID = id
c.ExternalKey = id
uid, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.MFKey = uid.String()
c.MFThing = uid.String()
c.ExternalID = uid.String()
c.ExternalKey = uid.String()
saved, err := repo.Save(c, channels)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
+8 -5
View File
@@ -8,9 +8,9 @@
package postgres
import (
"database/sql"
"fmt"
"github.com/jmoiron/sqlx"
_ "github.com/lib/pq" // required for SQL access
migrate "github.com/rubenv/sql-migrate"
)
@@ -31,10 +31,10 @@ type Config struct {
// Connect creates a connection to the PostgreSQL instance and applies any
// unapplied database migrations. A non-nil error is returned to indicate
// failure.
func Connect(cfg Config) (*sql.DB, error) {
func Connect(cfg Config) (*sqlx.DB, error) {
url := fmt.Sprintf("host=%s port=%s user=%s dbname=%s password=%s sslmode=%s sslcert=%s sslkey=%s sslrootcert=%s", cfg.Host, cfg.Port, cfg.User, cfg.Name, cfg.Pass, cfg.SSLMode, cfg.SSLCert, cfg.SSLKey, cfg.SSLRootCert)
db, err := sql.Open("postgres", url)
db, err := sqlx.Open("postgres", url)
if err != nil {
return nil, err
}
@@ -46,7 +46,7 @@ func Connect(cfg Config) (*sql.DB, error) {
return db, nil
}
func migrateDB(db *sql.DB) error {
func migrateDB(db *sqlx.DB) error {
migrations := &migrate.MemoryMigrationSource{
Migrations: []*migrate.Migration{
{
@@ -60,6 +60,9 @@ func migrateDB(db *sql.DB) error {
external_id TEXT UNIQUE NOT NULL,
external_key TEXT NOT NULL,
content TEXT,
client_cert TEXT,
client_key TEXT,
ca_cert TEXT,
state BIGINT NOT NULL,
PRIMARY KEY (mainflux_thing, owner)
)`,
@@ -95,7 +98,7 @@ func migrateDB(db *sql.DB) error {
},
}
_, err := migrate.Exec(db, "postgres", migrations, migrate.Up)
_, err := migrate.Exec(db.DB, "postgres", migrations, migrate.Up)
return err
}
+4 -4
View File
@@ -8,15 +8,15 @@
package postgres_test
import (
"database/sql"
"fmt"
"log"
"os"
"testing"
"github.com/jmoiron/sqlx"
"github.com/mainflux/mainflux/bootstrap/postgres"
"github.com/mainflux/mainflux/logger"
"gopkg.in/ory-am/dockertest.v3"
dockertest "gopkg.in/ory-am/dockertest.v3"
)
const (
@@ -26,7 +26,7 @@ const (
var (
testLog, _ = logger.New(os.Stdout, logger.Info.String())
db *sql.DB
db *sqlx.DB
)
func TestMain(m *testing.M) {
@@ -49,7 +49,7 @@ func TestMain(m *testing.M) {
if err := pool.Retry(func() error {
url := fmt.Sprintf("host=localhost port=%s user=test dbname=test password=test sslmode=disable", port)
db, err = sql.Open("postgres", url)
db, err = sqlx.Open("postgres", url)
if err != nil {
return err
}
+7 -6
View File
@@ -8,7 +8,6 @@
package bootstrap
import (
"errors"
"net/http"
"github.com/mainflux/mainflux"
@@ -21,7 +20,10 @@ type bootstrapRes struct {
MFThing string `json:"mainflux_id"`
MFKey string `json:"mainflux_key"`
MFChannels []channelRes `json:"mainflux_channels"`
Content string `json:"content"`
ClientCert string `json:"client_cert,omitempty"`
ClientKey string `json:"client_key,omitempty"`
CaCert string `json:"ca_cert,omitempty"`
Content string `json:"content,omitempty"`
}
type channelRes struct {
@@ -51,10 +53,6 @@ func NewConfigReader() ConfigReader {
}
func (r reader) ReadConfig(cfg Config) (mainflux.Response, error) {
if len(cfg.MFChannels) < 1 {
return bootstrapRes{}, errors.New("Invalid configuration")
}
var channels []channelRes
for _, ch := range cfg.MFChannels {
channels = append(channels, channelRes{ID: ch.ID, Name: ch.Name, Metadata: ch.Metadata})
@@ -64,6 +62,9 @@ func (r reader) ReadConfig(cfg Config) (mainflux.Response, error) {
MFKey: cfg.MFKey,
MFThing: cfg.MFThing,
MFChannels: channels,
ClientCert: cfg.ClientCert,
ClientKey: cfg.ClientKey,
CaCert: cfg.CACert,
Content: cfg.Content,
}
+10
View File
@@ -0,0 +1,10 @@
//
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
//
// Package consumer contains events consumer for events
// published by Things service.
package consumer
@@ -1,4 +1,11 @@
package redis
//
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
//
package consumer
type removeEvent struct {
id string
@@ -7,7 +14,7 @@ type removeEvent struct {
type updateChannelEvent struct {
id string
name string
metadata string
metadata map[string]interface{}
}
// Connection event is either connect or disconnect event.
@@ -1,6 +1,14 @@
package redis
//
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
//
package consumer
import (
"encoding/json"
"fmt"
"github.com/go-redis/redis"
@@ -97,10 +105,16 @@ func decodeRemoveThing(event map[string]interface{}) removeEvent {
}
func decodeUpdateChannel(event map[string]interface{}) updateChannelEvent {
strmeta := read(event, "metadata", "{}")
var metadata map[string]interface{}
if err := json.Unmarshal([]byte(strmeta), metadata); err != nil {
metadata = map[string]interface{}{}
}
return updateChannelEvent{
id: read(event, "id", ""),
name: read(event, "name", ""),
metadata: read(event, "metadata", ""),
metadata: metadata,
}
}
+10
View File
@@ -0,0 +1,10 @@
//
// Copyright (c) 2018
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
//
// Package producer contains the domain events needed to support
// event sourcing of Bootstrap service actions.
package producer
+138
View File
@@ -0,0 +1,138 @@
//
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
//
package producer
import (
"strings"
"time"
"github.com/mainflux/mainflux/bootstrap"
)
const (
configPrefix = "config."
configCreate = configPrefix + "create"
configUpdate = configPrefix + "update"
configRemove = configPrefix + "remove"
thingPrefix = "thing."
thingBootstrap = thingPrefix + "bootstrap"
thingStateChange = thingPrefix + "state_change"
thingUpdateConnections = thingPrefix + "update_connections"
)
type event interface {
encode() map[string]interface{}
}
var (
_ event = (*createConfigEvent)(nil)
_ event = (*updateConfigEvent)(nil)
_ event = (*removeConfigEvent)(nil)
_ event = (*bootstrapEvent)(nil)
_ event = (*changeStateEvent)(nil)
_ event = (*updateConnectionsEvent)(nil)
)
type createConfigEvent struct {
mfThing string
owner string
name string
mfChannels []string
externalID string
content string
timestamp time.Time
}
func (cce createConfigEvent) encode() map[string]interface{} {
return map[string]interface{}{
"thing_id": cce.mfThing,
"owner": cce.owner,
"name": cce.name,
"channels": strings.Join(cce.mfChannels, ", "),
"external_id": cce.externalID,
"content": cce.content,
"timestamp": cce.timestamp.Unix(),
"operation": configCreate,
}
}
type updateConfigEvent struct {
mfThing string
name string
content string
timestamp time.Time
}
func (uce updateConfigEvent) encode() map[string]interface{} {
return map[string]interface{}{
"thing_id": uce.mfThing,
"name": uce.name,
"content": uce.content,
"timestamp": uce.timestamp.Unix(),
"operation": configUpdate,
}
}
type removeConfigEvent struct {
mfThing string
timestamp time.Time
}
func (rce removeConfigEvent) encode() map[string]interface{} {
return map[string]interface{}{
"thing_id": rce.mfThing,
"timestamp": rce.timestamp.Unix(),
"operation": configRemove,
}
}
type bootstrapEvent struct {
externalID string
success bool
timestamp time.Time
}
func (be bootstrapEvent) encode() map[string]interface{} {
return map[string]interface{}{
"external_id": be.externalID,
"success": be.success,
"timestamp": be.timestamp.Unix(),
"operation": thingBootstrap,
}
}
type changeStateEvent struct {
mfThing string
state bootstrap.State
timestamp time.Time
}
func (cse changeStateEvent) encode() map[string]interface{} {
return map[string]interface{}{
"thing_id": cse.mfThing,
"state": cse.state.String(),
"timestamp": cse.timestamp.Unix(),
"operation": thingStateChange,
}
}
type updateConnectionsEvent struct {
mfThing string
mfChannels []string
timestamp time.Time
}
func (uce updateConnectionsEvent) encode() map[string]interface{} {
return map[string]interface{}{
"thing_id": uce.mfThing,
"channels": strings.Join(uce.mfChannels, ", "),
"timestamp": uce.timestamp.Unix(),
"operation": thingUpdateConnections,
}
}
+57
View File
@@ -0,0 +1,57 @@
//
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
//
package producer_test
import (
"fmt"
"log"
"os"
"testing"
"github.com/go-redis/redis"
dockertest "gopkg.in/ory-am/dockertest.v3"
)
const (
wrongID = 0
wrongValue = "wrong-value"
)
var redisClient *redis.Client
func TestMain(m *testing.M) {
pool, err := dockertest.NewPool("")
if err != nil {
log.Fatalf("Could not connect to docker: %s", err)
}
container, err := pool.Run("redis", "5.0-alpine", nil)
if err != nil {
log.Fatalf("Could not start container: %s", err)
}
if err := pool.Retry(func() error {
redisClient = redis.NewClient(&redis.Options{
Addr: fmt.Sprintf("localhost:%s", container.GetPort("6379/tcp")),
Password: "",
DB: 0,
})
return redisClient.Ping().Err()
}); err != nil {
log.Fatalf("Could not connect to docker: %s", err)
}
code := m.Run()
if err := pool.Purge(container); err != nil {
log.Fatalf("Could not purge container: %s", err)
}
os.Exit(code)
}
+182
View File
@@ -0,0 +1,182 @@
//
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
//
package producer
import (
"time"
"github.com/go-redis/redis"
"github.com/mainflux/mainflux/bootstrap"
)
const (
streamID = "mainflux.bootstrap"
streamLen = 1000
)
var _ bootstrap.Service = (*eventStore)(nil)
type eventStore struct {
svc bootstrap.Service
client *redis.Client
}
// NewEventStoreMiddleware returns wrapper around bootstrap service that sends
// events to event store.
func NewEventStoreMiddleware(svc bootstrap.Service, client *redis.Client) bootstrap.Service {
return eventStore{
svc: svc,
client: client,
}
}
func (es eventStore) Add(key string, cfg bootstrap.Config) (bootstrap.Config, error) {
saved, err := es.svc.Add(key, cfg)
if err != nil {
return saved, err
}
var channels []string
for _, ch := range saved.MFChannels {
channels = append(channels, ch.ID)
}
ev := createConfigEvent{
mfThing: saved.MFThing,
owner: saved.Owner,
name: saved.Name,
mfChannels: channels,
externalID: saved.ExternalID,
content: saved.Content,
timestamp: time.Now(),
}
es.add(ev)
return saved, err
}
func (es eventStore) View(key, id string) (bootstrap.Config, error) {
return es.svc.View(key, id)
}
func (es eventStore) Update(key string, cfg bootstrap.Config) error {
if err := es.svc.Update(key, cfg); err != nil {
return err
}
ev := updateConfigEvent{
mfThing: cfg.MFThing,
name: cfg.Name,
content: cfg.Content,
timestamp: time.Now(),
}
es.add(ev)
return nil
}
func (es eventStore) UpdateCert(key, thingKey, clientCert, clientKey, caCert string) error {
return es.svc.UpdateCert(key, thingKey, clientCert, clientKey, caCert)
}
func (es eventStore) UpdateConnections(key, id string, connections []string) error {
if err := es.svc.UpdateConnections(key, id, connections); err != nil {
return err
}
ev := updateConnectionsEvent{
mfThing: id,
mfChannels: connections,
timestamp: time.Now(),
}
es.add(ev)
return nil
}
func (es eventStore) List(key string, filter bootstrap.Filter, offset, limit uint64) (bootstrap.ConfigsPage, error) {
return es.svc.List(key, filter, offset, limit)
}
func (es eventStore) Remove(key, id string) error {
if err := es.svc.Remove(key, id); err != nil {
return err
}
ev := removeConfigEvent{
mfThing: id,
timestamp: time.Now(),
}
es.add(ev)
return nil
}
func (es eventStore) Bootstrap(externalKey, externalID string) (bootstrap.Config, error) {
cfg, err := es.svc.Bootstrap(externalKey, externalID)
ev := bootstrapEvent{
externalID: externalID,
timestamp: time.Now(),
success: true,
}
if err != nil {
ev.success = false
}
es.add(ev)
return cfg, err
}
func (es eventStore) ChangeState(key, id string, state bootstrap.State) error {
if err := es.svc.ChangeState(key, id, state); err != nil {
return err
}
ev := changeStateEvent{
mfThing: id,
state: state,
timestamp: time.Now(),
}
es.add(ev)
return nil
}
func (es eventStore) RemoveConfigHandler(id string) error {
return es.svc.RemoveConfigHandler(id)
}
func (es eventStore) RemoveChannelHandler(id string) error {
return es.svc.RemoveChannelHandler(id)
}
func (es eventStore) UpdateChannelHandler(channel bootstrap.Channel) error {
return es.UpdateChannelHandler(channel)
}
func (es eventStore) DisconnectThingHandler(channelID, thingID string) error {
return es.svc.DisconnectThingHandler(channelID, thingID)
}
func (es eventStore) add(ev event) error {
record := &redis.XAddArgs{
Stream: streamID,
MaxLenApprox: streamLen,
Values: ev.encode(),
}
return es.client.XAdd(record).Err()
}
+550
View File
@@ -0,0 +1,550 @@
//
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
//
package producer_test
import (
"fmt"
"net/http/httptest"
"strconv"
"strings"
"testing"
"time"
"github.com/go-redis/redis"
"github.com/mainflux/mainflux"
"github.com/opentracing/opentracing-go/mocktracer"
"github.com/mainflux/mainflux/bootstrap"
"github.com/mainflux/mainflux/bootstrap/mocks"
"github.com/mainflux/mainflux/bootstrap/redis/producer"
mfsdk "github.com/mainflux/mainflux/sdk/go"
"github.com/mainflux/mainflux/things"
httpapi "github.com/mainflux/mainflux/things/api/things/http"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const (
streamID = "mainflux.bootstrap"
email = "user@example.com"
validToken = "validToken"
unknownID = "1"
unknownKey = "2"
channelsNum = 3
defaultTimout = 5
configPrefix = "config."
configCreate = configPrefix + "create"
configUpdate = configPrefix + "update"
configRemove = configPrefix + "remove"
thingPrefix = "thing."
thingStateChange = thingPrefix + "state_change"
thingBootstrap = thingPrefix + "bootstrap"
thingUpdateConnections = thingPrefix + "update_connections"
)
var (
channel = bootstrap.Channel{
ID: "1",
Name: "name",
Metadata: map[string]interface{}{"name": "value"},
}
config = bootstrap.Config{
ExternalID: "external_id",
ExternalKey: "external_key",
MFChannels: []bootstrap.Channel{channel},
Content: "config",
}
)
func newService(users mainflux.UsersServiceClient, url string) bootstrap.Service {
configs := mocks.NewConfigsRepository(map[string]string{unknownID: unknownKey})
config := mfsdk.Config{
BaseURL: url,
}
sdk := mfsdk.NewSDK(config)
return bootstrap.New(users, configs, sdk)
}
func newThingsService(users mainflux.UsersServiceClient) things.Service {
channels := make(map[string]things.Channel, channelsNum)
for i := 0; i < channelsNum; i++ {
id := strconv.Itoa(i + 1)
channels[id] = things.Channel{
ID: id,
Owner: email,
Metadata: map[string]interface{}{"meta": "data"},
}
}
return mocks.NewThingsService(map[string]things.Thing{}, channels, users)
}
func newThingsServer(svc things.Service) *httptest.Server {
mux := httpapi.MakeHandler(mocktracer.New(), svc)
return httptest.NewServer(mux)
}
func TestAdd(t *testing.T) {
redisClient.FlushAll().Err()
users := mocks.NewUsersService(map[string]string{validToken: email})
server := newThingsServer(newThingsService(users))
svc := newService(users, server.URL)
svc = producer.NewEventStoreMiddleware(svc, redisClient)
var channels []string
for _, ch := range config.MFChannels {
channels = append(channels, ch.ID)
}
invalidConfig := config
invalidConfig.MFChannels = []bootstrap.Channel{bootstrap.Channel{ID: "empty"}}
cases := []struct {
desc string
config bootstrap.Config
key string
err error
event map[string]interface{}
}{
{
desc: "create config successfully",
config: config,
key: validToken,
err: nil,
event: map[string]interface{}{
"thing_id": "1",
"owner": email,
"name": config.Name,
"channels": strings.Join(channels, ", "),
"external_id": config.ExternalID,
"content": config.Content,
"timestamp": time.Now().Unix(),
"operation": configCreate,
},
},
{
desc: "create invalid config",
config: invalidConfig,
key: validToken,
err: bootstrap.ErrMalformedEntity,
event: nil,
},
}
lastID := "0"
for _, tc := range cases {
_, err := svc.Add(tc.key, tc.config)
assert.Equal(t, tc.err, err, fmt.Sprintf("%s: expected %s got %s\n", tc.desc, tc.err, err))
streams := redisClient.XRead(&redis.XReadArgs{
Streams: []string{streamID, lastID},
Count: 1,
Block: time.Second,
}).Val()
var event map[string]interface{}
if len(streams) > 0 && len(streams[0].Messages) > 0 {
msg := streams[0].Messages[0]
event = msg.Values
lastID = msg.ID
}
test(t, tc.event, event, tc.desc)
}
}
func TestView(t *testing.T) {
users := mocks.NewUsersService(map[string]string{validToken: email})
server := newThingsServer(newThingsService(users))
svc := newService(users, server.URL)
saved, err := svc.Add(validToken, config)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
svcConfig, svcErr := svc.View(validToken, saved.MFThing)
svc = producer.NewEventStoreMiddleware(svc, redisClient)
esConfig, esErr := svc.View(validToken, saved.MFThing)
assert.Equal(t, svcConfig, esConfig, fmt.Sprintf("event sourcing changed service behavior: expected %v got %v", svcConfig, esConfig))
assert.Equal(t, svcErr, esErr, fmt.Sprintf("event sourcing changed service behavior: expected %v got %v", svcErr, esErr))
}
func TestUpdate(t *testing.T) {
redisClient.FlushAll().Err()
users := mocks.NewUsersService(map[string]string{validToken: email})
server := newThingsServer(newThingsService(users))
svc := newService(users, server.URL)
svc = producer.NewEventStoreMiddleware(svc, redisClient)
c := config
ch := channel
ch.ID = "2"
c.MFChannels = append(c.MFChannels, ch)
saved, err := svc.Add(validToken, c)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
redisClient.FlushAll().Err()
modified := saved
modified.Content = "new-config"
modified.Name = "new name"
nonExisting := config
nonExisting.MFThing = "unknown"
cases := []struct {
desc string
config bootstrap.Config
key string
err error
event map[string]interface{}
}{
{
desc: "update config successfully",
config: modified,
key: validToken,
err: nil,
event: map[string]interface{}{
"thing_id": modified.MFThing,
"name": modified.Name,
"content": modified.Content,
"timestamp": time.Now().Unix(),
"operation": configUpdate,
},
},
{
desc: "update non-existing config",
config: nonExisting,
key: validToken,
err: bootstrap.ErrNotFound,
event: nil,
},
}
lastID := "0"
for _, tc := range cases {
err := svc.Update(tc.key, tc.config)
assert.Equal(t, tc.err, err, fmt.Sprintf("%s: expected %s got %s\n", tc.desc, tc.err, err))
streams := redisClient.XRead(&redis.XReadArgs{
Streams: []string{streamID, lastID},
Count: 1,
Block: time.Second,
}).Val()
var event map[string]interface{}
if len(streams) > 0 && len(streams[0].Messages) > 0 {
msg := streams[0].Messages[0]
event = msg.Values
lastID = msg.ID
}
test(t, tc.event, event, tc.desc)
}
}
func TestUpdateConnections(t *testing.T) {
redisClient.FlushAll().Err()
users := mocks.NewUsersService(map[string]string{validToken: email})
server := newThingsServer(newThingsService(users))
svc := newService(users, server.URL)
svc = producer.NewEventStoreMiddleware(svc, redisClient)
saved, err := svc.Add(validToken, config)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
redisClient.FlushAll().Err()
cases := []struct {
desc string
id string
key string
connections []string
err error
event map[string]interface{}
}{
{
desc: "update connections successfully",
id: saved.MFThing,
key: validToken,
connections: []string{"2"},
err: nil,
event: map[string]interface{}{
"thing_id": saved.MFThing,
"channels": "2",
"timestamp": time.Now().Unix(),
"operation": thingUpdateConnections,
},
},
{
desc: "update connections unsuccessfully",
id: saved.MFThing,
key: validToken,
connections: []string{"256"},
err: bootstrap.ErrMalformedEntity,
event: nil,
},
}
lastID := "0"
for _, tc := range cases {
err := svc.UpdateConnections(tc.key, tc.id, tc.connections)
assert.Equal(t, tc.err, err, fmt.Sprintf("%s: expected %s got %s\n", tc.desc, tc.err, err))
streams := redisClient.XRead(&redis.XReadArgs{
Streams: []string{streamID, lastID},
Count: 1,
Block: time.Second,
}).Val()
var event map[string]interface{}
if len(streams) > 0 && len(streams[0].Messages) > 0 {
msg := streams[0].Messages[0]
event = msg.Values
lastID = msg.ID
}
test(t, tc.event, event, tc.desc)
}
}
func TestList(t *testing.T) {
users := mocks.NewUsersService(map[string]string{validToken: email})
server := newThingsServer(newThingsService(users))
svc := newService(users, server.URL)
_, err := svc.Add(validToken, config)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
offset := uint64(0)
limit := uint64(10)
svcConfigs, svcErr := svc.List(validToken, bootstrap.Filter{}, offset, limit)
svc = producer.NewEventStoreMiddleware(svc, redisClient)
esConfigs, esErr := svc.List(validToken, bootstrap.Filter{}, offset, limit)
assert.Equal(t, svcConfigs, esConfigs, fmt.Sprintf("event sourcing changed service behavior: expected %v got %v", svcConfigs, esConfigs))
assert.Equal(t, svcErr, esErr, fmt.Sprintf("event sourcing changed service behavior: expected %v got %v", svcErr, esErr))
}
func TestRemove(t *testing.T) {
redisClient.FlushAll().Err()
users := mocks.NewUsersService(map[string]string{validToken: email})
server := newThingsServer(newThingsService(users))
svc := newService(users, server.URL)
svc = producer.NewEventStoreMiddleware(svc, redisClient)
c := config
saved, err := svc.Add(validToken, c)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
redisClient.FlushAll().Err()
cases := []struct {
desc string
id string
key string
err error
event map[string]interface{}
}{
{
desc: "remove config successfully",
id: saved.MFThing,
key: validToken,
err: nil,
event: map[string]interface{}{
"thing_id": saved.MFThing,
"timestamp": time.Now().Unix(),
"operation": configRemove,
},
},
{
desc: "remove config with invalid credentials",
id: saved.MFThing,
key: "",
err: bootstrap.ErrUnauthorizedAccess,
event: nil,
},
}
lastID := "0"
for _, tc := range cases {
err := svc.Remove(tc.key, tc.id)
assert.Equal(t, tc.err, err, fmt.Sprintf("%s: expected %s got %s\n", tc.desc, tc.err, err))
streams := redisClient.XRead(&redis.XReadArgs{
Streams: []string{streamID, lastID},
Count: 1,
Block: time.Second,
}).Val()
var event map[string]interface{}
if len(streams) > 0 && len(streams[0].Messages) > 0 {
msg := streams[0].Messages[0]
event = msg.Values
lastID = msg.ID
}
test(t, tc.event, event, tc.desc)
}
}
func TestBootstrap(t *testing.T) {
redisClient.FlushAll().Err()
users := mocks.NewUsersService(map[string]string{validToken: email})
server := newThingsServer(newThingsService(users))
svc := newService(users, server.URL)
svc = producer.NewEventStoreMiddleware(svc, redisClient)
c := config
saved, err := svc.Add(validToken, c)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
redisClient.FlushAll().Err()
cases := []struct {
desc string
externalID string
externalKey string
err error
event map[string]interface{}
}{
{
desc: "bootstrap successfully",
externalID: saved.ExternalID,
externalKey: saved.ExternalKey,
err: nil,
event: map[string]interface{}{
"external_id": saved.ExternalID,
"success": "1",
"timestamp": time.Now().Unix(),
"operation": thingBootstrap,
},
},
{
desc: "bootstrap with an error",
externalID: saved.ExternalID,
externalKey: "external",
err: bootstrap.ErrNotFound,
event: map[string]interface{}{
"external_id": saved.ExternalID,
"success": "0",
"timestamp": time.Now().Unix(),
"operation": thingBootstrap,
},
},
}
lastID := "0"
for _, tc := range cases {
_, err := svc.Bootstrap(tc.externalKey, tc.externalID)
assert.Equal(t, tc.err, err, fmt.Sprintf("%s: expected %s got %s\n", tc.desc, tc.err, err))
streams := redisClient.XRead(&redis.XReadArgs{
Streams: []string{streamID, lastID},
Count: 1,
Block: time.Second,
}).Val()
var event map[string]interface{}
if len(streams) > 0 && len(streams[0].Messages) > 0 {
msg := streams[0].Messages[0]
event = msg.Values
lastID = msg.ID
}
test(t, tc.event, event, tc.desc)
}
}
func TestChangeState(t *testing.T) {
redisClient.FlushAll().Err()
users := mocks.NewUsersService(map[string]string{validToken: email})
server := newThingsServer(newThingsService(users))
svc := newService(users, server.URL)
svc = producer.NewEventStoreMiddleware(svc, redisClient)
c := config
saved, err := svc.Add(validToken, c)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
redisClient.FlushAll().Err()
cases := []struct {
desc string
id string
key string
state bootstrap.State
err error
event map[string]interface{}
}{
{
desc: "change state to active",
id: saved.MFThing,
key: validToken,
state: bootstrap.Active,
err: nil,
event: map[string]interface{}{
"thing_id": saved.MFThing,
"state": bootstrap.Active.String(),
"timestamp": time.Now().Unix(),
"operation": thingStateChange,
},
},
{
desc: "change state invalid credentials",
id: saved.MFThing,
key: "",
state: bootstrap.Inactive,
err: bootstrap.ErrUnauthorizedAccess,
event: nil,
},
}
lastID := "0"
for _, tc := range cases {
err := svc.ChangeState(tc.key, tc.id, tc.state)
assert.Equal(t, tc.err, err, fmt.Sprintf("%s: expected %s got %s\n", tc.desc, tc.err, err))
streams := redisClient.XRead(&redis.XReadArgs{
Streams: []string{streamID, lastID},
Count: 1,
Block: time.Second,
}).Val()
var event map[string]interface{}
if len(streams) > 0 && len(streams[0].Messages) > 0 {
msg := streams[0].Messages[0]
event = msg.Values
lastID = msg.ID
}
test(t, tc.event, event, tc.desc)
}
}
func test(t *testing.T, expected, actual map[string]interface{}, description string) {
if expected != nil && actual != nil {
ts1 := expected["timestamp"].(int64)
ts2, err := strconv.ParseInt(actual["timestamp"].(string), 10, 64)
require.Nil(t, err, fmt.Sprintf("%s: expected to get a valid timestamp, got %s", description, err))
val := ts1 == ts2 || ts2 <= ts1+defaultTimout
assert.True(t, val, fmt.Sprintf("%s: timestamp is not in valid range", description))
delete(expected, "timestamp")
delete(actual, "timestamp")
assert.Equal(t, expected, actual, fmt.Sprintf("%s: expected %v got %v\n", description, expected, actual))
}
}
+58 -84
View File
@@ -9,7 +9,6 @@ package bootstrap
import (
"context"
"encoding/json"
"errors"
"time"
@@ -17,11 +16,6 @@ import (
mfsdk "github.com/mainflux/mainflux/sdk/go"
)
const (
thingType = "device"
chanName = "channel"
)
var (
// ErrNotFound indicates a non-existent entity request.
ErrNotFound = errors.New("non-existent entity")
@@ -55,6 +49,10 @@ type Service interface {
// Update updates editable fields of the provided Config.
Update(string, Config) error
// UpdateCert updates an existing Config certificate and key.
// A non-nil error is returned to indicate operation failure.
UpdateCert(string, string, string, string, string) error
// UpdateConnections updates list of Channels related to given Config.
UpdateConnections(string, string, []string) error
@@ -118,7 +116,7 @@ func (bs bootstrapService) Add(key string, cfg Config) (Config, error) {
toConnect := bs.toIDList(cfg.MFChannels)
// Check if channels exist. This is the way to prevent invalid configuration to be saved.
// Check if channels exist. This is the way to prevent fetching channels that already exist.
existing, err := bs.configs.ListExisting(owner, toConnect)
if err != nil {
return Config{}, err
@@ -162,20 +160,7 @@ func (bs bootstrapService) View(key, id string) (Config, error) {
return Config{}, err
}
cfg, err := bs.configs.RetrieveByID(owner, id)
if err != nil {
return Config{}, err
}
for i, ch := range cfg.MFChannels {
if meta, ok := ch.Metadata.([]byte); ok {
if err := json.Unmarshal(meta, &cfg.MFChannels[i].Metadata); err != nil {
return Config{}, err
}
}
}
return cfg, nil
return bs.configs.RetrieveByID(owner, id)
}
func (bs bootstrapService) Update(key string, cfg Config) error {
@@ -189,6 +174,14 @@ func (bs bootstrapService) Update(key string, cfg Config) error {
return bs.configs.Update(cfg)
}
func (bs bootstrapService) UpdateCert(key, thingKey, clientCert, clientKey, caCert string) error {
owner, err := bs.identify(key)
if err != nil {
return err
}
return bs.configs.UpdateCert(owner, thingKey, clientCert, clientKey, caCert)
}
func (bs bootstrapService) UpdateConnections(key, id string, connections []string) error {
owner, err := bs.identify(key)
if err != nil {
@@ -201,7 +194,14 @@ func (bs bootstrapService) UpdateConnections(key, id string, connections []strin
}
add, remove := bs.updateList(cfg, connections)
channels, err := bs.updateChannels(add, key)
// Check if channels exist. This is the way to prevent fetching channels that already exist.
existing, err := bs.configs.ListExisting(owner, connections)
if err != nil {
return err
}
channels, err := bs.connectionChannels(connections, bs.toIDList(existing), key)
if err != nil {
return err
}
@@ -266,14 +266,6 @@ func (bs bootstrapService) Bootstrap(externalKey, externalID string) (Config, er
return Config{}, ErrNotFound
}
for i, ch := range cfg.MFChannels {
if meta, ok := ch.Metadata.([]byte); ok {
if err := json.Unmarshal(meta, &cfg.MFChannels[i].Metadata); err != nil {
return Config{}, err
}
}
}
return cfg, nil
}
@@ -283,25 +275,25 @@ func (bs bootstrapService) ChangeState(key, id string, state State) error {
return err
}
thing, err := bs.configs.RetrieveByID(owner, id)
cfg, err := bs.configs.RetrieveByID(owner, id)
if err != nil {
return err
}
if thing.State == state {
if cfg.State == state {
return nil
}
switch state {
case Active:
for _, c := range thing.MFChannels {
if err := bs.sdk.ConnectThing(thing.MFThing, c.ID, key); err != nil {
for _, c := range cfg.MFChannels {
if err := bs.sdk.ConnectThing(cfg.MFThing, c.ID, key); err != nil {
return ErrThings
}
}
case Inactive:
for _, c := range thing.MFChannels {
if err := bs.sdk.DisconnectThing(thing.MFThing, c.ID, key); err != nil {
for _, c := range cfg.MFChannels {
if err := bs.sdk.DisconnectThing(cfg.MFThing, c.ID, key); err != nil {
if err == mfsdk.ErrNotFound {
continue
}
@@ -347,7 +339,7 @@ func (bs bootstrapService) thing(key, id string) (mfsdk.Thing, error) {
var err error
if id == "" {
thingID, err = bs.sdk.CreateThing(mfsdk.Thing{Type: thingType}, key)
thingID, err = bs.sdk.CreateThing(mfsdk.Thing{}, key)
if err != nil {
return mfsdk.Thing{}, err
}
@@ -369,6 +361,35 @@ func (bs bootstrapService) thing(key, id string) (mfsdk.Thing, error) {
return thing, nil
}
func (bs bootstrapService) connectionChannels(channels, existing []string, key string) ([]Channel, error) {
add := make(map[string]bool, len(channels))
for _, ch := range channels {
add[ch] = true
}
for _, ch := range existing {
if add[ch] == true {
delete(add, ch)
}
}
var ret []Channel
for id := range add {
ch, err := bs.sdk.Channel(id, key)
if err != nil {
return nil, ErrMalformedEntity
}
ret = append(ret, Channel{
ID: ch.ID,
Name: ch.Name,
Metadata: ch.Metadata,
})
}
return ret, nil
}
// Method updateList accepts config and channel IDs and returns three lists:
// 1) IDs of Channels to be added
// 2) IDs of Channels to be removed
@@ -405,50 +426,3 @@ func (bs bootstrapService) toIDList(channels []Channel) []string {
return ret
}
func (bs bootstrapService) updateChannels(add []string, key string) ([]Channel, error) {
var ret []Channel
for _, id := range add {
ch, err := bs.sdk.Channel(id, key)
if err != nil {
return []Channel{}, ErrMalformedEntity
}
ret = append(ret, Channel{
ID: ch.ID,
Name: ch.Name,
Metadata: ch.Metadata,
})
}
return ret, nil
}
func (bs bootstrapService) connectionChannels(channels, existing []string, key string) ([]Channel, error) {
add := make(map[string]bool, len(channels))
for _, ch := range channels {
add[ch] = true
}
for _, ch := range existing {
if add[ch] == true {
delete(add, ch)
}
}
var ret []Channel
for id := range add {
ch, err := bs.sdk.Channel(id, key)
if err != nil {
return nil, ErrMalformedEntity
}
ret = append(ret, Channel{
ID: ch.ID,
Name: ch.Name,
Metadata: ch.Metadata,
})
}
return ret, nil
}
+74 -11
View File
@@ -13,13 +13,15 @@ import (
"strconv"
"testing"
"github.com/opentracing/opentracing-go/mocktracer"
"github.com/gofrs/uuid"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/bootstrap"
"github.com/mainflux/mainflux/bootstrap/mocks"
mfsdk "github.com/mainflux/mainflux/sdk/go"
"github.com/mainflux/mainflux/things"
httpapi "github.com/mainflux/mainflux/things/api/http"
uuid "github.com/satori/go.uuid"
httpapi "github.com/mainflux/mainflux/things/api/things/http"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -38,7 +40,7 @@ var (
channel = bootstrap.Channel{
ID: "1",
Name: "name",
Metadata: `{"name":"value"}`,
Metadata: map[string]interface{}{"name": "value"},
}
config = bootstrap.Config{
@@ -66,7 +68,7 @@ func newThingsService(users mainflux.UsersServiceClient) things.Service {
channels[id] = things.Channel{
ID: id,
Owner: email,
Metadata: `{"meta":"data"}`,
Metadata: map[string]interface{}{"meta": "data"},
}
}
@@ -74,7 +76,7 @@ func newThingsService(users mainflux.UsersServiceClient) things.Service {
}
func newThingsServer(svc things.Service) *httptest.Server {
mux := httpapi.MakeHandler(svc)
mux := httpapi.MakeHandler(mocktracer.New(), svc)
return httptest.NewServer(mux)
}
@@ -223,6 +225,64 @@ func TestUpdate(t *testing.T) {
}
}
func TestUpdateCert(t *testing.T) {
users := mocks.NewUsersService(map[string]string{validToken: email})
server := newThingsServer(newThingsService(users))
svc := newService(users, server.URL)
c := config
ch := channel
ch.ID = "2"
c.MFChannels = append(c.MFChannels, ch)
saved, err := svc.Add(validToken, c)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
cases := []struct {
desc string
key string
thingKey string
clientCert string
clientKey string
caCert string
err error
}{
{
desc: "update certs for the valid config",
thingKey: saved.MFKey,
clientCert: "newCert",
clientKey: "newKey",
caCert: "newCert",
key: validToken,
err: nil,
},
{
desc: "update cert for a non-existing config",
thingKey: "empty",
clientCert: "newCert",
clientKey: "newKey",
caCert: "newCert",
key: validToken,
err: bootstrap.ErrNotFound,
},
{
desc: "update config cert with wrong credentials",
thingKey: saved.MFKey,
clientCert: "newCert",
clientKey: "newKey",
caCert: "newCert",
key: invalidToken,
err: bootstrap.ErrUnauthorizedAccess,
},
}
for _, tc := range cases {
err := svc.UpdateCert(tc.key, tc.thingKey, tc.clientCert, tc.clientKey, tc.caCert)
assert.Equal(t, tc.err, err, fmt.Sprintf("%s: expected %s got %s\n", tc.desc, tc.err, err))
}
}
func TestUpdateConnections(t *testing.T) {
users := mocks.NewUsersService(map[string]string{validToken: email})
@@ -236,7 +296,9 @@ func TestUpdateConnections(t *testing.T) {
created, err := svc.Add(validToken, c)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
c.ExternalID = uuid.NewV4().String()
externalID, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.ExternalID = externalID.String()
active, err := svc.Add(validToken, c)
require.Nil(t, err, fmt.Sprintf("Saving config expected to succeed: %s.\n", err))
err = svc.ChangeState(validToken, active.MFThing, bootstrap.Active)
@@ -253,7 +315,7 @@ func TestUpdateConnections(t *testing.T) {
err error
}{
{
desc: "update connections for config with state Created",
desc: "update connections for config with state Inactive",
key: validToken,
id: created.MFThing,
connections: []string{"2"},
@@ -305,9 +367,10 @@ func TestList(t *testing.T) {
var saved []bootstrap.Config
for i := 0; i < numThings; i++ {
c := config
id := uuid.NewV4().String()
c.ExternalID = id
c.ExternalKey = id
id, err := uuid.NewV4()
require.Nil(t, err, fmt.Sprintf("Got unexpected error: %s.\n", err))
c.ExternalID = id.String()
c.ExternalKey = id.String()
c.Name = fmt.Sprintf("%s-%d", config.Name, i)
s, err := svc.Add(validToken, c)
saved = append(saved, s)
@@ -584,7 +647,7 @@ func TestUpdateChannelHandler(t *testing.T) {
ch := bootstrap.Channel{
ID: channel.ID,
Name: "new name",
Metadata: "new meta",
Metadata: map[string]interface{}{"meta": "new"},
}
cases := []struct {
+65 -20
View File
@@ -1,7 +1,7 @@
swagger: "2.0"
info:
title: Mainflux Bootstrap service
description: HTTP API for managing platform devices and applications configuration.
description: HTTP API for managing platform things configuration.
version: "1.0.0"
consumes:
- "application/json"
@@ -56,7 +56,7 @@ paths:
- $ref: "#/parameters/Name"
responses:
200:
description: |
description: |
Data retrieved. Configs from this list don't contain channels.
schema:
$ref: "#/definitions/ConfigList"
@@ -156,11 +156,41 @@ paths:
description: Missing or invalid access token provided.
500:
$ref: "#/responses/ServiceError"
/things/configs/certs/{thingKey}:
put:
summary: Updates certs
description: |
Update is performed by replacing the current certificate data with values
provided in a request payload.
tags:
- configs
parameters:
- $ref: "#/parameters/Authorization"
- $ref: "#/parameters/ThingKey"
- name: config
description: JSON-formatted document describing the updated thing.
in: body
schema:
$ref: "#/definitions/ConfigUpdateCertReq"
required: true
responses:
200:
description: Config updated.
400:
description: Failed due to malformed JSON.
403:
description: Missing or invalid access token provided.
404:
description: Config does not exist.
415:
description: Missing or invalid content type.
500:
$ref: "#/responses/ServiceError"
/things/configs/connections/{configId}:
put:
summary: Updates channels the thing is connected to
description: |
Update connections performs update of the channel list corresponding
Update connections performs update of the channel list corresponding
Thing is connected to.
tags:
- configs
@@ -171,9 +201,7 @@ paths:
description: Array if IDs the thing is be connected to.
in: body
schema:
type: array
items:
type: string
$ref: "#/definitions/ConfigUpdateConnReq"
required: true
responses:
200:
@@ -260,10 +288,15 @@ parameters:
required: true
ConfigId:
name: configId
description: Unique Config identifier.
description: Unique Config identifier. It's the ID of the corresponding Thing.
in: path
type: integer
minimum: 1
type: string
required: true
ThingKey:
name: thingKey
description: Unique Thing key.
in: path
type: string
required: true
ExternalId:
name: externalId
@@ -342,7 +375,7 @@ definitions:
ConfigRes:
type: object
properties:
mainflux_thing:
mainflux_id:
type: string
description: Corresponding Mainflux Thing ID.
mainflux_key:
@@ -371,7 +404,7 @@ definitions:
description: External key.
content:
type: string
description: Free-form custom configuration
description: Free-form custom configuration.
state:
$ref: '#/definitions/State'
required:
@@ -380,7 +413,7 @@ definitions:
BootstrapRes:
type: object
properties:
mainflux_thing:
mainflux_id:
type: string
description: Corresponding Mainflux Thing ID.
mainflux_key:
@@ -393,9 +426,9 @@ definitions:
type: string
content:
type: string
description: Free-form custom configuration
description: Free-form custom configuration.
required:
- mainflux_thing
- mainflux_id
- mainflux_key
- mainflux_channels
- content
@@ -424,15 +457,27 @@ definitions:
ConfigUpdateReq:
type: object
properties:
channels:
type: array
minItems: 0
items:
type: string
content:
type: string
name:
type: string
required:
- content
- name
- name
ConfigUpdateConnReq:
type: object
properties:
channels:
type: array
minItems: 0
items:
type: string
ConfigUpdateCertReq:
type: object
properties:
client_cert:
type: string
client_key:
type: string
ca_cert:
type: string
+3 -3
View File
@@ -26,12 +26,12 @@ mainflux-cli users token john.doe@email.com password
### System Provisioning
#### Create Thing (type Device)
```
mainflux-cli things create '{"type":"device", "name":"myDevice"}' <user_auth_token>
mainflux-cli things create '{"name":"myDevice"}' <user_auth_token>
```
#### Create Thing (type Application)
```
mainflux-cli things create '{"type":"app", "name":"myDevice"}' <user_auth_token>
mainflux-cli things create '{"name":"myDevice"}' <user_auth_token>
```
#### Update Thing
@@ -104,5 +104,5 @@ mainflux-cli channels connections <channel_id> <user_auth_token>
### Messaging
#### Send a message over HTTP
```
mainflux-cli msg send <channel_id> '[{"bn":"Dev1","n":"temp","v":20}, {"n":"hum","v":40}, {"bn":"Dev2", "n":"temp","v":20}, {"n":"hum","v":40}]' <thing_auth_token>
mainflux-cli messages send <channel_id> '[{"bn":"Dev1","n":"temp","v":20}, {"n":"hum","v":40}, {"bn":"Dev2", "n":"temp","v":20}, {"n":"hum","v":40}]' <thing_auth_token>
```
+1 -1
View File
@@ -51,7 +51,7 @@ var cmdChannels = []cobra.Command{
}
if args[0] == "all" {
l, err := sdk.Channels(args[1], uint64(Offset), uint64(Limit))
l, err := sdk.Channels(args[1], uint64(Offset), uint64(Limit), Name)
if err != nil {
logError(err)
return
+5 -10
View File
@@ -22,8 +22,8 @@ import (
var errMalformedCSV = errors.New("malformed CSV")
func createThing(name, kind, token string) (mfxsdk.Thing, error) {
id, err := sdk.CreateThing(mfxsdk.Thing{Name: name, Type: kind}, token)
func createThing(name, token string) (mfxsdk.Thing, error) {
id, err := sdk.CreateThing(mfxsdk.Thing{Name: name}, token)
if err != nil {
return mfxsdk.Thing{}, err
}
@@ -36,7 +36,6 @@ func createThing(name, kind, token string) (mfxsdk.Thing, error) {
m := mfxsdk.Thing{
ID: id,
Name: name,
Type: kind,
Key: t.Key,
}
@@ -87,12 +86,12 @@ var cmdProvision = []cobra.Command{
return
}
if len(l) < 2 {
if len(l) < 1 {
logError(errMalformedCSV)
return
}
m, err := createThing(l[0], l[1], args[1])
m, err := createThing(l[0], args[1])
if err != nil {
logError(err)
return
@@ -187,12 +186,8 @@ var cmdProvision = []cobra.Command{
// Create things
for i := 0; i < numThings; i++ {
n := fmt.Sprintf("d%d", i)
k := "device"
if i%2 != 0 {
k = "app"
}
m, err := createThing(n, k, ut)
m, err := createThing(n, ut)
if err != nil {
logError(err)
return
+1 -1
View File
@@ -53,7 +53,7 @@ var cmdThings = []cobra.Command{
}
if args[0] == "all" {
l, err := sdk.Things(args[1], uint64(Offset), uint64(Limit))
l, err := sdk.Things(args[1], uint64(Offset), uint64(Limit), Name)
if err != nil {
logError(err)
return
+3 -1
View File
@@ -19,7 +19,9 @@ var (
// Limit query parameter
Limit uint = 10
// Offset query parameter
Offset uint
Offset uint = 0
// Name query parameter
Name string = ""
)
func logJSON(iList ...interface{}) {
+78 -21
View File
@@ -7,26 +7,33 @@
package main
import (
"database/sql"
"fmt"
"io"
"log"
"net/http"
"os"
"os/signal"
"strconv"
"syscall"
"time"
rediscons "github.com/mainflux/mainflux/bootstrap/redis/consumer"
redisprod "github.com/mainflux/mainflux/bootstrap/redis/producer"
"github.com/mainflux/mainflux/logger"
opentracing "github.com/opentracing/opentracing-go"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
r "github.com/go-redis/redis"
"github.com/jmoiron/sqlx"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/bootstrap"
api "github.com/mainflux/mainflux/bootstrap/api"
"github.com/mainflux/mainflux/bootstrap/postgres"
"github.com/mainflux/mainflux/bootstrap/redis"
"github.com/mainflux/mainflux/logger"
mflog "github.com/mainflux/mainflux/logger"
mfsdk "github.com/mainflux/mainflux/sdk/go"
usersapi "github.com/mainflux/mainflux/users/api/grpc"
stdprometheus "github.com/prometheus/client_golang/prometheus"
jconfig "github.com/uber/jaeger-client-go/config"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
)
@@ -50,10 +57,15 @@ const (
defBaseURL = "http://localhost"
defThingsPrefix = ""
defUsersURL = "localhost:8181"
defThingsESURL = "localhost:6379"
defThingsESPass = ""
defThingsESDB = "0"
defESURL = "localhost:6379"
defESPass = ""
defESDB = "0"
defInstanceName = "bootstrap"
defJaegerURL = "localhost:6831"
defUsersTimeout = "1" // in seconds
envLogLevel = "MF_BOOTSTRAP_LOG_LEVEL"
envDBHost = "MF_BOOTSTRAP_DB_HOST"
@@ -73,10 +85,15 @@ const (
envBaseURL = "MF_SDK_BASE_URL"
envThingsPrefix = "MF_SDK_THINGS_PREFIX"
envUsersURL = "MF_USERS_URL"
envESURL = "MF_THINGS_ES_URL"
envESPass = "MF_THINGS_ES_PASS"
envESDB = "MF_THINGS_ES_DB"
envThingsESURL = "MF_THINGS_ES_URL"
envThingsESPass = "MF_THINGS_ES_PASS"
envThingsESDB = "MF_THINGS_ES_DB"
envESURL = "MF_BOOTSTRAP_ES_URL"
envESPass = "MF_BOOTSTRAP_ES_PASS"
envESDB = "MF_BOOTSTRAP_ES_DB"
envInstanceName = "MF_BOOTSTRAP_INSTANCE_NAME"
envJaegerURL = "MF_JAEGER_URL"
envUsersTimeout = "MF_BOOTSTRAP_USERS_TIMEOUT"
)
type config struct {
@@ -90,17 +107,21 @@ type config struct {
baseURL string
thingsPrefix string
usersURL string
esThingsURL string
esThingsPass string
esThingsDB string
esURL string
esPass string
esDB string
instanceName string
jaegerURL string
usersTimeout time.Duration
}
func main() {
cfg := loadConfig()
logger, err := logger.New(os.Stdout, cfg.logLevel)
logger, err := mflog.New(os.Stdout, cfg.logLevel)
if err != nil {
log.Fatalf(err.Error())
}
@@ -111,14 +132,20 @@ func main() {
conn := connectToUsers(cfg, logger)
defer conn.Close()
esConn := connectToRedis(cfg.esURL, cfg.esPass, cfg.esDB, logger)
defer esConn.Close()
thingsESConn := connectToRedis(cfg.esThingsURL, cfg.esThingsPass, cfg.esThingsDB, logger)
defer thingsESConn.Close()
svc := newService(conn, db, logger, cfg)
esClient := connectToRedis(cfg.esURL, cfg.esPass, cfg.esDB, logger)
defer esClient.Close()
usersTracer, usersCloser := initJaeger("users", cfg.jaegerURL, logger)
defer usersCloser.Close()
svc := newService(conn, usersTracer, db, logger, esClient, cfg)
errs := make(chan error, 2)
go startHTTPServer(svc, cfg, logger, errs)
go subscribeToThingsES(svc, esConn, cfg.instanceName, logger)
go subscribeToThingsES(svc, thingsESConn, cfg.instanceName, logger)
go func() {
c := make(chan os.Signal)
@@ -147,6 +174,11 @@ func loadConfig() config {
SSLRootCert: mainflux.Env(envDBSSLRootCert, defDBSSLRootCert),
}
timeout, err := strconv.ParseInt(mainflux.Env(envUsersTimeout, defUsersTimeout), 10, 64)
if err != nil {
log.Fatalf("Invalid %s value: %s", envUsersTimeout, err.Error())
}
return config{
logLevel: mainflux.Env(envLogLevel, defLogLevel),
dbConfig: dbConfig,
@@ -158,15 +190,19 @@ func loadConfig() config {
baseURL: mainflux.Env(envBaseURL, defBaseURL),
thingsPrefix: mainflux.Env(envThingsPrefix, defThingsPrefix),
usersURL: mainflux.Env(envUsersURL, defUsersURL),
esThingsURL: mainflux.Env(envThingsESURL, defThingsESURL),
esThingsPass: mainflux.Env(envThingsESPass, defThingsESPass),
esThingsDB: mainflux.Env(envThingsESDB, defThingsESDB),
esURL: mainflux.Env(envESURL, defESURL),
esPass: mainflux.Env(envESPass, defESPass),
esDB: mainflux.Env(envESDB, defESDB),
instanceName: mainflux.Env(envInstanceName, defInstanceName),
jaegerURL: mainflux.Env(envJaegerURL, defJaegerURL),
usersTimeout: time.Duration(timeout) * time.Second,
}
}
func connectToDB(cfg postgres.Config, logger logger.Logger) *sql.DB {
func connectToDB(cfg postgres.Config, logger mflog.Logger) *sqlx.DB {
db, err := postgres.Connect(cfg)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to postgres: %s", err))
@@ -175,7 +211,7 @@ func connectToDB(cfg postgres.Config, logger logger.Logger) *sql.DB {
return db
}
func connectToRedis(redisURL, redisPass, redisDB string, logger logger.Logger) *r.Client {
func connectToRedis(redisURL, redisPass, redisDB string, logger mflog.Logger) *r.Client {
db, err := strconv.Atoi(redisDB)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to redis: %s", err))
@@ -189,7 +225,27 @@ func connectToRedis(redisURL, redisPass, redisDB string, logger logger.Logger) *
})
}
func newService(conn *grpc.ClientConn, db *sql.DB, logger logger.Logger, cfg config) bootstrap.Service {
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger client: %s", err))
os.Exit(1)
}
return tracer, closer
}
func newService(conn *grpc.ClientConn, usersTracer opentracing.Tracer, db *sqlx.DB, logger mflog.Logger, esClient *r.Client, cfg config) bootstrap.Service {
thingsRepo := postgres.NewConfigRepository(db, logger)
config := mfsdk.Config{
@@ -198,9 +254,10 @@ func newService(conn *grpc.ClientConn, db *sql.DB, logger logger.Logger, cfg con
}
sdk := mfsdk.NewSDK(config)
users := usersapi.NewClient(conn)
users := usersapi.NewClient(usersTracer, conn, cfg.usersTimeout)
svc := bootstrap.New(users, thingsRepo, sdk)
svc = redisprod.NewEventStoreMiddleware(svc, esClient)
svc = api.NewLoggingMiddleware(svc, logger)
svc = api.MetricsMiddleware(
svc,
@@ -220,7 +277,7 @@ func newService(conn *grpc.ClientConn, db *sql.DB, logger logger.Logger, cfg con
return svc
}
func connectToUsers(cfg config, logger logger.Logger) *grpc.ClientConn {
func connectToUsers(cfg config, logger mflog.Logger) *grpc.ClientConn {
var opts []grpc.DialOption
if cfg.clientTLS {
if cfg.caCerts != "" {
@@ -245,7 +302,7 @@ func connectToUsers(cfg config, logger logger.Logger) *grpc.ClientConn {
return conn
}
func startHTTPServer(svc bootstrap.Service, cfg config, logger logger.Logger, errs chan error) {
func startHTTPServer(svc bootstrap.Service, cfg config, logger mflog.Logger, errs chan error) {
p := fmt.Sprintf(":%s", cfg.httpPort)
if cfg.serverCert != "" || cfg.serverKey != "" {
logger.Info(fmt.Sprintf("Bootstrap service started using https on port %s with cert %s key %s",
@@ -257,8 +314,8 @@ func startHTTPServer(svc bootstrap.Service, cfg config, logger logger.Logger, er
errs <- http.ListenAndServe(p, api.MakeHandler(svc, bootstrap.NewConfigReader()))
}
func subscribeToThingsES(svc bootstrap.Service, client *r.Client, consumer string, logger logger.Logger) {
eventStore := redis.NewEventStore(svc, client, consumer, logger)
func subscribeToThingsES(svc bootstrap.Service, client *r.Client, consumer string, logger mflog.Logger) {
eventStore := rediscons.NewEventStore(svc, client, consumer, logger)
logger.Info("Subscribed to Redis Event Store")
if err := eventStore.Subscribe("mainflux.things"); err != nil {
logger.Warn(fmt.Sprintf("Botstrap service failed to subscribe to event sourcing: %s", err))
+90 -33
View File
@@ -9,6 +9,7 @@ package main
import (
"fmt"
"io"
"log"
"net/http"
"os"
@@ -16,6 +17,7 @@ import (
"strconv"
"strings"
"syscall"
"time"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
"github.com/gocql/gocql"
@@ -24,8 +26,10 @@ import (
"github.com/mainflux/mainflux/readers"
"github.com/mainflux/mainflux/readers/api"
"github.com/mainflux/mainflux/readers/cassandra"
thingsapi "github.com/mainflux/mainflux/things/api/grpc"
thingsapi "github.com/mainflux/mainflux/things/api/auth/grpc"
opentracing "github.com/opentracing/opentracing-go"
stdprometheus "github.com/prometheus/client_golang/prometheus"
jconfig "github.com/uber/jaeger-client-go/config"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
)
@@ -33,31 +37,42 @@ import (
const (
sep = ","
defLogLevel = "error"
defPort = "8180"
defCluster = "127.0.0.1"
defKeyspace = "mainflux"
defThingsURL = "localhost:8181"
defClientTLS = "false"
defCACerts = ""
defLogLevel = "error"
defPort = "8180"
defCluster = "127.0.0.1"
defKeyspace = "mainflux"
defDBUsername = ""
defDBPassword = ""
defDBPort = "9042"
defThingsURL = "localhost:8181"
defClientTLS = "false"
defCACerts = ""
defJaegerURL = "localhost:6831"
defThingsTimeout = "1" // in seconds
envLogLevel = "MF_CASSANDRA_READER_LOG_LEVEL"
envPort = "MF_CASSANDRA_READER_PORT"
envCluster = "MF_CASSANDRA_READER_DB_CLUSTER"
envKeyspace = "MF_CASSANDRA_READER_DB_KEYSPACE"
envThingsURL = "MF_THINGS_URL"
envClientTLS = "MF_CASSANDRA_READER_CLIENT_TLS"
envCACerts = "MF_CASSANDRA_READER_CA_CERTS"
envLogLevel = "MF_CASSANDRA_READER_LOG_LEVEL"
envPort = "MF_CASSANDRA_READER_PORT"
envCluster = "MF_CASSANDRA_READER_DB_CLUSTER"
envKeyspace = "MF_CASSANDRA_READER_DB_KEYSPACE"
envDBUsername = "MF_CASSANDRA_READER_DB_USERNAME"
envDBPassword = "MF_CASSANDRA_READER_DB_PASSWORD"
envDBPort = "MF_CASSANDRA_READER_DB_PORT"
envThingsURL = "MF_THINGS_URL"
envClientTLS = "MF_CASSANDRA_READER_CLIENT_TLS"
envCACerts = "MF_CASSANDRA_READER_CA_CERTS"
envJaegerURL = "MF_JAEGER_URL"
envThingsTimeout = "MF_CASSANDRA_READER_THINGS_TIMEOUT"
)
type config struct {
logLevel string
port string
cluster string
keyspace string
thingsURL string
clientTLS bool
caCerts string
logLevel string
port string
dbCfg cassandra.DBConfig
thingsURL string
clientTLS bool
caCerts string
jaegerURL string
thingsTimeout time.Duration
}
func main() {
@@ -68,13 +83,16 @@ func main() {
log.Fatalf(err.Error())
}
session := connectToCassandra(cfg.cluster, cfg.keyspace, logger)
session := connectToCassandra(cfg.dbCfg, logger)
defer session.Close()
conn := connectToThings(cfg, logger)
defer conn.Close()
tc := thingsapi.NewClient(conn)
thingsTracer, thingsCloser := initJaeger("things", cfg.jaegerURL, logger)
defer thingsCloser.Close()
tc := thingsapi.NewClient(conn, thingsTracer, cfg.thingsTimeout)
repo := newService(session, logger)
errs := make(chan error, 2)
@@ -92,24 +110,43 @@ func main() {
}
func loadConfig() config {
dbPort, err := strconv.Atoi(mainflux.Env(envDBPort, defDBPort))
if err != nil {
log.Fatal(err)
}
dbCfg := cassandra.DBConfig{
Hosts: strings.Split(mainflux.Env(envCluster, defCluster), sep),
Keyspace: mainflux.Env(envKeyspace, defKeyspace),
Username: mainflux.Env(envDBUsername, defDBUsername),
Password: mainflux.Env(envDBPassword, defDBPassword),
Port: dbPort,
}
tls, err := strconv.ParseBool(mainflux.Env(envClientTLS, defClientTLS))
if err != nil {
log.Fatalf("Invalid value passed for %s\n", envClientTLS)
}
timeout, err := strconv.ParseInt(mainflux.Env(envThingsTimeout, defThingsTimeout), 10, 64)
if err != nil {
log.Fatalf("Invalid %s value: %s", envThingsTimeout, err.Error())
}
return config{
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
cluster: mainflux.Env(envCluster, defCluster),
keyspace: mainflux.Env(envKeyspace, defKeyspace),
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
dbCfg: dbCfg,
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
jaegerURL: mainflux.Env(envJaegerURL, defJaegerURL),
thingsTimeout: time.Duration(timeout) * time.Second,
}
}
func connectToCassandra(cluster, keyspace string, logger logger.Logger) *gocql.Session {
session, err := cassandra.Connect(strings.Split(cluster, sep), keyspace)
func connectToCassandra(dbCfg cassandra.DBConfig, logger logger.Logger) *gocql.Session {
session, err := cassandra.Connect(dbCfg)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to Cassandra cluster: %s", err))
os.Exit(1)
@@ -142,6 +179,26 @@ func connectToThings(cfg config, logger logger.Logger) *grpc.ClientConn {
return conn
}
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger client: %s", err))
os.Exit(1)
}
return tracer, closer
}
func newService(session *gocql.Session, logger logger.Logger) readers.MessageRepository {
repo := cassandra.New(session)
repo = api.LoggingMiddleware(repo, logger)
+77 -24
View File
@@ -9,46 +9,58 @@ package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"github.com/BurntSushi/toml"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
"github.com/gocql/gocql"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/logger"
"github.com/mainflux/mainflux/writers"
"github.com/mainflux/mainflux/writers/api"
"github.com/mainflux/mainflux/writers/cassandra"
"github.com/nats-io/go-nats"
nats "github.com/nats-io/go-nats"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const (
queue = "cassandra-writer"
sep = ","
svcName = "cassandra-writer"
sep = ","
defNatsURL = nats.DefaultURL
defLogLevel = "error"
defPort = "8180"
defCluster = "127.0.0.1"
defKeyspace = "mainflux"
defNatsURL = nats.DefaultURL
defLogLevel = "error"
defPort = "8180"
defCluster = "127.0.0.1"
defKeyspace = "mainflux"
defDBUsername = ""
defDBPassword = ""
defDBPort = "9042"
defChanCfgPath = "/config/channels.toml"
envNatsURL = "MF_NATS_URL"
envLogLevel = "MF_CASSANDRA_WRITER_LOG_LEVEL"
envPort = "MF_CASSANDRA_WRITER_PORT"
envCluster = "MF_CASSANDRA_WRITER_DB_CLUSTER"
envKeyspace = "MF_CASSANDRA_WRITER_DB_KEYSPACE"
envNatsURL = "MF_NATS_URL"
envLogLevel = "MF_CASSANDRA_WRITER_LOG_LEVEL"
envPort = "MF_CASSANDRA_WRITER_PORT"
envCluster = "MF_CASSANDRA_WRITER_DB_CLUSTER"
envKeyspace = "MF_CASSANDRA_WRITER_DB_KEYSPACE"
envDBUsername = "MF_CASSANDRA_WRITER_DB_USERNAME"
envDBPassword = "MF_CASSANDRA_WRITER_DB_PASSWORD"
envDBPort = "MF_CASSANDRA_WRITER_DB_PORT"
envChanCfgPath = "MF_CASSANDRA_WRITER_CHANNELS_CONFIG"
)
type config struct {
natsURL string
logLevel string
port string
cluster string
keyspace string
dbCfg cassandra.DBConfig
channels map[string]bool
}
func main() {
@@ -62,11 +74,11 @@ func main() {
nc := connectToNATS(cfg.natsURL, logger)
defer nc.Close()
session := connectToCassandra(cfg.cluster, cfg.keyspace, logger)
session := connectToCassandra(cfg.dbCfg, logger)
defer session.Close()
repo := newService(session, logger)
if err := writers.Start(nc, repo, queue, logger); err != nil {
if err := writers.Start(nc, repo, svcName, cfg.channels, logger); err != nil {
logger.Error(fmt.Sprintf("Failed to create Cassandra writer: %s", err))
}
@@ -85,15 +97,56 @@ func main() {
}
func loadConfig() config {
dbPort, err := strconv.Atoi(mainflux.Env(envDBPort, defDBPort))
if err != nil {
log.Fatal(err)
}
dbCfg := cassandra.DBConfig{
Hosts: strings.Split(mainflux.Env(envCluster, defCluster), sep),
Keyspace: mainflux.Env(envKeyspace, defKeyspace),
Username: mainflux.Env(envDBUsername, defDBUsername),
Password: mainflux.Env(envDBPassword, defDBPassword),
Port: dbPort,
}
chanCfgPath := mainflux.Env(envChanCfgPath, defChanCfgPath)
return config{
natsURL: mainflux.Env(envNatsURL, defNatsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
cluster: mainflux.Env(envCluster, defCluster),
keyspace: mainflux.Env(envKeyspace, defKeyspace),
dbCfg: dbCfg,
channels: loadChansConfig(chanCfgPath),
}
}
type channels struct {
List []string `toml:"filter"`
}
type chanConfig struct {
Channels channels `toml:"channels"`
}
func loadChansConfig(chanConfigPath string) map[string]bool {
data, err := ioutil.ReadFile(chanConfigPath)
if err != nil {
log.Fatal(err)
}
var chanCfg chanConfig
if err := toml.Unmarshal(data, &chanCfg); err != nil {
log.Fatal(err)
}
chans := map[string]bool{}
for _, ch := range chanCfg.Channels.List {
chans[ch] = true
}
return chans
}
func connectToNATS(url string, logger logger.Logger) *nats.Conn {
nc, err := nats.Connect(url)
if err != nil {
@@ -104,8 +157,8 @@ func connectToNATS(url string, logger logger.Logger) *nats.Conn {
return nc
}
func connectToCassandra(cluster, keyspace string, logger logger.Logger) *gocql.Session {
session, err := cassandra.Connect(strings.Split(cluster, sep), keyspace)
func connectToCassandra(dbCfg cassandra.DBConfig, logger logger.Logger) *gocql.Session {
session, err := cassandra.Connect(dbCfg)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to Cassandra cluster: %s", err))
os.Exit(1)
@@ -116,8 +169,8 @@ func connectToCassandra(cluster, keyspace string, logger logger.Logger) *gocql.S
func newService(session *gocql.Session, logger logger.Logger) writers.MessageRepository {
repo := cassandra.New(session)
repo = writers.LoggingMiddleware(repo, logger)
repo = writers.MetricsMiddleware(
repo = api.LoggingMiddleware(repo, logger)
repo = api.MetricsMiddleware(
repo,
kitprometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: "cassandra",
@@ -139,5 +192,5 @@ func newService(session *gocql.Session, logger logger.Logger) writers.MessageRep
func startHTTPServer(port string, errs chan error, logger logger.Logger) {
p := fmt.Sprintf(":%s", port)
logger.Info(fmt.Sprintf("Cassandra writer service started, exposed port %s", port))
errs <- http.ListenAndServe(p, cassandra.MakeHandler())
errs <- http.ListenAndServe(p, api.MakeHandler(svcName))
}
+8
View File
@@ -120,6 +120,14 @@ func main() {
"offset query parameter",
)
rootCmd.PersistentFlags().StringVarP(
&cli.Name,
"name",
"n",
"",
"name query parameter",
)
if err := rootCmd.Execute(); err != nil {
log.Fatal(err)
}
+70 -31
View File
@@ -9,6 +9,7 @@ package main
import (
"fmt"
"io"
"log"
"net/http"
"os"
@@ -24,8 +25,10 @@ import (
"github.com/mainflux/mainflux/coap/api"
"github.com/mainflux/mainflux/coap/nats"
logger "github.com/mainflux/mainflux/logger"
thingsapi "github.com/mainflux/mainflux/things/api/grpc"
thingsapi "github.com/mainflux/mainflux/things/api/auth/grpc"
opentracing "github.com/opentracing/opentracing-go"
stdprometheus "github.com/prometheus/client_golang/prometheus"
jconfig "github.com/uber/jaeger-client-go/config"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
@@ -33,31 +36,37 @@ import (
)
const (
defPort = "5683"
defNatsURL = broker.DefaultURL
defThingsURL = "localhost:8181"
defLogLevel = "error"
defClientTLS = "false"
defCACerts = ""
defPingPeriod = "12"
defPort = "5683"
defNatsURL = broker.DefaultURL
defThingsURL = "localhost:8181"
defLogLevel = "error"
defClientTLS = "false"
defCACerts = ""
defPingPeriod = "12"
defJaegerURL = "localhost:6831"
defThingsTimeout = "1" // in seconds
envPort = "MF_COAP_ADAPTER_PORT"
envNatsURL = "MF_NATS_URL"
envThingsURL = "MF_THINGS_URL"
envLogLevel = "MF_COAP_ADAPTER_LOG_LEVEL"
envClientTLS = "MF_COAP_ADAPTER_CLIENT_TLS"
envCACerts = "MF_COAP_ADAPTER_CA_CERTS"
envPingPeriod = "MF_COAP_ADAPTER_PING_PERIOD"
envPort = "MF_COAP_ADAPTER_PORT"
envNatsURL = "MF_NATS_URL"
envThingsURL = "MF_THINGS_URL"
envLogLevel = "MF_COAP_ADAPTER_LOG_LEVEL"
envClientTLS = "MF_COAP_ADAPTER_CLIENT_TLS"
envCACerts = "MF_COAP_ADAPTER_CA_CERTS"
envPingPeriod = "MF_COAP_ADAPTER_PING_PERIOD"
envJaegerURL = "MF_JAEGER_URL"
envThingsTimeout = "MF_COAP_ADAPTER_THINGS_TIMEOUT"
)
type config struct {
port string
natsURL string
thingsURL string
logLevel string
clientTLS bool
caCerts string
pingPeriod time.Duration
port string
natsURL string
thingsURL string
logLevel string
clientTLS bool
caCerts string
pingPeriod time.Duration
jaegerURL string
thingsTimeout time.Duration
}
func main() {
@@ -78,10 +87,13 @@ func main() {
conn := connectToThings(cfg, logger)
defer conn.Close()
cc := thingsapi.NewClient(conn)
thingsTracer, thingsCloser := initJaeger("things", cfg.jaegerURL, logger)
defer thingsCloser.Close()
cc := thingsapi.NewClient(conn, thingsTracer, cfg.thingsTimeout)
respChan := make(chan string, 10000)
pubsub := nats.New(nc)
svc := coap.New(pubsub, respChan)
svc := coap.New(pubsub, cc, respChan)
svc = api.LoggingMiddleware(svc, logger)
svc = api.MetricsMiddleware(
@@ -130,14 +142,21 @@ func loadConfig() config {
log.Fatalf("Value of %s must be between 1 and 24", envPingPeriod)
}
timeout, err := strconv.ParseInt(mainflux.Env(envThingsTimeout, defThingsTimeout), 10, 64)
if err != nil {
log.Fatalf("Invalid %s value: %s", envThingsTimeout, err.Error())
}
return config{
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
natsURL: mainflux.Env(envNatsURL, defNatsURL),
port: mainflux.Env(envPort, defPort),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
pingPeriod: time.Duration(pp),
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
natsURL: mainflux.Env(envNatsURL, defNatsURL),
port: mainflux.Env(envPort, defPort),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
pingPeriod: time.Duration(pp),
jaegerURL: mainflux.Env(envJaegerURL, defJaegerURL),
thingsTimeout: time.Duration(timeout) * time.Second,
}
}
@@ -165,6 +184,26 @@ func connectToThings(cfg config, logger logger.Logger) *grpc.ClientConn {
return conn
}
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger client: %s", err))
os.Exit(1)
}
return tracer, closer
}
func startHTTPServer(port string, logger logger.Logger, errs chan error) {
p := fmt.Sprintf(":%s", port)
logger.Info(fmt.Sprintf("CoAP service started, exposed port %s", port))
+74 -30
View File
@@ -1,5 +1,5 @@
//
// Copyright (c) 2018
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
@@ -9,12 +9,14 @@ package main
import (
"fmt"
"io"
"log"
"net/http"
"os"
"os/signal"
"strconv"
"syscall"
"time"
"google.golang.org/grpc/credentials"
@@ -24,34 +26,43 @@ import (
"github.com/mainflux/mainflux/http/api"
"github.com/mainflux/mainflux/http/nats"
"github.com/mainflux/mainflux/logger"
thingsapi "github.com/mainflux/mainflux/things/api/grpc"
thingsapi "github.com/mainflux/mainflux/things/api/auth/grpc"
broker "github.com/nats-io/go-nats"
opentracing "github.com/opentracing/opentracing-go"
stdprometheus "github.com/prometheus/client_golang/prometheus"
jconfig "github.com/uber/jaeger-client-go/config"
"google.golang.org/grpc"
)
const (
defClientTLS = "false"
defCACerts = ""
defPort = "8180"
defLogLevel = "error"
defNatsURL = broker.DefaultURL
defThingsURL = "localhost:8181"
envClientTLS = "MF_HTTP_ADAPTER_CLIENT_TLS"
envCACerts = "MF_HTTP_ADAPTER_CA_CERTS"
envPort = "MF_HTTP_ADAPTER_PORT"
envLogLevel = "MF_HTTP_ADAPTER_LOG_LEVEL"
envNatsURL = "MF_NATS_URL"
envThingsURL = "MF_THINGS_URL"
defClientTLS = "false"
defCACerts = ""
defPort = "8180"
defLogLevel = "error"
defNatsURL = broker.DefaultURL
defThingsURL = "localhost:8181"
defJaegerURL = "localhost:6831"
defThingsTimeout = "1" // in seconds
envClientTLS = "MF_HTTP_ADAPTER_CLIENT_TLS"
envCACerts = "MF_HTTP_ADAPTER_CA_CERTS"
envPort = "MF_HTTP_ADAPTER_PORT"
envLogLevel = "MF_HTTP_ADAPTER_LOG_LEVEL"
envNatsURL = "MF_NATS_URL"
envThingsURL = "MF_THINGS_URL"
envJaegerURL = "MF_JAEGER_URL"
envThingsTimeout = "MF_HTTP_ADAPTER_THINGS_TIMEOUT"
)
type config struct {
thingsURL string
natsURL string
logLevel string
port string
clientTLS bool
caCerts string
thingsURL string
natsURL string
logLevel string
port string
clientTLS bool
caCerts string
jaegerURL string
thingsTimeout time.Duration
}
func main() {
@@ -73,10 +84,16 @@ func main() {
conn := connectToThings(cfg, logger)
defer conn.Close()
cc := thingsapi.NewClient(conn)
tracer, closer := initJaeger("http_adapter", cfg.jaegerURL, logger)
defer closer.Close()
thingsTracer, thingsCloser := initJaeger("things", cfg.jaegerURL, logger)
defer thingsCloser.Close()
cc := thingsapi.NewClient(conn, thingsTracer, cfg.thingsTimeout)
pub := nats.NewMessagePublisher(nc)
svc := adapter.New(pub)
svc := adapter.New(pub, cc)
svc = api.LoggingMiddleware(svc, logger)
svc = api.MetricsMiddleware(
svc,
@@ -99,7 +116,7 @@ func main() {
go func() {
p := fmt.Sprintf(":%s", cfg.port)
logger.Info(fmt.Sprintf("HTTP adapter service started on port %s", cfg.port))
errs <- http.ListenAndServe(p, api.MakeHandler(svc, cc))
errs <- http.ListenAndServe(p, api.MakeHandler(svc, tracer))
}()
go func() {
@@ -118,14 +135,41 @@ func loadConfig() config {
log.Fatalf("Invalid value passed for %s\n", envClientTLS)
}
return config{
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
natsURL: mainflux.Env(envNatsURL, defNatsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
timeout, err := strconv.ParseInt(mainflux.Env(envThingsTimeout, defThingsTimeout), 10, 64)
if err != nil {
log.Fatalf("Invalid %s value: %s", envThingsTimeout, err.Error())
}
return config{
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
natsURL: mainflux.Env(envNatsURL, defNatsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
jaegerURL: mainflux.Env(envJaegerURL, defJaegerURL),
thingsTimeout: time.Duration(timeout) * time.Second,
}
}
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger client: %s", err))
os.Exit(1)
}
return tracer, closer
}
func connectToThings(cfg config, logger logger.Logger) *grpc.ClientConn {
+80 -45
View File
@@ -2,12 +2,14 @@ package main
import (
"fmt"
"io"
"log"
"net/http"
"os"
"os/signal"
"strconv"
"syscall"
"time"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
influxdata "github.com/influxdata/influxdb/client/v2"
@@ -16,47 +18,55 @@ import (
"github.com/mainflux/mainflux/readers"
"github.com/mainflux/mainflux/readers/api"
"github.com/mainflux/mainflux/readers/influxdb"
thingsapi "github.com/mainflux/mainflux/things/api/grpc"
thingsapi "github.com/mainflux/mainflux/things/api/auth/grpc"
opentracing "github.com/opentracing/opentracing-go"
stdprometheus "github.com/prometheus/client_golang/prometheus"
jconfig "github.com/uber/jaeger-client-go/config"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
)
const (
defThingsURL = "localhost:8181"
defLogLevel = "error"
defPort = "8180"
defDBName = "mainflux"
defDBHost = "localhost"
defDBPort = "8086"
defDBUser = "mainflux"
defDBPass = "mainflux"
defClientTLS = "false"
defCACerts = ""
defThingsURL = "localhost:8181"
defLogLevel = "error"
defPort = "8180"
defDBName = "mainflux"
defDBHost = "localhost"
defDBPort = "8086"
defDBUser = "mainflux"
defDBPass = "mainflux"
defClientTLS = "false"
defCACerts = ""
defJaegerURL = "localhost:6831"
defThingsTimeout = "1" // in seconds
envThingsURL = "MF_THINGS_URL"
envLogLevel = "MF_INFLUX_READER_LOG_LEVEL"
envPort = "MF_INFLUX_READER_PORT"
envDBName = "MF_INFLUX_READER_DB_NAME"
envDBHost = "MF_INFLUX_READER_DB_HOST"
envDBPort = "MF_INFLUX_READER_DB_PORT"
envDBUser = "MF_INFLUX_READER_DB_USER"
envDBPass = "MF_INFLUX_READER_DB_PASS"
envClientTLS = "MF_INFLUX_READER_CLIENT_TLS"
envCACerts = "MF_INFLUX_READER_CA_CERTS"
envThingsURL = "MF_THINGS_URL"
envLogLevel = "MF_INFLUX_READER_LOG_LEVEL"
envPort = "MF_INFLUX_READER_PORT"
envDBName = "MF_INFLUX_READER_DB_NAME"
envDBHost = "MF_INFLUX_READER_DB_HOST"
envDBPort = "MF_INFLUX_READER_DB_PORT"
envDBUser = "MF_INFLUX_READER_DB_USER"
envDBPass = "MF_INFLUX_READER_DB_PASS"
envClientTLS = "MF_INFLUX_READER_CLIENT_TLS"
envCACerts = "MF_INFLUX_READER_CA_CERTS"
envJaegerURL = "MF_JAEGER_URL"
envThingsTimeout = "MF_INFLUX_READER_THINGS_TIMEOUT"
)
type config struct {
thingsURL string
logLevel string
port string
dbName string
dbHost string
dbPort string
dbUser string
dbPass string
clientTLS bool
caCerts string
thingsURL string
logLevel string
port string
dbName string
dbHost string
dbPort string
dbUser string
dbPass string
clientTLS bool
caCerts string
jaegerURL string
thingsTimeout time.Duration
}
func main() {
@@ -68,7 +78,10 @@ func main() {
conn := connectToThings(cfg, logger)
defer conn.Close()
tc := thingsapi.NewClient(conn)
thingsTracer, thingsCloser := initJaeger("things", cfg.jaegerURL, logger)
defer thingsCloser.Close()
tc := thingsapi.NewClient(conn, thingsTracer, cfg.thingsTimeout)
client, err := influxdata.NewHTTPClient(clientCfg)
if err != nil {
@@ -98,17 +111,24 @@ func loadConfigs() (config, influxdata.HTTPConfig) {
log.Fatalf("Invalid value passed for %s\n", envClientTLS)
}
timeout, err := strconv.ParseInt(mainflux.Env(envThingsTimeout, defThingsTimeout), 10, 64)
if err != nil {
log.Fatalf("Invalid %s value: %s", envThingsTimeout, err.Error())
}
cfg := config{
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
dbName: mainflux.Env(envDBName, defDBName),
dbHost: mainflux.Env(envDBHost, defDBHost),
dbPort: mainflux.Env(envDBPort, defDBPort),
dbUser: mainflux.Env(envDBUser, defDBUser),
dbPass: mainflux.Env(envDBPass, defDBPass),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
dbName: mainflux.Env(envDBName, defDBName),
dbHost: mainflux.Env(envDBHost, defDBHost),
dbPort: mainflux.Env(envDBPort, defDBPort),
dbUser: mainflux.Env(envDBUser, defDBUser),
dbPass: mainflux.Env(envDBPass, defDBPass),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
jaegerURL: mainflux.Env(envJaegerURL, defJaegerURL),
thingsTimeout: time.Duration(timeout) * time.Second,
}
clientCfg := influxdata.HTTPConfig{
@@ -144,13 +164,28 @@ func connectToThings(cfg config, logger logger.Logger) *grpc.ClientConn {
return conn
}
func newService(client influxdata.Client, dbName string, logger logger.Logger) readers.MessageRepository {
repo, err := influxdb.New(client, dbName)
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to create InfluxDB writer: %s", err))
logger.Error(fmt.Sprintf("Failed to init Jaeger client: %s", err))
os.Exit(1)
}
return tracer, closer
}
func newService(client influxdata.Client, dbName string, logger logger.Logger) readers.MessageRepository {
repo := influxdb.New(client, dbName)
repo = api.LoggingMiddleware(repo, logger)
repo = api.MetricsMiddleware(
repo,
+73 -36
View File
@@ -9,6 +9,7 @@ package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
@@ -17,18 +18,20 @@ import (
"syscall"
"time"
"github.com/BurntSushi/toml"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
influxdata "github.com/influxdata/influxdb/client/v2"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/logger"
"github.com/mainflux/mainflux/writers"
"github.com/mainflux/mainflux/writers/api"
"github.com/mainflux/mainflux/writers/influxdb"
"github.com/nats-io/go-nats"
nats "github.com/nats-io/go-nats"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const (
queue = "influxdb-writer"
svcName = "influxdb-writer"
defNatsURL = nats.DefaultURL
defLogLevel = "error"
@@ -40,6 +43,7 @@ const (
defDBPort = "8086"
defDBUser = "mainflux"
defDBPass = "mainflux"
defChanCfgPath = "/config/channels.toml"
envNatsURL = "MF_NATS_URL"
envLogLevel = "MF_INFLUX_WRITER_LOG_LEVEL"
@@ -51,28 +55,32 @@ const (
envDBPort = "MF_INFLUX_WRITER_DB_PORT"
envDBUser = "MF_INFLUX_WRITER_DB_USER"
envDBPass = "MF_INFLUX_WRITER_DB_PASS"
envChanCfgPath = "MF_INFLUX_WRITER_CHANNELS_CONFIG"
)
type config struct {
NatsURL string
LogLevel string
Port string
BatchSize string
BatchTimeout string
DBName string
DBHost string
DBPort string
DBUser string
DBPass string
natsURL string
logLevel string
port string
batchSize string
batchTimeout string
dbName string
dbHost string
dbPort string
dbUser string
dbPass string
channels map[string]bool
}
func main() {
cfg, clientCfg := loadConfigs()
logger, err := logger.New(os.Stdout, cfg.LogLevel)
logger, err := logger.New(os.Stdout, cfg.logLevel)
if err != nil {
log.Fatalf(err.Error())
}
nc, err := nats.Connect(cfg.NatsURL)
nc, err := nats.Connect(cfg.natsURL)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to NATS: %s", err))
os.Exit(1)
@@ -86,30 +94,30 @@ func main() {
}
defer client.Close()
batchTimeout, err := strconv.Atoi(cfg.BatchTimeout)
batchTimeout, err := strconv.Atoi(cfg.batchTimeout)
if err != nil {
logger.Error(fmt.Sprintf("Invalid value for batch timeout: %s", err))
os.Exit(1)
}
batchSize, err := strconv.Atoi(cfg.BatchSize)
batchSize, err := strconv.Atoi(cfg.batchSize)
if err != nil {
logger.Error(fmt.Sprintf("Invalid value of batch size: %s", err))
os.Exit(1)
}
timeout := time.Duration(batchTimeout) * time.Second
repo, err := influxdb.New(client, cfg.DBName, batchSize, timeout)
repo, err := influxdb.New(client, cfg.dbName, batchSize, timeout)
if err != nil {
logger.Error(fmt.Sprintf("Failed to create InfluxDB writer: %s", err))
os.Exit(1)
}
counter, latency := makeMetrics()
repo = writers.LoggingMiddleware(repo, logger)
repo = writers.MetricsMiddleware(repo, counter, latency)
if err := writers.Start(nc, repo, queue, logger); err != nil {
logger.Error(fmt.Sprintf("Failed to start message writer: %s", err))
repo = api.LoggingMiddleware(repo, logger)
repo = api.MetricsMiddleware(repo, counter, latency)
if err := writers.Start(nc, repo, svcName, cfg.channels, logger); err != nil {
logger.Error(fmt.Sprintf("Failed to start InfluxDB writer: %s", err))
os.Exit(1)
}
@@ -120,35 +128,64 @@ func main() {
errs <- fmt.Errorf("%s", <-c)
}()
go startHTTPService(cfg.Port, logger, errs)
go startHTTPService(cfg.port, logger, errs)
err = <-errs
logger.Error(fmt.Sprintf("InfluxDB writer service terminated: %s", err))
}
func loadConfigs() (config, influxdata.HTTPConfig) {
chanCfgPath := mainflux.Env(envChanCfgPath, defChanCfgPath)
cfg := config{
NatsURL: mainflux.Env(envNatsURL, defNatsURL),
LogLevel: mainflux.Env(envLogLevel, defLogLevel),
Port: mainflux.Env(envPort, defPort),
BatchSize: mainflux.Env(envBatchSize, defBatchSize),
BatchTimeout: mainflux.Env(envBatchTimeout, defBatchTimeout),
DBName: mainflux.Env(envDBName, defDBName),
DBHost: mainflux.Env(envDBHost, defDBHost),
DBPort: mainflux.Env(envDBPort, defDBPort),
DBUser: mainflux.Env(envDBUser, defDBUser),
DBPass: mainflux.Env(envDBPass, defDBPass),
natsURL: mainflux.Env(envNatsURL, defNatsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
batchSize: mainflux.Env(envBatchSize, defBatchSize),
batchTimeout: mainflux.Env(envBatchTimeout, defBatchTimeout),
dbName: mainflux.Env(envDBName, defDBName),
dbHost: mainflux.Env(envDBHost, defDBHost),
dbPort: mainflux.Env(envDBPort, defDBPort),
dbUser: mainflux.Env(envDBUser, defDBUser),
dbPass: mainflux.Env(envDBPass, defDBPass),
channels: loadChansConfig(chanCfgPath),
}
clientCfg := influxdata.HTTPConfig{
Addr: fmt.Sprintf("http://%s:%s", cfg.DBHost, cfg.DBPort),
Username: cfg.DBUser,
Password: cfg.DBPass,
Addr: fmt.Sprintf("http://%s:%s", cfg.dbHost, cfg.dbPort),
Username: cfg.dbUser,
Password: cfg.dbPass,
}
return cfg, clientCfg
}
type channels struct {
List []string `toml:"filter"`
}
type chanConfig struct {
Channels channels `toml:"channels"`
}
func loadChansConfig(chanConfigPath string) map[string]bool {
data, err := ioutil.ReadFile(chanConfigPath)
if err != nil {
log.Fatal(err)
}
var chanCfg chanConfig
if err := toml.Unmarshal(data, &chanCfg); err != nil {
log.Fatal(err)
}
chans := map[string]bool{}
for _, ch := range chanCfg.Channels.List {
chans[ch] = true
}
return chans
}
func makeMetrics() (*kitprometheus.Counter, *kitprometheus.Summary) {
counter := kitprometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: "influxdb",
@@ -170,5 +207,5 @@ func makeMetrics() (*kitprometheus.Counter, *kitprometheus.Summary) {
func startHTTPService(port string, logger logger.Logger, errs chan error) {
p := fmt.Sprintf(":%s", port)
logger.Info(fmt.Sprintf("InfluxDB writer service started, exposed port %s", p))
errs <- http.ListenAndServe(p, influxdb.MakeHandler())
errs <- http.ListenAndServe(p, api.MakeHandler(svcName))
}
+1 -1
View File
@@ -20,7 +20,7 @@ import (
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
"github.com/mainflux/mainflux/lora/redis"
"github.com/nats-io/go-nats"
nats "github.com/nats-io/go-nats"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
+78 -36
View File
@@ -10,12 +10,14 @@ package main
import (
"context"
"fmt"
"io"
"log"
"net/http"
"os"
"os/signal"
"strconv"
"syscall"
"time"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
"github.com/mainflux/mainflux"
@@ -23,42 +25,51 @@ import (
"github.com/mainflux/mainflux/readers"
"github.com/mainflux/mainflux/readers/api"
"github.com/mainflux/mainflux/readers/mongodb"
thingsapi "github.com/mainflux/mainflux/things/api/grpc"
"github.com/mongodb/mongo-go-driver/mongo"
thingsapi "github.com/mainflux/mainflux/things/api/auth/grpc"
opentracing "github.com/opentracing/opentracing-go"
stdprometheus "github.com/prometheus/client_golang/prometheus"
jconfig "github.com/uber/jaeger-client-go/config"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
)
const (
defThingsURL = "localhost:8181"
defLogLevel = "error"
defPort = "8180"
defDBName = "mainflux"
defDBHost = "localhost"
defDBPort = "27017"
defClientTLS = "false"
defCACerts = ""
defThingsURL = "localhost:8181"
defLogLevel = "error"
defPort = "8180"
defDBName = "mainflux"
defDBHost = "localhost"
defDBPort = "27017"
defClientTLS = "false"
defCACerts = ""
defJaegerURL = "localhost:6831"
defThingsTimeout = "1" // in seconds
envThingsURL = "MF_THINGS_URL"
envLogLevel = "MF_MONGO_READER_LOG_LEVEL"
envPort = "MF_MONGO_READER_PORT"
envDBName = "MF_MONGO_READER_DB_NAME"
envDBHost = "MF_MONGO_READER_DB_HOST"
envDBPort = "MF_MONGO_READER_DB_PORT"
envClientTLS = "MF_MONGO_READER_CLIENT_TLS"
envCACerts = "MF_MONGO_READER_CA_CERTS"
envThingsURL = "MF_THINGS_URL"
envLogLevel = "MF_MONGO_READER_LOG_LEVEL"
envPort = "MF_MONGO_READER_PORT"
envDBName = "MF_MONGO_READER_DB_NAME"
envDBHost = "MF_MONGO_READER_DB_HOST"
envDBPort = "MF_MONGO_READER_DB_PORT"
envClientTLS = "MF_MONGO_READER_CLIENT_TLS"
envCACerts = "MF_MONGO_READER_CA_CERTS"
envJaegerURL = "MF_JAEGER_URL"
envThingsTimeout = "MF_MONGO_READER_THINGS_TIMEOUT"
)
type config struct {
thingsURL string
logLevel string
port string
dbName string
dbHost string
dbPort string
clientTLS bool
caCerts string
thingsURL string
logLevel string
port string
dbName string
dbHost string
dbPort string
clientTLS bool
caCerts string
jaegerURL string
thingsTimeout time.Duration
}
func main() {
@@ -71,7 +82,10 @@ func main() {
conn := connectToThings(cfg, logger)
defer conn.Close()
tc := thingsapi.NewClient(conn)
thingsTracer, thingsCloser := initJaeger("things", cfg.jaegerURL, logger)
defer thingsCloser.Close()
tc := thingsapi.NewClient(conn, thingsTracer, cfg.thingsTimeout)
db := connectToMongoDB(cfg.dbHost, cfg.dbPort, cfg.dbName, logger)
@@ -96,20 +110,28 @@ func loadConfigs() config {
log.Fatalf("Invalid value passed for %s\n", envClientTLS)
}
timeout, err := strconv.ParseInt(mainflux.Env(envThingsTimeout, defThingsTimeout), 10, 64)
if err != nil {
log.Fatalf("Invalid %s value: %s", envThingsTimeout, err.Error())
}
return config{
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
dbName: mainflux.Env(envDBName, defDBName),
dbHost: mainflux.Env(envDBHost, defDBHost),
dbPort: mainflux.Env(envDBPort, defDBPort),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
dbName: mainflux.Env(envDBName, defDBName),
dbHost: mainflux.Env(envDBHost, defDBHost),
dbPort: mainflux.Env(envDBPort, defDBPort),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
jaegerURL: mainflux.Env(envJaegerURL, defJaegerURL),
thingsTimeout: time.Duration(timeout) * time.Second,
}
}
func connectToMongoDB(host, port, name string, logger logger.Logger) *mongo.Database {
client, err := mongo.Connect(context.Background(), fmt.Sprintf("mongodb://%s:%s", host, port), nil)
addr := fmt.Sprintf("mongodb://%s:%s", host, port)
client, err := mongo.Connect(context.Background(), options.Client().ApplyURI(addr))
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to database: %s", err))
os.Exit(1)
@@ -118,6 +140,26 @@ func connectToMongoDB(host, port, name string, logger logger.Logger) *mongo.Data
return client.Database(name)
}
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger client: %s", err))
os.Exit(1)
}
return tracer, closer
}
func connectToThings(cfg config, logger logger.Logger) *grpc.ClientConn {
var opts []grpc.DialOption
if cfg.clientTLS {
+77 -38
View File
@@ -10,76 +10,86 @@ package main
import (
"context"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"github.com/BurntSushi/toml"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/logger"
"github.com/mainflux/mainflux/writers"
"github.com/mainflux/mainflux/writers/api"
"github.com/mainflux/mainflux/writers/mongodb"
"github.com/mongodb/mongo-go-driver/mongo"
"github.com/nats-io/go-nats"
nats "github.com/nats-io/go-nats"
stdprometheus "github.com/prometheus/client_golang/prometheus"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
const (
queue = "mongodb-writer"
svcName = "mongodb-writer"
defNatsURL = nats.DefaultURL
defLogLevel = "error"
defPort = "8180"
defDBName = "mainflux"
defDBHost = "localhost"
defDBPort = "27017"
defNatsURL = nats.DefaultURL
defLogLevel = "error"
defPort = "8180"
defDBName = "mainflux"
defDBHost = "localhost"
defDBPort = "27017"
defChanCfgPath = "/config/channels.toml"
envNatsURL = "MF_NATS_URL"
envLogLevel = "MF_MONGO_WRITER_LOG_LEVEL"
envPort = "MF_MONGO_WRITER_PORT"
envDBName = "MF_MONGO_WRITER_DB_NAME"
envDBHost = "MF_MONGO_WRITER_DB_HOST"
envDBPort = "MF_MONGO_WRITER_DB_PORT"
envNatsURL = "MF_NATS_URL"
envLogLevel = "MF_MONGO_WRITER_LOG_LEVEL"
envPort = "MF_MONGO_WRITER_PORT"
envDBName = "MF_MONGO_WRITER_DB_NAME"
envDBHost = "MF_MONGO_WRITER_DB_HOST"
envDBPort = "MF_MONGO_WRITER_DB_PORT"
envChanCfgPath = "MF_MONGO_WRITER_CHANNELS_CONFIG"
)
type config struct {
NatsURL string
LogLevel string
Port string
DBName string
DBHost string
DBPort string
natsURL string
logLevel string
port string
dbName string
dbHost string
dbPort string
channels map[string]bool
}
func main() {
cfg := loadConfigs()
logger, err := logger.New(os.Stdout, cfg.LogLevel)
logger, err := logger.New(os.Stdout, cfg.logLevel)
if err != nil {
log.Fatalf(err.Error())
log.Fatal(err)
}
nc, err := nats.Connect(cfg.NatsURL)
nc, err := nats.Connect(cfg.natsURL)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to NATS: %s", err))
os.Exit(1)
}
defer nc.Close()
client, err := mongo.Connect(context.Background(), fmt.Sprintf("mongodb://%s:%s", cfg.DBHost, cfg.DBPort), nil)
addr := fmt.Sprintf("mongodb://%s:%s", cfg.dbHost, cfg.dbPort)
client, err := mongo.Connect(context.Background(), options.Client().ApplyURI(addr))
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to database: %s", err))
os.Exit(1)
}
db := client.Database(cfg.DBName)
db := client.Database(cfg.dbName)
repo := mongodb.New(db)
counter, latency := makeMetrics()
repo = writers.LoggingMiddleware(repo, logger)
repo = writers.MetricsMiddleware(repo, counter, latency)
if err := writers.Start(nc, repo, queue, logger); err != nil {
logger.Error(fmt.Sprintf("Failed to start message writer: %s", err))
repo = api.LoggingMiddleware(repo, logger)
repo = api.MetricsMiddleware(repo, counter, latency)
if err := writers.Start(nc, repo, svcName, cfg.channels, logger); err != nil {
logger.Error(fmt.Sprintf("Failed to start MongoDB writer: %s", err))
os.Exit(1)
}
@@ -90,23 +100,52 @@ func main() {
errs <- fmt.Errorf("%s", <-c)
}()
go startHTTPService(cfg.Port, logger, errs)
go startHTTPService(cfg.port, logger, errs)
err = <-errs
logger.Error(fmt.Sprintf("MongoDB writer service terminated: %s", err))
}
func loadConfigs() config {
chanCfgPath := mainflux.Env(envChanCfgPath, defChanCfgPath)
return config{
NatsURL: mainflux.Env(envNatsURL, defNatsURL),
LogLevel: mainflux.Env(envLogLevel, defLogLevel),
Port: mainflux.Env(envPort, defPort),
DBName: mainflux.Env(envDBName, defDBName),
DBHost: mainflux.Env(envDBHost, defDBHost),
DBPort: mainflux.Env(envDBPort, defDBPort),
natsURL: mainflux.Env(envNatsURL, defNatsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
dbName: mainflux.Env(envDBName, defDBName),
dbHost: mainflux.Env(envDBHost, defDBHost),
dbPort: mainflux.Env(envDBPort, defDBPort),
channels: loadChansConfig(chanCfgPath),
}
}
type channels struct {
List []string `toml:"filter"`
}
type chanConfig struct {
Channels channels `toml:"channels"`
}
func loadChansConfig(chanConfigPath string) map[string]bool {
data, err := ioutil.ReadFile(chanConfigPath)
if err != nil {
log.Fatal(err)
}
var chanCfg chanConfig
if err := toml.Unmarshal(data, &chanCfg); err != nil {
log.Fatal(err)
}
chans := map[string]bool{}
for _, ch := range chanCfg.Channels.List {
chans[ch] = true
}
return chans
}
func makeMetrics() (*kitprometheus.Counter, *kitprometheus.Summary) {
counter := kitprometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: "mongodb",
@@ -128,5 +167,5 @@ func makeMetrics() (*kitprometheus.Counter, *kitprometheus.Summary) {
func startHTTPService(port string, logger logger.Logger, errs chan error) {
p := fmt.Sprintf(":%s", port)
logger.Info(fmt.Sprintf("Mongodb writer service started, exposed port %s", p))
errs <- http.ListenAndServe(p, mongodb.MakeHandler())
errs <- http.ListenAndServe(p, api.MakeHandler(svcName))
}
+228
View File
@@ -0,0 +1,228 @@
//
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
//
package main
import (
"fmt"
"io"
"log"
"net/http"
"os"
"os/signal"
"strconv"
"syscall"
"time"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
"github.com/jmoiron/sqlx"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/logger"
"github.com/mainflux/mainflux/readers"
"github.com/mainflux/mainflux/readers/api"
"github.com/mainflux/mainflux/readers/postgres"
thingsapi "github.com/mainflux/mainflux/things/api/auth/grpc"
opentracing "github.com/opentracing/opentracing-go"
stdprometheus "github.com/prometheus/client_golang/prometheus"
jconfig "github.com/uber/jaeger-client-go/config"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
)
const (
svcName = "postgres-writer"
sep = ","
defThingsURL = "localhost:8183"
defLogLevel = "debug"
defPort = "9204"
defClientTLS = "false"
defCACerts = ""
defDBHost = "localhost"
defDBPort = "5432"
defDBUser = "mainflux"
defDBPass = "mainflux"
defDBName = "messages"
defDBSSLMode = "disable"
defDBSSLCert = ""
defDBSSLKey = ""
defDBSSLRootCert = ""
defJaegerURL = "localhost:6831"
defThingsTimeout = "1" // in seconds
envThingsURL = "MF_THINGS_URL"
envLogLevel = "MF_POSTGRES_READER_LOG_LEVEL"
envPort = "MF_POSTGRES_READER_PORT"
envClientTLS = "MF_POSTGRES_READER_CLIENT_TLS"
envCACerts = "MF_POSTGRES_READER_CA_CERTS"
envDBHost = "MF_POSTGRES_READER_DB_HOST"
envDBPort = "MF_POSTGRES_READER_DB_PORT"
envDBUser = "MF_POSTGRES_READER_DB_USER"
envDBPass = "MF_POSTGRES_READER_DB_PASS"
envDBName = "MF_POSTGRES_READER_DB_NAME"
envDBSSLMode = "MF_POSTGRES_READER_DB_SSL_MODE"
envDBSSLCert = "MF_POSTGRES_READER_DB_SSL_CERT"
envDBSSLKey = "MF_POSTGRES_READER_DB_SSL_KEY"
envDBSSLRootCert = "MF_POSTGRES_READER_DB_SSL_ROOT_CERT"
envJaegerURL = "MF_JAEGER_URL"
envThingsTimeout = "MF_POSTGRES_READER_THINGS_TIMEOUT"
)
type config struct {
thingsURL string
logLevel string
port string
clientTLS bool
caCerts string
dbConfig postgres.Config
jaegerURL string
thingsTimeout time.Duration
}
func main() {
cfg := loadConfig()
logger, err := logger.New(os.Stdout, cfg.logLevel)
if err != nil {
log.Fatalf(err.Error())
}
conn := connectToThings(cfg, logger)
defer conn.Close()
thingsTracer, thingsCloser := initJaeger("things", cfg.jaegerURL, logger)
defer thingsCloser.Close()
tc := thingsapi.NewClient(conn, thingsTracer, cfg.thingsTimeout)
db := connectToDB(cfg.dbConfig, logger)
defer db.Close()
repo := newService(db, logger)
errs := make(chan error, 2)
go startHTTPServer(repo, tc, cfg.port, logger, errs)
go func() {
c := make(chan os.Signal)
signal.Notify(c, syscall.SIGINT)
errs <- fmt.Errorf("%s", <-c)
}()
err = <-errs
logger.Error(fmt.Sprintf("Postgres writer service terminated: %s", err))
}
func loadConfig() config {
dbConfig := postgres.Config{
Host: mainflux.Env(envDBHost, defDBHost),
Port: mainflux.Env(envDBPort, defDBPort),
User: mainflux.Env(envDBUser, defDBUser),
Pass: mainflux.Env(envDBPass, defDBPass),
Name: mainflux.Env(envDBName, defDBName),
SSLMode: mainflux.Env(envDBSSLMode, defDBSSLMode),
SSLCert: mainflux.Env(envDBSSLCert, defDBSSLCert),
SSLKey: mainflux.Env(envDBSSLKey, defDBSSLKey),
SSLRootCert: mainflux.Env(envDBSSLRootCert, defDBSSLRootCert),
}
timeout, err := strconv.ParseInt(mainflux.Env(envThingsTimeout, defThingsTimeout), 10, 64)
if err != nil {
log.Fatalf("Invalid %s value: %s", envThingsTimeout, err.Error())
}
return config{
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
dbConfig: dbConfig,
jaegerURL: mainflux.Env(envJaegerURL, defJaegerURL),
thingsTimeout: time.Duration(timeout) * time.Second,
}
}
func connectToDB(dbConfig postgres.Config, logger logger.Logger) *sqlx.DB {
db, err := postgres.Connect(dbConfig)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to Postgres: %s", err))
os.Exit(1)
}
return db
}
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger client: %s", err))
os.Exit(1)
}
return tracer, closer
}
func connectToThings(cfg config, logger logger.Logger) *grpc.ClientConn {
var opts []grpc.DialOption
if cfg.clientTLS {
if cfg.caCerts != "" {
tpc, err := credentials.NewClientTLSFromFile(cfg.caCerts, "")
if err != nil {
logger.Error(fmt.Sprintf("Failed to load certs: %s", err))
os.Exit(1)
}
opts = append(opts, grpc.WithTransportCredentials(tpc))
}
} else {
logger.Info("gRPC communication is not encrypted")
opts = append(opts, grpc.WithInsecure())
}
conn, err := grpc.Dial(cfg.thingsURL, opts...)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to things service: %s", err))
os.Exit(1)
}
return conn
}
func newService(db *sqlx.DB, logger logger.Logger) readers.MessageRepository {
svc := postgres.New(db)
svc = api.LoggingMiddleware(svc, logger)
svc = api.MetricsMiddleware(
svc,
kitprometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: "postgres",
Subsystem: "message_writer",
Name: "request_count",
Help: "Number of requests received.",
}, []string{"method"}),
kitprometheus.NewSummaryFrom(stdprometheus.SummaryOpts{
Namespace: "postgres",
Subsystem: "message_writer",
Name: "request_latency_microseconds",
Help: "Total duration of requests in microseconds.",
}, []string{"method"}),
)
return svc
}
func startHTTPServer(repo readers.MessageRepository, tc mainflux.ThingsServiceClient, port string, logger logger.Logger, errs chan error) {
p := fmt.Sprintf(":%s", port)
logger.Info(fmt.Sprintf("Postgres reader service started, exposed port %s", port))
errs <- http.ListenAndServe(p, api.MakeHandler(repo, tc, svcName))
}
+200
View File
@@ -0,0 +1,200 @@
//
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
//
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"github.com/BurntSushi/toml"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
"github.com/jmoiron/sqlx"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/logger"
"github.com/mainflux/mainflux/writers"
"github.com/mainflux/mainflux/writers/api"
"github.com/mainflux/mainflux/writers/postgres"
nats "github.com/nats-io/go-nats"
stdprometheus "github.com/prometheus/client_golang/prometheus"
)
const (
svcName = "postgres-writer"
sep = ","
defNatsURL = nats.DefaultURL
defLogLevel = "error"
defPort = "9104"
defDBHost = "postgres"
defDBPort = "5432"
defDBUser = "mainflux"
defDBPass = "mainflux"
defDBName = "messages"
defDBSSLMode = "disable"
defDBSSLCert = ""
defDBSSLKey = ""
defDBSSLRootCert = ""
defChanCfgPath = "/config/channels.toml"
envNatsURL = "MF_NATS_URL"
envLogLevel = "MF_POSTGRES_WRITER_LOG_LEVEL"
envPort = "MF_POSTGRES_WRITER_PORT"
envDBHost = "MF_POSTGRES_WRITER_DB_HOST"
envDBPort = "MF_POSTGRES_WRITER_DB_PORT"
envDBUser = "MF_POSTGRES_WRITER_DB_USER"
envDBPass = "MF_POSTGRES_WRITER_DB_PASS"
envDBName = "MF_POSTGRES_WRITER_DB_NAME"
envDBSSLMode = "MF_POSTGRES_WRITER_DB_SSL_MODE"
envDBSSLCert = "MF_POSTGRES_WRITER_DB_SSL_CERT"
envDBSSLKey = "MF_POSTGRES_WRITER_DB_SSL_KEY"
envDBSSLRootCert = "MF_POSTGRES_WRITER_DB_SSL_ROOT_CERT"
envChanCfgPath = "MF_POSTGRES_WRITER_CHANNELS_CONFIG"
)
type config struct {
natsURL string
logLevel string
port string
dbConfig postgres.Config
channels map[string]bool
}
func main() {
cfg := loadConfig()
logger, err := logger.New(os.Stdout, cfg.logLevel)
if err != nil {
log.Fatalf(err.Error())
}
nc := connectToNATS(cfg.natsURL, logger)
defer nc.Close()
db := connectToDB(cfg.dbConfig, logger)
defer db.Close()
repo := newService(db, logger)
if err = writers.Start(nc, repo, svcName, cfg.channels, logger); err != nil {
logger.Error(fmt.Sprintf("Failed to create Postgres writer: %s", err))
}
errs := make(chan error, 2)
go startHTTPServer(cfg.port, errs, logger)
go func() {
c := make(chan os.Signal)
signal.Notify(c, syscall.SIGINT)
errs <- fmt.Errorf("%s", <-c)
}()
err = <-errs
logger.Error(fmt.Sprintf("Postgres writer service terminated: %s", err))
}
func loadConfig() config {
chanCfgPath := mainflux.Env(envChanCfgPath, defChanCfgPath)
dbConfig := postgres.Config{
Host: mainflux.Env(envDBHost, defDBHost),
Port: mainflux.Env(envDBPort, defDBPort),
User: mainflux.Env(envDBUser, defDBUser),
Pass: mainflux.Env(envDBPass, defDBPass),
Name: mainflux.Env(envDBName, defDBName),
SSLMode: mainflux.Env(envDBSSLMode, defDBSSLMode),
SSLCert: mainflux.Env(envDBSSLCert, defDBSSLCert),
SSLKey: mainflux.Env(envDBSSLKey, defDBSSLKey),
SSLRootCert: mainflux.Env(envDBSSLRootCert, defDBSSLRootCert),
}
return config{
natsURL: mainflux.Env(envNatsURL, defNatsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
dbConfig: dbConfig,
channels: loadChansConfig(chanCfgPath),
}
}
type channels struct {
List []string `toml:"filter"`
}
type chanConfig struct {
Channels channels `toml:"channels"`
}
func loadChansConfig(chanConfigPath string) map[string]bool {
data, err := ioutil.ReadFile(chanConfigPath)
if err != nil {
log.Fatal(err)
}
var chanCfg chanConfig
if err := toml.Unmarshal(data, &chanCfg); err != nil {
log.Fatal(err)
}
chans := map[string]bool{}
for _, ch := range chanCfg.Channels.List {
chans[ch] = true
}
return chans
}
func connectToNATS(url string, logger logger.Logger) *nats.Conn {
nc, err := nats.Connect(url)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to NATS: %s", err))
os.Exit(1)
}
return nc
}
func connectToDB(dbConfig postgres.Config, logger logger.Logger) *sqlx.DB {
db, err := postgres.Connect(dbConfig)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to Postgres: %s", err))
os.Exit(1)
}
return db
}
func newService(db *sqlx.DB, logger logger.Logger) writers.MessageRepository {
svc := postgres.New(db)
svc = api.LoggingMiddleware(svc, logger)
svc = api.MetricsMiddleware(
svc,
kitprometheus.NewCounterFrom(stdprometheus.CounterOpts{
Namespace: "postgres",
Subsystem: "message_writer",
Name: "request_count",
Help: "Number of requests received.",
}, []string{"method"}),
kitprometheus.NewSummaryFrom(stdprometheus.SummaryOpts{
Namespace: "postgres",
Subsystem: "message_writer",
Name: "request_latency_microseconds",
Help: "Total duration of requests in microseconds.",
}, []string{"method"}),
)
return svc
}
func startHTTPServer(port string, errs chan error, logger logger.Logger) {
p := fmt.Sprintf(":%s", port)
logger.Info(fmt.Sprintf("Postgres writer service started, exposed port %s", port))
errs <- http.ListenAndServe(p, api.MakeHandler(svcName))
}
+186 -101
View File
@@ -1,5 +1,5 @@
//
// Copyright (c) 2018
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
@@ -8,8 +8,8 @@
package main
import (
"database/sql"
"fmt"
"io"
"log"
"net"
"net/http"
@@ -17,7 +17,12 @@ import (
"os/signal"
"strconv"
"syscall"
"time"
"github.com/mainflux/mainflux/things/tracing"
"github.com/jmoiron/sqlx"
opentracing "github.com/opentracing/opentracing-go"
"google.golang.org/grpc/credentials"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
@@ -26,81 +31,100 @@ import (
"github.com/mainflux/mainflux/logger"
"github.com/mainflux/mainflux/things"
"github.com/mainflux/mainflux/things/api"
grpcapi "github.com/mainflux/mainflux/things/api/grpc"
httpapi "github.com/mainflux/mainflux/things/api/http"
authgrpcapi "github.com/mainflux/mainflux/things/api/auth/grpc"
authhttpapi "github.com/mainflux/mainflux/things/api/auth/http"
thhttpapi "github.com/mainflux/mainflux/things/api/things/http"
"github.com/mainflux/mainflux/things/postgres"
rediscache "github.com/mainflux/mainflux/things/redis"
localusers "github.com/mainflux/mainflux/things/users"
"github.com/mainflux/mainflux/things/uuid"
usersapi "github.com/mainflux/mainflux/users/api/grpc"
stdprometheus "github.com/prometheus/client_golang/prometheus"
jconfig "github.com/uber/jaeger-client-go/config"
"google.golang.org/grpc"
)
const (
defLogLevel = "error"
defDBHost = "localhost"
defDBPort = "5432"
defDBUser = "mainflux"
defDBPass = "mainflux"
defDBName = "things"
defDBSSLMode = "disable"
defDBSSLCert = ""
defDBSSLKey = ""
defDBSSLRootCert = ""
defClientTLS = "false"
defCACerts = ""
defCacheURL = "localhost:6379"
defCachePass = ""
defCacheDB = "0"
defESURL = "localhost:6379"
defESPass = ""
defESDB = "0"
defHTTPPort = "8180"
defGRPCPort = "8181"
defServerCert = ""
defServerKey = ""
defUsersURL = "localhost:8181"
envLogLevel = "MF_THINGS_LOG_LEVEL"
envDBHost = "MF_THINGS_DB_HOST"
envDBPort = "MF_THINGS_DB_PORT"
envDBUser = "MF_THINGS_DB_USER"
envDBPass = "MF_THINGS_DB_PASS"
envDBName = "MF_THINGS_DB"
envDBSSLMode = "MF_THINGS_DB_SSL_MODE"
envDBSSLCert = "MF_THINGS_DB_SSL_CERT"
envDBSSLKey = "MF_THINGS_DB_SSL_KEY"
envDBSSLRootCert = "MF_THINGS_DB_SSL_ROOT_CERT"
envClientTLS = "MF_THINGS_CLIENT_TLS"
envCACerts = "MF_THINGS_CA_CERTS"
envCacheURL = "MF_THINGS_CACHE_URL"
envCachePass = "MF_THINGS_CACHE_PASS"
envCacheDB = "MF_THINGS_CACHE_DB"
envESURL = "MF_THINGS_ES_URL"
envESPass = "MF_THINGS_ES_PASS"
envESDB = "MF_THINGS_ES_DB"
envHTTPPort = "MF_THINGS_HTTP_PORT"
envGRPCPort = "MF_THINGS_GRPC_PORT"
envUsersURL = "MF_USERS_URL"
envServerCert = "MF_THINGS_SERVER_CERT"
envServerKey = "MF_THINGS_SERVER_KEY"
defLogLevel = "error"
defDBHost = "localhost"
defDBPort = "5432"
defDBUser = "mainflux"
defDBPass = "mainflux"
defDBName = "things"
defDBSSLMode = "disable"
defDBSSLCert = ""
defDBSSLKey = ""
defDBSSLRootCert = ""
defClientTLS = "false"
defCACerts = ""
defCacheURL = "localhost:6379"
defCachePass = ""
defCacheDB = "0"
defESURL = "localhost:6379"
defESPass = ""
defESDB = "0"
defHTTPPort = "8180"
defAuthHTTPPort = "8989"
defAuthGRPCPort = "8181"
defServerCert = ""
defServerKey = ""
defUsersURL = "localhost:8181"
defSingleUserEmail = ""
defSingleUserToken = ""
defJaegerURL = "localhost:6831"
defUsersTimeout = "1" // in seconds
envLogLevel = "MF_THINGS_LOG_LEVEL"
envDBHost = "MF_THINGS_DB_HOST"
envDBPort = "MF_THINGS_DB_PORT"
envDBUser = "MF_THINGS_DB_USER"
envDBPass = "MF_THINGS_DB_PASS"
envDBName = "MF_THINGS_DB"
envDBSSLMode = "MF_THINGS_DB_SSL_MODE"
envDBSSLCert = "MF_THINGS_DB_SSL_CERT"
envDBSSLKey = "MF_THINGS_DB_SSL_KEY"
envDBSSLRootCert = "MF_THINGS_DB_SSL_ROOT_CERT"
envClientTLS = "MF_THINGS_CLIENT_TLS"
envCACerts = "MF_THINGS_CA_CERTS"
envCacheURL = "MF_THINGS_CACHE_URL"
envCachePass = "MF_THINGS_CACHE_PASS"
envCacheDB = "MF_THINGS_CACHE_DB"
envESURL = "MF_THINGS_ES_URL"
envESPass = "MF_THINGS_ES_PASS"
envESDB = "MF_THINGS_ES_DB"
envHTTPPort = "MF_THINGS_HTTP_PORT"
envAuthHTTPPort = "MF_THINGS_AUTH_HTTP_PORT"
envAuthGRPCPort = "MF_THINGS_AUTH_GRPC_PORT"
envUsersURL = "MF_USERS_URL"
envServerCert = "MF_THINGS_SERVER_CERT"
envServerKey = "MF_THINGS_SERVER_KEY"
envSingleUserEmail = "MF_THINGS_SINGLE_USER_EMAIL"
envSingleUserToken = "MF_THINGS_SINGLE_USER_TOKEN"
envJaegerURL = "MF_JAEGER_URL"
envUsersTimeout = "MF_THINGS_USERS_TIMEOUT"
)
type config struct {
logLevel string
dbConfig postgres.Config
clientTLS bool
caCerts string
cacheURL string
cachePass string
cacheDB string
esURL string
esPass string
esDB string
httpPort string
grpcPort string
usersURL string
serverCert string
serverKey string
logLevel string
dbConfig postgres.Config
clientTLS bool
caCerts string
cacheURL string
cachePass string
cacheDB string
esURL string
esPass string
esDB string
httpPort string
authHTTPPort string
authGRPCPort string
usersURL string
serverCert string
serverKey string
singleUserEmail string
singleUserToken string
jaegerURL string
usersTimeout time.Duration
}
func main() {
@@ -110,6 +134,10 @@ func main() {
if err != nil {
log.Fatalf(err.Error())
}
thingsTracer, thingsCloser := initJaeger("things", cfg.jaegerURL, logger)
defer thingsCloser.Close()
cacheClient := connectToRedis(cfg.cacheURL, cfg.cachePass, cfg.cacheDB, logger)
esClient := connectToRedis(cfg.esURL, cfg.esPass, cfg.esDB, logger)
@@ -117,14 +145,26 @@ func main() {
db := connectToDB(cfg.dbConfig, logger)
defer db.Close()
conn := connectToUsers(cfg, logger)
defer conn.Close()
usersTracer, usersCloser := initJaeger("users", cfg.jaegerURL, logger)
defer usersCloser.Close()
svc := newService(conn, db, cacheClient, esClient, logger)
users, close := createUsersClient(cfg, usersTracer, logger)
if close != nil {
defer close()
}
dbTracer, dbCloser := initJaeger("things_db", cfg.jaegerURL, logger)
defer dbCloser.Close()
cacheTracer, cacheCloser := initJaeger("things_cache", cfg.jaegerURL, logger)
defer cacheCloser.Close()
svc := newService(users, dbTracer, cacheTracer, db, cacheClient, esClient, logger)
errs := make(chan error, 2)
go startHTTPServer(svc, cfg, logger, errs)
go startGRPCServer(svc, cfg, logger, errs)
go startHTTPServer(thhttpapi.MakeHandler(thingsTracer, svc), cfg.httpPort, cfg, logger, errs)
go startHTTPServer(authhttpapi.MakeHandler(thingsTracer, svc), cfg.authHTTPPort, cfg, logger, errs)
go startGRPCServer(svc, thingsTracer, cfg, logger, errs)
go func() {
c := make(chan os.Signal)
@@ -142,6 +182,11 @@ func loadConfig() config {
log.Fatalf("Invalid value passed for %s\n", envClientTLS)
}
timeout, err := strconv.ParseInt(mainflux.Env(envUsersTimeout, defUsersTimeout), 10, 64)
if err != nil {
log.Fatalf("Invalid %s value: %s", envUsersTimeout, err.Error())
}
dbConfig := postgres.Config{
Host: mainflux.Env(envDBHost, defDBHost),
Port: mainflux.Env(envDBPort, defDBPort),
@@ -155,24 +200,49 @@ func loadConfig() config {
}
return config{
logLevel: mainflux.Env(envLogLevel, defLogLevel),
dbConfig: dbConfig,
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
cacheURL: mainflux.Env(envCacheURL, defCacheURL),
cachePass: mainflux.Env(envCachePass, defCachePass),
cacheDB: mainflux.Env(envCacheDB, defCacheDB),
esURL: mainflux.Env(envESURL, defESURL),
esPass: mainflux.Env(envESPass, defESPass),
esDB: mainflux.Env(envESDB, defESDB),
httpPort: mainflux.Env(envHTTPPort, defHTTPPort),
grpcPort: mainflux.Env(envGRPCPort, defGRPCPort),
usersURL: mainflux.Env(envUsersURL, defUsersURL),
serverCert: mainflux.Env(envServerCert, defServerCert),
serverKey: mainflux.Env(envServerKey, defServerKey),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
dbConfig: dbConfig,
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
cacheURL: mainflux.Env(envCacheURL, defCacheURL),
cachePass: mainflux.Env(envCachePass, defCachePass),
cacheDB: mainflux.Env(envCacheDB, defCacheDB),
esURL: mainflux.Env(envESURL, defESURL),
esPass: mainflux.Env(envESPass, defESPass),
esDB: mainflux.Env(envESDB, defESDB),
httpPort: mainflux.Env(envHTTPPort, defHTTPPort),
authHTTPPort: mainflux.Env(envAuthHTTPPort, defAuthHTTPPort),
authGRPCPort: mainflux.Env(envAuthGRPCPort, defAuthGRPCPort),
usersURL: mainflux.Env(envUsersURL, defUsersURL),
serverCert: mainflux.Env(envServerCert, defServerCert),
serverKey: mainflux.Env(envServerKey, defServerKey),
singleUserEmail: mainflux.Env(envSingleUserEmail, defSingleUserEmail),
singleUserToken: mainflux.Env(envSingleUserToken, defSingleUserToken),
jaegerURL: mainflux.Env(envJaegerURL, defJaegerURL),
usersTimeout: time.Duration(timeout) * time.Second,
}
}
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger client: %s", err))
os.Exit(1)
}
return tracer, closer
}
func connectToRedis(cacheURL, cachePass string, cacheDB string, logger logger.Logger) *redis.Client {
db, err := strconv.Atoi(cacheDB)
if err != nil {
@@ -187,7 +257,7 @@ func connectToRedis(cacheURL, cachePass string, cacheDB string, logger logger.Lo
})
}
func connectToDB(dbConfig postgres.Config, logger logger.Logger) *sql.DB {
func connectToDB(dbConfig postgres.Config, logger logger.Logger) *sqlx.DB {
db, err := postgres.Connect(dbConfig)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to postgres: %s", err))
@@ -196,6 +266,15 @@ func connectToDB(dbConfig postgres.Config, logger logger.Logger) *sql.DB {
return db
}
func createUsersClient(cfg config, tracer opentracing.Tracer, logger logger.Logger) (mainflux.UsersServiceClient, func() error) {
if cfg.singleUserEmail != "" && cfg.singleUserToken != "" {
return localusers.NewSingleUserService(cfg.singleUserEmail, cfg.singleUserToken), nil
}
conn := connectToUsers(cfg, logger)
return usersapi.NewClient(tracer, conn, cfg.usersTimeout), conn.Close
}
func connectToUsers(cfg config, logger logger.Logger) *grpc.ClientConn {
var opts []grpc.DialOption
if cfg.clientTLS {
@@ -221,12 +300,18 @@ func connectToUsers(cfg config, logger logger.Logger) *grpc.ClientConn {
return conn
}
func newService(conn *grpc.ClientConn, db *sql.DB, cacheClient *redis.Client, esClient *redis.Client, logger logger.Logger) things.Service {
users := usersapi.NewClient(conn)
thingsRepo := postgres.NewThingRepository(db, logger)
channelsRepo := postgres.NewChannelRepository(db, logger)
func newService(users mainflux.UsersServiceClient, dbTracer opentracing.Tracer, cacheTracer opentracing.Tracer, db *sqlx.DB, cacheClient *redis.Client, esClient *redis.Client, logger logger.Logger) things.Service {
thingsRepo := postgres.NewThingRepository(db)
thingsRepo = tracing.ThingRepositoryMiddleware(dbTracer, thingsRepo)
channelsRepo := postgres.NewChannelRepository(db)
channelsRepo = tracing.ChannelRepositoryMiddleware(dbTracer, channelsRepo)
chanCache := rediscache.NewChannelCache(cacheClient)
chanCache = tracing.ChannelCacheMiddleware(cacheTracer, chanCache)
thingCache := rediscache.NewThingCache(cacheClient)
thingCache = tracing.ThingCacheMiddleware(cacheTracer, thingCache)
idp := uuid.New()
svc := things.New(users, thingsRepo, channelsRepo, chanCache, thingCache, idp)
@@ -250,23 +335,23 @@ func newService(conn *grpc.ClientConn, db *sql.DB, cacheClient *redis.Client, es
return svc
}
func startHTTPServer(svc things.Service, cfg config, logger logger.Logger, errs chan error) {
p := fmt.Sprintf(":%s", cfg.httpPort)
func startHTTPServer(handler http.Handler, port string, cfg config, logger logger.Logger, errs chan error) {
p := fmt.Sprintf(":%s", port)
if cfg.serverCert != "" || cfg.serverKey != "" {
logger.Info(fmt.Sprintf("Things service started using https on port %s with cert %s key %s",
cfg.httpPort, cfg.serverCert, cfg.serverKey))
errs <- http.ListenAndServeTLS(p, cfg.serverCert, cfg.serverKey, httpapi.MakeHandler(svc))
port, cfg.serverCert, cfg.serverKey))
errs <- http.ListenAndServeTLS(p, cfg.serverCert, cfg.serverKey, handler)
return
}
logger.Info(fmt.Sprintf("Things service started using http on port %s", cfg.httpPort))
errs <- http.ListenAndServe(p, httpapi.MakeHandler(svc))
errs <- http.ListenAndServe(p, handler)
}
func startGRPCServer(svc things.Service, cfg config, logger logger.Logger, errs chan error) {
p := fmt.Sprintf(":%s", cfg.grpcPort)
func startGRPCServer(svc things.Service, tracer opentracing.Tracer, cfg config, logger logger.Logger, errs chan error) {
p := fmt.Sprintf(":%s", cfg.authGRPCPort)
listener, err := net.Listen("tcp", p)
if err != nil {
logger.Error(fmt.Sprintf("Failed to listen on port %s: %s", cfg.grpcPort, err))
logger.Error(fmt.Sprintf("Failed to listen on port %s: %s", cfg.authGRPCPort, err))
os.Exit(1)
}
@@ -278,13 +363,13 @@ func startGRPCServer(svc things.Service, cfg config, logger logger.Logger, errs
os.Exit(1)
}
logger.Info(fmt.Sprintf("Things gRPC service started using https on port %s with cert %s key %s",
cfg.grpcPort, cfg.serverCert, cfg.serverKey))
cfg.authGRPCPort, cfg.serverCert, cfg.serverKey))
server = grpc.NewServer(grpc.Creds(creds))
} else {
logger.Info(fmt.Sprintf("Things gRPC service started using http on port %s", cfg.grpcPort))
logger.Info(fmt.Sprintf("Things gRPC service started using http on port %s", cfg.authGRPCPort))
server = grpc.NewServer()
}
mainflux.RegisterThingsServiceServer(server, grpcapi.NewServer(svc))
mainflux.RegisterThingsServiceServer(server, authgrpcapi.NewServer(tracer, svc))
errs <- server.Serve(listener)
}
+50 -14
View File
@@ -1,5 +1,5 @@
//
// Copyright (c) 2018
// Copyright (c) 2019
// Mainflux
//
// SPDX-License-Identifier: Apache-2.0
@@ -8,8 +8,8 @@
package main
import (
"database/sql"
"fmt"
"io"
"log"
"net"
"net/http"
@@ -17,9 +17,12 @@ import (
"os/signal"
"syscall"
"github.com/mainflux/mainflux/users/tracing"
"google.golang.org/grpc/credentials"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
"github.com/jmoiron/sqlx"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/logger"
"github.com/mainflux/mainflux/users"
@@ -29,7 +32,9 @@ import (
"github.com/mainflux/mainflux/users/bcrypt"
"github.com/mainflux/mainflux/users/jwt"
"github.com/mainflux/mainflux/users/postgres"
opentracing "github.com/opentracing/opentracing-go"
stdprometheus "github.com/prometheus/client_golang/prometheus"
jconfig "github.com/uber/jaeger-client-go/config"
"google.golang.org/grpc"
)
@@ -49,6 +54,8 @@ const (
defSecret = "users"
defServerCert = ""
defServerKey = ""
defJaegerURL = "localhost:6831"
envLogLevel = "MF_USERS_LOG_LEVEL"
envDBHost = "MF_USERS_DB_HOST"
envDBPort = "MF_USERS_DB_PORT"
@@ -64,6 +71,7 @@ const (
envSecret = "MF_USERS_SECRET"
envServerCert = "MF_USERS_SERVER_CERT"
envServerKey = "MF_USERS_SERVER_KEY"
envJaegerURL = "MF_JAEGER_URL"
)
type config struct {
@@ -74,6 +82,7 @@ type config struct {
secret string
serverCert string
serverKey string
jaegerURL string
}
func main() {
@@ -83,14 +92,21 @@ func main() {
if err != nil {
log.Fatalf(err.Error())
}
db := connectToDB(cfg.dbConfig, logger)
defer db.Close()
svc := newService(db, cfg.secret, logger)
tracer, closer := initJaeger("users", cfg.jaegerURL, logger)
defer closer.Close()
dbTracer, dbCloser := initJaeger("users_db", cfg.jaegerURL, logger)
defer dbCloser.Close()
svc := newService(db, dbTracer, cfg.secret, logger)
errs := make(chan error, 2)
go startHTTPServer(svc, cfg.httpPort, cfg.serverCert, cfg.serverKey, logger, errs)
go startGRPCServer(svc, cfg.grpcPort, cfg.serverCert, cfg.serverKey, logger, errs)
go startHTTPServer(tracer, svc, cfg.httpPort, cfg.serverCert, cfg.serverKey, logger, errs)
go startGRPCServer(tracer, svc, cfg.grpcPort, cfg.serverCert, cfg.serverKey, logger, errs)
go func() {
c := make(chan os.Signal)
@@ -103,7 +119,6 @@ func main() {
}
func loadConfig() config {
dbConfig := postgres.Config{
Host: mainflux.Env(envDBHost, defDBHost),
Port: mainflux.Env(envDBPort, defDBPort),
@@ -124,10 +139,31 @@ func loadConfig() config {
secret: mainflux.Env(envSecret, defSecret),
serverCert: mainflux.Env(envServerCert, defServerCert),
serverKey: mainflux.Env(envServerKey, defServerKey),
jaegerURL: mainflux.Env(envJaegerURL, defJaegerURL),
}
}
func connectToDB(dbConfig postgres.Config, logger logger.Logger) *sql.DB {
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger: %s", err))
os.Exit(1)
}
return tracer, closer
}
func connectToDB(dbConfig postgres.Config, logger logger.Logger) *sqlx.DB {
db, err := postgres.Connect(dbConfig)
if err != nil {
logger.Error(fmt.Sprintf("Failed to connect to postgres: %s", err))
@@ -136,8 +172,8 @@ func connectToDB(dbConfig postgres.Config, logger logger.Logger) *sql.DB {
return db
}
func newService(db *sql.DB, secret string, logger logger.Logger) users.Service {
repo := postgres.New(db)
func newService(db *sqlx.DB, tracer opentracing.Tracer, secret string, logger logger.Logger) users.Service {
repo := tracing.UserRepositoryMiddleware(postgres.New(db), tracer)
hasher := bcrypt.New()
idp := jwt.New(secret)
@@ -161,18 +197,18 @@ func newService(db *sql.DB, secret string, logger logger.Logger) users.Service {
return svc
}
func startHTTPServer(svc users.Service, port string, certFile string, keyFile string, logger logger.Logger, errs chan error) {
func startHTTPServer(tracer opentracing.Tracer, svc users.Service, port string, certFile string, keyFile string, logger logger.Logger, errs chan error) {
p := fmt.Sprintf(":%s", port)
if certFile != "" || keyFile != "" {
logger.Info(fmt.Sprintf("Users service started using https, cert %s key %s, exposed port %s", certFile, keyFile, port))
errs <- http.ListenAndServeTLS(p, certFile, keyFile, httpapi.MakeHandler(svc, logger))
errs <- http.ListenAndServeTLS(p, certFile, keyFile, httpapi.MakeHandler(svc, tracer, logger))
} else {
logger.Info(fmt.Sprintf("Users service started using http, exposed port %s", port))
errs <- http.ListenAndServe(p, httpapi.MakeHandler(svc, logger))
errs <- http.ListenAndServe(p, httpapi.MakeHandler(svc, tracer, logger))
}
}
func startGRPCServer(svc users.Service, port string, certFile string, keyFile string, logger logger.Logger, errs chan error) {
func startGRPCServer(tracer opentracing.Tracer, svc users.Service, port string, certFile string, keyFile string, logger logger.Logger, errs chan error) {
p := fmt.Sprintf(":%s", port)
listener, err := net.Listen("tcp", p)
if err != nil {
@@ -193,7 +229,7 @@ func startGRPCServer(svc users.Service, port string, certFile string, keyFile st
server = grpc.NewServer()
}
mainflux.RegisterUsersServiceServer(server, grpcapi.NewServer(svc))
mainflux.RegisterUsersServiceServer(server, grpcapi.NewServer(tracer, svc))
logger.Info(fmt.Sprintf("Users gRPC service started, exposed port %s", port))
errs <- server.Serve(listener)
}
+67 -26
View File
@@ -9,48 +9,59 @@ package main
import (
"fmt"
"io"
"log"
"net/http"
"os"
"os/signal"
"strconv"
"syscall"
"time"
kitprometheus "github.com/go-kit/kit/metrics/prometheus"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/logger"
thingsapi "github.com/mainflux/mainflux/things/api/grpc"
thingsapi "github.com/mainflux/mainflux/things/api/auth/grpc"
adapter "github.com/mainflux/mainflux/ws"
"github.com/mainflux/mainflux/ws/api"
"github.com/mainflux/mainflux/ws/nats"
broker "github.com/nats-io/go-nats"
opentracing "github.com/opentracing/opentracing-go"
stdprometheus "github.com/prometheus/client_golang/prometheus"
jconfig "github.com/uber/jaeger-client-go/config"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
)
const (
defClientTLS = "false"
defCACerts = ""
defPort = "8180"
defLogLevel = "error"
defNatsURL = broker.DefaultURL
defThingsURL = "localhost:8181"
envClientTLS = "MF_WS_ADAPTER_CLIENT_TLS"
envCACerts = "MF_WS_ADAPTER_CA_CERTS"
envPort = "MF_WS_ADAPTER_PORT"
envLogLevel = "MF_WS_ADAPTER_LOG_LEVEL"
envNatsURL = "MF_NATS_URL"
envThingsURL = "MF_THINGS_URL"
defClientTLS = "false"
defCACerts = ""
defPort = "8180"
defLogLevel = "error"
defNatsURL = broker.DefaultURL
defThingsURL = "localhost:8181"
defJaegerURL = "localhost:6831"
defThingsTimeout = "1" // in seconds
envClientTLS = "MF_WS_ADAPTER_CLIENT_TLS"
envCACerts = "MF_WS_ADAPTER_CA_CERTS"
envPort = "MF_WS_ADAPTER_PORT"
envLogLevel = "MF_WS_ADAPTER_LOG_LEVEL"
envNatsURL = "MF_NATS_URL"
envThingsURL = "MF_THINGS_URL"
envJaegerURL = "MF_JAEGER_URL"
envThingsTimeout = "MF_WS_ADAPTER_THINGS_TIMEOUT"
)
type config struct {
clientTLS bool
caCerts string
thingsURL string
natsURL string
logLevel string
port string
clientTLS bool
caCerts string
thingsURL string
natsURL string
logLevel string
port string
jaegerURL string
thingsTimeout time.Duration
}
func main() {
@@ -71,7 +82,10 @@ func main() {
conn := connectToThings(cfg, logger)
defer conn.Close()
cc := thingsapi.NewClient(conn)
thingsTracer, thingsCloser := initJaeger("things", cfg.jaegerURL, logger)
defer thingsCloser.Close()
cc := thingsapi.NewClient(conn, thingsTracer, cfg.thingsTimeout)
pubsub := nats.New(nc)
svc := newService(pubsub, logger)
@@ -99,13 +113,20 @@ func loadConfig() config {
log.Fatalf("Invalid value passed for %s\n", envClientTLS)
}
timeout, err := strconv.ParseInt(mainflux.Env(envThingsTimeout, defThingsTimeout), 10, 64)
if err != nil {
log.Fatalf("Invalid %s value: %s", envThingsTimeout, err.Error())
}
return config{
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
natsURL: mainflux.Env(envNatsURL, defNatsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
clientTLS: tls,
caCerts: mainflux.Env(envCACerts, defCACerts),
thingsURL: mainflux.Env(envThingsURL, defThingsURL),
natsURL: mainflux.Env(envNatsURL, defNatsURL),
logLevel: mainflux.Env(envLogLevel, defLogLevel),
port: mainflux.Env(envPort, defPort),
jaegerURL: mainflux.Env(envJaegerURL, defJaegerURL),
thingsTimeout: time.Duration(timeout) * time.Second,
}
}
@@ -133,6 +154,26 @@ func connectToThings(cfg config, logger logger.Logger) *grpc.ClientConn {
return conn
}
func initJaeger(svcName, url string, logger logger.Logger) (opentracing.Tracer, io.Closer) {
tracer, closer, err := jconfig.Configuration{
ServiceName: svcName,
Sampler: &jconfig.SamplerConfig{
Type: "const",
Param: 1,
},
Reporter: &jconfig.ReporterConfig{
LocalAgentHostPort: url,
LogSpans: true,
},
}.NewTracer()
if err != nil {
logger.Error(fmt.Sprintf("Failed to init Jaeger client: %s", err))
os.Exit(1)
}
return tracer, closer
}
func newService(pubsub adapter.Service, logger logger.Logger) adapter.Service {
svc := adapter.New(pubsub)
svc = api.LoggingMiddleware(svc, logger)
+15 -11
View File
@@ -9,15 +9,17 @@ The service is configured using the environment variables presented in the
following table. Note that any unset variables will be replaced with their
default values.
| Variable | Description | Default |
|-----------------------------|--------------------------------------------------------|-----------------------|
| MF_COAP_ADAPTER_PORT | Service listening port | 5683 |
| MF_NATS_URL | NATS instance URL | nats://localhost:4222 |
| MF_THINGS_URL | Things service URL | localhost:8181 |
| MF_COAP_ADAPTER_LOG_LEVEL | Service log level | error |
| MF_COAP_ADAPTER_CLIENT_TLS | Flag that indicates if TLS should be turned on | false |
| MF_COAP_ADAPTER_CA_CERTS | Path to trusted CAs in PEM format | |
| MF_COAP_ADAPTER_PING_PERIOD | Hours between 1 and 24 to ping client with ACK message | 12 |
| Variable | Description | Default |
|--------------------------------|--------------------------------------------------------|-----------------------|
| MF_COAP_ADAPTER_PORT | Service listening port | 5683 |
| MF_NATS_URL | NATS instance URL | nats://localhost:4222 |
| MF_THINGS_URL | Things service URL | localhost:8181 |
| MF_COAP_ADAPTER_LOG_LEVEL | Service log level | error |
| MF_COAP_ADAPTER_CLIENT_TLS | Flag that indicates if TLS should be turned on | false |
| MF_COAP_ADAPTER_CA_CERTS | Path to trusted CAs in PEM format | |
| MF_COAP_ADAPTER_PING_PERIOD | Hours between 1 and 24 to ping client with ACK message | 12 |
| MF_JAEGER_URL | Jaeger server URL | localhost:6831 |
| MF_COAP_ADAPTER_THINGS_TIMEOUT | Things gRPC request timeout in seconds | 1 |
## Deployment
@@ -40,6 +42,8 @@ services:
MF_COAP_ADAPTER_CLIENT_TLS: [Flag that indicates if TLS should be turned on]
MF_COAP_ADAPTER_CA_CERTS: [Path to trusted CAs in PEM format]
MF_COAP_ADAPTER_PING_PERIOD: [Hours between 1 and 24 to ping client with ACK message]
MF_JAEGER_URL: [Jaeger server URL]
MF_COAP_ADAPTER_THINGS_TIMEOUT: [Things gRPC request timeout in seconds]
```
Running this service outside of container requires working instance of the NATS service.
@@ -58,10 +62,10 @@ make coap
make install
# set the environment variables and run the service
MF_THINGS_URL=[Things service URL] MF_NATS_URL=[NATS instance URL] MF_COAP_ADAPTER_PORT=[Service HTTP port] MF_COAP_ADAPTER_LOG_LEVEL=[Service log level] MF_COAP_ADAPTER_CLIENT_TLS=[Flag that indicates if TLS should be turned on] MF_COAP_ADAPTER_CA_CERTS=[Path to trusted CAs in PEM format] MF_COAP_ADAPTER_PING_PERIOD: [Hours between 1 and 24 to ping client with ACK message] $GOBIN/mainflux-coap
MF_THINGS_URL=[Things service URL] MF_NATS_URL=[NATS instance URL] MF_COAP_ADAPTER_PORT=[Service HTTP port] MF_COAP_ADAPTER_LOG_LEVEL=[Service log level] MF_COAP_ADAPTER_CLIENT_TLS=[Flag that indicates if TLS should be turned on] MF_COAP_ADAPTER_CA_CERTS=[Path to trusted CAs in PEM format] MF_COAP_ADAPTER_PING_PERIOD: [Hours between 1 and 24 to ping client with ACK message] MF_JAEGER_URL=[Jaeger server URL] MF_COAP_ADAPTER_THINGS_TIMEOUT=[Things gRPC request timeout in seconds] $GOBIN/mainflux-coap
```
## Usage
If CoAP adapter is running locally (on default 5683 port), a valid URL would be: `coap://localhost/channels/<channel_id>/messages?authorization=<thing_auth_key>`.
Since CoAP protocol does not support `Authorization` header (option) and options have limited size, in order to send CoAP messages, valid `authorization` value must be present in `Uri-Query` option.
Since CoAP protocol does not support `Authorization` header (option) and options have limited size, in order to send CoAP messages, valid `authorization` value (a valid Thing key) must be present in `Uri-Query` option.
+6 -3
View File
@@ -11,6 +11,7 @@
package coap
import (
"context"
"errors"
"sync"
"time"
@@ -62,14 +63,16 @@ type Service interface {
var _ Service = (*adapterService)(nil)
type adapterService struct {
auth mainflux.ThingsServiceClient
pubsub Broker
obs map[string]*Observer
obsLock sync.Mutex
}
// New instantiates the CoAP adapter implementation.
func New(pubsub Broker, responses <-chan string) Service {
func New(pubsub Broker, auth mainflux.ThingsServiceClient, responses <-chan string) Service {
as := &adapterService{
auth: auth,
pubsub: pubsub,
obs: make(map[string]*Observer),
obsLock: sync.Mutex{},
@@ -122,8 +125,8 @@ func (svc *adapterService) listenResponses(responses <-chan string) {
}
}
func (svc *adapterService) Publish(msg mainflux.RawMessage) error {
if err := svc.pubsub.Publish(msg); err != nil {
func (svc *adapterService) Publish(ctx context.Context, token string, msg mainflux.RawMessage) error {
if err := svc.pubsub.Publish(ctx, token, msg); err != nil {
switch err {
case broker.ErrConnectionClosed, broker.ErrInvalidConnection:
return ErrFailedConnection
+3 -2
View File
@@ -10,6 +10,7 @@
package api
import (
"context"
"fmt"
"time"
@@ -30,7 +31,7 @@ func LoggingMiddleware(svc coap.Service, logger log.Logger) coap.Service {
return &loggingMiddleware{logger, svc}
}
func (lm *loggingMiddleware) Publish(msg mainflux.RawMessage) (err error) {
func (lm *loggingMiddleware) Publish(ctx context.Context, token string, msg mainflux.RawMessage) (err error) {
defer func(begin time.Time) {
destChannel := msg.Channel
if msg.Subtopic != "" {
@@ -44,7 +45,7 @@ func (lm *loggingMiddleware) Publish(msg mainflux.RawMessage) (err error) {
lm.logger.Info(fmt.Sprintf("%s without errors.", message))
}(time.Now())
return lm.svc.Publish(msg)
return lm.svc.Publish(ctx, token, msg)
}
func (lm *loggingMiddleware) Subscribe(chanID, subtopic, obsID string, o *coap.Observer) (err error) {
+3 -2
View File
@@ -10,6 +10,7 @@
package api
import (
"context"
"time"
"github.com/go-kit/kit/metrics"
@@ -34,13 +35,13 @@ func MetricsMiddleware(svc coap.Service, counter metrics.Counter, latency metric
}
}
func (mm *metricsMiddleware) Publish(msg mainflux.RawMessage) error {
func (mm *metricsMiddleware) Publish(ctx context.Context, token string, msg mainflux.RawMessage) error {
defer func(begin time.Time) {
mm.counter.With("method", "publish").Add(1)
mm.latency.With("method", "publish").Observe(time.Since(begin).Seconds())
}(time.Now())
return mm.svc.Publish(msg)
return mm.svc.Publish(ctx, token, msg)
}
func (mm *metricsMiddleware) Subscribe(chanID, subtopic, clientID string, o *coap.Observer) error {
+103 -80
View File
@@ -11,24 +11,22 @@ import (
"bytes"
"context"
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"net"
"net/http"
"regexp"
"strings"
"time"
gocoap "github.com/dustin/go-coap"
"github.com/go-zoo/bone"
"github.com/mainflux/mainflux"
"github.com/mainflux/mainflux/coap"
log "github.com/mainflux/mainflux/logger"
"github.com/prometheus/client_golang/prometheus/promhttp"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
mux "github.com/dereulenspiegel/coap-mux"
gocoap "github.com/dustin/go-coap"
"github.com/mainflux/mainflux"
)
const protocol = "coap"
@@ -37,6 +35,7 @@ var (
errBadRequest = errors.New("bad request")
errBadOption = errors.New("bad option")
errMalformedSubtopic = errors.New("malformed subtopic")
channelRegExp = regexp.MustCompile(`^/?channels/([\w\-]+)/messages(/[^?]*)?(\?.*)?$`)
)
var (
@@ -47,16 +46,6 @@ var (
type handler func(conn *net.UDPConn, addr *net.UDPAddr, msg *gocoap.Message) *gocoap.Message
func notFoundHandler(l *net.UDPConn, a *net.UDPAddr, m *gocoap.Message) *gocoap.Message {
if m.IsConfirmable() {
return &gocoap.Message{
Type: gocoap.Acknowledgement,
Code: gocoap.NotFound,
}
}
return nil
}
//MakeHTTPHandler creates handler for version endpoint.
func MakeHTTPHandler() http.Handler {
b := bone.New()
@@ -71,14 +60,54 @@ func MakeCOAPHandler(svc coap.Service, tc mainflux.ThingsServiceClient, l log.Lo
auth = tc
logger = l
pingPeriod = pp
r := mux.NewRouter()
r.Handle("/channels/{id}/messages", gocoap.FuncHandler(receive(svc))).Methods(gocoap.POST)
r.Handle("/channels/{id}/messages/{subtopic:[^?]*}", gocoap.FuncHandler(receive(svc))).Methods(gocoap.POST)
r.Handle("/channels/{id}/messages", gocoap.FuncHandler(observe(svc, responses)))
r.Handle("/channels/{id}/messages/{subtopic:[^?]*}", gocoap.FuncHandler(observe(svc, responses)))
r.NotFoundHandler = gocoap.FuncHandler(notFoundHandler)
return mux(svc, responses)
}
return r
func mux(svc coap.Service, responses chan<- string) gocoap.Handler {
return gocoap.FuncHandler(func(conn *net.UDPConn, addr *net.UDPAddr, msg *gocoap.Message) *gocoap.Message {
path := msg.PathString()
if !channelRegExp.Match([]byte(path)) {
logger.Info(fmt.Sprintf("path %s not found", path))
return &gocoap.Message{
Type: gocoap.NonConfirmable,
Code: gocoap.NotFound,
MessageID: msg.MessageID,
Token: msg.Token,
}
}
// Allow "/" to be a part of the path.
if strings.HasPrefix(path, "/") {
msg.SetPathString(path[1:])
}
switch msg.Code {
case gocoap.GET:
return observe(svc, responses)(conn, addr, msg)
default:
return receive(svc, msg)
}
})
}
func id(msg *gocoap.Message) string {
vars := strings.Split(msg.PathString(), "/")
if len(vars) > 1 {
return vars[1]
}
return ""
}
func subtopic(msg *gocoap.Message) string {
path := msg.PathString()
pos := 0
for i, c := range path {
if c == '/' {
pos++
}
if pos == 2 {
return path[i:]
}
}
return ""
}
func authorize(msg *gocoap.Message, res *gocoap.Message, cid string) (string, error) {
@@ -116,7 +145,8 @@ func authorize(msg *gocoap.Message, res *gocoap.Message, cid string) (string, er
return id.GetValue(), nil
}
func fmtSubtopic(subtopic string) (string, error) {
func fmtSubtopic(msg *gocoap.Message) (string, error) {
subtopic := subtopic(msg)
if subtopic == "" {
return subtopic, nil
}
@@ -142,62 +172,60 @@ func fmtSubtopic(subtopic string) (string, error) {
return subtopic, nil
}
func receive(svc coap.Service) handler {
return func(conn *net.UDPConn, addr *net.UDPAddr, msg *gocoap.Message) *gocoap.Message {
// By default message is NonConfirmable, so
// NonConfirmable response is sent back.
res := &gocoap.Message{
Type: gocoap.NonConfirmable,
// According to https://tools.ietf.org/html/rfc7252#page-47: If the POST
// succeeds but does not result in a new resource being created on the
// server, the response SHOULD have a 2.04 (Changed) Response Code.
Code: gocoap.Changed,
MessageID: msg.MessageID,
Token: msg.Token,
Payload: []byte{},
}
func receive(svc coap.Service, msg *gocoap.Message) *gocoap.Message {
// By default message is NonConfirmable, so
// NonConfirmable response is sent back.
res := &gocoap.Message{
Type: gocoap.NonConfirmable,
// According to https://tools.ietf.org/html/rfc7252#page-47: If the POST
// succeeds but does not result in a new resource being created on the
// server, the response SHOULD have a 2.04 (Changed) Response Code.
Code: gocoap.Changed,
MessageID: msg.MessageID,
Token: msg.Token,
Payload: []byte{},
}
if msg.IsConfirmable() {
res.Type = gocoap.Acknowledgement
res.SetOption(gocoap.ContentFormat, gocoap.AppJSON)
if len(msg.Payload) == 0 {
res.Code = gocoap.BadRequest
return res
}
}
chanID := mux.Var(msg, "id")
if chanID == "" {
res.Code = gocoap.NotFound
return res
}
subtopic, err := fmtSubtopic(mux.Var(msg, "subtopic"))
if err != nil {
if msg.IsConfirmable() {
res.Type = gocoap.Acknowledgement
res.SetOption(gocoap.ContentFormat, gocoap.AppJSON)
if len(msg.Payload) == 0 {
res.Code = gocoap.BadRequest
return res
}
}
publisher, err := authorize(msg, res, chanID)
if err != nil {
res.Code = gocoap.Forbidden
return res
}
rawMsg := mainflux.RawMessage{
Channel: chanID,
Subtopic: subtopic,
Publisher: publisher,
Protocol: protocol,
Payload: msg.Payload,
}
if err := svc.Publish(rawMsg); err != nil {
res.Code = gocoap.InternalServerError
}
chanID := id(msg)
if chanID == "" {
res.Code = gocoap.NotFound
return res
}
subtopic, err := fmtSubtopic(msg)
if err != nil {
res.Code = gocoap.BadRequest
return res
}
publisher, err := authorize(msg, res, chanID)
if err != nil {
res.Code = gocoap.Forbidden
return res
}
rawMsg := mainflux.RawMessage{
Channel: chanID,
Subtopic: subtopic,
Publisher: publisher,
Protocol: protocol,
Payload: msg.Payload,
}
if err := svc.Publish(context.Background(), "", rawMsg); err != nil {
res.Code = gocoap.InternalServerError
}
return res
}
func observe(svc coap.Service, responses chan<- string) handler {
@@ -211,13 +239,13 @@ func observe(svc coap.Service, responses chan<- string) handler {
}
res.SetOption(gocoap.ContentFormat, gocoap.AppJSON)
chanID := mux.Var(msg, "id")
chanID := id(msg)
if chanID == "" {
res.Code = gocoap.NotFound
return res
}
subtopic, err := fmtSubtopic(mux.Var(msg, "subtopic"))
subtopic, err := fmtSubtopic(msg)
if err != nil {
res.Code = gocoap.BadRequest
return res
@@ -275,13 +303,8 @@ func handleMessage(conn *net.UDPConn, addr *net.UDPAddr, o *coap.Observer, msg *
if !ok {
return
}
payload, err := json.Marshal(msg)
if err != nil {
logger.Warn(fmt.Sprintf("Failed to parse received message: %s", err))
continue
}
notifyMsg.Payload = payload
notifyMsg.Payload = msg.Payload
notifyMsg.MessageID = o.LoadMessageID()
buff := new(bytes.Buffer)
observe := uint64(notifyMsg.MessageID)
+2 -1
View File
@@ -9,6 +9,7 @@
package nats
import (
"context"
"fmt"
"github.com/gogo/protobuf/proto"
@@ -38,7 +39,7 @@ func (pubsub *natsPublisher) fmtSubject(chanID, subtopic string) string {
return subject
}
func (pubsub *natsPublisher) Publish(msg mainflux.RawMessage) error {
func (pubsub *natsPublisher) Publish(_ context.Context, _ string, msg mainflux.RawMessage) error {
data, err := proto.Marshal(&msg)
if err != nil {
return err
+5 -3
View File
@@ -1,12 +1,14 @@
FROM golang:1.10-alpine AS builder
ARG SVC_NAME
ARG SVC
ARG GOARCH
ARG GOARM
WORKDIR /go/src/github.com/mainflux/mainflux
COPY . .
RUN apk update \
&& apk add make\
&& make $SVC_NAME \
&& mv build/mainflux-$SVC_NAME /exe
&& make $SVC \
&& mv build/mainflux-$SVC /exe
FROM scratch
COPY --from=builder /exe /
+2 -2
View File
@@ -1,4 +1,4 @@
FROM scratch
ARG SVC_NAME
COPY mainflux-$SVC_NAME /exe
ARG SVC
COPY mainflux-$SVC /exe
ENTRYPOINT ["/exe"]
+23
View File
@@ -0,0 +1,23 @@
# Docker Composition
Configure environment variables and run Mainflux Docker Composition.
*Note**: `docker-compose` uses `.env` file to set all environment variables. Ensure that you run the command from the same location as .env file.
## Installation
Follow the [official documentation](https://docs.docker.com/compose/install/).
## Usage
Run following commands from project root directory.
```
docker-compose -f docker/docker-compose.yml up
```
```
docker-compose -f docker/addons/<path>/docker-compose.yml up
```
+21 -13
View File
@@ -4,17 +4,22 @@ networks:
docker_mainflux-base-net:
external: true
volumes:
mainflux-bootstrap-db-volume:
services:
bootstrap-db:
image: postgres:10.2-alpine
container_name: mainflux-bootstrap-db
restart: on-failure
environment:
POSTGRES_USER: mainflux
POSTGRES_PASSWORD: mainflux
POSTGRES_DB: bootstrap
POSTGRES_USER: ${MF_BOOTSTRAP_DB_USER}
POSTGRES_PASSWORD: ${MF_BOOTSTRAP_DB_PASS}
POSTGRES_DB: ${MF_BOOTSTRAP_DB}
networks:
- docker_mainflux-base-net
volumes:
- mainflux-bootstrap-db-volume:/var/lib/postgresql/data
bootstrap:
image: mainflux/bootstrap:latest
@@ -23,17 +28,20 @@ services:
- bootstrap-db
restart: on-failure
ports:
- 8200:8200
- ${MF_BOOTSTRAP_PORT}:${MF_BOOTSTRAP_PORT}
environment:
MF_BOOTSTRAP_LOG_LEVEL: debug
MF_BOOTSTRAP_LOG_LEVEL: ${MF_BOOTSTRAP_LOG_LEVEL}
MF_BOOTSTRAP_DB_HOST: bootstrap-db
MF_BOOTSTRAP_DB_PORT: 5432
MF_BOOTSTRAP_DB_USER: mainflux
MF_BOOTSTRAP_DB_PASS: mainflux
MF_BOOTSTRAP_DB: bootstrap
MF_BOOTSTRAP_DB_SSL_MODE: disable
MF_BOOTSTRAP_PORT: 8200
MF_SDK_BASE_URL: http://mainflux-things:8182
MF_USERS_URL: mainflux-users:8181
MF_BOOTSTRAP_DB_PORT: ${MF_BOOTSTRAP_DB_PORT}
MF_BOOTSTRAP_DB_USER: ${MF_BOOTSTRAP_DB_USER}
MF_BOOTSTRAP_DB_PASS: ${MF_BOOTSTRAP_DB_PASS}
MF_BOOTSTRAP_DB: ${MF_BOOTSTRAP_DB}
MF_BOOTSTRAP_DB_SSL_MODE: ${MF_BOOTSTRAP_DB_SSL_MODE}
MF_BOOTSTRAP_PORT: ${MF_BOOTSTRAP_PORT}
MF_SDK_BASE_URL: http://mainflux-things:${MF_THINGS_HTTP_PORT}
MF_USERS_URL: mainflux-users:${MF_USERS_GRPC_PORT}
MF_THINGS_ES_URL: es-redis:${MF_REDIS_TCP_PORT}
MF_BOOTSTRAP_ES_URL: es-redis:${MF_REDIS_TCP_PORT}
MF_JAEGER_URL: ${MF_JAEGER_URL}
networks:
- docker_mainflux-base-net
@@ -18,12 +18,13 @@ services:
container_name: mainflux-cassandra-reader
restart: on-failure
environment:
MF_CASSANDRA_READER_LOG_LEVEL: debug
MF_THINGS_URL: things:8183
MF_CASSANDRA_READER_PORT: 8903
MF_CASSANDRA_READER_DB_CLUSTER: mainflux-cassandra
MF_CASSANDRA_READER_DB_KEYSPACE: mainflux
MF_CASSANDRA_READER_LOG_LEVEL: ${MF_CASSANDRA_READER_LOG_LEVEL}
MF_THINGS_URL: things:${MF_THINGS_AUTH_GRPC_PORT}
MF_CASSANDRA_READER_PORT: ${MF_CASSANDRA_READER_PORT}
MF_CASSANDRA_READER_DB_CLUSTER: ${MF_CASSANDRA_READER_DB_CLUSTER}
MF_CASSANDRA_READER_DB_KEYSPACE: ${MF_CASSANDRA_READER_DB_KEYSPACE}
MF_JAEGER_URL: ${MF_JAEGER_URL}
ports:
- 8903:8903
- ${MF_CASSANDRA_READER_PORT}:${MF_CASSANDRA_READER_PORT}
networks:
- docker_mainflux-base-net
@@ -0,0 +1,4 @@
# If you want to listen on all channels, just pass one element ["*"], otherwise
# pass the list of channels.
[channels]
filter = ["*"]
@@ -12,6 +12,9 @@ networks:
docker_mainflux-base-net:
external: true
volumes:
mainflux-cassandra-volume:
services:
cassandra:
image: cassandra:3.11.3
@@ -20,7 +23,9 @@ services:
networks:
- docker_mainflux-base-net
ports:
- 9042:9042
- ${MF_CASSANDRA_WRITER_DB_PORT}:${MF_CASSANDRA_WRITER_DB_PORT}
volumes:
- mainflux-cassandra-volume:/var/lib/cassandra
cassandra-writer:
image: mainflux/cassandra-writer:latest
@@ -29,12 +34,15 @@ services:
- cassandra
restart: on-failure
environment:
MF_CASSANDRA_WRITER_LOG_LEVEL: debug
MF_NATS_URL: nats://nats:4222
MF_CASSANDRA_WRITER_PORT: 8902
MF_CASSANDRA_WRITER_DB_CLUSTER: mainflux-cassandra
MF_CASSANDRA_WRITER_DB_KEYSPACE: mainflux
MF_CASSANDRA_WRITER_LOG_LEVEL: ${MF_CASSANDRA_WRITER_LOG_LEVEL}
MF_NATS_URL: ${MF_NATS_URL}
MF_CASSANDRA_WRITER_PORT: ${MF_CASSANDRA_WRITER_PORT}
MF_CASSANDRA_WRITER_DB_PORT: ${MF_CASSANDRA_WRITER_DB_PORT}
MF_CASSANDRA_WRITER_DB_CLUSTER: ${MF_CASSANDRA_WRITER_DB_CLUSTER}
MF_CASSANDRA_WRITER_DB_KEYSPACE: ${MF_CASSANDRA_WRITER_DB_KEYSPACE}
ports:
- 8902:8902
- ${MF_CASSANDRA_WRITER_PORT}:${MF_CASSANDRA_WRITER_PORT}
networks:
- docker_mainflux-base-net
volumes:
- ./channels.toml:/config/channels.toml
@@ -18,14 +18,15 @@ services:
restart: on-failure
environment:
MF_INFLUX_READER_LOG_LEVEL: debug
MF_THINGS_URL: things:8183
MF_INFLUX_READER_PORT: 8905
MF_INFLUX_READER_DB_NAME: mainflux
MF_THINGS_URL: things:${MF_THINGS_AUTH_GRPC_PORT}
MF_INFLUX_READER_PORT: ${MF_INFLUX_READER_PORT}
MF_INFLUX_READER_DB_NAME: ${MF_INFLUX_READER_DB_NAME}
MF_INFLUX_READER_DB_HOST: mainflux-influxdb
MF_INFLUX_READER_DB_PORT: 8086
MF_INFLUX_READER_DB_USER: mainflux
MF_INFLUX_READER_DB_PASS: mainflux
MF_INFLUX_READER_DB_PORT: ${MF_INFLUX_READER_DB_PORT}
MF_INFLUX_READER_DB_USER: ${MF_INFLUX_READER_DB_USER}
MF_INFLUX_READER_DB_PASS: ${MF_INFLUX_READER_DB_PASS}
MF_JAEGER_URL: ${MF_JAEGER_URL}
ports:
- 8905:8905
- ${MF_INFLUX_READER_PORT}:${MF_INFLUX_READER_PORT}
networks:
- docker_mainflux-base-net
@@ -0,0 +1,4 @@
# If you want to listen on all channels, just pass one element ["*"], otherwise
# pass the list of channels.
[channels]
filter = ["*"]
@@ -11,6 +11,10 @@ networks:
docker_mainflux-base-net:
external: true
volumes:
mainflux-influxdb-volume:
mainflux-grafana-volume:
services:
influxdb:
@@ -24,7 +28,9 @@ services:
networks:
- docker_mainflux-base-net
ports:
- 8086:8086
- ${MF_INFLUX_WRITER_DB_PORT}:${MF_INFLUX_WRITER_DB_PORT}
volumes:
- mainflux-influxdb-volume:/var/lib/influxdb
influxdb-writer:
image: mainflux/influxdb-writer:latest
@@ -34,19 +40,21 @@ services:
restart: on-failure
environment:
MF_INFLUX_WRITER_LOG_LEVEL: debug
MF_NATS_URL: nats://nats:4222
MF_INFLUX_WRITER_PORT: 8900
MF_INFLUX_WRITER_BATCH_SIZE: 5000
MF_INFLUX_WRITER_BATCH_TIMEOUT: 5
MF_INFLUX_WRITER_DB_NAME: mainflux
MF_NATS_URL: ${MF_NATS_URL}
MF_INFLUX_WRITER_PORT: ${MF_INFLUX_WRITER_PORT}
MF_INFLUX_WRITER_BATCH_SIZE: ${MF_INFLUX_WRITER_BATCH_SIZE}
MF_INFLUX_WRITER_BATCH_TIMEOUT: ${MF_INFLUX_WRITER_BATCH_TIMEOUT}
MF_INFLUX_WRITER_DB_NAME: ${MF_INFLUX_WRITER_DB_NAME}
MF_INFLUX_WRITER_DB_HOST: mainflux-influxdb
MF_INFLUX_WRITER_DB_PORT: 8086
MF_INFLUX_WRITER_DB_USER: mainflux
MF_INFLUX_WRITER_DB_PASS: mainflux
MF_INFLUX_WRITER_DB_PORT: ${MF_INFLUX_WRITER_DB_PORT}
MF_INFLUX_WRITER_DB_USER: ${MF_INFLUX_WRITER_DB_USER}
MF_INFLUX_WRITER_DB_PASS: ${MF_INFLUX_WRITER_DB_PASS}
ports:
- 8900:8900
- ${MF_INFLUX_WRITER_PORT}:${MF_INFLUX_WRITER_PORT}
networks:
- docker_mainflux-base-net
volumes:
- ./channels.toml:/config/channels.toml
grafana:
image: grafana/grafana:5.1.3
@@ -55,6 +63,8 @@ services:
- influxdb
restart: on-failure
ports:
- 3001:3000
- ${MF_INFLUX_WRITER_GRAFANA_PORT}:${MF_GRAFANA_PORT}
networks:
- docker_mainflux-base-net
volumes:
- mainflux-grafana-volume:/var/lib/grafana
@@ -24,13 +24,13 @@ services:
container_name: mainflux-lora
restart: on-failure
environment:
MF_LORA_ADAPTER_LOG_LEVEL: debug
MF_THINGS_ES_URL: things-redis:6379
MF_LORA_ADAPTER_ROUTEMAP_URL: lora-redis:6379
MF_LORA_ADAPTER_MESSAGES_URL: tcp://lora.mqtt.mainflux.io:1883
MF_LORA_ADAPTER_HTTP_PORT: 8187
MF_NATS_URL: nats://nats:4222
MF_LORA_ADAPTER_LOG_LEVEL: ${MF_LORA_ADAPTER_LOG_LEVEL}
MF_THINGS_ES_URL: es-redis:${MF_REDIS_TCP_PORT}
MF_LORA_ADAPTER_ROUTEMAP_URL: lora-redis:${MF_REDIS_TCP_PORT}
MF_LORA_ADAPTER_MESSAGES_URL: ${MF_LORA_ADAPTER_MESSAGES_URL}
MF_LORA_ADAPTER_HTTP_PORT: ${MF_LORA_ADAPTER_HTTP_PORT}
MF_NATS_URL: ${MF_NATS_URL}
ports:
- 8187:8187
- ${MF_LORA_ADAPTER_HTTP_PORT}:${MF_LORA_ADAPTER_HTTP_PORT}
networks:
- docker_mainflux-base-net
@@ -18,13 +18,14 @@ services:
container_name: mainflux-mongodb-reader
restart: on-failure
environment:
MF_MONGO_READER_LOG_LEVEL: debug
MF_THINGS_URL: things:8183
MF_MONGO_READER_PORT: 8904
MF_MONGO_READER_DB_NAME: mainflux
MF_MONGO_READER_LOG_LEVEL: ${MF_MONGO_READER_LOG_LEVEL}
MF_THINGS_URL: things:${MF_THINGS_AUTH_GRPC_PORT}
MF_MONGO_READER_PORT: ${MF_MONGO_READER_PORT}
MF_MONGO_READER_DB_NAME: ${MF_MONGO_READER_DB_NAME}
MF_MONGO_READER_DB_HOST: mongodb
MF_MONGO_READER_DB_PORT: 27017
MF_MONGO_READER_DB_PORT: ${MF_MONGO_READER_DB_PORT}
MF_JAEGER_URL: ${MF_JAEGER_URL}
ports:
- 8904:8904
- ${MF_MONGO_READER_PORT}:${MF_MONGO_READER_PORT}
networks:
- docker_mainflux-base-net
@@ -0,0 +1,4 @@
# If you want to listen on all channels, just pass one element ["*"], otherwise
# pass the list of channels.
[channels]
filter = ["*"]
@@ -13,17 +13,24 @@ networks:
docker_mainflux-base-net:
external: true
volumes:
mainflux-mongodb-db-volume:
mainflux-mongodb-configdb-volume:
services:
mongodb:
image: mongo:3.6-jessie
container_name: mainflux-mongodb
restart: on-failure
environment:
MONGO_INITDB_DATABASE: mainflux
MONGO_INITDB_DATABASE: ${MF_MONGO_WRITER_DB_NAME}
ports:
- 27017:27017
- ${MF_MONGO_WRITER_DB_PORT}:${MF_MONGO_WRITER_DB_PORT}
networks:
- docker_mainflux-base-net
volumes:
- mainflux-mongodb-db-volume:/data/db
- mainflux-mongodb-configdb-volume:/data/configdb
mongodb-writer:
image: mainflux/mongodb-writer:latest
@@ -32,13 +39,15 @@ services:
- mongodb
restart: on-failure
environment:
MF_MONGO_WRITER_LOG_LEVEL: debug
MF_NATS_URL: nats://nats:4222
MF_MONGO_WRITER_PORT: 8901
MF_MONGO_WRITER_DB_NAME: mainflux
MF_MONGO_WRITER_LOG_LEVEL: ${MF_MONGO_WRITER_LOG_LEVEL}
MF_NATS_URL: ${MF_NATS_URL}
MF_MONGO_WRITER_PORT: ${MF_MONGO_WRITER_PORT}
MF_MONGO_WRITER_DB_NAME: ${MF_MONGO_WRITER_DB_NAME}
MF_MONGO_WRITER_DB_HOST: mongodb
MF_MONGO_WRITER_DB_PORT: 27017
MF_MONGO_WRITER_DB_PORT: ${MF_MONGO_WRITER_DB_PORT}
ports:
- 8901:8901
- ${MF_MONGO_WRITER_PORT}:${MF_MONGO_WRITER_PORT}
networks:
- docker_mainflux-base-net
volumes:
- ./channels.toml:/config/channels.toml
@@ -0,0 +1,39 @@
###
# This docker-compose file contains optional Postgres-reader service for Mainflux platform.
# Since this service is optional, this file is dependent of docker-compose.yml file
# from <project_root>/docker. In order to run these optional service, execute command:
# docker-compose -f docker/docker-compose.yml -f docker/addons/postgres-reader/docker-compose.yml up
# from project root.
###
version: "3"
networks:
docker_mainflux-base-net:
external: true
services:
postgres-reader:
image: mainflux/postgres-reader:latest
container_name: mainflux-postgres-reader
restart: on-failure
environment:
MF_THINGS_URL: things:${MF_THINGS_AUTH_GRPC_PORT}
MF_POSTGRES_READER_LOG_LEVEL: ${MF_POSTGRES_READER_LOG_LEVEL}
MF_POSTGRES_READER_PORT: ${MF_POSTGRES_READER_PORT}
MF_POSTGRES_READER_CLIENT_TLS: ${MF_POSTGRES_READER_CLIENT_TLS}
MF_POSTGRES_READER_CA_CERTS: ${MF_POSTGRES_READER_CA_CERTS}
MF_POSTGRES_READER_DB_HOST: postgres
MF_POSTGRES_READER_DB_PORT: ${MF_POSTGRES_READER_DB_PORT}
MF_POSTGRES_READER_DB_USER: ${MF_POSTGRES_READER_DB_USER}
MF_POSTGRES_READER_DB_PASS: ${MF_POSTGRES_READER_DB_PASS}
MF_POSTGRES_READER_DB_NAME: ${MF_POSTGRES_READER_DB_NAME}
MF_POSTGRES_READER_DB_SSL_MODE: ${MF_POSTGRES_READER_DB_SSL_MODE}
MF_POSTGRES_READER_DB_SSL_CERT: ${MF_POSTGRES_READER_DB_SSL_CERT}
MF_POSTGRES_READER_DB_SSL_KEY: ${MF_POSTGRES_READER_DB_SSL_KEY}
MF_POSTGRES_READER_DB_SSL_ROOT_CERT: ${MF_POSTGRES_READER_DB_SSL_ROOT_CERT}
MF_JAEGER_URL: ${MF_JAEGER_URL}
ports:
- ${MF_POSTGRES_READER_PORT}:${MF_POSTGRES_READER_PORT}
networks:
- docker_mainflux-base-net
@@ -0,0 +1,4 @@
# If you want to listen on all channels, just pass one element ["*"], otherwise
# pass the list of channels.
[channels]
filter = ["*"]
@@ -0,0 +1,57 @@
###
# This docker-compose file contains optional Postgres and Postgres-writer services
# for Mainflux platform. Since these are optional, this file is dependent of docker-compose file
# from <project_root>/docker. In order to run these optional service, execute command:
# docker-compose -f docker/docker-compose.yml -f docker/addons/postgres-writer/docker-compose.yml up
# from project root. PostgreSQL default port (5432) is exposed, so you can use various tools for database
# inspection and data visualization.
###
version: "3"
networks:
docker_mainflux-base-net:
external: true
volumes:
mainflux-postgres-writer-volume:
services:
postgres:
image: postgres:10.2-alpine
container_name: mainflux-postgres
restart: on-failure
environment:
POSTGRES_USER: ${MF_POSTGRES_WRITER_DB_USER}
POSTGRES_PASSWORD: ${MF_POSTGRES_WRITER_DB_PASS}
POSTGRES_DB: ${MF_POSTGRES_WRITER_DB_NAME}
networks:
- docker_mainflux-base-net
volumes:
- mainflux-postgres-writer-volume:/var/lib/postgresql/data
postgres-writer:
image: mainflux/postgres-writer:latest
container_name: mainflux-postgres-writer
depends_on:
- postgres
restart: on-failure
environment:
MF_NATS_URL: ${MF_NATS_URL}
MF_POSTGRES_WRITER_LOG_LEVEL: ${MF_POSTGRES_WRITER_LOG_LEVEL}
MF_POSTGRES_WRITER_PORT: ${MF_POSTGRES_WRITER_PORT}
MF_POSTGRES_WRITER_DB_HOST: postgres
MF_POSTGRES_WRITER_DB_PORT: ${MF_POSTGRES_WRITER_DB_PORT}
MF_POSTGRES_WRITER_DB_USER: ${MF_POSTGRES_WRITER_DB_USER}
MF_POSTGRES_WRITER_DB_PASS: ${MF_POSTGRES_WRITER_DB_PASS}
MF_POSTGRES_WRITER_DB_NAME: ${MF_POSTGRES_WRITER_DB_NAME}
MF_POSTGRES_WRITER_DB_SSL_MODE: ${MF_POSTGRES_WRITER_DB_SSL_MODE}
MF_POSTGRES_WRITER_DB_SSL_CERT: ${MF_POSTGRES_WRITER_DB_SSL_CERT}
MF_POSTGRES_WRITER_DB_SSL_KEY: ${MF_POSTGRES_WRITER_DB_SSL_KEY}
MF_POSTGRES_WRITER_DB_SSL_ROOT_CERT: ${MF_POSTGRES_WRITER_DB_SSL_ROOT_CERT}
ports:
- ${MF_POSTGRES_WRITER_PORT}:${MF_POSTGRES_WRITER_PORT}
networks:
- docker_mainflux-base-net
volumes:
- ./channels.toml:/config/channels.toml
+127 -79
View File
@@ -12,22 +12,35 @@ networks:
mainflux-base-net:
driver: bridge
volumes:
mainflux-users-db-volume:
mainflux-things-db-volume:
mainflux-mqtt-redis-volume:
mainflux-things-redis-volume:
mainflux-es-redis-volume:
services:
nginx:
image: nginx:1.14.2-alpine
image: nginx:1.16.0-alpine
container_name: mainflux-nginx
restart: on-failure
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
- ./nginx/nginx-${AUTH-key}.conf:/etc/nginx/nginx.conf.template
- ./nginx/entrypoint.sh:/entrypoint.sh
- ./ssl/authorization.js:/etc/nginx/authorization.js
- ./ssl/certs/mainflux-server.crt:/etc/ssl/certs/mainflux-server.crt
- ./ssl/certs/ca.crt:/etc/ssl/certs/ca.crt
- ./ssl/certs/mainflux-server.key:/etc/ssl/private/mainflux-server.key
- ./ssl/dhparam.pem:/etc/ssl/certs/dhparam.pem
ports:
- 80:80
- 443:443
- 8883:8883
- ${MF_NGINX_HTTP_PORT}:${MF_NGINX_HTTP_PORT}
- ${MF_NGINX_SSL_PORT}:${MF_NGINX_SSL_PORT}
- ${MF_NGINX_MQTT_PORT}:${MF_NGINX_MQTT_PORT}
networks:
- mainflux-base-net
environment:
MF_UI_PORT: ${MF_UI_PORT}
command: /entrypoint.sh
nats:
image: nats:1.3.0
@@ -37,15 +50,17 @@ services:
- mainflux-base-net
users-db:
image: postgres:10.2-alpine
image: postgres:10.8-alpine
container_name: mainflux-users-db
restart: on-failure
environment:
POSTGRES_USER: mainflux
POSTGRES_PASSWORD: mainflux
POSTGRES_DB: users
POSTGRES_USER: ${MF_USERS_DB_USER}
POSTGRES_PASSWORD: ${MF_USERS_DB_PASS}
POSTGRES_DB: ${MF_USERS_DB}
networks:
- mainflux-base-net
volumes:
- mainflux-users-db-volume:/var/lib/postgresql/data
users:
image: mainflux/users:latest
@@ -53,33 +68,36 @@ services:
depends_on:
- users-db
expose:
- 8181
- ${MF_USERS_GRPC_PORT}
restart: on-failure
environment:
MF_USERS_LOG_LEVEL: debug
MF_USERS_LOG_LEVEL: ${MF_USERS_LOG_LEVEL}
MF_USERS_DB_HOST: users-db
MF_USERS_DB_PORT: 5432
MF_USERS_DB_USER: mainflux
MF_USERS_DB_PASS: mainflux
MF_USERS_DB: users
MF_USERS_HTTP_PORT: 8180
MF_USERS_GRPC_PORT: 8181
MF_USERS_SECRET: secret
MF_USERS_DB_PORT: ${MF_USERS_DB_PORT}
MF_USERS_DB_USER: ${MF_USERS_DB_USER}
MF_USERS_DB_PASS: ${MF_USERS_DB_PASS}
MF_USERS_DB: ${MF_USERS_DB}
MF_USERS_HTTP_PORT: ${MF_USERS_HTTP_PORT}
MF_USERS_GRPC_PORT: ${MF_USERS_GRPC_PORT}
MF_USERS_SECRET: ${MF_USERS_SECRET}
MF_JAEGER_URL: ${MF_JAEGER_URL}
ports:
- 8180:8180
- ${MF_USERS_HTTP_PORT}:${MF_USERS_HTTP_PORT}
networks:
- mainflux-base-net
things-db:
image: postgres:10.2-alpine
image: postgres:10.8-alpine
container_name: mainflux-things-db
restart: on-failure
environment:
POSTGRES_USER: mainflux
POSTGRES_PASSWORD: mainflux
POSTGRES_DB: things
POSTGRES_USER: ${MF_THINGS_DB_USER}
POSTGRES_PASSWORD: ${MF_THINGS_DB_PASS}
POSTGRES_DB: ${MF_THINGS_DB}
networks:
- mainflux-base-net
volumes:
- mainflux-things-db-volume:/var/lib/postgresql/data
things-redis:
image: redis:5.0-alpine
@@ -87,6 +105,8 @@ services:
restart: on-failure
networks:
- mainflux-base-net
volumes:
- mainflux-things-redis-volume:/data
things:
image: mainflux/things:latest
@@ -96,21 +116,34 @@ services:
- users
restart: on-failure
environment:
MF_THINGS_LOG_LEVEL: debug
MF_THINGS_LOG_LEVEL: ${MF_THINGS_LOG_LEVEL}
MF_THINGS_DB_HOST: things-db
MF_THINGS_DB_PORT: 5432
MF_THINGS_DB_USER: mainflux
MF_THINGS_DB_PASS: mainflux
MF_THINGS_DB: things
MF_THINGS_CACHE_URL: things-redis:6379
MF_THINGS_ES_URL: things-redis:6379
MF_THINGS_HTTP_PORT: 8182
MF_THINGS_GRPC_PORT: 8183
MF_USERS_URL: users:8181
MF_THINGS_SECRET: secret
MF_THINGS_DB_PORT: ${MF_THINGS_DB_PORT}
MF_THINGS_DB_USER: ${MF_THINGS_DB_USER}
MF_THINGS_DB_PASS: ${MF_THINGS_DB_PASS}
MF_THINGS_DB: ${MF_THINGS_DB}
MF_THINGS_CACHE_URL: things-redis:${MF_REDIS_TCP_PORT}
MF_THINGS_ES_URL: es-redis:${MF_REDIS_TCP_PORT}
MF_THINGS_HTTP_PORT: ${MF_THINGS_HTTP_PORT}
MF_THINGS_AUTH_HTTP_PORT: ${MF_THINGS_AUTH_HTTP_PORT}
MF_THINGS_AUTH_GRPC_PORT: ${MF_THINGS_AUTH_GRPC_PORT}
MF_USERS_URL: users:${MF_USERS_GRPC_PORT}
MF_THINGS_SECRET: ${MF_THINGS_SECRET}
MF_JAEGER_URL: ${MF_JAEGER_URL}
ports:
- 8182:8182
- 8183:8183
- ${MF_THINGS_HTTP_PORT}:${MF_THINGS_HTTP_PORT}
- ${MF_THINGS_AUTH_HTTP_PORT}:${MF_THINGS_AUTH_HTTP_PORT}
- ${MF_THINGS_AUTH_GRPC_PORT}:${MF_THINGS_AUTH_GRPC_PORT}
networks:
- mainflux-base-net
jaeger:
image: jaegertracing/all-in-one:1.13
ports:
- ${MF_JAEGER_PORT}:${MF_JAEGER_PORT}/udp
- ${MF_JAEGER_FRONTEND}:${MF_JAEGER_FRONTEND}
- ${MF_JAEGER_COLLECTOR}:${MF_JAEGER_COLLECTOR}
- ${MF_JAEGER_CONFIGS}:${MF_JAEGER_CONFIGS}
networks:
- mainflux-base-net
@@ -120,23 +153,12 @@ services:
restart: on-failure
depends_on:
- nats
expose:
- 8184
environment:
MF_NORMALIZER_LOG_LEVEL: debug
MF_NATS_URL: nats://nats:4222
MF_NORMALIZER_PORT: 8184
MF_NORMALIZER_LOG_LEVEL: ${MF_NORMALIZER_LOG_LEVEL}
MF_NATS_URL: ${MF_NATS_URL}
MF_NORMALIZER_PORT: ${MF_NORMALIZER_PORT}
ports:
- 8184:8184
networks:
- mainflux-base-net
ui:
image: mainflux/ui:latest
container_name: mainflux-ui
restart: on-failure
ports:
- 3000:3000
- ${MF_NORMALIZER_PORT}:${MF_NORMALIZER_PORT}
networks:
- mainflux-base-net
@@ -148,12 +170,13 @@ services:
- nats
restart: on-failure
environment:
MF_WS_ADAPTER_LOG_LEVEL: debug
MF_WS_ADAPTER_PORT: 8186
MF_NATS_URL: nats://nats:4222
MF_THINGS_URL: things:8183
MF_WS_ADAPTER_LOG_LEVEL: ${MF_WS_ADAPTER_LOG_LEVEL}
MF_WS_ADAPTER_PORT: ${MF_WS_ADAPTER_PORT}
MF_NATS_URL: ${MF_NATS_URL}
MF_THINGS_URL: things:${MF_THINGS_AUTH_GRPC_PORT}
MF_JAEGER_URL: ${MF_JAEGER_URL}
ports:
- 8186:8186
- ${MF_WS_ADAPTER_PORT}:${MF_WS_ADAPTER_PORT}
networks:
- mainflux-base-net
@@ -164,24 +187,34 @@ services:
- things
- nats
restart: on-failure
expose:
- 8185
environment:
MF_HTTP_ADAPTER_LOG_LEVEL: debug
MF_HTTP_ADAPTER_PORT: 8185
MF_NATS_URL: nats://nats:4222
MF_THINGS_URL: things:8183
MF_HTTP_ADAPTER_PORT: ${MF_HTTP_ADAPTER_PORT}
MF_NATS_URL: ${MF_NATS_URL}
MF_THINGS_URL: things:${MF_THINGS_AUTH_GRPC_PORT}
MF_JAEGER_URL: ${MF_JAEGER_URL}
ports:
- 8185:8185
- ${MF_HTTP_ADAPTER_PORT}:${MF_HTTP_ADAPTER_PORT}
networks:
- mainflux-base-net
redis:
image: redis:4.0.9-alpine
container_name: mainflux-redis
es-redis:
image: redis:5.0-alpine
container_name: mainflux-es-redis
restart: on-failure
networks:
- mainflux-base-net
volumes:
- mainflux-es-redis-volume:/data
mqtt-redis:
image: redis:5.0-alpine
container_name: mainflux-mqtt-redis
restart: on-failure
networks:
- mainflux-base-net
volumes:
- mainflux-mqtt-redis-volume:/data
mqtt-adapter:
image: mainflux/mqtt:latest
@@ -189,18 +222,21 @@ services:
depends_on:
- things
- nats
- redis
- mqtt-redis
restart: on-failure
environment:
MF_MQTT_ADAPTER_LOG_LEVEL: debug
MF_MQTT_ADAPTER_PORT: 1883
MF_MQTT_ADAPTER_WS_PORT: 8880
MF_MQTT_ADAPTER_REDIS_HOST: redis
MF_NATS_URL: nats://nats:4222
MF_THINGS_URL: things:8183
MF_MQTT_ADAPTER_LOG_LEVEL: ${MF_MQTT_ADAPTER_LOG_LEVEL}
MF_MQTT_INSTANCE_ID: mqtt-adapter-1
MF_MQTT_ADAPTER_PORT: ${MF_MQTT_ADAPTER_PORT}
MF_MQTT_ADAPTER_WS_PORT: ${MF_MQTT_ADAPTER_WS_PORT}
MF_MQTT_ADAPTER_REDIS_HOST: mqtt-redis
MF_MQTT_ADAPTER_ES_HOST: es-redis
MF_NATS_URL: ${MF_NATS_URL}
MF_THINGS_URL: things:${MF_THINGS_AUTH_GRPC_PORT}
MF_JAEGER_URL: ${MF_JAEGER_URL}
ports:
- 1883:1883
- 8880:8880
- ${MF_MQTT_ADAPTER_PORT}:${MF_MQTT_ADAPTER_PORT}
- ${MF_MQTT_ADAPTER_WS_PORT}:${MF_MQTT_ADAPTER_WS_PORT}
networks:
- mainflux-base-net
@@ -212,11 +248,23 @@ services:
- nats
restart: on-failure
environment:
MF_COAP_ADAPTER_LOG_LEVEL: debug
MF_COAP_ADAPTER_PORT: 5683
MF_NATS_URL: nats://nats:4222
MF_THINGS_URL: things:8183
MF_COAP_ADAPTER_LOG_LEVEL: ${MF_COAP_ADAPTER_LOG_LEVEL}
MF_COAP_ADAPTER_PORT: ${MF_COAP_ADAPTER_PORT}
MF_NATS_URL: ${MF_NATS_URL}
MF_THINGS_URL: things:${MF_THINGS_AUTH_GRPC_PORT}
MF_JAEGER_URL: ${MF_JAEGER_URL}
ports:
- 5683:5683/udp
- ${MF_COAP_ADAPTER_PORT}:${MF_COAP_ADAPTER_PORT}/udp
networks:
- mainflux-base-net
ui:
image: mainflux/ui:latest
container_name: mainflux-ui
restart: on-failure
ports:
- ${MF_UI_PORT}:${MF_UI_PORT}
networks:
- mainflux-base-net
environment:
MF_UI_PORT: ${MF_UI_PORT}
+9
View File
@@ -0,0 +1,9 @@
#!/bin/ash
if [ -n "$UI_PORT" ]; then
sed -e "s/UI_PORT/$UI_PORT/" /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf
else
sed -e "s/UI_PORT/3000/" /etc/nginx/nginx.conf.template > /etc/nginx/nginx.conf
fi
exec nginx -g "daemon off;"
@@ -1,16 +1,11 @@
###
# Mainflux NGINX Conf
#
# Taken for /etc/nginx/nginx.conf on Debian machine
# and https://github.com/nginxinc/docker-nginx/blob/master/mainline/alpine/nginx.conf
###
# Copyright (c) 2018
# Mainflux
#
# SPDX-License-Identifier: Apache-2.0
#
# This is the default Mainflux NGINX configuration.
##
# User:
# - 'www-data' on Debian
# - 'nginx' on Alpine
##
#user www-data;
user nginx;
worker_processes auto;
pid /run/nginx.pid;
@@ -18,77 +13,37 @@ include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
###
# HTTP
###
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Virtual Host Configs
##
# HTTPS
server {
listen 80 default_server;
listen [::]:80 default_server;
# SSL configuration
#
listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;
#
# Note: You should disable gzip for SSL traffic.
# See: https://bugs.debian.org/773332
#
# Read up on ssl_ciphers to ensure a secure configuration.
# See: https://bugs.debian.org/765782
#
# Self signed certs generated by the ssl-cert package
# Don't use them in a production server!
#
# include snippets/snakeoil.conf;
# Certificates
# These paths are set to its default values as
# a volume in the docker/docker-compose.yml file.
ssl_certificate /etc/ssl/certs/mainflux-server.crt;
ssl_certificate_key /etc/ssl/private/mainflux-server.key;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
# from https://cipherli.st/
# and https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
@@ -98,15 +53,12 @@ http {
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
# Disable preloading HSTS for now. You can use the commented out header line that includes
# the "preload" directive if you understand the implications.
#add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header Access-Control-Allow-Origin '*';
add_header Access-Control-Allow-Methods '*';
add_header Access-Control-Allow-Headers "*";
add_header Access-Control-Allow-Headers '*';
server_name localhost;
@@ -200,8 +152,8 @@ http {
return 200;
}
}
# Proxy pass to mainflux-mqtt-adapter
# Proxy pass to mainflux-mqtt-adapter over WS
location /mqtt {
proxy_redirect off;
proxy_set_header Host $host;
@@ -230,8 +182,7 @@ http {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://ui:3000/;
proxy_pass http://ui:UI_PORT/;
# Allow OPTIONS method CORS
if ($request_method = OPTIONS ) {
add_header Content-Length 0;
@@ -244,21 +195,17 @@ http {
# MQTT
stream {
# MQTT
server {
listen 8883 ssl;
listen [::]:8883 ssl;
# Certificates
# These paths are set to its default values as
# a volume in the docker/docker-compose.yml file.
ssl_certificate /etc/ssl/certs/mainflux-server.crt;
ssl_certificate_key /etc/ssl/private/mainflux-server.key;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
# from https://cipherli.st/
# and https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
+259
View File
@@ -0,0 +1,259 @@
#
# Copyright (c) 2018
# Mainflux
#
# SPDX-License-Identifier: Apache-2.0
#
# This is the Mainflux NGINX configuration for mututal authentication based on X.509 certifiactes.
user nginx;
worker_processes auto;
pid /run/nginx.pid;
load_module /etc/nginx/modules/ngx_stream_js_module.so;
load_module /etc/nginx/modules/ngx_http_js_module.so;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
js_include authorization.js;
js_set $auth_key setKey;
server {
listen 80 default_server;
listen [::]:80 default_server;
listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;
# These paths are set to its default values as
# a volume in the docker/docker-compose.yml file.
ssl_certificate /etc/ssl/certs/mainflux-server.crt;
ssl_certificate_key /etc/ssl/private/mainflux-server.key;
ssl_client_certificate /etc/ssl/certs/ca.crt;
ssl_verify_client optional;
ssl_verify_depth 2;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_tickets off;
ssl_stapling off;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header Access-Control-Allow-Origin '*';
add_header Access-Control-Allow-Methods '*';
add_header Access-Control-Allow-Headers '*';
server_name localhost;
# Proxy pass to users service
location ~ ^/(users|tokens) {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://users:8180;
# Allow OPTIONS method CORS
if ($request_method = OPTIONS ) {
add_header Content-Length 0;
add_header Content-Type text/plain;
return 200;
}
}
# Proxy pass to things service
location ~ ^/(things|channels) {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
add_header Access-Control-Expose-Headers Location;
proxy_pass http://things:8182;
# Allow OPTIONS method CORS
if ($request_method = OPTIONS ) {
add_header Content-Length 0;
add_header Content-Type text/plain;
return 200;
}
}
location /version {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://things:8182;
# Allow OPTIONS method CORS
if ($request_method = OPTIONS ) {
add_header Content-Length 0;
add_header Content-Type text/plain;
return 200;
}
}
# Proxy pass to mainflux-http-adapter
location /http/ {
if ($ssl_client_verify != SUCCESS) {
return 403;
}
if ($auth_key = '') {
return 403;
}
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Authorization $auth_key;
proxy_pass http://http-adapter:8185/;
# Allow OPTIONS method CORS
if ($request_method = OPTIONS ) {
add_header Content-Length 0;
add_header Content-Type text/plain;
return 200;
}
}
# Proxy pass to mainflux-ws-adapter
location /ws/ {
if ($ssl_client_verify != SUCCESS) {
return 403;
}
if ($auth_key = '') {
return 403;
}
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_connect_timeout 7d;
proxy_send_timeout 7d;
proxy_read_timeout 7d;
proxy_pass http://ws-adapter:8186/;
# Allow OPTIONS method CORS
if ($request_method = OPTIONS ) {
add_header Content-Length 0;
add_header Content-Type text/plain;
return 200;
}
}
# Proxy pass to mainflux-mqtt-adapter over WS
location /mqtt {
if ($ssl_client_verify != SUCCESS) {
return 403;
}
if ($auth_key = '') {
return 403;
}
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_connect_timeout 7d;
proxy_send_timeout 7d;
proxy_read_timeout 7d;
proxy_pass http://mqtt-adapter:8880/;
# Allow OPTIONS method CORS
if ($request_method = OPTIONS ) {
add_header Content-Length 0;
add_header Content-Type text/plain;
return 200;
}
}
location / {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://ui:UI_PORT/;
# Allow OPTIONS method CORS
if ($request_method = OPTIONS ) {
add_header Content-Length 0;
add_header Content-Type text/plain;
return 200;
}
}
}
error_log info.log info;
error_log error.log error;
error_log warn.log warn;
}
# MQTT
stream {
js_include authorization.js;
server {
listen 8883 ssl;
listen [::]:8883 ssl;
# These paths are set to its default values as
# a volume in the docker/docker-compose.yml file.
ssl_certificate /etc/ssl/certs/mainflux-server.crt;
ssl_certificate_key /etc/ssl/private/mainflux-server.key;
ssl_client_certificate /etc/ssl/certs/ca.crt;
ssl_verify_client on;
ssl_verify_depth 2;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_tickets off;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
js_preread authenticate;
proxy_pass mqtt-adapter:1883;
}
}
error_log info.log info;
error_log error.log error;
error_log warn.log warn;
+42
View File
@@ -0,0 +1,42 @@
CRT_LOCATION = certs
KEY = default
O = Mainflux
OU = mainflux
EA = info@mainflux.com
CN = localhost
CRT_FILE_NAME = thing
all: clean_certs ca server_crt
# CA name and key is "ca".
ca:
openssl req -newkey rsa:2048 -x509 -nodes -sha512 -days 1095 \
-keyout $(CRT_LOCATION)/ca.key -out $(CRT_LOCATION)/ca.crt -subj "/CN=localhost/O=Mainflux/OU=IoT/emailAddress=info@mainflux.com"
# Server cert and key name is "mainflux-server".
server_cert:
# Create mainflux server key and CSR.
openssl req -new -sha256 -newkey rsa:4096 -nodes -keyout $(CRT_LOCATION)/mainflux-server.key \
-out $(CRT_LOCATION)/mainflux-server.csr -subj "/CN=$(CN)/O=$(O)/OU=$(OU)/emailAddress=$(EA)"
# Sign server CSR.
openssl x509 -req -days 1000 -in $(CRT_LOCATION)/mainflux-server.csr -CA $(CRT_LOCATION)/ca.crt -CAkey $(CRT_LOCATION)/ca.key -CAcreateserial -out $(CRT_LOCATION)/mainflux-server.crt
# Remove CSR.
rm $(CRT_LOCATION)/mainflux-server.csr
thing_cert:
# Create mainflux server key and CSR.
openssl req -new -sha256 -newkey rsa:4096 -nodes -keyout $(CRT_LOCATION)/$(CRT_FILE_NAME).key \
-out $(CRT_LOCATION)/$(CRT_FILE_NAME).csr -subj "/CN=$(KEY)/O=$(O)/OU=$(OU)/emailAddress=$(EA)"
# Sign client CSR.
openssl x509 -req -days 730 -in $(CRT_LOCATION)/$(CRT_FILE_NAME).csr -CA $(CRT_LOCATION)/ca.crt -CAkey $(CRT_LOCATION)/ca.key -CAcreateserial -out $(CRT_LOCATION)/$(CRT_FILE_NAME).crt
# Remove CSR.
rm $(CRT_LOCATION)/$(CRT_FILE_NAME).csr
clean_certs:
rm -r $(CRT_LOCATION)/*.crt
rm -r $(CRT_LOCATION)/*.key
rm -r $(CRT_LOCATION)/*.srl
+176
View File
@@ -0,0 +1,176 @@
var clientKey = '';
// Check certificate MQTTS.
function authenticate(s) {
if (!s.variables.ssl_client_s_dn || !s.variables.ssl_client_s_dn.length ||
!s.variables.ssl_client_verify || s.variables.ssl_client_verify != "SUCCESS") {
s.deny();
return
}
s.on('upload', function (data) {
if (data == '') {
return;
}
var packet_type_flags_byte = data.codePointAt(0);
// First MQTT packet contain message type and flags. CONNECT message type
// is encoded as 0001, and we're not interested in flags, so only values
// 0001xxxx (which is between 16 and 32) should be checked.
if (packet_type_flags_byte < 16 || packet_type_flags_byte >= 32) {
s.off('upload');
s.allow();
return;
}
if (clientKey === '') {
clientKey = parseCert(s.variables.ssl_client_s_dn, 'CN');
}
var pass = parsePackage(s, data);
if (!clientKey.length || pass !== clientKey) {
s.error('Cert CN (' + clientKey + ') does not match client password');
s.off('upload')
s.deny();
return;
}
s.off('upload');
s.allow();
})
}
function parsePackage(s, data) {
// An explanation of MQTT packet structure can be found here:
// https://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#msg-format.
// CONNECT message is explained here:
// https://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html#connect.
/*
0 1 2 3
7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TYPE | RSRVD | REMAINING LEN | PROTOCOL NAME LEN |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PROTOCOL NAME |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
| VERSION | FLAGS | KEEP ALIVE |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
| Payload (if any) ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
First byte with remaining length represents fixed header.
Remaining Length is the length of the variable header (10 bytes) plus the length of the Payload.
It is encoded in the manner described here:
http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/errata01/os/mqtt-v3.1.1-errata01-os-complete.html#_Toc442180836.
Connect flags byte looks like this:
| 7 | 6 | 5 | 4 3 | 2 | 1 | 0 |
| Username Flag | Password Flag | Will Retain | Will QoS | Will Flag | Clean Session | Reserved |
The payload is determined by the flags and comes in this order:
1. Client ID (2 bytes length + ID value)
2. Will Topic (2 bytes length + Will Topic value) if Will Flag is 1.
3. Will Message (2 bytes length + Will Message value) if Will Flag is 1.
4. User Name (2 bytes length + User Name value) if User Name Flag is 1.
5. Password (2 bytes length + Password value) if Password Flag is 1.
This method extracts Password field.
*/
// Extract variable length header. It's 1-4 bytes. As long as continuation byte is
// 1, there are more bytes in this header. This algorithm is explained here:
// http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/errata01/os/mqtt-v3.1.1-errata01-os-complete.html#_Toc442180836
var len_size = 1;
for (var remaining_len = 1; remaining_len < 5; remaining_len++) {
if (data.codePointAt(remaining_len) > 128) {
len_size += 1;
continue;
}
break;
}
// CONTROL(1) + MSG_LEN(1-4) + PROTO_NAME_LEN(2) + PROTO_NAME(4) + PROTO_VERSION(1)
var flags_pos = 1 + len_size + 2 + 4 + 1;
var flags = data.codePointAt(flags_pos);
// If there are no username and password flags (11xxxxxx), return.
if (flags < 192) {
s.error('MQTT username or password not provided');
return '';
}
// FLAGS(1) + KEEP_ALIVE(2)
var shift = flags_pos + 1 + 2;
// Number of bytes to encode length.
var len_bytes_num = 2;
// If Wil Flag is present, Will Topic and Will Message need to be skipped as well.
var shift_flags = 196 <= flags ? 5 : 3;
var len_msb, len_lsb, len;
for (var i = 0; i < shift_flags; i++) {
len_msb = data.codePointAt(shift).toString(16);
len_lsb = data.codePointAt(shift + 1).toString(16);
len = calcLen(len_msb, len_lsb);
shift += len_bytes_num;
if (i != shift_flags - 1) {
shift += len;
}
}
var password = data.substring(shift, shift + len);
return password;
}
// Check certificate HTTPS and WSS.
function setKey(r) {
if (clientKey === '') {
clientKey = parseCert(r.variables.ssl_client_s_dn, 'CN');
}
var auth = r.headersIn['Authorization'];
if (auth && auth.length && auth != clientKey) {
r.error('Authorization header does not match certificate');
return '';
}
if (r.uri.startsWith('/ws') && (!auth || !auth.length)) {
var a;
for (a in r.args) {
if (a == 'authorization' && r.args[a] === clientKey) {
return clientKey;
}
}
r.error('Authorization param does not match certificate');
return '';
}
return clientKey;
}
function calcLen(msb, lsb) {
if (lsb < 2) {
lsb = '0' + lsb;
}
return parseInt(msb + lsb, 16);
}
function parseCert(cert, key) {
if (cert.length) {
var pairs = cert.split(',');
for (var i = 0; i < pairs.length; i++) {
var pair = pairs[i].split('=');
if (pair[0].toUpperCase() == key) {
return pair[1];
}
}
}
return '';
}
+20 -19
View File
@@ -1,21 +1,22 @@
-----BEGIN CERTIFICATE-----
MIIDhDCCAmygAwIBAgIJAK61I+11kc4nMA0GCSqGSIb3DQEBDQUAMFcxEjAQBgNV
BAMMCWxvY2FsaG9zdDERMA8GA1UECgwITWFpbmZsdXgxDDAKBgNVBAsMA0lvVDEg
MB4GCSqGSIb3DQEJARYRaW5mb0BtYWluZmx1eC5jb20wHhcNMTgxMTAyMTc1MTEx
WhcNMzIxMDI5MTc1MTExWjBXMRIwEAYDVQQDDAlsb2NhbGhvc3QxETAPBgNVBAoM
CE1haW5mbHV4MQwwCgYDVQQLDANJb1QxIDAeBgkqhkiG9w0BCQEWEWluZm9AbWFp
bmZsdXguY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAuTSZgdq0
BRjxYCNZV4QOAoN9eaoXSp5yD0DjdBfP4T5HZJYMmw1ca/EeO9raO1fOvAQ7CsCq
thKb8pw9HgPBLBL8GACTK/x2rjco9c2TbgTk7f6vSVoI+UUiU2Y5g4iIPIywYoGD
/WwCn38r7NyKxxzbEstnJ2TvcvoOBXTc0EfLJ3gox1Q+FlhYQ928FFSv78tWeXug
0X2KJKY8qUrXF1LDVLwv/93XduN+nGMJfoAYJQXRVmcPLtMFEycD90xUkKIPOPox
sSQwQl9RaM7r3M2ZsEirJpeJbvzVBxLdWWaG8HqT9oRZPrJZZvcpFgzX1Yf5oi2k
GMzSsYxJBnVsqwIDAQABo1MwUTAdBgNVHQ4EFgQUMmE/LCiFwPSKXDSAQvcRF/9y
M54wHwYDVR0jBBgwFoAUMmE/LCiFwPSKXDSAQvcRF/9yM54wDwYDVR0TAQH/BAUw
AwEB/zANBgkqhkiG9w0BAQ0FAAOCAQEACmbTBIq8cZebTa+IE8zUAj8KpaGLCn+7
nET6DYQzT1GoGToMVOdQ0goaGAGMhTGh8ezOxAPJoo3IYZwSErxSpyd20jASKkQG
p2Q+gBDZiohEumQkA2K6ywgTrVr/qNhGBvv+r40h3lJd2bbspfPLUq2zNnJpRhww
0QjObMnaDdXgD8kPy7poEUVmGxAYKhSBvi7gNInymaspGGwubNVrsakAjsi710r1
41KT4Pq4FpfHzqpSrrGq4VFbi1NSUZWGCIqIm+oYlA5l7/cMVPS7qtV/ScsMod8s
KSkNneFU0RqKeY7dMU2bkxlCcH+xUAmWefK9WFvwBJ4HjxE0Q83qPg==
MIIDjzCCAnegAwIBAgIUQ1AagVQXCuOIzmGXm+KhsbyBc18wDQYJKoZIhvcNAQEN
BQAwVzESMBAGA1UEAwwJbG9jYWxob3N0MREwDwYDVQQKDAhNYWluZmx1eDEMMAoG
A1UECwwDSW9UMSAwHgYJKoZIhvcNAQkBFhFpbmZvQG1haW5mbHV4LmNvbTAeFw0x
OTA0MDEwOTI3MDFaFw0yMjAzMzEwOTI3MDFaMFcxEjAQBgNVBAMMCWxvY2FsaG9z
dDERMA8GA1UECgwITWFpbmZsdXgxDDAKBgNVBAsMA0lvVDEgMB4GCSqGSIb3DQEJ
ARYRaW5mb0BtYWluZmx1eC5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQCq6O4PHwgGOmEafjea5KocG80GYSYbvN37ums6fQ1wcmCxn8LtZek8WkfJ
S2NQQPDvn8QWRY7aUkTAW7cEB4vxpT25bevP7KJNFAS8XZO7NTfF8fscJS+YWSXz
VS0OFZ2YuqTnjCiqWf5mvjAkkXBGIYq+k2ONM1tHlEA0lzbLun2a9H/XarCG+znj
pfYpW6R08zFzXyGb4sI2pyYpP7iZLla7PTSZTt9h6jkY3qqMDhEHhPdlXDhO1O9/
lA8yWMO9vKCzC7ngDXnV99Nl+tFhp9z9VkTUveLMuN9+riDJRfP25fOzHuRYzmsR
emYjD1NvSgsvFqSbFDVXB8kcyrXPAgMBAAGjUzBRMB0GA1UdDgQWBBRs4xR91qEj
NRGmw391xS7x6Tc+8jAfBgNVHSMEGDAWgBRs4xR91qEjNRGmw391xS7x6Tc+8jAP
BgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBDQUAA4IBAQAAPMf7bVFhzUG8AYq0
VS9BWVwVtdNzZ3X9FkG9O+tZZO43GlaToym8PmhJHF9wk3AA+pmgfcmBrHcTG0me
PeincN2euO0c4iv1f/i4bAY5/iq/Q0w/GiuTL5VLVpaH1SQrWhc0ZD7Ii+lVPpFQ
bJXKHFQBnZU7mWeQnL9W1SVhWfsSKShBkAEUeGXo3YMC7nYsFJkl/heC3sYqfrW4
7fq80u+TU6HjGetSAWKacae7eeNmprMn0lFw2VqPQG3M4M0l9pEfcrRygOAnqNKO
aNi2UYKBla3XeDjObovOsXRScTKmJZwJ/STJlu+x5UAwF34ZBJy0O2qdd+kOxAhj
5Yq2
-----END CERTIFICATE-----
+28
View File
@@ -0,0 +1,28 @@
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCq6O4PHwgGOmEa
fjea5KocG80GYSYbvN37ums6fQ1wcmCxn8LtZek8WkfJS2NQQPDvn8QWRY7aUkTA
W7cEB4vxpT25bevP7KJNFAS8XZO7NTfF8fscJS+YWSXzVS0OFZ2YuqTnjCiqWf5m
vjAkkXBGIYq+k2ONM1tHlEA0lzbLun2a9H/XarCG+znjpfYpW6R08zFzXyGb4sI2
pyYpP7iZLla7PTSZTt9h6jkY3qqMDhEHhPdlXDhO1O9/lA8yWMO9vKCzC7ngDXnV
99Nl+tFhp9z9VkTUveLMuN9+riDJRfP25fOzHuRYzmsRemYjD1NvSgsvFqSbFDVX
B8kcyrXPAgMBAAECggEAbp/el0MKup1HBRL1gvjHcvI7vwla1VFmje2YQn93F3Wx
SMeUMH1qfnohRRXa7rNaQIA1OAVF9eKSRcAXsjAAUSUX0tJndGpCk4mFlzcqzF4h
/6olU45uRDpP6jUTuK4dGCKXYpjCKaGenXo1RzYsafiECd707Qx05Nv8ww2tlifN
HtUR0xCZfVGDZfmNMZVrksUIZ1XHwZNtNLWQW6MBl3RhFaA0Wz/RfFMi2FzacEbj
75IqE6PLic1fin6P3GouzKamtZ6YPTyR5PqxCOCw97oZDCUGy2qGyAuPUi9O2HKB
fQgSyIxuR73S2korvxAmvekubjBFAqhan2oEjZs6oQKBgQDT28COlC33BSrpr2+V
pZIL4Bb1rGHreTi1M/4n9nP3GOZ9gqnSUsWXyxYVoZ2YfixorjZhUzHyx4SfZ2E9
p5PkIJ0wOiHLlKQ36vEVN9ZO1UyNCYUgs3seW40xnsAiMNczZjufIZrsejO3tc2j
Jhgp+B/9Bt5A8us2ewhz3LlQowKBgQDOhQmZAfL/xAjYBCUS73t/YO60i5e1yg2J
i6jXeKjd5gRZ32upkBzQ8UBvAGSQGqrcCnqIzrU5TeeD046bZzkokg7iKwHwQDrL
SXTthUB6ABZddP/VXCEUVBer3FEnUgJm9jw08RzmPyNEPjfp91FDmJ9GYcbdo/nL
hBPHh3lc5QKBgQCJYZ0yWACeiKlVNECFqAJW1Q/Oa+RrkAYn6vlK7NQyTeFZTlvV
WXtsfXNqv4y0kE037JCy+AIRzzO/MoiqNHsAme2Ukn3LyC3dXOrMuZKtOEAVzTCZ
Dgoum2up26n4AffrCsZq4J3X7z6OSMR6oX9V5+LGb6e8Mko43/uRNnatRQKBgEMH
bQkLV+ppnxE1ry7JKcU7Gd7hm9j1/pTRDnj5AZ4b5Peii1ganS+3zdj5QKqA7UnD
4Od8Z9d0kJr51EReKXAgj9IacWOgBTUr31akNDwkwR2ONubyIw5tCM3QEUr41CzE
6N+qDl4wyeqBYzZ9/hM5eyCl5ZzUduP2N1FAiER9AoGAW2T0OeM5ZsPABMKu9eEN
FB9bVysqWT1tExB34OGWrZvNEzsHTqvr/D3KSWv0PS1pM46M1XkVbybOzRmPrzab
AGMDJXgGhMuk2UtDA/s9mgqTOeDXpvmaFyThVkoH162j6GMuX2SwxHnH9D42zgMR
3LEZ/5Q5HMJ4jwEM880jvP4=
-----END PRIVATE KEY-----
+1
View File
@@ -0,0 +1 @@
27207EA9519D3D252E08AFA38D23BF2928FD5E20
+22 -26
View File
@@ -1,29 +1,25 @@
-----BEGIN CERTIFICATE-----
MIIFDDCCA/SgAwIBAgIJANUB7oWmhqk8MA0GCSqGSIb3DQEBDQUAMFcxEjAQBgNV
BAMMCWxvY2FsaG9zdDERMA8GA1UECgwITWFpbmZsdXgxDDAKBgNVBAsMA0lvVDEg
MB4GCSqGSIb3DQEJARYRaW5mb0BtYWluZmx1eC5jb20wHhcNMTgxMTAyMTc1MzQ0
WhcNMzIxMDI5MTc1MzQ0WjBdMRgwFgYDVQQDDA9tYWluZmx1eC1zZXJ2ZXIxETAP
BgNVBAoMCE1haW5mbHV4MQwwCgYDVQQLDANJb1QxIDAeBgkqhkiG9w0BCQEWEWlu
Zm9AbWFpbmZsdXguY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA
ovtHVZacXvf5m7iyhLdGqBaoTwUMSWR+LDaHyeUAcqLy4AHe2TGdJmbrsSKJOb94
Winru0dgQyBE1a/I+MOPYVNbYmShOoUSH6/a2Ph3DM8C7PjaFsc+uKd6NiVRmFid
c+3pQ62Um9mgJ5jD6kFB2uO5Bk9zxlDRGZz2BYvMFGcbhDZzf28O/Wwitfjb3dek
8kohknaIgHy50qstxt/GRFVHpqK0B+PVubC3Dr1Ext+lZORqM44o36jdUyVhnFXf
f8Fj/g4whrJfq3AOHeMsxm1VLqKeQb9CxpWvUi396w/bEXOuKXyoP5hyMUAvXEoE
pw2+M/CPHuCzj/ELafjWjQIDAQABo4IB0zCCAc8wDAYDVR0TAQH/BAIwADARBglg
hkgBhvhCAQEEBAMCBkAwCwYDVR0PBAQDAgXgMCoGCWCGSAGG+EIBDQQdFhtNYWlu
Zmx1eCBTZXJ2ZXIgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFDRwdZDGidKOpuJ29hXz
RWRFD8YkMIGIBgNVHSMEgYAwfoAUMmE/LCiFwPSKXDSAQvcRF/9yM56hW6RZMFcx
MIIEOjCCAyICFCcgfqlRnT0lLgivo40jvyko/V4fMA0GCSqGSIb3DQEBCwUAMFcx
EjAQBgNVBAMMCWxvY2FsaG9zdDERMA8GA1UECgwITWFpbmZsdXgxDDAKBgNVBAsM
A0lvVDEgMB4GCSqGSIb3DQEJARYRaW5mb0BtYWluZmx1eC5jb22CCQCutSPtdZHO
JzA7BgNVHREENDAyhwR/AAABhxAAAAAAAAAAAAAAAAAAAAABggV1c2Vyc4IGdGhp
bmdzgglsb2NhbGhvc3QwgYsGA1UdIASBgzCBgDB+BgMrBQgwdzAcBggrBgEFBQcC
ARYQaHR0cDovL2xvY2FsaG9zdDBXBggrBgEFBQcCAjBLMA8WCE1haW5mbHV4MAMC
AQEaOFRoaXMgQ0EgaXMgZm9yIGEgbG9jYWwgTWFpbmZsdXggc2VydmVyIGluc3Rh
bGxhdGlvbiBvbmx5MA0GCSqGSIb3DQEBDQUAA4IBAQAwrfzfLLe35aTGel9tWpWi
aWSATgYThCzaFqqzsQBNAmB/S+06xI2JSeXtHOestsqLZOrWdGG6LJcRiyCR/XWv
SxDRPUgjp14YQCml8GWQLairhdXNWZch1d+Bzr2XkJrTzQUex/XCJQnp56CzjFUO
XADhQdiaESvu/tk7nRuX8qYqwyIqwzRm5KlqHJIvsDXddGluS5EtsshdtAwbRQdR
jK1egJ3Z26vn68zrZiPQOYz9gmJs2Zl71bd4cGEBa6m7RGi9ww7gU6TsMoE8BTDg
i2ia+MB+COQB8ISx1+Pyx2migImZlnlYfSup1ObboyJV2jdqROBWbaosHqhh/7VF
A0lvVDEgMB4GCSqGSIb3DQEJARYRaW5mb0BtYWluZmx1eC5jb20wHhcNMTkwNDAx
MDkyNzA1WhcNMjExMjI2MDkyNzA1WjBcMRIwEAYDVQQDDAlsb2NhbGhvc3QxETAP
BgNVBAoMCE1haW5mbHV4MREwDwYDVQQLDAhtYWluZmx1eDEgMB4GCSqGSIb3DQEJ
ARYRaW5mb0BtYWluZmx1eC5jb20wggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK
AoICAQD1LayMnOYlTHWkK/7BIc2nRLkfkbfyejIujEKIuOPYZ1DbG36VeSM1MYlp
zl+E7gJvvK1RuCcL4DKG0uExI6HV2GdEq9kSe3Pj/512VTq+bXvMDRTcHMUkaN0N
J7GybHNk8J3fmFlB61zUpZUNy0M14YYX8tZRMMw8Ke6ThJyj8ulSky4Cp2tfiGK9
+YLP/UJkSm+0EOVAOMAtLNvXtg5+/0e63M+stdf+F3txLuiYXiOG399tXlI61r4L
5fKs0xau6P1V5uEPwAnQiXYVLCdahfGrUJIjHnHTU0TS2EpE8OxAu0krzQeONGSU
g6SMM8vCP0d8yqQrYZGkmaFmIiTgOmy/fs+8u/ykautiOR/SviTR3hi/ofjZ+NTd
T2Udg98BGuZBwKw+elajHUSUEkxtJVxeuFiVGzZNXkEhuxU6VNCnPeXxtl502rU9
nmhmO2WJ0/1KX+oe/uTC99b+olEPm72exsX0mwkSpIwDRBpX9meER4vJe4yX9fmo
tqEC2G30C9KYn+STcY9P7jptJgLLuN61DVBjeMPLW+0NTjqmtplcu73zYvyCsG4r
hIhY291wvz18iNLY7BfehU3beEx68ApdLMue6xi9JlFKxHf5FHBnBSvD2xrR47rH
9UMOHLglB+QkoidQ3KugHJ8r1sVHPhuS8mE7cENReFoNfh+N2wIDAQABMA0GCSqG
SIb3DQEBCwUAA4IBAQARH5ZD86TPaKW7Dty1bAnj1owp0o+DOp65hGZOZ2AqYVDF
UMz46ahAuBWhHPIiSkBnonBL5xVV3qihhlISaOQKe2FPdt/ekhUTzI/upAZDphN0
m4ZNllXaHAA0IQpXp3O/An6/IhrLCGLth9pnIzswi6sF+I5nIfpcuAV7TJfLUAG+
UTjy8GsZhE/ZCx0JSYzhpC1mDGxtyCQR7QY7rnEohXv0bHmv/jVVIZenT2SZZHJ5
sQEiaIZWbpHctpgbom1qi5BNmIz9APKus3f8ACGuMLOHiW1u6I8vl4b1kqc44Qoe
2c5uGEHh+Iv6v/V5JwzTrfbcaWeAv058NnN9rF8i
-----END CERTIFICATE-----
+52 -27
View File
@@ -1,27 +1,52 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEAovtHVZacXvf5m7iyhLdGqBaoTwUMSWR+LDaHyeUAcqLy4AHe
2TGdJmbrsSKJOb94Winru0dgQyBE1a/I+MOPYVNbYmShOoUSH6/a2Ph3DM8C7Pja
Fsc+uKd6NiVRmFidc+3pQ62Um9mgJ5jD6kFB2uO5Bk9zxlDRGZz2BYvMFGcbhDZz
f28O/Wwitfjb3dek8kohknaIgHy50qstxt/GRFVHpqK0B+PVubC3Dr1Ext+lZORq
M44o36jdUyVhnFXff8Fj/g4whrJfq3AOHeMsxm1VLqKeQb9CxpWvUi396w/bEXOu
KXyoP5hyMUAvXEoEpw2+M/CPHuCzj/ELafjWjQIDAQABAoIBACZM2KmjWCH1MkJG
+CS6ES+lC25uaEHDDOeE9Qob96v44QyFRAJUZ3LD0vVwgUEB3t2JZm0/S77akXKJ
+beD9WjQtvP7Y+wlo2mw4MQGN6vZ5f0cSdv6fKHWMaERygf8IxxAN4YA/6BUVw+X
FcsyBLDUvQFfoHxlN45GVYZeINpWNJEM5NkQpoz5JJmWplfRREtRDiFfdFvh8l25
nOeYaZkD9IAPW5k7ukzzUM4voNOKoutCV3h30AbD8zNMPblatLTrT9ZRtjEiofDn
iNT6DR2Ge4WWN8e+9h9CEklhhCwBULNlCg2mdNKsiU3q74oBExiKNrtJGxZXlhny
PIpciL0CgYEA01HmmXG1z82QmcvoMZBoHu9wNNl5krt5YsYUF9TWXTPijw6zxnLB
b0ef997KzMt1ZGRSq8Zae6YdMRXCvQJh1SMbhBTMfOwzbx+cqCcInmia47jAXgaV
agAc7im3gULeTnXtNRdo9cj2ilHmlJNFI0N0rl9NJxpYACEeSFaR6YMCgYEAxXD4
Z0+Boo+wsIvcxYWkYWpvhJfqqdETZJ5RdgkWLFG7OA2SWPgqIyWxPTnQRsK5ryjw
xyxNv7L9Ddd/+yCNsiuUIEdhLPtoIfBQ96722IJMTOxr9eWUXVngANBeMUypv/3q
v3AeD8MfTgJJPZ5F3zfLAeBPUxUx6JUIRxjFEq8CgYEAh+3q01EdjinAle1f0mH9
V8jU+GnbldMZ5btcOWgi65jwZu0iHsi6PIZqE9svwEQvowAVXYEduuPDa0uAFGxv
2dXXYUKvtruI6jX/YvqYxKys1UaGFvVNLv4bnecfrvoAXZ/lkX0ZeuBmFdtQ4slI
c8J0T6Xlzv1XFd43yHhHK1kCgYEAwwc0V9hRVSJahgmhqfq4xQE95tupENVVMq6w
CMgw/tY8+MFvLjL0bINu7+uLsFno0Py/2f4JTrKfBG06HfWqAbTKPJhFhQlRczTO
xdouOu96LwHaIqsEQrHkculgIZJ4mw1WNIOrLiavZX8cmaEdo8CY5uGLeaYWBogw
BQqSoEECgYEAuNXFrwDzUoqiFo9+7gjzT4pjQgh/zNTIODaXwIVBjTRe6Z4bQayd
Jel5Gf1IHu9iik6pvaVfK8tMc8eQisEC8F9U9l6mqw/Q4bpQ5k6CQtGe0roBEj2H
qJIQ/1TjMAOkOx7YiqTuFP4vs9LhCRAflQQ/Tg0fcHsMWBlPpWlEkRs=
-----END RSA PRIVATE KEY-----
-----BEGIN PRIVATE KEY-----
MIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQD1LayMnOYlTHWk
K/7BIc2nRLkfkbfyejIujEKIuOPYZ1DbG36VeSM1MYlpzl+E7gJvvK1RuCcL4DKG
0uExI6HV2GdEq9kSe3Pj/512VTq+bXvMDRTcHMUkaN0NJ7GybHNk8J3fmFlB61zU
pZUNy0M14YYX8tZRMMw8Ke6ThJyj8ulSky4Cp2tfiGK9+YLP/UJkSm+0EOVAOMAt
LNvXtg5+/0e63M+stdf+F3txLuiYXiOG399tXlI61r4L5fKs0xau6P1V5uEPwAnQ
iXYVLCdahfGrUJIjHnHTU0TS2EpE8OxAu0krzQeONGSUg6SMM8vCP0d8yqQrYZGk
maFmIiTgOmy/fs+8u/ykautiOR/SviTR3hi/ofjZ+NTdT2Udg98BGuZBwKw+elaj
HUSUEkxtJVxeuFiVGzZNXkEhuxU6VNCnPeXxtl502rU9nmhmO2WJ0/1KX+oe/uTC
99b+olEPm72exsX0mwkSpIwDRBpX9meER4vJe4yX9fmotqEC2G30C9KYn+STcY9P
7jptJgLLuN61DVBjeMPLW+0NTjqmtplcu73zYvyCsG4rhIhY291wvz18iNLY7Bfe
hU3beEx68ApdLMue6xi9JlFKxHf5FHBnBSvD2xrR47rH9UMOHLglB+QkoidQ3Kug
HJ8r1sVHPhuS8mE7cENReFoNfh+N2wIDAQABAoICAQDwIbfqUJGo3mYqUVzGVBFU
Tp7yKIp9VulnZcCUoGGoRiiPMTKdu3OcWdQ4aQRs8aA5SBaI/1Be9UsHeetNcZTE
GZurSpTk4Tz0hhr3Fyrd7+VcSUOxAgykSICYrdQA6O9sYa5+nHxvb9IQA80lIXvG
fggT1KfMBXtDQd6FZVD10qhrU/OwbcFgY/gyEPrqMyafi7g0KIgginTOyizX7Vvt
TqI2hqZwhfnyx5UDmen9sdYh94qhV8w6DLE+fg1c77I7xb66L9Mm1fPG55tbyU8R
/jZgkB1RgDQDwn6Z83VqaH08OTLFT56izPXl77luCBz9N3UQ6Hg6dOlsyXumyLJL
cKjZ3Yoaqu9GHEauiaIJqPX4bN6O0TjG/vW0yKdGSFgh7jfjIiYCmMnb526PolKM
YN4xZ/KcdGa4QGKuX0hfgYLaOAM4U+V2Flb6QiSSta+UAFPOC7d7fEZIwYrgieB9
10jnFqXqAef5w16Z1KZNI+X1FO5keHkOBqliXCdQJoLBwAt7nJFkgETatK5XCra1
WR/iVcanZrvwD7pITZXbROcOF9MxKxkAqxU8Xeftr47r5XteuqSd3uRmsa7034QV
0TNV2OUBv4UCa5DpEftSjoX5GQKKXkAFix3QdW+RqiZ4tGVnrCUShhgg9H3aOu2y
dePbf3F5R7P9g4SbfKFGAQKCAQEA/GVCzoihnM9j+DmP63xoNyWNGVQ3foeZw/o6
FvGYx5yUeYm+uwuq80hpd4KtHZoqbO3ohEsyZBHBwGjbmjxzwmrKbZukJ8Zxs7QQ
cB0YBmHduay59+MnmL3uiiLGsFedSlbReCJnFw+66yXfvDPVf4vs1I+DPtEIBkY5
mOOdfP7b4vQtQbhtw4EAPSORbcR9ap4DCRmedp9aWTh48VNvqGLJIxRxveHcX/F2
zAySGkw2s/pwQXq0htCComDn3X3yqw4y9WKirzmS5hYU4gsuk+dy3JSHhe0bsA8Y
daf4kEdutqyjYj1IaaDuvkiRW/6Pukb//R94/tejmwBLA5LXGwKCAQEA+K4HGruw
zlGKfQNc2uKX5uzB6N+rWQ//5oFhe/Lga2pQZlwDCay+3G9YWiVLP+wxEAkuoQcx
thfYFWLMy/8+Vyuiej/N73hWYXPgja7BA0d/j9/IKhtjvN2qIzA2xxKCkzqunXms
VnOoHVwhoqtVEZ9trB2gdO3ywE52aqSq131rVABOJDoFq9lVw4bl9Rj9wkp6D+tr
Dx1pMeKrBFKfxuCgAyj//BJrNSryxAxglAzyC75RKAPT6fvcw3Wcpnb10IXDt3Rd
g5YHMxas3g1fh2ieRVsQG4OvGytP9Uap6//AqM6c273Q86U3/pu3r8nvEMBKxdsx
pc4/raRsoUPQQQKCAQBNCBLFukmo9FsMjXTxaDzeZ+WSj3OIeJZji+Fi00XP1mgy
V+oQaFU6fyVBRm7TlBPSvyGyDslIZWr+8IHlpwGlmrZBkbkeMqDNOe2yag7FE+V6
H896aqfRJFbDbi258GOfJrQzuDxCe5iO4DZS2HcWwHv9u/dQmreaQqCdmwqb9aTi
taeCYWmOu7Z48nwWRlwIyEUg5+LHTYdjp6qx7MctW0kMHddBHsgFuEqLqGKHCC/B
6nOMaIjkhIr6SB08Ko5/youe/QWt+SJuetrQypzio0cZL3PVWjKTH2hVsHhagJK3
yiTrfMy3AFkdVkSXETCIp9bFSG/DR8k1K3e5lX11AoIBAAfSCT0o++VxIQbPbUMg
7x//ABYfupbBbw3DsdohCDe4jzC44guS2Cm8gq3LEHPBLMXRVBsSS9jrJQt/IOul
akN5htGLYiGOykCkUUKDZWSCAhv3MKdKVzegTPJwWLin911D8ivXoLjTSE0sEY65
DqLQPbW09M/Yj9LGZOjzpr/CHPb2T37KKFWALzdH7cFoeMp8ZxxLDgHare04sKIh
Kw8pDz8qMequdZqlcB8EOKFPSuldodW9URPBrO6kqzl88jwNiNsjGLHDrRRJOUR+
bSun+Zo6w+XpnT8gfJI9F6jpURi97qbmcETJRFqIcR1hH1iKg493VjddphkC27uy
k0ECggEAR1LyWFqTUxpP15EGA4vE64c0T6gmWmSyQiZ2VbMYWlgBCZJQN1EFDoFB
rLQvhy8jEU3zxbJPEOQmQL8OGzMBMgV/akEsTTEAPLbQc0ROSR7CW3YV8UlyBUP5
4/WK1NUR8GXyeCjJSWHgn/LclkcFmyJ5DCKmesMRAodMhkHkqmSTZxPAYmlipn87
PcGOoG02NlgjDADjwhRepRI7wVbb8HVXfGxhiPokAri/OgC/odnHIGzkdznur6JS
5eUoZkBcH16zBGxfwoGhqSGdip1BNNs2nUp4T0i+LGVKz8mYsZ9CdiGElfUwLpcc
eFet28DEMzSifudXY3LYE5N0Vl6g+A==
-----END PRIVATE KEY-----
+25
View File
@@ -0,0 +1,25 @@
-----BEGIN CERTIFICATE-----
MIIEODCCAyACFCcgfqlRnT0lLgivo40jvyko/V4gMA0GCSqGSIb3DQEBCwUAMFcx
EjAQBgNVBAMMCWxvY2FsaG9zdDERMA8GA1UECgwITWFpbmZsdXgxDDAKBgNVBAsM
A0lvVDEgMB4GCSqGSIb3DQEJARYRaW5mb0BtYWluZmx1eC5jb20wHhcNMTkwNDAx
MDkyNzEyWhcNMjEwMzMxMDkyNzEyWjBaMRAwDgYDVQQDDAdkZWZhdWx0MREwDwYD
VQQKDAhNYWluZmx1eDERMA8GA1UECwwIbWFpbmZsdXgxIDAeBgkqhkiG9w0BCQEW
EWluZm9AbWFpbmZsdXguY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC
AgEAuNyXOzUfhH4WOvXgaOIV+ViqXEXO/VfjxPBE4EOYxxk1S5N1tAM5STontEqJ
kv8sSInZkYPPPjcqv8yEh1q5GyZrsI3Th5wqovFQEKUwAP2m0bluJYwY0oqo8dcZ
vPhcMdEdJu1hzRI0LOiBv5EgkT6KT5z97CfDtV16uaVISywnQmo1TcY7tT0tQs2S
znc4kg6mbfGnbmkEHfiV5aOj4ahULd5grdPh+8YcnWXNE/maSZJLOKz2PIMuySS7
TmmmgJC6uMpsT5rCGI+WzNwYy8X6731zy/DVaMQ752QJfS0rUPwHa7KiEQNKAyrZ
NbYxiILKnbvd/JrN2iW6yllQDQn+XTft1aAui4fQcw/aBQZ5zbiedJeeDtKuve9+
X23PVhRumsI4Wfo4CzKBsdH6fT5oGOqmL8WFVCQl/p93vqPRbhadEeEqGgdE2om/
1SaA5nl7W4rbfo9beLpmi3KE+oldlLh5/mgH+7vWQQmmidC633AFaY7TabxU/59+
38Kzo6eAJauVoHFdXGCIgg/SemNS1KWo3t+pwBHJIPHdsLlWsRVtV5Vt8QW+MlD1
ODkApTstom0rtLvBoqBkI+2z29J+i07R4C2K/ZFdhv8Exf/MxUZeET+AznUwLHTE
SNxCsI/7wWQVyLVb1AwWLaBbt1cYd4YGVWe+QcslxNNayMkCAwEAATANBgkqhkiG
9w0BAQsFAAOCAQEAi7jvvUUMH2yVXfYgLUuBB8jRmwQcYKJo0jbPKZew07F+L3xM
WdYP+pDhdkyF79l99/fZS0Xs8dwYtAgU2tVkVoT6p/6vCvnqodgKgZJWi2dNCdG7
ftIJR9dkusHIy3cpSHNb+A/hYLvj1nY9IAmRiY1fBNrRflmQe73gUuIjuoqDQ8wV
5jteUUt33rH0wYhbMf4z9HFSDBK1Ti+Mw27ybDYnYb79FZjUnXAKR/Gb0QyyGQyI
N5sVboXyBEK6KlJ4xBQZ0gEvmhN0ZGgmje4u7+2E3pJxo3zRN8Qm5Poqyll+3Omd
3rPdUhkTrQhKC3iMi+hXr4ZjNSlcgF5f+zvRIA==
-----END CERTIFICATE-----
+52
View File
@@ -0,0 +1,52 @@
-----BEGIN PRIVATE KEY-----
MIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQC43Jc7NR+EfhY6
9eBo4hX5WKpcRc79V+PE8ETgQ5jHGTVLk3W0AzlJOie0SomS/yxIidmRg88+Nyq/
zISHWrkbJmuwjdOHnCqi8VAQpTAA/abRuW4ljBjSiqjx1xm8+Fwx0R0m7WHNEjQs
6IG/kSCRPopPnP3sJ8O1XXq5pUhLLCdCajVNxju1PS1CzZLOdziSDqZt8aduaQQd
+JXlo6PhqFQt3mCt0+H7xhydZc0T+ZpJkks4rPY8gy7JJLtOaaaAkLq4ymxPmsIY
j5bM3BjLxfrvfXPL8NVoxDvnZAl9LStQ/AdrsqIRA0oDKtk1tjGIgsqdu938ms3a
JbrKWVANCf5dN+3VoC6Lh9BzD9oFBnnNuJ50l54O0q69735fbc9WFG6awjhZ+jgL
MoGx0fp9PmgY6qYvxYVUJCX+n3e+o9FuFp0R4SoaB0Taib/VJoDmeXtbitt+j1t4
umaLcoT6iV2UuHn+aAf7u9ZBCaaJ0LrfcAVpjtNpvFT/n37fwrOjp4Alq5WgcV1c
YIiCD9J6Y1LUpaje36nAEckg8d2wuVaxFW1XlW3xBb4yUPU4OQClOy2ibSu0u8Gi
oGQj7bPb0n6LTtHgLYr9kV2G/wTF/8zFRl4RP4DOdTAsdMRI3EKwj/vBZBXItVvU
DBYtoFu3Vxh3hgZVZ75ByyXE01rIyQIDAQABAoICAFIiQFcgDTbSxpG/uMsg2F6G
1HpW0dah/CL+FbwGjJS5UIKZq8wlOoicfBOQontbQJOiG7aZd7TO0gGRnrh8yI2V
jndNLFSuQAtRaB9dJWzrRfkciCHKkyTIUrPQvDDHsD66CFfJVJDGq8PgMfWpR20A
+nbQ68jHCh9Ev0hIdUxg+7h4c+JwVwr5eWia6cUuF0Zdl/h1S8y0gA3I6uCyyhdy
sKQIj6/r2hYBOal9F5buaWySwTUXM/hC2MCpv0bhjgbFRxDfbywXOHGtKnUuvR2c
gxdxB8fu4wK/XVY7jjO9o+dBcxKYtYUVjwbFPOiuYGekpN1cIQ8gwKFR7iIFeJjx
FD/zrNUx+DhJcz9ovE4Tb/Gg1aKdddzeI5t/JzkG6uQCOshlErI7lKdwwhdwuQIx
R/EGXdnSHC6yCB9zKhM33iza7bEtYGC7ih78lGw8i3BQ/FvrJeQYs+CSNph+zQ0K
QRvqkNwODRXKA9Haqr3iPa5dKJhUMskYAz5FpHxC9oyB6YOh45PwKrUDo63Q6l6U
Snjl3w5pSFB5fcRZvb6Wfdv9eeLVTNksw35xE8kAaMTTk5x1pzQph7pqFjCmV7d+
CGsYYIl+855h789tbAnSY46JSGnvDKYcRitW88VyI1Lmjz2Mq9NhaT0Zj0iOJTMA
KU89EjtLzyp/dj8DM+35AoIBAQDgt77P0e2p0bKqTRWWubVZYsS0OatBq4wsYqV4
ustT1/IlT5fZhKN0cCxOXcwlhGyWrZt1ceVxLsFKl6IWOQHNjyRTyugICW/HVJM2
kpeZ0fMAMNWLPoFnX4hwhExGNhMN/hgLEqqefFhl+TSoGoWzj8D/TMXGSYx7C9Gp
9T2NXfORarNG5Xku/NmF2CTlSM9HZqzpikhkbpkK5rqB9nc+2T4XgskF7E7Rzx2v
cq7y0OfgGNlm8yWwv8mamGULT+jXWNGhfaugTmoph1F5+TmVliG0h2VPId2VoXRN
ex68UrRMKxP3qOpoZflwLFWOjrck+y/eK2l/ue5M/daN+ZUjAoIBAQDSmH8kxj+l
FCd3jL9KfzhzT10hYMPAiPh1uiMYZ2dQkHKbRoQq4I6s94Noow9o+KaNlJg8J8Bn
YZdrcLPT1semwNlE1FfI5t3gQjIzaIZ12FpaEwEEktCKk2SM/X7CwfkN1+FvNBnJ
2hj6TxjO8m+TkBQqAlHWMPM6P9uScn6SOM125iahdswOJeCAaJMkhcTvATE+m6Pc
CkWoxlILYHXyTQmH92Pka6ZfpBNpoI0ADCO5gFyOsL6VKahV0EsXa17yWzjsOpHI
2leJivls9dJgWr9hLSMgH3Qt/t8A35bGV5Q5PwODhJ+WJ9J19vFcHID73bzq43tD
56UBRUGNSdcjAoIBAAJXh+KMkoiBifYiZYYzm0M6N0iVjUZa7lQMFyNh9vqBtqFS
6gc3TajJ/nw2mAkQDz2mw4b+z+BVF2iamfLXV0B4LG2/IJns10BhjkM0VeYhfQHU
gHU6Cok0QqzBhDX7HEm6CzAaWrLaIuW1KipSVHBhoCZI+4qse41QuzelOaX+g6pR
TVsAyzmFIxM1BHVrQ9W/qS+p5EU/rdKiQvFVyzpZcz81erjYFJ41JV8Nt+sJ6FC6
kZF0GUF1TjmROwRaKdgMseqX77D1AEA8i8nUohf//4vtGU4w0SldDGQ+UzytM/nT
PRsIpKC/51CW9bFNpXT6NS6Aj1HocyZUQucp4bcCggEBAMCTNIjTRLXW1TRMH0yn
Q16mbzorezWfytwUxyz0uZQBUtvMwuVWjQF8IM1Zdqj934fOHtu7WgTvSAC2gaqw
V8eTx9pZ9qA/BRuiTLeX2IUAv7ZodGDTRCHEIImQ8Q51RCK1i28eDIr5hie2lq//
H6qncNjtYBpmjrRwWn/zdOyPRst4MFEsCfLSDhY+Cne2X1xTEc33kwKO3h40pCfF
IHXenl2YCt+A1RXWOu43I1iswSpLR9gvpUdPXaCDJXeX9q3WXxodgNxTVQLwc5+A
tsznjuP0247vVFUPIKtyyjQ7N86VYcgtSaWMarb2hsU9R3GJ1cxREpIIzGl6BDSI
FlMCggEAJmG2J8T0H6LT6CxCv5uhZW//uGV7gv+F4KTwpIx2oVEXt6gj6ORBx068
1nCbEG4ikPumiDMFXQ2GKa+m9vfSGIxhmYYbeEH29jRImNAgiXmEpSRtjSp6sRk+
g09K0Ee8N7UxK4ZhV9ozgPT9OUNY91MwfNlG+d5/qeOJqUCOtg9zsRf2kkFp7VBo
gTH597UDsHVrT98rpFo/XlOgsJb0OEUV8vkJkMtVguOyUnh6rp9uw+2kQocO1N3a
IT7YzeCaXcgjvvLZyILHy7tZnkMW7XUF70I18VzVFSNlzyOn/XD2JNeGwvAor26H
hqHUM7qo5k3nI6/dSdWQ1gBdmv104g==
-----END PRIVATE KEY-----
+2 -5
View File
@@ -26,11 +26,8 @@ an access token. Once logged into the system, user can manage his resources (i.e
things and channels) in CRUD fashion, and define access control policies by
connecting them.
`Thing` represents devices and applications connected to Mainflux. There are
two types of "things" supported at the moment: `device` and `app`. While device
is used to represent any physical device connected to the platform, app represents
any 3rd party service that uses the platform for message exchange with other
"things".
`Thing` represents devices (or applications) connected to Mainflux that uses the
platform for message exchange with other "things".
`Channel` represents a communication channel. It serves as message topic that
can be consumed by all of the things connected to it.
+85
View File
@@ -0,0 +1,85 @@
## Authentication with Mainflux keys
By default, Mainflux uses Mainflux keys for authentication. The Мainflux key is a secret key that's generated at the Thing creation. In order to authenticate, the Thing needs to send its key with the message. The way the key is passed depends on the protocol used to send a message and differs from adapter to adapter. For more details on how this key is passed around, please check out [messaging section](https://mainflux.readthedocs.io/en/latest/messaging).
This is the default Mainflux authentication mechanism and this method is used if the composition is started using the following command:
```bash
docker-compose -f docker/docker-compose.yml up
```
## Mutual TLS Authentication with X.509 Certificates
In most of the cases, HTTPS, WSS, MQTTS or secure CoAP are secure enough. However, sometimes you might need an even more secure connection. Mainflux supports mutual TLS authentication (_mTLS_) based on [X.509 certificates](https://tools.ietf.org/html/rfc5280). By default, the TLS protocol only proves the identity of the server to the client using the X.509 certificate and the authentication of the client to the server is left to the application layer. TLS also offers client-to-server authentication using client-side X.509 authentication. This is called two-way or mutual authentication. Mainflux currently supports mTLS over HTTP, WS, and MQTT protocols. In order to run Docker composition with mTLS turned on, you can execute the following command from the project root:
```bash
AUTH=x509 docker-compose -f docker/docker-compose.yml up -d
```
Mutual authentication includes client-side certificates. Certificates can be generated using the simple script provided [here](http://www.github.com/mainflux/mainflux/tree/master/docker/ssl/Makefile). In order to create a valid certificate, you need to create Mainflux thing using the process described in the [provisioning section](provisioning.md). After that, you need to fetch created thing key. Thing key will be used to create x.509 certificate for the corresponding thing. To create a certificate, execute the following commands:
```bash
cd docker/ssl
make ca
make server_cert
make thing_cert KEY=<thing_key> CRT_FILE_NAME=<cert_name>
```
These commands use [OpenSSL](https://www.openssl.org/) tool, so please make sure that you have it installed and set up before running these commands.
- Command `make ca` will generate a self-signed certificate that will later be used as a CA to sign other generated certificates. CA will expire in 3 years.
- Command `make server_cert` will generate and sign (with previously created CA) server cert, which will expire after 1000 days. This cert is used as a Mainflux server-side certificate in usual TLS flow to establish HTTPS, WSS, or MQTTS connection.
- Command `make thing_cert` will finally generate and sign a client-side certificate and private key for the thing.
In this example `<thing_key>` represents key of the thing, and `<cert_name>` represents the name of the certificate and key file which will be saved in `docker/ssl/certs` directory. Generated Certificate will expire after 2 years. The key must be stored in the x.509 certificate `CN` field. This script is created for testing purposes and is not meant to be used in production. We strongly recommend avoiding self-signed certificates and using a certificate management tool such as [Vault](https://www.vaultproject.io/) for the production.
Once you have created CA and server-side cert, you can spin the composition using:
```bash
AUTH=x509 docker-compose -f docker/docker-compose.yml up -d
```
Then, you can create user and provision things and channels. Now, in order to send a message from the specific thing to the channel, you need to connect thing to the channel and generate corresponding client certificate using aforementioned commands. To publish a message to the channel, thing should send following request:
### HTTPS
```bash
curl -s -S -i --cacert docker/ssl/certs/ca.crt --cert docker/ssl/certs/<thing_cert_name>.crt --key docker/ssl/certs/<thing_cert_key>.key --insecure -X POST -H "Content-Type: application/senml+json" https://localhost/http/channels/<channel_id>/messages -d '[{"bn":"some-base-name:","bt":1.276020076001e+09, "bu":"A","bver":5, "n":"voltage","u":"V","v":120.1}, {"n":"current","t":-5,"v":1.2}, {"n":"current","t":-4,"v":1.3}]'
```
### MQTTS
#### Publish
```bash
mosquitto_pub -u <thing_id> -P <thing_key> -t channels/<channel_id>/messages -h localhost --cafile docker/ssl/certs/ca.crt --cert docker/ssl/certs/<thing_cert_name>.crt --key docker/ssl/certs/<thing_cert_key>.key -m '[{"bn":"some-base-name:","bt":1.276020076001e+09, "bu":"A","bver":5, "n":"voltage","u":"V","v":120.1}, {"n":"current","t":-5,"v":1.2}, {"n":"current","t":-4,"v":1.3}]'
```
#### Subscribe
```
mosquitto_sub -u <thing_id> -P <thing_key> --cafile docker/ssl/certs/ca.crt --cert docker/ssl/certs/<thing_cert_name>.crt --key docker/ssl/certs/<thing_cert_key>.key -t channels/<channel_id>/messages -h localhost
```
### WSS
```javascript
const WebSocket = require('ws');
// Do not verify self-signed certificates if you are using one.
process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0'
// Replace <channel_id> and <thing_key> with real values.
const ws = new WebSocket('wss://localhost/ws/channels/<channel_id>/messages?authorization=<thing_key>',
// This is ClientOptions object that contains client cert and client key in the form of string. You can easily load these strings from cert and key files.
{
cert: `-----BEGIN CERTIFICATE-----....`,
key: `-----BEGIN RSA PRIVATE KEY-----.....`
})
ws.on('open', () => {
ws.send('something')
})
ws.on('message', (data) => {
console.log(data)
})
ws.on('error', (e) => {
console.log(e)
})
```
As you can see, `Authorization` header does not have to be present in the HTTP request, since the key is present in the certificate. However, if you pass `Authorization` header, it _must be the same as the key in the cert_. In the case of MQTTS, `password` filed in CONNECT message _must match the key from the certificate_. In the case of WSS, `Authorization` header or `authorization` query parameter _must match cert key_.
+30 -8
View File
@@ -8,19 +8,38 @@ Mainflux platform supports bootstrapping process, but some of the preconditions
> Bootstrapping and provisioning are two different procedures. Provisioning refers to entities management while bootstrapping is related to entity configuration.
![bootstrapping flow](img/bs_flow.gif)
Bootstrapping procedure is the following:
![Configure device](img/bootstrap/1.png)
*1) Configure device with Bootstrap service URL, an external key and external ID*
> ![Provision Mainflux channels](img/bootstrap/2.png)
*Optionally create Mainflux channels if they don't exist*
> ![Provision Mainflux things](img/bootstrap/3.png)
*Optionally create Mainflux thing if it doesn't exist*
![Upload configuration](img/bootstrap/4.png)
*2) Upload configuration for the Mainflux thing*
![Bootstrap](img/bootstrap/5.png)
*3) Bootstrap - send a request for the configuration*
![Update, enable/disable, remove](img/bootstrap/6.png)
*4) Connect/disconnect thing from channels, update or remove configuration*
### Configuration
The configuration of Mainflux thing consists of two major parts:
The configuration of Mainflux thing consists of three major parts:
- The list of Mainflux channels the thing is connected to
- Custom configuration related to the specific thing
- Thing key and certificate data related to that thing
Also, the configuration contains an external ID and external key, which will be explained later.
In order to enable the thing to start bootstrapping process, the user needs to upload a valid configuration for that specific thing. This can be done using the following HTTP request:
```
```bash
curl -s -S -i -X POST -H "Authorization: <user_token>" -H "Content-Type: application/json" http://localhost:8200/things/configs -d '{
"external_id":"09:6:0:sb:sa",
"thing_id": "1b9b8fae-9035-4969-a240-7fe5bdc0ed28",
@@ -32,11 +51,14 @@ curl -s -S -i -X POST -H "Authorization: <user_token>" -H "Content-Type: applica
"ff13ca9c-7322-4c28-a25c-4fe5c7b753fc",
"c3642289-501d-4974-82f2-ecccc71b2d82"
],
"content": "config..."
"content": "config...",
"client_cert": "PEM cert",
"client_key": "PEM client cert key",
"ca_cert": "PEM CA cert"
}'
```
In this example, `channels` field represents the list of Mainflux channel IDs the thing is connected to. These channels need to be provisioned before the configuration is uploaded. Field `content` represents custom configuration. This custom configuration contains parameters that can be used to set up the thing. It can also be empty if no additional set up is needed. Field `name` is human readable name and `thing_id` is an ID of the Mainflux thing. This field is not required. If `thing_id` is empty, corresponding Mainflux thing will be created implicitly and its ID will be sent as a part of `Location` header of the response.
In this example, `channels` field represents the list of Mainflux channel IDs the thing is connected to. These channels need to be provisioned before the configuration is uploaded. Field `content` represents custom configuration. This custom configuration contains parameters that can be used to set up the thing. It can also be empty if no additional set up is needed. Field `name` is human readable name and `thing_id` is an ID of the Mainflux thing. This field is not required. If `thing_id` is empty, corresponding Mainflux thing will be created implicitly and its ID will be sent as a part of `Location` header of the response. Fields `client_cert`, `client_key`, and `ca_cert` represent PEM or base64-encoded DER client certificate, client certificate key, and trusted CA, respectively.
There are two more fields: `external_id` and `external_key`. External ID represents an ID of the device that corresponds to the given thing. For example, this can be a MAC address or the serial number of the device. The external key represents the device key. This is the secret key that's safely stored on the device and it is used to authorize the thing during the bootstrapping process. Please note that external ID and external key and Mainflux ID and Mainflux key are _completely different concepts_. External id and key are only used to authenticate a device that corresponds to the specific Mainflux thing during the bootstrapping procedure.
@@ -44,13 +66,13 @@ There are two more fields: `external_id` and `external_key`. External ID represe
Currently, the bootstrapping procedure is executed over the HTTP protocol. Bootstrapping is nothing else but fetching and applying the configuration that corresponds to the given Mainflux thing. In order to fetch the configuration, _the thing_ needs to send a bootstrapping request:
```
```bash
curl -s -S -i -H "Authorization: <external_key>" http://localhost:8200/things/bootstrap/<external_id>
```
The response body should look something like:
```
```json
{
"mainflux_id":"7c9df5eb-d06b-4402-8c1a-df476e4394c8",
"mainflux_key":"86a4f870-eba4-46a0-bef9-d94db2b64392",
@@ -81,7 +103,7 @@ The response consists of an ID and key of the Mainflux thing, the list of channe
Uploading configuration does not automatically connect thing to the given list of channels. In order to connect the thing to the channels, user needs to send the following HTTP request:
```
```bash
curl -s -S -i -X PUT -H "Authorization: <user_token>" -H "Content-Type: application/json" http://localhost:8200/things/state/<thing_id> -d '{"state": 1}'
```
+2 -7
View File
@@ -81,14 +81,9 @@ mainflux-cli users token john.doe@email.com password
```
### System Provisioning
#### Create Thing (type Device)
#### Create Thing
```
mainflux-cli things create '{"type":"device", "name":"myDevice"}' <user_auth_token>
```
#### Create Thing (type Application)
```
mainflux-cli things create '{"type":"app", "name":"myDevice"}' <user_auth_token>
mainflux-cli things create '{"name":"myThing"}' <user_auth_token>
```
#### Update Thing
+316 -35
View File
@@ -65,7 +65,7 @@ make docker_http
> N.B. Mainflux creates `FROM scratch` docker containers which are compact and small in size.
> N.B. The `things-db` and `users-db` containers are built from a vanilla PostgreSQL docker image downloaded from docker hub which does not persist the data when these containers are rebuilt. Thus, __rebuilding of all docker containers with `make dockers` or rebuilding the `things-db` and `users-db` containers separately with `make docker_things-db` and `make docker_users-db` respectively, will cause data loss. All your users, things, channels and connections between them will be lost!__ As we use this setup only for development, we don't guarantee any permanent data persistence. If you need to retain the data between the container rebuilds you can attach volume to the `things-db` and `users-db` containers. Check the official docs on how to use volumes [here](https://docs.docker.com/storage/volumes/) and [here](https://docs.docker.com/compose/compose-file/#volumes). For examples on how to add persistent volumes check the [Overriding the default docker-compose configuration](#overriding-the-default-docker-compose-configuration) section.
> N.B. The `things-db` and `users-db` containers are built from a vanilla PostgreSQL docker image downloaded from docker hub which does not persist the data when these containers are rebuilt. Thus, __rebuilding of all docker containers with `make dockers` or rebuilding the `things-db` and `users-db` containers separately with `make docker_things-db` and `make docker_users-db` respectively, will cause data loss. All your users, things, channels and connections between them will be lost!__ As we use this setup only for development, we don't guarantee any permanent data persistence. Though, in order to enable data retention, we have configured persistent volumes for each container that stores some data. If you want to update your Mainflux dockerized installation and want to keep your data, use `make cleandocker` to clean the containers and images and keep the data (stored in docker persistent volumes) and then `make run` to update the images and the containers. Check the [Cleaning up your dockerized Mainflux setup](#cleaning-up-your-dockerized-mainflux-setup) section for details. Please note that this kind of updating might not work if there are database changes.
#### Building Docker images for development
@@ -85,56 +85,34 @@ Commands `make dockers` and `make dockers_dev` are similar. The main difference
### Overriding the default docker-compose configuration
Sometimes, depending on the use case and the user's needs it might be useful to override or add some extra parameters to the docker-compose configuration. These configuration changes can be done by specifying multiple compose files with the [docker-compose command line option -f](https://docs.docker.com/compose/reference/overview/) as described [here](https://docs.docker.com/compose/extends/).
The following format of the `docker-compose` can be used to extend or override the configuration:
The following format of the `docker-compose` command can be used to extend or override the configuration:
```
docker-compose -f docker/docker-compose.yml -f docker/docker-compose.custom1.yml -f docker/docker-compose.custom2.yml up [-d]
```
In the command above each successive file overrides the previous parameters.
A practical example in our case would be to add persistent volumes to the users-db, things-db, influxdb and grafana containers so that we don't loose data between updates of Mainflux. The volumes are mapped to the default location on the host machine, something like `/var/lib/docker/volumes/project-name_volume-name`:
A practical example in our case would be to enable debugging and tracing in NATS so that we can see better how are the messages moving around.
`docker-compose.nats-debugging.yml`
```yaml
# docker/docker-compose.persistence.yml
version: "3"
volumes:
mainflux-users-db-volume:
mainflux-things-db-volume:
services:
users-db:
volumes:
- mainflux-users-db-volume:/var/lib/postgresql/data
things-db:
volumes:
- mainflux-things-db-volume:/var/lib/postgresql/data
nats:
command: --debug -DV
```
```yaml
# docker/addons/influxdb-writer/docker-compose.persistence.yml
version: "3"
volumes:
influxdb-volume:
grafana-volume:
services:
influxdb:
volumes:
- influxdb-volume:/var/lib/influxdb
grafana:
volumes:
- grafana-volume:/var/lib/grafana
```
When we have the override files in place, to compose the whole infrastructure including the persistent volumes we can execute:
```
docker-compose -f docker/docker-compose.yml -f docker/docker-compose.persistence.yml up -d
docker-compose -f docker/addons/influxdb-writer/docker-compose.yml -f docker/addons/influxdb-writer/docker-compose.persistence.yml up -d
docker-compose -f docker/docker-compose.yml -f docker/docker-compose.nats-debugging.yml up -d
```
__Note:__ Please store your customizations to some folder outside the Mainflux's source folder and maybe add them to some other git repository. You can always apply your customizations by pointing to the right file using `docker-compose -f ...`. Also be sure not to use the `make cleandocker` and `make cleanghost` as they might delete something which you don't want to delete (i.e. the newly created persistent volumes).
__Note:__ Please store your customizations to some folder outside the Mainflux's source folder and maybe add them to some other git repository. You can always apply your customizations by pointing to the right file using `docker-compose -f ...`.
### Cleaning up your dockerized Mainflux setup
If you want to clean your whole dockerized Mainflux installation you can use the `make pv=true cleandocker` command. Please note that __by default the `make cleandocker` command will stop and delete all of the containers and images, but NOT DELETE persistent volumes__. If you want to delete the gathered data in the system (the persistent volumes) please use the following command `make pv=true cleandocker` (pv = persistent volumes). This form of the command will stop and delete the containers, the images and will also delete the persistent volumes.
### MQTT Microservice
The MQTT Microservice in Mainflux is special, as it is currently the only microservice written in NodeJS. It is not compiled, but node modules need to be downloaded in order to start the service:
@@ -255,3 +233,306 @@ which will properly configure and run all microservices.
Please assure that MQTT microservice has `node_modules` installed, as explained in _MQTT Microservice_ chapter.
> N.B. `make rundev` actually calls helper script `scripts/run.sh`, so you can inspect this script for the details.
## Events
In order to be easily integratable system, Mainflux is using [Redis Streams](https://redis.io/topics/streams-intro)
as an event log for event sourcing. Services that are publishing events to Redis Streams
are `things` service, `bootstrap` service, and `mqtt` adapter.
### Things Service
For every operation that has side effects (that is changing service state) `things`
service will generate new event and publish it to Redis Stream called `mainflux.things`.
Every event has its own event ID that is automatically generated and `operation`
field that can have one of the following values:
- `thing.create` for thing creation,
- `thing.update` for thing update,
- `thing.remove` for thing removal,
- `thing.connect` for connecting a thing to a channel,
- `thing.disconnect` for disconnecting thing from a channel,
- `channel.create` for channel creation,
- `channel.update` for channel update,
- `channel.remove` for channel removal.
By fetching and processing these events you can reconstruct `things` service state.
If you store some of your custom data in `metadata` field, this is the perfect
way to fetch it and process it. If you want to integrate through
[docker-compose.yml](https://github.com/mainflux/mainflux/blob/master/docker/docker-compose.yml)
you can use `mainflux-es-redis` service. Just connect to it and consume events
from Redis Stream named `mainflux.things`.
#### Thing create event
Whenever thing is created, `things` service will generate new `create` event. This
event will have the following format:
```
1) "1555334740911-0"
2) 1) "operation"
2) "thing.create"
3) "name"
4) "d0"
5) "id"
6) "3c36273a-94ea-4802-84d6-a51de140112e"
7) "owner"
8) "john.doe@email.com"
9) "metadata"
10) "{}"
```
As you can see from this example, every odd field represents field name while every
even field represents field value. This is standard event format for Redis Streams.
If you want to extract `metadata` field from this event, you'll have to read it as
string first, and then you can deserialize it to some structured format.
#### Thing update event
Whenever thing instance is updated, `things` service will generate new `update` event.
This event will have the following format:
```
1) "1555336161544-0"
2) 1) "operation"
2) "thing.update"
3) "name"
4) "weio"
5) "id"
6) "3c36273a-94ea-4802-84d6-a51de140112e"
```
Note that thing update event will contain only those fields that were updated using
update endpoint.
#### Thing remove event
Whenever thing instance is removed from the system, `things` service will generate and
publish new `remove` event. This event will have the following format:
```
1) 1) "1555339313003-0"
2) 1) "id"
2) "3c36273a-94ea-4802-84d6-a51de140112e"
3) "operation"
4) "thing.remove"
```
#### Channel create event
Whenever channel instance is created, `things` service will generate and publish new
`create` event. This event will have the following format:
```
1) "1555334740918-0"
2) 1) "id"
2) "16fb2748-8d3b-4783-b272-bb5f4ad4d661"
3) "owner"
4) "john.doe@email.com"
5) "operation"
6) "channel.create"
7) "name"
8) "c1"
```
#### Channel update event
Whenever channel instance is updated, `things` service will generate and publish new
`update` event. This event will have the following format:
```
1) "1555338870341-0"
2) 1) "name"
2) "chan"
3) "id"
4) "d9d8f31b-f8d4-49c5-b943-6db10d8e2949"
5) "operation"
6) "channel.update"
```
Note that update channel event will contain only those fields that were updated using
update channel endpoint.
#### Channel remove event
Whenever channel instance is removed from the system, `things` service will generate and
publish new `remove` event. This event will have the following format:
```
1) 1) "1555339429661-0"
2) 1) "id"
2) "d9d8f31b-f8d4-49c5-b943-6db10d8e2949"
3) "operation"
4) "channel.remove"
```
#### Connect thing to a channel event
Whenever thing is connected to a channel on `things` service, `things` service will
generate and publish new `connect` event. This event will have the following format:
```
1) "1555334740920-0"
2) 1) "chan_id"
2) "d9d8f31b-f8d4-49c5-b943-6db10d8e2949"
3) "thing_id"
4) "3c36273a-94ea-4802-84d6-a51de140112e"
5) "operation"
6) "thing.connect"
```
#### Disconnect thing from a channel event
Whenever thing is disconnected from a channel on `things` service, `things` service
will generate and publish new `disconnect` event. This event will have the following
format:
```
1) "1555334740920-0"
2) 1) "chan_id"
2) "d9d8f31b-f8d4-49c5-b943-6db10d8e2949"
3) "thing_id"
4) "3c36273a-94ea-4802-84d6-a51de140112e"
5) "operation"
6) "thing.disconnect"
```
> **Note:** Every one of these events will omit fields that were not used or are not
relevant for specific operation. Also, field ordering is not guaranteed, so DO NOT
rely on it.
### Bootstrap Service
Bootstrap service publishes events to Redis Stream called `mainflux.bootstrap`.
Every event from this service contains `operation` field which indicates one of
the following event types:
- `config.create` for configuration creation,
- `config.update` for configuration update,
- `config.remove` for configuration removal,
- `thing.bootstrap` for device bootstrap,
- `thing.state_change` for device state change,
- `thing.update_connections` for device connection update.
If you want to integrate through
[docker-compose.yml](https://github.com/mainflux/mainflux/blob/master/docker/addons/bootstrap/docker-compose.yml)
you can use `mainflux-es-redis` service. Just connect to it and consume events
from Redis Stream named `mainflux.bootstrap`.
#### Configuration create event
Whenever configuration is created, `bootstrap` service will generate and publish
new `create` event. This event will have the following format:
```
1) "1555404899581-0"
2) 1) "owner"
2) "john.doe@email.com"
3) "name"
4) "some"
5) "channels"
6) "ff13ca9c-7322-4c28-a25c-4fe5c7b753fc, c3642289-501d-4974-82f2-ecccc71b2d82, c3642289-501d-4974-82f2-ecccc71b2d83, cd4ce940-9173-43e3-86f7-f788e055eb14"
7) "externalID"
8) "9c:b6:d:eb:9f:fd"
9) "content"
10) "{}"
11) "timestamp"
12) "1555404899"
13) "operation"
14) "config.create"
15) "thing_id"
16) "63a110d4-2b77-48d2-aa46-2582681eeb82"
```
#### Configuration update event
Whenever configuration is updated, `bootstrap` service will generate and publish
new `update` event. This event will have the following format:
```
1) "1555405104368-0"
2) 1) "content"
2) "NOV_MGT_HOST: http://127.0.0.1:7000\nDOCKER_MGT_HOST: http://127.0.0.1:2375\nAGENT_MGT_HOST: https://127.0.0.1:7003\nMF_MQTT_HOST: tcp://104.248.142.133:8443"
3) "timestamp"
4) "1555405104"
5) "operation"
6) "config.update"
7) "thing_id"
8) "63a110d4-2b77-48d2-aa46-2582681eeb82"
9) "name"
10) "weio"
```
#### Configuration remove event
Whenever configuration is removed, `bootstrap` service will generate and publish
new `remove` event. This event will have the following format:
```
1) "1555405464328-0"
2) 1) "thing_id"
2) "63a110d4-2b77-48d2-aa46-2582681eeb82"
3) "timestamp"
4) "1555405464"
5) "operation"
6) "config.remove"
```
#### Thing bootstrap event
Whenever thing is bootstrapped, `bootstrap` service will generate and publish
new `bootstrap` event. This event will have the following format:
```
1) "1555405173785-0"
2) 1) "externalID"
2) "9c:b6:d:eb:9f:fd"
3) "success"
4) "1"
5) "timestamp"
6) "1555405173"
7) "operation"
8) "thing.bootstrap"
```
#### Thing change state event
Whenever thing's state changes, `bootstrap` service will generate and publish
new `change state` event. This event will have the following format:
```
1) "1555405294806-0"
2) 1) "thing_id"
2) "63a110d4-2b77-48d2-aa46-2582681eeb82"
3) "state"
4) "0"
5) "timestamp"
6) "1555405294"
7) "operation"
8) "thing.state_change"
```
#### Thing update connections event
Whenever thing's list of connections is updated, `bootstrap` service will generate
and publish new `update connections` event. This event will have the following format:
```
1) "1555405373360-0"
2) 1) "operation"
2) "thing.update_connections"
3) "thing_id"
4) "63a110d4-2b77-48d2-aa46-2582681eeb82"
5) "channels"
6) "ff13ca9c-7322-4c28-a25c-4fe5c7b753fc, 925461e6-edfb-4755-9242-8a57199b90a5, c3642289-501d-4974-82f2-ecccc71b2d82"
7) "timestamp"
8) "1555405373"
```
### MQTT Adapter
Instead of using heartbeat to know when client is connected through MQTT adapter one
can fetch events from Redis Streams that MQTT adapter publishes. MQTT adapter
publishes events every time client connects and disconnects to stream named `mainflux.mqtt`.
Events that are coming from MQTT adapter have following fields:
- `thing_id` ID of a thing that has connected to MQTT adapter,
- `timestamp` is in Epoch UNIX Time Stamp format,
- `event_type` can have two possible values, connect and disconnect,
- `instance` represents MQTT adapter instance.
If you want to integrate through
[docker-compose.yml](https://github.com/mainflux/mainflux/blob/master/docker/docker-compose.yml)
you can use `mainflux-es-redis` service. Just connect to it and consume events
from Redis Stream named `mainflux.mqtt`.
Example of connect event:
```
1) 1) "1555351214144-0"
2) 1) "thing_id"
2) "1c597a85-b68e-42ff-8ed8-a3a761884bc4"
3) "timestamp"
4) "1555351214"
5) "event_type"
6) "connect"
7) "instance"
8) "mqtt-adapter-1"
```
Example of disconnect event:
```
1) 1) "1555351214188-0"
2) 1) "thing_id"
2) "1c597a85-b68e-42ff-8ed8-a3a761884bc4"
3) "timestamp"
4) "1555351214"
5) "event_type"
6) "disconnect"
7) "instance"
8) "mqtt-adapter-1"
```
+1 -3
View File
@@ -50,13 +50,11 @@ Output of the command is something like this:
"id": "513d02d2-16c1-4f23-98be-9e12f8fee898",
"key": "69590b3a-9d76-4baa-adae-9b5fec0ea14f",
"name": "d0",
"type": "device"
},
{
"id": "bf78ca98-2fef-4cfc-9f26-e02da5ecdf67",
"key": "840c1ea1-2e8d-4809-a6d3-3433a5c489d2",
"name": "d1",
"type": "app"
}
]
@@ -106,4 +104,4 @@ mainflux-http | {"level":"info","message":"Method publish took 336.685µs to c
mainflux-normalizer | {"level":"info","message":"Method normalize took 108.126µs to complete without errors.","ts":"2019-01-08T22:19:30.149500543Z"}
```
This proves that messages have been well send through the system, via protocol adapter (`mainflux-http`) and `normalizer` service which corectly parsed messages.
This proves that messages have been well send through the system, via protocol adapter (`mainflux-http`) and `normalizer` service which correctly parsed messages.
Binary file not shown.

After

Width:  |  Height:  |  Size: 107 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 105 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

Some files were not shown because too many files have changed in this diff Show More