Compare commits

..

85 Commits

Author SHA1 Message Date
Nick Vollmar 980bee22a5 Release 2019.4.0 2019-04-03 14:31:34 -05:00
Chung-Ting Huang 80d3d989eb TUN-1673: Conflate Hello and Connect RPCs 2019-04-01 13:50:30 -05:00
Adam Chalmers 6804a5ff9d TUN-1648: ConnectionID is now a UUID 2019-03-28 15:03:30 -05:00
Chung-Ting Huang b5dab1f5da Release 2019.3.2 2019-03-25 13:54:40 -05:00
Chung-Ting Huang 8c6cfa34aa TUN-1604: Define Connect RPC call 2019-03-22 17:30:23 -05:00
Adam Chalmers 8560436487 TUN-1637: Free tunnels shouldn't require cert.pem 2019-03-22 14:43:30 -05:00
Nick Vollmar 619bc95501 Release 2019.3.1 2019-03-18 16:08:24 -05:00
Marek Vavruša 781e1fef7b TUN-1615: revert miekg/dns to last known working revision
The last known good commit is 6da3249dfb57fbaa16efafcd8744cee8809d80cd before the first release tag.
2019-03-17 20:40:18 -07:00
Adam Chalmers 541cf68608 TUN-1613: improved cloudflared RegisterTunnel fail metrics 2019-03-15 18:46:53 -05:00
Areg Harutyunyan eafc89bf73 Merge branch 'master' of github.com:cloudflare/cloudflared 2019-03-11 14:57:03 -05:00
Chung-Ting Huang fd451ca158 Release 2019.3.0 2019-03-11 13:55:10 -05:00
Nick Vollmar d22e214000 TUN-1522: If we can't get SRV from default resolver, get them from 1.1.1.1 DoT 2019-03-11 12:08:04 -04:00
Joel Miles 92d6d73f9c Add rdp as a supported protocol in URL validation (#76) 2019-03-09 22:23:30 -06:00
Areg Harutyunyan b02718f86b TUN-1389: Non-scalar flags in a cloudflared config.yml don't get logged 2019-03-08 02:43:07 -06:00
Areg Harutyunyan 5a0b7ecb62 Merge branch 'master' of github.com:cloudflare/cloudflared 2019-03-06 18:34:42 -06:00
Kristian Mide 07a409ffef Support unix sockets. 2019-03-06 18:33:23 -06:00
Austin Cherry 58e5e9c9a4 AUTH-1531: Named flags for ssh service tokens 2019-03-06 13:09:13 -06:00
Nick Vollmar 6ca642e572 TUN-1550: Add validation timeout for non-responsive origins 2019-03-06 11:30:29 -06:00
Adam Chalmers 073c5bfdaa TUN-1562: Refactor connectedSignal to be safe to close multiple times 2019-03-05 15:51:35 -06:00
Nick Vollmar fea3569956 TUN-1451: Make non-interactive, non-service execution possible on Windows 2019-03-04 19:49:57 -06:00
Adam Chalmers abdbc76a46 TUN-1559: fix nil dereference in TunnelConfig.CloseConnOnce 2019-02-28 16:56:47 -06:00
Areg Harutyunyan f22202b31b Merge branch 'master' of github.com:cloudflare/cloudflared 2019-02-27 18:38:01 -06:00
Adam Chalmers 4586ed3e51 Release 2019.2.1 2019-02-27 17:59:54 -06:00
Adam Chalmers 7475e3e487 TUN-1510: Wrap the close() in sync.Once.Do 2019-02-26 10:32:07 -06:00
Adam Chalmers e025a4cd7b TUN-1525: cloudflared metrics for registration success/fail 2019-02-19 17:11:04 -06:00
Austin Cherry 850f804c47 AUTH-1519: Added logging 2019-02-19 12:53:33 -06:00
Nick Vollmar 1558280185 TUN-1467: build with Go 1.11 2019-02-15 14:57:02 -06:00
Kane Dou 15d68a0eed make http transport aware of proxy from envvar 2019-02-15 11:26:34 -06:00
Areg Harutyunyan e3aed15c08 TUN-1381: should tell you if you're on the latest version rather than just exiting silently 2019-02-14 17:11:52 -06:00
Austin Cherry 99713e3439 Release 2019.2.0 2019-02-11 14:06:56 -06:00
Austin Cherry 27c6977746 AUTH-1403: Print the paths in the ssh-config instructions 2019-02-11 10:56:06 -06:00
Austin Cherry 200f9a3786 AUTH-1503: Added RDP support 2019-02-08 14:28:47 -06:00
Austin Cherry 92defa26d4 AUTH-1511: Add custom headers for ssh command 2019-02-07 16:38:52 -06:00
Areg Harutyunyan c9f1c1ec70 Merge branch 'master' of github.com:cloudflare/cloudflared 2019-02-01 14:57:53 -06:00
Philippe Grégoire 10e4a2940c cloudflared/linux_service: Add missing /etc/init.d shebang
When using sysv init scripts, `cloudflared` fails to start due to the
missing shebang interpreter line.

This patch adds the missing shebang.
2019-02-01 14:55:30 -06:00
Adam Chalmers 77f12c7c34 TUN-1456: Only make one UUID 2019-02-01 14:11:12 -06:00
Austin Cherry ca33ed9f6a AUTH-1462: better curl arg parsing 2019-02-01 10:46:36 -06:00
Adam Chalmers 6a331b13ba Release 2019.1.0 2019-01-28 17:03:33 -06:00
Chung-Ting Huang 47c878b9c4 TUN-1419: Identify request/response headers/content length with ray ID 2019-01-28 16:52:17 -06:00
Chung-Ting Huang 61cd4a918d TUN-1418: Rename ProtocolLogger to TransportLogger, and use TransportLogger to log RPC events. 2019-01-28 16:09:58 -06:00
Austin Cherry 9db4b7fdfb AUTH-1337: fix url path 2019-01-25 13:47:12 -06:00
Austin Cherry 2f59b8ac22 AUTH-1211: print all the versions 2019-01-25 13:44:26 -06:00
Austin Cherry f94699e07b AUTH-1459: improved ssh streaming error message 2019-01-25 10:45:50 -06:00
Austin Cherry f7cf597f54 AUTH-1404: reauth if the token is about to expire within 15 minutes 2019-01-25 10:43:07 -06:00
Austin Cherry aeb8dd7e51 AUTH-1423: move from stdout to stderr 2019-01-24 14:48:37 -06:00
Nick Vollmar 5bf6dd8f85 TUN-1358: Close readyList after Muxer.Serve() has stopped running 2019-01-22 15:54:29 -06:00
Nick Vollmar 62b1ab8c98 TUN-1350: Enhance error messages with cloudflarestatus.com link, if relevant 2019-01-18 13:37:05 -06:00
Areg Harutyunyan 8de19dc647 TUN-1265: Silent exit when failing to parse config 2019-01-08 15:51:43 -06:00
Areg Harutyunyan 8a099d8f6e TUN-1138: Install cloudflared service directory with 755 permissions 2019-01-07 13:46:21 -06:00
Leland Garofalo ef400afe00 TUN-632 Filter out common network exceptions from going to Sentry on StartServer 2019-01-07 13:36:52 -06:00
Nick Vollmar bf596c035e TUN-1140: Show usage if invoked with no args or config 2018-12-17 15:05:03 -06:00
Areg Harutyunyan 36276e957a TUN-1231: Horizontal overflow wrapping on the Hello page 2018-12-11 16:03:44 -06:00
Chung-Ting Huang c69437ba83 Release 2018.12.1 2018-12-11 14:20:51 -06:00
Adam Chalmers b2d0c612a5 TUN-1270: cloudflared panic (HA metrics missing label) 2018-12-11 11:41:49 -06:00
Areg Harutyunyan 8e2908f889 Release 2018.12.0 2018-12-10 13:50:13 -06:00
Nick Vollmar 3e8d886c25 TUN-1250: ValidateHTTPService shouldn't follow 302s 2018-12-07 16:59:15 -06:00
Areg Harutyunyan 446c5cf60c Merge branch 'master' of github.com:cloudflare/cloudflared 2018-12-07 11:36:41 -06:00
Mohammed Naser 13f88b3739 Fix license URL typo 2018-12-07 11:33:55 -06:00
Nick Vollmar 69ee6c1d88 TUN-1204: remove 'cloudflared hello' command 2018-12-03 16:31:20 -06:00
Adam Chalmers 192ae35728 TUN-1212: Expose tunnel_id in metrics 2018-11-27 15:26:45 -06:00
Chung-Ting Huang 10d547f528 TUN-1209: TLS Config Certificates and GetCertificate can both be set 2018-11-20 14:34:56 -06:00
Chung-Ting Huang b59fd4b7d8 TUN-1196: Allow TLS config client CA and root CA to be constructed from multiple certificates 2018-11-19 15:38:10 -06:00
Chung-Ting Huang c85c8526e8 Release 2018.11.0 2018-11-16 09:53:13 -06:00
Austin Cherry f49d9dcb67 AUTH-1320: Fixed request issue and unhide the ssh command 2018-11-15 13:08:56 -06:00
Nick Vollmar c2ac282aca TUN-1190: check URL parse error when starting SSH proxy server 2018-11-14 12:16:12 -06:00
Austin Cherry 58daf6bfed AUTH-1308: get jwt even when you are already logged in 2018-11-13 17:06:17 -06:00
Nick Vollmar 611b284e20 TUN-1179: Fix log message in cmd/cloudflared/transfer.Run 2018-11-06 09:23:03 -08:00
Austin Cherry 236a0a164d AUTH-1282: Fixed an issue where we were receiving as opposed sending on the channel. 2018-10-31 13:48:19 -05:00
Nick Vollmar 148eea5899 Release 2018.10.5 2018-10-30 14:15:58 -05:00
Nick Vollmar 83c6c8713b TUN-1160: pass Host header during origin url validation 2018-10-30 13:58:07 -05:00
Areg Harutyunyan 2b820d790c Merge remote-tracking branch 'upstream/master' 2018-10-30 12:06:25 -05:00
dstockton 36286301f7 #30: Fix the Content-Length header for HTTP2->HTTP1 2018-10-30 11:52:11 -05:00
Nick Vollmar 9a48fe959d TUN-1158: Windows: use process arguments rather than trivial service arguments
TUN-1158: Fix segfault when carrier test case fails
2018-10-29 14:14:53 -05:00
Nick Vollmar f6014cb2b4 TUN-968: Flow control for large requests/responses 2018-10-26 11:16:02 -05:00
Nick Vollmar 9fe21fa906 Release 2018.10.4 2018-10-25 15:56:25 -05:00
Austin Cherry 80a75e91d2 AUTH-1188: UX Review and Changes for CLI SSH Access 2018-10-25 15:50:27 -05:00
Austin Cherry 6acc95f756 TUN-1097: Host missing from WebSocket request 2018-10-19 16:51:54 -05:00
Austin Cherry 72412db4c2 AUTH-1235: fixed packaging of deb dev file 2018-10-19 14:01:08 -05:00
Austin Cherry fa92441415 AUTH-1070: added SSH/protocol forwarding 2018-10-11 11:34:37 -05:00
Chung-Ting Huang 41916365b6 Release 2018.10.3 2018-10-10 15:19:36 -05:00
Adam Chalmers 41429cc6a8 TUN-1101: False negatives in Cloudflared error reporting 2018-10-10 14:59:33 -05:00
Austin Cherry da0defcec9 TUN-1098: removed deprecation error 2018-10-08 16:38:33 -05:00
Areg Harutyunyan ca9902a8d1 TUN-1099: Bring back changes in 2018.10.1 2018-10-08 14:20:28 -05:00
Areg Harutyunyan 995e773096 Release 2018.10.2 2018-10-06 14:36:41 -05:00
Areg Harutyunyan faeba02e57 TUN-1093: Revert cloudflared to 2018.8.0 2018-10-06 14:30:51 -05:00
364 changed files with 6348 additions and 20355 deletions
+2
View File
@@ -7,3 +7,5 @@ guide/public
.idea
.vscode
\#*\#
cscope.*
cloudflared
Generated
+185 -29
View File
@@ -2,31 +2,48 @@
[[projects]]
digest = "1:b16fbfbcc20645cb419f78325bb2e85ec729b338e996a228124d68931a6f2a37"
name = "github.com/BurntSushi/toml"
packages = ["."]
pruneopts = "UT"
revision = "b26d9c308763d68093482582cea63d69be07a0f0"
version = "v0.3.0"
[[projects]]
branch = "master"
digest = "1:d6afaeed1502aa28e80a4ed0981d570ad91b2579193404256ce672ed0a609e0d"
name = "github.com/beorn7/perks"
packages = ["quantile"]
pruneopts = "UT"
revision = "3a771d992973f24aa725d07868b467d1ddfceafb"
[[projects]]
digest = "1:fed1f537c2f1269fe475a8556c393fe466641682d73ef8fd0491cd3aa1e47bad"
name = "github.com/certifi/gocertifi"
packages = ["."]
pruneopts = "UT"
revision = "deb3ae2ef2610fde3330947281941c562861188b"
version = "2018.01.18"
[[projects]]
branch = "master"
digest = "1:e5003c19d396d8b3cf1324ea0bf49b00f13e9466d0297d1268b641f1c617c3a2"
name = "github.com/cloudflare/brotli-go"
packages = ["."]
pruneopts = "T"
revision = "18c9f6c67e3dfc12e0ddaca748d2887f97a7ac28"
[[projects]]
branch = "master"
digest = "1:6dbb2bbc7e6333e691c4d82fd86485f0695a35902fbb9b2df5f72e22ab0040f3"
name = "github.com/cloudflare/golibs"
packages = ["lrucache"]
pruneopts = "UT"
revision = "333127dbecfcc23a8db7d9a4f52785d23aff44a1"
[[projects]]
branch = "master"
digest = "1:af68c8bdac3ceaf46c67f0c43cc1ee5f10e705d8e04f8a63fa41231d0be6b0fb"
name = "github.com/coredns/coredns"
packages = [
"core/dnsserver",
@@ -52,52 +69,64 @@
"plugin/pkg/uniq",
"plugin/pkg/watch",
"plugin/test",
"request"
"request",
]
pruneopts = "UT"
revision = "992e7928c7c258628d2b13b769acc86781b9faea"
[[projects]]
branch = "master"
digest = "1:6a503e232df389d94ebb97dfb22d4ae463b6e2f351660613e11d9e42f57ab6df"
name = "github.com/coreos/go-oidc"
packages = [
"http",
"jose",
"key",
"oauth2",
"oidc"
"oidc",
]
pruneopts = "UT"
revision = "a93f71fdfe73d2c0f5413c0565eea0af6523a6df"
[[projects]]
digest = "1:1da3a221f0bc090792d3a2a080ff09008427c0e0f0533a4ed6abd8994421da73"
name = "github.com/coreos/go-systemd"
packages = ["daemon"]
pruneopts = "UT"
revision = "39ca1b05acc7ad1220e09f133283b8859a8b71ab"
version = "v17"
[[projects]]
digest = "1:6fda0d7f5e52b081e075775b1ecebf1ea0c923e7be33604ed0225ae078e701b5"
name = "github.com/coreos/pkg"
packages = [
"health",
"httputil",
"timeutil"
"timeutil",
]
pruneopts = "UT"
revision = "97fdf19511ea361ae1c100dd393cc47f8dcfa1e1"
version = "v4"
[[projects]]
digest = "1:a2c1d0e43bd3baaa071d1b9ed72c27d78169b2b269f71c105ac4ba34b1be4a39"
name = "github.com/davecgh/go-spew"
packages = ["spew"]
pruneopts = "UT"
revision = "346938d642f2ec3594ed81d874461961cd0faa76"
version = "v1.1.0"
[[projects]]
branch = "master"
digest = "1:c013ffc6e15f9f898078f9d38441c68b228aa7b899659452170250ccb27f5f1e"
name = "github.com/elgs/gosqljson"
packages = ["."]
pruneopts = "UT"
revision = "027aa4915315a0b2825c0f025cea347829b974fa"
[[projects]]
branch = "master"
digest = "1:15eadee5930d325154e5e1f3c4f4b28ab62f9c0aa949580e8d9edabaf0b64e85"
name = "github.com/equinox-io/equinox"
packages = [
".",
@@ -105,209 +134,269 @@
"internal/go-update/internal/binarydist",
"internal/go-update/internal/osext",
"internal/osext",
"proto"
"proto",
]
pruneopts = "UT"
revision = "f24972fa72facf59d05c91c848b65eac38815915"
[[projects]]
branch = "master"
digest = "1:433763f10d88dba9b533a7ea2fe9f5ee11e57e00306eb97a1f6090fd978e8fa1"
name = "github.com/facebookgo/grace"
packages = ["gracenet"]
pruneopts = "UT"
revision = "75cf19382434e82df4dd84953f566b8ad23d6e9e"
[[projects]]
branch = "master"
digest = "1:50a46ab1d5edbbdd55125b4d37f1bf503d0807c26461f9ad7b358d6006641d09"
name = "github.com/flynn/go-shlex"
packages = ["."]
pruneopts = "UT"
revision = "3f9db97f856818214da2e1057f8ad84803971cff"
[[projects]]
branch = "master"
digest = "1:d4623fc7bf7e281d9107367cc4a9e76ed3e86b1eec1a4e30630c870bef1fedd0"
name = "github.com/getsentry/raven-go"
packages = ["."]
pruneopts = "UT"
revision = "ed7bcb39ff10f39ab08e317ce16df282845852fa"
[[projects]]
branch = "master"
digest = "1:3e6afc3ed8a72949aa735c00fddc23427dc9384ccfd51cf0d91a412e668da632"
name = "github.com/golang-collections/collections"
packages = ["queue"]
pruneopts = "UT"
revision = "604e922904d35e97f98a774db7881f049cd8d970"
[[projects]]
digest = "1:17fe264ee908afc795734e8c4e63db2accabaf57326dbf21763a7d6b86096260"
name = "github.com/golang/protobuf"
packages = [
"proto",
"ptypes",
"ptypes/any",
"ptypes/duration",
"ptypes/timestamp"
"ptypes/timestamp",
]
pruneopts = "UT"
revision = "b4deda0973fb4c70b50d226b1af49f3da59f5265"
version = "v1.1.0"
[[projects]]
digest = "1:582b704bebaa06b48c29b0cec224a6058a09c86883aaddabde889cd1a5f73e1b"
name = "github.com/google/uuid"
packages = ["."]
revision = "064e2069ce9c359c118179501254f67d7d37ba24"
version = "0.2"
pruneopts = "UT"
revision = "0cd6bf5da1e1c83f8b45653022c74f71af0538a4"
version = "v1.1.1"
[[projects]]
digest = "1:c79fb010be38a59d657c48c6ba1d003a8aa651fa56b579d959d74573b7dff8e1"
name = "github.com/gorilla/context"
packages = ["."]
pruneopts = "UT"
revision = "08b5f424b9271eedf6f9f0ce86cb9396ed337a42"
version = "v1.1.1"
[[projects]]
digest = "1:e73f5b0152105f18bc131fba127d9949305c8693f8a762588a82a48f61756f5f"
name = "github.com/gorilla/mux"
packages = ["."]
pruneopts = "UT"
revision = "e3702bed27f0d39777b0b37b664b6280e8ef8fbf"
version = "v1.6.2"
[[projects]]
digest = "1:43dd08a10854b2056e615d1b1d22ac94559d822e1f8b6fcc92c1a1057e85188e"
name = "github.com/gorilla/websocket"
packages = ["."]
pruneopts = "UT"
revision = "ea4d1f681babbce9545c9c5f3d5194a789c89f5b"
version = "v1.2.0"
[[projects]]
branch = "master"
digest = "1:1a1206efd03a54d336dce7bb8719e74f2f8932f661cb9f57d5813a1d99c083d8"
name = "github.com/grpc-ecosystem/grpc-opentracing"
packages = ["go/otgrpc"]
pruneopts = "UT"
revision = "8e809c8a86450a29b90dcc9efbf062d0fe6d9746"
[[projects]]
digest = "1:75ab90ae3f5d876167e60f493beadfe66f0ed861a710f283fb06c86437a09538"
name = "github.com/jonboulle/clockwork"
packages = ["."]
pruneopts = "UT"
revision = "2eee05ed794112d45db504eb05aa693efd2b8b09"
version = "v0.1.0"
[[projects]]
branch = "master"
digest = "1:37ce7d7d80531b227023331002c0d42b4b4b291a96798c82a049d03a54ba79e4"
name = "github.com/lib/pq"
packages = [
".",
"oid"
"oid",
]
pruneopts = "UT"
revision = "90697d60dd844d5ef6ff15135d0203f65d2f53b8"
[[projects]]
digest = "1:c658e84ad3916da105a761660dcaeb01e63416c8ec7bc62256a9b411a05fcd67"
name = "github.com/mattn/go-colorable"
packages = ["."]
pruneopts = "UT"
revision = "167de6bfdfba052fa6b2d3664c8f5272e23c9072"
version = "v0.0.9"
[[projects]]
digest = "1:d4d17353dbd05cb52a2a52b7fe1771883b682806f68db442b436294926bbfafb"
name = "github.com/mattn/go-isatty"
packages = ["."]
pruneopts = "UT"
revision = "0360b2af4f38e8d38c7fce2a9f4e702702d73a39"
version = "v0.0.3"
[[projects]]
digest = "1:ff5ebae34cfbf047d505ee150de27e60570e8c394b3b8fdbb720ff6ac71985fc"
name = "github.com/matttproud/golang_protobuf_extensions"
packages = ["pbutil"]
pruneopts = "UT"
revision = "c12348ce28de40eed0136aa2b644d0ee0650e56c"
version = "v1.0.1"
[[projects]]
branch = "master"
digest = "1:75fa16a231ef40da3e462d651c20b9df20bde0777bdc1ac0982242c79057ee71"
name = "github.com/mholt/caddy"
packages = [
".",
"caddyfile",
"telemetry"
"telemetry",
]
pruneopts = "UT"
revision = "d3b731e9255b72d4571a5aac125634cf1b6031dc"
[[projects]]
digest = "1:73f788f2df39614dd7d9e75c8ca5fb8b4a68937b5e6d4a15e4c3a596f6f8c07b"
name = "github.com/miekg/dns"
packages = ["."]
revision = "5a2b9fab83ff0f8bfc99684bd5f43a37abe560f1"
version = "v1.0.8"
packages = [
".",
"internal/socket",
]
pruneopts = "UT"
revision = "6da3249dfb57fbaa16efafcd8744cee8809d80cd"
[[projects]]
branch = "master"
digest = "1:8eb17c2ec4df79193ae65b621cd1c0c4697db3bc317fe6afdc76d7f2746abd05"
name = "github.com/mitchellh/go-homedir"
packages = ["."]
pruneopts = "UT"
revision = "3864e76763d94a6df2f9960b16a20a33da9f9a66"
[[projects]]
digest = "1:450b7623b185031f3a456801155c8320209f75d0d4c4e633c6b1e59d44d6e392"
name = "github.com/opentracing/opentracing-go"
packages = [
".",
"ext",
"log"
"log",
]
pruneopts = "UT"
revision = "1949ddbfd147afd4d964a9f00b24eb291e0e7c38"
version = "v1.0.2"
[[projects]]
digest = "1:40e195917a951a8bf867cd05de2a46aaf1806c50cf92eebf4c16f78cd196f747"
name = "github.com/pkg/errors"
packages = ["."]
pruneopts = "UT"
revision = "645ef00459ed84a119197bfb8d8205042c6df63d"
version = "v0.8.0"
[[projects]]
digest = "1:0028cb19b2e4c3112225cd871870f2d9cf49b9b4276531f03438a88e94be86fe"
name = "github.com/pmezard/go-difflib"
packages = ["difflib"]
pruneopts = "UT"
revision = "792786c7400a136282c1664665ae0a8db921c6c2"
version = "v1.0.0"
[[projects]]
digest = "1:c968b29db5d68ec97de404b6d058d5937fa015a141b3b4f7a0d87d5f8226f04c"
name = "github.com/prometheus/client_golang"
packages = [
"prometheus",
"prometheus/promhttp"
"prometheus/promhttp",
]
pruneopts = "UT"
revision = "967789050ba94deca04a5e84cce8ad472ce313c1"
version = "v0.9.0-pre1"
[[projects]]
branch = "master"
digest = "1:32d10bdfa8f09ecf13598324dba86ab891f11db3c538b6a34d1c3b5b99d7c36b"
name = "github.com/prometheus/client_model"
packages = ["go"]
pruneopts = "UT"
revision = "99fa1f4be8e564e8a6b613da7fa6f46c9edafc6c"
[[projects]]
branch = "master"
digest = "1:e469cd65badf7694aeb44874518606d93c1d59e7735d3754ad442782437d3cc3"
name = "github.com/prometheus/common"
packages = [
"expfmt",
"internal/bitbucket.org/ww/goautoneg",
"model"
"model",
]
pruneopts = "UT"
revision = "7600349dcfe1abd18d72d3a1770870d9800a7801"
[[projects]]
branch = "master"
digest = "1:20d9bb50dbee172242f9bcd6ec24a917dd7a5bb17421bf16a79c33111dea7db1"
name = "github.com/prometheus/procfs"
packages = [
".",
"internal/util",
"nfs",
"xfs"
"xfs",
]
pruneopts = "UT"
revision = "ae68e2d4c00fed4943b5f6698d504a5fe083da8a"
[[projects]]
digest = "1:9a6f766efd8d5752adb7052aebb6e3d85255b31a8dff5e58ab4efa740ba9efa0"
name = "github.com/rifflock/lfshook"
packages = ["."]
pruneopts = "UT"
revision = "bf539943797a1f34c1f502d07de419b5238ae6c6"
version = "v2.3"
[[projects]]
digest = "1:9e9193aa51197513b3abcb108970d831fbcf40ef96aa845c4f03276e1fa316d2"
name = "github.com/sirupsen/logrus"
packages = ["."]
pruneopts = "UT"
revision = "c155da19408a8799da419ed3eeb0cb5db0ad5dbc"
version = "v1.0.5"
[[projects]]
digest = "1:18752d0b95816a1b777505a97f71c7467a8445b8ffb55631a7bf779f6ba4fa83"
name = "github.com/stretchr/testify"
packages = ["assert"]
pruneopts = "UT"
revision = "f35b8ab0b5a2cef36673838d662e249dd9c94686"
version = "v1.2.2"
[[projects]]
branch = "master"
digest = "1:1182d12da264c8fe593504f150c3a479297767991fc1c06796cb67e4f61c4366"
name = "golang.org/x/crypto"
packages = [
"curve25519",
@@ -318,38 +407,39 @@
"nacl/secretbox",
"poly1305",
"salsa20/salsa",
"ssh/terminal"
"ssh/terminal",
]
pruneopts = "UT"
revision = "a49355c7e3f8fe157a85be2f77e6e269a0f89602"
[[projects]]
branch = "master"
digest = "1:9513f8a0f1f6918181b7d744225b1931ffacafeb57067a1c26893a51058afb80"
name = "golang.org/x/net"
packages = [
"bpf",
"context",
"http/httpguts",
"http2",
"http2/hpack",
"idna",
"internal/iana",
"internal/socket",
"internal/timeseries",
"ipv4",
"ipv6",
"trace",
"websocket"
"websocket",
]
pruneopts = "UT"
revision = "32a936f46389aa10549d60bd7833e54b01685d09"
[[projects]]
branch = "master"
digest = "1:39ebcc2b11457b703ae9ee2e8cca0f68df21969c6102cb3b705f76cca0ea0239"
name = "golang.org/x/sync"
packages = ["errgroup"]
pruneopts = "UT"
revision = "1d60e4601c6fd243af51cc01ddf169918a5407ca"
[[projects]]
branch = "master"
digest = "1:2e8e74aa73847379a370c2704ff08baf4a7303ae682766e70c23ded3fd9b3f37"
name = "golang.org/x/sys"
packages = [
"unix",
@@ -357,11 +447,13 @@
"windows/registry",
"windows/svc",
"windows/svc/eventlog",
"windows/svc/mgr"
"windows/svc/mgr",
]
pruneopts = "UT"
revision = "ce36f3865eeb42541ce3f87f32f8462c5687befa"
[[projects]]
digest = "1:a2ab62866c75542dd18d2b069fec854577a20211d7c0ea6ae746072a1dccdd18"
name = "golang.org/x/text"
packages = [
"collate",
@@ -377,18 +469,22 @@
"unicode/bidi",
"unicode/cldr",
"unicode/norm",
"unicode/rangetable"
"unicode/rangetable",
]
pruneopts = "UT"
revision = "f21a4dfb5e38f5895301dc265a8def02365cc3d0"
version = "v0.3.0"
[[projects]]
branch = "master"
digest = "1:601e63e7d4577f907118bec825902505291918859d223bce015539e79f1160e3"
name = "google.golang.org/genproto"
packages = ["googleapis/rpc/status"]
pruneopts = "UT"
revision = "ff3583edef7de132f219f0efc00e097cabcc0ec0"
[[projects]]
digest = "1:2dab32a43451e320e49608ff4542fdfc653c95dcc35d0065ec9c6c3dd540ed74"
name = "google.golang.org/grpc"
packages = [
".",
@@ -415,28 +511,34 @@
"stats",
"status",
"tap",
"transport"
"transport",
]
pruneopts = "UT"
revision = "168a6198bcb0ef175f7dacec0b8691fc141dc9b8"
version = "v1.13.0"
[[projects]]
branch = "altsrc-parse-durations"
digest = "1:0370b1bceda03dbfade3abbde639a43f1113bab711ec760452e5c0dcc0c14787"
name = "gopkg.in/urfave/cli.v2"
packages = [
".",
"altsrc"
"altsrc",
]
pruneopts = "UT"
revision = "d604b6ffeee878fbf084fd2761466b6649989cee"
source = "https://github.com/cbranch/cli"
[[projects]]
digest = "1:342378ac4dcb378a5448dd723f0784ae519383532f5e70ade24132c4c8693202"
name = "gopkg.in/yaml.v2"
packages = ["."]
pruneopts = "UT"
revision = "5420a8b6744d3b0345ab293f6fcba19c978f1183"
version = "v2.2.1"
[[projects]]
digest = "1:8ffc3ddc31414c0a71220957bb723b16510d7fcb5b3880dc0da4cf6d39c31642"
name = "zombiezen.com/go/capnproto2"
packages = [
".",
@@ -452,8 +554,9 @@
"rpc/internal/refcount",
"schemas",
"server",
"std/capnp/rpc"
"std/capnp/rpc",
]
pruneopts = "UT"
revision = "7cfd211c19c7f5783c695f3654efa46f0df259c3"
source = "https://github.com/zombiezen/go-capnproto2"
version = "v2.17.1"
@@ -461,6 +564,59 @@
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "ee681bef3527e49801c841e313f98b40116eafe8b60be21273956eeb96487486"
input-imports = [
"github.com/cloudflare/brotli-go",
"github.com/cloudflare/golibs/lrucache",
"github.com/coredns/coredns/core/dnsserver",
"github.com/coredns/coredns/plugin",
"github.com/coredns/coredns/plugin/cache",
"github.com/coredns/coredns/plugin/metrics/vars",
"github.com/coredns/coredns/plugin/pkg/dnstest",
"github.com/coredns/coredns/plugin/pkg/rcode",
"github.com/coredns/coredns/request",
"github.com/coreos/go-oidc/jose",
"github.com/coreos/go-oidc/oidc",
"github.com/coreos/go-systemd/daemon",
"github.com/elgs/gosqljson",
"github.com/equinox-io/equinox",
"github.com/facebookgo/grace/gracenet",
"github.com/getsentry/raven-go",
"github.com/golang-collections/collections/queue",
"github.com/google/uuid",
"github.com/gorilla/mux",
"github.com/gorilla/websocket",
"github.com/lib/pq",
"github.com/mattn/go-colorable",
"github.com/miekg/dns",
"github.com/mitchellh/go-homedir",
"github.com/pkg/errors",
"github.com/prometheus/client_golang/prometheus",
"github.com/prometheus/client_golang/prometheus/promhttp",
"github.com/rifflock/lfshook",
"github.com/sirupsen/logrus",
"github.com/stretchr/testify/assert",
"golang.org/x/crypto/nacl/box",
"golang.org/x/crypto/ssh/terminal",
"golang.org/x/net/context",
"golang.org/x/net/http2",
"golang.org/x/net/http2/hpack",
"golang.org/x/net/idna",
"golang.org/x/net/trace",
"golang.org/x/net/websocket",
"golang.org/x/sync/errgroup",
"golang.org/x/sys/windows",
"golang.org/x/sys/windows/svc",
"golang.org/x/sys/windows/svc/eventlog",
"golang.org/x/sys/windows/svc/mgr",
"gopkg.in/urfave/cli.v2",
"gopkg.in/urfave/cli.v2/altsrc",
"zombiezen.com/go/capnproto2",
"zombiezen.com/go/capnproto2/encoding/text",
"zombiezen.com/go/capnproto2/pogs",
"zombiezen.com/go/capnproto2/rpc",
"zombiezen.com/go/capnproto2/schemas",
"zombiezen.com/go/capnproto2/server",
"zombiezen.com/go/capnproto2/std/capnp/rpc",
]
solver-name = "gps-cdcl"
solver-version = 1
+8
View File
@@ -58,6 +58,10 @@
name = "github.com/coredns/coredns"
branch = "master"
[[constraint]]
name = "github.com/miekg/dns"
revision = "6da3249dfb57fbaa16efafcd8744cee8809d80cd"
[[constraint]]
name = "github.com/cloudflare/brotli-go"
branch = "master"
@@ -73,3 +77,7 @@
[[constraint]]
branch = "master"
name = "golang.org/x/crypto"
[[constraint]]
branch = "master"
name = "github.com/cloudflare/golibs"
+9 -1
View File
@@ -16,9 +16,17 @@ ifeq ($(EQUINOX_IS_DRAFT), true)
EQUINOX_FLAGS := --draft $(EQUINOX_FLAGS)
endif
ifeq ($(GOARCH),)
GOARCH := amd64
endif
.PHONY: all
all: cloudflared test
.PHONY: clean
clean:
go clean
.PHONY: cloudflared
cloudflared:
go build -v $(VERSION_FLAGS) $(IMPORT_PATH)/cmd/cloudflared
@@ -32,7 +40,7 @@ cloudflared-deb: cloudflared
mkdir -p $(PACKAGE_DIR)
cp cloudflared $(PACKAGE_DIR)/cloudflared
fakeroot fpm -C $(PACKAGE_DIR) -s dir -t deb --deb-compression bzip2 \
-a $(GOARCH) -v $(VERSION) -n cloudflared
-a $(GOARCH) -v $(VERSION) -n cloudflared cloudflared=/usr/local/bin/
.PHONY: cloudflared-darwin-amd64.tgz
cloudflared-darwin-amd64.tgz: cloudflared
+93
View File
@@ -1,3 +1,96 @@
2019.4.0
- 2019-03-28 TUN-1648: ConnectionID is now a UUID
- 2019-04-01 TUN-1673: Conflate Hello and Connect RPCs
2019.3.2
- 2019-03-22 TUN-1637: Free tunnels shouldn't require cert.pem
- 2019-03-18 TUN-1604: Define Connect RPC call
2019.3.1
- 2019-03-09 Add rdp as a supported protocol in URL validation (#76)
- 2019-03-15 TUN-1613: improved cloudflared RegisterTunnel fail metrics
- 2019-03-17 TUN-1615: revert miekg/dns to last known working revision
2019.3.0
- 2018-12-28 make http transport aware of proxy from envvar
- 2019-02-28 TUN-1559: fix nil dereference in TunnelConfig.CloseConnOnce
- 2019-03-04 TUN-1451: Make non-interactive, non-service execution possible on Windows
- 2019-03-04 TUN-1562: Refactor connectedSignal to be safe to close multiple times
- 2019-02-27 TUN-1550: Add validation timeout for non-responsive origins
- 2019-03-06 AUTH-1531: Named flags for ssh service tokens
- 2019-02-14 Support unix sockets.
- 2019-03-08 TUN-1389: Non-scalar flags in a cloudflared config.yml don't get logged
- 2019-03-07 TUN-1522: If we can't get SRV from default resolver, get them from 1.1.1.1 DoT
2019.2.1
- 2019-02-14 TUN-1381: should tell you if you're on the latest version rather than just exiting silently
- 2019-02-15 TUN-1467: build with Go 1.11
- 2019-02-15 AUTH-1519: Added logging
- 2019-02-19 TUN-1525: cloudflared metrics for registration success/fail
- 2019-02-19 TUN-1510: Wrap the close() in sync.Once.Do
2019.2.0
- 2019-01-24 AUTH-1462: better curl arg parsing
- 2019-02-01 TUN-1456: Only make one UUID
- 2019-01-30 cloudflared/linux_service: Add missing /etc/init.d shebang
- 2019-02-07 AUTH-1511: Add custom headers for ssh command
- 2019-02-01 AUTH-1503: Added RDP support
- 2019-02-01 AUTH-1403: Print the paths in the ssh-config instructions
2019.1.0
- 2018-12-10 TUN-1231: Horizontal overflow wrapping on the Hello page
- 2018-12-17 TUN-1140: Show usage if invoked with no args or config
- 2018-11-06 TUN-632 Filter out common network exceptions from going to Sentry on StartServer
- 2019-01-07 TUN-1138: Install cloudflared service directory with 755 permissions
- 2019-01-07 TUN-1265: Silent exit when failing to parse config
- 2019-01-10 TUN-1350: Enhance error messages with cloudflarestatus.com link, if relevant
- 2019-01-16 TUN-1358: Close readyList after Muxer.Serve() has stopped running
- 2019-01-24 AUTH-1423: move from stdout to stderr
- 2019-01-24 AUTH-1404: reauth if the token is about to expire within 15 minutes
- 2019-01-24 AUTH-1459: improved ssh streaming error message
- 2019-01-24 AUTH-1211: print all the versions
- 2019-01-24 AUTH-1337: fix url path
- 2019-01-28 TUN-1418: Rename ProtocolLogger to TransportLogger, and use TransportLogger to log RPC events.
- 2019-01-28 TUN-1419: Identify request/response headers/content length with ray ID
2018.12.1
- 2018-12-11 TUN-1270: cloudflared panic (HA metrics missing label)
2018.12.0
- 2018-11-15 TUN-1196: Allow TLS config client CA and root CA to be constructed from multiple certificates
- 2018-11-20 TUN-1209: TLS Config Certificates and GetCertificate can both be set
- 2018-11-26 TUN-1212: Expose tunnel_id in metrics
- 2018-11-30 TUN-1204: remove 'cloudflared hello' command
- 2018-12-04 Fix license URL typo
- 2018-12-07 TUN-1250: ValidateHTTPService shouldn't follow 302s
2018.11.0
- 2018-10-31 AUTH-1282: Fixed an issue where we were receiving as opposed sending on the channel.
- 2018-11-06 TUN-1179: Fix log message in cmd/cloudflared/transfer.Run
- 2018-11-13 AUTH-1308: get jwt even when you are already logged in
- 2018-11-12 TUN-1190: check URL parse error when starting SSH proxy server
- 2018-11-15 AUTH-1320: Fixed request issue and unhide the ssh command
2018.10.5
- 2018-10-18 TUN-968: Flow control for large requests/responses
- 2018-10-26 TUN-1158: Windows: use process arguments rather than trivial service arguments
- 2018-10-20 #30: Fix the Content-Length header for HTTP2->HTTP1
- 2018-10-29 TUN-1160: pass Host header during origin url validation
2018.10.4
- 2018-09-21 AUTH-1070: added SSH/protocol forwarding
- 2018-10-19 AUTH-1235: fixed packaging of deb dev file
- 2018-10-19 TUN-1097: Host missing from WebSocket request
- 2018-10-19 AUTH-1188: UX Review and Changes for CLI SSH Access
2018.10.3
- 2018-10-08 TUN-1099: Bring back changes in 2018.10.1
- 2018-10-08 TUN-1098: removed deprecation error
- 2018-10-08 TUN-1101: False negatives in Cloudflared error reporting
2018.10.2
- 2018-10-06 TUN-1093: Revert cloudflared to 2018.8.0
2018.10.1
- 2018-10-03 TUN-1012: Normalize config filename for Linux services
- 2018-10-05 TUN-1081: cloudflared now generates UUIDs
+141
View File
@@ -0,0 +1,141 @@
//Package carrier provides a WebSocket proxy to carry or proxy a connection
//from the local client to the edge. See it as a wrapper around any protocol
//that it packages up in a WebSocket connection to the edge.
package carrier
import (
"errors"
"io"
"net"
"net/http"
"os"
"strings"
"github.com/cloudflare/cloudflared/cmd/cloudflared/token"
"github.com/cloudflare/cloudflared/websocket"
"github.com/sirupsen/logrus"
)
// StdinoutStream is empty struct for wrapping stdin/stdout
// into a single ReadWriter
type StdinoutStream struct {
}
// Read will read from Stdin
func (c *StdinoutStream) Read(p []byte) (int, error) {
return os.Stdin.Read(p)
}
// Write will write to Stdout
func (c *StdinoutStream) Write(p []byte) (int, error) {
return os.Stdout.Write(p)
}
// StartClient will copy the data from stdin/stdout over a WebSocket connection
// to the edge (originURL)
func StartClient(logger *logrus.Logger, originURL string, stream io.ReadWriter, headers http.Header) error {
return serveStream(logger, originURL, stream, headers)
}
// StartServer will setup a server on a specified port and copy data over a WebSocket connection
// to the edge (originURL)
func StartServer(logger *logrus.Logger, address, originURL string, shutdownC <-chan struct{}, headers http.Header) error {
listener, err := net.Listen("tcp", address)
if err != nil {
logger.WithError(err).Error("failed to start forwarding server")
return err
}
logger.Info("Started listening on ", address)
defer listener.Close()
for {
select {
case <-shutdownC:
return nil
default:
conn, err := listener.Accept()
if err != nil {
return err
}
go serveConnection(logger, conn, originURL, headers)
}
}
}
// serveConnection handles connections for the StartServer call
func serveConnection(logger *logrus.Logger, c net.Conn, originURL string, headers http.Header) {
defer c.Close()
serveStream(logger, originURL, c, headers)
}
// serveStream will serve the data over the WebSocket stream
func serveStream(logger *logrus.Logger, originURL string, conn io.ReadWriter, headers http.Header) error {
wsConn, err := createWebsocketStream(originURL, headers)
if err != nil {
logger.WithError(err).Errorf("failed to connect to %s\n", originURL)
return err
}
defer wsConn.Close()
websocket.Stream(wsConn, conn)
return nil
}
// createWebsocketStream will create a WebSocket connection to stream data over
// It also handles redirects from Access and will present that flow if
// the token is not present on the request
func createWebsocketStream(originURL string, headers http.Header) (*websocket.Conn, error) {
req, err := http.NewRequest(http.MethodGet, originURL, nil)
if err != nil {
return nil, err
}
req.Header = headers
wsConn, resp, err := websocket.ClientConnect(req, nil)
if err != nil && resp != nil && resp.StatusCode > 300 {
location, err := resp.Location()
if err != nil {
return nil, err
}
if !strings.Contains(location.String(), "cdn-cgi/access/login") {
return nil, errors.New("not an Access redirect")
}
req, err := buildAccessRequest(originURL)
if err != nil {
return nil, err
}
wsConn, _, err = websocket.ClientConnect(req, nil)
if err != nil {
return nil, err
}
} else if err != nil {
return nil, err
}
return &websocket.Conn{Conn: wsConn}, nil
}
// buildAccessRequest builds an HTTP request with the Access token set
func buildAccessRequest(originURL string) (*http.Request, error) {
req, err := http.NewRequest(http.MethodGet, originURL, nil)
if err != nil {
return nil, err
}
token, err := token.FetchToken(req.URL)
if err != nil {
return nil, err
}
// We need to create a new request as FetchToken will modify req (boo mutable)
// as it has to follow redirect on the API and such, so here we init a new one
originRequest, err := http.NewRequest(http.MethodGet, originURL, nil)
if err != nil {
return nil, err
}
originRequest.Header.Set("cf-access-token", token)
return originRequest, nil
}
+122
View File
@@ -0,0 +1,122 @@
package carrier
import (
"bytes"
"io"
"net"
"net/http"
"net/http/httptest"
"sync"
"testing"
ws "github.com/gorilla/websocket"
"github.com/sirupsen/logrus"
"github.com/stretchr/testify/assert"
)
const (
// example in Sec-Websocket-Key in rfc6455
testSecWebsocketKey = "dGhlIHNhbXBsZSBub25jZQ=="
)
type testStreamer struct {
buf *bytes.Buffer
l sync.RWMutex
}
func newTestStream() *testStreamer {
return &testStreamer{buf: new(bytes.Buffer)}
}
func (s *testStreamer) Read(p []byte) (int, error) {
s.l.RLock()
defer s.l.RUnlock()
return s.buf.Read(p)
}
func (s *testStreamer) Write(p []byte) (int, error) {
s.l.Lock()
defer s.l.Unlock()
return s.buf.Write(p)
}
func TestStartClient(t *testing.T) {
message := "Good morning Austin! Time for another sunny day in the great state of Texas."
logger := logrus.New()
ts := newTestWebSocketServer()
defer ts.Close()
buf := newTestStream()
err := StartClient(logger, "http://"+ts.Listener.Addr().String(), buf, nil)
assert.NoError(t, err)
buf.Write([]byte(message))
readBuffer := make([]byte, len(message))
buf.Read(readBuffer)
assert.Equal(t, message, string(readBuffer))
}
func TestStartServer(t *testing.T) {
listenerAddress := "localhost:1117"
message := "Good morning Austin! Time for another sunny day in the great state of Texas."
logger := logrus.New()
shutdownC := make(chan struct{})
ts := newTestWebSocketServer()
defer ts.Close()
go func() {
err := StartServer(logger, listenerAddress, "http://"+ts.Listener.Addr().String(), shutdownC, nil)
if err != nil {
t.Fatalf("Error starting server: %v", err)
}
}()
conn, err := net.Dial("tcp", listenerAddress)
if err != nil {
t.Fatalf("Error connecting to server: %v", err)
}
conn.Write([]byte(message))
readBuffer := make([]byte, len(message))
conn.Read(readBuffer)
assert.Equal(t, string(readBuffer), message)
}
func newTestWebSocketServer() *httptest.Server {
upgrader := ws.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
}
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
conn, _ := upgrader.Upgrade(w, r, nil)
defer conn.Close()
for {
mt, message, err := conn.ReadMessage()
if err != nil {
break
}
if err := conn.WriteMessage(mt, []byte(message)); err != nil {
break
}
}
}))
}
func testRequest(t *testing.T, url string, stream io.ReadWriter) *http.Request {
req, err := http.NewRequest("GET", url, stream)
if err != nil {
t.Fatalf("testRequestHeader error")
}
req.Header.Add("Connection", "Upgrade")
req.Header.Add("Upgrade", "WebSocket")
req.Header.Add("Sec-Websocket-Key", testSecWebsocketKey)
req.Header.Add("Sec-Websocket-Protocol", "tunnel-protocol")
req.Header.Add("Sec-Websocket-Version", "13")
req.Header.Add("User-Agent", "curl/7.59.0")
return req
}
+4 -2
View File
@@ -1,6 +1,6 @@
pinned_go: &pinned_go go=1.9.3-1
pinned_go: &pinned_go go=1.11.5-1
build_dir: &build_dir /cfsetup_build/src/github.com/cloudflare/cloudflared/
stretch:
stretch: &stretch
build:
build_dir: *build_dir
builddeps:
@@ -87,3 +87,5 @@ stretch:
- export GOOS=linux
- export GOARCH=amd64
- make test
jessie: *stretch
+58
View File
@@ -0,0 +1,58 @@
package access
import (
"net/http"
"net/url"
"strings"
"github.com/cloudflare/cloudflared/carrier"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/validation"
"github.com/pkg/errors"
cli "gopkg.in/urfave/cli.v2"
)
// ssh will start a WS proxy server for server mode
// or copy from stdin/stdout for client mode
// useful for proxying other protocols (like ssh) over websockets
// (which you can put Access in front of)
func ssh(c *cli.Context) error {
hostname, err := validation.ValidateHostname(c.String("hostname"))
if err != nil || c.String("hostname") == "" {
return cli.ShowCommandHelp(c, "ssh")
}
headers := buildRequestHeaders(c.StringSlice("header"))
if c.IsSet("service-token-id") {
headers.Add("CF-Access-Client-Id", c.String("service-token-id"))
}
if c.IsSet("service-token-secret") {
headers.Add("CF-Access-Client-Secret", c.String("service-token-secret"))
}
if c.NArg() > 0 || c.IsSet("url") {
localForwarder, err := config.ValidateUrl(c)
if err != nil {
logger.WithError(err).Error("Error validating origin URL")
return errors.Wrap(err, "error validating origin URL")
}
forwarder, err := url.Parse(localForwarder)
if err != nil {
logger.WithError(err).Error("Error validating origin URL")
return errors.Wrap(err, "error validating origin URL")
}
return carrier.StartServer(logger, forwarder.Host, "https://"+hostname, shutdownC, headers)
}
return carrier.StartClient(logger, "https://"+hostname, &carrier.StdinoutStream{}, headers)
}
func buildRequestHeaders(values []string) http.Header {
headers := make(http.Header)
for _, valuePair := range values {
split := strings.Split(valuePair, ":")
if len(split) > 1 {
headers.Add(strings.TrimSpace(split[0]), strings.TrimSpace(split[1]))
}
}
return headers
}
+18
View File
@@ -0,0 +1,18 @@
package access
import (
"net/http"
"testing"
"github.com/stretchr/testify/assert"
)
func TestBuildRequestHeaders(t *testing.T) {
headers := make(http.Header)
headers.Add("client", "value")
headers.Add("secret", "safe-value")
values := buildRequestHeaders([]string{"client: value", "secret: safe-value", "trash"})
assert.Equal(t, headers.Get("client"), values.Get("client"))
assert.Equal(t, headers.Get("secret"), values.Get("secret"))
}
+123 -39
View File
@@ -5,8 +5,10 @@ import (
"fmt"
"net/url"
"os"
"strings"
"github.com/cloudflare/cloudflared/cmd/cloudflared/shell"
"github.com/cloudflare/cloudflared/cmd/cloudflared/token"
"golang.org/x/net/idna"
"github.com/cloudflare/cloudflared/log"
@@ -16,6 +18,17 @@ import (
const sentryDSN = "https://56a9c9fa5c364ab28f34b14f35ea0f1b@sentry.io/189878"
var (
logger = log.CreateLogger()
shutdownC chan struct{}
graceShutdownC chan struct{}
)
// Init will initialize and store vars from the main program
func Init(s, g chan struct{}) {
shutdownC, graceShutdownC = s, g
}
// Flags return the global flags for Access related commands (hopefully none)
func Flags() []cli.Flag {
return []cli.Flag{} // no flags yet.
@@ -53,7 +66,7 @@ func Commands() []*cli.Command {
{
Name: "curl",
Action: curl,
Usage: "curl <args>",
Usage: "curl [--allow-request, -ar] <url> [<curl args>...]",
Description: `The curl subcommand wraps curl and automatically injects the JWT into a cf-access-token
header when using curl to reach an application behind Access.`,
ArgsUsage: "allow-request will allow the curl request to continue even if the jwt is not present.",
@@ -61,7 +74,7 @@ func Commands() []*cli.Command {
},
{
Name: "token",
Action: token,
Action: generateToken,
Usage: "token -app=<url of access application>",
ArgsUsage: "url of Access application",
Description: `The token subcommand produces a JWT which can be used to authenticate requests.`,
@@ -71,6 +84,45 @@ func Commands() []*cli.Command {
},
},
},
{
Name: "ssh",
Action: ssh,
Aliases: []string{"rdp"},
Usage: "",
ArgsUsage: "",
Description: `The ssh subcommand sends data over a proxy to the Cloudflare edge.`,
Flags: []cli.Flag{
&cli.StringFlag{
Name: "hostname",
Usage: "specifics the hostname of your application.",
},
&cli.StringFlag{
Name: "url",
Usage: "specifics the host:port to forward data to Cloudflare edge.",
},
&cli.StringSliceFlag{
Name: "header",
Aliases: []string{"H"},
Usage: "specific additional headers you wish to send.",
},
&cli.StringSliceFlag{
Name: "service-token-id",
Aliases: []string{"id"},
Usage: "specific an Access service token ID you wish to use.",
},
&cli.StringSliceFlag{
Name: "service-token-secret",
Aliases: []string{"secret"},
Usage: "specific an Access service token secret you wish to use.",
},
},
},
{
Name: "ssh-config",
Action: sshConfig,
Usage: "ssh-config",
Description: `Prints an example configuration ~/.ssh/config`,
},
},
},
}
@@ -86,10 +138,12 @@ func login(c *cli.Context) error {
logger.Errorf("Please provide the url of the Access application\n")
return err
}
if _, err := fetchToken(c, appURL); err != nil {
token, err := token.FetchToken(appURL)
if err != nil {
logger.Errorf("Failed to fetch token: %s\n", err)
return err
}
fmt.Fprintf(os.Stdout, "Successfully fetched your token:\n\n%s\n\n", string(token))
return nil
}
@@ -104,18 +158,19 @@ func curl(c *cli.Context) error {
return errors.New("incorrect args")
}
cmdArgs, appURL, allowRequest, err := buildCurlCmdArgs(args.Slice())
cmdArgs, allowRequest := parseAllowRequest(args.Slice())
appURL, err := getAppURL(cmdArgs)
if err != nil {
return err
}
token, err := getTokenIfExists(appURL)
if err != nil || token == "" {
tok, err := token.GetTokenIfExists(appURL)
if err != nil || tok == "" {
if allowRequest {
logger.Warn("You don't have an Access token set. Please run access token <access application> to fetch one.")
return shell.Run("curl", cmdArgs...)
}
token, err = fetchToken(c, appURL)
tok, err = token.FetchToken(appURL)
if err != nil {
logger.Error("Failed to refresh token: ", err)
return err
@@ -123,37 +178,77 @@ func curl(c *cli.Context) error {
}
cmdArgs = append(cmdArgs, "-H")
cmdArgs = append(cmdArgs, fmt.Sprintf("cf-access-token: %s", token))
cmdArgs = append(cmdArgs, fmt.Sprintf("cf-access-token: %s", tok))
return shell.Run("curl", cmdArgs...)
}
// token dumps provided token to stdout
func token(c *cli.Context) error {
func generateToken(c *cli.Context) error {
raven.SetDSN(sentryDSN)
appURL, err := url.Parse(c.String("app"))
if err != nil || c.NumFlags() < 1 {
fmt.Fprintln(os.Stderr, "Please provide a url.")
return err
}
token, err := getTokenIfExists(appURL)
if err != nil || token == "" {
tok, err := token.GetTokenIfExists(appURL)
if err != nil || tok == "" {
fmt.Fprintln(os.Stderr, "Unable to find token for provided application. Please run token command to generate token.")
return err
}
if _, err := fmt.Fprint(os.Stdout, token); err != nil {
if _, err := fmt.Fprint(os.Stdout, tok); err != nil {
fmt.Fprintln(os.Stderr, "Failed to write token to stdout.")
return err
}
return nil
}
// sshConfig prints an example SSH config to stdout
func sshConfig(c *cli.Context) error {
outputMessage := "Add this configuration block to your %s/.ssh/config:\n\nHost [your hostname]\n\tProxyCommand %s access ssh --hostname %%h\n"
logger.Printf(outputMessage, os.Getenv("HOME"), cloudflaredPath())
return nil
}
// getAppURL will pull the appURL needed for fetching a user's Access token
func getAppURL(cmdArgs []string) (*url.URL, error) {
if len(cmdArgs) < 1 {
logger.Error("Please provide a valid URL as the first argument to curl.")
return nil, errors.New("not a valid url")
}
u, err := processURL(cmdArgs[0])
if err != nil {
logger.Error("Please provide a valid URL as the first argument to curl.")
return nil, err
}
return u, err
}
// parseAllowRequest will parse cmdArgs and return a copy of the args and result
// of the allow request was present
func parseAllowRequest(cmdArgs []string) ([]string, bool) {
if len(cmdArgs) > 1 {
if cmdArgs[0] == "--allow-request" || cmdArgs[0] == "-ar" {
return cmdArgs[1:], true
}
}
return cmdArgs, false
}
// processURL will preprocess the string (parse to a url, convert to punycode, etc).
func processURL(s string) (*url.URL, error) {
u, err := url.ParseRequestURI(s)
if err != nil {
return nil, err
}
if u.Host == "" {
return nil, errors.New("not a valid host")
}
host, err := idna.ToASCII(u.Hostname())
if err != nil { // we fail to convert to punycode, just return the url we parsed.
return u, nil
@@ -167,33 +262,22 @@ func processURL(s string) (*url.URL, error) {
return u, nil
}
// buildCurlCmdArgs will build the curl cmd args
func buildCurlCmdArgs(cmdArgs []string) ([]string, *url.URL, bool, error) {
allowRequest, iAllowRequest := false, 0
var appURL *url.URL
for i, arg := range cmdArgs {
if arg == "-allow-request" || arg == "-ar" {
iAllowRequest = i
allowRequest = true
}
u, err := processURL(arg)
if err == nil {
appURL = u
cmdArgs[i] = appURL.String()
// cloudflaredPath pulls the full path of cloudflared on disk
func cloudflaredPath() string {
for _, p := range strings.Split(os.Getenv("PATH"), ":") {
path := fmt.Sprintf("%s/%s", p, "cloudflared")
if isFileThere(path) {
return path
}
}
if appURL == nil {
logger.Error("Please provide a valid URL.")
return cmdArgs, appURL, allowRequest, errors.New("invalid url")
}
if allowRequest {
// remove from cmdArgs
cmdArgs[iAllowRequest] = cmdArgs[len(cmdArgs)-1]
cmdArgs = cmdArgs[:len(cmdArgs)-1]
}
return cmdArgs, appURL, allowRequest, nil
return "cloudflared"
}
// isFileThere will check for the presence of candidate path
func isFileThere(candidate string) bool {
fi, err := os.Stat(candidate)
if err != nil || fi.IsDir() || !fi.Mode().IsRegular() {
return false
}
return true
}
+25
View File
@@ -1,9 +1,11 @@
package config
import (
"errors"
"os"
"path/filepath"
"github.com/cloudflare/cloudflared/validation"
homedir "github.com/mitchellh/go-homedir"
"gopkg.in/urfave/cli.v2"
"gopkg.in/urfave/cli.v2/altsrc"
@@ -60,3 +62,26 @@ func FindDefaultConfigPath() string {
}
return ""
}
// ValidateUnixSocket ensures --unix-socket param is used exclusively
// i.e. it fails if a user specifies both --url and --unix-socket
func ValidateUnixSocket(c *cli.Context) (string, error) {
if c.IsSet("unix-socket") && (c.IsSet("url") || c.NArg() > 0) {
return "", errors.New("--unix-socket must be used exclusivly.")
}
return c.String("unix-socket"), nil
}
// ValidateUrl will validate url flag correctness. It can be either from --url or argument
// Notice ValidateUnixSocket, it will enforce --unix-socket is not used with --url or argument
func ValidateUrl(c *cli.Context) (string, error) {
var url = c.String("url")
if c.NArg() > 0 {
if c.IsSet("url") {
return "", errors.New("Specified origin urls using both --url and argument. Decide which one you want, I can only support one.")
}
url = c.Args().Get(0)
}
validUrl, err := validation.ValidateUrl(url)
return validUrl, err
}
+2 -1
View File
@@ -84,7 +84,8 @@ WantedBy=timers.target
var sysvTemplate = ServiceTemplate{
Path: "/etc/init.d/cloudflared",
FileMode: 0755,
Content: `# For RedHat and cousins:
Content: `#!/bin/sh
# For RedHat and cousins:
# chkconfig: 2345 99 01
# description: Argo Tunnel agent
# processname: {{.Path}}
+48 -13
View File
@@ -2,11 +2,9 @@ package main
import (
"fmt"
"os"
"strings"
"time"
"golang.org/x/crypto/ssh/terminal"
"github.com/cloudflare/cloudflared/cmd/cloudflared/access"
"github.com/cloudflare/cloudflared/cmd/cloudflared/tunnel"
"github.com/cloudflare/cloudflared/cmd/cloudflared/updater"
@@ -22,13 +20,25 @@ import (
const (
developerPortal = "https://developers.cloudflare.com/argo-tunnel"
licenseUrl = developerPortal + "/licence/"
licenseUrl = developerPortal + "/license/"
versionText = "Print the version"
)
var (
Version = "DEV"
BuildTime = "unknown"
logger = log.CreateLogger()
// Mostly network errors that we don't want reported back to Sentry, this is done by substring match.
ignoredErrors = []string{
"connection reset by peer",
"An existing connection was forcibly closed by the remote host.",
"use of closed connection",
"You need to enable Argo Smart Routing",
"3001 connection closed",
"3002 connection dropped",
"rpc exception: dial tcp",
"rpc exception: EOF",
}
)
func main() {
@@ -42,6 +52,12 @@ func main() {
// Windows service manager closes this channel when it receives stop command.
graceShutdownC := make(chan struct{})
cli.VersionFlag = &cli.BoolFlag{
Name: "version",
Aliases: []string{"v", "V"},
Usage: versionText,
}
app := &cli.App{}
app.Name = "cloudflared"
app.Usage = "Cloudflare's command-line tool and agent"
@@ -55,13 +71,14 @@ func main() {
app.Flags = flags()
app.Action = action(Version, shutdownC, graceShutdownC)
app.Before = tunnel.Before
app.Commands = commands()
app.Commands = commands(cli.ShowVersion)
tunnel.Init(Version, shutdownC, graceShutdownC) // we need this to support the tunnel sub command...
access.Init(shutdownC, graceShutdownC)
runApp(app, shutdownC, graceShutdownC)
}
func commands() []*cli.Command {
func commands(version func(c *cli.Context)) []*cli.Command {
cmds := []*cli.Command{
{
Name: "update",
@@ -74,6 +91,15 @@ Otherwise, does nothing.
To determine if an update happened in a script, check for error code 64.`,
},
{
Name: "version",
Action: func(c *cli.Context) (err error) {
version(c)
return nil
},
Usage: versionText,
Description: versionText,
},
}
cmds = append(cmds, tunnel.Commands()...)
cmds = append(cmds, access.Commands()...)
@@ -85,19 +111,21 @@ func flags() []cli.Flag {
return append(flags, access.Flags()...)
}
func isEmptyInvocation(c *cli.Context) bool {
return c.NArg() == 0 && c.NumFlags() == 0
}
func action(version string, shutdownC, graceShutdownC chan struct{}) cli.ActionFunc {
return func(c *cli.Context) (err error) {
if isRunningFromTerminal() {
logger.Error("Use of cloudflared without commands is deprecated.")
cli.ShowAppHelp(c)
return nil
if isEmptyInvocation(c) {
cli.ShowAppHelpAndExit(c, 1)
}
tags := make(map[string]string)
tags["hostname"] = c.String("hostname")
raven.SetTagsContext(tags)
raven.CapturePanic(func() { err = tunnel.StartServer(c, version, shutdownC, graceShutdownC) }, nil)
if err != nil {
raven.CaptureError(err, nil)
handleError(err)
}
return err
}
@@ -116,6 +144,13 @@ func userHomeDir() (string, error) {
return homeDir, nil
}
func isRunningFromTerminal() bool {
return terminal.IsTerminal(int(os.Stdout.Fd()))
// In order to keep the amount of noise sent to Sentry low, typical network errors can be filtered out here by a substring match.
func handleError(err error) {
errorMessage := err.Error()
for _, ignoredErrorMessage := range ignoredErrors {
if strings.Contains(errorMessage, ignoredErrorMessage) {
return
}
}
raven.CaptureError(err, nil)
}
+1 -1
View File
@@ -96,7 +96,7 @@ func runCommand(command string, args ...string) error {
func ensureConfigDirExists(configDir string) error {
ok, err := config.FileExists(configDir)
if !ok && err == nil {
err = os.Mkdir(configDir, 0700)
err = os.Mkdir(configDir, 0755)
}
return err
}
@@ -1,4 +1,4 @@
package access
package token
import (
"fmt"
@@ -15,15 +15,13 @@ import (
"github.com/coreos/go-oidc/jose"
"github.com/coreos/go-oidc/oidc"
homedir "github.com/mitchellh/go-homedir"
cli "gopkg.in/urfave/cli.v2"
)
var logger = log.CreateLogger()
// fetchToken will either load a stored token or generate a new one
func fetchToken(c *cli.Context, appURL *url.URL) (string, error) {
if token, err := getTokenIfExists(appURL); token != "" && err == nil {
fmt.Fprintf(os.Stdout, "You have an existing token:\n\n%s\n\n", token)
// FetchToken will either load a stored token or generate a new one
func FetchToken(appURL *url.URL) (string, error) {
if token, err := GetTokenIfExists(appURL); token != "" && err == nil {
return token, nil
}
@@ -36,17 +34,16 @@ func fetchToken(c *cli.Context, appURL *url.URL) (string, error) {
// we want to send to the transfer service. the key is token and the value
// is blank (basically just the id generated in the transfer service)
const resourceName, key, value = "token", "token", ""
token, err := transfer.Run(c, appURL, resourceName, key, value, path, true)
token, err := transfer.Run(appURL, resourceName, key, value, path, true)
if err != nil {
return "", err
}
fmt.Fprintf(os.Stdout, "Successfully fetched your token:\n\n%s\n\n", string(token))
return string(token), nil
}
// getTokenIfExists will return the token from local storage if it exists
func getTokenIfExists(url *url.URL) (string, error) {
// GetTokenIfExists will return the token from local storage if it exists
func GetTokenIfExists(url *url.URL) (string, error) {
path, err := generateFilePathForTokenURL(url)
if err != nil {
return "", err
@@ -65,7 +62,8 @@ func getTokenIfExists(url *url.URL) (string, error) {
return "", err
}
ident, err := oidc.IdentityFromClaims(claims)
if err == nil && ident.ExpiresAt.After(time.Now()) {
// AUTH-1404, reauth if the token is about to expire within 15 minutes
if err == nil && ident.ExpiresAt.After(time.Now().Add(time.Minute*15)) {
return token.Encode(), nil
}
return "", err
+13 -22
View File
@@ -10,13 +10,11 @@ import (
"net/http"
"net/url"
"os"
"path/filepath"
"time"
"github.com/cloudflare/cloudflared/cmd/cloudflared/encrypter"
"github.com/cloudflare/cloudflared/cmd/cloudflared/shell"
"github.com/cloudflare/cloudflared/log"
cli "gopkg.in/urfave/cli.v2"
)
const (
@@ -32,7 +30,7 @@ var logger = log.CreateLogger()
// The "dance" we refer to is building a HTTP request, opening that in a browser waiting for
// the user to complete an action, while it long polls in the background waiting for an
// action to be completed to download the resource.
func Run(c *cli.Context, transferURL *url.URL, resourceName, key, value, path string, shouldEncrypt bool) ([]byte, error) {
func Run(transferURL *url.URL, resourceName, key, value, path string, shouldEncrypt bool) ([]byte, error) {
encrypterClient, err := encrypter.New("cloudflared_priv.pem", "cloudflared_pub.pem")
if err != nil {
return nil, err
@@ -42,23 +40,18 @@ func Run(c *cli.Context, transferURL *url.URL, resourceName, key, value, path st
return nil, err
}
// See AUTH-1423 for why we use stderr (the way git wraps ssh)
err = shell.OpenBrowser(requestURL)
if err != nil {
fmt.Fprintf(os.Stdout, "Please open the following URL and log in with your Cloudflare account:\n\n%s\n\nLeave cloudflared running to download the %s automatically.\n", resourceName, requestURL)
fmt.Fprintf(os.Stderr, "Please open the following URL and log in with your Cloudflare account:\n\n%s\n\nLeave cloudflared running to download the %s automatically.\n", requestURL, resourceName)
} else {
fmt.Fprintf(os.Stdout, "A browser window should have opened at the following URL:\n\n%s\n\nIf the browser failed to open, open it yourself and visit the URL above.\n", requestURL)
}
// for local debugging
baseURL := baseStoreURL
if c.IsSet("url") {
baseURL = c.String("url")
fmt.Fprintf(os.Stderr, "A browser window should have opened at the following URL:\n\n%s\n\nIf the browser failed to open, open it yourself and visit the URL above.\n", requestURL)
}
var resourceData []byte
if shouldEncrypt {
buf, key, err := transferRequest(baseURL + filepath.Join("transfer", encrypterClient.PublicKey()))
buf, key, err := transferRequest(baseStoreURL + "transfer/" + encrypterClient.PublicKey())
if err != nil {
return nil, err
}
@@ -74,7 +67,7 @@ func Run(c *cli.Context, transferURL *url.URL, resourceName, key, value, path st
resourceData = decrypted
} else {
buf, _, err := transferRequest(baseURL + filepath.Join(encrypterClient.PublicKey()))
buf, _, err := transferRequest(baseStoreURL + encrypterClient.PublicKey())
if err != nil {
return nil, err
}
@@ -90,21 +83,19 @@ func Run(c *cli.Context, transferURL *url.URL, resourceName, key, value, path st
// BuildRequestURL creates a request suitable for a resource transfer.
// it will return a constructed url based off the base url and query key/value provided.
// follow will follow redirects.
func buildRequestURL(baseURL *url.URL, key, value string, follow bool) (string, error) {
// cli will build a url for cli transfer request.
func buildRequestURL(baseURL *url.URL, key, value string, cli bool) (string, error) {
q := baseURL.Query()
q.Set(key, value)
baseURL.RawQuery = q.Encode()
if !follow {
if !cli {
return baseURL.String(), nil
}
response, err := http.Get(baseURL.String())
if err != nil {
return "", err
}
return response.Request.URL.String(), nil
q.Set("redirect_url", baseURL.String()) // we add the token as a query param on both the redirect_url
baseURL.RawQuery = q.Encode() // and this actual baseURL.
baseURL.Path = "cdn-cgi/access/cli"
return baseURL.String(), nil
}
// transferRequest downloads the requested resource from the request URL
+71 -36
View File
@@ -3,6 +3,8 @@ package tunnel
import (
"fmt"
"io/ioutil"
"net"
"net/url"
"os"
"runtime/trace"
"sync"
@@ -18,11 +20,13 @@ import (
"github.com/cloudflare/cloudflared/hello"
"github.com/cloudflare/cloudflared/metrics"
"github.com/cloudflare/cloudflared/origin"
"github.com/cloudflare/cloudflared/signal"
"github.com/cloudflare/cloudflared/tunneldns"
"github.com/cloudflare/cloudflared/websocket"
"github.com/coreos/go-systemd/daemon"
"github.com/facebookgo/grace/gracenet"
"github.com/pkg/errors"
cli "gopkg.in/urfave/cli.v2"
"gopkg.in/urfave/cli.v2"
"gopkg.in/urfave/cli.v2/altsrc"
)
@@ -53,20 +57,6 @@ func Commands() []*cli.Command {
},
Hidden: true,
},
{
Name: "hello",
Action: helloWorld,
Usage: "Run a simple \"Hello World\" server for testing Argo Tunnel.",
Flags: []cli.Flag{
&cli.IntFlag{
Name: "port",
Usage: "Listen on the selected port.",
Value: 8080,
},
},
ArgsUsage: " ", // can't be the empty string or we get the default output
Hidden: true,
},
{
Name: "proxy-dns",
Action: tunneldns.Run,
@@ -187,11 +177,11 @@ func Init(v string, s, g chan struct{}) {
}
func StartServer(c *cli.Context, version string, shutdownC, graceShutdownC chan struct{}) error {
raven.SetDSN(sentryDSN)
_ = raven.SetDSN(sentryDSN)
var wg sync.WaitGroup
listeners := gracenet.Net{}
errC := make(chan error)
connectedSignal := make(chan struct{})
connectedSignal := signal.New(make(chan struct{}))
dnsReadySignal := make(chan struct{})
if c.String("config") == "" {
@@ -202,9 +192,9 @@ func StartServer(c *cli.Context, version string, shutdownC, graceShutdownC chan
return errors.Wrap(err, "Error configuring logger")
}
protoLogger, err := configProtoLogger(c)
transportLogger, err := configTransportLogger(c)
if err != nil {
return errors.Wrap(err, "Error configuring protocol logger")
return errors.Wrap(err, "Error configuring transport logger")
}
if c.IsSet("trace-output") {
@@ -220,7 +210,10 @@ func StartServer(c *cli.Context, version string, shutdownC, graceShutdownC chan
if err := os.Rename(tmpTraceFile.Name(), c.String("trace-output")); err != nil {
logger.WithError(err).Errorf("Failed to rename temporary trace output file %s to %s", tmpTraceFile.Name(), c.String("trace-output"))
} else {
os.Remove(tmpTraceFile.Name())
err := os.Remove(tmpTraceFile.Name())
if err != nil {
logger.WithError(err).Errorf("Failed to remove the temporary trace file %s", tmpTraceFile.Name())
}
}
}()
@@ -232,7 +225,7 @@ func StartServer(c *cli.Context, version string, shutdownC, graceShutdownC chan
}
if c.String("logfile") != "" {
if err := initLogFile(c, logger, protoLogger); err != nil {
if err := initLogFile(c, logger, transportLogger); err != nil {
logger.Error(err)
}
}
@@ -288,7 +281,7 @@ func StartServer(c *cli.Context, version string, shutdownC, graceShutdownC chan
// Serve DNS proxy stand-alone if no hostname or tag or app is going to run
if dnsProxyStandAlone(c) {
close(connectedSignal)
connectedSignal.Notify()
// no grace period, handle SIGINT/SIGTERM immediately
return waitToShutdown(&wg, errC, shutdownC, graceShutdownC, 0)
}
@@ -308,7 +301,21 @@ func StartServer(c *cli.Context, version string, shutdownC, graceShutdownC chan
c.Set("url", "https://"+helloListener.Addr().String())
}
tunnelConfig, err := prepareTunnelConfig(c, buildInfo, version, logger, protoLogger)
if host := hostnameFromURI(c.String("url")); host != "" {
listener, err := net.Listen("tcp", "127.0.0.1:")
if err != nil {
logger.WithError(err).Error("Cannot start Websocket Proxy Server")
return errors.Wrap(err, "Cannot start Websocket Proxy Server")
}
wg.Add(1)
go func() {
defer wg.Done()
errC <- websocket.StartProxyServer(logger, listener, host, shutdownC)
}()
c.Set("url", "http://"+listener.Addr().String())
}
tunnelConfig, err := prepareTunnelConfig(c, buildInfo, version, logger, transportLogger)
if err != nil {
return err
}
@@ -324,19 +331,19 @@ func StartServer(c *cli.Context, version string, shutdownC, graceShutdownC chan
func Before(c *cli.Context) error {
if c.String("config") == "" {
logger.Warnf("Cannot determine default configuration path. No file %v in %v", config.DefaultConfigFiles, config.DefaultConfigDirs)
logger.Debugf("Cannot determine default configuration path. No file %v in %v", config.DefaultConfigFiles, config.DefaultConfigDirs)
}
inputSource, err := config.FindInputSourceContext(c)
if err != nil {
logger.WithError(err).Infof("Cannot load configuration from %s", c.String("config"))
logger.WithError(err).Errorf("Cannot load configuration from %s", c.String("config"))
return err
} else if inputSource != nil {
err := altsrc.ApplyInputSourceValues(c, inputSource, c.App.Flags)
if err != nil {
logger.WithError(err).Infof("Cannot apply configuration from %s", c.String("config"))
logger.WithError(err).Errorf("Cannot apply configuration from %s", c.String("config"))
return err
}
logger.Infof("Applied configuration from %s", c.String("config"))
logger.Debugf("Applied configuration from %s", c.String("config"))
}
return nil
}
@@ -368,13 +375,13 @@ func waitToShutdown(wg *sync.WaitGroup,
return err
}
func notifySystemd(waitForSignal chan struct{}) {
<-waitForSignal
func notifySystemd(waitForSignal *signal.Signal) {
<-waitForSignal.Wait()
daemon.SdNotify(false, "READY=1")
}
func writePidFile(waitForSignal chan struct{}, pidFile string) {
<-waitForSignal
func writePidFile(waitForSignal *signal.Signal, pidFile string) {
<-waitForSignal.Wait()
file, err := os.Create(pidFile)
if err != nil {
logger.WithError(err).Errorf("Unable to write pid to %s", pidFile)
@@ -383,6 +390,27 @@ func writePidFile(waitForSignal chan struct{}, pidFile string) {
fmt.Fprintf(file, "%d", os.Getpid())
}
func hostnameFromURI(uri string) string {
u, err := url.Parse(uri)
if err != nil {
return ""
}
switch u.Scheme {
case "ssh":
return addPortIfMissing(u, 22)
case "rdp":
return addPortIfMissing(u, 3389)
}
return ""
}
func addPortIfMissing(uri *url.URL, port int) string {
if uri.Port() != "" {
return uri.Host
}
return fmt.Sprintf("%s:%d", uri.Hostname(), port)
}
func tunnelFlags(shouldHide bool) []cli.Flag {
return []cli.Flag{
&cli.StringFlag{
@@ -417,7 +445,7 @@ func tunnelFlags(shouldHide bool) []cli.Flag {
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "cacert",
Usage: "Certificate Authority authenticating the Cloudflare tunnel connection.",
Usage: "Certificate Authority authenticating connections with Cloudflare's edge network.",
EnvVars: []string{"TUNNEL_CACERT"},
Hidden: true,
}),
@@ -447,6 +475,12 @@ func tunnelFlags(shouldHide bool) []cli.Flag {
EnvVars: []string{"TUNNEL_URL"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "unix-socket",
Usage: "Path to unix socket to use instead of --url",
EnvVars: []string{"TUNNEL_UNIX_SOCKET"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "hostname",
Usage: "Set a hostname on a Cloudflare zone to route traffic through this tunnel.",
@@ -524,15 +558,16 @@ func tunnelFlags(shouldHide bool) []cli.Flag {
altsrc.NewStringFlag(&cli.StringFlag{
Name: "loglevel",
Value: "info",
Usage: "Application logging level {panic, fatal, error, warn, info, debug}",
Usage: "Application logging level {panic, fatal, error, warn, info, debug}. " + debugLevelWarning,
EnvVars: []string{"TUNNEL_LOGLEVEL"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "proto-loglevel",
Name: "transport-loglevel",
Aliases: []string{"proto-loglevel"}, // This flag used to be called proto-loglevel
Value: "warn",
Usage: "Protocol logging level {panic, fatal, error, warn, info, debug}",
EnvVars: []string{"TUNNEL_PROTO_LOGLEVEL"},
Usage: "Transport logging level(previously called protocol logging level) {panic, fatal, error, warn, info, debug}",
EnvVars: []string{"TUNNEL_PROTO_LOGLEVEL", "TUNNEL_TRANSPORT_LOGLEVEL"},
Hidden: shouldHide,
}),
altsrc.NewUintFlag(&cli.UintFlag{
+15
View File
@@ -0,0 +1,15 @@
package tunnel
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TesthostnameFromURI(t *testing.T) {
assert.Equal(t, "ssh://awesome.warptunnels.horse:22", hostnameFromURI("ssh://awesome.warptunnels.horse:22"))
assert.Equal(t, "ssh://awesome.warptunnels.horse:22", hostnameFromURI("ssh://awesome.warptunnels.horse"))
assert.Equal(t, "rdp://localhost:3389", hostnameFromURI("rdp://localhost"))
assert.Equal(t, "", hostnameFromURI("trash"))
assert.Equal(t, "", hostnameFromURI("https://awesomesauce.com"))
}
+155 -42
View File
@@ -1,12 +1,11 @@
package tunnel
import (
"context"
"crypto/tls"
"crypto/x509"
"encoding/hex"
"fmt"
"io/ioutil"
"math/rand"
"net"
"net/http"
"os"
@@ -20,13 +19,13 @@ import (
"github.com/cloudflare/cloudflared/tlsconfig"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/cloudflare/cloudflared/validation"
"golang.org/x/crypto/ssh/terminal"
"github.com/sirupsen/logrus"
"gopkg.in/urfave/cli.v2"
"github.com/google/uuid"
"github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/crypto/ssh/terminal"
"gopkg.in/urfave/cli.v2"
)
var (
@@ -48,11 +47,13 @@ func findDefaultOriginCertPath() string {
return ""
}
func generateRandomClientID() string {
r := rand.New(rand.NewSource(time.Now().UnixNano()))
id := make([]byte, 32)
r.Read(id)
return hex.EncodeToString(id)
func generateRandomClientID(logger *logrus.Logger) (string, error) {
u, err := uuid.NewRandom()
if err != nil {
logger.WithError(err).Error("couldn't create UUID for client ID")
return "", err
}
return u.String(), nil
}
func handleDeprecatedOptions(c *cli.Context) error {
@@ -64,26 +65,21 @@ func handleDeprecatedOptions(c *cli.Context) error {
return nil
}
// validate url. It can be either from --url or argument
func validateUrl(c *cli.Context) (string, error) {
var url = c.String("url")
if c.NArg() > 0 {
if c.IsSet("url") {
return "", errors.New("Specified origin urls using both --url and argument. Decide which one you want, I can only support one.")
}
url = c.Args().Get(0)
}
validUrl, err := validation.ValidateUrl(url)
return validUrl, err
}
func logClientOptions(c *cli.Context) {
flags := make(map[string]interface{})
for _, flag := range c.LocalFlagNames() {
flags[flag] = c.Generic(flag)
}
sliceFlags := []string{"header", "tag", "proxy-dns-upstream", "upstream", "edge"}
for _, sliceFlag := range sliceFlags {
if len(c.StringSlice(sliceFlag)) > 0 {
flags[sliceFlag] = strings.Join(c.StringSlice(sliceFlag), ", ")
}
}
if len(flags) > 0 {
logger.Infof("Flags %v", flags)
logger.WithFields(flags).Info("Flags")
}
envs := make(map[string]string)
@@ -149,15 +145,24 @@ If you don't have a certificate signed by Cloudflare, run the command:
return originCert, nil
}
func prepareTunnelConfig(c *cli.Context, buildInfo *origin.BuildInfo, version string, logger, protoLogger *logrus.Logger) (*origin.TunnelConfig, error) {
func prepareTunnelConfig(
c *cli.Context,
buildInfo *origin.BuildInfo,
version string, logger,
transportLogger *logrus.Logger,
) (*origin.TunnelConfig, error) {
hostname, err := validation.ValidateHostname(c.String("hostname"))
if err != nil {
logger.WithError(err).Error("Invalid hostname")
return nil, errors.Wrap(err, "Invalid hostname")
}
isFreeTunnel := hostname == ""
clientID := c.String("id")
if !c.IsSet("id") {
clientID = generateRandomClientID()
clientID, err = generateRandomClientID(logger)
if err != nil {
return nil, err
}
}
tags, err := NewTagSliceFromCLI(c.StringSlice("tag"))
@@ -168,16 +173,18 @@ func prepareTunnelConfig(c *cli.Context, buildInfo *origin.BuildInfo, version st
tags = append(tags, tunnelpogs.Tag{Name: "ID", Value: clientID})
originURL, err := validateUrl(c)
originURL, err := config.ValidateUrl(c)
if err != nil {
logger.WithError(err).Error("Error validating origin URL")
return nil, errors.Wrap(err, "Error validating origin URL")
}
logger.Infof("Proxying tunnel requests to %s", originURL)
originCert, err := getOriginCert(c)
if err != nil {
return nil, errors.Wrap(err, "Error getting origin cert")
var originCert []byte
if !isFreeTunnel {
originCert, err = getOriginCert(c)
if err != nil {
return nil, errors.Wrap(err, "Error getting origin cert")
}
}
originCertPool, err := loadCertPool(c, logger)
@@ -188,12 +195,7 @@ func prepareTunnelConfig(c *cli.Context, buildInfo *origin.BuildInfo, version st
tunnelMetrics := origin.NewTunnelMetrics()
httpTransport := &http.Transport{
Proxy: http.ProxyFromEnvironment,
DialContext: (&net.Dialer{
Timeout: c.Duration("proxy-connect-timeout"),
KeepAlive: c.Duration("proxy-tcp-keepalive"),
DualStack: !c.Bool("proxy-no-happy-eyeballs"),
}).DialContext,
Proxy: http.ProxyFromEnvironment,
MaxIdleConns: c.Int("proxy-keepalive-connections"),
IdleConnTimeout: c.Duration("proxy-keepalive-timeout"),
TLSHandshakeTimeout: c.Duration("proxy-tls-timeout"),
@@ -201,22 +203,51 @@ func prepareTunnelConfig(c *cli.Context, buildInfo *origin.BuildInfo, version st
TLSClientConfig: &tls.Config{RootCAs: originCertPool, InsecureSkipVerify: c.IsSet("no-tls-verify")},
}
dialContext := (&net.Dialer{
Timeout: c.Duration("proxy-connect-timeout"),
KeepAlive: c.Duration("proxy-tcp-keepalive"),
DualStack: !c.Bool("proxy-no-happy-eyeballs"),
}).DialContext
if c.IsSet("unix-socket") {
unixSocket, err := config.ValidateUnixSocket(c)
if err != nil {
logger.WithError(err).Error("Error validating --unix-socket")
return nil, errors.Wrap(err, "Error validating --unix-socket")
}
logger.Infof("Proxying tunnel requests to unix:%s", unixSocket)
httpTransport.DialContext = func(ctx context.Context, _, _ string) (net.Conn, error) {
// if --unix-socket specified, enforce network type "unix"
return dialContext(ctx, "unix", unixSocket)
}
} else {
logger.Infof("Proxying tunnel requests to %s", originURL)
httpTransport.DialContext = dialContext
}
if !c.IsSet("hello-world") && c.IsSet("origin-server-name") {
httpTransport.TLSClientConfig.ServerName = c.String("origin-server-name")
}
err = validation.ValidateHTTPService(originURL, httpTransport)
err = validation.ValidateHTTPService(originURL, hostname, httpTransport)
if err != nil {
logger.WithError(err).Error("unable to connect to the origin")
return nil, errors.Wrap(err, "unable to connect to the origin")
}
toEdgeTLSConfig, err := createTunnelConfig(c)
if err != nil {
logger.WithError(err).Error("unable to create TLS config to connect with edge")
return nil, errors.Wrap(err, "unable to create TLS config to connect with edge")
}
return &origin.TunnelConfig{
EdgeAddrs: c.StringSlice("edge"),
OriginUrl: originURL,
Hostname: hostname,
OriginCert: originCert,
TlsConfig: tlsconfig.CreateTunnelConfig(c, c.StringSlice("edge")),
TlsConfig: toEdgeTLSConfig,
ClientTlsConfig: httpTransport.TLSClientConfig,
Retries: c.Uint("retries"),
HeartbeatInterval: c.Duration("heartbeat-interval"),
@@ -230,13 +261,15 @@ func prepareTunnelConfig(c *cli.Context, buildInfo *origin.BuildInfo, version st
HTTPTransport: httpTransport,
Metrics: tunnelMetrics,
MetricsUpdateFreq: c.Duration("metrics-update-freq"),
ProtocolLogger: protoLogger,
TransportLogger: transportLogger,
Logger: logger,
IsAutoupdated: c.Bool("is-autoupdated"),
GracePeriod: c.Duration("grace-period"),
RunFromTerminal: isRunningFromTerminal(),
NoChunkedEncoding: c.Bool("no-chunked-encoding"),
CompressionQuality: c.Uint64("compression-quality"),
IncidentLookup: origin.NewIncidentLookup(),
IsFreeTunnel: isFreeTunnel,
}, nil
}
@@ -253,7 +286,7 @@ func loadCertPool(c *cli.Context, logger *logrus.Logger) (*x509.CertPool, error)
}
}
originCertPool, err := tlsconfig.LoadOriginCertPool(originCustomCAPool)
originCertPool, err := loadOriginCertPool(originCustomCAPool)
if err != nil {
return nil, errors.Wrap(err, "error loading the certificate pool")
}
@@ -266,6 +299,86 @@ func loadCertPool(c *cli.Context, logger *logrus.Logger) (*x509.CertPool, error)
return originCertPool, nil
}
func loadOriginCertPool(originCAPoolPEM []byte) (*x509.CertPool, error) {
// Get the global pool
certPool, err := loadGlobalCertPool()
if err != nil {
return nil, err
}
// Then, add any custom origin CA pool the user may have passed
if originCAPoolPEM != nil {
if !certPool.AppendCertsFromPEM(originCAPoolPEM) {
logger.Warn("could not append the provided origin CA to the cloudflared certificate pool")
}
}
return certPool, nil
}
func loadGlobalCertPool() (*x509.CertPool, error) {
// First, obtain the system certificate pool
certPool, err := x509.SystemCertPool()
if err != nil {
if runtime.GOOS != "windows" {
logger.WithError(err).Warn("error obtaining the system certificates")
}
certPool = x509.NewCertPool()
}
// Next, append the Cloudflare CAs into the system pool
cfRootCA, err := tlsconfig.GetCloudflareRootCA()
if err != nil {
return nil, errors.Wrap(err, "could not append Cloudflare Root CAs to cloudflared certificate pool")
}
for _, cert := range cfRootCA {
certPool.AddCert(cert)
}
// Finally, add the Hello certificate into the pool (since it's self-signed)
helloCert, err := tlsconfig.GetHelloCertificateX509()
if err != nil {
return nil, errors.Wrap(err, "could not append Hello server certificate to cloudflared certificate pool")
}
certPool.AddCert(helloCert)
return certPool, nil
}
func createTunnelConfig(c *cli.Context) (*tls.Config, error) {
var rootCAs []string
if c.String("cacert") != "" {
rootCAs = append(rootCAs, c.String("cacert"))
}
edgeAddrs := c.StringSlice("edge")
userConfig := &tlsconfig.TLSParameters{RootCAs: rootCAs}
tlsConfig, err := tlsconfig.GetConfig(userConfig)
if err != nil {
return nil, err
}
if tlsConfig.RootCAs == nil {
rootCAPool := x509.NewCertPool()
cfRootCA, err := tlsconfig.GetCloudflareRootCA()
if err != nil {
return nil, errors.Wrap(err, "could not append Cloudflare Root CAs to cloudflared certificate pool")
}
for _, cert := range cfRootCA {
rootCAPool.AddCert(cert)
}
tlsConfig.RootCAs = rootCAPool
tlsConfig.ServerName = "cftunnel.com"
} else if len(edgeAddrs) > 0 {
// Set for development environments and for testing specific origintunneld instances
tlsConfig.ServerName, _, _ = net.SplitHostPort(edgeAddrs[0])
}
if tlsConfig.ServerName == "" && !tlsConfig.InsecureSkipVerify {
return nil, fmt.Errorf("either ServerName or InsecureSkipVerify must be specified in the tls.Config")
}
return tlsConfig, nil
}
func isRunningFromTerminal() bool {
return terminal.IsTerminal(int(os.Stdout.Fd()))
}
@@ -0,0 +1,214 @@
// +build ignore
// TODO: Remove the above build tag and include this test when we start compiling with Golang 1.10.0+
package tunnel
import (
"crypto/x509"
"crypto/x509/pkix"
"encoding/asn1"
"os"
"testing"
"github.com/stretchr/testify/assert"
)
// Generated using `openssl req -newkey rsa:512 -nodes -x509 -days 3650`
var samplePEM = []byte(`
-----BEGIN CERTIFICATE-----
MIIB4DCCAYoCCQCb/H0EUrdXEjANBgkqhkiG9w0BAQsFADB3MQswCQYDVQQGEwJV
UzEOMAwGA1UECAwFVGV4YXMxDzANBgNVBAcMBkF1c3RpbjEZMBcGA1UECgwQQ2xv
dWRmbGFyZSwgSW5jLjEZMBcGA1UECwwQUHJvZHVjdCBTdHJhdGVneTERMA8GA1UE
AwwIVGVzdCBPbmUwHhcNMTgwNDI2MTYxMDUxWhcNMjgwNDIzMTYxMDUxWjB3MQsw
CQYDVQQGEwJVUzEOMAwGA1UECAwFVGV4YXMxDzANBgNVBAcMBkF1c3RpbjEZMBcG
A1UECgwQQ2xvdWRmbGFyZSwgSW5jLjEZMBcGA1UECwwQUHJvZHVjdCBTdHJhdGVn
eTERMA8GA1UEAwwIVGVzdCBPbmUwXDANBgkqhkiG9w0BAQEFAANLADBIAkEAwVQD
K0SJ25UFLznm2pU3zhzMEvpDEofHVNnCjk4mlDrtVop7PkKZ8pDEmuQANltUrxC8
yHBE2wXMv+GlH+bDtwIDAQABMA0GCSqGSIb3DQEBCwUAA0EAjVYQzozIFPkt/HRY
uUoZ8zEHIDICb0syFf5VAjm9AgTwIPzUmD+c5vl6LWDnxq7L45nLCzhhQ6YmiwDz
X7Wcyg==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB4DCCAYoCCQDZfCdAJ+mwzDANBgkqhkiG9w0BAQsFADB3MQswCQYDVQQGEwJV
UzEOMAwGA1UECAwFVGV4YXMxDzANBgNVBAcMBkF1c3RpbjEZMBcGA1UECgwQQ2xv
dWRmbGFyZSwgSW5jLjEZMBcGA1UECwwQUHJvZHVjdCBTdHJhdGVneTERMA8GA1UE
AwwIVGVzdCBUd28wHhcNMTgwNDI2MTYxMTIwWhcNMjgwNDIzMTYxMTIwWjB3MQsw
CQYDVQQGEwJVUzEOMAwGA1UECAwFVGV4YXMxDzANBgNVBAcMBkF1c3RpbjEZMBcG
A1UECgwQQ2xvdWRmbGFyZSwgSW5jLjEZMBcGA1UECwwQUHJvZHVjdCBTdHJhdGVn
eTERMA8GA1UEAwwIVGVzdCBUd28wXDANBgkqhkiG9w0BAQEFAANLADBIAkEAoHKp
ROVK3zCSsH7ocYeyRAML4V7SFAbZcb4WIwDnE08oMBVRkQVcW5tqEkvG3RiClfzV
wZIJ3CfqKIeSNSDU9wIDAQABMA0GCSqGSIb3DQEBCwUAA0EAJw2gUbnPiq4C2p5b
iWzlA9Q7aKo+VQ4H7IZS7tTccr59nVjvH/TG3eWujpnocr4TOqW9M3CK1DF9mUGP
3pQ3Jg==
-----END CERTIFICATE-----
`)
var systemCertPoolSubjects []*pkix.Name
type certificateFixture struct {
ou string
cn string
}
func TestMain(m *testing.M) {
systemCertPool, err := x509.SystemCertPool()
if isUnrecoverableError(err) {
os.Exit(1)
}
if systemCertPool == nil {
// On Windows, let's just assume the system cert pool was empty
systemCertPool = x509.NewCertPool()
}
systemCertPoolSubjects, err = getCertPoolSubjects(systemCertPool)
if err != nil {
os.Exit(1)
}
os.Exit(m.Run())
}
func TestLoadOriginCertPoolJustSystemPool(t *testing.T) {
certPoolSubjects := loadCertPoolSubjects(t, nil)
extraSubjects := subjectSubtract(systemCertPoolSubjects, certPoolSubjects)
// Remove extra subjects from the cert pool
var filteredSystemCertPoolSubjects []*pkix.Name
t.Log(extraSubjects)
OUTER:
for _, subject := range certPoolSubjects {
for _, extraSubject := range extraSubjects {
if subject == extraSubject {
t.Log(extraSubject)
continue OUTER
}
}
filteredSystemCertPoolSubjects = append(filteredSystemCertPoolSubjects, subject)
}
assert.Equal(t, len(filteredSystemCertPoolSubjects), len(systemCertPoolSubjects))
difference := subjectSubtract(systemCertPoolSubjects, filteredSystemCertPoolSubjects)
assert.Equal(t, 0, len(difference))
}
func TestLoadOriginCertPoolCFCertificates(t *testing.T) {
certPoolSubjects := loadCertPoolSubjects(t, nil)
extraSubjects := subjectSubtract(systemCertPoolSubjects, certPoolSubjects)
expected := []*certificateFixture{
{ou: "CloudFlare Origin SSL ECC Certificate Authority"},
{ou: "CloudFlare Origin SSL Certificate Authority"},
{cn: "origin-pull.cloudflare.net"},
{cn: "Argo Tunnel Sample Hello Server Certificate"},
}
assertFixturesMatchSubjects(t, expected, extraSubjects)
}
func TestLoadOriginCertPoolWithExtraPEMs(t *testing.T) {
certPoolWithoutPEMSubjects := loadCertPoolSubjects(t, nil)
certPoolWithPEMSubjects := loadCertPoolSubjects(t, samplePEM)
difference := subjectSubtract(certPoolWithoutPEMSubjects, certPoolWithPEMSubjects)
assert.Equal(t, 2, len(difference))
expected := []*certificateFixture{
{cn: "Test One"},
{cn: "Test Two"},
}
assertFixturesMatchSubjects(t, expected, difference)
}
func loadCertPoolSubjects(t *testing.T, originCAPoolPEM []byte) []*pkix.Name {
certPool, err := loadOriginCertPool(originCAPoolPEM)
if isUnrecoverableError(err) {
t.Fatal(err)
}
assert.NotEmpty(t, certPool.Subjects())
certPoolSubjects, err := getCertPoolSubjects(certPool)
if err != nil {
t.Fatal(err)
}
return certPoolSubjects
}
func assertFixturesMatchSubjects(t *testing.T, fixtures []*certificateFixture, subjects []*pkix.Name) {
assert.Equal(t, len(fixtures), len(subjects))
for _, fixture := range fixtures {
found := false
for _, subject := range subjects {
found = found || fixtureMatchesSubjectPredicate(fixture, subject)
}
if !found {
t.Fail()
}
}
}
func fixtureMatchesSubjectPredicate(fixture *certificateFixture, subject *pkix.Name) bool {
cnMatch := true
if fixture.cn != "" {
cnMatch = fixture.cn == subject.CommonName
}
ouMatch := true
if fixture.ou != "" {
ouMatch = len(subject.OrganizationalUnit) > 0 && fixture.ou == subject.OrganizationalUnit[0]
}
return cnMatch && ouMatch
}
func subjectSubtract(left []*pkix.Name, right []*pkix.Name) []*pkix.Name {
var difference []*pkix.Name
var found bool
for _, r := range right {
found = false
for _, l := range left {
if (*l).String() == (*r).String() {
found = true
}
}
if !found {
difference = append(difference, r)
}
}
return difference
}
func getCertPoolSubjects(certPool *x509.CertPool) ([]*pkix.Name, error) {
var subjects []*pkix.Name
for _, subject := range certPool.Subjects() {
var sequence pkix.RDNSequence
_, err := asn1.Unmarshal(subject, &sequence)
if err != nil {
return nil, err
}
name := pkix.Name{}
name.FillFromRDNSequence(&sequence)
subjects = append(subjects, &name)
}
return subjects, nil
}
func isUnrecoverableError(err error) bool {
return err != nil && err.Error() != "crypto/x509: system root pool is not available on Windows"
}
-20
View File
@@ -1,20 +0,0 @@
package tunnel
import (
"fmt"
"gopkg.in/urfave/cli.v2"
"github.com/cloudflare/cloudflared/hello"
)
func helloWorld(c *cli.Context) error {
address := fmt.Sprintf(":%d", c.Int("port"))
listener, err := hello.CreateTLSListener(address)
if err != nil {
return err
}
defer listener.Close()
err = hello.StartHelloWorldServer(logger, listener, nil)
return err
}
+13 -7
View File
@@ -13,6 +13,9 @@ import (
"github.com/pkg/errors"
)
const debugLevelWarning = "At debug level, request URL, method, protocol, content legnth and header will be logged. " +
"Response status, content length and header will also be logged in debug level."
var logger = log.CreateLogger()
func configMainLogger(c *cli.Context) error {
@@ -22,18 +25,21 @@ func configMainLogger(c *cli.Context) error {
return errors.Wrap(err, "Unknown logging level specified")
}
logger.SetLevel(logLevel)
if logLevel == logrus.DebugLevel {
logger.Warn(debugLevelWarning)
}
return nil
}
func configProtoLogger(c *cli.Context) (*logrus.Logger, error) {
protoLogLevel, err := logrus.ParseLevel(c.String("proto-loglevel"))
func configTransportLogger(c *cli.Context) (*logrus.Logger, error) {
transportLogLevel, err := logrus.ParseLevel(c.String("transport-loglevel"))
if err != nil {
logger.WithError(err).Fatal("Unknown protocol logging level specified")
return nil, errors.Wrap(err, "Unknown protocol logging level specified")
logger.WithError(err).Fatal("Unknown transport logging level specified")
return nil, errors.Wrap(err, "Unknown transport logging level specified")
}
protoLogger := logrus.New()
protoLogger.Level = protoLogLevel
return protoLogger, nil
transportLogger := logrus.New()
transportLogger.Level = transportLogLevel
return transportLogger, nil
}
func initLogFile(c *cli.Context, loggers ...*logrus.Logger) error {
+1 -1
View File
@@ -33,7 +33,7 @@ func login(c *cli.Context) error {
return err
}
_, err = transfer.Run(c, loginURL, "cert", "callback", callbackStoreURL, path, false)
_, err = transfer.Run(loginURL, "cert", "callback", callbackStoreURL, path, false)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to write the certificate due to the following error:\n%v\n\nYour browser will download the certificate instead. You will have to manually\ncopy it to the following path:\n\n%s\n", err, path)
return err
+31 -19
View File
@@ -30,45 +30,56 @@ EKx0BZogHSor9Wy5VztdFaAaVbsJiCbO
logger = log.CreateLogger()
)
type ReleaseInfo struct {
type UpdateOutcome struct {
Updated bool
Version string
Error error
}
func checkForUpdates() ReleaseInfo {
func (uo *UpdateOutcome) noUpdate() bool {
return uo.Error != nil && uo.Updated == false
}
func checkForUpdateAndApply() UpdateOutcome {
var opts equinox.Options
if err := opts.SetPublicKeyPEM(publicKey); err != nil {
return ReleaseInfo{Error: err}
return UpdateOutcome{Error: err}
}
resp, err := equinox.Check(appID, opts)
switch {
case err == equinox.NotAvailableErr:
return ReleaseInfo{}
return UpdateOutcome{}
case err != nil:
return ReleaseInfo{Error: err}
return UpdateOutcome{Error: err}
}
err = resp.Apply()
if err != nil {
return ReleaseInfo{Error: err}
return UpdateOutcome{Error: err}
}
return ReleaseInfo{Updated: true, Version: resp.ReleaseVersion}
return UpdateOutcome{Updated: true, Version: resp.ReleaseVersion}
}
func Update(_ *cli.Context) error {
if updateApplied() {
os.Exit(64)
updateOutcome := loggedUpdate()
if updateOutcome.Error != nil {
os.Exit(10)
}
return nil
if updateOutcome.noUpdate() {
logger.Infof("cloudflared is up to date (%s)", updateOutcome.Version)
}
return updateOutcome.Error
}
func Autoupdate(freq time.Duration, listeners *gracenet.Net, shutdownC chan struct{}) error {
tickC := time.Tick(freq)
for {
if updateApplied() {
updateOutcome := loggedUpdate()
if updateOutcome.Updated {
os.Args = append(os.Args, "--is-autoupdated=true")
pid, err := listeners.StartProcess()
if err != nil {
@@ -88,16 +99,17 @@ func Autoupdate(freq time.Duration, listeners *gracenet.Net, shutdownC chan stru
}
}
func updateApplied() bool {
releaseInfo := checkForUpdates()
if releaseInfo.Updated {
logger.Infof("Updated to version %s", releaseInfo.Version)
return true
// Checks for an update and applies it if one is available
func loggedUpdate() UpdateOutcome {
updateOutcome := checkForUpdateAndApply()
if updateOutcome.Updated {
logger.Infof("cloudflared has been updated to version %s", updateOutcome.Version)
}
if releaseInfo.Error != nil {
logger.WithError(releaseInfo.Error).Error("Update check failed")
if updateOutcome.Error != nil {
logger.WithError(updateOutcome.Error).Error("update check failed")
}
return false
return updateOutcome
}
func IsAutoupdateEnabled(c *cli.Context) bool {
+59 -25
View File
@@ -8,6 +8,7 @@ package main
import (
"fmt"
"os"
"syscall"
"time"
"unsafe"
@@ -30,6 +31,10 @@ const (
// not defined in golang.org/x/sys/windows package
// https://msdn.microsoft.com/en-us/library/windows/desktop/ms681988(v=vs.85).aspx
serviceConfigFailureActionsFlag = 4
// ERROR_FAILED_SERVICE_CONTROLLER_CONNECT
// https://docs.microsoft.com/en-us/windows/desktop/debug/system-error-codes--1000-1299-
serviceControllerConnectionFailure = 1063
)
func runApp(app *cli.App, shutdownC, graceShutdownC chan struct{}) {
@@ -50,16 +55,48 @@ func runApp(app *cli.App, shutdownC, graceShutdownC chan struct{}) {
},
})
// `IsAnInteractiveSession()` isn't exactly equivalent to "should the
// process run as a normal EXE?" There are legitimate non-service cases,
// like running cloudflared in a GCP startup script, for which
// `IsAnInteractiveSession()` returns false. For more context, see:
// https://github.com/judwhite/go-svc/issues/6
// It seems that the "correct way" to check "is this a normal EXE?" is:
// 1. attempt to connect to the Service Control Manager
// 2. get ERROR_FAILED_SERVICE_CONTROLLER_CONNECT
// This involves actually trying to start the service.
isIntSess, err := svc.IsAnInteractiveSession()
if err != nil {
logger.Fatalf("failed to determine if we are running in an interactive session: %v", err)
}
if isIntSess {
app.Run(os.Args)
return
}
// Run executes service name by calling windowsService which is a Handler
// interface that implements Execute method.
// It will set service status to stop after Execute returns
err = svc.Run(windowsServiceName, &windowsService{app: app, shutdownC: shutdownC, graceShutdownC: graceShutdownC})
if err != nil {
if errno, ok := err.(syscall.Errno); ok && int(errno) == serviceControllerConnectionFailure {
// Hack: assume this is a false negative from the IsAnInteractiveSession() check above.
// Run the app in "interactive" mode anyway.
app.Run(os.Args)
return
}
logger.Fatalf("%s service failed: %v", windowsServiceName, err)
}
}
type windowsService struct {
app *cli.App
shutdownC chan struct{}
graceShutdownC chan struct{}
}
// called by the package code at the start of the service
func (s *windowsService) Execute(serviceArgs []string, r <-chan svc.ChangeRequest, statusChan chan<- svc.Status) (ssec bool, errno uint32) {
elog, err := eventlog.Open(windowsServiceName)
if err != nil {
logger.WithError(err).Errorf("Cannot open event log for %s", windowsServiceName)
@@ -68,26 +105,23 @@ func runApp(app *cli.App, shutdownC, graceShutdownC chan struct{}) {
defer elog.Close()
elog.Info(1, fmt.Sprintf("%s service starting", windowsServiceName))
// Run executes service name by calling windowsService which is a Handler
// interface that implements Execute method.
// It will set service status to stop after Execute returns
err = svc.Run(windowsServiceName, &windowsService{app: app, elog: elog, shutdownC: shutdownC, graceShutdownC: graceShutdownC})
if err != nil {
elog.Error(1, fmt.Sprintf("%s service failed: %v", windowsServiceName, err))
return
defer func() {
elog.Info(1, fmt.Sprintf("%s service stopped", windowsServiceName))
}()
// the arguments passed here are only meaningful if they were manually
// specified by the user, e.g. using the Services console or `sc start`.
// https://docs.microsoft.com/en-us/windows/desktop/services/service-entry-point
// https://stackoverflow.com/a/6235139
var args []string
if len(serviceArgs) > 1 {
args = serviceArgs
} else {
// fall back to the arguments from ImagePath (or, as sc calls it, binPath)
args = os.Args
}
elog.Info(1, fmt.Sprintf("%s service stopped", windowsServiceName))
}
elog.Info(1, fmt.Sprintf("%s service arguments: %v", windowsServiceName, args))
type windowsService struct {
app *cli.App
elog *eventlog.Log
shutdownC chan struct{}
graceShutdownC chan struct{}
}
// called by the package code at the start of the service
func (s *windowsService) Execute(args []string, r <-chan svc.ChangeRequest, statusChan chan<- svc.Status) (ssec bool, errno uint32) {
const cmdsAccepted = svc.AcceptStop | svc.AcceptShutdown
statusChan <- svc.Status{State: svc.StartPending}
errC := make(chan error)
@@ -101,26 +135,26 @@ func (s *windowsService) Execute(args []string, r <-chan svc.ChangeRequest, stat
case c := <-r:
switch c.Cmd {
case svc.Interrogate:
s.elog.Info(1, fmt.Sprintf("control request 1 #%d", c))
elog.Info(1, fmt.Sprintf("control request 1 #%d", c))
statusChan <- c.CurrentStatus
case svc.Stop:
s.elog.Info(1, "received stop control request")
elog.Info(1, "received stop control request")
close(s.graceShutdownC)
statusChan <- svc.Status{State: svc.StopPending}
case svc.Shutdown:
s.elog.Info(1, "received shutdown control request")
elog.Info(1, "received shutdown control request")
close(s.shutdownC)
statusChan <- svc.Status{State: svc.StopPending}
default:
s.elog.Error(1, fmt.Sprintf("unexpected control request #%d", c))
elog.Error(1, fmt.Sprintf("unexpected control request #%d", c))
}
case err := <-errC:
ssec = true
if err != nil {
s.elog.Error(1, fmt.Sprintf("cloudflared terminated with error %v", err))
elog.Error(1, fmt.Sprintf("cloudflared terminated with error %v", err))
errno = 1
} else {
s.elog.Info(1, "cloudflared terminated without error")
elog.Info(1, "cloudflared terminated without error")
errno = 0
}
return
+64 -52
View File
@@ -15,11 +15,12 @@ import (
)
const (
defaultFrameSize uint32 = 1 << 14 // Minimum frame size in http2 spec
defaultWindowSize uint32 = 65535
maxWindowSize uint32 = (1 << 31) - 1 // 2^31-1 = 2147483647, max window size specified in http2 spec
defaultTimeout time.Duration = 5 * time.Second
defaultRetries uint64 = 5
defaultFrameSize uint32 = 1 << 14 // Minimum frame size in http2 spec
defaultWindowSize uint32 = (1 << 16) - 1 // Minimum window size in http2 spec
maxWindowSize uint32 = (1 << 31) - 1 // 2^31-1 = 2147483647, max window size in http2 spec
defaultTimeout time.Duration = 5 * time.Second
defaultRetries uint64 = 5
defaultWriteBufferMaxLen int = 1024 * 1024 * 512 // 500mb
SettingMuxerMagic http2.SettingID = 0x42db
MuxerMagicOrigin uint32 = 0xa2e43c8b
@@ -49,6 +50,12 @@ type MuxerConfig struct {
// Logger to use
Logger *log.Entry
CompressionQuality CompressionSetting
// Initial size for HTTP2 flow control windows
DefaultWindowSize uint32
// Largest allowable size for HTTP2 flow control windows
MaxWindowSize uint32
// Largest allowable capacity for the buffer of data to be sent
StreamWriteBufferMaxLen int
}
type Muxer struct {
@@ -64,7 +71,7 @@ type Muxer struct {
// muxWriter is the write process.
muxWriter *MuxWriter
// muxMetricsUpdater is the process to update metrics
muxMetricsUpdater *muxMetricsUpdater
muxMetricsUpdater muxMetricsUpdater
// newStreamChan is used to create new streams on the writer thread.
// The writer will assign the next available stream ID.
newStreamChan chan MuxedStreamRequest
@@ -98,6 +105,15 @@ func Handshake(
if config.Timeout == 0 {
config.Timeout = defaultTimeout
}
if config.DefaultWindowSize == 0 {
config.DefaultWindowSize = defaultWindowSize
}
if config.MaxWindowSize == 0 {
config.MaxWindowSize = maxWindowSize
}
if config.StreamWriteBufferMaxLen == 0 {
config.StreamWriteBufferMaxLen = defaultWriteBufferMaxLen
}
// Initialise connection state fields
m := &Muxer{
f: http2.NewFramer(w, r), // A framer that writes to w and reads from r
@@ -147,11 +163,6 @@ func Handshake(
// set up reader/writer pair ready for serve
streamErrors := NewStreamErrorMap()
goAwayChan := make(chan http2.ErrCode, 1)
updateRTTChan := make(chan *roundTripMeasurement, 1)
updateReceiveWindowChan := make(chan uint32, 1)
updateSendWindowChan := make(chan uint32, 1)
updateInBoundBytesChan := make(chan uint64)
updateOutBoundBytesChan := make(chan uint64)
inBoundCounter := NewAtomicCounter(0)
outBoundCounter := NewAtomicCounter(0)
pingTimestamp := NewPingTimestamp()
@@ -168,6 +179,14 @@ func Handshake(
config.Logger.Warn("Minimum number of unacked heartbeats to send before closing the connection has been adjusted to ", maxRetries)
}
compBytesBefore, compBytesAfter := NewAtomicCounter(0), NewAtomicCounter(0)
m.muxMetricsUpdater = newMuxMetricsUpdater(
m.abortChan,
compBytesBefore,
compBytesAfter,
)
m.explicitShutdown = NewBooleanFuse()
m.muxReader = &MuxReader{
f: m.f,
@@ -179,47 +198,30 @@ func Handshake(
abortChan: m.abortChan,
pingTimestamp: pingTimestamp,
connActive: connActive,
initialStreamWindow: defaultWindowSize,
streamWindowMax: maxWindowSize,
r: m.r,
updateRTTChan: updateRTTChan,
updateReceiveWindowChan: updateReceiveWindowChan,
updateSendWindowChan: updateSendWindowChan,
bytesRead: inBoundCounter,
updateInBoundBytesChan: updateInBoundBytesChan,
initialStreamWindow: m.config.DefaultWindowSize,
streamWindowMax: m.config.MaxWindowSize,
streamWriteBufferMaxLen: m.config.StreamWriteBufferMaxLen,
r: m.r,
metricsUpdater: m.muxMetricsUpdater,
bytesRead: inBoundCounter,
}
m.muxWriter = &MuxWriter{
f: m.f,
streams: m.streams,
streamErrors: streamErrors,
readyStreamChan: m.readyList.ReadyChannel(),
newStreamChan: m.newStreamChan,
goAwayChan: goAwayChan,
abortChan: m.abortChan,
pingTimestamp: pingTimestamp,
idleTimer: NewIdleTimer(idleDuration, maxRetries),
connActiveChan: connActive.WaitChannel(),
maxFrameSize: defaultFrameSize,
updateReceiveWindowChan: updateReceiveWindowChan,
updateSendWindowChan: updateSendWindowChan,
bytesWrote: outBoundCounter,
updateOutBoundBytesChan: updateOutBoundBytesChan,
f: m.f,
streams: m.streams,
streamErrors: streamErrors,
readyStreamChan: m.readyList.ReadyChannel(),
newStreamChan: m.newStreamChan,
goAwayChan: goAwayChan,
abortChan: m.abortChan,
pingTimestamp: pingTimestamp,
idleTimer: NewIdleTimer(idleDuration, maxRetries),
connActiveChan: connActive.WaitChannel(),
maxFrameSize: defaultFrameSize,
metricsUpdater: m.muxMetricsUpdater,
bytesWrote: outBoundCounter,
}
m.muxWriter.headerEncoder = hpack.NewEncoder(&m.muxWriter.headerBuffer)
compBytesBefore, compBytesAfter := NewAtomicCounter(0), NewAtomicCounter(0)
m.muxMetricsUpdater = newMuxMetricsUpdater(
updateRTTChan,
updateReceiveWindowChan,
updateSendWindowChan,
updateInBoundBytesChan,
updateOutBoundBytesChan,
m.abortChan,
compBytesBefore,
compBytesAfter,
)
if m.compressionQuality.dictSize > 0 && m.compressionQuality.nDicts > 0 {
nd, sz := m.compressionQuality.nDicts, m.compressionQuality.dictSize
writeDicts, dictChan := newH2WriteDictionaries(
@@ -305,6 +307,12 @@ func joinErrorsWithTimeout(errChan <-chan error, receiveCount int, timeout time.
return nil
}
// Serve runs the event loops that comprise h2mux:
// - MuxReader.run()
// - MuxWriter.run()
// - muxMetricsUpdater.run()
// In the normal case, Shutdown() is called concurrently with Serve() to stop
// these loops.
func (m *Muxer) Serve(ctx context.Context) error {
errGroup, _ := errgroup.WithContext(ctx)
errGroup.Go(func() error {
@@ -335,6 +343,7 @@ func (m *Muxer) Serve(ctx context.Context) error {
return nil
}
// Shutdown is called to initiate the "happy path" of muxer termination.
func (m *Muxer) Shutdown() {
m.explicitShutdown.Fuse(true)
m.muxReader.Shutdown()
@@ -375,10 +384,12 @@ func (m *Muxer) OpenStream(headers []Header, body io.Reader) (*MuxedStream, erro
responseHeadersReceived: make(chan struct{}),
readBuffer: NewSharedBuffer(),
writeBuffer: &bytes.Buffer{},
receiveWindow: defaultWindowSize,
receiveWindowCurrentMax: defaultWindowSize, // Initial window size limit. exponentially increase it when receiveWindow is exhausted
receiveWindowMax: maxWindowSize,
sendWindow: defaultWindowSize,
writeBufferMaxLen: m.config.StreamWriteBufferMaxLen,
writeBufferHasSpace: make(chan struct{}, 1),
receiveWindow: m.config.DefaultWindowSize,
receiveWindowCurrentMax: m.config.DefaultWindowSize,
receiveWindowMax: m.config.MaxWindowSize,
sendWindow: m.config.DefaultWindowSize,
readyList: m.readyList,
writeHeaders: headers,
dictionaries: m.muxReader.dictionaries,
@@ -399,12 +410,13 @@ func (m *Muxer) OpenStream(headers []Header, body io.Reader) (*MuxedStream, erro
}
func (m *Muxer) Metrics() *MuxerMetrics {
return m.muxMetricsUpdater.Metrics()
return m.muxMetricsUpdater.metrics()
}
func (m *Muxer) abort() {
m.abortOnce.Do(func() {
close(m.abortChan)
m.readyList.Close()
m.streams.Abort()
})
}
+56 -46
View File
@@ -39,20 +39,26 @@ func NewDefaultMuxerPair() *DefaultMuxerPair {
origin, edge := net.Pipe()
return &DefaultMuxerPair{
OriginMuxConfig: MuxerConfig{
Timeout: time.Second,
IsClient: true,
Name: "origin",
Logger: log.NewEntry(log.New()),
Timeout: time.Second,
IsClient: true,
Name: "origin",
Logger: log.NewEntry(log.New()),
DefaultWindowSize: (1 << 8) - 1,
MaxWindowSize: (1 << 15) - 1,
StreamWriteBufferMaxLen: 1024,
},
OriginConn: origin,
EdgeMuxConfig: MuxerConfig{
Timeout: time.Second,
IsClient: false,
Name: "edge",
Logger: log.NewEntry(log.New()),
OriginConn: origin,
EdgeMuxConfig: MuxerConfig{
Timeout: time.Second,
IsClient: false,
Name: "edge",
Logger: log.NewEntry(log.New()),
DefaultWindowSize: (1 << 8) - 1,
MaxWindowSize: (1 << 15) - 1,
StreamWriteBufferMaxLen: 1024,
},
EdgeConn: edge,
doneC: make(chan struct{}),
EdgeConn: edge,
doneC: make(chan struct{}),
}
}
@@ -60,22 +66,22 @@ func NewCompressedMuxerPair(quality CompressionSetting) *DefaultMuxerPair {
origin, edge := net.Pipe()
return &DefaultMuxerPair{
OriginMuxConfig: MuxerConfig{
Timeout: time.Second,
IsClient: true,
Name: "origin",
Timeout: time.Second,
IsClient: true,
Name: "origin",
CompressionQuality: quality,
Logger: log.NewEntry(log.New()),
Logger: log.NewEntry(log.New()),
},
OriginConn: origin,
EdgeMuxConfig: MuxerConfig{
Timeout: time.Second,
IsClient: false,
Name: "edge",
OriginConn: origin,
EdgeMuxConfig: MuxerConfig{
Timeout: time.Second,
IsClient: false,
Name: "edge",
CompressionQuality: quality,
Logger: log.NewEntry(log.New()),
Logger: log.NewEntry(log.New()),
},
EdgeConn: edge,
doneC: make(chan struct{}),
EdgeConn: edge,
doneC: make(chan struct{}),
}
}
@@ -230,7 +236,6 @@ func TestSingleStream(t *testing.T) {
func TestSingleStreamLargeResponseBody(t *testing.T) {
muxPair := NewDefaultMuxerPair()
bodySize := 1 << 24
streamReady := make(chan struct{})
muxPair.OriginMuxConfig.Handler = MuxedStreamFunc(func(stream *MuxedStream) error {
if len(stream.Headers) != 1 {
t.Fatalf("expected %d headers, got %d", 1, len(stream.Headers))
@@ -257,8 +262,6 @@ func TestSingleStreamLargeResponseBody(t *testing.T) {
if n != len(payload) {
t.Fatalf("origin short write: %d/%d bytes", n, len(payload))
}
t.Log("Payload written; signaling that the stream is ready")
streamReady <- struct{}{}
return nil
})
@@ -282,9 +285,6 @@ func TestSingleStreamLargeResponseBody(t *testing.T) {
}
responseBody := make([]byte, bodySize)
<-streamReady
t.Log("Received stream ready signal; resuming the test")
n, err := io.ReadFull(stream, responseBody)
if err != nil {
t.Fatalf("error from (*MuxedStream).Read: %s", err)
@@ -367,14 +367,13 @@ func TestMultipleStreams(t *testing.T) {
log.Error(err)
}
if testFail {
t.Fatalf("TestMultipleStreamsFlowControl failed")
t.Fatalf("TestMultipleStreams failed")
}
}
func TestMultipleStreamsFlowControl(t *testing.T) {
maxStreams := 32
errorsC := make(chan error, maxStreams)
streamReady := make(chan struct{})
responseSizes := make([]int32, maxStreams)
for i := 0; i < maxStreams; i++ {
responseSizes[i] = rand.Int31n(int32(defaultWindowSize << 4))
@@ -398,7 +397,6 @@ func TestMultipleStreamsFlowControl(t *testing.T) {
payload[i] = byte(i % 256)
}
n, err := stream.Write(payload)
streamReady <- struct{}{}
if err != nil {
t.Fatalf("origin write error: %s", err)
}
@@ -435,7 +433,6 @@ func TestMultipleStreamsFlowControl(t *testing.T) {
return
}
<-streamReady
responseBody := make([]byte, responseSizes[(stream.streamID-2)/2])
n, err := io.ReadFull(stream, responseBody)
if err != nil {
@@ -782,9 +779,11 @@ func TestMultipleStreamsWithDictionaries(t *testing.T) {
}
wg.Add(len(paths))
errorsC := make(chan error, len(paths))
for i, s := range paths {
go func(i int, path string) {
defer wg.Done()
stream, err := muxPair.EdgeMux.OpenStream(
[]Header{
{Name: ":method", Value: "GET"},
@@ -805,24 +804,33 @@ func TestMultipleStreamsWithDictionaries(t *testing.T) {
responseBody := make([]byte, len(expectBody)*2)
n, err := stream.Read(responseBody)
if err != nil {
log.Printf("error from (*MuxedStream).Read: %s", err)
t.Fatalf("error from (*MuxedStream).Read: %s", err)
errorsC <- fmt.Errorf("stream %d error from (*MuxedStream).Read: %s", stream.streamID, err)
return
}
if n != len(expectBody) {
log.Printf("expected response body to have %d bytes, got %d", len(expectBody), n)
t.Fatalf("expected response body to have %d bytes, got %d", len(expectBody), n)
errorsC <- fmt.Errorf("stream %d expected response body to have %d bytes, got %d", stream.streamID, len(expectBody), n)
return
}
if string(responseBody[:n]) != expectBody {
log.Printf("expected response body %s, got %s", expectBody, responseBody[:n])
t.Fatalf("expected response body %s, got %s", expectBody, responseBody[:n])
errorsC <- fmt.Errorf("stream %d expected response body %s, got %s", stream.streamID, expectBody, responseBody[:n])
return
}
wg.Done()
}(i, s)
time.Sleep(1 * time.Millisecond)
}
wg.Wait()
if q > CompressionNone && muxPair.OriginMux.muxMetricsUpdater.compBytesBefore.Value() <= 10*muxPair.OriginMux.muxMetricsUpdater.compBytesAfter.Value() {
wg.Wait()
close(errorsC)
testFail := false
for err := range errorsC {
testFail = true
log.Error(err)
}
if testFail {
t.Fatalf("TestMultipleStreams failed")
}
originMuxMetrics := muxPair.OriginMux.Metrics()
if q > CompressionNone && originMuxMetrics.CompBytesBefore.Value() <= 10*originMuxMetrics.CompBytesAfter.Value() {
t.Fatalf("Cross-stream compression is expected to give a better compression ratio")
}
}
@@ -920,7 +928,8 @@ func TestSampleSiteWithDictionaries(t *testing.T) {
}
wg.Wait()
if q > CompressionNone && muxPair.OriginMux.muxMetricsUpdater.compBytesBefore.Value() <= 10*muxPair.OriginMux.muxMetricsUpdater.compBytesAfter.Value() {
originMuxMetrics := muxPair.OriginMux.Metrics()
if q > CompressionNone && originMuxMetrics.CompBytesBefore.Value() <= 10*originMuxMetrics.CompBytesAfter.Value() {
t.Fatalf("Cross-stream compression is expected to give a better compression ratio")
}
}
@@ -953,7 +962,8 @@ func TestLongSiteWithDictionaries(t *testing.T) {
}
wg.Wait()
if q > CompressionNone && muxPair.OriginMux.muxMetricsUpdater.compBytesBefore.Value() <= 100*muxPair.OriginMux.muxMetricsUpdater.compBytesAfter.Value() {
originMuxMetrics := muxPair.OriginMux.Metrics()
if q > CompressionNone && originMuxMetrics.CompBytesBefore.Value() <= 100*originMuxMetrics.CompBytesAfter.Value() {
t.Fatalf("Cross-stream compression is expected to give a better compression ratio")
}
}
+103 -35
View File
@@ -17,32 +17,51 @@ type ReadWriteClosedCloser interface {
Closed() bool
}
// MuxedStream is logically an HTTP/2 stream, with an additional buffer for outgoing data.
type MuxedStream struct {
Headers []Header
streamID uint32
// The "Receive" end of the stream
readBufferLock sync.RWMutex
readBuffer ReadWriteClosedCloser
// This is the amount of bytes that are in our receive window
// (how much data we can receive into this stream).
receiveWindow uint32
// current receive window size limit. Exponentially increase it when it's exhausted
receiveWindowCurrentMax uint32
// hard limit set in http2 spec. 2^31-1
receiveWindowMax uint32
// The desired size increment for receiveWindow.
// If this is nonzero, a WINDOW_UPDATE frame needs to be sent.
windowUpdate uint32
// The headers that were most recently received.
// Particularly:
// * for an eyeball-initiated stream (as passed to TunnelHandler::ServeStream),
// these are the request headers
// * for a cloudflared-initiated stream (as created by Register/UnregisterTunnel),
// these are the response headers.
// They are useful in both of these contexts; hence `Headers` is public.
Headers []Header
// For use in the context of a cloudflared-initiated stream.
responseHeadersReceived chan struct{}
readBuffer ReadWriteClosedCloser
receiveWindow uint32
// current window size limit. Exponentially increase it when it's exhausted
receiveWindowCurrentMax uint32
// limit set in http2 spec. 2^31-1
receiveWindowMax uint32
// nonzero if a WINDOW_UPDATE frame for a stream needs to be sent
windowUpdate uint32
writeLock sync.Mutex
// The zero value for Buffer is an empty buffer ready to use.
// The "Send" end of the stream
writeLock sync.Mutex
writeBuffer ReadWriteLengther
// The maximum capacity that the send buffer should grow to.
writeBufferMaxLen int
// A channel to be notified when the send buffer is not full.
writeBufferHasSpace chan struct{}
// This is the amount of bytes that are in the peer's receive window
// (how much data we can send from this stream).
sendWindow uint32
readyList *ReadyList
// Reference to the muxer's readyList; signal this for stream data to be sent.
readyList *ReadyList
// The headers that should be sent, and a flag so we only send them once.
headersSent bool
writeHeaders []Header
// EOF-related fields
// true if the write end of this stream has been closed
writeEOF bool
// true if we have sent EOF to the peer
@@ -50,40 +69,63 @@ type MuxedStream struct {
// true if the peer sent us an EOF
receivedEOF bool
// dictionary that was used to compress the stream
// Compression-related fields
receivedUseDict bool
method string
contentType string
path string
dictionaries h2Dictionaries
readBufferLock sync.RWMutex
}
func (s *MuxedStream) Read(p []byte) (n int, err error) {
var readBuffer ReadWriteClosedCloser
if s.dictionaries.read != nil {
s.readBufferLock.RLock()
b := s.readBuffer
readBuffer = s.readBuffer
s.readBufferLock.RUnlock()
return b.Read(p)
} else {
readBuffer = s.readBuffer
}
return s.readBuffer.Read(p)
n, err = readBuffer.Read(p)
s.replenishReceiveWindow(uint32(n))
return
}
func (s *MuxedStream) Write(p []byte) (n int, err error) {
// Blocks until len(p) bytes have been written to the buffer
func (s *MuxedStream) Write(p []byte) (int, error) {
// If assignDictToStream returns success, then it will have acquired the
// writeLock. Otherwise we must acquire it ourselves.
ok := assignDictToStream(s, p)
if !ok {
s.writeLock.Lock()
}
defer s.writeLock.Unlock()
if s.writeEOF {
return 0, io.EOF
}
n, err = s.writeBuffer.Write(p)
if n != len(p) || err != nil {
return n, err
totalWritten := 0
for totalWritten < len(p) {
// If the buffer is full, block till there is more room.
// Use a loop to recheck the buffer size after the lock is reacquired.
for s.writeBufferMaxLen <= s.writeBuffer.Len() {
s.writeLock.Unlock()
<-s.writeBufferHasSpace
s.writeLock.Lock()
}
amountToWrite := len(p) - totalWritten
spaceAvailable := s.writeBufferMaxLen - s.writeBuffer.Len()
if spaceAvailable < amountToWrite {
amountToWrite = spaceAvailable
}
amountWritten, err := s.writeBuffer.Write(p[totalWritten : totalWritten+amountToWrite])
totalWritten += amountWritten
if err != nil {
return totalWritten, err
}
s.writeNotify()
}
s.writeNotify()
return n, nil
return totalWritten, nil
}
func (s *MuxedStream) Close() error {
@@ -164,9 +206,9 @@ func (s *MuxedStream) writeNotify() {
// receive window (how much data we can send).
func (s *MuxedStream) replenishSendWindow(bytes uint32) {
s.writeLock.Lock()
defer s.writeLock.Unlock()
s.sendWindow += bytes
s.writeNotify()
s.writeLock.Unlock()
}
// Call by muxreader when it receives a data frame
@@ -178,17 +220,30 @@ func (s *MuxedStream) consumeReceiveWindow(bytes uint32) bool {
return false
}
s.receiveWindow -= bytes
if s.receiveWindow < s.receiveWindowCurrentMax/2 {
if s.receiveWindow < s.receiveWindowCurrentMax/2 && s.receiveWindowCurrentMax < s.receiveWindowMax {
// exhausting client send window (how much data client can send)
if s.receiveWindowCurrentMax < s.receiveWindowMax {
s.receiveWindowCurrentMax <<= 1
// and there is room to grow the receive window
newMax := s.receiveWindowCurrentMax << 1
if newMax > s.receiveWindowMax {
newMax = s.receiveWindowMax
}
s.windowUpdate += s.receiveWindowCurrentMax - s.receiveWindow
s.windowUpdate += newMax - s.receiveWindowCurrentMax
s.receiveWindowCurrentMax = newMax
// notify MuxWriter to write WINDOW_UPDATE frame
s.writeNotify()
}
return true
}
// Arranges for the MuxWriter to send a WINDOW_UPDATE
// Called by MuxedStream::Read when data has left the read buffer.
func (s *MuxedStream) replenishReceiveWindow(bytes uint32) {
s.writeLock.Lock()
defer s.writeLock.Unlock()
s.windowUpdate += bytes
s.writeNotify()
}
// receiveEOF should be called when the peer indicates no more data will be sent.
// Returns true if the socket is now closed (i.e. the write side is already closed).
func (s *MuxedStream) receiveEOF() (closed bool) {
@@ -226,7 +281,8 @@ type streamChunk struct {
// true if a HEADERS frame should be sent
sendHeaders bool
headers []Header
// nonzero if a WINDOW_UPDATE frame should be sent
// nonzero if a WINDOW_UPDATE frame should be sent;
// in that case, it is the increment value to use
windowUpdate uint32
// true if data frames should be sent
sendData bool
@@ -249,11 +305,23 @@ func (s *MuxedStream) getChunk() *streamChunk {
eof: s.writeEOF && uint32(s.writeBuffer.Len()) <= s.sendWindow,
}
// Copies at most s.sendWindow bytes
// Copy at most s.sendWindow bytes, adjust the sendWindow accordingly
writeLen, _ := io.CopyN(&chunk.buffer, s.writeBuffer, int64(s.sendWindow))
s.sendWindow -= uint32(writeLen)
// Non-blocking channel send. This will allow MuxedStream::Write() to continue, if needed
if s.writeBuffer.Len() < s.writeBufferMaxLen {
select {
case s.writeBufferHasSpace <- struct{}{}:
default:
}
}
// When we write the chunk, we'll write the WINDOW_UPDATE frame if needed
s.receiveWindow += s.windowUpdate
s.windowUpdate = 0
// When we write the chunk, we'll write the headers if needed
s.headersSent = true
// if this chunk contains the end of the stream, close the stream now
+27 -19
View File
@@ -23,47 +23,55 @@ func TestFlowControlSingleStream(t *testing.T) {
sendWindow: testWindowSize,
readyList: NewReadyList(),
}
var tempWindowUpdate uint32
var tempStreamChunk *streamChunk
assert.True(t, stream.consumeReceiveWindow(testWindowSize/2))
dataSent := testWindowSize / 2
assert.Equal(t, testWindowSize-dataSent, stream.receiveWindow)
assert.Equal(t, testWindowSize, stream.receiveWindowCurrentMax)
assert.Equal(t, uint32(0), stream.windowUpdate)
tempWindowUpdate := stream.windowUpdate
streamChunk := stream.getChunk()
assert.Equal(t, tempWindowUpdate, streamChunk.windowUpdate)
assert.Equal(t, testWindowSize-dataSent, stream.receiveWindow)
assert.Equal(t, uint32(0), stream.windowUpdate)
assert.Equal(t, testWindowSize, stream.sendWindow)
assert.Equal(t, uint32(0), stream.windowUpdate)
tempStreamChunk = stream.getChunk()
assert.Equal(t, uint32(0), tempStreamChunk.windowUpdate)
assert.Equal(t, testWindowSize-dataSent, stream.receiveWindow)
assert.Equal(t, testWindowSize, stream.receiveWindowCurrentMax)
assert.Equal(t, testWindowSize, stream.sendWindow)
assert.Equal(t, uint32(0), stream.windowUpdate)
assert.True(t, stream.consumeReceiveWindow(2))
dataSent += 2
assert.Equal(t, testWindowSize-dataSent, stream.receiveWindow)
assert.Equal(t, testWindowSize<<1, stream.receiveWindowCurrentMax)
assert.Equal(t, (testWindowSize<<1)-stream.receiveWindow, stream.windowUpdate)
assert.Equal(t, testWindowSize, stream.sendWindow)
assert.Equal(t, testWindowSize, stream.windowUpdate)
tempWindowUpdate = stream.windowUpdate
streamChunk = stream.getChunk()
assert.Equal(t, tempWindowUpdate, streamChunk.windowUpdate)
assert.Equal(t, testWindowSize<<1, stream.receiveWindow)
assert.Equal(t, uint32(0), stream.windowUpdate)
tempStreamChunk = stream.getChunk()
assert.Equal(t, tempWindowUpdate, tempStreamChunk.windowUpdate)
assert.Equal(t, (testWindowSize<<1)-dataSent, stream.receiveWindow)
assert.Equal(t, testWindowSize<<1, stream.receiveWindowCurrentMax)
assert.Equal(t, testWindowSize, stream.sendWindow)
assert.Equal(t, uint32(0), stream.windowUpdate)
assert.True(t, stream.consumeReceiveWindow(testWindowSize+10))
dataSent = testWindowSize + 10
dataSent += testWindowSize + 10
assert.Equal(t, (testWindowSize<<1)-dataSent, stream.receiveWindow)
assert.Equal(t, testWindowSize<<2, stream.receiveWindowCurrentMax)
assert.Equal(t, (testWindowSize<<2)-stream.receiveWindow, stream.windowUpdate)
assert.Equal(t, testWindowSize, stream.sendWindow)
assert.Equal(t, testWindowSize<<1, stream.windowUpdate)
tempWindowUpdate = stream.windowUpdate
streamChunk = stream.getChunk()
assert.Equal(t, tempWindowUpdate, streamChunk.windowUpdate)
assert.Equal(t, testWindowSize<<2, stream.receiveWindow)
assert.Equal(t, uint32(0), stream.windowUpdate)
tempStreamChunk = stream.getChunk()
assert.Equal(t, tempWindowUpdate, tempStreamChunk.windowUpdate)
assert.Equal(t, (testWindowSize<<2)-dataSent, stream.receiveWindow)
assert.Equal(t, testWindowSize<<2, stream.receiveWindowCurrentMax)
assert.Equal(t, testWindowSize, stream.sendWindow)
assert.Equal(t, uint32(0), stream.windowUpdate)
assert.False(t, stream.consumeReceiveWindow(testMaxWindowSize+1))
assert.Equal(t, testWindowSize<<2, stream.receiveWindow)
assert.Equal(t, (testWindowSize<<2)-dataSent, stream.receiveWindow)
assert.Equal(t, testMaxWindowSize, stream.receiveWindowCurrentMax)
}
+75 -20
View File
@@ -17,7 +17,24 @@ const (
updateFreq = time.Second
)
type muxMetricsUpdater struct {
type muxMetricsUpdater interface {
// metrics returns the latest metrics
metrics() *MuxerMetrics
// run is a blocking call to start the event loop
run(logger *log.Entry) error
// updateRTTChan is called by muxReader to report new RTT measurements
updateRTT(rtt *roundTripMeasurement)
//updateReceiveWindowChan is called by muxReader and muxWriter when receiveWindow size is updated
updateReceiveWindow(receiveWindow uint32)
//updateSendWindowChan is called by muxReader and muxWriter when sendWindow size is updated
updateSendWindow(sendWindow uint32)
// updateInBoundBytesChan is called periodicallyby muxReader to report bytesRead
updateInBoundBytes(inBoundBytes uint64)
// updateOutBoundBytesChan is called periodically by muxWriter to report bytesWrote
updateOutBoundBytes(outBoundBytes uint64)
}
type muxMetricsUpdaterImpl struct {
// rttData keeps record of rtt, rttMin, rttMax and last measured time
rttData *rttData
// receiveWindowData keeps record of receive window measurement
@@ -28,16 +45,16 @@ type muxMetricsUpdater struct {
inBoundRate *rate
// outBoundRate is outgoing bytes/sec
outBoundRate *rate
// updateRTTChan is the channel to receive new RTT measurement from muxReader
updateRTTChan <-chan *roundTripMeasurement
//updateReceiveWindowChan is the channel to receive updated receiveWindow size from muxReader and muxWriter
updateReceiveWindowChan <-chan uint32
//updateSendWindowChan is the channel to receive updated sendWindow size from muxReader and muxWriter
updateSendWindowChan <-chan uint32
// updateInBoundBytesChan us the channel to receive bytesRead from muxReader
updateInBoundBytesChan <-chan uint64
// updateOutBoundBytesChan us the channel to receive bytesWrote from muxWriter
updateOutBoundBytesChan <-chan uint64
// updateRTTChan is the channel to receive new RTT measurement
updateRTTChan chan *roundTripMeasurement
//updateReceiveWindowChan is the channel to receive updated receiveWindow size
updateReceiveWindowChan chan uint32
//updateSendWindowChan is the channel to receive updated sendWindow size
updateSendWindowChan chan uint32
// updateInBoundBytesChan us the channel to receive bytesRead
updateInBoundBytesChan chan uint64
// updateOutBoundBytesChan us the channel to receive bytesWrote
updateOutBoundBytesChan chan uint64
// shutdownC is to signal the muxerMetricsUpdater to shutdown
abortChan <-chan struct{}
@@ -84,15 +101,16 @@ type rate struct {
}
func newMuxMetricsUpdater(
updateRTTChan <-chan *roundTripMeasurement,
updateReceiveWindowChan <-chan uint32,
updateSendWindowChan <-chan uint32,
updateInBoundBytesChan <-chan uint64,
updateOutBoundBytesChan <-chan uint64,
abortChan <-chan struct{},
compBytesBefore, compBytesAfter *AtomicCounter,
) *muxMetricsUpdater {
return &muxMetricsUpdater{
) muxMetricsUpdater {
updateRTTChan := make(chan *roundTripMeasurement, 1)
updateReceiveWindowChan := make(chan uint32, 1)
updateSendWindowChan := make(chan uint32, 1)
updateInBoundBytesChan := make(chan uint64)
updateOutBoundBytesChan := make(chan uint64)
return &muxMetricsUpdaterImpl{
rttData: newRTTData(),
receiveWindowData: newFlowControlData(),
sendWindowData: newFlowControlData(),
@@ -109,7 +127,7 @@ func newMuxMetricsUpdater(
}
}
func (updater *muxMetricsUpdater) Metrics() *MuxerMetrics {
func (updater *muxMetricsUpdaterImpl) metrics() *MuxerMetrics {
m := &MuxerMetrics{}
m.RTT, m.RTTMin, m.RTTMax = updater.rttData.metrics()
m.ReceiveWindowAve, m.ReceiveWindowMin, m.ReceiveWindowMax = updater.receiveWindowData.metrics()
@@ -120,7 +138,7 @@ func (updater *muxMetricsUpdater) Metrics() *MuxerMetrics {
return m
}
func (updater *muxMetricsUpdater) run(parentLogger *log.Entry) error {
func (updater *muxMetricsUpdaterImpl) run(parentLogger *log.Entry) error {
logger := parentLogger.WithFields(log.Fields{
"subsystem": "mux",
"dir": "metrics",
@@ -152,6 +170,43 @@ func (updater *muxMetricsUpdater) run(parentLogger *log.Entry) error {
}
}
func (updater *muxMetricsUpdaterImpl) updateRTT(rtt *roundTripMeasurement) {
select {
case updater.updateRTTChan <- rtt:
case <-updater.abortChan:
}
}
func (updater *muxMetricsUpdaterImpl) updateReceiveWindow(receiveWindow uint32) {
select {
case updater.updateReceiveWindowChan <- receiveWindow:
case <-updater.abortChan:
}
}
func (updater *muxMetricsUpdaterImpl) updateSendWindow(sendWindow uint32) {
select {
case updater.updateSendWindowChan <- sendWindow:
case <-updater.abortChan:
}
}
func (updater *muxMetricsUpdaterImpl) updateInBoundBytes(inBoundBytes uint64) {
select {
case updater.updateInBoundBytesChan <- inBoundBytes:
case <-updater.abortChan:
}
}
func (updater *muxMetricsUpdaterImpl) updateOutBoundBytes(outBoundBytes uint64) {
select {
case updater.updateOutBoundBytesChan <- outBoundBytes:
case <-updater.abortChan:
}
}
func newRTTData() *rttData {
return &rttData{}
}
+18 -29
View File
@@ -86,24 +86,11 @@ func TestFlowControlDataUpdate(t *testing.T) {
}
func TestMuxMetricsUpdater(t *testing.T) {
t.Skip("Race condition")
updateRTTChan := make(chan *roundTripMeasurement)
updateReceiveWindowChan := make(chan uint32)
updateSendWindowChan := make(chan uint32)
updateInBoundBytesChan := make(chan uint64)
updateOutBoundBytesChan := make(chan uint64)
abortChan := make(chan struct{})
t.Skip("Inherently racy test due to muxMetricsUpdaterImpl.run()")
errChan := make(chan error)
abortChan := make(chan struct{})
compBefore, compAfter := NewAtomicCounter(0), NewAtomicCounter(0)
m := newMuxMetricsUpdater(updateRTTChan,
updateReceiveWindowChan,
updateSendWindowChan,
updateInBoundBytesChan,
updateOutBoundBytesChan,
abortChan,
compBefore,
compAfter,
)
m := newMuxMetricsUpdater(abortChan, compBefore, compAfter)
logger := log.NewEntry(log.New())
go func() {
@@ -116,42 +103,44 @@ func TestMuxMetricsUpdater(t *testing.T) {
// mock muxReader
readerStart := time.Now()
rm := &roundTripMeasurement{receiveTime: readerStart, sendTime: readerStart}
updateRTTChan <- rm
m.updateRTT(rm)
go func() {
defer wg.Done()
// Becareful if dataPoints is not divisibile by 4
assert.Equal(t, 0, dataPoints%4,
"dataPoints is not divisible by 4; this test should be adjusted accordingly")
readerSend := readerStart.Add(time.Millisecond)
for i := 1; i <= dataPoints/4; i++ {
readerReceive := readerSend.Add(time.Duration(i) * time.Millisecond)
rm := &roundTripMeasurement{receiveTime: readerReceive, sendTime: readerSend}
updateRTTChan <- rm
m.updateRTT(rm)
readerSend = readerReceive.Add(time.Millisecond)
m.updateReceiveWindow(uint32(i))
m.updateSendWindow(uint32(i))
updateReceiveWindowChan <- uint32(i)
updateSendWindowChan <- uint32(i)
updateInBoundBytesChan <- uint64(i)
m.updateInBoundBytes(uint64(i))
}
}()
// mock muxWriter
go func() {
defer wg.Done()
assert.Equal(t, 0, dataPoints%4,
"dataPoints is not divisible by 4; this test should be adjusted accordingly")
for j := dataPoints/4 + 1; j <= dataPoints/2; j++ {
updateReceiveWindowChan <- uint32(j)
updateSendWindowChan <- uint32(j)
m.updateReceiveWindow(uint32(j))
m.updateSendWindow(uint32(j))
// should always be disgard since the send time is before readerSend
// should always be disgarded since the send time is before readerSend
rm := &roundTripMeasurement{receiveTime: readerStart, sendTime: readerStart.Add(-time.Duration(j*dataPoints) * time.Millisecond)}
updateRTTChan <- rm
m.updateRTT(rm)
updateOutBoundBytesChan <- uint64(j)
m.updateOutBoundBytes(uint64(j))
}
}()
wg.Wait()
metrics := m.Metrics()
metrics := m.metrics()
points := dataPoints / 2
assert.Equal(t, time.Millisecond, metrics.RTTMin)
assert.Equal(t, time.Duration(dataPoints/4)*time.Millisecond, metrics.RTTMax)
+15 -17
View File
@@ -35,18 +35,14 @@ type MuxReader struct {
initialStreamWindow uint32
// The max value for the send window of a stream.
streamWindowMax uint32
// The max size for the write buffer of a stream
streamWriteBufferMaxLen int
// r is a reference to the underlying connection used when shutting down.
r io.Closer
// updateRTTChan is the channel to send new RTT measurement to muxerMetricsUpdater
updateRTTChan chan<- *roundTripMeasurement
// updateReceiveWindowChan is the channel to update receiveWindow size to muxerMetricsUpdater
updateReceiveWindowChan chan<- uint32
// updateSendWindowChan is the channel to update sendWindow size to muxerMetricsUpdater
updateSendWindowChan chan<- uint32
// bytesRead is the amount of bytes read from data frame since the last time we send bytes read to metrics
// metricsUpdater is used to report metrics
metricsUpdater muxMetricsUpdater
// bytesRead is the amount of bytes read from data frames since the last time we called metricsUpdater.updateInBoundBytes()
bytesRead *AtomicCounter
// updateOutBoundBytesChan is the channel to send bytesWrote to muxerMetricsUpdater
updateInBoundBytesChan chan<- uint64
// dictionaries holds the h2 cross-stream compression dictionaries
dictionaries h2Dictionaries
}
@@ -79,7 +75,7 @@ func (r *MuxReader) run(parentLogger *log.Entry) error {
case <-r.abortChan:
return
case <-tickC:
r.updateInBoundBytesChan <- r.bytesRead.Count()
r.metricsUpdater.updateInBoundBytes(r.bytesRead.Count())
}
}
}()
@@ -153,6 +149,8 @@ func (r *MuxReader) newMuxedStream(streamID uint32) *MuxedStream {
streamID: streamID,
readBuffer: NewSharedBuffer(),
writeBuffer: &bytes.Buffer{},
writeBufferMaxLen: r.streamWriteBufferMaxLen,
writeBufferHasSpace: make(chan struct{}, 1),
receiveWindow: r.initialStreamWindow,
receiveWindowCurrentMax: r.initialStreamWindow,
receiveWindowMax: r.streamWindowMax,
@@ -285,7 +283,7 @@ func (r *MuxReader) receiveFrameData(frame *http2.DataFrame, parentLogger *log.E
if !stream.consumeReceiveWindow(uint32(len(data))) {
return r.streamError(stream.streamID, http2.ErrCodeFlowControl)
}
r.updateReceiveWindowChan <- stream.getReceiveWindow()
r.metricsUpdater.updateReceiveWindow(stream.getReceiveWindow())
return nil
}
@@ -297,13 +295,13 @@ func (r *MuxReader) receivePingData(frame *http2.PingFrame) {
return
}
// Update updates the computed values with a new measurement.
// outgoingTime is the time that the probe was sent.
// We assume that time.Now() is the time we received that probe.
r.updateRTTChan <- &roundTripMeasurement{
// Update the computed RTT aggregations with a new measurement.
// `ts` is the time that the probe was sent.
// We assume that `time.Now()` is the time we received that probe.
r.metricsUpdater.updateRTT(&roundTripMeasurement{
receiveTime: time.Now(),
sendTime: time.Unix(0, ts),
}
})
}
// Receive a GOAWAY from the peer. Gracefully shut down our connection.
@@ -464,7 +462,7 @@ func (r *MuxReader) updateStreamWindow(frame *http2.WindowUpdateFrame) error {
return nil
}
stream.replenishSendWindow(frame.Increment)
r.updateSendWindowChan <- stream.getSendWindow()
r.metricsUpdater.updateSendWindow(stream.getSendWindow())
return nil
}
+7 -10
View File
@@ -40,14 +40,11 @@ type MuxWriter struct {
headerEncoder *hpack.Encoder
// headerBuffer is the temporary buffer used by headerEncoder.
headerBuffer bytes.Buffer
// updateReceiveWindowChan is the channel to update receiveWindow size to muxerMetricsUpdater
updateReceiveWindowChan chan<- uint32
// updateSendWindowChan is the channel to update sendWindow size to muxerMetricsUpdater
updateSendWindowChan chan<- uint32
// bytesWrote is the amount of bytes wrote to data frame since the last time we send bytes wrote to metrics
// metricsUpdater is used to report metrics
metricsUpdater muxMetricsUpdater
// bytesWrote is the amount of bytes written to data frames since the last time we called metricsUpdater.updateOutBoundBytes()
bytesWrote *AtomicCounter
// updateOutBoundBytesChan is the channel to send bytesWrote to muxerMetricsUpdater
updateOutBoundBytesChan chan<- uint64
useDictChan <-chan useDictRequest
}
@@ -83,7 +80,7 @@ func (w *MuxWriter) run(parentLogger *log.Entry) error {
case <-w.abortChan:
return
case <-tickC:
w.updateOutBoundBytesChan <- w.bytesWrote.Count()
w.metricsUpdater.updateOutBoundBytes(w.bytesWrote.Count())
}
}
}()
@@ -172,8 +169,8 @@ func (w *MuxWriter) run(parentLogger *log.Entry) error {
func (w *MuxWriter) writeStreamData(stream *MuxedStream, logger *log.Entry) error {
logger.Debug("writable")
chunk := stream.getChunk()
w.updateReceiveWindowChan <- stream.getReceiveWindow()
w.updateSendWindowChan <- stream.getSendWindow()
w.metricsUpdater.updateReceiveWindow(stream.getReceiveWindow())
w.metricsUpdater.updateSendWindow(stream.getSendWindow())
if chunk.sendHeadersFrame() {
err := w.writeHeaders(chunk.streamID, chunk.headers)
if err != nil {
+24 -13
View File
@@ -1,15 +1,23 @@
package h2mux
import "sync"
// ReadyList multiplexes several event signals onto a single channel.
type ReadyList struct {
// signalC is used to signal that a stream can be enqueued
signalC chan uint32
waitC chan uint32
// waitC is used to signal the ID of the first ready descriptor
waitC chan uint32
// doneC is used to signal that run should terminate
doneC chan struct{}
closeOnce sync.Once
}
func NewReadyList() *ReadyList {
rl := &ReadyList{
signalC: make(chan uint32),
waitC: make(chan uint32),
doneC: make(chan struct{}),
}
go rl.run()
return rl
@@ -17,7 +25,11 @@ func NewReadyList() *ReadyList {
// ID is the stream ID
func (r *ReadyList) Signal(ID uint32) {
r.signalC <- ID
select {
case r.signalC <- ID:
// ReadyList already closed
case <-r.doneC:
}
}
func (r *ReadyList) ReadyChannel() <-chan uint32 {
@@ -25,7 +37,9 @@ func (r *ReadyList) ReadyChannel() <-chan uint32 {
}
func (r *ReadyList) Close() {
close(r.signalC)
r.closeOnce.Do(func() {
close(r.doneC)
})
}
func (r *ReadyList) run() {
@@ -35,28 +49,25 @@ func (r *ReadyList) run() {
activeDescriptors := newReadyDescriptorMap()
for {
if firstReady == nil {
// Wait for first ready descriptor
i, ok := <-r.signalC
if !ok {
// closed
select {
case i := <-r.signalC:
firstReady = activeDescriptors.SetIfMissing(i)
case <-r.doneC:
return
}
firstReady = activeDescriptors.SetIfMissing(i)
}
select {
case r.waitC <- firstReady.ID:
activeDescriptors.Delete(firstReady.ID)
firstReady = queue.Dequeue()
case i, ok := <-r.signalC:
if !ok {
// closed
return
}
case i := <-r.signalC:
newReady := activeDescriptors.SetIfMissing(i)
if newReady != nil {
// key doesn't exist
queue.Enqueue(newReady)
}
case <-r.doneC:
return
}
}
}
+80 -24
View File
@@ -3,36 +3,59 @@ package h2mux
import (
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestReadyList(t *testing.T) {
func assertEmpty(t *testing.T, rl *ReadyList) {
select {
case <-rl.ReadyChannel():
t.Fatal("Spurious wakeup")
default:
}
}
func assertClosed(t *testing.T, rl *ReadyList) {
select {
case _, ok := <-rl.ReadyChannel():
assert.False(t, ok, "ReadyChannel was not closed")
case <-time.After(100 * time.Millisecond):
t.Fatalf("Timeout")
}
}
func receiveWithTimeout(t *testing.T, rl *ReadyList) uint32 {
select {
case i := <-rl.ReadyChannel():
return i
case <-time.After(100 * time.Millisecond):
t.Fatalf("Timeout")
return 0
}
}
func TestReadyListEmpty(t *testing.T) {
rl := NewReadyList()
c := rl.ReadyChannel()
// helper functions
assertEmpty := func() {
select {
case <-c:
t.Fatalf("Spurious wakeup")
default:
}
}
receiveWithTimeout := func() uint32 {
select {
case i := <-c:
return i
case <-time.After(100 * time.Millisecond):
t.Fatalf("Timeout")
return 0
}
}
// no signals, receive should fail
assertEmpty()
assertEmpty(t, rl)
}
func TestReadyListSignal(t *testing.T) {
rl := NewReadyList()
assertEmpty(t, rl)
rl.Signal(0)
if receiveWithTimeout() != 0 {
if receiveWithTimeout(t, rl) != 0 {
t.Fatalf("Received wrong ID of signalled event")
}
// no new signals, receive should fail
assertEmpty()
assertEmpty(t, rl)
}
func TestReadyListMultipleSignals(t *testing.T) {
rl := NewReadyList()
assertEmpty(t, rl)
// Signals should not block;
// Duplicate unhandled signals should not cause multiple wakeups
signalled := [5]bool{}
@@ -42,12 +65,45 @@ func TestReadyList(t *testing.T) {
}
// All signals should be received once (in any order)
for range signalled {
i := receiveWithTimeout()
i := receiveWithTimeout(t, rl)
if signalled[i] {
t.Fatalf("Received signal %d more than once", i)
}
signalled[i] = true
}
for i := range signalled {
if !signalled[i] {
t.Fatalf("Never received signal %d", i)
}
}
assertEmpty(t, rl)
}
func TestReadyListClose(t *testing.T) {
rl := NewReadyList()
rl.Close()
// readyList.run() occurs in a separate goroutine,
// so there's no way to directly check that run() has terminated.
// Perform an indirect check: is the ready channel closed?
assertClosed(t, rl)
// a second rl.Close() shouldn't cause a panic
rl.Close()
// Signal shouldn't block after Close()
done := make(chan struct{})
go func() {
for i := 0; i < 5; i++ {
rl.Signal(uint32(i))
}
close(done)
}()
select {
case <-done:
case <-time.After(100 * time.Millisecond):
t.Fatal("Test timed out")
}
}
func TestReadyDescriptorQueue(t *testing.T) {
+1 -3
View File
@@ -43,7 +43,7 @@ const indexTemplate = `
<meta name="description" content="Argo Tunnel Connection">
<meta name="viewport" content="width=device-width, initial-scale=1">
<style>
html{line-height:1.15;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%}body{margin:0}section{display:block}h1{font-size:2em;margin:.67em 0}a{background-color:transparent;-webkit-text-decoration-skip:objects}/* 1 */::-webkit-file-upload-button{-webkit-appearance:button;font:inherit}/* 1 */a,body,dd,div,dl,dt,h1,h4,html,p,section{box-sizing:border-box}.bt{border-top-style:solid;border-top-width:1px}.bl{border-left-style:solid;border-left-width:1px}.b--orange{border-color:#f38020}.br1{border-radius:.125rem}.bw2{border-width:.25rem}.dib{display:inline-block}.sans-serif{font-family:open sans,-apple-system,BlinkMacSystemFont,avenir next,avenir,helvetica neue,helvetica,ubuntu,roboto,noto,segoe ui,arial,sans-serif}.code{font-family:Consolas,monaco,monospace}.b{font-weight:700}.fw3{font-weight:300}.fw4{font-weight:400}.fw5{font-weight:500}.fw6{font-weight:600}.lh-copy{line-height:1.5}.link{text-decoration:none}.link,.link:active,.link:focus,.link:hover,.link:link,.link:visited{transition:color .15s ease-in}.link:focus{outline:1px dotted currentColor}.mw-100{max-width:100%}.mw4{max-width:8rem}.mw7{max-width:48rem}.bg-light-gray{background-color:#f7f7f7}.link-hover:hover{background-color:#1f679e}.white{color:#fff}.bg-white{background-color:#fff}.bg-blue{background-color:#408bc9}.pb2{padding-bottom:.5rem}.pb6{padding-bottom:8rem}.pt3{padding-top:1rem}.pt5{padding-top:4rem}.pv2{padding-top:.5rem;padding-bottom:.5rem}.ph3{padding-left:1rem;padding-right:1rem}.ph4{padding-left:2rem;padding-right:2rem}.ml0{margin-left:0}.mb1{margin-bottom:.25rem}.mb2{margin-bottom:.5rem}.mb3{margin-bottom:1rem}.mt5{margin-top:4rem}.ttu{text-transform:uppercase}.f4{font-size:1.25rem}.f5{font-size:1rem}.f6{font-size:.875rem}.f7{font-size:.75rem}.measure{max-width:30em}.center{margin-left:auto}.center{margin-right:auto}@media screen and (min-width:30em){.f2-ns{font-size:2.25rem}}@media screen and (min-width:30em) and (max-width:60em){.f5-m{font-size:1rem}}@media screen and (min-width:60em){.f4-l{font-size:1.25rem}}
html{line-height:1.15;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%}body{margin:0}section{display:block}h1{font-size:2em;margin:.67em 0}a{background-color:transparent;-webkit-text-decoration-skip:objects}/* 1 */::-webkit-file-upload-button{-webkit-appearance:button;font:inherit}/* 1 */a,body,dd,div,dl,dt,h1,h4,html,p,section{box-sizing:border-box}.bt{border-top-style:solid;border-top-width:1px}.bl{border-left-style:solid;border-left-width:1px}.b--orange{border-color:#f38020}.br1{border-radius:.125rem}.bw2{border-width:.25rem}.dib{display:inline-block}.sans-serif{font-family:open sans,-apple-system,BlinkMacSystemFont,avenir next,avenir,helvetica neue,helvetica,ubuntu,roboto,noto,segoe ui,arial,sans-serif}.overflow-x-auto{overflow-x:auto}.code{font-family:Consolas,monaco,monospace}.b{font-weight:700}.fw3{font-weight:300}.fw4{font-weight:400}.fw5{font-weight:500}.fw6{font-weight:600}.lh-copy{line-height:1.5}.link{text-decoration:none}.link,.link:active,.link:focus,.link:hover,.link:link,.link:visited{transition:color .15s ease-in}.link:focus{outline:1px dotted currentColor}.mw-100{max-width:100%}.mw4{max-width:8rem}.mw7{max-width:48rem}.bg-light-gray{background-color:#f7f7f7}.link-hover:hover{background-color:#1f679e}.white{color:#fff}.bg-white{background-color:#fff}.bg-blue{background-color:#408bc9}.pb2{padding-bottom:.5rem}.pb6{padding-bottom:8rem}.pt3{padding-top:1rem}.pt5{padding-top:4rem}.pv2{padding-top:.5rem;padding-bottom:.5rem}.ph3{padding-left:1rem;padding-right:1rem}.ph4{padding-left:2rem;padding-right:2rem}.ml0{margin-left:0}.mb1{margin-bottom:.25rem}.mb2{margin-bottom:.5rem}.mb3{margin-bottom:1rem}.mt5{margin-top:4rem}.ttu{text-transform:uppercase}.f4{font-size:1.25rem}.f5{font-size:1rem}.f6{font-size:.875rem}.f7{font-size:.75rem}.measure{max-width:30em}.center{margin-left:auto}.center{margin-right:auto}@media screen and (min-width:30em){.f2-ns{font-size:2.25rem}}@media screen and (min-width:30em) and (max-width:60em){.f5-m{font-size:1rem}}@media screen and (min-width:60em){.f4-l{font-size:1.25rem}}
.st0{fill:#FFF}.st1{fill:#f48120}.st2{fill:#faad3f}.st3{fill:#404041}
</style>
</head>
@@ -91,7 +91,6 @@ const indexTemplate = `
</html>
`
func StartHelloWorldServer(logger *logrus.Logger, listener net.Listener, shutdownC <-chan struct{}) error {
logger.Infof("Starting Hello World server at %s", listener.Addr())
serverName := defaultServerName
@@ -132,7 +131,6 @@ func CreateTLSListener(address string) (net.Listener, error) {
return listener, err
}
func uptimeHandler(startTime time.Time) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// Note that if autoupdate is enabled, the uptime is reset when a new client
+1 -1
View File
@@ -1,6 +1,7 @@
package metrics
import (
"context"
"net"
"net/http"
_ "net/http/pprof"
@@ -8,7 +9,6 @@ import (
"sync"
"time"
"golang.org/x/net/context"
"golang.org/x/net/trace"
"github.com/prometheus/client_golang/prometheus"
+1 -2
View File
@@ -1,9 +1,8 @@
package origin
import (
"context"
"time"
"golang.org/x/net/context"
)
// Redeclare time functions so they can be overridden in tests.
+1 -2
View File
@@ -1,10 +1,9 @@
package origin
import (
"context"
"testing"
"time"
"golang.org/x/net/context"
)
func immediateTimeAfter(time.Duration) <-chan time.Time {
+117
View File
@@ -0,0 +1,117 @@
package origin
import (
"encoding/json"
"io/ioutil"
"net/http"
"strings"
"time"
"github.com/cloudflare/golibs/lrucache"
)
// StatusPage.io API docs:
// https://www.cloudflarestatus.com/api/v2/#incidents-unresolved
const (
activeIncidentsURL = "https://yh6f0r4529hb.statuspage.io/api/v2/incidents/unresolved.json"
argoTunnelKeyword = "argo tunnel"
incidentDetailsPrefix = "https://www.cloudflarestatus.com/incidents/"
)
// IncidentLookup is an object that checks for active incidents in
// the Cloudflare infrastructure.
type IncidentLookup interface {
ActiveIncidents() []Incident
}
// NewIncidentLookup returns a new IncidentLookup instance that caches its
// results with a 1-minute TTL.
func NewIncidentLookup() IncidentLookup {
return newCachedIncidentLookup(fetchActiveIncidents)
}
type IncidentUpdate struct {
Body string
}
type Incident struct {
Name string
ID string `json:"id"`
Updates []IncidentUpdate `json:"incident_updates"`
}
type StatusPage struct {
Incidents []Incident
}
func (i Incident) URL() string {
return incidentDetailsPrefix + i.ID
}
func parseStatusPage(data []byte) (*StatusPage, error) {
var result StatusPage
err := json.Unmarshal(data, &result)
return &result, err
}
func isArgoTunnelIncident(i Incident) bool {
if strings.Contains(strings.ToLower(i.Name), argoTunnelKeyword) {
return true
}
for _, u := range i.Updates {
if strings.Contains(strings.ToLower(u.Body), argoTunnelKeyword) {
return true
}
}
return false
}
func fetchActiveIncidents() (incidents []Incident) {
resp, err := http.Get(activeIncidentsURL)
if err != nil {
return
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return
}
statusPage, err := parseStatusPage(body)
if err != nil {
return
}
for _, i := range statusPage.Incidents {
if isArgoTunnelIncident(i) {
incidents = append(incidents, i)
}
}
return incidents
}
type cachedIncidentLookup struct {
cache *lrucache.LRUCache
ttl time.Duration
uncachedLookup func() []Incident
}
func newCachedIncidentLookup(uncachedLookup func() []Incident) *cachedIncidentLookup {
return &cachedIncidentLookup{
cache: lrucache.NewLRUCache(1),
ttl: time.Minute,
uncachedLookup: uncachedLookup,
}
}
// We only need one cache entry. Always use the empty string as its key.
const cacheKey = ""
func (c *cachedIncidentLookup) ActiveIncidents() []Incident {
if cached, ok := c.cache.GetNotStale(cacheKey); ok {
if incidents, ok := cached.([]Incident); ok {
return incidents
}
}
incidents := c.uncachedLookup()
c.cache.Set(cacheKey, incidents, time.Now().Add(c.ttl))
return incidents
}
+386
View File
@@ -0,0 +1,386 @@
package origin
import (
"testing"
"time"
"github.com/cloudflare/golibs/lrucache"
"github.com/stretchr/testify/assert"
)
func TestParseStatusPage(t *testing.T) {
testCases := []struct {
input []byte
output *StatusPage
fail bool
}{
{
input: []byte(`<html>
<head><title>504 Gateway Time-out</title></head>
<body><center><h1>504 Gateway Time-out</h1></center></body>
</html>`),
output: nil,
fail: true,
},
{
input: []byte(`{
"page": {
"id": "yh6f0r4529hb",
"name": "Cloudflare",
"url": "https://www.cloudflarestatus.com",
"time_zone": "Etc/UTC",
"updated_at": "2019-01-10T20:11:38.750Z"
},
"incidents": [
{
"name": "Cloudflare API service issues",
"status": "resolved",
"created_at": "2018-09-17T19:29:21.132Z",
"updated_at": "2018-09-18T07:45:41.313Z",
"monitoring_at": "2018-09-17T21:35:06.492Z",
"resolved_at": "2018-09-18T07:45:41.290Z",
"shortlink": "http://stspg.io/7f079791e",
"id": "q746ybtyb6q0",
"page_id": "yh6f0r4529hb",
"incident_updates": [
{
"status": "resolved",
"body": "Cloudflare has resolved the issue and the service have resumed normal operation.",
"created_at": "2018-09-18T07:45:41.290Z",
"updated_at": "2018-09-18T07:45:41.290Z",
"display_at": "2018-09-18T07:45:41.290Z",
"affected_components": [
{
"code": "g4tb35rs9yw7",
"name": "Cloudflare customer dashboard and APIs - Cloudflare APIs",
"old_status": "operational",
"new_status": "operational"
}
],
"deliver_notifications": true,
"tweet_id": null,
"id": "zl5g2pl5zhfs",
"incident_id": "q746ybtyb6q0",
"custom_tweet": null
},
{
"status": "monitoring",
"body": "Cloudflare has implemented a fix for this issue and is currently monitoring the results.\r\n\r\nWe will update the status once the issue is resolved.",
"created_at": "2018-09-17T21:35:06.492Z",
"updated_at": "2018-09-17T21:35:06.492Z",
"display_at": "2018-09-17T21:35:06.492Z",
"affected_components": [
{
"code": "g4tb35rs9yw7",
"name": "Cloudflare customer dashboard and APIs - Cloudflare APIs",
"old_status": "degraded_performance",
"new_status": "operational"
}
],
"deliver_notifications": false,
"tweet_id": null,
"id": "0001sv3chdnx",
"incident_id": "q746ybtyb6q0",
"custom_tweet": null
},
{
"status": "investigating",
"body": "We are continuing to investigate this issue.",
"created_at": "2018-09-17T19:30:08.049Z",
"updated_at": "2018-09-17T19:30:08.049Z",
"display_at": "2018-09-17T19:30:08.049Z",
"affected_components": [
{
"code": "g4tb35rs9yw7",
"name": "Cloudflare customer dashboard and APIs - Cloudflare APIs",
"old_status": "operational",
"new_status": "degraded_performance"
}
],
"deliver_notifications": false,
"tweet_id": null,
"id": "qdr164tfpq7m",
"incident_id": "q746ybtyb6q0",
"custom_tweet": null
},
{
"status": "investigating",
"body": "Cloudflare is investigating issues with APIs and Page Rule delays for Page Rule updates. Cloudflare Page Rule service delivery is unaffected and is operating normally. Also, these issues do not affect the Cloudflare CDN and therefore, do not impact customer websites.",
"created_at": "2018-09-17T19:29:21.201Z",
"updated_at": "2018-09-17T19:29:21.201Z",
"display_at": "2018-09-17T19:29:21.201Z",
"affected_components": [
{
"code": "g4tb35rs9yw7",
"name": "Cloudflare customer dashboard and APIs - Cloudflare APIs",
"old_status": "operational",
"new_status": "operational"
}
],
"deliver_notifications": false,
"tweet_id": null,
"id": "qzl2n0q8tskg",
"incident_id": "q746ybtyb6q0",
"custom_tweet": null
}
],
"components": [
{
"status": "operational",
"name": "Cloudflare APIs",
"created_at": "2014-10-09T03:32:07.158Z",
"updated_at": "2019-01-01T22:58:30.846Z",
"position": 2,
"description": null,
"showcase": false,
"id": "g4tb35rs9yw7",
"page_id": "yh6f0r4529hb",
"group_id": "1km35smx8p41",
"group": false,
"only_show_if_degraded": false,
"automation_email": "component+g4tb35rs9yw7@notifications.statuspage.io"
}
],
"impact": "minor"
},
{
"name": "Web Analytics Delays",
"status": "resolved",
"created_at": "2018-09-17T18:05:39.907Z",
"updated_at": "2018-09-17T22:53:05.078Z",
"monitoring_at": null,
"resolved_at": "2018-09-17T22:53:05.057Z",
"shortlink": "http://stspg.io/cb208928c",
"id": "wqfk9mzs5qt1",
"page_id": "yh6f0r4529hb",
"incident_updates": [
{
"status": "resolved",
"body": "Cloudflare has resolved the issue and Web Analytics have resumed normal operation.",
"created_at": "2018-09-17T22:53:05.057Z",
"updated_at": "2018-09-17T22:53:05.057Z",
"display_at": "2018-09-17T22:53:05.057Z",
"affected_components": [
{
"code": "4c231tkdlpcl",
"name": "Cloudflare customer dashboard and APIs - Analytics",
"old_status": "degraded_performance",
"new_status": "operational"
}
],
"deliver_notifications": false,
"tweet_id": null,
"id": "93y1w00yqzk4",
"incident_id": "wqfk9mzs5qt1",
"custom_tweet": null
},
{
"status": "investigating",
"body": "There is a delay in processing Cloudflare Web Analytics. This affects timely delivery of customer data.\n\nThese delays do not impact analytics for DNS and Rate Limiting.",
"created_at": "2018-09-17T18:05:40.033Z",
"updated_at": "2018-09-17T18:05:40.033Z",
"display_at": "2018-09-17T18:05:40.033Z",
"affected_components": [
{
"code": "4c231tkdlpcl",
"name": "Cloudflare customer dashboard and APIs - Analytics",
"old_status": "operational",
"new_status": "degraded_performance"
}
],
"deliver_notifications": false,
"tweet_id": null,
"id": "362t6lv0vrpk",
"incident_id": "wqfk9mzs5qt1",
"custom_tweet": null
}
],
"components": [
{
"status": "operational",
"name": "Analytics",
"created_at": "2014-11-13T11:54:10.191Z",
"updated_at": "2018-12-31T08:20:52.349Z",
"position": 3,
"description": "Customer data",
"showcase": false,
"id": "4c231tkdlpcl",
"page_id": "yh6f0r4529hb",
"group_id": "1km35smx8p41",
"group": false,
"only_show_if_degraded": false,
"automation_email": "component+4c231tkdlpcl@notifications.statuspage.io"
}
],
"impact": "minor"
}
]
}`),
output: &StatusPage{
Incidents: []Incident{
Incident{
Name: "Cloudflare API service issues",
ID: "q746ybtyb6q0",
Updates: []IncidentUpdate{
IncidentUpdate{
Body: "Cloudflare has resolved the issue and the service have resumed normal operation.",
},
IncidentUpdate{
Body: "Cloudflare has implemented a fix for this issue and is currently monitoring the results.\r\n\r\nWe will update the status once the issue is resolved.",
},
IncidentUpdate{
Body: "We are continuing to investigate this issue.",
},
IncidentUpdate{
Body: "Cloudflare is investigating issues with APIs and Page Rule delays for Page Rule updates. Cloudflare Page Rule service delivery is unaffected and is operating normally. Also, these issues do not affect the Cloudflare CDN and therefore, do not impact customer websites.",
},
},
},
Incident{
Name: "Web Analytics Delays",
ID: "wqfk9mzs5qt1",
Updates: []IncidentUpdate{
IncidentUpdate{
Body: "Cloudflare has resolved the issue and Web Analytics have resumed normal operation.",
},
IncidentUpdate{
Body: "There is a delay in processing Cloudflare Web Analytics. This affects timely delivery of customer data.\n\nThese delays do not impact analytics for DNS and Rate Limiting.",
},
},
},
},
},
fail: false,
},
}
for _, testCase := range testCases {
output, err := parseStatusPage(testCase.input)
if testCase.fail {
assert.Error(t, err)
} else {
assert.Nil(t, err)
assert.Equal(t, testCase.output, output)
}
}
}
func TestIsArgoTunnelIncident(t *testing.T) {
testCases := []struct {
input Incident
output bool
}{
{
input: Incident{},
output: false,
},
{
input: Incident{Name: "An Argo Tunnel incident"},
output: true,
},
{
input: Incident{Name: "an argo tunnel incident"},
output: true,
},
{
input: Incident{Name: "an aRgO TuNnEl incident"},
output: true,
},
{
input: Incident{Name: "an argotunnel incident"},
output: false,
},
{
input: Incident{Name: "irrelevant"},
output: false,
},
{
input: Incident{
Name: "irrelevant",
Updates: []IncidentUpdate{
IncidentUpdate{Body: "irrelevant"},
IncidentUpdate{Body: "an Argo Tunnel incident"},
IncidentUpdate{Body: "irrelevant"},
},
},
output: true,
},
{
input: Incident{
Name: "an Argo Tunnel incident",
Updates: []IncidentUpdate{
IncidentUpdate{Body: "irrelevant"},
IncidentUpdate{Body: "irrelevant"},
IncidentUpdate{Body: "irrelevant"},
},
},
output: true,
},
}
for _, testCase := range testCases {
actual := isArgoTunnelIncident(testCase.input)
assert.Equal(t, testCase.output, actual, "Test case failed: %v", testCase.input)
}
}
func TestIncidentURL(t *testing.T) {
incident := Incident{
ID: "s6k0dnn5347b",
}
assert.Equal(t, "https://www.cloudflarestatus.com/incidents/s6k0dnn5347b", incident.URL())
}
func TestNewCachedIncidentLookup(t *testing.T) {
c := newCachedIncidentLookup(func() []Incident { return nil })
assert.Equal(t, time.Minute, c.ttl)
assert.Equal(t, 1, c.cache.Capacity())
}
func TestCachedIncidentLookup(t *testing.T) {
expected := []Incident{
Incident{
Name: "An incident",
ID: "incidentID",
},
}
var shouldCallUncachedLookup bool
c := &cachedIncidentLookup{
cache: lrucache.NewLRUCache(1),
ttl: 50 * time.Millisecond,
uncachedLookup: func() []Incident {
if !shouldCallUncachedLookup {
t.Fatal("uncachedLookup shouldn't have been called")
}
return expected
},
}
shouldCallUncachedLookup = true
assert.Equal(t, expected, c.ActiveIncidents())
shouldCallUncachedLookup = false
assert.Equal(t, expected, c.ActiveIncidents())
assert.Equal(t, expected, c.ActiveIncidents())
time.Sleep(50 * time.Millisecond)
shouldCallUncachedLookup = true
assert.Equal(t, expected, c.ActiveIncidents())
}
func TestCachedIncidentLookupDoesntPanic(t *testing.T) {
expected := []Incident{
Incident{
Name: "An incident",
ID: "incidentID",
},
}
c := &cachedIncidentLookup{
cache: lrucache.NewLRUCache(1),
ttl: 50 * time.Millisecond,
uncachedLookup: func() []Incident { return expected },
}
c.cache.Set(cacheKey, 42, time.Now().Add(30*time.Minute))
actual := c.ActiveIncidents()
assert.Equal(t, expected, actual)
}
+63 -2
View File
@@ -1,8 +1,14 @@
package origin
import (
"context"
"crypto/tls"
"fmt"
"net"
"time"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
)
const (
@@ -10,9 +16,25 @@ const (
srvService = "warp"
srvProto = "tcp"
srvName = "cloudflarewarp.com"
// Used to fallback to DoT when we can't use the default resolver to
// discover HA Warp servers (GitHub issue #75).
dotServerName = "cloudflare-dns.com"
dotServerAddr = "1.1.1.1:853"
dotTimeout = time.Duration(15 * time.Second)
)
func ResolveEdgeIPs(addresses []string) ([]*net.TCPAddr, error) {
var friendlyDNSErrorLines = []string{
`Please try the following things to diagnose this issue:`,
` 1. ensure that cloudflarewarp.com is returning "warp" service records.`,
` Run your system's equivalent of: dig srv _warp._tcp.cloudflarewarp.com`,
` 2. ensure that your DNS resolver is not returning compressed SRV records.`,
` See GitHub issue https://github.com/golang/go/issues/27546`,
` For example, you could use Cloudflare's 1.1.1.1 as your resolver:`,
` https://developers.cloudflare.com/1.1.1.1/setting-up-1.1.1.1/`,
}
func ResolveEdgeIPs(logger *log.Logger, addresses []string) ([]*net.TCPAddr, error) {
if len(addresses) > 0 {
var tcpAddrs []*net.TCPAddr
for _, address := range addresses {
@@ -28,7 +50,30 @@ func ResolveEdgeIPs(addresses []string) ([]*net.TCPAddr, error) {
// HA service discovery lookup
_, addrs, err := net.LookupSRV(srvService, srvProto, srvName)
if err != nil {
return nil, err
// Try to fall back to DoT from Cloudflare directly.
//
// Note: Instead of DoT, we could also have used DoH. Either of these:
// - directly via the JSON API (https://1.1.1.1/dns-query?ct=application/dns-json&name=_warp._tcp.cloudflarewarp.com&type=srv)
// - indirectly via `tunneldns.NewUpstreamHTTPS()`
// But both of these cases miss out on a key feature from the stdlib:
// "The returned records are sorted by priority and randomized by weight within a priority."
// (https://golang.org/pkg/net/#Resolver.LookupSRV)
// Does this matter? I don't know. It may someday. Let's use DoT so we don't need to worry about it.
// See also: Go feature request for stdlib-supported DoH: https://github.com/golang/go/issues/27552
r := fallbackResolver(dotServerName, dotServerAddr)
ctx, cancel := context.WithTimeout(context.Background(), dotTimeout)
defer cancel()
_, fallbackAddrs, fallbackErr := r.LookupSRV(ctx, srvService, srvProto, srvName)
if fallbackErr != nil || len(fallbackAddrs) == 0 {
// use the original DNS error `err` in messages, not `fallbackErr`
logger.Errorln("Error looking up Cloudflare edge IPs: the DNS query failed:", err)
for _, s := range friendlyDNSErrorLines {
logger.Errorln(s)
}
return nil, errors.Wrap(err, "Could not lookup srv records on _warp._tcp.cloudflarewarp.com")
}
// Accept the fallback results and keep going
addrs = fallbackAddrs
}
var resolvedIPsPerCNAME [][]*net.TCPAddr
var lookupErr error
@@ -80,3 +125,19 @@ func FlattenServiceIPs(ipsByService [][]*net.TCPAddr) []*net.TCPAddr {
}
return result
}
// Inspiration: https://github.com/artyom/dot/blob/master/dot.go
func fallbackResolver(serverName, serverAddress string) *net.Resolver {
return &net.Resolver{
PreferGo: true,
Dial: func(ctx context.Context, _ string, _ string) (net.Conn, error) {
var dialer net.Dialer
conn, err := dialer.DialContext(ctx, "tcp", serverAddress)
if err != nil {
return nil, err
}
tlsConfig := &tls.Config{ServerName: serverName}
return tls.Client(conn, tlsConfig), nil
},
}
}
+36 -2
View File
@@ -51,7 +51,12 @@ type TunnelMetrics struct {
// oldServerLocations stores the last server the tunnel was connected to
oldServerLocations map[string]string
regSuccess prometheus.Counter
regFail *prometheus.CounterVec
rpcFail *prometheus.CounterVec
muxerMetrics *muxerMetrics
tunnelsHA tunnelsForHA
}
func newMuxerMetrics() *muxerMetrics {
@@ -341,6 +346,31 @@ func NewTunnelMetrics() *TunnelMetrics {
)
prometheus.MustRegister(serverLocations)
rpcFail := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "tunnel_rpc_fail",
Help: "Count of RPC connection errors by type",
},
[]string{"error"},
)
prometheus.MustRegister(rpcFail)
registerFail := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "tunnel_register_fail",
Help: "Count of tunnel registration errors by type",
},
[]string{"error"},
)
prometheus.MustRegister(registerFail)
registerSuccess := prometheus.NewCounter(
prometheus.CounterOpts{
Name: "tunnel_register_success",
Help: "Count of successful tunnel registrations",
})
prometheus.MustRegister(registerSuccess)
return &TunnelMetrics{
haConnections: haConnections,
totalRequests: totalRequests,
@@ -355,6 +385,10 @@ func NewTunnelMetrics() *TunnelMetrics {
serverLocations: serverLocations,
oldServerLocations: make(map[string]string),
muxerMetrics: newMuxerMetrics(),
tunnelsHA: NewTunnelsForHA(),
regSuccess: registerSuccess,
regFail: registerFail,
rpcFail: rpcFail,
}
}
@@ -375,7 +409,7 @@ func (t *TunnelMetrics) incrementRequests(connectionID string) {
var concurrentRequests uint64
var ok bool
if concurrentRequests, ok = t.concurrentRequests[connectionID]; ok {
t.concurrentRequests[connectionID] += 1
t.concurrentRequests[connectionID]++
concurrentRequests++
} else {
t.concurrentRequests[connectionID] = 1
@@ -395,7 +429,7 @@ func (t *TunnelMetrics) incrementRequests(connectionID string) {
func (t *TunnelMetrics) decrementConcurrentRequests(connectionID string) {
t.concurrentRequestsLock.Lock()
if _, ok := t.concurrentRequests[connectionID]; ok {
t.concurrentRequests[connectionID] -= 1
t.concurrentRequests[connectionID]--
}
t.concurrentRequestsLock.Unlock()
+24 -20
View File
@@ -1,11 +1,14 @@
package origin
import (
"context"
"fmt"
"net"
"time"
"golang.org/x/net/context"
"github.com/cloudflare/cloudflared/signal"
"github.com/google/uuid"
)
const (
@@ -50,9 +53,9 @@ func NewSupervisor(config *TunnelConfig) *Supervisor {
}
}
func (s *Supervisor) Run(ctx context.Context, connectedSignal chan struct{}) error {
func (s *Supervisor) Run(ctx context.Context, connectedSignal *signal.Signal, u uuid.UUID) error {
logger := s.config.Logger
if err := s.initialize(ctx, connectedSignal); err != nil {
if err := s.initialize(ctx, connectedSignal, u); err != nil {
return err
}
var tunnelsWaiting []int
@@ -95,7 +98,7 @@ func (s *Supervisor) Run(ctx context.Context, connectedSignal chan struct{}) err
case <-backoffTimer:
backoffTimer = nil
for _, index := range tunnelsWaiting {
go s.startTunnel(ctx, index, s.newConnectedTunnelSignal(index))
go s.startTunnel(ctx, index, s.newConnectedTunnelSignal(index), u)
}
tunnelsActive += len(tunnelsWaiting)
tunnelsWaiting = nil
@@ -119,9 +122,9 @@ func (s *Supervisor) Run(ctx context.Context, connectedSignal chan struct{}) err
}
}
func (s *Supervisor) initialize(ctx context.Context, connectedSignal chan struct{}) error {
func (s *Supervisor) initialize(ctx context.Context, connectedSignal *signal.Signal, u uuid.UUID) error {
logger := s.config.Logger
edgeIPs, err := ResolveEdgeIPs(s.config.EdgeAddrs)
edgeIPs, err := ResolveEdgeIPs(logger, s.config.EdgeAddrs)
if err != nil {
logger.Infof("ResolveEdgeIPs err")
return err
@@ -134,7 +137,7 @@ func (s *Supervisor) initialize(ctx context.Context, connectedSignal chan struct
s.lastResolve = time.Now()
// check entitlement and version too old error before attempting to register more tunnels
s.nextUnusedEdgeIP = s.config.HAConnections
go s.startFirstTunnel(ctx, connectedSignal)
go s.startFirstTunnel(ctx, connectedSignal, u)
select {
case <-ctx.Done():
<-s.tunnelErrors
@@ -142,11 +145,12 @@ func (s *Supervisor) initialize(ctx context.Context, connectedSignal chan struct
return fmt.Errorf("context was canceled")
case tunnelError := <-s.tunnelErrors:
return tunnelError.err
case <-connectedSignal:
case <-connectedSignal.Wait():
}
// At least one successful connection, so start the rest
for i := 1; i < s.config.HAConnections; i++ {
go s.startTunnel(ctx, i, make(chan struct{}))
ch := signal.New(make(chan struct{}))
go s.startTunnel(ctx, i, ch, u)
time.Sleep(registrationInterval)
}
return nil
@@ -154,8 +158,8 @@ func (s *Supervisor) initialize(ctx context.Context, connectedSignal chan struct
// startTunnel starts the first tunnel connection. The resulting error will be sent on
// s.tunnelErrors. It will send a signal via connectedSignal if registration succeed
func (s *Supervisor) startFirstTunnel(ctx context.Context, connectedSignal chan struct{}) {
err := ServeTunnelLoop(ctx, s.config, s.getEdgeIP(0), 0, connectedSignal)
func (s *Supervisor) startFirstTunnel(ctx context.Context, connectedSignal *signal.Signal, u uuid.UUID) {
err := ServeTunnelLoop(ctx, s.config, s.getEdgeIP(0), 0, connectedSignal, u)
defer func() {
s.tunnelErrors <- tunnelError{index: 0, err: err}
}()
@@ -176,23 +180,23 @@ func (s *Supervisor) startFirstTunnel(ctx context.Context, connectedSignal chan
default:
return
}
err = ServeTunnelLoop(ctx, s.config, s.getEdgeIP(0), 0, connectedSignal)
err = ServeTunnelLoop(ctx, s.config, s.getEdgeIP(0), 0, connectedSignal, u)
}
}
// startTunnel starts a new tunnel connection. The resulting error will be sent on
// s.tunnelErrors.
func (s *Supervisor) startTunnel(ctx context.Context, index int, connectedSignal chan struct{}) {
err := ServeTunnelLoop(ctx, s.config, s.getEdgeIP(index), uint8(index), connectedSignal)
func (s *Supervisor) startTunnel(ctx context.Context, index int, connectedSignal *signal.Signal, u uuid.UUID) {
err := ServeTunnelLoop(ctx, s.config, s.getEdgeIP(index), uint8(index), connectedSignal, u)
s.tunnelErrors <- tunnelError{index: index, err: err}
}
func (s *Supervisor) newConnectedTunnelSignal(index int) chan struct{} {
signal := make(chan struct{})
s.tunnelsConnecting[index] = signal
s.nextConnectedSignal = signal
func (s *Supervisor) newConnectedTunnelSignal(index int) *signal.Signal {
sig := make(chan struct{})
s.tunnelsConnecting[index] = sig
s.nextConnectedSignal = sig
s.nextConnectedIndex = index
return signal
return signal.New(sig)
}
func (s *Supervisor) waitForNextTunnel(index int) bool {
@@ -219,7 +223,7 @@ func (s *Supervisor) refreshEdgeIPs() {
}
s.resolverC = make(chan resolveResult)
go func() {
edgeIPs, err := ResolveEdgeIPs(s.config.EdgeAddrs)
edgeIPs, err := ResolveEdgeIPs(s.config.Logger, s.config.EdgeAddrs)
s.resolverC <- resolveResult{edgeIPs: edgeIPs, err: err}
}()
}
+126 -47
View File
@@ -2,30 +2,32 @@ package origin
import (
"bufio"
"context"
"crypto/tls"
"fmt"
"github.com/google/uuid"
"io"
"net"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
"time"
"golang.org/x/net/context"
"golang.org/x/sync/errgroup"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/signal"
"github.com/cloudflare/cloudflared/tunnelrpc"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/cloudflare/cloudflared/validation"
"github.com/cloudflare/cloudflared/websocket"
raven "github.com/getsentry/raven-go"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
_ "github.com/prometheus/client_golang/prometheus"
log "github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
rpc "zombiezen.com/go/capnproto2/rpc"
)
@@ -37,8 +39,10 @@ const (
)
type TunnelConfig struct {
// OriginUrl may not be used if a user specifies a unix socket.
OriginUrl string
EdgeAddrs []string
OriginUrl string
Hostname string
OriginCert []byte
TlsConfig *tls.Config
@@ -55,7 +59,7 @@ type TunnelConfig struct {
HTTPTransport http.RoundTripper
Metrics *TunnelMetrics
MetricsUpdateFreq time.Duration
ProtocolLogger *log.Logger
TransportLogger *log.Logger
Logger *log.Logger
IsAutoupdated bool
GracePeriod time.Duration
@@ -63,6 +67,9 @@ type TunnelConfig struct {
NoChunkedEncoding bool
WSGI bool
CompressionQuality uint64
IncidentLookup IncidentLookup
CloseConnOnce *sync.Once // Used to close connectedSignal no more than once
IsFreeTunnel bool
}
type dialError struct {
@@ -100,6 +107,11 @@ type clientRegisterTunnelError struct {
cause error
}
func newClientRegisterTunnelError(cause error, counter *prometheus.CounterVec) clientRegisterTunnelError {
counter.WithLabelValues(cause.Error()).Inc()
return clientRegisterTunnelError{cause: cause}
}
func (e clientRegisterTunnelError) Error() string {
return e.cause.Error()
}
@@ -125,21 +137,27 @@ func (c *TunnelConfig) RegistrationOptions(connectionID uint8, OriginLocalIP str
}
}
func StartTunnelDaemon(config *TunnelConfig, shutdownC <-chan struct{}, connectedSignal chan struct{}) error {
func StartTunnelDaemon(config *TunnelConfig, shutdownC <-chan struct{}, connectedSignal *signal.Signal) error {
ctx, cancel := context.WithCancel(context.Background())
go func() {
<-shutdownC
cancel()
}()
u, err := uuid.NewRandom()
if err != nil {
return err
}
// If a user specified negative HAConnections, we will treat it as requesting 1 connection
if config.HAConnections > 1 {
return NewSupervisor(config).Run(ctx, connectedSignal)
return NewSupervisor(config).Run(ctx, connectedSignal, u)
} else {
addrs, err := ResolveEdgeIPs(config.EdgeAddrs)
addrs, err := ResolveEdgeIPs(config.Logger, config.EdgeAddrs)
if err != nil {
return err
}
return ServeTunnelLoop(ctx, config, addrs[0], 0, connectedSignal)
return ServeTunnelLoop(ctx, config, addrs[0], 0, connectedSignal, u)
}
}
@@ -147,23 +165,23 @@ func ServeTunnelLoop(ctx context.Context,
config *TunnelConfig,
addr *net.TCPAddr,
connectionID uint8,
connectedSignal chan struct{},
connectedSignal *signal.Signal,
u uuid.UUID,
) error {
logger := config.Logger
config.Metrics.incrementHaConnections()
defer config.Metrics.decrementHaConnections()
backoff := BackoffHandler{MaxRetries: config.Retries}
// Used to close connectedSignal no more than once
connectedFuse := h2mux.NewBooleanFuse()
go func() {
if connectedFuse.Await() {
close(connectedSignal)
connectedSignal.Notify()
}
}()
// Ensure the above goroutine will terminate if we return without connecting
defer connectedFuse.Fuse(false)
for {
err, recoverable := ServeTunnel(ctx, config, addr, connectionID, connectedFuse, &backoff)
err, recoverable := ServeTunnel(ctx, config, addr, connectionID, connectedFuse, &backoff, u)
if recoverable {
if duration, ok := backoff.GetBackoffDuration(ctx); ok {
logger.Infof("Retrying in %s seconds", duration)
@@ -182,6 +200,7 @@ func ServeTunnel(
connectionID uint8,
connectedFuse *h2mux.BooleanFuse,
backoff *BackoffHandler,
u uuid.UUID,
) (err error, recoverable bool) {
// Treat panics as recoverable errors
defer func() {
@@ -221,7 +240,7 @@ func ServeTunnel(
errGroup, serveCtx := errgroup.WithContext(ctx)
errGroup.Go(func() error {
err := RegisterTunnel(serveCtx, handler.muxer, config, connectionID, originLocalIP)
err := RegisterTunnel(serveCtx, handler.muxer, config, connectionID, originLocalIP, u)
if err == nil {
connectedFuse.Fuse(true)
backoff.SetGracePeriod()
@@ -235,7 +254,7 @@ func ServeTunnel(
select {
case <-serveCtx.Done():
// UnregisterTunnel blocks until the RPC call returns
err := UnregisterTunnel(handler.muxer, config.GracePeriod, config.Logger)
err := UnregisterTunnel(handler.muxer, config.GracePeriod, config.TransportLogger)
handler.muxer.Shutdown()
return err
case <-updateMetricsTickC:
@@ -265,6 +284,9 @@ func ServeTunnel(
logger.WithError(castedErr.cause).Error("Register tunnel error from server side")
// Don't send registration error return from server to Sentry. They are
// logged on server side
if incidents := config.IncidentLookup.ActiveIncidents(); len(incidents) > 0 {
logger.Error(activeIncidentsMsg(incidents))
}
return castedErr.cause, !castedErr.permanent
case clientRegisterTunnelError:
logger.WithError(castedErr.cause).Error("Register tunnel error on client side")
@@ -292,8 +314,15 @@ func IsRPCStreamResponse(headers []h2mux.Header) bool {
return true
}
func RegisterTunnel(ctx context.Context, muxer *h2mux.Muxer, config *TunnelConfig, connectionID uint8, originLocalIP string) error {
config.Logger.Debug("initiating RPC stream to register")
func RegisterTunnel(
ctx context.Context,
muxer *h2mux.Muxer,
config *TunnelConfig,
connectionID uint8,
originLocalIP string,
uuid uuid.UUID,
) error {
config.TransportLogger.Debug("initiating RPC stream to register")
stream, err := muxer.OpenStream([]h2mux.Header{
{Name: ":method", Value: "RPC"},
{Name: ":scheme", Value: "capnp"},
@@ -301,15 +330,15 @@ func RegisterTunnel(ctx context.Context, muxer *h2mux.Muxer, config *TunnelConfi
}, nil)
if err != nil {
// RPC stream open error
return clientRegisterTunnelError{cause: err}
return newClientRegisterTunnelError(err, config.Metrics.rpcFail)
}
if !IsRPCStreamResponse(stream.Headers) {
// stream response error
return clientRegisterTunnelError{cause: err}
return newClientRegisterTunnelError(err, config.Metrics.rpcFail)
}
conn := rpc.NewConn(
tunnelrpc.NewTransportLogger(config.Logger.WithField("subsystem", "rpc-register"), rpc.StreamTransport(stream)),
tunnelrpc.ConnLog(config.Logger.WithField("subsystem", "rpc-transport")),
tunnelrpc.NewTransportLogger(config.TransportLogger.WithField("subsystem", "rpc-register"), rpc.StreamTransport(stream)),
tunnelrpc.ConnLog(config.TransportLogger.WithField("subsystem", "rpc-transport")),
)
defer conn.Close()
ts := tunnelpogs.TunnelServer_PogsClient{Client: conn.Bootstrap(ctx)}
@@ -318,10 +347,6 @@ func RegisterTunnel(ctx context.Context, muxer *h2mux.Muxer, config *TunnelConfi
serverInfoPromise := tsClient.GetServerInfo(ctx, func(tunnelrpc.TunnelServer_getServerInfo_Params) error {
return nil
})
uuid, err := uuid.NewRandom()
if err != nil {
return clientRegisterTunnelError{cause: err}
}
registration, err := ts.RegisterTunnel(
ctx,
config.OriginCert,
@@ -331,28 +356,30 @@ func RegisterTunnel(ctx context.Context, muxer *h2mux.Muxer, config *TunnelConfi
LogServerInfo(serverInfoPromise.Result(), connectionID, config.Metrics, config.Logger)
if err != nil {
// RegisterTunnel RPC failure
return clientRegisterTunnelError{cause: err}
return newClientRegisterTunnelError(err, config.Metrics.regFail)
}
for _, logLine := range registration.LogLines {
config.Logger.Info(logLine)
}
if registration.Err == DuplicateConnectionError {
return dupConnRegisterTunnelError{}
} else if registration.Err != "" {
return serverRegisterTunnelError{
cause: fmt.Errorf("Server error: %s", registration.Err),
permanent: registration.PermanentFailure,
}
if regErr := processRegisterTunnelError(registration.Err, registration.PermanentFailure, config.Metrics); regErr != nil {
return regErr
}
if registration.TunnelID != "" {
config.Logger.Info("Tunnel ID: " + registration.TunnelID)
config.Metrics.tunnelsHA.AddTunnelID(connectionID, registration.TunnelID)
config.Logger.Infof("Each HA connection's tunnel IDs: %v", config.Metrics.tunnelsHA.String())
}
// Print out the user's trial zone URL in a nice box (if they requested and got one)
if isTrialTunnel := config.Hostname == "" && registration.Url != ""; isTrialTunnel {
for _, line := range asciiBox(trialZoneMsg(registration.Url), 2) {
config.Logger.Infoln(line)
if isTrialTunnel := config.Hostname == ""; isTrialTunnel {
if url, err := url.Parse(registration.Url); err == nil {
for _, line := range asciiBox(trialZoneMsg(url.String()), 2) {
config.Logger.Infoln(line)
}
} else {
config.Logger.Errorln("Failed to connect tunnel, please try again.")
return fmt.Errorf("empty URL in response from Cloudflare edge")
}
}
@@ -360,6 +387,22 @@ func RegisterTunnel(ctx context.Context, muxer *h2mux.Muxer, config *TunnelConfi
return nil
}
func processRegisterTunnelError(err string, permanentFailure bool, metrics *TunnelMetrics) error {
if err == "" {
metrics.regSuccess.Inc()
return nil
}
metrics.regFail.WithLabelValues(err).Inc()
if err == DuplicateConnectionError {
return dupConnRegisterTunnelError{}
}
return serverRegisterTunnelError{
cause: fmt.Errorf("Server error: %s", err),
permanent: permanentFailure,
}
}
func UnregisterTunnel(muxer *h2mux.Muxer, gracePeriod time.Duration, logger *log.Logger) error {
logger.Debug("initiating RPC stream to unregister")
stream, err := muxer.OpenStream([]h2mux.Header{
@@ -426,6 +469,12 @@ func H2RequestHeadersToH1Request(h2 []h2mux.Header, h1 *http.Request) error {
return fmt.Errorf("invalid path")
}
h1.URL = resolved
case "content-length":
contentLength, err := strconv.ParseInt(header.Value, 10, 64)
if err != nil {
return fmt.Errorf("unparseable content length")
}
h1.ContentLength = contentLength
default:
h1.Header.Add(http.CanonicalHeaderKey(header.Name), header.Value)
}
@@ -509,7 +558,7 @@ func NewTunnelHandler(ctx context.Context,
IsClient: true,
HeartbeatInterval: config.HeartbeatInterval,
MaxHeartbeats: config.MaxHeartbeats,
Logger: config.ProtocolLogger.WithFields(log.Fields{}),
Logger: config.TransportLogger.WithFields(log.Fields{}),
CompressionQuality: h2mux.CompressionSetting(config.CompressionQuality),
})
if err != nil {
@@ -614,25 +663,41 @@ func (h *TunnelHandler) logError(stream *h2mux.MuxedStream, err error) {
}
func (h *TunnelHandler) logRequest(req *http.Request, cfRay string, lbProbe bool) {
logger := log.NewEntry(h.logger)
if cfRay != "" {
h.logger.WithField("CF-RAY", cfRay).Debugf("%s %s %s", req.Method, req.URL, req.Proto)
logger = logger.WithField("CF-RAY", cfRay)
logger.Debugf("%s %s %s", req.Method, req.URL, req.Proto)
} else if lbProbe {
h.logger.Debugf("Load Balancer health check %s %s %s", req.Method, req.URL, req.Proto)
logger.Debugf("Load Balancer health check %s %s %s", req.Method, req.URL, req.Proto)
} else {
h.logger.Warnf("All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. %s %s %s ", req.Method, req.URL, req.Proto)
logger.Warnf("All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. %s %s %s ", req.Method, req.URL, req.Proto)
}
logger.Debugf("Request Headers %+v", req.Header)
if contentLen := req.ContentLength; contentLen == -1 {
logger.Debugf("Request Content length unknown")
} else {
logger.Debugf("Request content length %d", contentLen)
}
h.logger.Debugf("Request Headers %+v", req.Header)
}
func (h *TunnelHandler) logResponse(r *http.Response, cfRay string, lbProbe bool) {
logger := log.NewEntry(h.logger)
if cfRay != "" {
h.logger.WithField("CF-RAY", cfRay).Debugf("%s", r.Status)
logger = logger.WithField("CF-RAY", cfRay)
logger.Debugf("%s", r.Status)
} else if lbProbe {
h.logger.Debugf("Response to Load Balancer health check %s", r.Status)
logger.Debugf("Response to Load Balancer health check %s", r.Status)
} else {
h.logger.Infof("%s", r.Status)
logger.Infof("%s", r.Status)
}
logger.Debugf("Response Headers %+v", r.Header)
if contentLen := r.ContentLength; contentLen == -1 {
logger.Debugf("Response content length unknown")
} else {
logger.Debugf("Response content length %d", contentLen)
}
h.logger.Debugf("Response Headers %+v", r.Header)
}
func (h *TunnelHandler) UpdateMetrics(connectionID string) {
@@ -678,3 +743,17 @@ func trialZoneMsg(url string) []string {
" " + url,
}
}
func activeIncidentsMsg(incidents []Incident) string {
preamble := "There is an active Cloudflare incident that may be related:"
if len(incidents) > 1 {
preamble = "There are active Cloudflare incidents that may be related:"
}
incidentStrings := []string{}
for _, incident := range incidents {
incidentString := fmt.Sprintf("%s (%s)", incident.Name, incident.URL())
incidentStrings = append(incidentStrings, incidentString)
}
return preamble + " " + strings.Join(incidentStrings, "; ")
}
+50
View File
@@ -0,0 +1,50 @@
package origin
import (
"fmt"
"sync"
"github.com/prometheus/client_golang/prometheus"
)
// tunnelsForHA maps this cloudflared instance's HA connections to the tunnel IDs they serve.
type tunnelsForHA struct {
sync.Mutex
metrics *prometheus.GaugeVec
entries map[uint8]string
}
// NewTunnelsForHA initializes the Prometheus metrics etc for a tunnelsForHA.
func NewTunnelsForHA() tunnelsForHA {
metrics := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "tunnel_ids",
Help: "The ID of all tunnels (and their corresponding HA connection ID) running in this instance of cloudflared.",
},
[]string{"tunnel_id", "ha_conn_id"},
)
prometheus.MustRegister(metrics)
return tunnelsForHA{
metrics: metrics,
entries: make(map[uint8]string),
}
}
// Track a new tunnel ID, removing the disconnected tunnel (if any) and update metrics.
func (t *tunnelsForHA) AddTunnelID(haConn uint8, tunnelID string) {
t.Lock()
defer t.Unlock()
haStr := fmt.Sprintf("%v", haConn)
if oldTunnelID, ok := t.entries[haConn]; ok {
t.metrics.WithLabelValues(oldTunnelID, haStr).Dec()
}
t.entries[haConn] = tunnelID
t.metrics.WithLabelValues(tunnelID, haStr).Inc()
}
func (t *tunnelsForHA) String() string {
t.Lock()
defer t.Unlock()
return fmt.Sprintf("%v", t.entries)
}
+33
View File
@@ -0,0 +1,33 @@
package signal
import (
"sync"
)
// Signal lets goroutines signal that some event has occurred. Other goroutines can wait for the signal.
type Signal struct {
ch chan struct{}
once sync.Once
}
// New wraps a channel and turns it into a signal for a one-time event.
func New(ch chan struct{}) *Signal {
return &Signal{
ch: ch,
once: sync.Once{},
}
}
// Notify alerts any goroutines waiting on this signal that the event has occurred.
// After the first call to Notify(), future calls are no-op.
func (s *Signal) Notify() {
s.once.Do(func() {
close(s.ch)
})
}
// Wait returns a channel which will be written to when Notify() is called for the first time.
// This channel will never be written to a second time.
func (s *Signal) Wait() <-chan struct{} {
return s.ch
}
+25
View File
@@ -0,0 +1,25 @@
package signal
import (
"testing"
)
func TestMultiNotifyDoesntCrash(t *testing.T) {
sig := New(make(chan struct{}))
sig.Notify()
sig.Notify()
// If code has reached here without crashing, the test has passed.
}
func TestWait(t *testing.T) {
sig := New(make(chan struct{}))
sig.Notify()
select {
case <-sig.Wait():
// Test succeeds
return
default:
// sig.Wait() should have been read from, because sig.Notify() wrote to it.
t.Fail()
}
}
+10 -19
View File
@@ -2,14 +2,10 @@ package tlsconfig
import (
"crypto/tls"
"errors"
"fmt"
"sync"
tunnellog "github.com/cloudflare/cloudflared/log"
"github.com/getsentry/raven-go"
log "github.com/sirupsen/logrus"
"gopkg.in/urfave/cli.v2"
)
// CertReloader can load and reload a TLS certificate from a particular filepath.
@@ -21,18 +17,14 @@ type CertReloader struct {
keyPath string
}
// NewCertReloader makes a CertReloader, memorizing the filepaths in the context/flags.
func NewCertReloader(c *cli.Context, f CLIFlags) (*CertReloader, error) {
if !c.IsSet(f.Cert) {
return nil, errors.New("CertReloader: cert not provided")
}
if !c.IsSet(f.Key) {
return nil, errors.New("CertReloader: key not provided")
}
// NewCertReloader makes a CertReloader. It loads the cert during initialization to make sure certPath and keyPath are valid
func NewCertReloader(certPath, keyPath string) (*CertReloader, error) {
cr := new(CertReloader)
cr.certPath = c.String(f.Cert)
cr.keyPath = c.String(f.Key)
cr.LoadCert()
cr.certPath = certPath
cr.keyPath = keyPath
if err := cr.LoadCert(); err != nil {
return nil, err
}
return cr, nil
}
@@ -45,18 +37,17 @@ func (cr *CertReloader) Cert(clientHello *tls.ClientHelloInfo) (*tls.Certificate
// LoadCert loads a TLS certificate from the CertReloader's specified filepath.
// Call this after writing a new certificate to the disk (e.g. after renewing a certificate)
func (cr *CertReloader) LoadCert() {
func (cr *CertReloader) LoadCert() error {
cr.Lock()
defer cr.Unlock()
log.SetFormatter(&tunnellog.JSONFormatter{})
log.Info("Reloading certificate")
cert, err := tls.LoadX509KeyPair(cr.certPath, cr.keyPath)
// Keep the old certificate if there's a problem reading the new one.
if err != nil {
raven.CaptureError(fmt.Errorf("Error parsing X509 key pair: %v", err), nil)
return
return err
}
cr.certificate = &cert
return nil
}
+22 -6
View File
@@ -2,6 +2,7 @@ package tlsconfig
import (
"crypto/x509"
"encoding/pem"
)
// TODO: remove the Origin CA root certs when migrated to Authenticated Origin Pull certs
@@ -85,11 +86,26 @@ QzMmZpRpIBB321ZBlcnlxiTJvWxvbCPHKHj20VwwAz7LONF59s84ZsOqfoBv8gKM
s0s5dsq5zpLeaw==
-----END CERTIFICATE-----`)
func GetCloudflareRootCA() *x509.CertPool {
ca := x509.NewCertPool()
if !ca.AppendCertsFromPEM([]byte(cloudflareRootCA)) {
// should never happen
panic("failure loading Cloudflare origin CA pem")
func GetCloudflareRootCA() ([]*x509.Certificate, error) {
var certs []*x509.Certificate
pemBlocks := cloudflareRootCA
for len(pemBlocks) > 0 {
var block *pem.Block
block, pemBlocks = pem.Decode(pemBlocks)
if block == nil {
break
}
if block.Type != "CERTIFICATE" {
continue
}
cert, err := x509.ParseCertificate(block.Bytes)
if err != nil {
return nil, err
}
certs = append(certs, cert)
}
return ca
return certs, nil
}
+13
View File
@@ -0,0 +1,13 @@
-----BEGIN CERTIFICATE-----
MIICBjCCAbCgAwIBAgIJAPKk4bYMrSFMMA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV
BAYTAlVTMQ4wDAYDVQQIDAVUZXhhczEPMA0GA1UEBwwGQXVzdGluMRkwFwYDVQQK
DBBDbG91ZGZsYXJlLCBJbmMuMRIwEAYDVQQDDAlsb2NhbGhvc3QwHhcNMTgxMTE1
MjA1NzU3WhcNMjgxMTEyMjA1NzU3WjBdMQswCQYDVQQGEwJVUzEOMAwGA1UECAwF
VGV4YXMxDzANBgNVBAcMBkF1c3RpbjEZMBcGA1UECgwQQ2xvdWRmbGFyZSwgSW5j
LjESMBAGA1UEAwwJbG9jYWxob3N0MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAOQN
pTRn5wLf8SSI5x2kpDvbdDy7lfhamJ2En4Q+wy1cSKp8bn8/oyhVF7QTsimDGTI4
45pV9nDfNJPYB3IW0x0CAwEAAaNTMFEwHQYDVR0OBBYEFE4jIa97mIEiYFa02X++
uu5mCEn+MB8GA1UdIwQYMBaAFE4jIa97mIEiYFa02X++uu5mCEn+MA8GA1UdEwEB
/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADQQAkE+pDee0o5cNcZRUszy8sTQzB1Wlp
J6ucfmo16crqRaK7uGvhkMyibIc4D8z2Cxw3aI3IMMFoIIlYoYKiUcbd
-----END CERTIFICATE-----
+13
View File
@@ -0,0 +1,13 @@
-----BEGIN CERTIFICATE-----
MIICBjCCAbCgAwIBAgIJAN6cXRTbJtFnMA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV
BAYTAlVTMQ4wDAYDVQQIDAVUZXhhczEPMA0GA1UEBwwGQXVzdGluMRgwFgYDVQQK
DA9DbG91ZGZsYXJlLCBJbmMxEzARBgNVBAMMCmxvY2FsaG9zdDIwHhcNMTgxMTE1
MjExMTU4WhcNMjgxMTEyMjExMTU4WjBdMQswCQYDVQQGEwJVUzEOMAwGA1UECAwF
VGV4YXMxDzANBgNVBAcMBkF1c3RpbjEYMBYGA1UECgwPQ2xvdWRmbGFyZSwgSW5j
MRMwEQYDVQQDDApsb2NhbGhvc3QyMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAKQx
IMZ6QgXoul2ITF/7sly4fW2Ol+a/AYw42zCWhVqOXv8AhY21I0Q8lkRR6wOroQwZ
O7jKKOcE5TnR/NRcZr8CAwEAAaNTMFEwHQYDVR0OBBYEFONKxLZc2RUD0KTHkAz4
8nrb5688MB8GA1UdIwQYMBaAFONKxLZc2RUD0KTHkAz48nrb5688MA8GA1UdEwEB
/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADQQA56pwhvGpNPjyLcWfJHu/vI3ZjdoLB
LnrkRaMjJmv0H0Beh4upJhoz8u6lhMACerKQrrdQhEPB2u+maFrEBtmN
-----END CERTIFICATE-----
+10
View File
@@ -0,0 +1,10 @@
-----BEGIN PRIVATE KEY-----
MIIBVQIBADANBgkqhkiG9w0BAQEFAASCAT8wggE7AgEAAkEA5A2lNGfnAt/xJIjn
HaSkO9t0PLuV+FqYnYSfhD7DLVxIqnxufz+jKFUXtBOyKYMZMjjjmlX2cN80k9gH
chbTHQIDAQABAkAoeDtu91lJa1AxuZG58vOqI6GW/Xr5naojmdts7m5YaAhDa7DE
zJUp4d8SP5cGBf1/PB3x6Cu9UviFNQ16wmzJAiEA8gUm4UYpWZD4Ze2l/xb+BK8D
IglSUIy1VxW+X1G55wMCIQDxOfXiFzPqnv/e5avKGv6CU11Dhmbi1OpiyybZTjGz
XwIhAM3bE/cJdqJ4bNBGE6umIupY8pFA3IMnLBempwbsvPOBAiEAgzJ+5OSxu92W
VGidsmJUIhWtF9i1hJFAmVLcYjwBFAkCICtjP/vv0qOZWk4mAAn2zz9UVWp45DSR
p/FA8V77ohXD
-----END PRIVATE KEY-----
+58 -122
View File
@@ -6,150 +6,86 @@ import (
"crypto/tls"
"crypto/x509"
"io/ioutil"
"net"
"github.com/cloudflare/cloudflared/log"
"github.com/pkg/errors"
"gopkg.in/urfave/cli.v2"
"runtime"
)
var logger = log.CreateLogger()
// CLIFlags names the flags used to configure TLS for a command or subsystem.
// The nil value for a field means the flag is ignored.
type CLIFlags struct {
Cert string
Key string
ClientCert string
RootCA string
// Config is the user provided parameters to create a tls.Config
type TLSParameters struct {
Cert string
Key string
GetCertificate *CertReloader
ClientCAs []string
RootCAs []string
ServerName string
CurvePreferences []tls.CurveID
}
// GetConfig returns a TLS configuration according to the flags defined in f and
// set by the user.
func (f CLIFlags) GetConfig(c *cli.Context) *tls.Config {
config := &tls.Config{}
if c.IsSet(f.Cert) && c.IsSet(f.Key) {
cert, err := tls.LoadX509KeyPair(c.String(f.Cert), c.String(f.Key))
// GetConfig returns a TLS configuration according to the Config set by the user.
func GetConfig(p *TLSParameters) (*tls.Config, error) {
tlsconfig := &tls.Config{}
if p.Cert != "" && p.Key != "" {
cert, err := tls.LoadX509KeyPair(p.Cert, p.Key)
if err != nil {
logger.WithError(err).Fatal("Error parsing X509 key pair")
return nil, errors.Wrap(err, "Error parsing X509 key pair")
}
config.Certificates = []tls.Certificate{cert}
config.BuildNameToCertificate()
tlsconfig.Certificates = []tls.Certificate{cert}
// BuildNameToCertificate parses Certificates and builds NameToCertificate from common name
// and SAN fields of leaf certificates
tlsconfig.BuildNameToCertificate()
}
return f.finishGettingConfig(c, config)
}
func (f CLIFlags) GetConfigReloadableCert(c *cli.Context, cr *CertReloader) *tls.Config {
config := &tls.Config{
GetCertificate: cr.Cert,
if p.GetCertificate != nil {
// GetCertificate is called when client supplies SNI info or Certificates is empty.
// Order of retrieving certificate is GetCertificate, NameToCertificate and lastly first element of Certificates
tlsconfig.GetCertificate = p.GetCertificate.Cert
}
config.BuildNameToCertificate()
return f.finishGettingConfig(c, config)
}
func (f CLIFlags) finishGettingConfig(c *cli.Context, config *tls.Config) *tls.Config {
if c.IsSet(f.ClientCert) {
if len(p.ClientCAs) > 0 {
// set of root certificate authorities that servers use if required to verify a client certificate
// by the policy in ClientAuth
config.ClientCAs = LoadCert(c.String(f.ClientCert))
clientCAs, err := LoadCert(p.ClientCAs)
if err != nil {
return nil, errors.Wrap(err, "Error loading client CAs")
}
tlsconfig.ClientCAs = clientCAs
// server's policy for TLS Client Authentication. Default is no client cert
config.ClientAuth = tls.RequireAndVerifyClientCert
tlsconfig.ClientAuth = tls.RequireAndVerifyClientCert
}
// set of root certificate authorities that clients use when verifying server certificates
if c.IsSet(f.RootCA) {
config.RootCAs = LoadCert(c.String(f.RootCA))
if len(p.RootCAs) > 0 {
rootCAs, err := LoadCert(p.RootCAs)
if err != nil {
return nil, errors.Wrap(err, "Error loading root CAs")
}
tlsconfig.RootCAs = rootCAs
}
// we optimize CurveP256
config.CurvePreferences = []tls.CurveID{tls.CurveP256}
return config
if p.ServerName != "" {
tlsconfig.ServerName = p.ServerName
}
if len(p.CurvePreferences) > 0 {
tlsconfig.CurvePreferences = p.CurvePreferences
} else {
// Cloudflare optimize CurveP256
tlsconfig.CurvePreferences = []tls.CurveID{tls.CurveP256}
}
return tlsconfig, nil
}
// LoadCert creates a CertPool containing all certificates in a PEM-format file.
func LoadCert(certPath string) *x509.CertPool {
caCert, err := ioutil.ReadFile(certPath)
if err != nil {
logger.WithError(err).Fatalf("Error reading certificate %s", certPath)
}
func LoadCert(certPaths []string) (*x509.CertPool, error) {
ca := x509.NewCertPool()
if !ca.AppendCertsFromPEM(caCert) {
logger.WithError(err).Fatalf("Error parsing certificate %s", certPath)
}
return ca
}
func LoadGlobalCertPool() (*x509.CertPool, error) {
success := false
// First, obtain the system certificate pool
certPool, systemCertPoolErr := x509.SystemCertPool()
if systemCertPoolErr != nil {
if runtime.GOOS != "windows" {
logger.Warnf("error obtaining the system certificates: %s", systemCertPoolErr)
for _, certPath := range certPaths {
caCert, err := ioutil.ReadFile(certPath)
if err != nil {
return nil, errors.Wrapf(err, "Error reading certificate %s", certPath)
}
certPool = x509.NewCertPool()
} else {
success = true
}
// Next, append the Cloudflare CA pool into the system pool
if !certPool.AppendCertsFromPEM(cloudflareRootCA) {
logger.Warn("could not append the CF certificate to the cloudflared certificate pool")
} else {
success = true
}
if success != true { // Obtaining any of the CAs has failed; this is a fatal error
return nil, errors.New("error loading any of the CAs into the global certificate pool")
}
// Finally, add the Hello certificate into the pool (since it's self-signed)
helloCertificate, err := GetHelloCertificateX509()
if err != nil {
logger.Warn("error obtaining the Hello server certificate")
}
certPool.AddCert(helloCertificate)
return certPool, nil
}
func LoadOriginCertPool(originCAPoolPEM []byte) (*x509.CertPool, error) {
success := false
// Get the global pool
certPool, globalPoolErr := LoadGlobalCertPool()
if globalPoolErr != nil {
certPool = x509.NewCertPool()
} else {
success = true
}
// Then, add any custom origin CA pool the user may have passed
if originCAPoolPEM != nil {
if !certPool.AppendCertsFromPEM(originCAPoolPEM) {
logger.Warn("could not append the provided origin CA to the cloudflared certificate pool")
} else {
success = true
if !ca.AppendCertsFromPEM(caCert) {
return nil, errors.Wrapf(err, "Error parsing certificate %s", certPath)
}
}
if success != true {
return nil, errors.New("error loading any of the CAs into the origin certificate pool")
}
return certPool, nil
}
func CreateTunnelConfig(c *cli.Context, addrs []string) *tls.Config {
tlsConfig := CLIFlags{RootCA: "cacert"}.GetConfig(c)
if tlsConfig.RootCAs == nil {
tlsConfig.RootCAs = GetCloudflareRootCA()
tlsConfig.ServerName = "cftunnel.com"
} else if len(addrs) > 0 {
// Set for development environments and for testing specific origintunneld instances
tlsConfig.ServerName, _, _ = net.SplitHostPort(addrs[0])
}
return tlsConfig
return ca, nil
}
+58 -188
View File
@@ -1,214 +1,84 @@
// +build ignore
// TODO: Remove the above build tag and include this test when we start compiling with Golang 1.10.0+
package tlsconfig
import (
"crypto/x509"
"crypto/x509/pkix"
"encoding/asn1"
"os"
"crypto/tls"
"testing"
"github.com/stretchr/testify/assert"
)
// Generated using `openssl req -newkey rsa:512 -nodes -x509 -days 3650`
var samplePEM = []byte(`
-----BEGIN CERTIFICATE-----
MIIB4DCCAYoCCQCb/H0EUrdXEjANBgkqhkiG9w0BAQsFADB3MQswCQYDVQQGEwJV
UzEOMAwGA1UECAwFVGV4YXMxDzANBgNVBAcMBkF1c3RpbjEZMBcGA1UECgwQQ2xv
dWRmbGFyZSwgSW5jLjEZMBcGA1UECwwQUHJvZHVjdCBTdHJhdGVneTERMA8GA1UE
AwwIVGVzdCBPbmUwHhcNMTgwNDI2MTYxMDUxWhcNMjgwNDIzMTYxMDUxWjB3MQsw
CQYDVQQGEwJVUzEOMAwGA1UECAwFVGV4YXMxDzANBgNVBAcMBkF1c3RpbjEZMBcG
A1UECgwQQ2xvdWRmbGFyZSwgSW5jLjEZMBcGA1UECwwQUHJvZHVjdCBTdHJhdGVn
eTERMA8GA1UEAwwIVGVzdCBPbmUwXDANBgkqhkiG9w0BAQEFAANLADBIAkEAwVQD
K0SJ25UFLznm2pU3zhzMEvpDEofHVNnCjk4mlDrtVop7PkKZ8pDEmuQANltUrxC8
yHBE2wXMv+GlH+bDtwIDAQABMA0GCSqGSIb3DQEBCwUAA0EAjVYQzozIFPkt/HRY
uUoZ8zEHIDICb0syFf5VAjm9AgTwIPzUmD+c5vl6LWDnxq7L45nLCzhhQ6YmiwDz
X7Wcyg==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIB4DCCAYoCCQDZfCdAJ+mwzDANBgkqhkiG9w0BAQsFADB3MQswCQYDVQQGEwJV
UzEOMAwGA1UECAwFVGV4YXMxDzANBgNVBAcMBkF1c3RpbjEZMBcGA1UECgwQQ2xv
dWRmbGFyZSwgSW5jLjEZMBcGA1UECwwQUHJvZHVjdCBTdHJhdGVneTERMA8GA1UE
AwwIVGVzdCBUd28wHhcNMTgwNDI2MTYxMTIwWhcNMjgwNDIzMTYxMTIwWjB3MQsw
CQYDVQQGEwJVUzEOMAwGA1UECAwFVGV4YXMxDzANBgNVBAcMBkF1c3RpbjEZMBcG
A1UECgwQQ2xvdWRmbGFyZSwgSW5jLjEZMBcGA1UECwwQUHJvZHVjdCBTdHJhdGVn
eTERMA8GA1UEAwwIVGVzdCBUd28wXDANBgkqhkiG9w0BAQEFAANLADBIAkEAoHKp
ROVK3zCSsH7ocYeyRAML4V7SFAbZcb4WIwDnE08oMBVRkQVcW5tqEkvG3RiClfzV
wZIJ3CfqKIeSNSDU9wIDAQABMA0GCSqGSIb3DQEBCwUAA0EAJw2gUbnPiq4C2p5b
iWzlA9Q7aKo+VQ4H7IZS7tTccr59nVjvH/TG3eWujpnocr4TOqW9M3CK1DF9mUGP
3pQ3Jg==
-----END CERTIFICATE-----
`)
// testcert.pem and testcert2.pem are Generated using `openssl req -newkey rsa:512 -nodes -x509 -days 3650`
const (
testcertCommonName = "localhost"
)
var systemCertPoolSubjects []*pkix.Name
func TestGetFromEmptyConfig(t *testing.T) {
c := &TLSParameters{}
type certificateFixture struct {
ou string
cn string
tlsConfig, err := GetConfig(c)
assert.NoError(t, err)
assert.Empty(t, tlsConfig.Certificates)
assert.Empty(t, tlsConfig.NameToCertificate)
assert.Nil(t, tlsConfig.ClientCAs)
assert.Equal(t, tls.NoClientCert, tlsConfig.ClientAuth)
assert.Nil(t, tlsConfig.RootCAs)
assert.Len(t, tlsConfig.CurvePreferences, 1)
assert.Equal(t, tls.CurveP256, tlsConfig.CurvePreferences[0])
}
func TestMain(m *testing.M) {
systemCertPool, err := x509.SystemCertPool()
if isUnrecoverableError(err) {
os.Exit(1)
}
func TestGetConfig(t *testing.T) {
cert, err := tls.LoadX509KeyPair("testcert.pem", "testkey.pem")
assert.NoError(t, err)
if systemCertPool == nil {
// On Windows, let's just assume the system cert pool was empty
systemCertPool = x509.NewCertPool()
c := &TLSParameters{
Cert: "testcert.pem",
Key: "testkey.pem",
ClientCAs: []string{"testcert.pem", "testcert2.pem"},
RootCAs: []string{"testcert.pem", "testcert2.pem"},
ServerName: "test",
CurvePreferences: []tls.CurveID{tls.CurveP384},
}
tlsConfig, err := GetConfig(c)
assert.NoError(t, err)
assert.Len(t, tlsConfig.Certificates, 1)
assert.Equal(t, cert, tlsConfig.Certificates[0])
systemCertPoolSubjects, err = getCertPoolSubjects(systemCertPool)
if err != nil {
os.Exit(1)
}
assert.Equal(t, cert, *tlsConfig.NameToCertificate[testcertCommonName])
os.Exit(m.Run())
assert.NotNil(t, tlsConfig.ClientCAs)
assert.Equal(t, tls.RequireAndVerifyClientCert, tlsConfig.ClientAuth)
assert.NotNil(t, tlsConfig.RootCAs)
assert.Len(t, tlsConfig.CurvePreferences, 1)
assert.Equal(t, tls.CurveP384, tlsConfig.CurvePreferences[0])
}
func TestLoadOriginCertPoolJustSystemPool(t *testing.T) {
certPoolSubjects := loadCertPoolSubjects(t, nil)
extraSubjects := subjectSubtract(systemCertPoolSubjects, certPoolSubjects)
func TestCertReloader(t *testing.T) {
expectedCert, err := tls.LoadX509KeyPair("testcert.pem", "testkey.pem")
assert.NoError(t, err)
// Remove extra subjects from the cert pool
var filteredSystemCertPoolSubjects []*pkix.Name
certReloader, err := NewCertReloader("testcert.pem", "testkey.pem")
assert.NoError(t, err)
t.Log(extraSubjects)
chi := &tls.ClientHelloInfo{ServerName: testcertCommonName}
cert, err := certReloader.Cert(chi)
assert.NoError(t, err)
assert.Equal(t, expectedCert, *cert)
OUTER:
for _, subject := range certPoolSubjects {
for _, extraSubject := range extraSubjects {
if subject == extraSubject {
t.Log(extraSubject)
continue OUTER
}
}
filteredSystemCertPoolSubjects = append(filteredSystemCertPoolSubjects, subject)
c := &TLSParameters{
GetCertificate: certReloader,
}
tlsConfig, err := GetConfig(c)
assert.NoError(t, err)
assert.Equal(t, len(filteredSystemCertPoolSubjects), len(systemCertPoolSubjects))
difference := subjectSubtract(systemCertPoolSubjects, filteredSystemCertPoolSubjects)
assert.Equal(t, 0, len(difference))
}
func TestLoadOriginCertPoolCFCertificates(t *testing.T) {
certPoolSubjects := loadCertPoolSubjects(t, nil)
extraSubjects := subjectSubtract(systemCertPoolSubjects, certPoolSubjects)
expected := []*certificateFixture{
{ou: "CloudFlare Origin SSL ECC Certificate Authority"},
{ou: "CloudFlare Origin SSL Certificate Authority"},
{cn: "origin-pull.cloudflare.net"},
{cn: "Argo Tunnel Sample Hello Server Certificate"},
}
assertFixturesMatchSubjects(t, expected, extraSubjects)
}
func TestLoadOriginCertPoolWithExtraPEMs(t *testing.T) {
certPoolWithoutPEMSubjects := loadCertPoolSubjects(t, nil)
certPoolWithPEMSubjects := loadCertPoolSubjects(t, samplePEM)
difference := subjectSubtract(certPoolWithoutPEMSubjects, certPoolWithPEMSubjects)
assert.Equal(t, 2, len(difference))
expected := []*certificateFixture{
{cn: "Test One"},
{cn: "Test Two"},
}
assertFixturesMatchSubjects(t, expected, difference)
}
func loadCertPoolSubjects(t *testing.T, originCAPoolPEM []byte) []*pkix.Name {
certPool, err := LoadOriginCertPool(originCAPoolPEM)
if isUnrecoverableError(err) {
t.Fatal(err)
}
assert.NotEmpty(t, certPool.Subjects())
certPoolSubjects, err := getCertPoolSubjects(certPool)
if err != nil {
t.Fatal(err)
}
return certPoolSubjects
}
func assertFixturesMatchSubjects(t *testing.T, fixtures []*certificateFixture, subjects []*pkix.Name) {
assert.Equal(t, len(fixtures), len(subjects))
for _, fixture := range fixtures {
found := false
for _, subject := range subjects {
found = found || fixtureMatchesSubjectPredicate(fixture, subject)
}
if !found {
t.Fail()
}
}
}
func fixtureMatchesSubjectPredicate(fixture *certificateFixture, subject *pkix.Name) bool {
cnMatch := true
if fixture.cn != "" {
cnMatch = fixture.cn == subject.CommonName
}
ouMatch := true
if fixture.ou != "" {
ouMatch = len(subject.OrganizationalUnit) > 0 && fixture.ou == subject.OrganizationalUnit[0]
}
return cnMatch && ouMatch
}
func subjectSubtract(left []*pkix.Name, right []*pkix.Name) []*pkix.Name {
var difference []*pkix.Name
var found bool
for _, r := range right {
found = false
for _, l := range left {
if (*l).String() == (*r).String() {
found = true
}
}
if !found {
difference = append(difference, r)
}
}
return difference
}
func getCertPoolSubjects(certPool *x509.CertPool) ([]*pkix.Name, error) {
var subjects []*pkix.Name
for _, subject := range certPool.Subjects() {
var sequence pkix.RDNSequence
_, err := asn1.Unmarshal(subject, &sequence)
if err != nil {
return nil, err
}
name := pkix.Name{}
name.FillFromRDNSequence(&sequence)
subjects = append(subjects, &name)
}
return subjects, nil
}
func isUnrecoverableError(err error) bool {
return err != nil && err.Error() != "crypto/x509: system root pool is not available on Windows"
cert, err = tlsConfig.GetCertificate(chi)
assert.NoError(t, err)
assert.Equal(t, expectedCert, *cert)
}
+2 -1
View File
@@ -1,10 +1,11 @@
package tunneldns
import (
"context"
"github.com/coredns/coredns/plugin"
"github.com/miekg/dns"
"github.com/pkg/errors"
"golang.org/x/net/context"
)
// Upstream is a simplified interface for proxy destination
+2 -1
View File
@@ -2,6 +2,7 @@ package tunneldns
import (
"bytes"
"context"
"crypto/tls"
"fmt"
"io/ioutil"
@@ -12,7 +13,6 @@ import (
"github.com/miekg/dns"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
"golang.org/x/net/context"
"golang.org/x/net/http2"
)
@@ -39,6 +39,7 @@ func NewUpstreamHTTPS(endpoint string) (Upstream, error) {
TLSClientConfig: tls,
DisableCompression: true,
MaxIdleConns: 1,
Proxy: http.ProxyFromEnvironment,
}
http2.ConfigureTransport(transport)
+2 -1
View File
@@ -1,6 +1,8 @@
package tunneldns
import (
"context"
"github.com/coredns/coredns/plugin"
"github.com/coredns/coredns/plugin/metrics/vars"
"github.com/coredns/coredns/plugin/pkg/dnstest"
@@ -8,7 +10,6 @@ import (
"github.com/coredns/coredns/request"
"github.com/miekg/dns"
"github.com/prometheus/client_golang/prometheus"
"golang.org/x/net/context"
)
// MetricsPlugin is an adapter for CoreDNS and built-in metrics
+2 -1
View File
@@ -1,8 +1,9 @@
package tunnelrpc
import (
"context"
log "github.com/sirupsen/logrus"
"golang.org/x/net/context"
"golang.org/x/net/trace"
"zombiezen.com/go/capnproto2/rpc"
)
+1 -1
View File
@@ -3,9 +3,9 @@ package tunnelrpc
import (
"bytes"
"context"
log "github.com/sirupsen/logrus"
"golang.org/x/net/context"
"zombiezen.com/go/capnproto2/encoding/text"
"zombiezen.com/go/capnproto2/rpc"
rpccapnp "zombiezen.com/go/capnproto2/std/capnp/rpc"
+125
View File
@@ -2,8 +2,10 @@ package pogs
import (
"context"
"time"
"github.com/cloudflare/cloudflared/tunnelrpc"
"github.com/google/uuid"
log "github.com/sirupsen/logrus"
"zombiezen.com/go/capnproto2"
@@ -71,6 +73,41 @@ func UnmarshalRegistrationOptions(s tunnelrpc.RegistrationOptions) (*Registratio
return p, err
}
type ConnectResult struct {
Err *ConnectError
ServerInfo ServerInfo
}
func MarshalConnectResult(s tunnelrpc.ConnectResult, p *ConnectResult) error {
return pogs.Insert(tunnelrpc.ConnectResult_TypeID, s.Struct, p)
}
func UnmarshalConnectResult(s tunnelrpc.ConnectResult) (*ConnectResult, error) {
p := new(ConnectResult)
err := pogs.Extract(p, tunnelrpc.ConnectResult_TypeID, s.Struct)
return p, err
}
type ConnectError struct {
Cause string
RetryAfter time.Duration
ShouldRetry bool
}
func MarshalConnectError(s tunnelrpc.ConnectError, p *ConnectError) error {
return pogs.Insert(tunnelrpc.ConnectError_TypeID, s.Struct, p)
}
func UnmarshalConnectError(s tunnelrpc.ConnectError) (*ConnectError, error) {
p := new(ConnectError)
err := pogs.Extract(p, tunnelrpc.ConnectError_TypeID, s.Struct)
return p, err
}
func (e *ConnectError) Error() string {
return e.Cause
}
type Tag struct {
Name string `json:"name"`
Value string `json:"value"`
@@ -90,10 +127,56 @@ func UnmarshalServerInfo(s tunnelrpc.ServerInfo) (*ServerInfo, error) {
return p, err
}
type ConnectParameters struct {
OriginCert []byte
CloudflaredID uuid.UUID
NumPreviousAttempts uint8
Tags []Tag
}
// CapnpConnectParameters is ConnectParameters represented in Cap'n Proto build-in types
type CapnpConnectParameters struct {
OriginCert []byte
CloudflaredID []byte
NumPreviousAttempts uint8
Tags []Tag
}
func MarshalConnectParameters(s tunnelrpc.CapnpConnectParameters, p *ConnectParameters) error {
cloudflaredIDBytes, err := p.CloudflaredID.MarshalBinary()
if err != nil {
return err
}
capnpConnectParameters := &CapnpConnectParameters{
OriginCert: p.OriginCert,
CloudflaredID: cloudflaredIDBytes,
NumPreviousAttempts: p.NumPreviousAttempts,
}
return pogs.Insert(tunnelrpc.CapnpConnectParameters_TypeID, s.Struct, capnpConnectParameters)
}
func UnmarshalConnectParameters(s tunnelrpc.CapnpConnectParameters) (*ConnectParameters, error) {
p := new(CapnpConnectParameters)
err := pogs.Extract(p, tunnelrpc.CapnpConnectParameters_TypeID, s.Struct)
if err != nil {
return nil, err
}
cloudflaredID, err := uuid.FromBytes(p.CloudflaredID)
if err != nil {
return nil, err
}
return &ConnectParameters{
OriginCert: p.OriginCert,
CloudflaredID: cloudflaredID,
NumPreviousAttempts: p.NumPreviousAttempts,
}, nil
}
type TunnelServer interface {
RegisterTunnel(ctx context.Context, originCert []byte, hostname string, options *RegistrationOptions) (*TunnelRegistration, error)
GetServerInfo(ctx context.Context) (*ServerInfo, error)
UnregisterTunnel(ctx context.Context, gracePeriodNanoSec int64) error
Connect(ctx context.Context, paramaters *ConnectParameters) (*ConnectResult, error)
}
func TunnelServer_ServerToClient(s TunnelServer) tunnelrpc.TunnelServer {
@@ -151,7 +234,27 @@ func (i TunnelServer_PogsImpl) UnregisterTunnel(p tunnelrpc.TunnelServer_unregis
gracePeriodNanoSec := p.Params.GracePeriodNanoSec()
server.Ack(p.Options)
return i.impl.UnregisterTunnel(p.Ctx, gracePeriodNanoSec)
}
func (i TunnelServer_PogsImpl) Connect(p tunnelrpc.TunnelServer_connect) error {
paramaters, err := p.Params.Parameters()
if err != nil {
return err
}
pogsParameters, err := UnmarshalConnectParameters(paramaters)
if err != nil {
return err
}
server.Ack(p.Options)
connectResult, err := i.impl.Connect(p.Ctx, pogsParameters)
if err != nil {
return err
}
result, err := p.Results.NewResult()
if err != nil {
return err
}
return MarshalConnectResult(result, connectResult)
}
type TunnelServer_PogsClient struct {
@@ -212,3 +315,25 @@ func (c TunnelServer_PogsClient) UnregisterTunnel(ctx context.Context, gracePeri
_, err := promise.Struct()
return err
}
func (c TunnelServer_PogsClient) Connect(ctx context.Context,
parameters *ConnectParameters,
) (*ConnectResult, error) {
client := tunnelrpc.TunnelServer{Client: c.Client}
promise := client.Connect(ctx, func(p tunnelrpc.TunnelServer_connect_Params) error {
connectParameters, err := p.NewParameters()
if err != nil {
return err
}
err = MarshalConnectParameters(connectParameters, parameters)
if err != nil {
return err
}
return nil
})
retval, err := promise.Result().Struct()
if err != nil {
return nil, err
}
return UnmarshalConnectResult(retval)
}
+25
View File
@@ -46,6 +46,30 @@ struct RegistrationOptions {
uuid @11 :Text;
}
struct CapnpConnectParameters {
# certificate and token to prove ownership of a zone
originCert @0 :Data;
# UUID assigned to this cloudflared obtained from Hello
cloudflaredID @1 :Data;
# number of previous attempts to send Connect
numPreviousAttempts @2 :UInt8;
# user defined labels for this cloudflared
tags @3 :List(Tag);
}
struct ConnectResult {
err @0 :ConnectError;
# Information about the server this connection is established with
serverInfo @1 :ServerInfo;
}
struct ConnectError {
cause @0 :Text;
# How long should this connection wait to retry in ns
retryAfter @1 :Int64;
shouldRetry @2 :Bool;
}
struct Tag {
name @0 :Text;
value @1 :Text;
@@ -65,4 +89,5 @@ interface TunnelServer {
registerTunnel @0 (originCert :Data, hostname :Text, options :RegistrationOptions) -> (result :TunnelRegistration);
getServerInfo @1 () -> (result :ServerInfo);
unregisterTunnel @2 (gracePeriodNanoSec :Int64) -> ();
connect @3 (parameters :CapnpConnectParameters) -> (result :ConnectResult);
}
+617 -145
View File
@@ -105,11 +105,6 @@ func (s Authentication_List) At(i int) Authentication { return Authentication{s.
func (s Authentication_List) Set(i int, v Authentication) error { return s.List.SetStruct(i, v.Struct) }
func (s Authentication_List) String() string {
str, _ := text.MarshalList(0xc082ef6e0d42ed1d, s.List)
return str
}
// Authentication_Promise is a wrapper for a Authentication promised by a client call.
type Authentication_Promise struct{ *capnp.Pipeline }
@@ -250,11 +245,6 @@ func (s TunnelRegistration_List) Set(i int, v TunnelRegistration) error {
return s.List.SetStruct(i, v.Struct)
}
func (s TunnelRegistration_List) String() string {
str, _ := text.MarshalList(0xf41a0f001ad49e46, s.List)
return str
}
// TunnelRegistration_Promise is a wrapper for a TunnelRegistration promised by a client call.
type TunnelRegistration_Promise struct{ *capnp.Pipeline }
@@ -484,11 +474,6 @@ func (s RegistrationOptions_List) Set(i int, v RegistrationOptions) error {
return s.List.SetStruct(i, v.Struct)
}
func (s RegistrationOptions_List) String() string {
str, _ := text.MarshalList(0xc793e50592935b4a, s.List)
return str
}
// RegistrationOptions_Promise is a wrapper for a RegistrationOptions promised by a client call.
type RegistrationOptions_Promise struct{ *capnp.Pipeline }
@@ -497,6 +482,302 @@ func (p RegistrationOptions_Promise) Struct() (RegistrationOptions, error) {
return RegistrationOptions{s}, err
}
type CapnpConnectParameters struct{ capnp.Struct }
// CapnpConnectParameters_TypeID is the unique identifier for the type CapnpConnectParameters.
const CapnpConnectParameters_TypeID = 0xa78f37418c1077c8
func NewCapnpConnectParameters(s *capnp.Segment) (CapnpConnectParameters, error) {
st, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 8, PointerCount: 3})
return CapnpConnectParameters{st}, err
}
func NewRootCapnpConnectParameters(s *capnp.Segment) (CapnpConnectParameters, error) {
st, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 8, PointerCount: 3})
return CapnpConnectParameters{st}, err
}
func ReadRootCapnpConnectParameters(msg *capnp.Message) (CapnpConnectParameters, error) {
root, err := msg.RootPtr()
return CapnpConnectParameters{root.Struct()}, err
}
func (s CapnpConnectParameters) String() string {
str, _ := text.Marshal(0xa78f37418c1077c8, s.Struct)
return str
}
func (s CapnpConnectParameters) OriginCert() ([]byte, error) {
p, err := s.Struct.Ptr(0)
return []byte(p.Data()), err
}
func (s CapnpConnectParameters) HasOriginCert() bool {
p, err := s.Struct.Ptr(0)
return p.IsValid() || err != nil
}
func (s CapnpConnectParameters) SetOriginCert(v []byte) error {
return s.Struct.SetData(0, v)
}
func (s CapnpConnectParameters) CloudflaredID() ([]byte, error) {
p, err := s.Struct.Ptr(1)
return []byte(p.Data()), err
}
func (s CapnpConnectParameters) HasCloudflaredID() bool {
p, err := s.Struct.Ptr(1)
return p.IsValid() || err != nil
}
func (s CapnpConnectParameters) SetCloudflaredID(v []byte) error {
return s.Struct.SetData(1, v)
}
func (s CapnpConnectParameters) NumPreviousAttempts() uint8 {
return s.Struct.Uint8(0)
}
func (s CapnpConnectParameters) SetNumPreviousAttempts(v uint8) {
s.Struct.SetUint8(0, v)
}
func (s CapnpConnectParameters) Tags() (Tag_List, error) {
p, err := s.Struct.Ptr(2)
return Tag_List{List: p.List()}, err
}
func (s CapnpConnectParameters) HasTags() bool {
p, err := s.Struct.Ptr(2)
return p.IsValid() || err != nil
}
func (s CapnpConnectParameters) SetTags(v Tag_List) error {
return s.Struct.SetPtr(2, v.List.ToPtr())
}
// NewTags sets the tags field to a newly
// allocated Tag_List, preferring placement in s's segment.
func (s CapnpConnectParameters) NewTags(n int32) (Tag_List, error) {
l, err := NewTag_List(s.Struct.Segment(), n)
if err != nil {
return Tag_List{}, err
}
err = s.Struct.SetPtr(2, l.List.ToPtr())
return l, err
}
// CapnpConnectParameters_List is a list of CapnpConnectParameters.
type CapnpConnectParameters_List struct{ capnp.List }
// NewCapnpConnectParameters creates a new list of CapnpConnectParameters.
func NewCapnpConnectParameters_List(s *capnp.Segment, sz int32) (CapnpConnectParameters_List, error) {
l, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 8, PointerCount: 3}, sz)
return CapnpConnectParameters_List{l}, err
}
func (s CapnpConnectParameters_List) At(i int) CapnpConnectParameters {
return CapnpConnectParameters{s.List.Struct(i)}
}
func (s CapnpConnectParameters_List) Set(i int, v CapnpConnectParameters) error {
return s.List.SetStruct(i, v.Struct)
}
// CapnpConnectParameters_Promise is a wrapper for a CapnpConnectParameters promised by a client call.
type CapnpConnectParameters_Promise struct{ *capnp.Pipeline }
func (p CapnpConnectParameters_Promise) Struct() (CapnpConnectParameters, error) {
s, err := p.Pipeline.Struct()
return CapnpConnectParameters{s}, err
}
type ConnectResult struct{ capnp.Struct }
// ConnectResult_TypeID is the unique identifier for the type ConnectResult.
const ConnectResult_TypeID = 0xff8d9848747c956a
func NewConnectResult(s *capnp.Segment) (ConnectResult, error) {
st, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2})
return ConnectResult{st}, err
}
func NewRootConnectResult(s *capnp.Segment) (ConnectResult, error) {
st, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2})
return ConnectResult{st}, err
}
func ReadRootConnectResult(msg *capnp.Message) (ConnectResult, error) {
root, err := msg.RootPtr()
return ConnectResult{root.Struct()}, err
}
func (s ConnectResult) String() string {
str, _ := text.Marshal(0xff8d9848747c956a, s.Struct)
return str
}
func (s ConnectResult) Err() (ConnectError, error) {
p, err := s.Struct.Ptr(0)
return ConnectError{Struct: p.Struct()}, err
}
func (s ConnectResult) HasErr() bool {
p, err := s.Struct.Ptr(0)
return p.IsValid() || err != nil
}
func (s ConnectResult) SetErr(v ConnectError) error {
return s.Struct.SetPtr(0, v.Struct.ToPtr())
}
// NewErr sets the err field to a newly
// allocated ConnectError struct, preferring placement in s's segment.
func (s ConnectResult) NewErr() (ConnectError, error) {
ss, err := NewConnectError(s.Struct.Segment())
if err != nil {
return ConnectError{}, err
}
err = s.Struct.SetPtr(0, ss.Struct.ToPtr())
return ss, err
}
func (s ConnectResult) ServerInfo() (ServerInfo, error) {
p, err := s.Struct.Ptr(1)
return ServerInfo{Struct: p.Struct()}, err
}
func (s ConnectResult) HasServerInfo() bool {
p, err := s.Struct.Ptr(1)
return p.IsValid() || err != nil
}
func (s ConnectResult) SetServerInfo(v ServerInfo) error {
return s.Struct.SetPtr(1, v.Struct.ToPtr())
}
// NewServerInfo sets the serverInfo field to a newly
// allocated ServerInfo struct, preferring placement in s's segment.
func (s ConnectResult) NewServerInfo() (ServerInfo, error) {
ss, err := NewServerInfo(s.Struct.Segment())
if err != nil {
return ServerInfo{}, err
}
err = s.Struct.SetPtr(1, ss.Struct.ToPtr())
return ss, err
}
// ConnectResult_List is a list of ConnectResult.
type ConnectResult_List struct{ capnp.List }
// NewConnectResult creates a new list of ConnectResult.
func NewConnectResult_List(s *capnp.Segment, sz int32) (ConnectResult_List, error) {
l, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2}, sz)
return ConnectResult_List{l}, err
}
func (s ConnectResult_List) At(i int) ConnectResult { return ConnectResult{s.List.Struct(i)} }
func (s ConnectResult_List) Set(i int, v ConnectResult) error { return s.List.SetStruct(i, v.Struct) }
// ConnectResult_Promise is a wrapper for a ConnectResult promised by a client call.
type ConnectResult_Promise struct{ *capnp.Pipeline }
func (p ConnectResult_Promise) Struct() (ConnectResult, error) {
s, err := p.Pipeline.Struct()
return ConnectResult{s}, err
}
func (p ConnectResult_Promise) Err() ConnectError_Promise {
return ConnectError_Promise{Pipeline: p.Pipeline.GetPipeline(0)}
}
func (p ConnectResult_Promise) ServerInfo() ServerInfo_Promise {
return ServerInfo_Promise{Pipeline: p.Pipeline.GetPipeline(1)}
}
type ConnectError struct{ capnp.Struct }
// ConnectError_TypeID is the unique identifier for the type ConnectError.
const ConnectError_TypeID = 0xb14ce48f4e2abb0d
func NewConnectError(s *capnp.Segment) (ConnectError, error) {
st, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 1})
return ConnectError{st}, err
}
func NewRootConnectError(s *capnp.Segment) (ConnectError, error) {
st, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 16, PointerCount: 1})
return ConnectError{st}, err
}
func ReadRootConnectError(msg *capnp.Message) (ConnectError, error) {
root, err := msg.RootPtr()
return ConnectError{root.Struct()}, err
}
func (s ConnectError) String() string {
str, _ := text.Marshal(0xb14ce48f4e2abb0d, s.Struct)
return str
}
func (s ConnectError) Cause() (string, error) {
p, err := s.Struct.Ptr(0)
return p.Text(), err
}
func (s ConnectError) HasCause() bool {
p, err := s.Struct.Ptr(0)
return p.IsValid() || err != nil
}
func (s ConnectError) CauseBytes() ([]byte, error) {
p, err := s.Struct.Ptr(0)
return p.TextBytes(), err
}
func (s ConnectError) SetCause(v string) error {
return s.Struct.SetText(0, v)
}
func (s ConnectError) RetryAfter() int64 {
return int64(s.Struct.Uint64(0))
}
func (s ConnectError) SetRetryAfter(v int64) {
s.Struct.SetUint64(0, uint64(v))
}
func (s ConnectError) ShouldRetry() bool {
return s.Struct.Bit(64)
}
func (s ConnectError) SetShouldRetry(v bool) {
s.Struct.SetBit(64, v)
}
// ConnectError_List is a list of ConnectError.
type ConnectError_List struct{ capnp.List }
// NewConnectError creates a new list of ConnectError.
func NewConnectError_List(s *capnp.Segment, sz int32) (ConnectError_List, error) {
l, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 16, PointerCount: 1}, sz)
return ConnectError_List{l}, err
}
func (s ConnectError_List) At(i int) ConnectError { return ConnectError{s.List.Struct(i)} }
func (s ConnectError_List) Set(i int, v ConnectError) error { return s.List.SetStruct(i, v.Struct) }
// ConnectError_Promise is a wrapper for a ConnectError promised by a client call.
type ConnectError_Promise struct{ *capnp.Pipeline }
func (p ConnectError_Promise) Struct() (ConnectError, error) {
s, err := p.Pipeline.Struct()
return ConnectError{s}, err
}
type Tag struct{ capnp.Struct }
// Tag_TypeID is the unique identifier for the type Tag.
@@ -573,11 +854,6 @@ func (s Tag_List) At(i int) Tag { return Tag{s.List.Struct(i)} }
func (s Tag_List) Set(i int, v Tag) error { return s.List.SetStruct(i, v.Struct) }
func (s Tag_List) String() string {
str, _ := text.MarshalList(0xcbd96442ae3bb01a, s.List)
return str
}
// Tag_Promise is a wrapper for a Tag promised by a client call.
type Tag_Promise struct{ *capnp.Pipeline }
@@ -703,11 +979,6 @@ func (s ServerInfo_List) At(i int) ServerInfo { return ServerInfo{s.List.Struct(
func (s ServerInfo_List) Set(i int, v ServerInfo) error { return s.List.SetStruct(i, v.Struct) }
func (s ServerInfo_List) String() string {
str, _ := text.MarshalList(0xf2c68e2547ec3866, s.List)
return str
}
// ServerInfo_Promise is a wrapper for a ServerInfo promised by a client call.
type ServerInfo_Promise struct{ *capnp.Pipeline }
@@ -781,6 +1052,26 @@ func (c TunnelServer) UnregisterTunnel(ctx context.Context, params func(TunnelSe
}
return TunnelServer_unregisterTunnel_Results_Promise{Pipeline: capnp.NewPipeline(c.Client.Call(call))}
}
func (c TunnelServer) Connect(ctx context.Context, params func(TunnelServer_connect_Params) error, opts ...capnp.CallOption) TunnelServer_connect_Results_Promise {
if c.Client == nil {
return TunnelServer_connect_Results_Promise{Pipeline: capnp.NewPipeline(capnp.ErrorAnswer(capnp.ErrNullClient))}
}
call := &capnp.Call{
Ctx: ctx,
Method: capnp.Method{
InterfaceID: 0xea58385c65416035,
MethodID: 3,
InterfaceName: "tunnelrpc/tunnelrpc.capnp:TunnelServer",
MethodName: "connect",
},
Options: capnp.NewCallOptions(opts),
}
if params != nil {
call.ParamsSize = capnp.ObjectSize{DataSize: 0, PointerCount: 1}
call.ParamsFunc = func(s capnp.Struct) error { return params(TunnelServer_connect_Params{Struct: s}) }
}
return TunnelServer_connect_Results_Promise{Pipeline: capnp.NewPipeline(c.Client.Call(call))}
}
type TunnelServer_Server interface {
RegisterTunnel(TunnelServer_registerTunnel) error
@@ -788,6 +1079,8 @@ type TunnelServer_Server interface {
GetServerInfo(TunnelServer_getServerInfo) error
UnregisterTunnel(TunnelServer_unregisterTunnel) error
Connect(TunnelServer_connect) error
}
func TunnelServer_ServerToClient(s TunnelServer_Server) TunnelServer {
@@ -797,7 +1090,7 @@ func TunnelServer_ServerToClient(s TunnelServer_Server) TunnelServer {
func TunnelServer_Methods(methods []server.Method, s TunnelServer_Server) []server.Method {
if cap(methods) == 0 {
methods = make([]server.Method, 0, 3)
methods = make([]server.Method, 0, 4)
}
methods = append(methods, server.Method{
@@ -842,6 +1135,20 @@ func TunnelServer_Methods(methods []server.Method, s TunnelServer_Server) []serv
ResultsSize: capnp.ObjectSize{DataSize: 0, PointerCount: 0},
})
methods = append(methods, server.Method{
Method: capnp.Method{
InterfaceID: 0xea58385c65416035,
MethodID: 3,
InterfaceName: "tunnelrpc/tunnelrpc.capnp:TunnelServer",
MethodName: "connect",
},
Impl: func(c context.Context, opts capnp.CallOptions, p, r capnp.Struct) error {
call := TunnelServer_connect{c, opts, TunnelServer_connect_Params{Struct: p}, TunnelServer_connect_Results{Struct: r}}
return s.Connect(call)
},
ResultsSize: capnp.ObjectSize{DataSize: 0, PointerCount: 1},
})
return methods
}
@@ -869,6 +1176,14 @@ type TunnelServer_unregisterTunnel struct {
Results TunnelServer_unregisterTunnel_Results
}
// TunnelServer_connect holds the arguments for a server call to TunnelServer.connect.
type TunnelServer_connect struct {
Ctx context.Context
Options capnp.CallOptions
Params TunnelServer_connect_Params
Results TunnelServer_connect_Results
}
type TunnelServer_registerTunnel_Params struct{ capnp.Struct }
// TunnelServer_registerTunnel_Params_TypeID is the unique identifier for the type TunnelServer_registerTunnel_Params.
@@ -969,11 +1284,6 @@ func (s TunnelServer_registerTunnel_Params_List) Set(i int, v TunnelServer_regis
return s.List.SetStruct(i, v.Struct)
}
func (s TunnelServer_registerTunnel_Params_List) String() string {
str, _ := text.MarshalList(0xb70431c0dc014915, s.List)
return str
}
// TunnelServer_registerTunnel_Params_Promise is a wrapper for a TunnelServer_registerTunnel_Params promised by a client call.
type TunnelServer_registerTunnel_Params_Promise struct{ *capnp.Pipeline }
@@ -1053,11 +1363,6 @@ func (s TunnelServer_registerTunnel_Results_List) Set(i int, v TunnelServer_regi
return s.List.SetStruct(i, v.Struct)
}
func (s TunnelServer_registerTunnel_Results_List) String() string {
str, _ := text.MarshalList(0xf2c122394f447e8e, s.List)
return str
}
// TunnelServer_registerTunnel_Results_Promise is a wrapper for a TunnelServer_registerTunnel_Results promised by a client call.
type TunnelServer_registerTunnel_Results_Promise struct{ *capnp.Pipeline }
@@ -1112,11 +1417,6 @@ func (s TunnelServer_getServerInfo_Params_List) Set(i int, v TunnelServer_getSer
return s.List.SetStruct(i, v.Struct)
}
func (s TunnelServer_getServerInfo_Params_List) String() string {
str, _ := text.MarshalList(0xdc3ed6801961e502, s.List)
return str
}
// TunnelServer_getServerInfo_Params_Promise is a wrapper for a TunnelServer_getServerInfo_Params promised by a client call.
type TunnelServer_getServerInfo_Params_Promise struct{ *capnp.Pipeline }
@@ -1192,11 +1492,6 @@ func (s TunnelServer_getServerInfo_Results_List) Set(i int, v TunnelServer_getSe
return s.List.SetStruct(i, v.Struct)
}
func (s TunnelServer_getServerInfo_Results_List) String() string {
str, _ := text.MarshalList(0xe3e37d096a5b564e, s.List)
return str
}
// TunnelServer_getServerInfo_Results_Promise is a wrapper for a TunnelServer_getServerInfo_Results promised by a client call.
type TunnelServer_getServerInfo_Results_Promise struct{ *capnp.Pipeline }
@@ -1259,11 +1554,6 @@ func (s TunnelServer_unregisterTunnel_Params_List) Set(i int, v TunnelServer_unr
return s.List.SetStruct(i, v.Struct)
}
func (s TunnelServer_unregisterTunnel_Params_List) String() string {
str, _ := text.MarshalList(0x9b87b390babc2ccf, s.List)
return str
}
// TunnelServer_unregisterTunnel_Params_Promise is a wrapper for a TunnelServer_unregisterTunnel_Params promised by a client call.
type TunnelServer_unregisterTunnel_Params_Promise struct{ *capnp.Pipeline }
@@ -1314,11 +1604,6 @@ func (s TunnelServer_unregisterTunnel_Results_List) Set(i int, v TunnelServer_un
return s.List.SetStruct(i, v.Struct)
}
func (s TunnelServer_unregisterTunnel_Results_List) String() string {
str, _ := text.MarshalList(0xa29a916d4ebdd894, s.List)
return str
}
// TunnelServer_unregisterTunnel_Results_Promise is a wrapper for a TunnelServer_unregisterTunnel_Results promised by a client call.
type TunnelServer_unregisterTunnel_Results_Promise struct{ *capnp.Pipeline }
@@ -1327,100 +1612,285 @@ func (p TunnelServer_unregisterTunnel_Results_Promise) Struct() (TunnelServer_un
return TunnelServer_unregisterTunnel_Results{s}, err
}
const schema_db8274f9144abc7e = "x\xda\x9cV_\x8c\x13U\x17?\xbf{\xdb\x9d\x16v" +
"\xe9N\xa6$\xd0|\xa4\xf9\x08||\x10A\x101\xb0" +
"\xfe\xd9?.h\xd7e\xe9\xd0\xd5\x10\xc0\x84\xa1\xbd\x94" +
"Y\xdb\x99ff\x8a,\xe1\x7f\xd6\x88D\x09\x08<\x88" +
"`\x90\x84\x08\x86\xa8\x89&&\x06\x13|P\x1ex " +
"&b4\x92\x18%<H$\xc4\x8d<hb\xc6\xdc" +
"\xe9N[v\x15\xd0\xb7\xdb\xdf=\xf7\x9c\xdf9s\xce" +
"\xeft\xe1\xffx\x17[\x14\xfd>J\xa4\xf7F[\xfc" +
"\xc7\xca\x97O=r\xf4\xd2\x08\xa9)\xe6\xef<\xdf\x97" +
"\xfc\xcd\xdb\xfb\x1d\x11\x16_a\xdb\xa0\xfd\xc4\x14\"\xed" +
":[E\xf0/?p\xfe\x93\x83\x1f\xbe\xfc&\xe9s" +
"\x00\xa2\x88B\xb4\xf8\x0f\xf6;\x08\x9a\xca;\x09\xfe\x91" +
"o>\x1d(\x1f:v\x8a\xd49\xe1\xfd2\xce\x18E" +
"\xfc\xa9\x19\\\xbd\xb0(\xf2q\xed&\xca\xe5\xd5|~" +
"C>\xed\xe6\xef\x13\xfc\x197{\xda\xac[{/\x90" +
"\x9aB\x83E\xcd\xf0\x07\xde\x07\xed\xb6<j\xbf\x04\xc6" +
"}\xeb\x0e\xbf\x1e\xbd~\xf8\"\xe9)4q\x8e*\xd2" +
"\xfa\xb5\x88\x03\xed\xed \xf8\x89\xc8;\x8c\xe0\xa7>x" +
"\xf4\xbd\x9e\xc2\xb7\x97\xc6\xf9\x0e2{E\x19\xd5\x8e\xca" +
"w\xda!\xe5E\x82\xcf\xae\x1b\xd3w\x7f\xfd\xc4\xd5\xa6" +
"\x14n*?\x82\"\xfe\xc0s\xeb\x86\xe2;\xae]\x1b" +
"K\x01\x013%H\xe1\xb6\"\xb3_\xb2\xa1[\xac_" +
"\xba\xe6\x06\xa9)~G!\xa7\xc6:\xa0\xcd\x8e\xc9 " +
"\xff\x8d]\xd4\xce\xca\x93\x7f`g\xef\xaae3?\x1b" +
"mvw(6*\xdd\x9d\x8eIw\x9b\x96\xfe\xfc\xd4" +
"\xec\x03_\x8c\x8ec\x1d\x18~\x1e\x9b\x07\xedJ\xe0\xf1" +
"Ki|k\xc5[_\xa5\x12\xa9_\xf5\x14\x9am\x03" +
"\xfa\xb7cC\xd0\xe2qy\x8c\xc6\xd3\xa0\xf9\xbeW\xb5" +
",Qr*\x91\xfc\x83\xe11\xbf oT\xacJ\xc7" +
"\xf2\xad\xa6\xeb\x99Vq0\xc0;\xb3v\xc9\xcc\x0fg" +
"\x01\xbd\x15\x8cH\x9d\xd1A\x04\xa8S\xd7\x12\x81\xa9j" +
"\x0fQ\xa7Y\xb4lG\xf8\x05\xd3\xcd\xdb\x96%\x88\xe7" +
"\xbd]\x1b\x8d\x92a\xe5E=P\xcb\xc4@\xb5\x009" +
"\xe1l\x11\xce\x82\xaa\xe5\x88\xa2\xe9z\xc2\xa9\xc1\xb3:" +
"\xb3\x86c\x94]=\xc2#D\x11\x10\xa9m\xc7\x88\xf4" +
"v\x0e\xfd?\x0c~\xd11\xf2\"+\x1c\x98va\xc0" +
"\xb0\xec\x1c\x17yD\x89!J\xa8\x07\x9d\xf2O\x83\xae" +
"\x16n\xb5\xe4\xb9T\x7fu\xf7\xf7\xe3^g\x8dD@" +
"\xb9\xb5Ny\xf9Z9a\x1cz\x96A\x05\x92rf" +
"\xd4\x95}Dz?\x87\xbe\x86Ae,\x19\x94\xf5\xd9" +
"\x1e\"=\xcb\xa1\xafg\xf0m\xc7,\x9a\xd6\x93\x82\xb8" +
"\xe3\xa1\x8d\x18\xda\x08\xfef\xdb\xf5,\xa3,\x88\x08\xad" +
"\xc4\xd0J\xd8eW<\xd3\xb6\\\xb47\xe6\x81\x80\xf6" +
"\xa6\x12\xfc\xc5\x07\xee\xaez\x9b\x85\xe5\x99yC>&" +
"\x0a\xbem\x83\xf2L\"\xbd\x8bC\xefo\xa2\x9cy\xa8" +
")\x8f\x90\xf2\xca\x8d\x8d<\x94\x17\xc4p\xc8*-\xca" +
"\x86Y\x0a\x7f\x85\xc9t\x93\xf2L\xc3\xe6n\xfcV\x07" +
"Uu\x02v\xab*\xe9 C\xc9qa\xc8Q\xebF" +
"\x1fQ\xae\x0b\x1c\xb9~4hj\x19\xf4\x10\xe5z%" +
"\x9eE\x83\xa9\xb6\x12)\xa2\xdc\xd3\x12\x1f\x04\x03x\x12" +
"\x9cH\xd3\xf1.QnP\xc2\x1b\xa4y\x84'\x11!" +
"\xd2\x9e\x0f\xdc\xaf\x97\xf8f\x89G#ID\x894\x81" +
"yD\xb9\x0d\x12\xdf.\xf1\x16\x96D\x0b\x916\x8c!" +
"\xa2\xdcV\x89\x8fH\\\x89&\xe5\x88j{\xe0\x10\xe5" +
"vK\xfcU\x89\xc7\xa6%\x11\x93\x82\x13\xe0\xfb$~" +
"D\xe2\xf1\xe9I\xc4\xa5\xfc`/Q\xee\xa0\xc4\x8fK" +
"|\x12\x92\x98D\xa4\xbd\x81cD\xb9\xe3\x12?#\xf1" +
"\xc9-IL&\xd2N\x07|NJ\xfc\x1c\x18\xfc|" +
"\xc9\x14\x96\x97)4w\xc8\x16\xe1\xb8\xa6m\x85\xbf\xb9" +
"\xed\xd6?\x81\x18\x1bt\xd4\xda7k'\xe4\xa4#\xd1" +
"X\x07\x04$\x08~\xc5\xb6K\x03wv^\xc23\x8a" +
".\xa6\x10\xb2\x1cho\xc8+A\x82~\xa0\x03y\xcf" +
"\xa4\x84me\x0ah!\x86\x96z+\xf4\xdb\x94\xce\x1b" +
"\xa5L\xa5\xce\xc4t\xbb\xab\x9e]\xadP\xba`x\xa2" +
"\x00\x10\x03\x08\xbeS\xb5V8vy\x10\xc2)\x9b\x96" +
"Q\xa2\xfaM\xde.W\x1c\xe1\xba0mK\xaf\x1a%" +
"\x93{\xc3\x88\x13C\x9c\x90\xa8V\xcd\xc2\x84Fc\xe3" +
"\x1b-]\xe9\x184\x8a\xb2\xb1b\xf5\xe6\x9f;\x8fH" +
"\x9f\xc5\xa1/lj\xfe\xf9\xb2\xf9\xff\xcf\xa1?\xcc\x90" +
"\x90\x13Xo\xf4-F\xa9*&D\xba\x97\xd4\x15\x85" +
"W;e\xacM\xf6\xac\xac\xe1(F\xd9\xfd\x97\xafW" +
"\x0b7!\x15\xabY&;\x88\xf4\x18\x87\x9ed\xe8t" +
"\x02AC{c\x95\x8c\xd3\x07\xfew\xe1:kQj" +
"\xe2\x10%\xaa\xefo\x84kK\xd5\xb7\x11S3\x0a\x1a" +
"+\x13\xe1\x86T\x1fw\x88\xa9K\x14\xb0\xfa_\x06\x84" +
"\x7f\x0d\xd4\xb9\xfb\x89\xa9\xb3\x15?\x94O\xea\xac\x85\xec" +
"\x82\x1ffG\xe9 \xbf.\xf8\xa1F#\x94Y\xa2." +
"dq\xff\xe5\x9e \xf1i\xf7~*\x16\xee\xd3{\xd7" +
"\xab\x16'!\xf9\xcaj5\xf9\x1d\"\xd2[9\xf4i" +
"\x0c~\xc9\x1e\xd3\xdb\xc4@S\x0b\xddM\x07k\x84C" +
"5L\xc8\xc7\xd2\x7f\xb2\xee\x7f\x87\x94\xea\xad\x1c\xfaH" +
"S\xb7\xee\x91\xe0v\x0e}_\x93T\xbf$W\xce\x08" +
"\x87~\xb2\xae~\xea\x89\xfdD\xfaI\x0e\xfd\\C\xfa" +
"\xd4\xb3\xd2\xf0\x0c\x87\xfe\x11\x83\"\x1c'\xe4\xa9T\x9d" +
"\x86\xa2\x97\xecb\xbfi\x09W\xea\xc1\x98\x04\xc8+9" +
"\xf8\x15\xe1\x94\x0dKX\xf0V\x18f\xa9\xea\x08j\xcc" +
"l-\xbfLo\x93\x8c\xfc\x19\x00\x00\xff\xff@V\xc0" +
"\xb7"
type TunnelServer_connect_Params struct{ capnp.Struct }
// TunnelServer_connect_Params_TypeID is the unique identifier for the type TunnelServer_connect_Params.
const TunnelServer_connect_Params_TypeID = 0xa766b24d4fe5da35
func NewTunnelServer_connect_Params(s *capnp.Segment) (TunnelServer_connect_Params, error) {
st, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})
return TunnelServer_connect_Params{st}, err
}
func NewRootTunnelServer_connect_Params(s *capnp.Segment) (TunnelServer_connect_Params, error) {
st, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})
return TunnelServer_connect_Params{st}, err
}
func ReadRootTunnelServer_connect_Params(msg *capnp.Message) (TunnelServer_connect_Params, error) {
root, err := msg.RootPtr()
return TunnelServer_connect_Params{root.Struct()}, err
}
func (s TunnelServer_connect_Params) String() string {
str, _ := text.Marshal(0xa766b24d4fe5da35, s.Struct)
return str
}
func (s TunnelServer_connect_Params) Parameters() (CapnpConnectParameters, error) {
p, err := s.Struct.Ptr(0)
return CapnpConnectParameters{Struct: p.Struct()}, err
}
func (s TunnelServer_connect_Params) HasParameters() bool {
p, err := s.Struct.Ptr(0)
return p.IsValid() || err != nil
}
func (s TunnelServer_connect_Params) SetParameters(v CapnpConnectParameters) error {
return s.Struct.SetPtr(0, v.Struct.ToPtr())
}
// NewParameters sets the parameters field to a newly
// allocated CapnpConnectParameters struct, preferring placement in s's segment.
func (s TunnelServer_connect_Params) NewParameters() (CapnpConnectParameters, error) {
ss, err := NewCapnpConnectParameters(s.Struct.Segment())
if err != nil {
return CapnpConnectParameters{}, err
}
err = s.Struct.SetPtr(0, ss.Struct.ToPtr())
return ss, err
}
// TunnelServer_connect_Params_List is a list of TunnelServer_connect_Params.
type TunnelServer_connect_Params_List struct{ capnp.List }
// NewTunnelServer_connect_Params creates a new list of TunnelServer_connect_Params.
func NewTunnelServer_connect_Params_List(s *capnp.Segment, sz int32) (TunnelServer_connect_Params_List, error) {
l, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1}, sz)
return TunnelServer_connect_Params_List{l}, err
}
func (s TunnelServer_connect_Params_List) At(i int) TunnelServer_connect_Params {
return TunnelServer_connect_Params{s.List.Struct(i)}
}
func (s TunnelServer_connect_Params_List) Set(i int, v TunnelServer_connect_Params) error {
return s.List.SetStruct(i, v.Struct)
}
// TunnelServer_connect_Params_Promise is a wrapper for a TunnelServer_connect_Params promised by a client call.
type TunnelServer_connect_Params_Promise struct{ *capnp.Pipeline }
func (p TunnelServer_connect_Params_Promise) Struct() (TunnelServer_connect_Params, error) {
s, err := p.Pipeline.Struct()
return TunnelServer_connect_Params{s}, err
}
func (p TunnelServer_connect_Params_Promise) Parameters() CapnpConnectParameters_Promise {
return CapnpConnectParameters_Promise{Pipeline: p.Pipeline.GetPipeline(0)}
}
type TunnelServer_connect_Results struct{ capnp.Struct }
// TunnelServer_connect_Results_TypeID is the unique identifier for the type TunnelServer_connect_Results.
const TunnelServer_connect_Results_TypeID = 0xfeac5c8f4899ef7c
func NewTunnelServer_connect_Results(s *capnp.Segment) (TunnelServer_connect_Results, error) {
st, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})
return TunnelServer_connect_Results{st}, err
}
func NewRootTunnelServer_connect_Results(s *capnp.Segment) (TunnelServer_connect_Results, error) {
st, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1})
return TunnelServer_connect_Results{st}, err
}
func ReadRootTunnelServer_connect_Results(msg *capnp.Message) (TunnelServer_connect_Results, error) {
root, err := msg.RootPtr()
return TunnelServer_connect_Results{root.Struct()}, err
}
func (s TunnelServer_connect_Results) String() string {
str, _ := text.Marshal(0xfeac5c8f4899ef7c, s.Struct)
return str
}
func (s TunnelServer_connect_Results) Result() (ConnectResult, error) {
p, err := s.Struct.Ptr(0)
return ConnectResult{Struct: p.Struct()}, err
}
func (s TunnelServer_connect_Results) HasResult() bool {
p, err := s.Struct.Ptr(0)
return p.IsValid() || err != nil
}
func (s TunnelServer_connect_Results) SetResult(v ConnectResult) error {
return s.Struct.SetPtr(0, v.Struct.ToPtr())
}
// NewResult sets the result field to a newly
// allocated ConnectResult struct, preferring placement in s's segment.
func (s TunnelServer_connect_Results) NewResult() (ConnectResult, error) {
ss, err := NewConnectResult(s.Struct.Segment())
if err != nil {
return ConnectResult{}, err
}
err = s.Struct.SetPtr(0, ss.Struct.ToPtr())
return ss, err
}
// TunnelServer_connect_Results_List is a list of TunnelServer_connect_Results.
type TunnelServer_connect_Results_List struct{ capnp.List }
// NewTunnelServer_connect_Results creates a new list of TunnelServer_connect_Results.
func NewTunnelServer_connect_Results_List(s *capnp.Segment, sz int32) (TunnelServer_connect_Results_List, error) {
l, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 1}, sz)
return TunnelServer_connect_Results_List{l}, err
}
func (s TunnelServer_connect_Results_List) At(i int) TunnelServer_connect_Results {
return TunnelServer_connect_Results{s.List.Struct(i)}
}
func (s TunnelServer_connect_Results_List) Set(i int, v TunnelServer_connect_Results) error {
return s.List.SetStruct(i, v.Struct)
}
// TunnelServer_connect_Results_Promise is a wrapper for a TunnelServer_connect_Results promised by a client call.
type TunnelServer_connect_Results_Promise struct{ *capnp.Pipeline }
func (p TunnelServer_connect_Results_Promise) Struct() (TunnelServer_connect_Results, error) {
s, err := p.Pipeline.Struct()
return TunnelServer_connect_Results{s}, err
}
func (p TunnelServer_connect_Results_Promise) Result() ConnectResult_Promise {
return ConnectResult_Promise{Pipeline: p.Pipeline.GetPipeline(0)}
}
const schema_db8274f9144abc7e = "x\xda\x9cW]l\x1cW\x15>\xe7\xde\xdd\x9dx\x9d" +
"e=\x9a\x8d\xd4XT+\xa2\xa0B\xd4\x94\xa4\xa1\x90" +
"\x9a\x9f\xb5\x1d;d\x8d\xed\xec\xf5\xda(\xb4\xa9\x94\xc9" +
"\xee\xb5=fwf{g&\x89\xab\xa4I\xa3 J" +
"T\x9a\x86&\x82\x86\x82\xa2HQ\x0a\x8a\x80\x14\x10j" +
"i\xa5\xe6\x81\x1f\xa9\x0f}A\x88@_\xa0D\x88\x8a" +
"*\"\x82\x07x`\xd0\x99\xdd\x99\x9d\xaeM\xdc\xf6\xc5" +
"\x1a\x7f\xf7\xde\xf3}\xe7\xdes\xbf{v[)5\xcc" +
"\xb6\xa7\xbf\xa5\x01\x88}\xe9L\xf0\xd9\xe6\x1b\x97>u" +
"\xfe\xf5S\xa0\x0f\xb2\xe0\xf1W&\x0a\xff\xf6N\xfe\x11" +
"\x00w\\\xe3\x8f\xa1\xf1K\xae\x01\x18\xd7\xf9^\xc0\xe0" +
"\x8d{_y\xf9\x99\x9f|\xed; \xeeA\x04Hi" +
"\x00;n\xf0\xff \xa0\xf1\x0e/\x01\x06\xe7~\xff\xea" +
"t\xf3\xec\x85K\xa0\xdf\x13\x8doH1\x06\xa9\xe0\x81" +
"?\xdc\xdc;\xf5\xe2\xfc\x95\xf6H\x1ai\xa8/\xf5\"" +
"-\xdd\x98\xa2\xa5\xbf9<\xf0\xd4\xc8\xa7\xcf\\\x011" +
"\x88\xd8\x95\x91&\xfa\x1d\x9fK\x9dFc\x8e\xe2\x19\"" +
"\xf5W\xc0 \xf7\x8b-\xd3g\xfe2y\x8df'D" +
"\xb7\xe3\x8e\xa7\x87\xd0\x98K\x87\xb3\xd3?\x02\x0c6\x94" +
"\xf1\xcd\xd7\xb6\xa7~\xde!\x0fC\xfe#\xfd6\x91\xa7" +
"34\xe1\xeewFs\xf6\xad\x93\xaf\x81>\xb8\x82\xfb" +
"rf\x02\x8d\x972\x14\xedg\xe1\xe4\x89\x87\x9f\xfdf" +
"\xfa\xe6\xb3\xbf\xee\xe5\xd6h\xb6\xd0\x14\x1a2\xfc4\xb5" +
"+\x0c0\x18\xfc\xf1g~8Z\xbf\xf1zOlF" +
"\x01\xa7\xb2\xb7\x8d/g\xe9k.{\x180`7\xcd" +
"\x8d'~\xf7\xf97\x13\xfbw-\xfbg\x84T0\xfd" +
"\xa5\x87\x97\xfa\x8e\xbd\xf5Vr\xff.g\xc3\x14^\xca" +
"\xd2\xfe=p`D\xee\xdf\xb9\xefm\xd0\x07\xf9\xbbN" +
"\xf1Fv\x08\x8d\xbf\x11\xc9\x8e\x9bY\x0d\x8d\xf3\xfd\x1a" +
"@\xf0\xf4\xe3c{\x1f\xdct\xfdv2\xde\xb1\xfe\xdb" +
"\x14\xefl?\xc5\x9b\xdf\xf9\xf7/|\xf4\xe9_\xdd\xee" +
"\x91\x1dN\xbc\xd6\xbf\x05\x8d\xeb\x14\xc7x\x95&\xdf\xda" +
"\xfd\xbd\xdf\x0e\xe6\x07\xff\xd9st\xa1\xfe?\xf5/\xa1" +
"\xf1\xaf\xfep\xcb\xfb\x8b\x08\x18\x1c\xbd\xf5\xdc\x9e3\xfb" +
"\xaf\xfe7I\xfd\x91\xdc\xcbD\xbd=G\xd4K\xe7\x8f" +
"z{\xbe\xfd\x8d`\x95\x1d\xdb!r\xa3h\x989\xa2" +
"~$w\x18\xb6\x06\x9eo\xdb\xb2\xa1Z\xa9\xda'\xa2" +
"\xcf\xda}5\xb3e\xb7\x86\xc6\x8fX\xaeg\xd9\x0b\xb3" +
"!^\xaa8\x0d\xab\xb6\\A\x14\xeb\x91\x01\xe8w\x0f" +
"\x01 \xea\x1b\x1e\x02@\xa6\xeb\xa3\x00%k\xc1v\x94" +
"\x0c\xea\x96[sl[\x02\xafy\xc7\x0f\x9a\x0d\xd3\xae" +
"\xc9\x98(\xb3\x92\xa8MP\x95\xea\x90T\xf7\xf9\xb6\x92" +
"\x0b\x96\xebI\xd5\x867\x97*\xa62\x9b\xaeH\xf1\x14" +
"@\x0a\x01\xf4\xdc\x05\x001\xc0Q|\x98a\xb0\xa0\xcc" +
"\x9a\xacH\x85\x96S\x9f6m\xa7\xcae\x0d\xd3\xc00" +
"\x0d\x18\x93\xe6\xde/\xe9\x8ct\xfd\x86\xe7B\xbc\xea\xce" +
"\xeb\xc3tk\xde\xe6Jq\x85\xd6\x87\x00\xc4z\x8e\xe2" +
".\x86A\x8bF\xa5'\x81+\x17\x07\xbaw\x16\x10\x07" +
"\x12j\xd3+\xd9v\xd1\xdf]m\x96J'\x8ar\x01" +
"\xe88\x06b2\x93\xc8\x0ep\x14\x0d\x86:b\x81\\" +
"F\xb7\x14\x80X\xe4(<\x86\xc8\x0a\xe1\xd9=z\x09" +
"@x\x1c\xc5\x09\x86:g\x05\xe4\x00\xfa\xb1-\x00\xe2" +
"\x08Gq\x8ea\xe0(k\xc1\xb2w\x91T\x0fs\xc0" +
"0\x07\x18\xd4\x1a\x8e_\x9fo\x98PT\xb2^\x1e\x8b" +
"q\xdboV\x94<d\xa1\xe3\xbb#\x9e'\x9bZ\xcb" +
"s1\x03\x0c3\x80y\xcf\\p\xf1C\x80\x15\x8e8" +
"\xd0\xbd\xcf\x80\x04\xc6I\xf3U\x92\x0e\xf3-y\xe3J" +
"9*,\xbc8\xd3\xf1\xfb\x01\xc40G1\xc90J" +
"\xb4L\xd9\xef\xe1(f\x19\xeal\xb8\x9d\xa98\x08 " +
"*\x1c\xc5~\x86\xc5\x9a\xe9\xbb\x12\xd7\x03\xc3\xf5\x80\x81" +
"\x92\x9eZ\x1e\x99\xf7\x80K\x15W\x8c\xbb\xe8\xf8\x8d\xfa" +
"\x8c\x04\xcdS\xcb\x88\xc0\x10\x13\"\xd7*\xde\x9e*\xaa" +
"\x98\xf9\xb0\x1c\x12\xbaI\xe3\x18GQI\x9c\xd0\xd4\x04" +
"\x80\x98\xe4(\xf6\x91\xf0\xce\x11\xcd\x8dv\x85\xaf~\x1a" +
"\x8b\x8e\xeb\xd9fS\x02@\x94\xd3q\xa7\xe5Y\x8eM" +
"\xc5\x15\xdblOq\xadr\xd1G|oQ\xda\x9eU" +
"3iq\xbb\xa8\x12\x927u\xb7:\x96\\\xbe?\x91" +
"G$y\xea`7\x0f\xed+r9RU\x94M\xd3" +
"j\xc4\xfb\xdeIf\x04\xb4/v\xe7\xdcI\xdfL\xb8" +
"\xab*T\xb7\xb7U\x0c3$\x8d\xdb\"\x8d\xc6\x08N" +
"\x00T\x87\x91cu\x12\xbb2\x8d2\x8e\x02T\xc7\x08" +
"\xaf`W\xa91\x85\x83\x00\xd5=\x84\xcf\"C\xe4\xe1" +
"\x0d0\x04\xfe\x00\xa0:K\xf0\x01\x9a\x9e\xe2\x05L\x91" +
"U\x86\xe1\xf7\x13\xbeHx:U\xc04\x80!q\x0b" +
"@\xf5\x00\xe1G\x09\xcf\xb0\x02f\x00\x8ce\\\x02\xa8" +
"\x1e!\xfc\x14\xe1Z\xba@6m<\x81\x0a\xa0z\x82" +
"\xf0\xa7\x08_wW\x01\xd7\x01\x18_\x0f\xf1'\x09?" +
"Gx\xdf\xc6\x02\xf6\x01\x18g\xf1$@\xf5\x19\xc2\x9f" +
"'<\x8b\x05\xcc\x02\x18\xcf\xe1\x05\x80\xea\xf3\x84\xbf@" +
"x\x7f\xa6\x80\xfd\x00\xc6\xe5P\xcfE\xc2\xaf\"\xa3\xfb" +
"jI\xdb+\xd7\x93\x15rH*\xd7r\xec\xe8\x7f\xee" +
"\xb8\xf1\x11\xc8\x8e\xe1c\xbb|+N\x9e\x1c\x1f\xf3\xdd" +
"\x16\x07\x10\xf3\x80A\xcbq\x1a\xd3\xef\xae\xbc5ny" +
"\xc7 -\xc8;v\xb9\x1eYC\xa7\x14&\x1d(\xd6" +
"\xccF\xb9\x15+\xb1\xdc\x11\xdfs\xfc\x16\x14\xeb\xa6'" +
"\xeb\xf1=T\xbe\xbd[9\xcdY\x94\xaai\xd9f\x03" +
"\xe2\x91\x9a\xd3l)\xe9\xbah9\xb6\xf0\xcd\x86\xc5\xbd" +
"e\xec\x03\x86}\x80y\xdf\xb7\xea+\x0a\x8d\xf5\x16Z" +
"\xb154k.Pa\xad\x8b\x8b\xff\xe3\xe4\x89\x9b9" +
"\x8am\x89\xe2\xdfJ\xc5\xff1\x8e\xe2\x93\x0c\xf3t\x03" +
"\xe3B?d6|\xb9\x82i-\xd7X\x90^\xfb\xab" +
"l\xcf;\x9b+\xa6\xd2\xcc\xa6\xfb\x01W\xcfH7O" +
"/W\xf2\x09\x1a\x02\x10\xeb8\x8a\x02\xc3\x92\x0a\x1f6" +
"\x1c\xe86(=\xfe\xc0\xff\x1f]\xa9\xcd\xd2~q\xd2" +
"\x00q[\x88Q3\xa4?\xfa\x180\xdd\xd2\xb0\xdb\x89" +
"a\xd4x\xe9\x8f(`\xfa\x9c\x86,n\x831jw" +
"\xf5\xf2i`\xfa\xb8\x86<\xees1\xear\xf4\x07G" +
"\x81\xe9[\xb5 \xb2V(\xb5\xe5\x0cc\x10e\x0e\xc5" +
"0\xf7a\x0c\xa2w\x1c#\x0b\x06\x18\xc6\xe3\x9d\xe2\x1b" +
"\xc6\x0a~`+\x9f\x91E\xf7\xbd\xeck\xd4\xcb\xad\xbd" +
"\xabm\x9e<)\xa7=M\xc4]J\xb4\x0c\x0d\xa7\xe3" +
"\xca\xf9\xe9D\xa1\xdd\xc9-\xdb\x82#\xcf\xcc\xd3b\x8a" +
"_\x88\xe3\x1f\xdb\xd4y\xe7O%j\xfa\x09\x02\x8fr" +
"\x14O&\x0c\xfd\xab\xf40\x9d\xe2(.\xc6\x1e\xa9\x7f" +
"\xf74\x80\xb8\xc8Q\\\xed\x1a\xa4\xfe}\x9a\xf8\x02G" +
"\xf1S\x86\x9aT*\xd2\xa9\xf9\xaa\xeb\xfb\x0dga\xd2" +
"\xb2\xa5K\xae\xd11\x0a\x1a\"{hI\xd54mi" +
"\xa3\xb7\xdb\xb4\x1a\xbe\x92\x00=ooy,a6w" +
"\xea\x94V\xed\xcbfJ\xf2=\x9d^\xb7w^\xfb\xd5" +
"\xectc\xedN\x11zlc\xd3j\xb6Ao\xff\xbd" +
"\x1c\xc5\xce\xce&\x0dt\x7f\x87u\xe8\xdc\xceE\x06>" +
"\xef\xac\xbc\xa3\xff\x0b\x00\x00\xff\xffR\x8f\xce\x88"
func init() {
schemas.Register(schema_db8274f9144abc7e,
0x84cb9536a2cf6d3c,
0x9b87b390babc2ccf,
0xa29a916d4ebdd894,
0xa766b24d4fe5da35,
0xa78f37418c1077c8,
0xb14ce48f4e2abb0d,
0xb70431c0dc014915,
0xc082ef6e0d42ed1d,
0xc793e50592935b4a,
@@ -1430,5 +1900,7 @@ func init() {
0xea58385c65416035,
0xf2c122394f447e8e,
0xf2c68e2547ec3866,
0xf41a0f001ad49e46)
0xf41a0f001ad49e46,
0xfeac5c8f4899ef7c,
0xff8d9848747c956a)
}
+31 -11
View File
@@ -5,15 +5,20 @@ import (
"net"
"net/url"
"strings"
"time"
"net/http"
"github.com/pkg/errors"
"golang.org/x/net/idna"
"net/http"
)
const defaultScheme = "http"
var supportedProtocol = [2]string{"http", "https"}
var (
supportedProtocols = []string{"http", "https", "rdp"}
validationTimeout = time.Duration(30 * time.Second)
)
func ValidateHostname(hostname string) (string, error) {
if hostname == "" {
@@ -116,7 +121,7 @@ func ValidateUrl(originUrl string) (string, error) {
}
func validateScheme(scheme string) error {
for _, protocol := range supportedProtocol {
for _, protocol := range supportedProtocols {
if scheme == protocol {
return nil
}
@@ -137,26 +142,41 @@ func validateIP(scheme, host, port string) (string, error) {
return fmt.Sprintf("%s://%s", scheme, host), nil
}
func ValidateHTTPService(originURL string, transport http.RoundTripper) error {
func ValidateHTTPService(originURL string, hostname string, transport http.RoundTripper) error {
parsedURL, err := url.Parse(originURL)
if err != nil {
return err
}
client := &http.Client{Transport: transport}
client := &http.Client{
Transport: transport,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
Timeout: validationTimeout,
}
initialResponse, initialErr := client.Get(parsedURL.String())
if initialErr != nil || initialResponse.StatusCode != http.StatusOK {
initialRequest, err := http.NewRequest("GET", parsedURL.String(), nil)
if err != nil {
return err
}
initialRequest.Host = hostname
_, initialErr := client.Do(initialRequest)
if initialErr != nil {
// Attempt the same endpoint via the other protocol (http/https); maybe we have better luck?
oldScheme := parsedURL.Scheme
parsedURL.Scheme = toggleProtocol(parsedURL.Scheme)
secondResponse, _ := client.Get(parsedURL.String())
if secondResponse != nil && secondResponse.StatusCode == http.StatusOK { // Worked this time--advise the user to switch protocols
secondRequest, err := http.NewRequest("GET", parsedURL.String(), nil)
if err != nil {
return err
}
secondRequest.Host = hostname
_, secondErr := client.Do(secondRequest)
if secondErr == nil { // Worked this time--advise the user to switch protocols
return errors.Errorf(
"%s doesn't seem to work over %s, but does seem to work over %s. Consider changing the origin URL to %s",
parsedURL.Hostname(),
parsedURL.Host,
oldScheme,
parsedURL.Scheme,
parsedURL,
+251 -27
View File
@@ -1,18 +1,22 @@
package validation
import (
"bytes"
"fmt"
"io/ioutil"
"testing"
"time"
"context"
"crypto/tls"
"crypto/x509"
"github.com/stretchr/testify/assert"
"net"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"github.com/stretchr/testify/assert"
)
func TestValidateHostname(t *testing.T) {
@@ -150,58 +154,278 @@ func TestToggleProtocol(t *testing.T) {
assert.Equal(t, "", toggleProtocol(""))
}
// Happy path 1: originURL is HTTP, and HTTP connections work
func TestValidateHTTPService_HTTP2HTTP(t *testing.T) {
originURL := "http://127.0.0.1/"
hostname := "example.com"
assert.Nil(t, ValidateHTTPService(originURL, hostname, testRoundTripper(func(req *http.Request) (*http.Response, error) {
assert.Equal(t, req.Host, hostname)
if req.URL.Scheme == "http" {
return emptyResponse(200), nil
}
if req.URL.Scheme == "https" {
t.Fatal("http works, shouldn't have tried with https")
}
panic("Shouldn't reach here")
})))
assert.Nil(t, ValidateHTTPService(originURL, hostname, testRoundTripper(func(req *http.Request) (*http.Response, error) {
assert.Equal(t, req.Host, hostname)
if req.URL.Scheme == "http" {
return emptyResponse(503), nil
}
if req.URL.Scheme == "https" {
t.Fatal("http works, shouldn't have tried with https")
}
panic("Shouldn't reach here")
})))
// Integration-style test with a mock server
server, client, err := createMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, hostname, r.Host)
w.WriteHeader(200)
}))
assert.NoError(t, err)
defer server.Close()
assert.Nil(t, ValidateHTTPService(originURL, hostname, client.Transport))
assert.Equal(t, nil, ValidateHTTPService("http://example.com/", client.Transport))
}
func TestValidateHTTPService_ServerNonOKResponse(t *testing.T) {
server, client, err := createMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(400)
// this will fail if the client follows the 302
redirectServer, redirectClient, err := createMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/followedRedirect" {
t.Fatal("shouldn't have followed the 302")
}
if r.Method == "CONNECT" {
assert.Equal(t, "127.0.0.1:443", r.Host)
} else {
assert.Equal(t, hostname, r.Host)
}
w.Header().Set("Location", "/followedRedirect")
w.WriteHeader(302)
}))
assert.NoError(t, err)
defer server.Close()
defer redirectServer.Close()
assert.Nil(t, ValidateHTTPService(originURL, hostname, redirectClient.Transport))
assert.Equal(t, nil, ValidateHTTPService("http://example.com/", client.Transport))
}
func TestValidateHTTPService_HTTPS2HTTP(t *testing.T) {
server, client, err := createMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(200)
}))
assert.NoError(t, err)
defer server.Close()
assert.Equal(t,
"example.com doesn't seem to work over https, but does seem to work over http. Consider changing the origin URL to http://example.com:1234/",
ValidateHTTPService("https://example.com:1234/", client.Transport).Error())
}
// Happy path 2: originURL is HTTPS, and HTTPS connections work
func TestValidateHTTPService_HTTPS2HTTPS(t *testing.T) {
originURL := "https://127.0.0.1/"
hostname := "example.com"
assert.Nil(t, ValidateHTTPService(originURL, hostname, testRoundTripper(func(req *http.Request) (*http.Response, error) {
assert.Equal(t, req.Host, hostname)
if req.URL.Scheme == "http" {
t.Fatal("https works, shouldn't have tried with http")
}
if req.URL.Scheme == "https" {
return emptyResponse(200), nil
}
panic("Shouldn't reach here")
})))
assert.Nil(t, ValidateHTTPService(originURL, hostname, testRoundTripper(func(req *http.Request) (*http.Response, error) {
assert.Equal(t, req.Host, hostname)
if req.URL.Scheme == "http" {
t.Fatal("https works, shouldn't have tried with http")
}
if req.URL.Scheme == "https" {
return emptyResponse(503), nil
}
panic("Shouldn't reach here")
})))
// Integration-style test with a mock server
server, client, err := createSecureMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == "CONNECT" {
assert.Equal(t, "127.0.0.1:443", r.Host)
} else {
assert.Equal(t, hostname, r.Host)
}
w.WriteHeader(200)
}))
assert.NoError(t, err)
defer server.Close()
assert.Nil(t, ValidateHTTPService(originURL, hostname, client.Transport))
assert.Equal(t, nil, ValidateHTTPService("https://example.com/", client.Transport))
// this will fail if the client follows the 302
redirectServer, redirectClient, err := createSecureMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/followedRedirect" {
t.Fatal("shouldn't have followed the 302")
}
if r.Method == "CONNECT" {
assert.Equal(t, "127.0.0.1:443", r.Host)
} else {
assert.Equal(t, hostname, r.Host)
}
w.Header().Set("Location", "/followedRedirect")
w.WriteHeader(302)
}))
assert.NoError(t, err)
defer redirectServer.Close()
assert.Nil(t, ValidateHTTPService(originURL, hostname, redirectClient.Transport))
}
func TestValidateHTTPService_HTTP2HTTPS(t *testing.T) {
server, client, err := createSecureMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Error path 1: originURL is HTTPS, but HTTP connections work
func TestValidateHTTPService_HTTPS2HTTP(t *testing.T) {
originURL := "https://127.0.0.1:1234/"
hostname := "example.com"
assert.Error(t, ValidateHTTPService(originURL, hostname, testRoundTripper(func(req *http.Request) (*http.Response, error) {
assert.Equal(t, req.Host, hostname)
if req.URL.Scheme == "http" {
return emptyResponse(200), nil
}
if req.URL.Scheme == "https" {
return nil, assert.AnError
}
panic("Shouldn't reach here")
})))
assert.Error(t, ValidateHTTPService(originURL, hostname, testRoundTripper(func(req *http.Request) (*http.Response, error) {
assert.Equal(t, req.Host, hostname)
if req.URL.Scheme == "http" {
return emptyResponse(503), nil
}
if req.URL.Scheme == "https" {
return nil, assert.AnError
}
panic("Shouldn't reach here")
})))
// Integration-style test with a mock server
server, client, err := createMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == "CONNECT" {
assert.Equal(t, "127.0.0.1:1234", r.Host)
} else {
assert.Equal(t, hostname, r.Host)
}
w.WriteHeader(200)
}))
assert.NoError(t, err)
defer server.Close()
assert.Error(t, ValidateHTTPService(originURL, hostname, client.Transport))
assert.Equal(t,
"example.com doesn't seem to work over http, but does seem to work over https. Consider changing the origin URL to https://example.com:1234/",
ValidateHTTPService("http://example.com:1234/", client.Transport).Error())
// this will fail if the client follows the 302
redirectServer, redirectClient, err := createMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/followedRedirect" {
t.Fatal("shouldn't have followed the 302")
}
if r.Method == "CONNECT" {
assert.Equal(t, "127.0.0.1:1234", r.Host)
} else {
assert.Equal(t, hostname, r.Host)
}
w.Header().Set("Location", "/followedRedirect")
w.WriteHeader(302)
}))
assert.NoError(t, err)
defer redirectServer.Close()
assert.Error(t, ValidateHTTPService(originURL, hostname, redirectClient.Transport))
}
// Error path 2: originURL is HTTP, but HTTPS connections work
func TestValidateHTTPService_HTTP2HTTPS(t *testing.T) {
originURL := "http://127.0.0.1:1234/"
hostname := "example.com"
assert.Error(t, ValidateHTTPService(originURL, hostname, testRoundTripper(func(req *http.Request) (*http.Response, error) {
assert.Equal(t, req.Host, hostname)
if req.URL.Scheme == "http" {
return nil, assert.AnError
}
if req.URL.Scheme == "https" {
return emptyResponse(200), nil
}
panic("Shouldn't reach here")
})))
assert.Error(t, ValidateHTTPService(originURL, hostname, testRoundTripper(func(req *http.Request) (*http.Response, error) {
assert.Equal(t, req.Host, hostname)
if req.URL.Scheme == "http" {
return nil, assert.AnError
}
if req.URL.Scheme == "https" {
return emptyResponse(503), nil
}
panic("Shouldn't reach here")
})))
// Integration-style test with a mock server
server, client, err := createSecureMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == "CONNECT" {
assert.Equal(t, "127.0.0.1:1234", r.Host)
} else {
assert.Equal(t, hostname, r.Host)
}
w.WriteHeader(200)
}))
assert.NoError(t, err)
defer server.Close()
assert.Error(t, ValidateHTTPService(originURL, hostname, client.Transport))
// this will fail if the client follows the 302
redirectServer, redirectClient, err := createSecureMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/followedRedirect" {
t.Fatal("shouldn't have followed the 302")
}
if r.Method == "CONNECT" {
assert.Equal(t, "127.0.0.1:443", r.Host)
} else {
assert.Equal(t, hostname, r.Host)
}
w.Header().Set("Location", "/followedRedirect")
w.WriteHeader(302)
}))
assert.NoError(t, err)
defer redirectServer.Close()
assert.Error(t, ValidateHTTPService(originURL, hostname, redirectClient.Transport))
}
// error path 3: origin URL is nonresponsive
func TestValidateHTTPService_NonResponsiveOrigin(t *testing.T) {
originURL := "https://127.0.0.1/"
hostname := "example.com"
oldValidationTimeout := validationTimeout
defer func() {
validationTimeout = oldValidationTimeout
}()
validationTimeout = 500 * time.Millisecond
server, client, err := createSecureMockServerAndClient(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == "CONNECT" {
assert.Equal(t, "127.0.0.1:443", r.Host)
} else {
assert.Equal(t, hostname, r.Host)
}
time.Sleep(1 * time.Second)
w.WriteHeader(200)
}))
if !assert.NoError(t, err) {
t.FailNow()
}
defer server.Close()
err = ValidateHTTPService(originURL, hostname, client.Transport)
if err, ok := err.(net.Error); assert.True(t, ok) {
assert.True(t, err.Timeout())
}
}
type testRoundTripper func(req *http.Request) (*http.Response, error)
func (f testRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
func emptyResponse(statusCode int) *http.Response {
return &http.Response{
StatusCode: statusCode,
Body: ioutil.NopCloser(bytes.NewReader(nil)),
Header: make(http.Header),
}
}
func createMockServerAndClient(handler http.Handler) (*httptest.Server, *http.Client, error) {
+27
View File
@@ -0,0 +1,27 @@
Copyright (c) 2013 CloudFlare, Inc. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of the CloudFlare, Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+3
View File
@@ -0,0 +1,3 @@
cover.out~
benchmark/benchmark
+41
View File
@@ -0,0 +1,41 @@
# Copyright (c) 2013 CloudFlare, Inc.
RACE+=--race
PKGNAME=github.com/cloudflare/golibs/lrucache
SKIPCOVER=list.go|list_extension.go|priorityqueue.go
.PHONY: all test bench cover clean
all:
@echo "Targets:"
@echo " test: run tests with race detector"
@echo " cover: print test coverage"
@echo " bench: run basic benchmarks"
test:
@go test $(RACE) -bench=. -v $(PKGNAME)
COVEROUT=cover.out
cover:
@go test -coverprofile=$(COVEROUT) -v $(PKGNAME)
@cat $(COVEROUT) | egrep -v '$(SKIPCOVER)' > $(COVEROUT)~
@go tool cover -func=$(COVEROUT)~|sed 's|^.*/\([^/]*/[^/]*/[^/]*\)$$|\1|g'
bench:
@echo "[*] Scalability of cache/lrucache"
@echo "[ ] Operations in shared cache using one core"
@GOMAXPROCS=1 go test -run=- -bench='.*LRUCache.*' $(PKGNAME) \
| egrep -v "^PASS|^ok"
@echo "[*] Scalability of cache/multilru"
@echo "[ ] Operations in four caches using four cores "
@GOMAXPROCS=4 go test -run=- -bench='.*MultiLRU.*' $(PKGNAME) \
| egrep -v "^PASS|^ok"
@(cd benchmark; go build $(PKGNAME)/benchmark)
@./benchmark/benchmark
clean:
rm -rf $(COVEROUT) $(COVEROUT)~ benchmark/benchmark
+40
View File
@@ -0,0 +1,40 @@
LRU Cache
---------
A `golang` implementation of last recently used cache data structure.
To install:
go get github.com/cloudflare/golibs/lrucache
To test:
cd $GOPATH/src/github.com/cloudflare/golibs/lrucache
make test
For coverage:
make cover
Basic benchmarks:
$ make bench # As tested on my two core i5
[*] Scalability of cache/lrucache
[ ] Operations in shared cache using one core
BenchmarkConcurrentGetLRUCache 5000000 450 ns/op
BenchmarkConcurrentSetLRUCache 2000000 821 ns/op
BenchmarkConcurrentSetNXLRUCache 5000000 664 ns/op
[*] Scalability of cache/multilru
[ ] Operations in four caches using four cores
BenchmarkConcurrentGetMultiLRU-4 5000000 475 ns/op
BenchmarkConcurrentSetMultiLRU-4 2000000 809 ns/op
BenchmarkConcurrentSetNXMultiLRU-4 5000000 643 ns/op
[*] Capacity=4096 Keys=30000 KeySpace=15625
vitess LRUCache MultiLRUCache-4
create 1.709us 1.626374ms 343.54us
Get (miss) 144.266083ms 132.470397ms 177.277193ms
SetNX #1 338.637977ms 380.733302ms 411.709204ms
Get (hit) 195.896066ms 173.252112ms 234.109494ms
SetNX #2 349.785951ms 367.255624ms 419.129127ms
+69
View File
@@ -0,0 +1,69 @@
// Copyright (c) 2013 CloudFlare, Inc.
// Package lrucache implements a last recently used cache data structure.
//
// This code tries to avoid dynamic memory allocations - all required
// memory is allocated on creation. Access to the data structure is
// O(1). Modification O(log(n)) if expiry is used, O(1)
// otherwise.
//
// This package exports three things:
// LRUCache: is the main implementation. It supports multithreading by
// using guarding mutex lock.
//
// MultiLRUCache: is a sharded implementation. It supports the same
// API as LRUCache and uses it internally, but is not limited to
// a single CPU as every shard is separately locked. Use this
// data structure instead of LRUCache if you have have lock
// contention issues.
//
// Cache interface: Both implementations fulfill it.
package lrucache
import (
"time"
)
// Cache interface is fulfilled by the LRUCache and MultiLRUCache
// implementations.
type Cache interface {
// Methods not needing to know current time.
//
// Get a key from the cache, possibly stale. Update its LRU
// score.
Get(key string) (value interface{}, ok bool)
// Get a key from the cache, possibly stale. Don't modify its LRU score. O(1)
GetQuiet(key string) (value interface{}, ok bool)
// Get and remove a key from the cache.
Del(key string) (value interface{}, ok bool)
// Evict all items from the cache.
Clear() int
// Number of entries used in the LRU
Len() int
// Get the total capacity of the LRU
Capacity() int
// Methods use time.Now() when neccessary to determine expiry.
//
// Add an item to the cache overwriting existing one if it
// exists.
Set(key string, value interface{}, expire time.Time)
// Get a key from the cache, make sure it's not stale. Update
// its LRU score.
GetNotStale(key string) (value interface{}, ok bool)
// Evict all the expired items.
Expire() int
// Methods allowing to explicitly specify time used to
// determine if items are expired.
//
// Add an item to the cache overwriting existing one if it
// exists. Allows specifing current time required to expire an
// item when no more slots are used.
SetNow(key string, value interface{}, expire time.Time, now time.Time)
// Get a key from the cache, make sure it's not stale. Update
// its LRU score.
GetNotStaleNow(key string, now time.Time) (value interface{}, ok bool)
// Evict items that expire before Now.
ExpireNow(now time.Time) int
}
+238
View File
@@ -0,0 +1,238 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
/* This file is a slightly modified file from the go package sources
and is released on the following license:
Copyright (c) 2012 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
// Package list implements a doubly linked list.
//
// To iterate over a list (where l is a *List):
// for e := l.Front(); e != nil; e = e.Next() {
// // do something with e.Value
// }
//
package lrucache
// Element is an element of a linked list.
type element struct {
// Next and previous pointers in the doubly-linked list of elements.
// To simplify the implementation, internally a list l is implemented
// as a ring, such that &l.root is both the next element of the last
// list element (l.Back()) and the previous element of the first list
// element (l.Front()).
next, prev *element
// The list to which this element belongs.
list *list
// The value stored with this element.
Value interface{}
}
// Next returns the next list element or nil.
func (e *element) Next() *element {
if p := e.next; e.list != nil && p != &e.list.root {
return p
}
return nil
}
// Prev returns the previous list element or nil.
func (e *element) Prev() *element {
if p := e.prev; e.list != nil && p != &e.list.root {
return p
}
return nil
}
// List represents a doubly linked list.
// The zero value for List is an empty list ready to use.
type list struct {
root element // sentinel list element, only &root, root.prev, and root.next are used
len int // current list length excluding (this) sentinel element
}
// Init initializes or clears list l.
func (l *list) Init() *list {
l.root.next = &l.root
l.root.prev = &l.root
l.len = 0
return l
}
// New returns an initialized list.
// func New() *list { return new(list).Init() }
// Len returns the number of elements of list l.
// The complexity is O(1).
func (l *list) Len() int { return l.len }
// Front returns the first element of list l or nil
func (l *list) Front() *element {
if l.len == 0 {
return nil
}
return l.root.next
}
// Back returns the last element of list l or nil.
func (l *list) Back() *element {
if l.len == 0 {
return nil
}
return l.root.prev
}
// insert inserts e after at, increments l.len, and returns e.
func (l *list) insert(e, at *element) *element {
n := at.next
at.next = e
e.prev = at
e.next = n
n.prev = e
e.list = l
l.len++
return e
}
// insertValue is a convenience wrapper for insert(&Element{Value: v}, at).
func (l *list) insertValue(v interface{}, at *element) *element {
return l.insert(&element{Value: v}, at)
}
// remove removes e from its list, decrements l.len, and returns e.
func (l *list) remove(e *element) *element {
e.prev.next = e.next
e.next.prev = e.prev
e.next = nil // avoid memory leaks
e.prev = nil // avoid memory leaks
e.list = nil
l.len--
return e
}
// Remove removes e from l if e is an element of list l.
// It returns the element value e.Value.
func (l *list) Remove(e *element) interface{} {
if e.list == l {
// if e.list == l, l must have been initialized when e was inserted
// in l or l == nil (e is a zero Element) and l.remove will crash
l.remove(e)
}
return e.Value
}
// PushFront inserts a new element e with value v at the front of list l and returns e.
func (l *list) PushFront(v interface{}) *element {
return l.insertValue(v, &l.root)
}
// PushBack inserts a new element e with value v at the back of list l and returns e.
func (l *list) PushBack(v interface{}) *element {
return l.insertValue(v, l.root.prev)
}
// InsertBefore inserts a new element e with value v immediately before mark and returns e.
// If mark is not an element of l, the list is not modified.
func (l *list) InsertBefore(v interface{}, mark *element) *element {
if mark.list != l {
return nil
}
// see comment in List.Remove about initialization of l
return l.insertValue(v, mark.prev)
}
// InsertAfter inserts a new element e with value v immediately after mark and returns e.
// If mark is not an element of l, the list is not modified.
func (l *list) InsertAfter(v interface{}, mark *element) *element {
if mark.list != l {
return nil
}
// see comment in List.Remove about initialization of l
return l.insertValue(v, mark)
}
// MoveToFront moves element e to the front of list l.
// If e is not an element of l, the list is not modified.
func (l *list) MoveToFront(e *element) {
if e.list != l || l.root.next == e {
return
}
// see comment in List.Remove about initialization of l
l.insert(l.remove(e), &l.root)
}
// MoveToBack moves element e to the back of list l.
// If e is not an element of l, the list is not modified.
func (l *list) MoveToBack(e *element) {
if e.list != l || l.root.prev == e {
return
}
// see comment in List.Remove about initialization of l
l.insert(l.remove(e), l.root.prev)
}
// MoveBefore moves element e to its new position before mark.
// If e is not an element of l, or e == mark, the list is not modified.
func (l *list) MoveBefore(e, mark *element) {
if e.list != l || e == mark {
return
}
l.insert(l.remove(e), mark.prev)
}
// MoveAfter moves element e to its new position after mark.
// If e is not an element of l, or e == mark, the list is not modified.
func (l *list) MoveAfter(e, mark *element) {
if e.list != l || e == mark {
return
}
l.insert(l.remove(e), mark)
}
// PushBackList inserts a copy of an other list at the back of list l.
// The lists l and other may be the same.
func (l *list) PushBackList(other *list) {
for i, e := other.Len(), other.Front(); i > 0; i, e = i-1, e.Next() {
l.insertValue(e.Value, l.root.prev)
}
}
// PushFrontList inserts a copy of an other list at the front of list l.
// The lists l and other may be the same.
func (l *list) PushFrontList(other *list) {
for i, e := other.Len(), other.Back(); i > 0; i, e = i-1, e.Prev() {
l.insertValue(e.Value, &l.root)
}
}
+25
View File
@@ -0,0 +1,25 @@
// Copyright (c) 2013 CloudFlare, Inc.
// Extensions to "container/list" that allowing reuse of Elements.
package lrucache
func (l *list) PushElementFront(e *element) *element {
return l.insert(e, &l.root)
}
func (l *list) PushElementBack(e *element) *element {
return l.insert(e, l.root.prev)
}
func (l *list) PopElementFront() *element {
el := l.Front()
l.Remove(el)
return el
}
func (l *list) PopFront() interface{} {
el := l.Front()
l.Remove(el)
return el.Value
}
+316
View File
@@ -0,0 +1,316 @@
// Copyright (c) 2013 CloudFlare, Inc.
package lrucache
import (
"container/heap"
"sync"
"time"
)
// Every element in the cache is linked to three data structures:
// Table map, PriorityQueue heap ordered by expiry and a LruList list
// ordered by decreasing popularity.
type entry struct {
element element // list element. value is a pointer to this entry
key string // key is a key!
value interface{} //
expire time.Time // time when the item is expired. it's okay to be stale.
index int // index for priority queue needs. -1 if entry is free
}
// LRUCache data structure. Never dereference it or copy it by
// value. Always use it through a pointer.
type LRUCache struct {
lock sync.Mutex
table map[string]*entry // all entries in table must be in lruList
priorityQueue priorityQueue // some elements from table may be in priorityQueue
lruList list // every entry is either used and resides in lruList
freeList list // or free and is linked to freeList
ExpireGracePeriod time.Duration // time after an expired entry is purged from cache (unless pushed out of LRU)
}
// Initialize the LRU cache instance. O(capacity)
func (b *LRUCache) Init(capacity uint) {
b.table = make(map[string]*entry, capacity)
b.priorityQueue = make([]*entry, 0, capacity)
b.lruList.Init()
b.freeList.Init()
heap.Init(&b.priorityQueue)
// Reserve all the entries in one giant continous block of memory
arrayOfEntries := make([]entry, capacity)
for i := uint(0); i < capacity; i++ {
e := &arrayOfEntries[i]
e.element.Value = e
e.index = -1
b.freeList.PushElementBack(&e.element)
}
}
// Create new LRU cache instance. Allocate all the needed memory. O(capacity)
func NewLRUCache(capacity uint) *LRUCache {
b := &LRUCache{}
b.Init(capacity)
return b
}
// Give me the entry with lowest expiry field if it's before now.
func (b *LRUCache) expiredEntry(now time.Time) *entry {
if len(b.priorityQueue) == 0 {
return nil
}
if now.IsZero() {
// Fill it only when actually used.
now = time.Now()
}
if e := b.priorityQueue[0]; e.expire.Before(now) {
return e
}
return nil
}
// Give me the least used entry.
func (b *LRUCache) leastUsedEntry() *entry {
return b.lruList.Back().Value.(*entry)
}
func (b *LRUCache) freeSomeEntry(now time.Time) (e *entry, used bool) {
if b.freeList.Len() > 0 {
return b.freeList.Front().Value.(*entry), false
}
e = b.expiredEntry(now)
if e != nil {
return e, true
}
if b.lruList.Len() == 0 {
return nil, false
}
return b.leastUsedEntry(), true
}
// Move entry from used/lru list to a free list. Clear the entry as well.
func (b *LRUCache) removeEntry(e *entry) {
if e.element.list != &b.lruList {
panic("list lruList")
}
if e.index != -1 {
heap.Remove(&b.priorityQueue, e.index)
}
b.lruList.Remove(&e.element)
b.freeList.PushElementFront(&e.element)
delete(b.table, e.key)
e.key = ""
e.value = nil
}
func (b *LRUCache) insertEntry(e *entry) {
if e.element.list != &b.freeList {
panic("list freeList")
}
if !e.expire.IsZero() {
heap.Push(&b.priorityQueue, e)
}
b.freeList.Remove(&e.element)
b.lruList.PushElementFront(&e.element)
b.table[e.key] = e
}
func (b *LRUCache) touchEntry(e *entry) {
b.lruList.MoveToFront(&e.element)
}
// Add an item to the cache overwriting existing one if it
// exists. Allows specifing current time required to expire an item
// when no more slots are used. O(log(n)) if expiry is set, O(1) when
// clear.
func (b *LRUCache) SetNow(key string, value interface{}, expire time.Time, now time.Time) {
b.lock.Lock()
defer b.lock.Unlock()
var used bool
e := b.table[key]
if e != nil {
used = true
} else {
e, used = b.freeSomeEntry(now)
if e == nil {
return
}
}
if used {
b.removeEntry(e)
}
e.key = key
e.value = value
e.expire = expire
b.insertEntry(e)
}
// Add an item to the cache overwriting existing one if it
// exists. O(log(n)) if expiry is set, O(1) when clear.
func (b *LRUCache) Set(key string, value interface{}, expire time.Time) {
b.SetNow(key, value, expire, time.Time{})
}
// Get a key from the cache, possibly stale. Update its LRU score. O(1)
func (b *LRUCache) Get(key string) (v interface{}, ok bool) {
b.lock.Lock()
defer b.lock.Unlock()
e := b.table[key]
if e == nil {
return nil, false
}
b.touchEntry(e)
return e.value, true
}
// Get a key from the cache, possibly stale. Don't modify its LRU score. O(1)
func (b *LRUCache) GetQuiet(key string) (v interface{}, ok bool) {
b.lock.Lock()
defer b.lock.Unlock()
e := b.table[key]
if e == nil {
return nil, false
}
return e.value, true
}
// Get a key from the cache, make sure it's not stale. Update its
// LRU score. O(log(n)) if the item is expired.
func (b *LRUCache) GetNotStale(key string) (value interface{}, ok bool) {
return b.GetNotStaleNow(key, time.Now())
}
// Get a key from the cache, make sure it's not stale. Update its
// LRU score. O(log(n)) if the item is expired.
func (b *LRUCache) GetNotStaleNow(key string, now time.Time) (value interface{}, ok bool) {
b.lock.Lock()
defer b.lock.Unlock()
e := b.table[key]
if e == nil {
return nil, false
}
if e.expire.Before(now) {
// Remove entries expired for more than a graceful period
if b.ExpireGracePeriod == 0 || e.expire.Sub(now) > b.ExpireGracePeriod {
b.removeEntry(e)
}
return nil, false
}
b.touchEntry(e)
return e.value, true
}
// Get a key from the cache, possibly stale. Update its LRU
// score. O(1) always.
func (b *LRUCache) GetStale(key string) (value interface{}, ok, expired bool) {
return b.GetStaleNow(key, time.Now())
}
// Get a key from the cache, possibly stale. Update its LRU
// score. O(1) always.
func (b *LRUCache) GetStaleNow(key string, now time.Time) (value interface{}, ok, expired bool) {
b.lock.Lock()
defer b.lock.Unlock()
e := b.table[key]
if e == nil {
return nil, false, false
}
b.touchEntry(e)
return e.value, true, e.expire.Before(now)
}
// Get and remove a key from the cache. O(log(n)) if the item is using expiry, O(1) otherwise.
func (b *LRUCache) Del(key string) (v interface{}, ok bool) {
b.lock.Lock()
defer b.lock.Unlock()
e := b.table[key]
if e == nil {
return nil, false
}
value := e.value
b.removeEntry(e)
return value, true
}
// Evict all items from the cache. O(n*log(n))
func (b *LRUCache) Clear() int {
b.lock.Lock()
defer b.lock.Unlock()
// First, remove entries that have expiry set
l := len(b.priorityQueue)
for i := 0; i < l; i++ {
// This could be reduced to O(n).
b.removeEntry(b.priorityQueue[0])
}
// Second, remove all remaining entries
r := b.lruList.Len()
for i := 0; i < r; i++ {
b.removeEntry(b.leastUsedEntry())
}
return l + r
}
// Evict all the expired items. O(n*log(n))
func (b *LRUCache) Expire() int {
return b.ExpireNow(time.Now())
}
// Evict items that expire before `now`. O(n*log(n))
func (b *LRUCache) ExpireNow(now time.Time) int {
b.lock.Lock()
defer b.lock.Unlock()
i := 0
for {
e := b.expiredEntry(now)
if e == nil {
break
}
b.removeEntry(e)
i += 1
}
return i
}
// Number of entries used in the LRU
func (b *LRUCache) Len() int {
// yes. this stupid thing requires locking
b.lock.Lock()
defer b.lock.Unlock()
return b.lruList.Len()
}
// Get the total capacity of the LRU
func (b *LRUCache) Capacity() int {
// yes. this stupid thing requires locking
b.lock.Lock()
defer b.lock.Unlock()
return b.lruList.Len() + b.freeList.Len()
}
+118
View File
@@ -0,0 +1,118 @@
// Copyright (c) 2013 CloudFlare, Inc.
package lrucache
import (
"hash/crc32"
"time"
)
// MultiLRUCache data structure. Never dereference it or copy it by
// value. Always use it through a pointer.
type MultiLRUCache struct {
buckets uint
cache []*LRUCache
}
// Using this constructor is almost always wrong. Use NewMultiLRUCache instead.
func (m *MultiLRUCache) Init(buckets, bucket_capacity uint) {
m.buckets = buckets
m.cache = make([]*LRUCache, buckets)
for i := uint(0); i < buckets; i++ {
m.cache[i] = NewLRUCache(bucket_capacity)
}
}
// Set the stale expiry grace period for each cache in the multicache instance.
func (m *MultiLRUCache) SetExpireGracePeriod(p time.Duration) {
for _, c := range m.cache {
c.ExpireGracePeriod = p
}
}
func NewMultiLRUCache(buckets, bucket_capacity uint) *MultiLRUCache {
m := &MultiLRUCache{}
m.Init(buckets, bucket_capacity)
return m
}
func (m *MultiLRUCache) bucketNo(key string) uint {
// Arbitrary choice. Any fast hash will do.
return uint(crc32.ChecksumIEEE([]byte(key))) % m.buckets
}
func (m *MultiLRUCache) Set(key string, value interface{}, expire time.Time) {
m.cache[m.bucketNo(key)].Set(key, value, expire)
}
func (m *MultiLRUCache) SetNow(key string, value interface{}, expire time.Time, now time.Time) {
m.cache[m.bucketNo(key)].SetNow(key, value, expire, now)
}
func (m *MultiLRUCache) Get(key string) (value interface{}, ok bool) {
return m.cache[m.bucketNo(key)].Get(key)
}
func (m *MultiLRUCache) GetQuiet(key string) (value interface{}, ok bool) {
return m.cache[m.bucketNo(key)].Get(key)
}
func (m *MultiLRUCache) GetNotStale(key string) (value interface{}, ok bool) {
return m.cache[m.bucketNo(key)].GetNotStale(key)
}
func (m *MultiLRUCache) GetNotStaleNow(key string, now time.Time) (value interface{}, ok bool) {
return m.cache[m.bucketNo(key)].GetNotStaleNow(key, now)
}
func (m *MultiLRUCache) GetStale(key string) (value interface{}, ok, expired bool) {
return m.cache[m.bucketNo(key)].GetStale(key)
}
func (m *MultiLRUCache) GetStaleNow(key string, now time.Time) (value interface{}, ok, expired bool) {
return m.cache[m.bucketNo(key)].GetStaleNow(key, now)
}
func (m *MultiLRUCache) Del(key string) (value interface{}, ok bool) {
return m.cache[m.bucketNo(key)].Del(key)
}
func (m *MultiLRUCache) Clear() int {
var s int
for _, c := range m.cache {
s += c.Clear()
}
return s
}
func (m *MultiLRUCache) Len() int {
var s int
for _, c := range m.cache {
s += c.Len()
}
return s
}
func (m *MultiLRUCache) Capacity() int {
var s int
for _, c := range m.cache {
s += c.Capacity()
}
return s
}
func (m *MultiLRUCache) Expire() int {
var s int
for _, c := range m.cache {
s += c.Expire()
}
return s
}
func (m *MultiLRUCache) ExpireNow(now time.Time) int {
var s int
for _, c := range m.cache {
s += c.ExpireNow(now)
}
return s
}
+37
View File
@@ -0,0 +1,37 @@
// Copyright (c) 2013 CloudFlare, Inc.
// This code is based on golang example from "container/heap" package.
package lrucache
type priorityQueue []*entry
func (pq priorityQueue) Len() int {
return len(pq)
}
func (pq priorityQueue) Less(i, j int) bool {
return pq[i].expire.Before(pq[j].expire)
}
func (pq priorityQueue) Swap(i, j int) {
pq[i], pq[j] = pq[j], pq[i]
pq[i].index = i
pq[j].index = j
}
func (pq *priorityQueue) Push(e interface{}) {
n := len(*pq)
item := e.(*entry)
item.index = n
*pq = append(*pq, item)
}
func (pq *priorityQueue) Pop() interface{} {
old := *pq
n := len(old)
item := old[n-1]
item.index = -1
*pq = old[0 : n-1]
return item
}
-4
View File
@@ -1,7 +1,3 @@
**This package is currently in development and the API may not be stable.**
The API will become stable with v1.
# uuid ![build status](https://travis-ci.org/google/uuid.svg?branch=master)
The uuid package generates and inspects UUIDs based on
[RFC 4122](http://tools.ietf.org/html/rfc4122)
+2 -2
View File
@@ -42,7 +42,7 @@ func NewDCESecurity(domain Domain, id uint32) (UUID, error) {
// NewDCEPerson returns a DCE Security (Version 2) UUID in the person
// domain with the id returned by os.Getuid.
//
// NewDCEPerson(Person, uint32(os.Getuid()))
// NewDCESecurity(Person, uint32(os.Getuid()))
func NewDCEPerson() (UUID, error) {
return NewDCESecurity(Person, uint32(os.Getuid()))
}
@@ -50,7 +50,7 @@ func NewDCEPerson() (UUID, error) {
// NewDCEGroup returns a DCE Security (Version 2) UUID in the group
// domain with the id returned by os.Getgid.
//
// NewDCEGroup(Group, uint32(os.Getgid()))
// NewDCESecurity(Group, uint32(os.Getgid()))
func NewDCEGroup() (UUID, error) {
return NewDCESecurity(Group, uint32(os.Getgid()))
}
+1
View File
@@ -0,0 +1 @@
module github.com/google/uuid
+1 -1
View File
@@ -27,7 +27,7 @@ var (
func NewHash(h hash.Hash, space UUID, data []byte, version int) UUID {
h.Reset()
h.Write(space[:])
h.Write([]byte(data))
h.Write(data)
s := h.Sum(nil)
var uuid UUID
copy(uuid[:], s)
-2
View File
@@ -15,8 +15,6 @@ func (uuid UUID) MarshalText() ([]byte, error) {
// UnmarshalText implements encoding.TextUnmarshaler.
func (uuid *UUID) UnmarshalText(data []byte) error {
// See comment in ParseBytes why we do this.
// id, err := ParseBytes(data)
id, err := ParseBytes(data)
if err == nil {
*uuid = id
+10 -23
View File
@@ -5,16 +5,14 @@
package uuid
import (
"net"
"sync"
)
var (
nodeMu sync.Mutex
interfaces []net.Interface // cached list of interfaces
ifname string // name of interface being used
nodeID [6]byte // hardware for version 1 UUIDs
zeroID [6]byte // nodeID with only 0's
nodeMu sync.Mutex
ifname string // name of interface being used
nodeID [6]byte // hardware for version 1 UUIDs
zeroID [6]byte // nodeID with only 0's
)
// NodeInterface returns the name of the interface from which the NodeID was
@@ -39,26 +37,18 @@ func SetNodeInterface(name string) bool {
}
func setNodeInterface(name string) bool {
if interfaces == nil {
var err error
interfaces, err = net.Interfaces()
if err != nil && name != "" {
return false
}
}
for _, ifs := range interfaces {
if len(ifs.HardwareAddr) >= 6 && (name == "" || name == ifs.Name) {
copy(nodeID[:], ifs.HardwareAddr)
ifname = ifs.Name
return true
}
iname, addr := getHardwareInterface(name) // null implementation for js
if iname != "" && addr != nil {
ifname = iname
copy(nodeID[:], addr)
return true
}
// We found no interfaces with a valid hardware address. If name
// does not specify a specific interface generate a random Node ID
// (section 4.1.6)
if name == "" {
ifname = "random"
randomBits(nodeID[:])
return true
}
@@ -94,9 +84,6 @@ func SetNodeID(id []byte) bool {
// NodeID returns the 6 byte node id encoded in uuid. It returns nil if uuid is
// not valid. The NodeID is only well defined for version 1 and 2 UUIDs.
func (uuid UUID) NodeID() []byte {
if len(uuid) != 16 {
return nil
}
var node [6]byte
copy(node[:], uuid[10:])
return node[:]
+12
View File
@@ -0,0 +1,12 @@
// Copyright 2017 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build js
package uuid
// getHardwareInterface returns nil values for the JS version of the code.
// This remvoves the "net" dependency, because it is not used in the browser.
// Using the "net" library inflates the size of the transpiled JS code by 673k bytes.
func getHardwareInterface(name string) (string, []byte) { return "", nil }
+33
View File
@@ -0,0 +1,33 @@
// Copyright 2017 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !js
package uuid
import "net"
var interfaces []net.Interface // cached list of interfaces
// getHardwareInterface returns the name and hardware address of interface name.
// If name is "" then the name and hardware address of one of the system's
// interfaces is returned. If no interfaces are found (name does not exist or
// there are no interfaces) then "", nil is returned.
//
// Only addresses of at least 6 bytes are returned.
func getHardwareInterface(name string) (string, []byte) {
if interfaces == nil {
var err error
interfaces, err = net.Interfaces()
if err != nil {
return "", nil
}
}
for _, ifs := range interfaces {
if len(ifs.HardwareAddr) >= 6 && (name == "" || name == ifs.Name) {
return ifs.Name, ifs.HardwareAddr
}
}
return "", nil
}
+11 -10
View File
@@ -13,35 +13,36 @@ import (
// Currently, database types that map to string and []byte are supported. Please
// consult database-specific driver documentation for matching types.
func (uuid *UUID) Scan(src interface{}) error {
switch src.(type) {
switch src := src.(type) {
case nil:
return nil
case string:
// if an empty UUID comes from a table, we return a null UUID
if src.(string) == "" {
if src == "" {
return nil
}
// see Parse for required string format
u, err := Parse(src.(string))
u, err := Parse(src)
if err != nil {
return fmt.Errorf("Scan: %v", err)
}
*uuid = u
case []byte:
b := src.([]byte)
case []byte:
// if an empty UUID comes from a table, we return a null UUID
if len(b) == 0 {
if len(src) == 0 {
return nil
}
// assumes a simple slice of bytes if 16 bytes
// otherwise attempts to parse
if len(b) != 16 {
return uuid.Scan(string(b))
if len(src) != 16 {
return uuid.Scan(string(src))
}
copy((*uuid)[:], b)
copy((*uuid)[:], src)
default:
return fmt.Errorf("Scan: unable to scan type %T into UUID", src)
+3 -3
View File
@@ -86,7 +86,7 @@ func clockSequence() int {
return int(clockSeq & 0x3fff)
}
// SetClockSeq sets the clock sequence to the lower 14 bits of seq. Setting to
// SetClockSequence sets the clock sequence to the lower 14 bits of seq. Setting to
// -1 causes a new sequence to be generated.
func SetClockSequence(seq int) {
defer timeMu.Unlock()
@@ -100,9 +100,9 @@ func setClockSequence(seq int) {
randomBits(b[:]) // clock sequence
seq = int(b[0])<<8 | int(b[1])
}
old_seq := clockSeq
oldSeq := clockSeq
clockSeq = uint16(seq&0x3fff) | 0x8000 // Set our variant
if old_seq != clockSeq {
if oldSeq != clockSeq {
lasttime = 0
}
}
+74 -20
View File
@@ -1,4 +1,4 @@
// Copyright 2016 Google Inc. All rights reserved.
// Copyright 2018 Google Inc. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
@@ -35,20 +35,43 @@ const (
var rander = rand.Reader // random function
// Parse decodes s into a UUID or returns an error. Both the UUID form of
// xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx and
// urn:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx are decoded.
// Parse decodes s into a UUID or returns an error. Both the standard UUID
// forms of xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx and
// urn:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx are decoded as well as the
// Microsoft encoding {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx} and the raw hex
// encoding: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
func Parse(s string) (UUID, error) {
var uuid UUID
if len(s) != 36 {
if len(s) != 36+9 {
return uuid, fmt.Errorf("invalid UUID length: %d", len(s))
}
switch len(s) {
// xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
case 36:
// urn:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
case 36 + 9:
if strings.ToLower(s[:9]) != "urn:uuid:" {
return uuid, fmt.Errorf("invalid urn prefix: %q", s[:9])
}
s = s[9:]
// {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}
case 36 + 2:
s = s[1:]
// xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
case 32:
var ok bool
for i := range uuid {
uuid[i], ok = xtob(s[i*2], s[i*2+1])
if !ok {
return uuid, errors.New("invalid UUID format")
}
}
return uuid, nil
default:
return uuid, fmt.Errorf("invalid UUID length: %d", len(s))
}
// s is now at least 36 bytes long
// it must be of the form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
if s[8] != '-' || s[13] != '-' || s[18] != '-' || s[23] != '-' {
return uuid, errors.New("invalid UUID format")
}
@@ -58,11 +81,11 @@ func Parse(s string) (UUID, error) {
14, 16,
19, 21,
24, 26, 28, 30, 32, 34} {
if v, ok := xtob(s[x], s[x+1]); !ok {
v, ok := xtob(s[x], s[x+1])
if !ok {
return uuid, errors.New("invalid UUID format")
} else {
uuid[i] = v
}
uuid[i] = v
}
return uuid, nil
}
@@ -70,15 +93,29 @@ func Parse(s string) (UUID, error) {
// ParseBytes is like Parse, except it parses a byte slice instead of a string.
func ParseBytes(b []byte) (UUID, error) {
var uuid UUID
if len(b) != 36 {
if len(b) != 36+9 {
return uuid, fmt.Errorf("invalid UUID length: %d", len(b))
}
switch len(b) {
case 36: // xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
case 36 + 9: // urn:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
if !bytes.Equal(bytes.ToLower(b[:9]), []byte("urn:uuid:")) {
return uuid, fmt.Errorf("invalid urn prefix: %q", b[:9])
}
b = b[9:]
case 36 + 2: // {xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}
b = b[1:]
case 32: // xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
var ok bool
for i := 0; i < 32; i += 2 {
uuid[i/2], ok = xtob(b[i], b[i+1])
if !ok {
return uuid, errors.New("invalid UUID format")
}
}
return uuid, nil
default:
return uuid, fmt.Errorf("invalid UUID length: %d", len(b))
}
// s is now at least 36 bytes long
// it must be of the form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
if b[8] != '-' || b[13] != '-' || b[18] != '-' || b[23] != '-' {
return uuid, errors.New("invalid UUID format")
}
@@ -88,15 +125,32 @@ func ParseBytes(b []byte) (UUID, error) {
14, 16,
19, 21,
24, 26, 28, 30, 32, 34} {
if v, ok := xtob(b[x], b[x+1]); !ok {
v, ok := xtob(b[x], b[x+1])
if !ok {
return uuid, errors.New("invalid UUID format")
} else {
uuid[i] = v
}
uuid[i] = v
}
return uuid, nil
}
// MustParse is like Parse but panics if the string cannot be parsed.
// It simplifies safe initialization of global variables holding compiled UUIDs.
func MustParse(s string) UUID {
uuid, err := Parse(s)
if err != nil {
panic(`uuid: Parse(` + s + `): ` + err.Error())
}
return uuid
}
// FromBytes creates a new UUID from a byte slice. Returns an error if the slice
// does not have a length of 16. The bytes are copied from the slice.
func FromBytes(b []byte) (uuid UUID, err error) {
err = uuid.UnmarshalBinary(b)
return uuid, err
}
// Must returns uuid if err is nil and panics otherwise.
func Must(uuid UUID, err error) UUID {
if err != nil {
@@ -123,7 +177,7 @@ func (uuid UUID) URN() string {
}
func encodeHex(dst []byte, uuid UUID) {
hex.Encode(dst[:], uuid[:4])
hex.Encode(dst, uuid[:4])
dst[8] = '-'
hex.Encode(dst[9:13], uuid[4:6])
dst[13] = '-'
@@ -176,7 +230,7 @@ func (v Variant) String() string {
return fmt.Sprintf("BadVariant%d", int(v))
}
// SetRand sets the random number generator to r, which implents io.Reader.
// SetRand sets the random number generator to r, which implements io.Reader.
// If r.Read returns an error when the package requests random data then
// a panic will be issued.
//
+1 -1
View File
@@ -13,7 +13,7 @@ import (
// or SetNodeInterface then it will be set automatically. If the NodeID cannot
// be set NewUUID returns nil. If clock sequence has not been set by
// SetClockSequence then it will be set automatically. If GetTime fails to
// return the current NewUUID returns Nil and an error.
// return the current NewUUID returns nil and an error.
//
// In most cases, New should be used.
func NewUUID() (UUID, error) {
+3 -3
View File
@@ -6,7 +6,7 @@ package uuid
import "io"
// New is creates a new random UUID or panics. New is equivalent to
// New creates a new random UUID or panics. New is equivalent to
// the expression
//
// uuid.Must(uuid.NewRandom())
@@ -14,12 +14,12 @@ func New() UUID {
return Must(NewRandom())
}
// NewRandom returns a Random (Version 4) UUID or panics.
// NewRandom returns a Random (Version 4) UUID.
//
// The strength of the UUIDs is based on the strength of the crypto/rand
// package.
//
// A note about uniqueness derived from from the UUID Wikipedia entry:
// A note about uniqueness derived from the UUID Wikipedia entry:
//
// Randomly generated UUIDs have 122 random bits. One's annual risk of being
// hit by a meteorite is estimated to be one chance in 17 billion, that
-8
View File
@@ -1,8 +0,0 @@
coverage:
status:
project:
default:
target: 40%
threshold: null
patch: false
changes: false
+1 -7
View File
@@ -7,15 +7,9 @@
packages = ["ed25519","ed25519/internal/edwards25519"]
revision = "b080dc9a8c480b08e698fb1219160d598526310f"
[[projects]]
branch = "master"
name = "golang.org/x/net"
packages = ["bpf","internal/iana","internal/socket","ipv4","ipv6"]
revision = "894f8ed5849b15b810ae41e9590a0d05395bba27"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
inputs-digest = "c4abc38abaeeeeb9be92455c9c02cae32841122b8982aaa067ef25bb8e86ff9d"
inputs-digest = "5046e265393bd5e54f570ce29ae8bc6fa3f30ef5110e922996540400f287c64a"
solver-name = "gps-cdcl"
solver-version = 1
-52
View File
@@ -1,52 +0,0 @@
# Makefile for releasing.
#
# The release is controlled from version.go. The version found there is
# used to tag the git repo, we're not building any artifects so there is nothing
# to upload to github.
#
# * Up the version in version.go
# * Run: make -f Makefile.release release
# * will *commit* your change with 'Release $VERSION'
# * push to github
#
define GO
//+build ignore
package main
import (
"fmt"
"github.com/miekg/dns"
)
func main() {
fmt.Println(dns.Version.String())
}
endef
$(file > version_release.go,$(GO))
VERSION:=$(shell go run version_release.go)
TAG="v$(VERSION)"
all:
@echo Use the \'release\' target to start a release $(VERSION)
rm -f version_release.go
.PHONY: release
release: commit push
@echo Released $(VERSION)
rm -f version_release.go
.PHONY: commit
commit:
@echo Committing release $(VERSION)
git commit -am"Release $(VERSION)"
git tag $(TAG)
.PHONY: push
push:
@echo Pushing release $(VERSION) to master
git push --tags
git push
-2
View File
@@ -62,8 +62,6 @@ A not-so-up-to-date-list-that-may-be-actually-current:
* https://dnssectest.net/
* https://dns.apebits.com
* https://github.com/oif/apex
* https://github.com/jedisct1/dnscrypt-proxy
* https://github.com/jedisct1/rpdns
Send pull request if you want to be listed here.
+19 -98
View File
@@ -7,27 +7,22 @@ import (
"context"
"crypto/tls"
"encoding/binary"
"fmt"
"io"
"io/ioutil"
"net"
"net/http"
"strings"
"time"
)
const (
dnsTimeout time.Duration = 2 * time.Second
tcpIdleTimeout time.Duration = 8 * time.Second
dohMimeType = "application/dns-message"
)
const dnsTimeout time.Duration = 2 * time.Second
const tcpIdleTimeout time.Duration = 8 * time.Second
// A Conn represents a connection to a DNS server.
type Conn struct {
net.Conn // a net.Conn holding the connection
UDPSize uint16 // minimum receive buffer for UDP messages
TsigSecret map[string]string // secret(s) for Tsig map[<zonename>]<base64 secret>, zonename must be in canonical form (lowercase, fqdn, see RFC 4034 Section 6.2)
rtt time.Duration
t time.Time
tsigRequestMAC string
}
@@ -44,7 +39,6 @@ type Client struct {
DialTimeout time.Duration // net.DialTimeout, defaults to 2 seconds, or net.Dialer.Timeout if expiring earlier - overridden by Timeout when that value is non-zero
ReadTimeout time.Duration // net.Conn.SetReadTimeout value for connections, defaults to 2 seconds - overridden by Timeout when that value is non-zero
WriteTimeout time.Duration // net.Conn.SetWriteTimeout value for connections, defaults to 2 seconds - overridden by Timeout when that value is non-zero
HTTPClient *http.Client // The http.Client to use for DNS-over-HTTPS
TsigSecret map[string]string // secret(s) for Tsig map[<zonename>]<base64 secret>, zonename must be in canonical form (lowercase, fqdn, see RFC 4034 Section 6.2)
SingleInflight bool // if true suppress multiple outstanding queries for the same Qname, Qtype and Qclass
group singleflight
@@ -89,10 +83,11 @@ func (c *Client) Dial(address string) (conn *Conn, err error) {
// create a new dialer with the appropriate timeout
var d net.Dialer
if c.Dialer == nil {
d = net.Dialer{Timeout:c.getTimeoutForRequest(c.dialTimeout())}
d = net.Dialer{}
} else {
d = net.Dialer(*c.Dialer)
}
d.Timeout = c.getTimeoutForRequest(c.writeTimeout())
network := "udp"
useTLS := false
@@ -141,11 +136,6 @@ func (c *Client) Dial(address string) (conn *Conn, err error) {
// attribute appropriately
func (c *Client) Exchange(m *Msg, address string) (r *Msg, rtt time.Duration, err error) {
if !c.SingleInflight {
if c.Net == "https" {
// TODO(tmthrgd): pipe timeouts into exchangeDOH
return c.exchangeDOH(context.TODO(), m, address)
}
return c.exchange(m, address)
}
@@ -158,11 +148,6 @@ func (c *Client) Exchange(m *Msg, address string) (r *Msg, rtt time.Duration, er
cl = cl1
}
r, rtt, err, shared := c.group.Do(m.Question[0].Name+t+cl, func() (*Msg, time.Duration, error) {
if c.Net == "https" {
// TODO(tmthrgd): pipe timeouts into exchangeDOH
return c.exchangeDOH(context.TODO(), m, address)
}
return c.exchange(m, address)
})
if r != nil && shared {
@@ -192,9 +177,8 @@ func (c *Client) exchange(m *Msg, a string) (r *Msg, rtt time.Duration, err erro
}
co.TsigSecret = c.TsigSecret
t := time.Now()
// write with the appropriate write timeout
co.SetWriteDeadline(t.Add(c.getTimeoutForRequest(c.writeTimeout())))
co.SetWriteDeadline(time.Now().Add(c.getTimeoutForRequest(c.writeTimeout())))
if err = co.WriteMsg(m); err != nil {
return nil, 0, err
}
@@ -204,76 +188,12 @@ func (c *Client) exchange(m *Msg, a string) (r *Msg, rtt time.Duration, err erro
if err == nil && r.Id != m.Id {
err = ErrId
}
rtt = time.Since(t)
return r, rtt, err
}
func (c *Client) exchangeDOH(ctx context.Context, m *Msg, a string) (r *Msg, rtt time.Duration, err error) {
p, err := m.Pack()
if err != nil {
return nil, 0, err
}
req, err := http.NewRequest(http.MethodPost, a, bytes.NewReader(p))
if err != nil {
return nil, 0, err
}
req.Header.Set("Content-Type", dohMimeType)
req.Header.Set("Accept", dohMimeType)
hc := http.DefaultClient
if c.HTTPClient != nil {
hc = c.HTTPClient
}
if ctx != context.Background() && ctx != context.TODO() {
req = req.WithContext(ctx)
}
t := time.Now()
resp, err := hc.Do(req)
if err != nil {
return nil, 0, err
}
defer closeHTTPBody(resp.Body)
if resp.StatusCode != http.StatusOK {
return nil, 0, fmt.Errorf("dns: server returned HTTP %d error: %q", resp.StatusCode, resp.Status)
}
if ct := resp.Header.Get("Content-Type"); ct != dohMimeType {
return nil, 0, fmt.Errorf("dns: unexpected Content-Type %q; expected %q", ct, dohMimeType)
}
p, err = ioutil.ReadAll(resp.Body)
if err != nil {
return nil, 0, err
}
rtt = time.Since(t)
r = new(Msg)
if err := r.Unpack(p); err != nil {
return r, 0, err
}
// TODO: TSIG? Is it even supported over DoH?
return r, rtt, nil
}
func closeHTTPBody(r io.ReadCloser) error {
io.Copy(ioutil.Discard, io.LimitReader(r, 8<<20))
return r.Close()
return r, co.rtt, err
}
// ReadMsg reads a message from the connection co.
// If the received message contains a TSIG record the transaction signature
// is verified. This method always tries to return the message, however if an
// error is returned there are no guarantees that the returned message is a
// valid representation of the packet read.
// If the received message contains a TSIG record the transaction
// signature is verified.
func (co *Conn) ReadMsg() (*Msg, error) {
p, err := co.ReadMsgHeader(nil)
if err != nil {
@@ -282,10 +202,13 @@ func (co *Conn) ReadMsg() (*Msg, error) {
m := new(Msg)
if err := m.Unpack(p); err != nil {
// If an error was returned, we still want to allow the user to use
// If ErrTruncated was returned, we still want to allow the user to use
// the message, but naively they can just check err if they don't want
// to use an erroneous message
return m, err
// to use a truncated message
if err == ErrTruncated {
return m, err
}
return nil, err
}
if t := m.IsTsig(); t != nil {
if _, ok := co.TsigSecret[t.Hdr.Name]; !ok {
@@ -318,6 +241,7 @@ func (co *Conn) ReadMsgHeader(hdr *Header) ([]byte, error) {
}
p = make([]byte, l)
n, err = tcpRead(r, p)
co.rtt = time.Since(co.t)
default:
if co.UDPSize > MinMsgSize {
p = make([]byte, co.UDPSize)
@@ -325,6 +249,7 @@ func (co *Conn) ReadMsgHeader(hdr *Header) ([]byte, error) {
p = make([]byte, MinMsgSize)
}
n, err = co.Read(p)
co.rtt = time.Since(co.t)
}
if err != nil {
@@ -437,6 +362,7 @@ func (co *Conn) WriteMsg(m *Msg) (err error) {
if err != nil {
return err
}
co.t = time.Now()
if _, err = co.Write(out); err != nil {
return err
}
@@ -568,10 +494,6 @@ func DialTimeoutWithTLS(network, address string, tlsConfig *tls.Config, timeout
// context, if present. If there is both a context deadline and a configured
// timeout on the client, the earliest of the two takes effect.
func (c *Client) ExchangeContext(ctx context.Context, m *Msg, a string) (r *Msg, rtt time.Duration, err error) {
if !c.SingleInflight && c.Net == "https" {
return c.exchangeDOH(ctx, m, a)
}
var timeout time.Duration
if deadline, ok := ctx.Deadline(); !ok {
timeout = 0
@@ -580,7 +502,6 @@ func (c *Client) ExchangeContext(ctx context.Context, m *Msg, a string) (r *Msg,
}
// not passing the context to the underlying calls, as the API does not support
// context. For timeouts you should set up Client.Dialer and call Client.Exchange.
// TODO(tmthrgd): this is a race condition
c.Dialer = &net.Dialer{Timeout: timeout}
return c.Exchange(m, a)
}
+1 -1
View File
@@ -91,7 +91,7 @@ func ClientConfigFromReader(resolvconf io.Reader) (*ClientConfig, error) {
n = 1
}
c.Timeout = n
case len(s) >= 9 && s[:9] == "attempts:":
case len(s) >= 8 && s[:9] == "attempts:":
n, _ := strconv.Atoi(s[9:])
if n < 1 {
n = 1
+11 -20
View File
@@ -18,7 +18,8 @@ import (
)
var packageHdr = `
// Code generated by "go run compress_generate.go"; DO NOT EDIT.
// *** DO NOT MODIFY ***
// AUTOGENERATED BY go generate from compress_generate.go
package dns
@@ -101,8 +102,7 @@ Names:
// compressionLenHelperType - all types that have domain-name/cdomain-name can be used for compressing names
fmt.Fprint(b, "func compressionLenHelperType(c map[string]int, r RR, initLen int) int {\n")
fmt.Fprint(b, "currentLen := initLen\n")
fmt.Fprint(b, "func compressionLenHelperType(c map[string]int, r RR) {\n")
fmt.Fprint(b, "switch x := r.(type) {\n")
for _, name := range domainTypes {
o := scope.Lookup(name)
@@ -110,10 +110,7 @@ Names:
fmt.Fprintf(b, "case *%s:\n", name)
for i := 1; i < st.NumFields(); i++ {
out := func(s string) {
fmt.Fprintf(b, "currentLen -= len(x.%s) + 1\n", st.Field(i).Name())
fmt.Fprintf(b, "currentLen += compressionLenHelper(c, x.%s, currentLen)\n", st.Field(i).Name())
}
out := func(s string) { fmt.Fprintf(b, "compressionLenHelper(c, x.%s)\n", st.Field(i).Name()) }
if _, ok := st.Field(i).Type().(*types.Slice); ok {
switch st.Tag(i) {
@@ -122,12 +119,8 @@ Names:
case `dns:"cdomain-name"`:
// For HIP we need to slice over the elements in this slice.
fmt.Fprintf(b, `for i := range x.%s {
currentLen -= len(x.%s[i]) + 1
}
`, st.Field(i).Name(), st.Field(i).Name())
fmt.Fprintf(b, `for i := range x.%s {
currentLen += compressionLenHelper(c, x.%s[i], currentLen)
}
compressionLenHelper(c, x.%s[i])
}
`, st.Field(i).Name(), st.Field(i).Name())
}
continue
@@ -141,11 +134,11 @@ Names:
}
}
}
fmt.Fprintln(b, "}\nreturn currentLen - initLen\n}\n\n")
fmt.Fprintln(b, "}\n}\n\n")
// compressionLenSearchType - search cdomain-tags types for compressible names.
fmt.Fprint(b, "func compressionLenSearchType(c map[string]int, r RR) (int, bool, int) {\n")
fmt.Fprint(b, "func compressionLenSearchType(c map[string]int, r RR) (int, bool) {\n")
fmt.Fprint(b, "switch x := r.(type) {\n")
for _, name := range cdomainTypes {
o := scope.Lookup(name)
@@ -155,7 +148,7 @@ Names:
j := 1
for i := 1; i < st.NumFields(); i++ {
out := func(s string, j int) {
fmt.Fprintf(b, "k%d, ok%d, sz%d := compressionLenSearch(c, x.%s)\n", j, j, j, st.Field(i).Name())
fmt.Fprintf(b, "k%d, ok%d := compressionLenSearch(c, x.%s)\n", j, j, st.Field(i).Name())
}
// There are no slice types with names that can be compressed.
@@ -168,15 +161,13 @@ Names:
}
k := "k1"
ok := "ok1"
sz := "sz1"
for i := 2; i < j; i++ {
k += fmt.Sprintf(" + k%d", i)
ok += fmt.Sprintf(" && ok%d", i)
sz += fmt.Sprintf(" + sz%d", i)
}
fmt.Fprintf(b, "return %s, %s, %s\n", k, ok, sz)
fmt.Fprintf(b, "return %s, %s\n", k, ok)
}
fmt.Fprintln(b, "}\nreturn 0, false, 0\n}\n\n")
fmt.Fprintln(b, "}\nreturn 0, false\n}\n\n")
// gofmt
res, err := format.Source(b.Bytes())
+2 -5
View File
@@ -273,11 +273,8 @@ func (t Type) String() string {
// String returns the string representation for the class c.
func (c Class) String() string {
if s, ok := ClassToString[uint16(c)]; ok {
// Only emit mnemonics when they are unambiguous, specically ANY is in both.
if _, ok := StringToType[s]; !ok {
return s
}
if c1, ok := ClassToString[uint16(c)]; ok {
return c1
}
return "CLASS" + strconv.Itoa(int(c))
}
+10
View File
@@ -55,6 +55,16 @@ func (h *RR_Header) Header() *RR_Header { return h }
// Just to implement the RR interface.
func (h *RR_Header) copy() RR { return nil }
func (h *RR_Header) copyHeader() *RR_Header {
r := new(RR_Header)
r.Name = h.Name
r.Rrtype = h.Rrtype
r.Class = h.Class
r.Ttl = h.Ttl
r.Rdlength = h.Rdlength
return r
}
func (h *RR_Header) String() string {
var s string
+11 -12
View File
@@ -73,7 +73,6 @@ var StringToAlgorithm = reverseInt8(AlgorithmToString)
// AlgorithmToHash is a map of algorithm crypto hash IDs to crypto.Hash's.
var AlgorithmToHash = map[uint8]crypto.Hash{
RSAMD5: crypto.MD5, // Deprecated in RFC 6725
DSA: crypto.SHA1,
RSASHA1: crypto.SHA1,
RSASHA1NSEC3SHA1: crypto.SHA1,
RSASHA256: crypto.SHA256,
@@ -240,7 +239,7 @@ func (k *DNSKEY) ToDS(h uint8) *DS {
// ToCDNSKEY converts a DNSKEY record to a CDNSKEY record.
func (k *DNSKEY) ToCDNSKEY() *CDNSKEY {
c := &CDNSKEY{DNSKEY: *k}
c.Hdr = k.Hdr
c.Hdr = *k.Hdr.copyHeader()
c.Hdr.Rrtype = TypeCDNSKEY
return c
}
@@ -248,7 +247,7 @@ func (k *DNSKEY) ToCDNSKEY() *CDNSKEY {
// ToCDS converts a DS record to a CDS record.
func (d *DS) ToCDS() *CDS {
c := &CDS{DS: *d}
c.Hdr = d.Hdr
c.Hdr = *d.Hdr.copyHeader()
c.Hdr.Rrtype = TypeCDS
return c
}
@@ -542,20 +541,20 @@ func (k *DNSKEY) publicKeyRSA() *rsa.PublicKey {
explen = uint16(keybuf[1])<<8 | uint16(keybuf[2])
keyoff = 3
}
if explen > 4 {
// Larger exponent than supported by the crypto package.
return nil
}
pubkey := new(rsa.PublicKey)
pubkey.N = big.NewInt(0)
shift := uint64((explen - 1) * 8)
expo := uint64(0)
for i := 0; i < int(explen); i++ {
expo <<= 8
expo |= uint64(keybuf[keyoff+i])
for i := int(explen - 1); i > 0; i-- {
expo += uint64(keybuf[keyoff+i]) << shift
shift -= 8
}
if expo > 1<<31-1 {
// Larger exponent than supported by the crypto package.
// Remainder
expo += uint64(keybuf[keyoff])
if expo > (2<<31)+1 {
// Larger expo than supported.
// println("dns: F5 primes (or larger) are not supported")
return nil
}
pubkey.E = int(expo)
+2 -2
View File
@@ -73,11 +73,11 @@ and port to use for the connection:
Port: 12345,
Zone: "",
}
c.Dialer := &net.Dialer{
d := net.Dialer{
Timeout: 200 * time.Millisecond,
LocalAddr: &laddr,
}
in, rtt, err := c.Exchange(m1, "8.8.8.8:53")
in, rtt, err := c.ExchangeWithDialer(&d, m1, "8.8.8.8:53")
If these "advanced" features are not needed, a simple UDP query can be sent,
with:

Some files were not shown because too many files have changed in this diff Show More