Compare commits

..

37 Commits

Author SHA1 Message Date
Devin Carr 7ecb6d3e88 Release 2023.4.2 2023-04-24 12:48:58 -07:00
Devin Carr 88c25d2c67 TUN-7133: Add sampling support for streaming logs
In addition to supporting sampling support for streaming logs,
cloudflared tail also supports this via `--sample 0.5` to sample 50%
of your log events.
2023-04-24 09:39:26 -07:00
Devin Carr 38cd455e4d TUN-7373: Streaming logs override for same actor
To help accommodate web browser interactions with websockets, when a
streaming logs session is requested for the same actor while already
serving a session for that user in a separate request, the original
request will be closed and the new request start streaming logs
instead. This should help with rogue sessions holding on for too long
with no client on the other side (before idle timeout or connection
close).
2023-04-21 11:54:37 -07:00
Devin Carr ee5e447d44 TUN-7141: Add component tests for streaming logs 2023-04-21 10:14:03 -07:00
Sudarsan Reddy 4d30a71434 TUN-7383: Bump requirements.txt 2023-04-20 16:49:26 +01:00
Jesse Li 39b7aed24e AUTH-4887 Add aud parameter to token transfer url 2023-04-19 21:01:24 +00:00
Devin Carr 4de1bc4bba TUN-7378: Remove RPC debug logs 2023-04-19 18:35:51 +00:00
Sudarsan Reddy e426693330 TUN-7361: Add a label to override hostname
It might make sense for users to sometimes name their cloudflared
connectors to make identification easier than relying on hostnames that
TUN-7360 provides. This PR provides a new --label option to cloudflared
tunnel that a user could provide to give custom names to their
connectors.
2023-04-19 13:56:32 +00:00
Devin Carr 0b5b9b8297 TUN-7130: Categorize UDP logs for streaming logs 2023-04-18 20:49:36 +00:00
Devin Carr 7a014b06ec TUN-7129: Categorize TCP logs for streaming logs 2023-04-18 20:49:29 +00:00
James Royal 171d4ac77c AUTH-3122 Verify that Access tokens are still valid in curl command
Before this change, the only sure fire way to make sure you had a valid
Access token was to run `cloudflared access login <your domain>`. That
was because that command would actually make a preflight request to ensure
that the edge considered that token valid. The most common reasons a token
was no longer valid was expiration and revocation. Expiration is easy to
check client side, but revocation can only be checked at the edge.

This change adds the same flow that cfd access login did to the curl command.
It will preflight the request with the token and ensure that the edge thinks
its valid before making the real request.
2023-04-18 13:38:50 +00:00
Sudarsan Reddy 5e212a6bf3 TUN-7360: Add Get Host Details handler in management service
With the management tunnels work, we allow calls to our edge service
   using an access JWT provided by Tunnelstore. Given a connector ID,
   this request is then proxied to the appropriate Cloudflare Tunnel.

   This PR takes advantage of this flow and adds a new host_details
   endpoint. Calls to this endpoint will result in cloudflared gathering
   some details about the host: hostname (os.hostname()) and ip address
   (localAddr in a dial).

   Note that the mini spec lists 4 alternatives and this picks alternative
   3 because:

   1. Ease of implementation: This is quick and non-intrusive to any of our
      code path. We expect to change how connection tracking works and
      regardless of the direction we take, it may be easy to keep, morph
      or throw this away.

   2. The cloudflared part of this round trip takes some time with a
      hostname call and a dial. But note that this is off the critical path
      and not an API that will be exercised often.
2023-04-18 09:54:54 +00:00
Devin Carr 3996b1adca Release 2023.4.1 2023-04-17 10:04:12 -07:00
Devin Carr 71997be90e TUN-7368: Report destination address for TCP requests in logs 2023-04-13 16:49:42 -07:00
Devin Carr 991f01fe34 TUN-7131: Add cloudflared log event to connection messages and enable streaming logs 2023-04-12 14:41:11 -07:00
Devin Carr b89c092c1b TUN-7134: Acquire token for cloudflared tail
cloudflared tail will now fetch the management token from by making
a request to the Cloudflare API using the cert.pem (acquired from
cloudflared login).

Refactored some of the credentials code into it's own package as
to allow for easier use between subcommands outside of
`cloudflared tunnel`.
2023-04-12 09:43:38 -07:00
Devin Carr 8dc0697a8f TUN-7132 TUN-7136: Add filter support for streaming logs
Additionally adds similar support in cloudflared tail to provide
filters for events and log level.
2023-04-11 20:20:52 +00:00
Sudarsan Reddy 5dbf76a7aa TUN-7335: Fix cloudflared update not working in windows
This PR fixes some long standing bugs in the windows update
paths. We previously did not surface the errors at all leading to
this function failing silently.

This PR:

1. Now returns the ExitError if the bat run for update fails.
2. Fixes the errors surfaced by that return:
    a. The batch file doesnt play well with spaces. This is fixed by
    using PROGRA~1/2 which are aliases windows uses.
    b. The existing script also seemed to be irregular about where batch
    files were put and looked for. This is also fixed in this script.
2023-04-11 08:54:38 +00:00
Devin Carr 8d87d4facd TUN-7351: Add streaming logs session ping and timeout
Sends a ping every 15 seconds to keep the session alive even if no
protocol messages are being propagated. Additionally, sets a hard
timeout of 5 minutes when not actively streaming logs to drop the
connection.
2023-04-10 22:14:58 +00:00
Devin Carr 3fd571063e TUN-7128: Categorize logs from public hostname locations
Updates the HTTP ingress request log events to have more structured
fields to adapt to streaming logs reporting.
2023-04-10 22:14:12 +00:00
Devin Carr 5d0bb25572 TUN-7354: Don't warn for empty ingress rules when using --token 2023-04-10 22:12:40 +00:00
Devin Carr c51b651afb Release 2023.4.0 2023-04-10 09:22:27 -07:00
Devin Carr 04367b0f63 TUN-7357: Bump to go 1.19.6 2023-04-07 18:35:06 +00:00
Devin Carr 69eb9698b5 TUN-7356: Bump golang.org/x/net package to 0.7.0 2023-04-07 09:41:23 -07:00
Devin Carr 55ed995bf0 TUN-7127: Disconnect logger level requirement for management
By default, we want streaming logs to be able to stream debug logs
from cloudflared without needing to update the remote cloudflared's
configuration. This disconnects the provided local log level sent
to console, file, etc. from the level that management tunnel will
utilize via requested filters.
2023-04-06 11:31:47 -07:00
Devin Carr 820a201603 TUN-7135: Add cloudflared tail 2023-04-05 10:20:53 -07:00
Devin Carr 93acdaface TUN-7125: Add management streaming logs WebSocket protocol 2023-04-05 16:25:16 +00:00
João Oliveirinha 5972540efa TUN-7332: Remove legacy tunnel force flag 2023-04-05 16:13:59 +01:00
Han Li 5e37a65dac Fix typo (#918)
UUID not UUUD
2023-04-04 16:15:12 +01:00
pufferfish bfbe426905 Add suport for OpenBSD (#916) 2023-04-04 16:14:51 +01:00
Devin Carr 39ed5dc182 TUN-7126: Add Management logger io.Writer 2023-03-30 14:12:00 -07:00
Devin Carr bbc8d9431b TUN-7333: Default features checkable at runtime across all packages 2023-03-30 17:42:54 +00:00
Sudarsan Reddy b5e03dd66c TUN-9999: Remove classic tunnel component tests 2023-03-30 15:07:14 +00:00
Devin Carr 87f81cc57c TUN-7324: Add http.Hijacker to connection.ResponseWriter
Allows connection.ResponseWriter implemenations to be Hijacked to properly
handle WebSocket connection downgrades from proper HTTP requests.
2023-03-29 09:21:19 -07:00
Devin Carr be64362fdb TUN-7124: Add intercept ingress rule for management requests 2023-03-21 11:42:25 -07:00
João Oliveirinha f686da832f TUN-7275: Make QuickTunnels only use a single connection to the edge 2023-03-13 15:32:46 +00:00
Sudarsan Reddy be341fa055 Updated CHANGES.md for 2023.3.1 2023-03-13 15:15:13 +00:00
394 changed files with 31444 additions and 3056 deletions
+15 -1
View File
@@ -1,7 +1,21 @@
## 2023.4.1
### New Features
- You can now stream your logs from your remote cloudflared to your local terminal with `cloudflared tail <TUNNEL-ID>`. This new feature requires the remote cloudflared to be version 2023.4.1 or higher.
## 2023.3.2
### Notices
- Due to the nature of QuickTunnels (https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/do-more-with-tunnels/trycloudflare/) and its intended usage for testing and experiment of Cloudflare Tunnels, starting from 2023.3.2, QuickTunnels only make a single connection to the edge. If users want to use Tunnels in a production environment, they should move to Named Tunnels instead. (https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide/remote/#set-up-a-tunnel-remotely-dashboard-setup)
## 2023.3.1
### Breaking Change
- Running a tunnel without ingress rules defined in configuration file nor from the CLI flags will no longer provide a default ingress rule to localhost:8080 and instead will return HTTP response code 503 for all incoming HTTP requests.
### Security Fixes
- Windows 32 bit machines MSI now defaults to Program Files to install cloudflared. (See CVE-2023-1314). The cloudflared client itself is unaffected. This just changes how the installer works on 32 bit windows machines.
### Bug Fixes
- Fixed a bug that would cause running tunnel on Bastion mode and without ingress rules to crash.
## 2023.2.2
### Notices
- Legacy tunnels were officially deprecated on December 1, 2022. Starting with this version, cloudflared no longer supports connecting legacy tunnels.
@@ -15,7 +29,7 @@
## 2022.12.0
### Improvements
- cloudflared now attempts to try other edge addresses before falling back to a lower protoocol.
- cloudflared now attempts to try other edge addresses before falling back to a lower protocol.
- cloudflared tunnel no longer spins up a quick tunnel. The call has to be explicit and provide a --url flag.
- cloudflared will now randomly pick the first or second region to connect to instead of always connecting to region2 first.
+2
View File
@@ -82,6 +82,8 @@ else ifeq ($(LOCAL_OS),windows)
TARGET_OS ?= windows
else ifeq ($(LOCAL_OS),freebsd)
TARGET_OS ?= freebsd
else ifeq ($(LOCAL_OS),openbsd)
TARGET_OS ?= openbsd
else
$(error This system's OS $(LOCAL_OS) isn't supported)
endif
+38
View File
@@ -1,3 +1,41 @@
2023.4.2
- 2023-04-24 TUN-7133: Add sampling support for streaming logs
- 2023-04-21 TUN-7141: Add component tests for streaming logs
- 2023-04-21 TUN-7373: Streaming logs override for same actor
- 2023-04-20 TUN-7383: Bump requirements.txt
- 2023-04-19 TUN-7361: Add a label to override hostname
- 2023-04-19 TUN-7378: Remove RPC debug logs
- 2023-04-18 TUN-7360: Add Get Host Details handler in management service
- 2023-04-17 AUTH-3122 Verify that Access tokens are still valid in curl command
- 2023-04-17 TUN-7129: Categorize TCP logs for streaming logs
- 2023-04-17 TUN-7130: Categorize UDP logs for streaming logs
- 2023-04-10 AUTH-4887 Add aud parameter to token transfer url
2023.4.1
- 2023-04-13 TUN-7368: Report destination address for TCP requests in logs
- 2023-04-12 TUN-7134: Acquire token for cloudflared tail
- 2023-04-12 TUN-7131: Add cloudflared log event to connection messages and enable streaming logs
- 2023-04-11 TUN-7132 TUN-7136: Add filter support for streaming logs
- 2023-04-06 TUN-7354: Don't warn for empty ingress rules when using --token
- 2023-04-06 TUN-7128: Categorize logs from public hostname locations
- 2023-04-06 TUN-7351: Add streaming logs session ping and timeout
- 2023-04-06 TUN-7335: Fix cloudflared update not working in windows
2023.4.0
- 2023-04-07 TUN-7356: Bump golang.org/x/net package to 0.7.0
- 2023-04-07 TUN-7357: Bump to go 1.19.6
- 2023-04-06 TUN-7127: Disconnect logger level requirement for management
- 2023-04-05 TUN-7332: Remove legacy tunnel force flag
- 2023-04-05 TUN-7135: Add cloudflared tail
- 2023-04-04 Add suport for OpenBSD (#916)
- 2023-04-04 Fix typo (#918)
- 2023-04-04 TUN-7125: Add management streaming logs WebSocket protocol
- 2023-03-30 TUN-9999: Remove classic tunnel component tests
- 2023-03-30 TUN-7126: Add Management logger io.Writer
- 2023-03-29 TUN-7324: Add http.Hijacker to connection.ResponseWriter
- 2023-03-29 TUN-7333: Default features checkable at runtime across all packages
- 2023-03-21 TUN-7124: Add intercept ingress rule for management requests
2023.3.1
- 2023-03-13 TUN-7271: Return 503 status code when no ingress rules configured
- 2023-03-10 TUN-7272: Fix cloudflared returning non supported status service which breaks configuration migration
-58
View File
@@ -1,58 +0,0 @@
package certutil
import (
"encoding/json"
"encoding/pem"
"fmt"
)
type namedTunnelToken struct {
ZoneID string `json:"zoneID"`
AccountID string `json:"accountID"`
APIToken string `json:"apiToken"`
}
type OriginCert struct {
ZoneID string
APIToken string
AccountID string
}
func DecodeOriginCert(blocks []byte) (*OriginCert, error) {
if len(blocks) == 0 {
return nil, fmt.Errorf("Cannot decode empty certificate")
}
originCert := OriginCert{}
block, rest := pem.Decode(blocks)
for {
if block == nil {
break
}
switch block.Type {
case "PRIVATE KEY", "CERTIFICATE":
// this is for legacy purposes.
break
case "ARGO TUNNEL TOKEN":
if originCert.ZoneID != "" || originCert.APIToken != "" {
return nil, fmt.Errorf("Found multiple tokens in the certificate")
}
// The token is a string,
// Try the newer JSON format
ntt := namedTunnelToken{}
if err := json.Unmarshal(block.Bytes, &ntt); err == nil {
originCert.ZoneID = ntt.ZoneID
originCert.APIToken = ntt.APIToken
originCert.AccountID = ntt.AccountID
}
default:
return nil, fmt.Errorf("Unknown block %s in the certificate", block.Type)
}
block, rest = pem.Decode(rest)
}
if originCert.ZoneID == "" || originCert.APIToken == "" {
return nil, fmt.Errorf("Missing token in the certificate")
}
return &originCert, nil
}
-51
View File
@@ -1,51 +0,0 @@
package certutil
import (
"fmt"
"io/ioutil"
"testing"
"github.com/stretchr/testify/assert"
)
func TestLoadOriginCert(t *testing.T) {
cert, err := DecodeOriginCert([]byte{})
assert.Equal(t, fmt.Errorf("Cannot decode empty certificate"), err)
assert.Nil(t, cert)
blocks, err := ioutil.ReadFile("test-cert-unknown-block.pem")
assert.Nil(t, err)
cert, err = DecodeOriginCert(blocks)
assert.Equal(t, fmt.Errorf("Unknown block RSA PRIVATE KEY in the certificate"), err)
assert.Nil(t, cert)
}
func TestJSONArgoTunnelTokenEmpty(t *testing.T) {
cert, err := DecodeOriginCert([]byte{})
blocks, err := ioutil.ReadFile("test-cert-no-token.pem")
assert.Nil(t, err)
cert, err = DecodeOriginCert(blocks)
assert.Equal(t, fmt.Errorf("Missing token in the certificate"), err)
assert.Nil(t, cert)
}
func TestJSONArgoTunnelToken(t *testing.T) {
// The given cert's Argo Tunnel Token was generated by base64 encoding this JSON:
// {
// "zoneID": "7b0a4d77dfb881c1a3b7d61ea9443e19",
// "apiToken": "test-service-key",
// "accountID": "abcdabcdabcdabcd1234567890abcdef"
// }
CloudflareTunnelTokenTest(t, "test-cloudflare-tunnel-cert-json.pem")
}
func CloudflareTunnelTokenTest(t *testing.T, path string) {
blocks, err := ioutil.ReadFile(path)
assert.Nil(t, err)
cert, err := DecodeOriginCert(blocks)
assert.Nil(t, err)
assert.NotNil(t, cert)
assert.Equal(t, "7b0a4d77dfb881c1a3b7d61ea9443e19", cert.ZoneID)
key := "test-service-key"
assert.Equal(t, key, cert.APIToken)
}
+1
View File
@@ -8,6 +8,7 @@ type TunnelClient interface {
CreateTunnel(name string, tunnelSecret []byte) (*TunnelWithToken, error)
GetTunnel(tunnelID uuid.UUID) (*Tunnel, error)
GetTunnelToken(tunnelID uuid.UUID) (string, error)
GetManagementToken(tunnelID uuid.UUID) (string, error)
DeleteTunnel(tunnelID uuid.UUID) error
ListTunnels(filter *TunnelFilter) ([]*Tunnel, error)
ListActiveClients(tunnelID uuid.UUID) ([]*ActiveClient, error)
+26
View File
@@ -50,6 +50,10 @@ type newTunnel struct {
TunnelSecret []byte `json:"tunnel_secret"`
}
type managementRequest struct {
Resources []string `json:"resources"`
}
type CleanupParams struct {
queryParams url.Values
}
@@ -133,6 +137,28 @@ func (r *RESTClient) GetTunnelToken(tunnelID uuid.UUID) (token string, err error
return "", r.statusCodeToError("get tunnel token", resp)
}
func (r *RESTClient) GetManagementToken(tunnelID uuid.UUID) (token string, err error) {
endpoint := r.baseEndpoints.accountLevel
endpoint.Path = path.Join(endpoint.Path, fmt.Sprintf("%v/management", tunnelID))
body := &managementRequest{
Resources: []string{"logs"},
}
resp, err := r.sendRequest("POST", endpoint, body)
if err != nil {
return "", errors.Wrap(err, "REST request failed")
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusOK {
err = parseResponse(resp.Body, &token)
return token, err
}
return "", r.statusCodeToError("get tunnel token", resp)
}
func (r *RESTClient) DeleteTunnel(tunnelID uuid.UUID) error {
endpoint := r.baseEndpoints.accountLevel
endpoint.Path = path.Join(endpoint.Path, fmt.Sprintf("%v", tunnelID))
+2 -2
View File
@@ -1,5 +1,5 @@
pinned_go: &pinned_go go=1.19.3-1
pinned_go_fips: &pinned_go_fips go-boring=1.19.3-1
pinned_go: &pinned_go go=1.19.6-1
pinned_go_fips: &pinned_go_fips go-boring=1.19.6-1
build_dir: &build_dir /cfsetup_build
default-flavor: bullseye
+7
View File
@@ -280,6 +280,13 @@ func curl(c *cli.Context) error {
if err != nil {
return err
}
// Verify that the existing token is still good; if not fetch a new one
if err := verifyTokenAtEdge(appURL, appInfo, c, log); err != nil {
log.Err(err).Msg("Could not verify token")
return err
}
tok, err := token.GetAppTokenIfExists(appInfo)
if err != nil || tok == "" {
if allowRequest {
+4
View File
@@ -47,3 +47,7 @@ func (bi *BuildInfo) GetBuildTypeMsg() string {
}
return fmt.Sprintf(" with %s", bi.BuildType)
}
func (bi *BuildInfo) UserAgent() string {
return fmt.Sprintf("cloudflared/%s", bi.CloudflaredVersion)
}
+51
View File
@@ -0,0 +1,51 @@
package cliutil
import (
"github.com/urfave/cli/v2"
"github.com/urfave/cli/v2/altsrc"
"github.com/cloudflare/cloudflared/logger"
)
var (
debugLevelWarning = "At debug level cloudflared will log request URL, method, protocol, content length, as well as, all request and response headers. " +
"This can expose sensitive information in your logs."
)
func ConfigureLoggingFlags(shouldHide bool) []cli.Flag {
return []cli.Flag{
altsrc.NewStringFlag(&cli.StringFlag{
Name: logger.LogLevelFlag,
Value: "info",
Usage: "Application logging level {debug, info, warn, error, fatal}. " + debugLevelWarning,
EnvVars: []string{"TUNNEL_LOGLEVEL"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: logger.LogTransportLevelFlag,
Aliases: []string{"proto-loglevel"}, // This flag used to be called proto-loglevel
Value: "info",
Usage: "Transport logging level(previously called protocol logging level) {debug, info, warn, error, fatal}",
EnvVars: []string{"TUNNEL_PROTO_LOGLEVEL", "TUNNEL_TRANSPORT_LOGLEVEL"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: logger.LogFileFlag,
Usage: "Save application log to this file for reporting issues.",
EnvVars: []string{"TUNNEL_LOGFILE"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: logger.LogDirectoryFlag,
Usage: "Save application log to this directory for reporting issues.",
EnvVars: []string{"TUNNEL_LOGDIRECTORY"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "trace-output",
Usage: "Name of trace output file, generated when cloudflared stops.",
EnvVars: []string{"TUNNEL_TRACE_OUTPUT"},
Hidden: shouldHide,
}),
}
}
+3
View File
@@ -15,6 +15,7 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/access"
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/cmd/cloudflared/proxydns"
"github.com/cloudflare/cloudflared/cmd/cloudflared/tail"
"github.com/cloudflare/cloudflared/cmd/cloudflared/tunnel"
"github.com/cloudflare/cloudflared/cmd/cloudflared/updater"
"github.com/cloudflare/cloudflared/config"
@@ -89,6 +90,7 @@ func main() {
updater.Init(Version)
tracing.Init(Version)
token.Init(Version)
tail.Init(bInfo)
runApp(app, graceShutdownC)
}
@@ -138,6 +140,7 @@ To determine if an update happened in a script, check for error code 11.`,
cmds = append(cmds, tunnel.Commands()...)
cmds = append(cmds, proxydns.Command(false))
cmds = append(cmds, access.Commands()...)
cmds = append(cmds, tail.Command())
return cmds
}
+395
View File
@@ -0,0 +1,395 @@
package tail
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
"os"
"os/signal"
"syscall"
"time"
"github.com/google/uuid"
"github.com/mattn/go-colorable"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
"nhooyr.io/websocket"
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/credentials"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/management"
)
var (
buildInfo *cliutil.BuildInfo
)
func Init(bi *cliutil.BuildInfo) {
buildInfo = bi
}
func Command() *cli.Command {
subcommands := []*cli.Command{
buildTailManagementTokenSubcommand(),
}
return buildTailCommand(subcommands)
}
func buildTailManagementTokenSubcommand() *cli.Command {
return &cli.Command{
Name: "token",
Action: cliutil.ConfiguredAction(managementTokenCommand),
Usage: "Get management access jwt",
UsageText: "cloudflared tail token TUNNEL_ID",
Description: `Get management access jwt for a tunnel`,
Hidden: true,
}
}
func managementTokenCommand(c *cli.Context) error {
log := createLogger(c)
token, err := getManagementToken(c, log)
if err != nil {
return err
}
var tokenResponse = struct {
Token string `json:"token"`
}{Token: token}
return json.NewEncoder(os.Stdout).Encode(tokenResponse)
}
func buildTailCommand(subcommands []*cli.Command) *cli.Command {
return &cli.Command{
Name: "tail",
Action: Run,
Usage: "Stream logs from a remote cloudflared",
UsageText: "cloudflared tail [tail command options] [TUNNEL-ID]",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "connector-id",
Usage: "Access a specific cloudflared instance by connector id (for when a tunnel has multiple cloudflared's)",
Value: "",
EnvVars: []string{"TUNNEL_MANAGEMENT_CONNECTOR"},
},
&cli.StringSliceFlag{
Name: "event",
Usage: "Filter by specific Events (cloudflared, http, tcp, udp) otherwise, defaults to send all events",
EnvVars: []string{"TUNNEL_MANAGEMENT_FILTER_EVENTS"},
},
&cli.StringFlag{
Name: "level",
Usage: "Filter by specific log levels (debug, info, warn, error). Filters by debug log level by default.",
EnvVars: []string{"TUNNEL_MANAGEMENT_FILTER_LEVEL"},
Value: "debug",
},
&cli.Float64Flag{
Name: "sample",
Usage: "Sample log events by percentage (0.0 .. 1.0). No sampling by default.",
EnvVars: []string{"TUNNEL_MANAGEMENT_FILTER_SAMPLE"},
Value: 1.0,
},
&cli.StringFlag{
Name: "token",
Usage: "Access token for a specific tunnel",
Value: "",
EnvVars: []string{"TUNNEL_MANAGEMENT_TOKEN"},
},
&cli.StringFlag{
Name: "management-hostname",
Usage: "Management hostname to signify incoming management requests",
EnvVars: []string{"TUNNEL_MANAGEMENT_HOSTNAME"},
Hidden: true,
Value: "management.argotunnel.com",
},
&cli.StringFlag{
Name: "trace",
Usage: "Set a cf-trace-id for the request",
Hidden: true,
Value: "",
},
&cli.StringFlag{
Name: logger.LogLevelFlag,
Value: "info",
Usage: "Application logging level {debug, info, warn, error, fatal}",
EnvVars: []string{"TUNNEL_LOGLEVEL"},
},
&cli.StringFlag{
Name: credentials.OriginCertFlag,
Usage: "Path to the certificate generated for your origin when you run cloudflared login.",
EnvVars: []string{"TUNNEL_ORIGIN_CERT"},
Value: credentials.FindDefaultOriginCertPath(),
},
},
Subcommands: subcommands,
}
}
// Middleware validation error struct for returning to the eyeball
type managementError struct {
Code int `json:"code,omitempty"`
Message string `json:"message,omitempty"`
}
// Middleware validation error HTTP response JSON for returning to the eyeball
type managementErrorResponse struct {
Success bool `json:"success,omitempty"`
Errors []managementError `json:"errors,omitempty"`
}
func handleValidationError(resp *http.Response, log *zerolog.Logger) {
if resp.StatusCode == 530 {
log.Error().Msgf("no cloudflared connector available or reachable via management request (a recent version of cloudflared is required to use streaming logs)")
}
var managementErr managementErrorResponse
err := json.NewDecoder(resp.Body).Decode(&managementErr)
if err != nil {
log.Error().Msgf("unable to start management log streaming session: http response code returned %d", resp.StatusCode)
return
}
if managementErr.Success || len(managementErr.Errors) == 0 {
log.Error().Msgf("management tunnel validation returned success with invalid HTTP response code to convert to a WebSocket request")
return
}
for _, e := range managementErr.Errors {
log.Error().Msgf("management request failed validation: (%d) %s", e.Code, e.Message)
}
}
// logger will be created to emit only against the os.Stderr as to not obstruct with normal output from
// management requests
func createLogger(c *cli.Context) *zerolog.Logger {
level, levelErr := zerolog.ParseLevel(c.String(logger.LogLevelFlag))
if levelErr != nil {
level = zerolog.InfoLevel
}
log := zerolog.New(zerolog.ConsoleWriter{
Out: colorable.NewColorable(os.Stderr),
TimeFormat: time.RFC3339,
}).With().Timestamp().Logger().Level(level)
return &log
}
// parseFilters will attempt to parse provided filters to send to with the EventStartStreaming
func parseFilters(c *cli.Context) (*management.StreamingFilters, error) {
var level *management.LogLevel
var events []management.LogEventType
var sample float64
argLevel := c.String("level")
argEvents := c.StringSlice("event")
argSample := c.Float64("sample")
if argLevel != "" {
l, ok := management.ParseLogLevel(argLevel)
if !ok {
return nil, fmt.Errorf("invalid --level filter provided, please use one of the following Log Levels: debug, info, warn, error")
}
level = &l
}
for _, v := range argEvents {
t, ok := management.ParseLogEventType(v)
if !ok {
return nil, fmt.Errorf("invalid --event filter provided, please use one of the following EventTypes: cloudflared, http, tcp, udp")
}
events = append(events, t)
}
if argSample <= 0.0 || argSample > 1.0 {
return nil, fmt.Errorf("invalid --sample value provided, please make sure it is in the range (0.0 .. 1.0)")
}
sample = argSample
if level == nil && len(events) == 0 && argSample != 1.0 {
// When no filters are provided, do not return a StreamingFilters struct
return nil, nil
}
return &management.StreamingFilters{
Level: level,
Events: events,
Sampling: sample,
}, nil
}
// getManagementToken will make a call to the Cloudflare API to acquire a management token for the requested tunnel.
func getManagementToken(c *cli.Context, log *zerolog.Logger) (string, error) {
userCreds, err := credentials.Read(c.String(credentials.OriginCertFlag), log)
if err != nil {
return "", err
}
client, err := userCreds.Client(c.String("api-url"), buildInfo.UserAgent(), log)
if err != nil {
return "", err
}
tunnelIDString := c.Args().First()
if tunnelIDString == "" {
return "", errors.New("no tunnel ID provided")
}
tunnelID, err := uuid.Parse(tunnelIDString)
if err != nil {
return "", errors.New("unable to parse provided tunnel id as a valid UUID")
}
token, err := client.GetManagementToken(tunnelID)
if err != nil {
return "", err
}
return token, nil
}
// buildURL will build the management url to contain the required query parameters to authenticate the request.
func buildURL(c *cli.Context, log *zerolog.Logger) (url.URL, error) {
var err error
managementHostname := c.String("management-hostname")
token := c.String("token")
if token == "" {
token, err = getManagementToken(c, log)
if err != nil {
return url.URL{}, fmt.Errorf("unable to acquire management token for requested tunnel id: %w", err)
}
}
query := url.Values{}
query.Add("access_token", token)
connector := c.String("connector-id")
if connector != "" {
connectorID, err := uuid.Parse(connector)
if err != nil {
return url.URL{}, fmt.Errorf("unabled to parse 'connector-id' flag into a valid UUID: %w", err)
}
query.Add("connector_id", connectorID.String())
}
return url.URL{Scheme: "wss", Host: managementHostname, Path: "/logs", RawQuery: query.Encode()}, nil
}
// Run implements a foreground runner
func Run(c *cli.Context) error {
log := createLogger(c)
signals := make(chan os.Signal, 10)
signal.Notify(signals, syscall.SIGTERM, syscall.SIGINT)
defer signal.Stop(signals)
filters, err := parseFilters(c)
if err != nil {
log.Error().Err(err).Msgf("invalid filters provided")
return nil
}
u, err := buildURL(c, log)
if err != nil {
log.Err(err).Msg("unable to construct management request URL")
return nil
}
header := make(http.Header)
header.Add("User-Agent", buildInfo.UserAgent())
trace := c.String("trace")
if trace != "" {
header["cf-trace-id"] = []string{trace}
}
ctx := c.Context
conn, resp, err := websocket.Dial(ctx, u.String(), &websocket.DialOptions{
HTTPHeader: header,
})
if err != nil {
if resp != nil && resp.StatusCode != http.StatusSwitchingProtocols {
handleValidationError(resp, log)
return nil
}
log.Error().Err(err).Msgf("unable to start management log streaming session")
return nil
}
defer conn.Close(websocket.StatusInternalError, "management connection was closed abruptly")
// Once connection is established, send start_streaming event to begin receiving logs
err = management.WriteEvent(conn, ctx, &management.EventStartStreaming{
ClientEvent: management.ClientEvent{Type: management.StartStreaming},
Filters: filters,
})
if err != nil {
log.Error().Err(err).Msg("unable to request logs from management tunnel")
return nil
}
log.Debug().
Str("tunnel-id", c.Args().First()).
Str("connector-id", c.String("connector-id")).
Interface("filters", filters).
Msg("connected")
readerDone := make(chan struct{})
go func() {
defer close(readerDone)
for {
select {
case <-ctx.Done():
return
default:
event, err := management.ReadServerEvent(conn, ctx)
if err != nil {
if closeErr := management.AsClosed(err); closeErr != nil {
// If the client (or the server) already closed the connection, don't continue to
// attempt to read from the client.
if closeErr.Code == websocket.StatusNormalClosure {
return
}
// Only log abnormal closures
log.Error().Msgf("received remote closure: (%d) %s", closeErr.Code, closeErr.Reason)
return
}
log.Err(err).Msg("unable to read event from server")
return
}
switch event.Type {
case management.Logs:
logs, ok := management.IntoServerEvent(event, management.Logs)
if !ok {
log.Error().Msgf("invalid logs event")
continue
}
// Output all the logs received to stdout
for _, l := range logs.Logs {
fields, err := json.Marshal(l.Fields)
if err != nil {
fields = []byte("unable to parse fields")
log.Debug().Msgf("unable to parse fields from event %+v", l)
}
fmt.Printf("%s %s %s %s %s\n", l.Time, l.Level, l.Event, l.Message, fields)
}
case management.UnknownServerEventType:
fallthrough
default:
log.Debug().Msgf("unexpected log event type: %s", event.Type)
}
}
}
}()
for {
select {
case <-ctx.Done():
return nil
case <-readerDone:
return nil
case <-signals:
log.Debug().Msg("closing management connection")
// Cleanly close the connection by sending a close message and then
// waiting (with timeout) for the server to close the connection.
conn.Close(websocket.StatusNormalClosure, "")
select {
case <-readerDone:
case <-time.After(time.Second):
}
return nil
}
}
}
+52 -47
View File
@@ -28,8 +28,12 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/updater"
"github.com/cloudflare/cloudflared/config"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/credentials"
"github.com/cloudflare/cloudflared/edgediscovery"
"github.com/cloudflare/cloudflared/features"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/metrics"
"github.com/cloudflare/cloudflared/orchestration"
"github.com/cloudflare/cloudflared/signal"
@@ -42,6 +46,9 @@ import (
const (
sentryDSN = "https://56a9c9fa5c364ab28f34b14f35ea0f1b:3e8827f6f9f740738eb11138f7bebb68@sentry.io/189878"
// ha-Connections specifies how many connections to make to the edge
haConnectionsFlag = "ha-connections"
// sshPortFlag is the port on localhost the cloudflared ssh server will run on
sshPortFlag = "local-ssh-port"
@@ -75,9 +82,6 @@ const (
// uiFlag is to enable launching cloudflared in interactive UI mode
uiFlag = "ui"
debugLevelWarning = "At debug level cloudflared will log request URL, method, protocol, content length, as well as, all request and response headers. " +
"This can expose sensitive information in your logs."
LogFieldCommand = "command"
LogFieldExpandedPath = "expandedPath"
LogFieldPIDPathname = "pidPathname"
@@ -92,6 +96,7 @@ Eg. cloudflared tunnel --url localhost:8080/.
Please note that Quick Tunnels are meant to be ephemeral and should only be used for testing purposes.
For production usage, we recommend creating Named Tunnels. (https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide/)
`
connectorLabelFlag = "label"
)
var (
@@ -394,7 +399,26 @@ func StartServer(
}
}
orchestrator, err := orchestration.NewOrchestrator(ctx, orchestratorConfig, tunnelConfig.Tags, tunnelConfig.Log)
localRules := []ingress.Rule{}
if features.Contains(features.FeatureManagementLogs) {
serviceIP := c.String("service-op-ip")
if edgeAddrs, err := edgediscovery.ResolveEdge(log, tunnelConfig.Region, tunnelConfig.EdgeIPVersion); err == nil {
if serviceAddr, err := edgeAddrs.GetAddrForRPC(); err == nil {
serviceIP = serviceAddr.TCP.String()
}
}
mgmt := management.New(
c.String("management-hostname"),
serviceIP,
clientID,
c.String(connectorLabelFlag),
logger.ManagementLogger.Log,
logger.ManagementLogger,
)
localRules = []ingress.Rule{ingress.NewManagementRule(mgmt)}
}
orchestrator, err := orchestration.NewOrchestrator(ctx, orchestratorConfig, tunnelConfig.Tags, localRules, tunnelConfig.Log)
if err != nil {
return err
}
@@ -418,7 +442,7 @@ func StartServer(
errC <- metrics.ServeMetrics(metricsListener, ctx, metricsConfig, log)
}()
reconnectCh := make(chan supervisor.ReconnectSignal, c.Int("ha-connections"))
reconnectCh := make(chan supervisor.ReconnectSignal, c.Int(haConnectionsFlag))
if c.IsSet("stdin-control") {
log.Info().Msg("Enabling control through stdin")
go stdinControl(reconnectCh, log)
@@ -531,7 +555,7 @@ func addPortIfMissing(uri *url.URL, port int) string {
func tunnelFlags(shouldHide bool) []cli.Flag {
flags := configureCloudflaredFlags(shouldHide)
flags = append(flags, configureProxyFlags(shouldHide)...)
flags = append(flags, configureLoggingFlags(shouldHide)...)
flags = append(flags, cliutil.ConfigureLoggingFlags(shouldHide)...)
flags = append(flags, configureProxyDNSFlags(shouldHide)...)
flags = append(flags, []cli.Flag{
credentialsFileFlag,
@@ -655,10 +679,15 @@ func tunnelFlags(shouldHide bool) []cli.Flag {
Hidden: shouldHide,
}),
altsrc.NewIntFlag(&cli.IntFlag{
Name: "ha-connections",
Name: haConnectionsFlag,
Value: 4,
Hidden: true,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: connectorLabelFlag,
Usage: "Use this option to give a meaningful label to a specific connector. When a tunnel starts up, a connector id unique to the tunnel is generated. This is a uuid. To make it easier to identify a connector, we will use the hostname of the machine the tunnel is running on along with the connector ID. This option exists if one wants to have more control over what their individual connectors are called.",
Value: "",
}),
altsrc.NewDurationFlag(&cli.DurationFlag{
Name: "grace-period",
Usage: "When cloudflared receives SIGINT/SIGTERM it will stop accepting new requests, wait for in-progress requests to terminate, then shutdown. Waiting for in-progress requests will timeout after this grace period, or when a second SIGTERM/SIGINT is received.",
@@ -744,10 +773,10 @@ func configureCloudflaredFlags(shouldHide bool) []cli.Flag {
Hidden: shouldHide,
},
altsrc.NewStringFlag(&cli.StringFlag{
Name: "origincert",
Name: credentials.OriginCertFlag,
Usage: "Path to the certificate generated for your origin when you run cloudflared login.",
EnvVars: []string{"TUNNEL_ORIGIN_CERT"},
Value: findDefaultOriginCertPath(),
Value: credentials.FindDefaultOriginCertPath(),
Hidden: shouldHide,
}),
altsrc.NewDurationFlag(&cli.DurationFlag{
@@ -892,6 +921,20 @@ func configureProxyFlags(shouldHide bool) []cli.Flag {
Hidden: shouldHide,
Value: false,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "management-hostname",
Usage: "Management hostname to signify incoming management requests",
EnvVars: []string{"TUNNEL_MANAGEMENT_HOSTNAME"},
Hidden: true,
Value: "management.argotunnel.com",
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "service-op-ip",
Usage: "Fallback IP for service operations run by the management service.",
EnvVars: []string{"TUNNEL_SERVICE_OP_IP"},
Hidden: true,
Value: "198.41.200.113:80",
}),
}
return append(flags, sshFlags(shouldHide)...)
}
@@ -1000,44 +1043,6 @@ func sshFlags(shouldHide bool) []cli.Flag {
}
}
func configureLoggingFlags(shouldHide bool) []cli.Flag {
return []cli.Flag{
altsrc.NewStringFlag(&cli.StringFlag{
Name: logger.LogLevelFlag,
Value: "info",
Usage: "Application logging level {debug, info, warn, error, fatal}. " + debugLevelWarning,
EnvVars: []string{"TUNNEL_LOGLEVEL"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: logger.LogTransportLevelFlag,
Aliases: []string{"proto-loglevel"}, // This flag used to be called proto-loglevel
Value: "info",
Usage: "Transport logging level(previously called protocol logging level) {debug, info, warn, error, fatal}",
EnvVars: []string{"TUNNEL_PROTO_LOGLEVEL", "TUNNEL_TRANSPORT_LOGLEVEL"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: logger.LogFileFlag,
Usage: "Save application log to this file for reporting issues.",
EnvVars: []string{"TUNNEL_LOGFILE"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: logger.LogDirectoryFlag,
Usage: "Save application log to this directory for reporting issues.",
EnvVars: []string{"TUNNEL_LOGDIRECTORY"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "trace-output",
Usage: "Name of trace output file, generated when cloudflared stops.",
EnvVars: []string{"TUNNEL_TRACE_OUTPUT"},
Hidden: shouldHide,
}),
}
}
func configureProxyDNSFlags(shouldHide bool) []cli.Flag {
return []cli.Flag{
altsrc.NewBoolFlag(&cli.BoolFlag{
+6 -78
View File
@@ -3,17 +3,14 @@ package tunnel
import (
"crypto/tls"
"fmt"
"io/ioutil"
mathRand "math/rand"
"net"
"net/netip"
"os"
"path/filepath"
"strings"
"time"
"github.com/google/uuid"
homedir "github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
@@ -25,6 +22,7 @@ import (
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/edgediscovery"
"github.com/cloudflare/cloudflared/edgediscovery/allregions"
"github.com/cloudflare/cloudflared/features"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/orchestration"
"github.com/cloudflare/cloudflared/supervisor"
@@ -32,7 +30,6 @@ import (
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
const LogFieldOriginCertPath = "originCertPath"
const secretValue = "*****"
var (
@@ -40,24 +37,11 @@ var (
serviceUrl = developerPortal + "/reference/service/"
argumentsUrl = developerPortal + "/reference/arguments/"
secretFlags = [2]*altsrc.StringFlag{credentialsContentsFlag, tunnelTokenFlag}
defaultFeatures = []string{supervisor.FeatureAllowRemoteConfig, supervisor.FeatureSerializedHeaders, supervisor.FeatureDatagramV2, supervisor.FeatureQUICSupportEOF}
secretFlags = [2]*altsrc.StringFlag{credentialsContentsFlag, tunnelTokenFlag}
configFlags = []string{"autoupdate-freq", "no-autoupdate", "retries", "protocol", "loglevel", "transport-loglevel", "origincert", "metrics", "metrics-update-freq", "edge-ip-version", "edge-bind-address"}
)
// returns the first path that contains a cert.pem file. If none of the DefaultConfigSearchDirectories
// contains a cert.pem file, return empty string
func findDefaultOriginCertPath() string {
for _, defaultConfigDir := range config.DefaultConfigSearchDirectories() {
originCertPath, _ := homedir.Expand(filepath.Join(defaultConfigDir, config.DefaultCredentialFile))
if ok, _ := config.FileExists(originCertPath); ok {
return originCertPath
}
}
return ""
}
func generateRandomClientID(log *zerolog.Logger) (string, error) {
u, err := uuid.NewRandom()
if err != nil {
@@ -128,62 +112,6 @@ func dnsProxyStandAlone(c *cli.Context, namedTunnel *connection.NamedTunnelPrope
namedTunnel != nil) // named tunnel
}
func findOriginCert(originCertPath string, log *zerolog.Logger) (string, error) {
if originCertPath == "" {
log.Info().Msgf("Cannot determine default origin certificate path. No file %s in %v", config.DefaultCredentialFile, config.DefaultConfigSearchDirectories())
if isRunningFromTerminal() {
log.Error().Msgf("You need to specify the origin certificate path with --origincert option, or set TUNNEL_ORIGIN_CERT environment variable. See %s for more information.", argumentsUrl)
return "", fmt.Errorf("client didn't specify origincert path when running from terminal")
} else {
log.Error().Msgf("You need to specify the origin certificate path by specifying the origincert option in the configuration file, or set TUNNEL_ORIGIN_CERT environment variable. See %s for more information.", serviceUrl)
return "", fmt.Errorf("client didn't specify origincert path")
}
}
var err error
originCertPath, err = homedir.Expand(originCertPath)
if err != nil {
log.Err(err).Msgf("Cannot resolve origin certificate path")
return "", fmt.Errorf("cannot resolve path %s", originCertPath)
}
// Check that the user has acquired a certificate using the login command
ok, err := config.FileExists(originCertPath)
if err != nil {
log.Error().Err(err).Msgf("Cannot check if origin cert exists at path %s", originCertPath)
return "", fmt.Errorf("cannot check if origin cert exists at path %s", originCertPath)
}
if !ok {
log.Error().Msgf(`Cannot find a valid certificate for your origin at the path:
%s
If the path above is wrong, specify the path with the -origincert option.
If you don't have a certificate signed by Cloudflare, run the command:
%s login
`, originCertPath, os.Args[0])
return "", fmt.Errorf("cannot find a valid certificate at the path %s", originCertPath)
}
return originCertPath, nil
}
func readOriginCert(originCertPath string) ([]byte, error) {
// Easier to send the certificate as []byte via RPC than decoding it at this point
originCert, err := ioutil.ReadFile(originCertPath)
if err != nil {
return nil, fmt.Errorf("cannot read %s to load origin certificate", originCertPath)
}
return originCert, nil
}
func getOriginCert(originCertPath string, log *zerolog.Logger) ([]byte, error) {
if originCertPath, err := findOriginCert(originCertPath, log); err != nil {
return nil, err
} else {
return readOriginCert(originCertPath)
}
}
func prepareTunnelConfig(
c *cli.Context,
info *cliutil.BuildInfo,
@@ -216,13 +144,13 @@ func prepareTunnelConfig(
transportProtocol = connection.QUIC.String()
}
features := dedup(append(c.StringSlice("features"), defaultFeatures...))
clientFeatures := dedup(append(c.StringSlice("features"), features.DefaultFeatures...))
if needPQ {
features = append(features, supervisor.FeaturePostQuantum)
clientFeatures = append(clientFeatures, features.FeaturePostQuantum)
}
namedTunnel.Client = tunnelpogs.ClientInfo{
ClientID: clientID[:],
Features: features,
Features: clientFeatures,
Version: info.Version(),
Arch: info.OSArch(),
}
@@ -293,7 +221,7 @@ func prepareTunnelConfig(
Region: c.String("region"),
EdgeIPVersion: edgeIPVersion,
EdgeBindAddr: edgeBindAddr,
HAConnections: c.Int("ha-connections"),
HAConnections: c.Int(haConnectionsFlag),
IncidentLookup: supervisor.NewIncidentLookup(),
IsAutoupdated: c.Bool("is-autoupdated"),
LBPool: c.String("lb-pool"),
+4 -3
View File
@@ -5,6 +5,7 @@ import (
"path/filepath"
"github.com/cloudflare/cloudflared/config"
"github.com/cloudflare/cloudflared/credentials"
"github.com/google/uuid"
"github.com/rs/zerolog"
@@ -56,13 +57,13 @@ func newSearchByID(id uuid.UUID, c *cli.Context, log *zerolog.Logger, fs fileSys
}
func (s searchByID) Path() (string, error) {
originCertPath := s.c.String("origincert")
originCertPath := s.c.String(credentials.OriginCertFlag)
originCertLog := s.log.With().
Str(LogFieldOriginCertPath, originCertPath).
Str("originCertPath", originCertPath).
Logger()
// Fallback to look for tunnel credentials in the origin cert directory
if originCertPath, err := findOriginCert(originCertPath, &originCertLog); err == nil {
if originCertPath, err := credentials.FindOriginCert(originCertPath, &originCertLog); err == nil {
originCertDir := filepath.Dir(originCertPath)
if filePath, err := tunnelFilePath(s.id, originCertDir); err == nil {
if s.fs.validFilePath(filePath) {
+3 -1
View File
@@ -14,6 +14,7 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/config"
"github.com/cloudflare/cloudflared/credentials"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/token"
)
@@ -52,6 +53,7 @@ func login(c *cli.Context) error {
resourceData, err := token.RunTransfer(
loginURL,
"",
"cert",
"callback",
callbackStoreURL,
@@ -85,7 +87,7 @@ func checkForExistingCert() (string, bool, error) {
if err != nil {
return "", false, err
}
path := filepath.Join(configPath, config.DefaultCredentialFile)
path := filepath.Join(configPath, credentials.DefaultCredentialFile)
fileInfo, err := os.Stat(path)
if err == nil && fileInfo.Size() > 0 {
return path, true, nil
+3
View File
@@ -73,6 +73,9 @@ func RunQuickTunnel(sc *subcommandContext) error {
sc.c.Set("protocol", "quic")
}
// Override the number of connections used. Quick tunnels shouldn't be used for production usage,
// so, use a single connection instead.
sc.c.Set(haConnectionsFlag, "1")
return StartServer(
sc.c,
buildInfo,
+11 -48
View File
@@ -13,9 +13,9 @@ import (
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
"github.com/cloudflare/cloudflared/certutil"
"github.com/cloudflare/cloudflared/cfapi"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/credentials"
"github.com/cloudflare/cloudflared/logger"
)
@@ -37,7 +37,7 @@ type subcommandContext struct {
// These fields should be accessed using their respective Getter
tunnelstoreClient cfapi.Client
userCredential *userCredential
userCredential *credentials.User
}
func newSubcommandContext(c *cli.Context) (*subcommandContext, error) {
@@ -56,65 +56,28 @@ func (sc *subcommandContext) credentialFinder(tunnelID uuid.UUID) CredFinder {
return newSearchByID(tunnelID, sc.c, sc.log, sc.fs)
}
type userCredential struct {
cert *certutil.OriginCert
certPath string
}
func (sc *subcommandContext) client() (cfapi.Client, error) {
if sc.tunnelstoreClient != nil {
return sc.tunnelstoreClient, nil
}
credential, err := sc.credential()
cred, err := sc.credential()
if err != nil {
return nil, err
}
userAgent := fmt.Sprintf("cloudflared/%s", buildInfo.Version())
client, err := cfapi.NewRESTClient(
sc.c.String("api-url"),
credential.cert.AccountID,
credential.cert.ZoneID,
credential.cert.APIToken,
userAgent,
sc.log,
)
sc.tunnelstoreClient, err = cred.Client(sc.c.String("api-url"), buildInfo.UserAgent(), sc.log)
if err != nil {
return nil, err
}
sc.tunnelstoreClient = client
return client, nil
return sc.tunnelstoreClient, nil
}
func (sc *subcommandContext) credential() (*userCredential, error) {
func (sc *subcommandContext) credential() (*credentials.User, error) {
if sc.userCredential == nil {
originCertPath := sc.c.String("origincert")
originCertLog := sc.log.With().
Str(LogFieldOriginCertPath, originCertPath).
Logger()
originCertPath, err := findOriginCert(originCertPath, &originCertLog)
uc, err := credentials.Read(sc.c.String(credentials.OriginCertFlag), sc.log)
if err != nil {
return nil, errors.Wrap(err, "Error locating origin cert")
}
blocks, err := readOriginCert(originCertPath)
if err != nil {
return nil, errors.Wrapf(err, "Can't read origin cert from %s", originCertPath)
}
cert, err := certutil.DecodeOriginCert(blocks)
if err != nil {
return nil, errors.Wrap(err, "Error decoding origin cert")
}
if cert.AccountID == "" {
return nil, errors.Errorf(`Origin certificate needs to be refreshed before creating new tunnels.\nDelete %s and run "cloudflared login" to obtain a new cert.`, originCertPath)
}
sc.userCredential = &userCredential{
cert: cert,
certPath: originCertPath,
return nil, err
}
sc.userCredential = uc
}
return sc.userCredential, nil
}
@@ -175,13 +138,13 @@ func (sc *subcommandContext) create(name string, credentialsFilePath string, sec
return nil, err
}
tunnelCredentials := connection.Credentials{
AccountTag: credential.cert.AccountID,
AccountTag: credential.AccountID(),
TunnelSecret: tunnelSecret,
TunnelID: tunnel.ID,
}
usedCertPath := false
if credentialsFilePath == "" {
originCertDir := filepath.Dir(credential.certPath)
originCertDir := filepath.Dir(credential.CertPath())
credentialsFilePath, err = tunnelFilePath(tunnelCredentials.TunnelID, originCertDir)
if err != nil {
return nil, err
@@ -16,6 +16,7 @@ import (
"github.com/cloudflare/cloudflared/cfapi"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/credentials"
)
type mockFileSystem struct {
@@ -37,7 +38,7 @@ func Test_subcommandContext_findCredentials(t *testing.T) {
log *zerolog.Logger
fs fileSystem
tunnelstoreClient cfapi.Client
userCredential *userCredential
userCredential *credentials.User
}
type args struct {
tunnelID uuid.UUID
@@ -249,7 +250,7 @@ func Test_subcommandContext_Delete(t *testing.T) {
isUIEnabled bool
fs fileSystem
tunnelstoreClient *deleteMockTunnelStore
userCredential *userCredential
userCredential *credentials.User
}
type args struct {
tunnelIDs []uuid.UUID
+2 -11
View File
@@ -95,14 +95,6 @@ var (
Usage: "Inverts the sort order of the tunnel list.",
EnvVars: []string{"TUNNEL_LIST_INVERT_SORT"},
}
forceFlag = altsrc.NewBoolFlag(&cli.BoolFlag{
Name: "force",
Aliases: []string{"f"},
Usage: "By default, if a tunnel is currently being run from a cloudflared, you can't " +
"simultaneously rerun it again from a second cloudflared. The --force flag lets you " +
"overwrite the previous tunnel. If you want to use a single hostname with multiple " +
"tunnels, you can do so with Cloudflare's Load Balancer product.",
})
featuresFlag = altsrc.NewStringSliceFlag(&cli.StringSliceFlag{
Name: "features",
Aliases: []string{"F"},
@@ -616,7 +608,6 @@ func renderOutput(format string, v interface{}) error {
func buildRunCommand() *cli.Command {
flags := []cli.Flag{
forceFlag,
credentialsFileFlag,
credentialsContentsFlag,
postQuantumFlag,
@@ -632,7 +623,7 @@ func buildRunCommand() *cli.Command {
Action: cliutil.ConfiguredAction(runCommand),
Usage: "Proxy a local web server by running the given tunnel",
UsageText: "cloudflared tunnel [tunnel command options] run [subcommand options] [TUNNEL]",
Description: `Runs the tunnel identified by name or UUUD, creating highly available connections
Description: `Runs the tunnel identified by name or UUID, creating highly available connections
between your server and the Cloudflare edge. You can provide name or UUID of tunnel to run either as the
last command line argument or in the configuration file using "tunnel: TUNNEL".
@@ -941,7 +932,7 @@ func commandHelpTemplate() string {
for _, f := range configureCloudflaredFlags(false) {
parentFlagsHelp += fmt.Sprintf(" %s\n\t", f)
}
for _, f := range configureLoggingFlags(false) {
for _, f := range cliutil.ConfigureLoggingFlags(false) {
parentFlagsHelp += fmt.Sprintf(" %s\n\t", f)
}
const template = `NAME:
+14
View File
@@ -6,6 +6,7 @@ import (
"os"
"path/filepath"
"runtime"
"strings"
"time"
"github.com/facebookgo/grace/gracenet"
@@ -95,11 +96,24 @@ func CheckForUpdate(options updateOptions) (CheckResult, error) {
url = StagingUpdateURL
}
if runtime.GOOS == "windows" {
cfdPath = encodeWindowsPath(cfdPath)
}
s := NewWorkersService(version, url, cfdPath, Options{IsBeta: options.isBeta,
IsForced: options.isForced, RequestedVersion: options.intendedVersion})
return s.Check()
}
func encodeWindowsPath(path string) string {
// We do this because Windows allows spaces in directories such as
// Program Files but does not allow these directories to be spaced in batch files.
targetPath := strings.Replace(path, "Program Files (x86)", "PROGRA~2", -1)
// This is to do the same in 32 bit systems. We do this second so that the first
// replace is for x86 dirs.
targetPath = strings.Replace(targetPath, "Program Files", "PROGRA~1", -1)
return targetPath
}
func applyUpdate(options updateOptions, update CheckResult) UpdateOutcome {
if update.Version() == "" || options.updateDisabled {
+18 -8
View File
@@ -25,14 +25,13 @@ const (
// rename cloudflared.exe.new to cloudflared.exe
// delete cloudflared.exe.old
// start the service
// delete the batch file
windowsUpdateCommandTemplate = `@echo off
sc stop cloudflared >nul 2>&1
// exit with code 0 if we've reached this point indicating success.
windowsUpdateCommandTemplate = `sc stop cloudflared >nul 2>&1
rename "{{.TargetPath}}" {{.OldName}}
rename "{{.NewPath}}" {{.BinaryName}}
del "{{.OldPath}}"
sc start cloudflared >nul 2>&1
del {{.BatchName}}`
exit /b 0`
batchFileName = "cfd_update.bat"
)
@@ -214,8 +213,9 @@ func isValidChecksum(checksum, filePath string) error {
// writeBatchFile writes a batch file out to disk
// see the dicussion on why it has to be done this way
func writeBatchFile(targetPath string, newPath string, oldPath string) error {
os.Remove(batchFileName) //remove any failed updates before download
f, err := os.Create(batchFileName)
batchFilePath := filepath.Join(filepath.Dir(targetPath), batchFileName)
os.Remove(batchFilePath) //remove any failed updates before download
f, err := os.Create(batchFilePath)
if err != nil {
return err
}
@@ -241,6 +241,16 @@ func writeBatchFile(targetPath string, newPath string, oldPath string) error {
// run each OS command for windows
func runWindowsBatch(batchFile string) error {
cmd := exec.Command("cmd", "/c", batchFile)
return cmd.Start()
defer os.Remove(batchFile)
cmd := exec.Command("cmd", "/C", batchFile)
_, err := cmd.Output()
// Remove the batch file we created. Don't let this interfere with the error
// we report.
if err != nil {
if exitError, ok := err.(*exec.ExitError); ok {
return fmt.Errorf("Error during update : %s;", string(exitError.Stderr))
}
}
return err
}
-1
View File
@@ -10,7 +10,6 @@
cloudflared_binary: "cloudflared"
tunnel: "3d539f97-cd3a-4d8e-c33b-65e9099c7a8d"
credentials_file: "/Users/tunnel/.cloudflared/3d539f97-cd3a-4d8e-c33b-65e9099c7a8d.json"
classic_hostname: "classic-tunnel-component-tests.example.com"
origincert: "/Users/tunnel/.cloudflared/cert.pem"
ingress:
- hostname: named-tunnel-component-tests.example.com
+32
View File
@@ -2,7 +2,9 @@ import json
import subprocess
from time import sleep
from constants import MANAGEMENT_HOST_NAME
from setup import get_config_from_file
from util import get_tunnel_connector_id
SINGLE_CASE_TIMEOUT = 600
@@ -28,6 +30,36 @@ class CloudflaredCli:
listed = self._run_command(cmd_args, "list")
return json.loads(listed.stdout)
def get_management_token(self, config, config_path):
basecmd = [config.cloudflared_binary]
if config_path is not None:
basecmd += ["--config", str(config_path)]
origincert = get_config_from_file()["origincert"]
if origincert:
basecmd += ["--origincert", origincert]
cmd_args = ["tail", "token", config.get_tunnel_id()]
cmd = basecmd + cmd_args
result = run_subprocess(cmd, "token", self.logger, check=True, capture_output=True, timeout=15)
return json.loads(result.stdout.decode("utf-8").strip())["token"]
def get_management_url(self, path, config, config_path):
access_jwt = self.get_management_token(config, config_path)
connector_id = get_tunnel_connector_id()
return f"https://{MANAGEMENT_HOST_NAME}/{path}?connector_id={connector_id}&access_token={access_jwt}"
def get_management_wsurl(self, path, config, config_path):
access_jwt = self.get_management_token(config, config_path)
connector_id = get_tunnel_connector_id()
return f"wss://{MANAGEMENT_HOST_NAME}/{path}?connector_id={connector_id}&access_token={access_jwt}"
def get_connector_id(self, config):
op = self.get_tunnel_info(config.get_tunnel_id())
connectors = []
for conn in op["conns"]:
connectors.append(conn["id"])
return connectors
def get_tunnel_info(self, tunnel_id):
info = self._run_command(["info", "--output", "json", tunnel_id], "info")
return json.loads(info.stdout)
+8
View File
@@ -0,0 +1,8 @@
cloudflared_binary: "cloudflared"
tunnel: "ae21a96c-24d1-4ce8-a6ba-962cba5976d3"
credentials_file: "/Users/sudarsan/.cloudflared/ae21a96c-24d1-4ce8-a6ba-962cba5976d3.json"
origincert: "/Users/sudarsan/.cloudflared/cert.pem"
ingress:
- hostname: named-tunnel-component-tests.example.com
service: hello_world
- service: http_status:404
+1
View File
@@ -4,6 +4,7 @@ BACKOFF_SECS = 7
MAX_LOG_LINES = 50
PROXY_DNS_PORT = 9053
MANAGEMENT_HOST_NAME = "management.argotunnel.com"
def protocols():
+5 -3
View File
@@ -1,6 +1,8 @@
cloudflare==2.8.15
flaky==3.7.0
pytest==6.2.2
pytest==7.3.1
pytest-asyncio==0.21.0
pyyaml==5.4.1
requests==2.25.1
retrying==1.3.3
requests==2.28.2
retrying==1.3.4
websockets==11.0.1
-9
View File
@@ -81,12 +81,6 @@ def create_dns(config, hostname, type, content):
)
def create_classic_dns(config, random_uuid):
classic_hostname = "classic-" + random_uuid + "." + config["zone_domain"]
create_dns(config, classic_hostname, "AAAA", "fd10:aec2:5dae::")
return classic_hostname
def create_named_dns(config, random_uuid):
hostname = "named-" + random_uuid + "." + config["zone_domain"]
create_dns(config, hostname, "CNAME", config["tunnel"] + ".cfargotunnel.com")
@@ -119,7 +113,6 @@ def create():
Creates the necessary resources for the components test to run.
- Creates a named tunnel with a random name.
- Creates a random CNAME DNS entry for that tunnel.
- Creates a random AAAA DNS entry for a classic tunnel.
Those created resources are added to the config (obtained from an environment variable).
The resulting configuration is persisted for the tests to use.
@@ -129,7 +122,6 @@ def create():
random_uuid = str(uuid.uuid4())
config["tunnel"] = create_tunnel(config, origincert_path, random_uuid)
config["classic_hostname"] = create_classic_dns(config, random_uuid)
config["ingress"] = [
{
"hostname": create_named_dns(config, random_uuid),
@@ -150,7 +142,6 @@ def cleanup():
"""
config = get_config_from_file()
delete_tunnel(config)
delete_dns(config, config["classic_hostname"])
delete_dns(config, config["ingress"][0]["hostname"])
+3 -3
View File
@@ -74,7 +74,7 @@ def assert_log_to_dir(config, log_dir):
class TestLogging:
def test_logging_to_terminal(self, tmp_path, component_tests_config):
config = component_tests_config()
with start_cloudflared(tmp_path, config, new_process=True) as cloudflared:
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True) as cloudflared:
wait_tunnel_ready(tunnel_url=config.get_url())
assert_log_to_terminal(cloudflared)
@@ -85,7 +85,7 @@ class TestLogging:
"logfile": str(log_file),
}
config = component_tests_config(extra_config)
with start_cloudflared(tmp_path, config, new_process=True, capture_output=False):
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, capture_output=False):
wait_tunnel_ready(tunnel_url=config.get_url(), cfd_logs=str(log_file))
assert_log_in_file(log_file)
assert_json_log(log_file)
@@ -98,6 +98,6 @@ class TestLogging:
"log-directory": str(log_dir),
}
config = component_tests_config(extra_config)
with start_cloudflared(tmp_path, config, new_process=True, capture_output=False):
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, capture_output=False):
wait_tunnel_ready(tunnel_url=config.get_url(), cfd_logs=str(log_dir))
assert_log_to_dir(config, log_dir)
+2 -2
View File
@@ -12,6 +12,6 @@ class TestPostQuantum:
def test_post_quantum(self, tmp_path, component_tests_config):
config = component_tests_config(self._extra_config())
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--post-quantum"], new_process=True):
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["run", "--post-quantum"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(),
require_min_connections=4)
require_min_connections=1)
+1 -1
View File
@@ -25,7 +25,7 @@ def run_test_scenario(tmp_path, component_tests_config, cfd_mode, run_proxy_dns)
if cfd_mode == CfdModes.NAMED:
expect_tunnel = True
pre_args = ["tunnel"]
pre_args = ["tunnel", "--ha-connections", "1"]
args = ["run"]
elif cfd_mode == CfdModes.PROXY_DNS:
expect_proxy_dns = True
+4 -4
View File
@@ -7,16 +7,16 @@ class TestQuickTunnels:
def test_quick_tunnel(self, tmp_path, component_tests_config):
config = component_tests_config(cfd_mode=CfdModes.QUICK, run_proxy_dns=False)
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["--hello-world"], new_process=True):
wait_tunnel_ready(require_min_connections=4)
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["--hello-world"], new_process=True):
wait_tunnel_ready(require_min_connections=1)
url = get_quicktunnel_url()
send_requests(url, 3, True)
def test_quick_tunnel_url(self, tmp_path, component_tests_config):
config = component_tests_config(cfd_mode=CfdModes.QUICK, run_proxy_dns=False)
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["--url", f"http://localhost:{METRICS_PORT}/"], new_process=True):
wait_tunnel_ready()
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["--url", f"http://localhost:{METRICS_PORT}/"], new_process=True):
wait_tunnel_ready(require_min_connections=1)
url = get_quicktunnel_url()
send_requests(url+"/ready", 3, True)
+2 -2
View File
@@ -13,7 +13,7 @@ from util import start_cloudflared, wait_tunnel_ready, check_tunnel_not_connecte
@flaky(max_runs=3, min_passes=1)
class TestReconnect:
default_ha_conns = 4
default_ha_conns = 1
default_reconnect_secs = 15
extra_config = {
"stdin-control": True,
@@ -29,7 +29,7 @@ class TestReconnect:
@pytest.mark.parametrize("protocol", protocols())
def test_named_reconnect(self, tmp_path, component_tests_config, protocol):
config = component_tests_config(self._extra_config(protocol))
with start_cloudflared(tmp_path, config, new_process=True, allow_input=True, capture_output=False) as cloudflared:
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, allow_input=True, capture_output=False) as cloudflared:
# Repeat the test multiple times because some issues only occur after multiple reconnects
self.assert_reconnect(config, cloudflared, 5)
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env python
import asyncio
import json
import pytest
import requests
import websockets
from websockets.client import connect, WebSocketClientProtocol
from conftest import CfdModes
from constants import MAX_RETRIES, BACKOFF_SECS
from retrying import retry
from cli import CloudflaredCli
from util import LOGGER, start_cloudflared, write_config, wait_tunnel_ready
class TestTail:
@pytest.mark.asyncio
async def test_start_stop_streaming(self, tmp_path, component_tests_config):
"""
Validates that a websocket connection to management.argotunnel.com/logs can be opened
with the access token and start and stop streaming on-demand.
"""
print("test_start_stop_streaming")
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(), require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
url = cfd_cli.get_management_wsurl("logs", config, config_path)
async with connect(url, open_timeout=5, close_timeout=3) as websocket:
await websocket.send('{"type": "start_streaming"}')
await websocket.send('{"type": "stop_streaming"}')
await websocket.send('{"type": "start_streaming"}')
await websocket.send('{"type": "stop_streaming"}')
@pytest.mark.asyncio
async def test_streaming_logs(self, tmp_path, component_tests_config):
"""
Validates that a streaming logs connection will stream logs
"""
print("test_streaming_logs")
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(), require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
url = cfd_cli.get_management_wsurl("logs", config, config_path)
async with connect(url, open_timeout=5, close_timeout=5) as websocket:
# send start_streaming
await websocket.send('{"type": "start_streaming"}')
# send some http requests to the tunnel to trigger some logs
await generate_and_validate_http_events(websocket, config.get_url(), 10)
# send stop_streaming
await websocket.send('{"type": "stop_streaming"}')
@pytest.mark.asyncio
async def test_streaming_logs_filters(self, tmp_path, component_tests_config):
"""
Validates that a streaming logs connection will stream logs
but not http when filters applied.
"""
print("test_streaming_logs_filters")
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(), require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
url = cfd_cli.get_management_wsurl("logs", config, config_path)
async with connect(url, open_timeout=5, close_timeout=5) as websocket:
# send start_streaming with tcp logs only
await websocket.send(json.dumps({
"type": "start_streaming",
"filters": {
"events": ["tcp"],
"level": "debug"
}
}))
# don't expect any http logs
await generate_and_validate_no_log_event(websocket, config.get_url())
# send stop_streaming
await websocket.send('{"type": "stop_streaming"}')
@pytest.mark.asyncio
async def test_streaming_logs_sampling(self, tmp_path, component_tests_config):
"""
Validates that a streaming logs connection will stream logs with sampling.
"""
print("test_streaming_logs_sampling")
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(), require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
url = cfd_cli.get_management_wsurl("logs", config, config_path)
async with connect(url, open_timeout=5, close_timeout=5) as websocket:
# send start_streaming with info logs only
await websocket.send(json.dumps({
"type": "start_streaming",
"filters": {
"sampling": 0.5
}
}))
# don't expect any http logs
count = await generate_and_validate_http_events(websocket, config.get_url(), 10)
assert count < (10 * 2) # There are typically always two log lines for http requests (request and response)
# send stop_streaming
await websocket.send('{"type": "stop_streaming"}')
@pytest.mark.asyncio
async def test_streaming_logs_actor_override(self, tmp_path, component_tests_config):
"""
Validates that a streaming logs session can be overriden by the same actor
"""
print("test_streaming_logs_actor_override")
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(), require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
url = cfd_cli.get_management_wsurl("logs", config, config_path)
task = asyncio.ensure_future(start_streaming_to_be_remotely_closed(url))
override_task = asyncio.ensure_future(start_streaming_override(url))
await asyncio.wait([task, override_task])
assert task.exception() == None, task.exception()
assert override_task.exception() == None, override_task.exception()
async def start_streaming_to_be_remotely_closed(url):
async with connect(url, open_timeout=5, close_timeout=5) as websocket:
try:
await websocket.send(json.dumps({"type": "start_streaming"}))
await asyncio.sleep(10)
assert websocket.closed, "expected this request to be forcibly closed by the override"
except websockets.ConnectionClosed:
# we expect the request to be closed
pass
async def start_streaming_override(url):
# wait for the first connection to be established
await asyncio.sleep(1)
async with connect(url, open_timeout=5, close_timeout=5) as websocket:
await websocket.send(json.dumps({"type": "start_streaming"}))
await asyncio.sleep(1)
await websocket.send(json.dumps({"type": "stop_streaming"}))
await asyncio.sleep(1)
# Every http request has two log lines sent
async def generate_and_validate_http_events(websocket: WebSocketClientProtocol, url: str, count_send: int):
for i in range(count_send):
send_request(url)
# There are typically always two log lines for http requests (request and response)
count = 0
while True:
try:
req_line = await asyncio.wait_for(websocket.recv(), 2)
log_line = json.loads(req_line)
assert log_line["type"] == "logs"
assert log_line["logs"][0]["event"] == "http"
count += 1
except asyncio.TimeoutError:
# ignore timeout from waiting for recv
break
return count
# Every http request has two log lines sent
async def generate_and_validate_no_log_event(websocket: WebSocketClientProtocol, url: str):
send_request(url)
try:
# wait for 5 seconds and make sure we hit the timeout and not recv any events
req_line = await asyncio.wait_for(websocket.recv(), 5)
assert req_line == None, "expected no logs for the specified filters"
except asyncio.TimeoutError:
pass
@retry(stop_max_attempt_number=MAX_RETRIES, wait_fixed=BACKOFF_SECS * 1000)
def send_request(url, headers={}):
with requests.Session() as s:
resp = s.get(url, timeout=BACKOFF_SECS, headers=headers)
assert resp.status_code == 200, f"{url} returned {resp}"
return resp.status_code == 200
+3 -3
View File
@@ -34,7 +34,7 @@ class TestTermination:
def test_graceful_shutdown(self, tmp_path, component_tests_config, signal, protocol):
config = component_tests_config(self._extra_config(protocol))
with start_cloudflared(
tmp_path, config, new_process=True, capture_output=False) as cloudflared:
tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, capture_output=False) as cloudflared:
wait_tunnel_ready(tunnel_url=config.get_url())
connected = threading.Condition()
@@ -56,7 +56,7 @@ class TestTermination:
def test_shutdown_once_no_connection(self, tmp_path, component_tests_config, signal, protocol):
config = component_tests_config(self._extra_config(protocol))
with start_cloudflared(
tmp_path, config, new_process=True, capture_output=False) as cloudflared:
tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, capture_output=False) as cloudflared:
wait_tunnel_ready(tunnel_url=config.get_url())
connected = threading.Condition()
@@ -76,7 +76,7 @@ class TestTermination:
def test_no_connection_shutdown(self, tmp_path, component_tests_config, signal, protocol):
config = component_tests_config(self._extra_config(protocol))
with start_cloudflared(
tmp_path, config, new_process=True, capture_output=False) as cloudflared:
tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, capture_output=False) as cloudflared:
wait_tunnel_ready(tunnel_url=config.get_url())
with self.within_grace_period():
self.terminate_by_signal(cloudflared, signal)
+43 -9
View File
@@ -3,7 +3,9 @@ import requests
from conftest import CfdModes
from constants import METRICS_PORT, MAX_RETRIES, BACKOFF_SECS
from retrying import retry
from util import LOGGER, start_cloudflared, wait_tunnel_ready, send_requests
from cli import CloudflaredCli
from util import LOGGER, write_config, start_cloudflared, wait_tunnel_ready, send_requests
import platform
class TestTunnel:
'''Test tunnels with no ingress rules from config.yaml but ingress rules from CLI only'''
@@ -11,15 +13,47 @@ class TestTunnel:
def test_tunnel_hello_world(self, tmp_path, component_tests_config):
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(),
require_min_connections=4)
require_min_connections=1)
"""
test_get_host_details does the following:
1. It gets a management token from Tunnelstore using cloudflared tail token <tunnel_id>
2. It gets the connector_id after starting a cloudflare tunnel
3. It sends a request to the management host with the connector_id and management token
4. Asserts that the response has a hostname and ip.
"""
def test_get_host_details(self, tmp_path, component_tests_config):
# TUN-7377 : wait_tunnel_ready does not work properly in windows.
# Skipping this test for windows for now and will address it as part of tun-7377
if platform.system() == "Windows":
return
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
headers = {}
headers["Content-Type"] = "application/json"
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1", "--label" , "test"], cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(),
require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
connector_id = cfd_cli.get_connector_id(config)[0]
url = cfd_cli.get_management_url("host_details", config, config_path)
resp = send_request(url, headers=headers)
# Assert response json.
assert resp.status_code == 200, "Expected cloudflared to return 200 for host details"
assert resp.json()["hostname"] == "custom:test", "Expected cloudflared to return hostname"
assert resp.json()["ip"] != "", "Expected cloudflared to return ip"
assert resp.json()["connector_id"] == connector_id, "Expected cloudflared to return connector_id"
def test_tunnel_url(self, tmp_path, component_tests_config):
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--url", f"http://localhost:{METRICS_PORT}/"], new_process=True):
wait_tunnel_ready(require_min_connections=4)
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["run", "--url", f"http://localhost:{METRICS_PORT}/"], new_process=True):
wait_tunnel_ready(require_min_connections=1)
send_requests(config.get_url()+"/ready", 3, True)
def test_tunnel_no_ingress(self, tmp_path, component_tests_config):
@@ -29,8 +63,8 @@ class TestTunnel:
'''
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["run"], new_process=True):
wait_tunnel_ready(require_min_connections=4)
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["run"], new_process=True):
wait_tunnel_ready(require_min_connections=1)
resp = send_request(config.get_url()+"/")
assert resp.status_code == 503, "Expected cloudflared to return 503 for all requests with no ingress defined"
resp = send_request(config.get_url()+"/test")
@@ -38,6 +72,6 @@ class TestTunnel:
@retry(stop_max_attempt_number=MAX_RETRIES, wait_fixed=BACKOFF_SECS * 1000)
def send_request(url):
def send_request(url, headers={}):
with requests.Session() as s:
return s.get(url, timeout=BACKOFF_SECS)
return s.get(url, timeout=BACKOFF_SECS, headers=headers)
+4 -3
View File
@@ -9,6 +9,7 @@ import pytest
import requests
import yaml
import json
from retrying import retry
from constants import METRICS_PORT, MAX_RETRIES, BACKOFF_SECS
@@ -48,7 +49,6 @@ def start_cloudflared(directory, config, cfd_args=["run"], cfd_pre_args=["tunnel
# By setting check=True, it will raise an exception if the process exits with non-zero exit code
return subprocess.run(cmd, check=expect_success, capture_output=capture_output)
def cloudflared_cmd(config, config_path, cfd_args, cfd_pre_args, root):
cmd = []
if root:
@@ -106,13 +106,14 @@ def inner_wait_tunnel_ready(tunnel_url=None, require_min_connections=1):
with requests.Session() as s:
resp = send_request(s, metrics_url, True)
assert resp.json()["readyConnections"] >= require_min_connections, \
ready_connections = resp.json()["readyConnections"]
assert ready_connections >= require_min_connections, \
f"Ready endpoint returned {resp.json()} but we expect at least {require_min_connections} connections"
if tunnel_url is not None:
send_request(s, tunnel_url, True)
def _log_cloudflared_logs(cfd_logs):
log_file = cfd_logs
if os.path.isdir(cfd_logs):
-2
View File
@@ -39,8 +39,6 @@ var (
)
const (
DefaultCredentialFile = "cert.pem"
// BastionFlag is to enable bastion, or jump host, operation
BastionFlag = "bastion"
)
+46
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"io"
"math"
"net"
"net/http"
"strconv"
"strings"
@@ -197,10 +198,55 @@ func (h *HTTPResponseReadWriteAcker) AckConnection(tracePropagation string) erro
return h.w.WriteRespHeaders(resp.StatusCode, resp.Header)
}
// localProxyConnection emulates an incoming connection to cloudflared as a net.Conn.
// Used when handling a "hijacked" connection from connection.ResponseWriter
type localProxyConnection struct {
io.ReadWriteCloser
}
func (c *localProxyConnection) Read(b []byte) (int, error) {
return c.ReadWriteCloser.Read(b)
}
func (c *localProxyConnection) Write(b []byte) (int, error) {
return c.ReadWriteCloser.Write(b)
}
func (c *localProxyConnection) Close() error {
return c.ReadWriteCloser.Close()
}
func (c *localProxyConnection) LocalAddr() net.Addr {
// Unused LocalAddr
return &net.TCPAddr{IP: net.IPv6loopback, Port: 0, Zone: ""}
}
func (c *localProxyConnection) RemoteAddr() net.Addr {
// Unused RemoteAddr
return &net.TCPAddr{IP: net.IPv6loopback, Port: 0, Zone: ""}
}
func (c *localProxyConnection) SetDeadline(t time.Time) error {
// ignored since we can't set the read/write Deadlines for the tunnel back to origintunneld
return nil
}
func (c *localProxyConnection) SetReadDeadline(t time.Time) error {
// ignored since we can't set the read/write Deadlines for the tunnel back to origintunneld
return nil
}
func (c *localProxyConnection) SetWriteDeadline(t time.Time) error {
// ignored since we can't set the read/write Deadlines for the tunnel back to origintunneld
return nil
}
// ResponseWriter is the response path for a request back through cloudflared's tunnel.
type ResponseWriter interface {
WriteRespHeaders(status int, header http.Header) error
AddTrailer(trailerName, trailerValue string)
http.ResponseWriter
http.Hijacker
io.Writer
}
+7 -3
View File
@@ -2,13 +2,13 @@ package connection
import (
"context"
"fmt"
"io"
"net"
"time"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/management"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
@@ -85,7 +85,7 @@ func (c *controlStream) ServeControlStream(
return err
}
c.observer.logServerInfo(c.connIndex, registrationDetails.Location, c.edgeAddress, fmt.Sprintf("Connection %s registered with protocol: %s", registrationDetails.UUID, c.protocol))
c.observer.logConnected(registrationDetails.UUID, c.connIndex, registrationDetails.Location, c.edgeAddress, c.protocol)
c.observer.sendConnectedEvent(c.connIndex, c.protocol, registrationDetails.Location)
c.connectedFuse.Connected()
@@ -116,7 +116,11 @@ func (c *controlStream) waitForUnregister(ctx context.Context, rpcClient NamedTu
c.observer.sendUnregisteringEvent(c.connIndex)
rpcClient.GracefulShutdown(ctx, c.gracePeriod)
c.observer.log.Info().Uint8(LogFieldConnIndex, c.connIndex).Msg("Unregistered tunnel connection")
c.observer.log.Info().
Int(management.EventTypeKey, int(management.Cloudflared)).
Uint8(LogFieldConnIndex, c.connIndex).
IPAddr(LogFieldIPAddress, c.edgeAddress).
Msg("Unregistered tunnel connection")
}
func (c *controlStream) IsStopped() bool {
+42
View File
@@ -1,6 +1,7 @@
package connection
import (
"bufio"
"context"
gojson "encoding/json"
"fmt"
@@ -198,6 +199,8 @@ type http2RespWriter struct {
shouldFlush bool
statusWritten bool
respHeaders http.Header
hijackedMutex sync.Mutex
hijackedv bool
log *zerolog.Logger
}
@@ -218,6 +221,7 @@ func NewHTTP2RespWriter(r *http.Request, w http.ResponseWriter, connType Type, l
w: w,
flusher: flusher,
shouldFlush: connType.shouldFlush(),
respHeaders: make(http.Header),
log: log,
}, nil
}
@@ -232,6 +236,10 @@ func (rp *http2RespWriter) AddTrailer(trailerName, trailerValue string) {
}
func (rp *http2RespWriter) WriteRespHeaders(status int, header http.Header) error {
if rp.hijacked() {
rp.log.Warn().Msg("WriteRespHeaders after hijack")
return nil
}
dest := rp.w.Header()
userHeaders := make(http.Header, len(header))
for name, values := range header {
@@ -282,9 +290,43 @@ func (rp *http2RespWriter) Header() http.Header {
}
func (rp *http2RespWriter) WriteHeader(status int) {
if rp.hijacked() {
rp.log.Warn().Msg("WriteHeader after hijack")
return
}
rp.WriteRespHeaders(status, rp.respHeaders)
}
func (rp *http2RespWriter) hijacked() bool {
rp.hijackedMutex.Lock()
defer rp.hijackedMutex.Unlock()
return rp.hijackedv
}
func (rp *http2RespWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
if !rp.statusWritten {
return nil, nil, fmt.Errorf("status not yet written before attempting to hijack connection")
}
// Make sure to flush anything left in the buffer before hijacking
if rp.shouldFlush {
rp.flusher.Flush()
}
rp.hijackedMutex.Lock()
defer rp.hijackedMutex.Unlock()
if rp.hijackedv {
return nil, nil, http.ErrHijacked
}
rp.hijackedv = true
conn := &localProxyConnection{rp}
// We return the http2RespWriter here because we want to make sure that we flush after every write
// otherwise the HTTP2 write buffer waits a few seconds before sending.
readWriter := bufio.NewReadWriter(
bufio.NewReader(rp),
bufio.NewWriter(rp),
)
return conn, readWriter, nil
}
func (rp *http2RespWriter) WriteErrorResponse() bool {
if rp.statusWritten {
return false
+10 -2
View File
@@ -4,12 +4,17 @@ import (
"net"
"strings"
"github.com/google/uuid"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/management"
)
const (
LogFieldConnectionID = "connection"
LogFieldLocation = "location"
LogFieldIPAddress = "ip"
LogFieldProtocol = "protocol"
observerChannelBufferSize = 16
)
@@ -41,13 +46,16 @@ func (o *Observer) RegisterSink(sink EventSink) {
o.addSinkChan <- sink
}
func (o *Observer) logServerInfo(connIndex uint8, location string, address net.IP, msg string) {
func (o *Observer) logConnected(connectionID uuid.UUID, connIndex uint8, location string, address net.IP, protocol Protocol) {
o.sendEvent(Event{Index: connIndex, EventType: Connected, Location: location})
o.log.Info().
Int(management.EventTypeKey, int(management.Cloudflared)).
Str(LogFieldConnectionID, connectionID.String()).
Uint8(LogFieldConnIndex, connIndex).
Str(LogFieldLocation, location).
IPAddr(LogFieldIPAddress, address).
Msg(msg)
Str(LogFieldProtocol, protocol.String()).
Msg("Registered tunnel connection")
o.metrics.registerServerLocation(uint8ToString(connIndex), location)
}
+27 -6
View File
@@ -1,6 +1,7 @@
package connection
import (
"bufio"
"context"
"crypto/tls"
"fmt"
@@ -24,6 +25,7 @@ import (
"github.com/cloudflare/cloudflared/datagramsession"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/packet"
quicpogs "github.com/cloudflare/cloudflared/quic"
"github.com/cloudflare/cloudflared/tracing"
@@ -297,11 +299,12 @@ func (q *QUICConnection) RegisterUdpSession(ctx context.Context, sessionID uuid.
attribute.String("session-id", sessionID.String()),
attribute.String("dst", fmt.Sprintf("%s:%d", dstIP, dstPort)),
))
log := q.logger.With().Int(management.EventTypeKey, int(management.UDP)).Logger()
// Each session is a series of datagram from an eyeball to a dstIP:dstPort.
// (src port, dst IP, dst port) uniquely identifies a session, so it needs a dedicated connected socket.
originProxy, err := ingress.DialUDP(dstIP, dstPort)
if err != nil {
q.logger.Err(err).Msgf("Failed to create udp proxy to %s:%d", dstIP, dstPort)
log.Err(err).Msgf("Failed to create udp proxy to %s:%d", dstIP, dstPort)
tracing.EndWithErrorStatus(registerSpan, err)
return nil, err
}
@@ -312,14 +315,18 @@ func (q *QUICConnection) RegisterUdpSession(ctx context.Context, sessionID uuid.
session, err := q.sessionManager.RegisterSession(ctx, sessionID, originProxy)
if err != nil {
q.logger.Err(err).Str("sessionID", sessionID.String()).Msgf("Failed to register udp session")
log.Err(err).Str("sessionID", sessionID.String()).Msgf("Failed to register udp session")
tracing.EndWithErrorStatus(registerSpan, err)
return nil, err
}
go q.serveUDPSession(session, closeAfterIdleHint)
q.logger.Debug().Str("sessionID", sessionID.String()).Str("src", originProxy.LocalAddr().String()).Str("dst", fmt.Sprintf("%s:%d", dstIP, dstPort)).Msgf("Registered session")
log.Debug().
Str("sessionID", sessionID.String()).
Str("src", originProxy.LocalAddr().String()).
Str("dst", fmt.Sprintf("%s:%d", dstIP, dstPort)).
Msgf("Registered session")
tracing.End(registerSpan)
resp := tunnelpogs.RegisterUdpSessionResponse{
@@ -340,7 +347,10 @@ func (q *QUICConnection) serveUDPSession(session *datagramsession.Session, close
q.closeUDPSession(ctx, session.ID, "terminated without error")
}
}
q.logger.Debug().Err(err).Str("sessionID", session.ID.String()).Msg("Session terminated")
q.logger.Debug().Err(err).
Int(management.EventTypeKey, int(management.UDP)).
Str("sessionID", session.ID.String()).
Msg("Session terminated")
}
// closeUDPSession first unregisters the session from session manager, then it tries to unregister from edge
@@ -350,7 +360,9 @@ func (q *QUICConnection) closeUDPSession(ctx context.Context, sessionID uuid.UUI
if err != nil {
// Log this at debug because this is not an error if session was closed due to lost connection
// with edge
q.logger.Debug().Err(err).Str("sessionID", sessionID.String()).
q.logger.Debug().Err(err).
Int(management.EventTypeKey, int(management.UDP)).
Str("sessionID", sessionID.String()).
Msgf("Failed to open quic stream to unregister udp session with edge")
return
}
@@ -407,7 +419,7 @@ type httpResponseAdapter struct {
}
func newHTTPResponseAdapter(s *quicpogs.RequestServerStream) httpResponseAdapter {
return httpResponseAdapter{RequestServerStream: s}
return httpResponseAdapter{RequestServerStream: s, headers: make(http.Header)}
}
func (hrw *httpResponseAdapter) AddTrailer(trailerName, trailerValue string) {
@@ -435,6 +447,15 @@ func (hrw *httpResponseAdapter) WriteHeader(status int) {
hrw.WriteRespHeaders(status, hrw.headers)
}
func (hrw *httpResponseAdapter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
conn := &localProxyConnection{hrw.ReadWriteCloser}
readWriter := bufio.NewReadWriter(
bufio.NewReader(hrw.ReadWriteCloser),
bufio.NewWriter(hrw.ReadWriteCloser),
)
return conn, readWriter, nil
}
func (hrw *httpResponseAdapter) WriteErrorResponse(err error) {
hrw.WriteConnectResponseData(err, quicpogs.Metadata{Key: "HttpStatus", Val: strconv.Itoa(http.StatusBadGateway)})
}
+83
View File
@@ -0,0 +1,83 @@
package credentials
import (
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/cfapi"
)
const (
logFieldOriginCertPath = "originCertPath"
)
type User struct {
cert *OriginCert
certPath string
}
func (c User) AccountID() string {
return c.cert.AccountID
}
func (c User) ZoneID() string {
return c.cert.ZoneID
}
func (c User) APIToken() string {
return c.cert.APIToken
}
func (c User) CertPath() string {
return c.certPath
}
// Client uses the user credentials to create a Cloudflare API client
func (c *User) Client(apiURL string, userAgent string, log *zerolog.Logger) (cfapi.Client, error) {
if apiURL == "" {
return nil, errors.New("An api-url was not provided for the Cloudflare API client")
}
client, err := cfapi.NewRESTClient(
apiURL,
c.cert.AccountID,
c.cert.ZoneID,
c.cert.APIToken,
userAgent,
log,
)
if err != nil {
return nil, err
}
return client, nil
}
// Read will load and read the origin cert.pem to load the user credentials
func Read(originCertPath string, log *zerolog.Logger) (*User, error) {
originCertLog := log.With().
Str(logFieldOriginCertPath, originCertPath).
Logger()
originCertPath, err := FindOriginCert(originCertPath, &originCertLog)
if err != nil {
return nil, errors.Wrap(err, "Error locating origin cert")
}
blocks, err := readOriginCert(originCertPath)
if err != nil {
return nil, errors.Wrapf(err, "Can't read origin cert from %s", originCertPath)
}
cert, err := decodeOriginCert(blocks)
if err != nil {
return nil, errors.Wrap(err, "Error decoding origin cert")
}
if cert.AccountID == "" {
return nil, errors.Errorf(`Origin certificate needs to be refreshed before creating new tunnels.\nDelete %s and run "cloudflared login" to obtain a new cert.`, originCertPath)
}
return &User{
cert: cert,
certPath: originCertPath,
}, nil
}
+38
View File
@@ -0,0 +1,38 @@
package credentials
import (
"io/fs"
"os"
"path"
"testing"
"github.com/stretchr/testify/require"
)
func TestCredentialsRead(t *testing.T) {
file, err := os.ReadFile("test-cloudflare-tunnel-cert-json.pem")
require.NoError(t, err)
dir := t.TempDir()
certPath := path.Join(dir, originCertFile)
os.WriteFile(certPath, file, fs.ModePerm)
user, err := Read(certPath, &nopLog)
require.NoError(t, err)
require.Equal(t, certPath, user.CertPath())
require.Equal(t, "test-service-key", user.APIToken())
require.Equal(t, "7b0a4d77dfb881c1a3b7d61ea9443e19", user.ZoneID())
require.Equal(t, "abcdabcdabcdabcd1234567890abcdef", user.AccountID())
}
func TestCredentialsClient(t *testing.T) {
user := User{
certPath: "/tmp/cert.pem",
cert: &OriginCert{
ZoneID: "7b0a4d77dfb881c1a3b7d61ea9443e19",
AccountID: "abcdabcdabcdabcd1234567890abcdef",
APIToken: "test-service-key",
},
}
client, err := user.Client("example.com", "cloudflared/test", &nopLog)
require.NoError(t, err)
require.NotNil(t, client)
}
+130
View File
@@ -0,0 +1,130 @@
package credentials
import (
"encoding/json"
"encoding/pem"
"fmt"
"os"
"path/filepath"
"github.com/mitchellh/go-homedir"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/config"
)
const (
DefaultCredentialFile = "cert.pem"
OriginCertFlag = "origincert"
)
type namedTunnelToken struct {
ZoneID string `json:"zoneID"`
AccountID string `json:"accountID"`
APIToken string `json:"apiToken"`
}
type OriginCert struct {
ZoneID string
APIToken string
AccountID string
}
// FindDefaultOriginCertPath returns the first path that contains a cert.pem file. If none of the
// DefaultConfigSearchDirectories contains a cert.pem file, return empty string
func FindDefaultOriginCertPath() string {
for _, defaultConfigDir := range config.DefaultConfigSearchDirectories() {
originCertPath, _ := homedir.Expand(filepath.Join(defaultConfigDir, DefaultCredentialFile))
if ok := fileExists(originCertPath); ok {
return originCertPath
}
}
return ""
}
func decodeOriginCert(blocks []byte) (*OriginCert, error) {
if len(blocks) == 0 {
return nil, fmt.Errorf("Cannot decode empty certificate")
}
originCert := OriginCert{}
block, rest := pem.Decode(blocks)
for {
if block == nil {
break
}
switch block.Type {
case "PRIVATE KEY", "CERTIFICATE":
// this is for legacy purposes.
break
case "ARGO TUNNEL TOKEN":
if originCert.ZoneID != "" || originCert.APIToken != "" {
return nil, fmt.Errorf("Found multiple tokens in the certificate")
}
// The token is a string,
// Try the newer JSON format
ntt := namedTunnelToken{}
if err := json.Unmarshal(block.Bytes, &ntt); err == nil {
originCert.ZoneID = ntt.ZoneID
originCert.APIToken = ntt.APIToken
originCert.AccountID = ntt.AccountID
}
default:
return nil, fmt.Errorf("Unknown block %s in the certificate", block.Type)
}
block, rest = pem.Decode(rest)
}
if originCert.ZoneID == "" || originCert.APIToken == "" {
return nil, fmt.Errorf("Missing token in the certificate")
}
return &originCert, nil
}
func readOriginCert(originCertPath string) ([]byte, error) {
originCert, err := os.ReadFile(originCertPath)
if err != nil {
return nil, fmt.Errorf("cannot read %s to load origin certificate", originCertPath)
}
return originCert, nil
}
// FindOriginCert will check to make sure that the certificate exists at the specified file path.
func FindOriginCert(originCertPath string, log *zerolog.Logger) (string, error) {
if originCertPath == "" {
log.Error().Msgf("Cannot determine default origin certificate path. No file %s in %v. You need to specify the origin certificate path by specifying the origincert option in the configuration file, or set TUNNEL_ORIGIN_CERT environment variable", DefaultCredentialFile, config.DefaultConfigSearchDirectories())
return "", fmt.Errorf("client didn't specify origincert path")
}
var err error
originCertPath, err = homedir.Expand(originCertPath)
if err != nil {
log.Err(err).Msgf("Cannot resolve origin certificate path")
return "", fmt.Errorf("cannot resolve path %s", originCertPath)
}
// Check that the user has acquired a certificate using the login command
ok := fileExists(originCertPath)
if !ok {
log.Error().Msgf(`Cannot find a valid certificate for your origin at the path:
%s
If the path above is wrong, specify the path with the -origincert option.
If you don't have a certificate signed by Cloudflare, run the command:
cloudflared login
`, originCertPath)
return "", fmt.Errorf("cannot find a valid certificate at the path %s", originCertPath)
}
return originCertPath, nil
}
// FileExists checks to see if a file exist at the provided path.
func fileExists(path string) bool {
fileStat, err := os.Stat(path)
if err != nil {
return false
}
return !fileStat.IsDir()
}
+110
View File
@@ -0,0 +1,110 @@
package credentials
import (
"fmt"
"io/fs"
"os"
"path"
"testing"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const (
originCertFile = "cert.pem"
)
var (
nopLog = zerolog.Nop().With().Logger()
)
func TestLoadOriginCert(t *testing.T) {
cert, err := decodeOriginCert([]byte{})
assert.Equal(t, fmt.Errorf("Cannot decode empty certificate"), err)
assert.Nil(t, cert)
blocks, err := os.ReadFile("test-cert-unknown-block.pem")
assert.NoError(t, err)
cert, err = decodeOriginCert(blocks)
assert.Equal(t, fmt.Errorf("Unknown block RSA PRIVATE KEY in the certificate"), err)
assert.Nil(t, cert)
}
func TestJSONArgoTunnelTokenEmpty(t *testing.T) {
blocks, err := os.ReadFile("test-cert-no-token.pem")
assert.NoError(t, err)
cert, err := decodeOriginCert(blocks)
assert.Equal(t, fmt.Errorf("Missing token in the certificate"), err)
assert.Nil(t, cert)
}
func TestJSONArgoTunnelToken(t *testing.T) {
// The given cert's Argo Tunnel Token was generated by base64 encoding this JSON:
// {
// "zoneID": "7b0a4d77dfb881c1a3b7d61ea9443e19",
// "apiToken": "test-service-key",
// "accountID": "abcdabcdabcdabcd1234567890abcdef"
// }
CloudflareTunnelTokenTest(t, "test-cloudflare-tunnel-cert-json.pem")
}
func CloudflareTunnelTokenTest(t *testing.T, path string) {
blocks, err := os.ReadFile(path)
assert.NoError(t, err)
cert, err := decodeOriginCert(blocks)
assert.NoError(t, err)
assert.NotNil(t, cert)
assert.Equal(t, "7b0a4d77dfb881c1a3b7d61ea9443e19", cert.ZoneID)
key := "test-service-key"
assert.Equal(t, key, cert.APIToken)
}
type mockFile struct {
path string
data []byte
err error
}
type mockFileSystem struct {
files map[string]mockFile
}
func newMockFileSystem(files ...mockFile) *mockFileSystem {
fs := mockFileSystem{map[string]mockFile{}}
for _, f := range files {
fs.files[f.path] = f
}
return &fs
}
func (fs *mockFileSystem) ReadFile(path string) ([]byte, error) {
if f, ok := fs.files[path]; ok {
return f.data, f.err
}
return nil, os.ErrNotExist
}
func (fs *mockFileSystem) ValidFilePath(path string) bool {
_, exists := fs.files[path]
return exists
}
func TestFindOriginCert_Valid(t *testing.T) {
file, err := os.ReadFile("test-cloudflare-tunnel-cert-json.pem")
require.NoError(t, err)
dir := t.TempDir()
certPath := path.Join(dir, originCertFile)
os.WriteFile(certPath, file, fs.ModePerm)
path, err := FindOriginCert(certPath, &nopLog)
require.NoError(t, err)
require.Equal(t, certPath, path)
}
func TestFindOriginCert_Missing(t *testing.T) {
dir := t.TempDir()
certPath := path.Join(dir, originCertFile)
_, err := FindOriginCert(certPath, &nopLog)
require.Error(t, err)
}
+4 -1
View File
@@ -9,6 +9,7 @@ import (
"github.com/google/uuid"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/packet"
)
@@ -120,7 +121,9 @@ func (m *manager) registerSession(ctx context.Context, registration *registerSes
}
func (m *manager) newSession(id uuid.UUID, dstConn io.ReadWriteCloser) *Session {
logger := m.log.With().Str("sessionID", id.String()).Logger()
logger := m.log.With().
Int(management.EventTypeKey, int(management.UDP)).
Str("sessionID", id.String()).Logger()
return &Session{
ID: id,
sendFunc: m.sendFunc,
+19 -7
View File
@@ -9,6 +9,8 @@ import (
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/management"
)
const (
@@ -108,16 +110,20 @@ var friendlyDNSErrorLines = []string{
// EdgeDiscovery implements HA service discovery lookup.
func edgeDiscovery(log *zerolog.Logger, srvService string) ([][]*EdgeAddr, error) {
log.Debug().Str("domain", "_"+srvService+"._"+srvProto+"."+srvName).Msg("looking up edge SRV record")
logger := log.With().Int(management.EventTypeKey, int(management.Cloudflared)).Logger()
logger.Debug().
Int(management.EventTypeKey, int(management.Cloudflared)).
Str("domain", "_"+srvService+"._"+srvProto+"."+srvName).
Msg("edge discovery: looking up edge SRV record")
_, addrs, err := netLookupSRV(srvService, srvProto, srvName)
if err != nil {
_, fallbackAddrs, fallbackErr := fallbackLookupSRV(srvService, srvProto, srvName)
if fallbackErr != nil || len(fallbackAddrs) == 0 {
// use the original DNS error `err` in messages, not `fallbackErr`
log.Err(err).Msg("Error looking up Cloudflare edge IPs: the DNS query failed")
logger.Err(err).Msg("edge discovery: error looking up Cloudflare edge IPs: the DNS query failed")
for _, s := range friendlyDNSErrorLines {
log.Error().Msg(s)
logger.Error().Msg(s)
}
return nil, errors.Wrapf(err, "Could not lookup srv records on _%v._%v.%v", srvService, srvProto, srvName)
}
@@ -131,9 +137,13 @@ func edgeDiscovery(log *zerolog.Logger, srvService string) ([][]*EdgeAddr, error
if err != nil {
return nil, err
}
for _, e := range edgeAddrs {
log.Debug().Msgf("Edge Address: %+v", *e)
logAddrs := make([]string, len(edgeAddrs))
for i, e := range edgeAddrs {
logAddrs[i] = e.UDP.IP.String()
}
logger.Debug().
Strs("addresses", logAddrs).
Msg("edge discovery: resolved edge addresses")
resolvedAddrPerCNAME = append(resolvedAddrPerCNAME, edgeAddrs)
}
@@ -188,13 +198,15 @@ func ResolveAddrs(addrs []string, log *zerolog.Logger) (resolved []*EdgeAddr) {
for _, addr := range addrs {
tcpAddr, err := net.ResolveTCPAddr("tcp", addr)
if err != nil {
log.Error().Str(logFieldAddress, addr).Err(err).Msg("failed to resolve to TCP address")
log.Error().Int(management.EventTypeKey, int(management.Cloudflared)).
Str(logFieldAddress, addr).Err(err).Msg("edge discovery: failed to resolve to TCP address")
continue
}
udpAddr, err := net.ResolveUDPAddr("udp", addr)
if err != nil {
log.Error().Str(logFieldAddress, addr).Err(err).Msg("failed to resolve to UDP address")
log.Error().Int(management.EventTypeKey, int(management.Cloudflared)).
Str(logFieldAddress, addr).Err(err).Msg("edge discovery: failed to resolve to UDP address")
continue
}
version := V6
+21 -17
View File
@@ -6,6 +6,7 @@ import (
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/edgediscovery/allregions"
"github.com/cloudflare/cloudflared/management"
)
const (
@@ -74,33 +75,35 @@ func (ed *Edge) GetAddrForRPC() (*allregions.EdgeAddr, error) {
// GetAddr gives this proxy connection an edge Addr. Prefer Addrs this connection has already used.
func (ed *Edge) GetAddr(connIndex int) (*allregions.EdgeAddr, error) {
log := ed.log.With().Int(LogFieldConnIndex, connIndex).Logger()
log := ed.log.With().
Int(LogFieldConnIndex, connIndex).
Int(management.EventTypeKey, int(management.Cloudflared)).
Logger()
ed.Lock()
defer ed.Unlock()
// If this connection has already used an edge addr, return it.
if addr := ed.regions.AddrUsedBy(connIndex); addr != nil {
log.Debug().Msg("edgediscovery - GetAddr: Returning same address back to proxy connection")
log.Debug().IPAddr(LogFieldIPAddress, addr.UDP.IP).Msg("edge discovery: returning same edge address back to pool")
return addr, nil
}
// Otherwise, give it an unused one
addr := ed.regions.GetUnusedAddr(nil, connIndex)
if addr == nil {
log.Debug().Msg("edgediscovery - GetAddr: No addresses left to give proxy connection")
log.Debug().Msg("edge discovery: no addresses left in pool to give proxy connection")
return nil, errNoAddressesLeft
}
log = ed.log.With().
Int(LogFieldConnIndex, connIndex).
IPAddr(LogFieldIPAddress, addr.UDP.IP).Logger()
log.Debug().Msgf("edgediscovery - GetAddr: Giving connection its new address")
log.Debug().IPAddr(LogFieldIPAddress, addr.UDP.IP).Msg("edge discovery: giving new address to connection")
return addr, nil
}
// GetDifferentAddr gives back the proxy connection's edge Addr and uses a new one.
func (ed *Edge) GetDifferentAddr(connIndex int, hasConnectivityError bool) (*allregions.EdgeAddr, error) {
log := ed.log.With().Int(LogFieldConnIndex, connIndex).Logger()
log := ed.log.With().
Int(LogFieldConnIndex, connIndex).
Int(management.EventTypeKey, int(management.Cloudflared)).
Logger()
ed.Lock()
defer ed.Unlock()
@@ -110,14 +113,14 @@ func (ed *Edge) GetDifferentAddr(connIndex int, hasConnectivityError bool) (*all
}
addr := ed.regions.GetUnusedAddr(oldAddr, connIndex)
if addr == nil {
log.Debug().Msg("edgediscovery - GetDifferentAddr: No addresses left to give proxy connection")
log.Debug().Msg("edge discovery: no addresses left in pool to give proxy connection")
// note: if oldAddr were not nil, it will become available on the next iteration
return nil, errNoAddressesLeft
}
log = ed.log.With().
Int(LogFieldConnIndex, connIndex).
IPAddr(LogFieldIPAddress, addr.UDP.IP).Logger()
log.Debug().Msgf("edgediscovery - GetDifferentAddr: Giving connection its new address from the address list: %v", ed.regions.AvailableAddrs())
log.Debug().
IPAddr(LogFieldIPAddress, addr.UDP.IP).
Int("available", ed.regions.AvailableAddrs()).
Msg("edge discovery: giving new address to connection")
return addr, nil
}
@@ -133,8 +136,9 @@ func (ed *Edge) AvailableAddrs() int {
func (ed *Edge) GiveBack(addr *allregions.EdgeAddr, hasConnectivityError bool) bool {
ed.Lock()
defer ed.Unlock()
log := ed.log.With().
IPAddr(LogFieldIPAddress, addr.UDP.IP).Logger()
log.Debug().Msgf("edgediscovery - GiveBack: Address now unused")
ed.log.Debug().
Int(management.EventTypeKey, int(management.Cloudflared)).
IPAddr(LogFieldIPAddress, addr.UDP.IP).
Msg("edge discovery: gave back address to the pool")
return ed.regions.GiveBack(addr, hasConnectivityError)
}
+30
View File
@@ -0,0 +1,30 @@
package features
const (
FeatureSerializedHeaders = "serialized_headers"
FeatureQuickReconnects = "quick_reconnects"
FeatureAllowRemoteConfig = "allow_remote_config"
FeatureDatagramV2 = "support_datagram_v2"
FeaturePostQuantum = "postquantum"
FeatureQUICSupportEOF = "support_quic_eof"
FeatureManagementLogs = "management_logs"
)
var (
DefaultFeatures = []string{
FeatureAllowRemoteConfig,
FeatureSerializedHeaders,
FeatureDatagramV2,
FeatureQUICSupportEOF,
FeatureManagementLogs,
}
)
func Contains(feature string) bool {
for _, f := range DefaultFeatures {
if f == feature {
return true
}
}
return false
}
+11 -7
View File
@@ -12,6 +12,8 @@ require (
github.com/fsnotify/fsnotify v1.4.9
github.com/getsentry/raven-go v0.2.0
github.com/getsentry/sentry-go v0.16.0
github.com/go-chi/chi/v5 v5.0.8
github.com/go-jose/go-jose/v3 v3.0.0
github.com/gobwas/ws v1.0.4
github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3
github.com/google/gopacket v1.1.19
@@ -35,16 +37,17 @@ require (
go.opentelemetry.io/otel/trace v1.6.3
go.opentelemetry.io/proto/otlp v0.15.0
go.uber.org/automaxprocs v1.4.0
golang.org/x/crypto v0.5.0
golang.org/x/net v0.5.0
golang.org/x/crypto v0.8.0
golang.org/x/net v0.9.0
golang.org/x/sync v0.1.0
golang.org/x/sys v0.4.0
golang.org/x/term v0.4.0
golang.org/x/sys v0.7.0
golang.org/x/term v0.7.0
google.golang.org/protobuf v1.28.1
gopkg.in/coreos/go-oidc.v2 v2.2.1
gopkg.in/natefinch/lumberjack.v2 v2.0.0
gopkg.in/square/go-jose.v2 v2.6.0
gopkg.in/yaml.v3 v3.0.1
nhooyr.io/websocket v1.8.7
zombiezen.com/go/capnproto2 v2.18.0+incompatible
)
@@ -72,6 +75,7 @@ require (
github.com/golang/protobuf v1.5.2 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0 // indirect
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645 // indirect
github.com/klauspost/compress v1.15.11 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/marten-seemann/qtls-go1-16 v0.1.5 // indirect
@@ -91,10 +95,10 @@ require (
github.com/prometheus/common v0.37.0 // indirect
github.com/prometheus/procfs v0.8.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 // indirect
golang.org/x/mod v0.8.0 // indirect
golang.org/x/oauth2 v0.4.0 // indirect
golang.org/x/text v0.6.0 // indirect
golang.org/x/tools v0.1.12 // indirect
golang.org/x/text v0.9.0 // indirect
golang.org/x/tools v0.6.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20221202195650-67e5cbc046fd // indirect
google.golang.org/grpc v1.51.0 // indirect
+51 -14
View File
@@ -166,12 +166,20 @@ github.com/getsentry/raven-go v0.2.0/go.mod h1:KungGk8q33+aIAZUIVWZDr2OfAEBsO49P
github.com/getsentry/sentry-go v0.16.0 h1:owk+S+5XcgJLlGR/3+3s6N4d+uKwqYvh/eS0AIMjPWo=
github.com/getsentry/sentry-go v0.16.0/go.mod h1:ZXCloQLj0pG7mja5NK6NPf2V4A88YJ4pNlc2mOHwh6Y=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
github.com/gin-gonic/gin v1.6.3/go.mod h1:75u5sXoLsGZoRN5Sgbi1eraJ4GU3++wFwWzhwvtwp4M=
github.com/gin-gonic/gin v1.8.1 h1:4+fr/el88TOO3ewCmQr8cx/CtZ/umlIRIs5M4NTNjf8=
github.com/gliderlabs/ssh v0.1.1/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0=
github.com/go-chi/chi/v5 v5.0.8 h1:lD+NLqFcAi1ovnVZpsnObHGW4xb4J8lNmoYVfECH1Y0=
github.com/go-chi/chi/v5 v5.0.8/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
github.com/go-errors/errors v1.0.1/go.mod h1:f4zRHt4oKfwPJE5k8C9vpYG+aDHdBFUsgrm6/TyX73Q=
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-jose/go-jose/v3 v3.0.0 h1:s6rrhirfEP/CGIoc6p+PZAeogN2SxKav6Wp7+dyMWVo=
github.com/go-jose/go-jose/v3 v3.0.0/go.mod h1:RNkWWRld676jZEYoV3+XK8L2ZnNSvIsxFMht0mSX+u8=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=
@@ -185,15 +193,26 @@ github.com/go-logr/logr v1.2.3 h1:2DntVwHkVopvECVRSlL5PSo9eG+cAkDCuckLubN+rq0=
github.com/go-logr/logr v1.2.3/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-playground/assert/v2 v2.0.1/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
github.com/go-playground/locales v0.14.0 h1:u50s323jtVGugKlcYeyzC0etD1HifMjqmJqb8WugfUU=
github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
github.com/go-playground/universal-translator v0.18.0 h1:82dyy6p4OuJq4/CByFNOn/jYrnRPArHwAcmLoJZxyho=
github.com/go-playground/validator/v10 v10.2.0/go.mod h1:uOYAAleCW8F/7oMFd6aG0GOhaH6EGOAJShg8Id5JGkI=
github.com/go-playground/validator/v10 v10.11.1 h1:prmOlTVv+YjZjmRmNSF3VmspqJIxJWXmqUsHwfTRRkQ=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0 h1:p104kn46Q8WdvHunIJ9dAyjPVtrBPhSr3KT2yUst43I=
github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE=
github.com/gobwas/httphead v0.0.0-20180130184737-2c6c146eadee/go.mod h1:L0fX3K22YWvt/FAX9NnzrNzcI4wNYi9Yku4O0LKYflo=
github.com/gobwas/httphead v0.0.0-20200921212729-da3d93bc3c58 h1:YyrUZvJaU8Q0QsoVo+xLFBgWDTam29PKea6GYmwvSiQ=
github.com/gobwas/httphead v0.0.0-20200921212729-da3d93bc3c58/go.mod h1:L0fX3K22YWvt/FAX9NnzrNzcI4wNYi9Yku4O0LKYflo=
github.com/gobwas/pool v0.2.0/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw=
github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og=
github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw=
github.com/gobwas/ws v1.0.2/go.mod h1:szmBTxLgaFppYjEmNtny/v3w89xOydFnnZMcgRRu/EM=
github.com/gobwas/ws v1.0.4 h1:5eXU1CZhpQdq5kXbKb+sECH5Ia5KiO6CYzIzdlVx6Bs=
github.com/gobwas/ws v1.0.4/go.mod h1:szmBTxLgaFppYjEmNtny/v3w89xOydFnnZMcgRRu/EM=
github.com/goccy/go-json v0.9.11 h1:/pAaQDLHEoCq/5FFmSKBswWmK6H0e8g4159Kc/X/nqk=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3 h1:zN2lZNZRflqFyxVaTIU61KNKQ9C0055u9CAfpmqUvo4=
github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3/go.mod h1:nPpo7qLxd6XL3hWJG/O60sR8ZKfMCiIoNap5GvD12KU=
@@ -290,6 +309,7 @@ github.com/googleapis/gax-go/v2 v2.3.0/go.mod h1:b8LNqSzNabLiUpXKkY7HAR5jr6bIT99
github.com/googleapis/gax-go/v2 v2.4.0/go.mod h1:XOTVJ59hdnfJLIP/dh8n5CGryZR2LxK9wbMD5+iXC6c=
github.com/googleapis/go-type-adapters v1.0.0/go.mod h1:zHW75FOG2aur7gAO2B+MLby+cLsWGBF62rFAi7WjWO4=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gorilla/websocket v1.4.1/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gorilla/websocket v1.4.2 h1:+/TMaTYc4QFitKJxsQ7Yye35DkWvkdLcvGKqM+x0Ufc=
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA=
@@ -309,6 +329,7 @@ github.com/ipostelnik/cli/v2 v2.3.1-0.20210324024421-b6ea8234fe3d/go.mod h1:LJmU
github.com/jellevandenhooff/dkim v0.0.0-20150330215556-f50fe3d243e1/go.mod h1:E0B/fFc00Y+Rasa88328GlI/XbtyysCtTHZS8h7IrBU=
github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4=
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
@@ -318,6 +339,9 @@ github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/X
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.10.3/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.15.11 h1:Lcadnb3RKGin4FYM/orgq0qde+nc15E5Cbqg4B9Sx9c=
github.com/klauspost/compress v1.15.11/go.mod h1:QPwzmACJjUTFsnSHH934V6woptycfrDDJnH7hvFVbGM=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
@@ -331,6 +355,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII=
github.com/leodido/go-urn v1.2.1 h1:BqpAaACuzVSgi/VLzGZIobT2z4v53pjosyNd9Yv6n/w=
github.com/lunixbochs/vtclean v1.0.0/go.mod h1:pHhQNgMf3btfWnGBVipUOjRYhoOsdGqdm/+2c2E2WMI=
github.com/mailru/easyjson v0.0.0-20190312143242-1de009706dbe/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/marten-seemann/qpack v0.2.1/go.mod h1:F7Gl5L1jIgN1D11ucXefiuJS9UMVP2opoCp2jDKb7wc=
@@ -340,6 +366,7 @@ github.com/marten-seemann/qtls-go1-17 v0.1.2 h1:JADBlm0LYiVbuSySCHeY863dNkcpMmDR
github.com/marten-seemann/qtls-go1-17 v0.1.2/go.mod h1:C2ekUKcDdz9SDWxec1N/MvcXBpaX9l3Nx67XaR84L5s=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-isatty v0.0.16 h1:bq3VjFmv/sOjHtdEhmkEV4x1AJtvUvOJ2PFAZ5+peKQ=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
@@ -378,6 +405,7 @@ github.com/onsi/gomega v1.23.0/go.mod h1:Z/NWtiqwBrwUt4/2loMmHL63EDLnYHmVbuBpDr2
github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs=
github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc=
github.com/openzipkin/zipkin-go v0.1.1/go.mod h1:NtoC/o8u3JlF1lSlyPNswIbeQH9bJTmOf0Erfk+hxe8=
github.com/pelletier/go-toml/v2 v2.0.5 h1:ipoSadvV8oGUjnUbMub59IDPPwfxF694nG/jwbMiyQg=
github.com/philhofer/fwd v1.1.1 h1:GdGcTjf5RNAxwS4QLsiMzJYj5KEvPJD3Abr261yRQXQ=
github.com/pingcap/errors v0.11.4 h1:lFuQV/oaUMGcD2tqt+01ROSmJs75VG1ToEOkZIZ4nE4=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -471,6 +499,10 @@ github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKs
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07/go.mod h1:kDXzergiv9cbyO7IOYJZWg1U88JhDg3PB6klq9Hg2pA=
github.com/tinylib/msgp v1.1.2 h1:gWmO7n0Ys2RBEb7GPYB9Ujq8Mk5p2U08lRnmMcGy6BQ=
github.com/ugorji/go v1.1.7 h1:/68gy2h+1mWMrwZFeD1kQialdSzAb432dtpeJ42ovdo=
github.com/ugorji/go v1.1.7/go.mod h1:kZn38zHttfInRq0xu/PH0az30d+z6vm202qpg1oXVMw=
github.com/ugorji/go/codec v1.1.7/go.mod h1:Ax+UKWsSmolVDwsd+7N3ZtXu+yMGCf907BLYF3GoBXY=
github.com/ugorji/go/codec v1.2.7 h1:YPXUKf7fYbp/y8xloBqZOw2qaVggbfwMlI8WM3wZUJ0=
github.com/viant/assertly v0.4.8/go.mod h1:aGifi++jvCrUaklKEKT0BU95igDNaqkvz+49uaYMPRU=
github.com/viant/toolbox v0.24.0/go.mod h1:OxMCG57V0PXuIP2HNQrtJf2CjqdmbrOx5EkMILuUhzM=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
@@ -512,12 +544,13 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk
golang.org/x/crypto v0.0.0-20190313024323-a1f597ede03a/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190911031432-227b76d455e7/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200221231518-2aa609cf4a9d/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.5.0 h1:U/0M97KRkSFvyD/3FSmdP5W5swImpNgle/EHFhOsQPE=
golang.org/x/crypto v0.5.0/go.mod h1:NK/OQwhpMQP3MwtdjgLlYHnH9ebylxKWv3e0fK+mkQU=
golang.org/x/crypto v0.8.0 h1:pd9TJtTueMTVQXzk8E2XESSMQDj/U7OUu0PqJqPXQjQ=
golang.org/x/crypto v0.8.0/go.mod h1:mRqEX+O9/h5TFCrQhkgjo2yKi0yYA+9ecGkdQoHrywE=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -554,8 +587,8 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0 h1:LUYupSeNrTNCGzR/hVBk2NHZO4hXcVaW1k4Qx7rjPx8=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -610,8 +643,8 @@ golang.org/x/net v0.0.0-20220425223048-2871e0cb64e4/go.mod h1:CfG3xpIq0wQ8r1q4Su
golang.org/x/net v0.0.0-20220607020251-c690dde0001d/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20220826154423-83b083e8dc8b/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk=
golang.org/x/net v0.5.0 h1:GyT4nK/YDHSqa1c4753ouYCDajOYKTja9Xb/OHtgvSw=
golang.org/x/net v0.5.0/go.mod h1:DivGGAXEgPSlEBzxGzZI+ZLohi+xUj054jfeKui00ws=
golang.org/x/net v0.9.0 h1:aWJ/m6xSmxWBx+V0XRHTlrYrPG56jKsLdTFmsSsCzOM=
golang.org/x/net v0.9.0/go.mod h1:d48xBJpPfHeWQsugry2m+kC02ZBRGRgulfHnEXEuWns=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181017192945-9dcd33a902f4/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
@@ -675,6 +708,7 @@ golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -731,12 +765,12 @@ golang.org/x/sys v0.0.0-20220610221304-9f5ed59c137d/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220808155132-1c4a2a72c664/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.4.0 h1:Zr2JFtRQNX3BCZ8YtxRE9hNJYC8J6I1MVbMg6owUp18=
golang.org/x/sys v0.4.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.7.0 h1:3jlCCIQZPdOYu1h8BkNvLz8Kgwtae2cagcG/VamtZRU=
golang.org/x/sys v0.7.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.4.0 h1:O7UWfv5+A2qiuulQk30kVinPoMtoIPeVaKLEgLpVkvg=
golang.org/x/term v0.4.0/go.mod h1:9P2UbLfCdcvo3p/nzKvsmas4TnlujnuoV9hGgYzW1lQ=
golang.org/x/term v0.7.0 h1:BEvjmm5fURWqcfbSKTdpkDXYBrUS1c0m8agp14W48vQ=
golang.org/x/term v0.7.0/go.mod h1:P32HKFT3hSsZrRxla30E9HqToFYAQPCMs/zFMBUFqPY=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -746,8 +780,8 @@ golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.6.0 h1:3XmdazWV+ubf7QgHSTWeykHOci5oeekaGJBLkrkaw4k=
golang.org/x/text v0.6.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0 h1:2sjJmO8cDvYveuX97RDLsxlyUxLl+GHoLxBiRdHllBE=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -808,8 +842,8 @@ golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.6-0.20210726203631-07bc1bf47fb2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.12 h1:VveCTK38A2rkS8ZqFY25HIDFscX5X9OoEhJd3quQmXU=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0 h1:BOw41kyTf3PuCW1pVQf8+Cyg8pMlkYB1oo9iJ6D/lKM=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -1029,6 +1063,7 @@ gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
@@ -1043,6 +1078,8 @@ honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWh
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
nhooyr.io/websocket v1.8.7 h1:usjR2uOr/zjjkVMy0lW+PPohFok7PCow5sDjLgX4P4g=
nhooyr.io/websocket v1.8.7/go.mod h1:B70DZP8IakI65RVQ51MsWP/8jndNma26DVA/nFSCgW0=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
+19 -21
View File
@@ -35,13 +35,22 @@ const (
// FindMatchingRule returns the index of the Ingress Rule which matches the given
// hostname and path. This function assumes the last rule matches everything,
// which is the case if the rules were instantiated via the ingress#Validate method
// which is the case if the rules were instantiated via the ingress#Validate method.
//
// Negative index rule signifies local cloudflared rules (not-user defined).
func (ing Ingress) FindMatchingRule(hostname, path string) (*Rule, int) {
// The hostname might contain port. We only want to compare the host part with the rule
host, _, err := net.SplitHostPort(hostname)
if err == nil {
hostname = host
}
for i, rule := range ing.LocalRules {
if rule.Matches(hostname, path) {
// Local rule matches return a negative rule index to distiguish local rules from user-defined rules in logs
// Full range would be [-1 .. )
return &rule, -1 - i
}
}
for i, rule := range ing.Rules {
if rule.Matches(hostname, path) {
return &rule, i
@@ -67,6 +76,9 @@ func matchHost(ruleHost, reqHost string) bool {
// Ingress maps eyeball requests to origins.
type Ingress struct {
// Set of ingress rules that are not added to remote config, e.g. management
LocalRules []Rule
// Rules that are provided by the user from remote or local configuration
Rules []Rule `json:"ingress"`
Defaults OriginRequestConfig `json:"originRequest"`
}
@@ -82,7 +94,7 @@ func ParseIngress(conf *config.Configuration) (Ingress, error) {
// ParseIngressFromConfigAndCLI will parse the configuration rules from config files for ingress
// rules and then attempt to parse CLI for ingress rules.
// Will always return at least one valid ingress rule. If none are provided by the user, the default
// will be to return 502 status code for all incoming requests.
// will be to return 503 status code for all incoming requests.
func ParseIngressFromConfigAndCLI(conf *config.Configuration, c *cli.Context, log *zerolog.Logger) (Ingress, error) {
// Attempt to parse ingress rules from configuration
ingressRules, err := ParseIngress(conf)
@@ -98,7 +110,11 @@ func ParseIngressFromConfigAndCLI(conf *config.Configuration, c *cli.Context, lo
// --bastion for ssh bastion service
ingressRules, err = parseCLIIngress(c, false)
if errors.Is(err, ErrNoIngressRulesCLI) {
log.Warn().Msgf(ErrNoIngressRulesCLI.Error())
// Only log a warning if the tunnel is not a remotely managed tunnel and the config
// will be loaded after connecting.
if !c.IsSet("token") {
log.Warn().Msgf(ErrNoIngressRulesCLI.Error())
}
return newDefaultOrigin(c, log), nil
}
if err != nil {
@@ -145,24 +161,6 @@ func newDefaultOrigin(c *cli.Context, log *zerolog.Logger) Ingress {
return ingress
}
// WarpRoutingService starts a tcp stream between the origin and requests from
// warp clients.
type WarpRoutingService struct {
Proxy StreamBasedOriginProxy
}
func NewWarpRoutingService(config WarpRoutingConfig) *WarpRoutingService {
svc := &rawTCPService{
name: ServiceWarpRouting,
dialer: net.Dialer{
Timeout: config.ConnectTimeout.Duration,
KeepAlive: config.TCPKeepAlive.Duration,
},
}
return &WarpRoutingService{Proxy: svc}
}
// Get a single origin service from the CLI/config.
func parseSingleOriginService(c *cli.Context, allowURLFromArgs bool) (OriginService, error) {
if c.IsSet(HelloWorldFlag) {
+6
View File
@@ -17,6 +17,12 @@ type StreamBasedOriginProxy interface {
EstablishConnection(ctx context.Context, dest string) (OriginConnection, error)
}
// HTTPLocalProxy can be implemented by cloudflared services that want to handle incoming http requests.
type HTTPLocalProxy interface {
// Handler is how cloudflared proxies eyeball requests to the local cloudflared services
http.Handler
}
func (o *unixSocketPath) RoundTrip(req *http.Request) (*http.Response, error) {
req.URL.Scheme = o.scheme
return o.transport.RoundTrip(req)
+49
View File
@@ -17,6 +17,7 @@ import (
"github.com/cloudflare/cloudflared/hello"
"github.com/cloudflare/cloudflared/ipaccess"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/socks"
"github.com/cloudflare/cloudflared/tlsconfig"
)
@@ -278,6 +279,54 @@ func (o statusCode) MarshalJSON() ([]byte, error) {
return json.Marshal(o.String())
}
// WarpRoutingService starts a tcp stream between the origin and requests from
// warp clients.
type WarpRoutingService struct {
Proxy StreamBasedOriginProxy
}
func NewWarpRoutingService(config WarpRoutingConfig) *WarpRoutingService {
svc := &rawTCPService{
name: ServiceWarpRouting,
dialer: net.Dialer{
Timeout: config.ConnectTimeout.Duration,
KeepAlive: config.TCPKeepAlive.Duration,
},
}
return &WarpRoutingService{Proxy: svc}
}
// ManagementService starts a local HTTP server to handle incoming management requests.
type ManagementService struct {
HTTPLocalProxy
}
func newManagementService(managementProxy HTTPLocalProxy) *ManagementService {
return &ManagementService{
HTTPLocalProxy: managementProxy,
}
}
func (o *ManagementService) start(log *zerolog.Logger, _ <-chan struct{}, cfg OriginRequestConfig) error {
return nil
}
func (o *ManagementService) String() string {
return "management"
}
func (o ManagementService) MarshalJSON() ([]byte, error) {
return json.Marshal(o.String())
}
func NewManagementRule(management *management.ManagementService) Rule {
return Rule{
Hostname: management.Hostname,
Service: newManagementService(management),
}
}
type NopReadCloser struct{}
// Read always returns EOF to signal end of input
+66
View File
@@ -0,0 +1,66 @@
package test
// copied from https://github.com/nhooyr/websocket/blob/master/internal/test/wstest/pipe.go
import (
"bufio"
"context"
"net"
"net/http"
"net/http/httptest"
"nhooyr.io/websocket"
)
// Pipe is used to create an in memory connection
// between two websockets analogous to net.Pipe.
func WSPipe(dialOpts *websocket.DialOptions, acceptOpts *websocket.AcceptOptions) (clientConn, serverConn *websocket.Conn) {
tt := fakeTransport{
h: func(w http.ResponseWriter, r *http.Request) {
serverConn, _ = websocket.Accept(w, r, acceptOpts)
},
}
if dialOpts == nil {
dialOpts = &websocket.DialOptions{}
}
dialOpts = &*dialOpts
dialOpts.HTTPClient = &http.Client{
Transport: tt,
}
clientConn, _, _ = websocket.Dial(context.Background(), "ws://example.com", dialOpts)
return clientConn, serverConn
}
type fakeTransport struct {
h http.HandlerFunc
}
func (t fakeTransport) RoundTrip(r *http.Request) (*http.Response, error) {
clientConn, serverConn := net.Pipe()
hj := testHijacker{
ResponseRecorder: httptest.NewRecorder(),
serverConn: serverConn,
}
t.h.ServeHTTP(hj, r)
resp := hj.ResponseRecorder.Result()
if resp.StatusCode == http.StatusSwitchingProtocols {
resp.Body = clientConn
}
return resp, nil
}
type testHijacker struct {
*httptest.ResponseRecorder
serverConn net.Conn
}
var _ http.Hijacker = testHijacker{}
func (hj testHijacker) Hijack() (net.Conn, *bufio.ReadWriter, error) {
return hj.serverConn, bufio.NewReadWriter(bufio.NewReader(hj.serverConn), bufio.NewWriter(hj.serverConn)), nil
}
+45 -7
View File
@@ -15,6 +15,9 @@ import (
"github.com/urfave/cli/v2"
"golang.org/x/term"
"gopkg.in/natefinch/lumberjack.v2"
"github.com/cloudflare/cloudflared/features"
"github.com/cloudflare/cloudflared/management"
)
const (
@@ -35,8 +38,19 @@ const (
consoleTimeFormat = time.RFC3339
)
var (
ManagementLogger *management.Logger
)
func init() {
zerolog.TimeFieldFormat = time.RFC3339
zerolog.TimestampFunc = utcNow
if features.Contains(features.FeatureManagementLogs) {
// Management logger needs to be initialized before any of the other loggers as to not capture
// it's own logging events.
ManagementLogger = management.NewLogger()
}
}
func utcNow() time.Time {
@@ -50,17 +64,36 @@ func fallbackLogger(err error) *zerolog.Logger {
return &failLog
}
type resilientMultiWriter struct {
writers []io.Writer
}
// This custom resilientMultiWriter is an alternative to zerolog's so that we can make it resilient to individual
// resilientMultiWriter is an alternative to zerolog's so that we can make it resilient to individual
// writer's errors. E.g., when running as a Windows service, the console writer fails, but we don't want to
// allow that to prevent all logging to fail due to breaking the for loop upon an error.
type resilientMultiWriter struct {
level zerolog.Level
writers []io.Writer
managementWriter zerolog.LevelWriter
}
func (t resilientMultiWriter) Write(p []byte) (n int, err error) {
for _, w := range t.writers {
_, _ = w.Write(p)
}
if t.managementWriter != nil {
_, _ = t.managementWriter.Write(p)
}
return len(p), nil
}
func (t resilientMultiWriter) WriteLevel(level zerolog.Level, p []byte) (n int, err error) {
// Only write the event to normal writers if it exceeds the level, but always write to the
// management logger and let it decided with the provided level of the log event.
if t.level <= level {
for _, w := range t.writers {
_, _ = w.Write(p)
}
}
if t.managementWriter != nil {
_, _ = t.managementWriter.WriteLevel(level, p)
}
return len(p), nil
}
@@ -91,13 +124,18 @@ func newZerolog(loggerConfig *Config) *zerolog.Logger {
writers = append(writers, rollingLogger)
}
multi := resilientMultiWriter{writers}
var managementWriter zerolog.LevelWriter
if features.Contains(features.FeatureManagementLogs) {
managementWriter = ManagementLogger
}
level, levelErr := zerolog.ParseLevel(loggerConfig.MinLevel)
if levelErr != nil {
level = zerolog.InfoLevel
}
log := zerolog.New(multi).With().Timestamp().Logger().Level(level)
multi := resilientMultiWriter{level, writers, managementWriter}
log := zerolog.New(multi).With().Timestamp().Logger()
if !levelErrorLogged && levelErr != nil {
log.Error().Msgf("Failed to parse log level %q, using %q instead", loggerConfig.MinLevel, level)
levelErrorLogged = true
+68 -26
View File
@@ -9,14 +9,13 @@ import (
"github.com/stretchr/testify/assert"
)
var writeCalls int
type mockedWriter struct {
wantErr bool
wantErr bool
writeCalls int
}
func (c mockedWriter) Write(p []byte) (int, error) {
writeCalls++
func (c *mockedWriter) Write(p []byte) (int, error) {
c.writeCalls++
if c.wantErr {
return -1, errors.New("Expected error")
@@ -26,65 +25,108 @@ func (c mockedWriter) Write(p []byte) (int, error) {
}
// Tests that a new writer is only used if it actually works.
func TestResilientMultiWriter(t *testing.T) {
func TestResilientMultiWriter_Errors(t *testing.T) {
tests := []struct {
name string
writers []io.Writer
writers []*mockedWriter
}{
{
name: "All valid writers",
writers: []io.Writer{
mockedWriter{
writers: []*mockedWriter{
{
wantErr: false,
},
mockedWriter{
{
wantErr: false,
},
},
},
{
name: "All invalid writers",
writers: []io.Writer{
mockedWriter{
writers: []*mockedWriter{
{
wantErr: true,
},
mockedWriter{
{
wantErr: true,
},
},
},
{
name: "First invalid writer",
writers: []io.Writer{
mockedWriter{
writers: []*mockedWriter{
{
wantErr: true,
},
mockedWriter{
{
wantErr: false,
},
},
},
{
name: "First valid writer",
writers: []io.Writer{
mockedWriter{
writers: []*mockedWriter{
{
wantErr: false,
},
mockedWriter{
{
wantErr: true,
},
},
},
}
for _, tt := range tests {
writers := tt.writers
multiWriter := resilientMultiWriter{writers}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
writers := []io.Writer{}
for _, w := range test.writers {
writers = append(writers, w)
}
multiWriter := resilientMultiWriter{zerolog.InfoLevel, writers, nil}
logger := zerolog.New(multiWriter).With().Timestamp().Logger().Level(zerolog.InfoLevel)
logger.Info().Msg("Test msg")
logger := zerolog.New(multiWriter).With().Timestamp().Logger()
logger.Info().Msg("Test msg")
assert.Equal(t, len(writers), writeCalls)
writeCalls = 0
for _, w := range test.writers {
// Expect each writer to be written to regardless of the previous writers returning an error
assert.Equal(t, 1, w.writeCalls)
}
})
}
}
type mockedManagementWriter struct {
WriteCalls int
}
func (c *mockedManagementWriter) Write(p []byte) (int, error) {
return len(p), nil
}
func (c *mockedManagementWriter) WriteLevel(level zerolog.Level, p []byte) (int, error) {
c.WriteCalls++
return len(p), nil
}
// Tests that management writer receives write calls of all levels except Disabled
func TestResilientMultiWriter_Management(t *testing.T) {
for _, level := range []zerolog.Level{
zerolog.DebugLevel,
zerolog.InfoLevel,
zerolog.WarnLevel,
zerolog.ErrorLevel,
zerolog.FatalLevel,
zerolog.PanicLevel,
} {
t.Run(level.String(), func(t *testing.T) {
managementWriter := mockedManagementWriter{}
multiWriter := resilientMultiWriter{level, []io.Writer{&mockedWriter{}}, &managementWriter}
logger := zerolog.New(multiWriter).With().Timestamp().Logger()
logger.Info().Msg("Test msg")
// Always write to management
assert.Equal(t, 1, managementWriter.WriteCalls)
})
}
}
+317
View File
@@ -0,0 +1,317 @@
package management
import (
"context"
"errors"
"fmt"
"io"
jsoniter "github.com/json-iterator/go"
"github.com/rs/zerolog"
"nhooyr.io/websocket"
)
var (
errInvalidMessageType = fmt.Errorf("invalid message type was provided")
)
// ServerEventType represents the event types that can come from the server
type ServerEventType string
// ClientEventType represents the event types that can come from the client
type ClientEventType string
const (
UnknownClientEventType ClientEventType = ""
StartStreaming ClientEventType = "start_streaming"
StopStreaming ClientEventType = "stop_streaming"
UnknownServerEventType ServerEventType = ""
Logs ServerEventType = "logs"
)
// ServerEvent is the base struct that informs, based of the Type field, which Event type was provided from the server.
type ServerEvent struct {
Type ServerEventType `json:"type,omitempty"`
// The raw json message is provided to allow better deserialization once the type is known
event jsoniter.RawMessage
}
// ClientEvent is the base struct that informs, based of the Type field, which Event type was provided from the client.
type ClientEvent struct {
Type ClientEventType `json:"type,omitempty"`
// The raw json message is provided to allow better deserialization once the type is known
event jsoniter.RawMessage
}
// EventStartStreaming signifies that the client wishes to start receiving log events.
// Additional filters can be provided to augment the log events requested.
type EventStartStreaming struct {
ClientEvent
Filters *StreamingFilters `json:"filters,omitempty"`
}
type StreamingFilters struct {
Events []LogEventType `json:"events,omitempty"`
Level *LogLevel `json:"level,omitempty"`
Sampling float64 `json:"sampling,omitempty"`
}
// EventStopStreaming signifies that the client wishes to halt receiving log events.
type EventStopStreaming struct {
ClientEvent
}
// EventLog is the event that the server sends to the client with the log events.
type EventLog struct {
ServerEvent
Logs []*Log `json:"logs"`
}
// LogEventType is the way that logging messages are able to be filtered.
// Example: assigning LogEventType.Cloudflared to a zerolog event will allow the client to filter for only
// the Cloudflared-related events.
type LogEventType int8
const (
// Cloudflared events are signficant to cloudflared operations like connection state changes.
// Cloudflared is also the default event type for any events that haven't been separated into a proper event type.
Cloudflared LogEventType = iota
HTTP
TCP
UDP
)
func ParseLogEventType(s string) (LogEventType, bool) {
switch s {
case "cloudflared":
return Cloudflared, true
case "http":
return HTTP, true
case "tcp":
return TCP, true
case "udp":
return UDP, true
}
return -1, false
}
func (l LogEventType) String() string {
switch l {
case Cloudflared:
return "cloudflared"
case HTTP:
return "http"
case TCP:
return "tcp"
case UDP:
return "udp"
default:
return ""
}
}
func (l LogEventType) MarshalJSON() ([]byte, error) {
return json.Marshal(l.String())
}
func (e *LogEventType) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return errors.New("unable to unmarshal LogEventType string")
}
if event, ok := ParseLogEventType(s); ok {
*e = event
return nil
}
return errors.New("unable to unmarshal LogEventType")
}
// LogLevel corresponds to the zerolog logging levels
// "panic", "fatal", and "trace" are exempt from this list as they are rarely used and, at least
// the the first two are limited to failure conditions that lead to cloudflared shutting down.
type LogLevel int8
const (
Debug LogLevel = 0
Info LogLevel = 1
Warn LogLevel = 2
Error LogLevel = 3
)
func ParseLogLevel(l string) (LogLevel, bool) {
switch l {
case "debug":
return Debug, true
case "info":
return Info, true
case "warn":
return Warn, true
case "error":
return Error, true
}
return -1, false
}
func (l LogLevel) String() string {
switch l {
case Debug:
return "debug"
case Info:
return "info"
case Warn:
return "warn"
case Error:
return "error"
default:
return ""
}
}
func (l LogLevel) MarshalJSON() ([]byte, error) {
return json.Marshal(l.String())
}
func (l *LogLevel) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return errors.New("unable to unmarshal LogLevel string")
}
if level, ok := ParseLogLevel(s); ok {
*l = level
return nil
}
return fmt.Errorf("unable to unmarshal LogLevel")
}
const (
// TimeKey aligns with the zerolog.TimeFieldName
TimeKey = "time"
// LevelKey aligns with the zerolog.LevelFieldName
LevelKey = "level"
// LevelKey aligns with the zerolog.MessageFieldName
MessageKey = "message"
// EventTypeKey is the custom JSON key of the LogEventType in ZeroLogEvent
EventTypeKey = "event"
// FieldsKey is a custom JSON key to match and store every other key for a zerolog event
FieldsKey = "fields"
)
// Log is the basic structure of the events that are sent to the client.
type Log struct {
Time string `json:"time,omitempty"`
Level LogLevel `json:"level,omitempty"`
Message string `json:"message,omitempty"`
Event LogEventType `json:"event,omitempty"`
Fields map[string]interface{} `json:"fields,omitempty"`
}
// IntoClientEvent unmarshals the provided ClientEvent into the proper type.
func IntoClientEvent[T EventStartStreaming | EventStopStreaming](e *ClientEvent, eventType ClientEventType) (*T, bool) {
if e.Type != eventType {
return nil, false
}
event := new(T)
err := json.Unmarshal(e.event, event)
if err != nil {
return nil, false
}
return event, true
}
// IntoServerEvent unmarshals the provided ServerEvent into the proper type.
func IntoServerEvent[T EventLog](e *ServerEvent, eventType ServerEventType) (*T, bool) {
if e.Type != eventType {
return nil, false
}
event := new(T)
err := json.Unmarshal(e.event, event)
if err != nil {
return nil, false
}
return event, true
}
// ReadEvent will read a message from the websocket connection and parse it into a valid ServerEvent.
func ReadServerEvent(c *websocket.Conn, ctx context.Context) (*ServerEvent, error) {
message, err := readMessage(c, ctx)
if err != nil {
return nil, err
}
event := ServerEvent{}
if err := json.Unmarshal(message, &event); err != nil {
return nil, err
}
switch event.Type {
case Logs:
event.event = message
return &event, nil
case UnknownServerEventType:
return nil, errInvalidMessageType
default:
return nil, fmt.Errorf("invalid server message type was provided: %s", event.Type)
}
}
// ReadEvent will read a message from the websocket connection and parse it into a valid ClientEvent.
func ReadClientEvent(c *websocket.Conn, ctx context.Context) (*ClientEvent, error) {
message, err := readMessage(c, ctx)
if err != nil {
return nil, err
}
event := ClientEvent{}
if err := json.Unmarshal(message, &event); err != nil {
return nil, err
}
switch event.Type {
case StartStreaming, StopStreaming:
event.event = message
return &event, nil
case UnknownClientEventType:
return nil, errInvalidMessageType
default:
return nil, fmt.Errorf("invalid client message type was provided: %s", event.Type)
}
}
// readMessage will read a message from the websocket connection and return the payload.
func readMessage(c *websocket.Conn, ctx context.Context) ([]byte, error) {
messageType, reader, err := c.Reader(ctx)
if err != nil {
return nil, err
}
if messageType != websocket.MessageText {
return nil, errInvalidMessageType
}
return io.ReadAll(reader)
}
// WriteEvent will write a Event type message to the websocket connection.
func WriteEvent(c *websocket.Conn, ctx context.Context, event any) error {
payload, err := json.Marshal(event)
if err != nil {
return err
}
return c.Write(ctx, websocket.MessageText, payload)
}
// IsClosed returns true if the websocket error is a websocket.CloseError; returns false if not a
// websocket.CloseError
func IsClosed(err error, log *zerolog.Logger) bool {
var closeErr websocket.CloseError
if errors.As(err, &closeErr) {
if closeErr.Code != websocket.StatusNormalClosure {
log.Debug().Msgf("connection is already closed: (%d) %s", closeErr.Code, closeErr.Reason)
}
return true
}
return false
}
func AsClosed(err error) *websocket.CloseError {
var closeErr websocket.CloseError
if errors.As(err, &closeErr) {
return &closeErr
}
return nil
}
+247
View File
@@ -0,0 +1,247 @@
package management
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/require"
"nhooyr.io/websocket"
"github.com/cloudflare/cloudflared/internal/test"
)
var (
debugLevel *LogLevel
infoLevel *LogLevel
warnLevel *LogLevel
errorLevel *LogLevel
)
func init() {
// created here because we can't do a reference to a const enum, i.e. &Info
debugLevel := new(LogLevel)
*debugLevel = Debug
infoLevel := new(LogLevel)
*infoLevel = Info
warnLevel := new(LogLevel)
*warnLevel = Warn
errorLevel := new(LogLevel)
*errorLevel = Error
}
func TestIntoClientEvent_StartStreaming(t *testing.T) {
for _, test := range []struct {
name string
expected EventStartStreaming
}{
{
name: "no filters",
expected: EventStartStreaming{ClientEvent: ClientEvent{Type: StartStreaming}},
},
{
name: "level filter",
expected: EventStartStreaming{
ClientEvent: ClientEvent{Type: StartStreaming},
Filters: &StreamingFilters{
Level: infoLevel,
},
},
},
{
name: "events filter",
expected: EventStartStreaming{
ClientEvent: ClientEvent{Type: StartStreaming},
Filters: &StreamingFilters{
Events: []LogEventType{Cloudflared, HTTP},
},
},
},
{
name: "sampling filter",
expected: EventStartStreaming{
ClientEvent: ClientEvent{Type: StartStreaming},
Filters: &StreamingFilters{
Sampling: 0.5,
},
},
},
{
name: "level and events filters",
expected: EventStartStreaming{
ClientEvent: ClientEvent{Type: StartStreaming},
Filters: &StreamingFilters{
Level: infoLevel,
Events: []LogEventType{Cloudflared},
Sampling: 0.5,
},
},
},
} {
t.Run(test.name, func(t *testing.T) {
data, err := json.Marshal(test.expected)
require.NoError(t, err)
event := ClientEvent{}
err = json.Unmarshal(data, &event)
require.NoError(t, err)
event.event = data
ce, ok := IntoClientEvent[EventStartStreaming](&event, StartStreaming)
require.True(t, ok)
require.Equal(t, test.expected.ClientEvent, ce.ClientEvent)
if test.expected.Filters != nil {
f := ce.Filters
ef := test.expected.Filters
if ef.Level != nil {
require.Equal(t, *ef.Level, *f.Level)
}
require.ElementsMatch(t, ef.Events, f.Events)
}
})
}
}
func TestIntoClientEvent_StopStreaming(t *testing.T) {
event := ClientEvent{
Type: StopStreaming,
event: []byte(`{"type": "stop_streaming"}`),
}
ce, ok := IntoClientEvent[EventStopStreaming](&event, StopStreaming)
require.True(t, ok)
require.Equal(t, EventStopStreaming{ClientEvent: ClientEvent{Type: StopStreaming}}, *ce)
}
func TestIntoClientEvent_Invalid(t *testing.T) {
event := ClientEvent{
Type: UnknownClientEventType,
event: []byte(`{"type": "invalid"}`),
}
_, ok := IntoClientEvent[EventStartStreaming](&event, StartStreaming)
require.False(t, ok)
}
func TestIntoServerEvent_Logs(t *testing.T) {
event := ServerEvent{
Type: Logs,
event: []byte(`{"type": "logs"}`),
}
ce, ok := IntoServerEvent(&event, Logs)
require.True(t, ok)
require.Equal(t, EventLog{ServerEvent: ServerEvent{Type: Logs}}, *ce)
}
func TestIntoServerEvent_Invalid(t *testing.T) {
event := ServerEvent{
Type: UnknownServerEventType,
event: []byte(`{"type": "invalid"}`),
}
_, ok := IntoServerEvent(&event, Logs)
require.False(t, ok)
}
func TestReadServerEvent(t *testing.T) {
sentEvent := EventLog{
ServerEvent: ServerEvent{Type: Logs},
Logs: []*Log{
{
Time: time.Now().UTC().Format(time.RFC3339),
Event: HTTP,
Level: Info,
Message: "test",
},
},
}
client, server := test.WSPipe(nil, nil)
server.CloseRead(context.Background())
defer func() {
server.Close(websocket.StatusInternalError, "")
}()
go func() {
err := WriteEvent(server, context.Background(), &sentEvent)
require.NoError(t, err)
}()
event, err := ReadServerEvent(client, context.Background())
require.NoError(t, err)
require.Equal(t, sentEvent.Type, event.Type)
client.Close(websocket.StatusInternalError, "")
}
func TestReadServerEvent_InvalidWebSocketMessageType(t *testing.T) {
client, server := test.WSPipe(nil, nil)
server.CloseRead(context.Background())
defer func() {
server.Close(websocket.StatusInternalError, "")
}()
go func() {
err := server.Write(context.Background(), websocket.MessageBinary, []byte("test1234"))
require.NoError(t, err)
}()
_, err := ReadServerEvent(client, context.Background())
require.Error(t, err)
client.Close(websocket.StatusInternalError, "")
}
func TestReadServerEvent_InvalidMessageType(t *testing.T) {
sentEvent := ClientEvent{Type: ClientEventType(UnknownServerEventType)}
client, server := test.WSPipe(nil, nil)
server.CloseRead(context.Background())
defer func() {
server.Close(websocket.StatusInternalError, "")
}()
go func() {
err := WriteEvent(server, context.Background(), &sentEvent)
require.NoError(t, err)
}()
_, err := ReadServerEvent(client, context.Background())
require.ErrorIs(t, err, errInvalidMessageType)
client.Close(websocket.StatusInternalError, "")
}
func TestReadClientEvent(t *testing.T) {
sentEvent := EventStartStreaming{
ClientEvent: ClientEvent{Type: StartStreaming},
}
client, server := test.WSPipe(nil, nil)
client.CloseRead(context.Background())
defer func() {
client.Close(websocket.StatusInternalError, "")
}()
go func() {
err := WriteEvent(client, context.Background(), &sentEvent)
require.NoError(t, err)
}()
event, err := ReadClientEvent(server, context.Background())
require.NoError(t, err)
require.Equal(t, sentEvent.Type, event.Type)
server.Close(websocket.StatusInternalError, "")
}
func TestReadClientEvent_InvalidWebSocketMessageType(t *testing.T) {
client, server := test.WSPipe(nil, nil)
client.CloseRead(context.Background())
defer func() {
client.Close(websocket.StatusInternalError, "")
}()
go func() {
err := client.Write(context.Background(), websocket.MessageBinary, []byte("test1234"))
require.NoError(t, err)
}()
_, err := ReadClientEvent(server, context.Background())
require.Error(t, err)
server.Close(websocket.StatusInternalError, "")
}
func TestReadClientEvent_InvalidMessageType(t *testing.T) {
sentEvent := ClientEvent{Type: UnknownClientEventType}
client, server := test.WSPipe(nil, nil)
client.CloseRead(context.Background())
defer func() {
client.Close(websocket.StatusInternalError, "")
}()
go func() {
err := WriteEvent(client, context.Background(), &sentEvent)
require.NoError(t, err)
}()
_, err := ReadClientEvent(server, context.Background())
require.ErrorIs(t, err, errInvalidMessageType)
server.Close(websocket.StatusInternalError, "")
}
+177
View File
@@ -0,0 +1,177 @@
package management
import (
"os"
"sync"
"time"
jsoniter "github.com/json-iterator/go"
"github.com/rs/zerolog"
)
var json = jsoniter.ConfigFastest
// Logger manages the number of management streaming log sessions
type Logger struct {
sessions []*session
mu sync.RWMutex
// Unique logger that isn't a io.Writer of the list of zerolog writers. This helps prevent management log
// statements from creating infinite recursion to export messages to a session and allows basic debugging and
// error statements to be issued in the management code itself.
Log *zerolog.Logger
}
func NewLogger() *Logger {
log := zerolog.New(zerolog.ConsoleWriter{
Out: os.Stdout,
TimeFormat: time.RFC3339,
}).With().Timestamp().Logger().Level(zerolog.InfoLevel)
return &Logger{
Log: &log,
}
}
type LoggerListener interface {
// ActiveSession returns the first active session for the requested actor.
ActiveSession(actor) *session
// ActiveSession returns the count of active sessions.
ActiveSessions() int
// Listen appends the session to the list of sessions that receive log events.
Listen(*session)
// Remove a session from the available sessions that were receiving log events.
Remove(*session)
}
func (l *Logger) ActiveSession(actor actor) *session {
l.mu.RLock()
defer l.mu.RUnlock()
for _, session := range l.sessions {
if session.actor.ID == actor.ID && session.active.Load() {
return session
}
}
return nil
}
func (l *Logger) ActiveSessions() int {
l.mu.RLock()
defer l.mu.RUnlock()
count := 0
for _, session := range l.sessions {
if session.active.Load() {
count += 1
}
}
return count
}
func (l *Logger) Listen(session *session) {
l.mu.Lock()
defer l.mu.Unlock()
session.active.Store(true)
l.sessions = append(l.sessions, session)
}
func (l *Logger) Remove(session *session) {
l.mu.Lock()
defer l.mu.Unlock()
index := -1
for i, v := range l.sessions {
if v == session {
index = i
break
}
}
if index == -1 {
// Not found
return
}
copy(l.sessions[index:], l.sessions[index+1:])
l.sessions = l.sessions[:len(l.sessions)-1]
}
// Write will write the log event to all sessions that have available capacity. For those that are full, the message
// will be dropped.
// This function is the interface that zerolog expects to call when a log event is to be written out.
func (l *Logger) Write(p []byte) (int, error) {
l.mu.RLock()
defer l.mu.RUnlock()
// return early if no active sessions
if len(l.sessions) == 0 {
return len(p), nil
}
event, err := parseZerologEvent(p)
// drop event if unable to parse properly
if err != nil {
l.Log.Debug().Msg("unable to parse log event")
return len(p), nil
}
for _, session := range l.sessions {
session.Insert(event)
}
return len(p), nil
}
func (l *Logger) WriteLevel(level zerolog.Level, p []byte) (n int, err error) {
return l.Write(p)
}
func parseZerologEvent(p []byte) (*Log, error) {
var fields map[string]interface{}
iter := json.BorrowIterator(p)
defer json.ReturnIterator(iter)
iter.ReadVal(&fields)
if iter.Error != nil {
return nil, iter.Error
}
logTime := time.Now().UTC().Format(zerolog.TimeFieldFormat)
if t, ok := fields[TimeKey]; ok {
if t, ok := t.(string); ok {
logTime = t
}
}
logLevel := Debug
// A zerolog Debug event can be created and then an error can be added
// via .Err(error), if so, we upgrade the level to error.
if _, hasError := fields["error"]; hasError {
logLevel = Error
} else {
if level, ok := fields[LevelKey]; ok {
if level, ok := level.(string); ok {
if logLevel, ok = ParseLogLevel(level); !ok {
logLevel = Debug
}
}
}
}
// Assume the event type is Cloudflared if unable to parse/find. This could be from log events that haven't
// yet been tagged with the appropriate EventType yet.
logEvent := Cloudflared
e := fields[EventTypeKey]
if e != nil {
if eventNumber, ok := e.(float64); ok {
logEvent = LogEventType(eventNumber)
}
}
logMessage := ""
if m, ok := fields[MessageKey]; ok {
if m, ok := m.(string); ok {
logMessage = m
}
}
event := Log{
Time: logTime,
Level: logLevel,
Event: logEvent,
Message: logMessage,
}
// Remove the keys that have top level keys on Log
delete(fields, TimeKey)
delete(fields, LevelKey)
delete(fields, EventTypeKey)
delete(fields, MessageKey)
// The rest of the keys go into the Fields
event.Fields = fields
return &event, nil
}
+152
View File
@@ -0,0 +1,152 @@
package management
import (
"context"
"testing"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// No listening sessions will not write to the channel
func TestLoggerWrite_NoSessions(t *testing.T) {
logger := NewLogger()
zlog := zerolog.New(logger).With().Timestamp().Logger().Level(zerolog.InfoLevel)
zlog.Info().Msg("hello")
}
// Validate that the session receives the event
func TestLoggerWrite_OneSession(t *testing.T) {
logger := NewLogger()
zlog := zerolog.New(logger).With().Timestamp().Logger().Level(zerolog.InfoLevel)
_, cancel := context.WithCancel(context.Background())
defer cancel()
session := newSession(logWindow, actor{ID: actorID}, cancel)
logger.Listen(session)
defer logger.Remove(session)
assert.Equal(t, 1, logger.ActiveSessions())
assert.Equal(t, session, logger.ActiveSession(actor{ID: actorID}))
zlog.Info().Int(EventTypeKey, int(HTTP)).Msg("hello")
select {
case event := <-session.listener:
assert.NotEmpty(t, event.Time)
assert.Equal(t, "hello", event.Message)
assert.Equal(t, Info, event.Level)
assert.Equal(t, HTTP, event.Event)
default:
assert.Fail(t, "expected an event to be in the listener")
}
}
// Validate all sessions receive the same event
func TestLoggerWrite_MultipleSessions(t *testing.T) {
logger := NewLogger()
zlog := zerolog.New(logger).With().Timestamp().Logger().Level(zerolog.InfoLevel)
_, cancel := context.WithCancel(context.Background())
defer cancel()
session1 := newSession(logWindow, actor{}, cancel)
logger.Listen(session1)
defer logger.Remove(session1)
assert.Equal(t, 1, logger.ActiveSessions())
session2 := newSession(logWindow, actor{}, cancel)
logger.Listen(session2)
assert.Equal(t, 2, logger.ActiveSessions())
zlog.Info().Int(EventTypeKey, int(HTTP)).Msg("hello")
for _, session := range []*session{session1, session2} {
select {
case event := <-session.listener:
assert.NotEmpty(t, event.Time)
assert.Equal(t, "hello", event.Message)
assert.Equal(t, Info, event.Level)
assert.Equal(t, HTTP, event.Event)
default:
assert.Fail(t, "expected an event to be in the listener")
}
}
// Close session2 and make sure session1 still receives events
logger.Remove(session2)
zlog.Info().Int(EventTypeKey, int(HTTP)).Msg("hello2")
select {
case event := <-session1.listener:
assert.NotEmpty(t, event.Time)
assert.Equal(t, "hello2", event.Message)
assert.Equal(t, Info, event.Level)
assert.Equal(t, HTTP, event.Event)
default:
assert.Fail(t, "expected an event to be in the listener")
}
// Make sure a held reference to session2 doesn't receive events after being closed
select {
case <-session2.listener:
assert.Fail(t, "An event was not expected to be in the session listener")
default:
// pass
}
}
type mockWriter struct {
event *Log
err error
}
func (m *mockWriter) Write(p []byte) (int, error) {
m.event, m.err = parseZerologEvent(p)
return len(p), nil
}
// Validate all event types are set properly
func TestParseZerologEvent_EventTypes(t *testing.T) {
writer := mockWriter{}
zlog := zerolog.New(&writer).With().Timestamp().Logger().Level(zerolog.InfoLevel)
for _, test := range []LogEventType{
Cloudflared,
HTTP,
TCP,
UDP,
} {
t.Run(test.String(), func(t *testing.T) {
defer func() { writer.err = nil }()
zlog.Info().Int(EventTypeKey, int(test)).Msg("test")
require.NoError(t, writer.err)
require.Equal(t, test, writer.event.Event)
})
}
// Invalid defaults to Cloudflared LogEventType
t.Run("invalid", func(t *testing.T) {
defer func() { writer.err = nil }()
zlog.Info().Str(EventTypeKey, "unknown").Msg("test")
require.NoError(t, writer.err)
require.Equal(t, Cloudflared, writer.event.Event)
})
}
// Validate top-level keys are removed from Fields
func TestParseZerologEvent_Fields(t *testing.T) {
writer := mockWriter{}
zlog := zerolog.New(&writer).With().Timestamp().Logger().Level(zerolog.InfoLevel)
zlog.Info().Int(EventTypeKey, int(Cloudflared)).Str("test", "test").Msg("test message")
require.NoError(t, writer.err)
event := writer.event
require.NotEmpty(t, event.Time)
require.Equal(t, Cloudflared, event.Event)
require.Equal(t, Info, event.Level)
require.Equal(t, "test message", event.Message)
// Make sure Fields doesn't have other set keys used in the Log struct
require.NotEmpty(t, event.Fields)
require.Equal(t, "test", event.Fields["test"])
require.NotContains(t, event.Fields, EventTypeKey)
require.NotContains(t, event.Fields, LevelKey)
require.NotContains(t, event.Fields, MessageKey)
require.NotContains(t, event.Fields, TimeKey)
}
+76
View File
@@ -0,0 +1,76 @@
package management
import (
"context"
"fmt"
"net/http"
)
type ctxKey int
const (
accessClaimsCtxKey ctxKey = iota
)
const (
connectorIDQuery = "connector_id"
accessTokenQuery = "access_token"
)
var (
errMissingAccessToken = managementError{Code: 1001, Message: "missing access_token query parameter"}
)
// HTTP middleware setting the parsed access_token claims in the request context
func ValidateAccessTokenQueryMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Validate access token
accessToken := r.URL.Query().Get("access_token")
if accessToken == "" {
writeHTTPErrorResponse(w, errMissingAccessToken)
return
}
token, err := parseToken(accessToken)
if err != nil {
writeHTTPErrorResponse(w, errMissingAccessToken)
return
}
r = r.WithContext(context.WithValue(r.Context(), accessClaimsCtxKey, token))
next.ServeHTTP(w, r)
})
}
// Middleware validation error struct for returning to the eyeball
type managementError struct {
Code int `json:"code,omitempty"`
Message string `json:"message,omitempty"`
}
func (m *managementError) Error() string {
return m.Message
}
// Middleware validation error HTTP response JSON for returning to the eyeball
type managementErrorResponse struct {
Success bool `json:"success,omitempty"`
Errors []managementError `json:"errors,omitempty"`
}
// writeErrorResponse will respond to the eyeball with basic HTTP JSON payloads with validation failure information
func writeHTTPErrorResponse(w http.ResponseWriter, errResp managementError) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusBadRequest)
err := json.NewEncoder(w).Encode(managementErrorResponse{
Success: false,
Errors: []managementError{errResp},
})
// we have already written the header, so write a basic error response if unable to encode the error
if err != nil {
// fallback to text message
http.Error(w, fmt.Sprintf(
"%d %s",
http.StatusBadRequest,
http.StatusText(http.StatusBadRequest),
), http.StatusBadRequest)
}
}
+71
View File
@@ -0,0 +1,71 @@
package management
import (
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
"github.com/stretchr/testify/assert"
)
func TestValidateAccessTokenQueryMiddleware(t *testing.T) {
r := chi.NewRouter()
r.Use(ValidateAccessTokenQueryMiddleware)
r.Get("/valid", func(w http.ResponseWriter, r *http.Request) {
claims, ok := r.Context().Value(accessClaimsCtxKey).(*managementTokenClaims)
assert.True(t, ok)
assert.True(t, claims.verify())
w.WriteHeader(http.StatusOK)
})
r.Get("/invalid", func(w http.ResponseWriter, r *http.Request) {
_, ok := r.Context().Value(accessClaimsCtxKey).(*managementTokenClaims)
assert.False(t, ok)
w.WriteHeader(http.StatusOK)
})
ts := httptest.NewServer(r)
defer ts.Close()
// valid: with access_token query param
path := "/valid?access_token=" + validToken
resp, _ := testRequest(t, ts, "GET", path, nil)
assert.Equal(t, http.StatusOK, resp.StatusCode)
// invalid: unset token
path = "/invalid"
resp, err := testRequest(t, ts, "GET", path, nil)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
assert.NotNil(t, err)
assert.Equal(t, errMissingAccessToken, err.Errors[0])
// invalid: invalid token
path = "/invalid?access_token=eyJ"
resp, err = testRequest(t, ts, "GET", path, nil)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
assert.NotNil(t, err)
assert.Equal(t, errMissingAccessToken, err.Errors[0])
}
func testRequest(t *testing.T, ts *httptest.Server, method, path string, body io.Reader) (*http.Response, *managementErrorResponse) {
req, err := http.NewRequest(method, ts.URL+path, body)
if err != nil {
t.Fatal(err)
return nil, nil
}
resp, err := ts.Client().Do(req)
if err != nil {
t.Fatal(err)
return nil, nil
}
var claims managementErrorResponse
err = json.NewDecoder(resp.Body).Decode(&claims)
if err != nil {
return resp, nil
}
defer resp.Body.Close()
return resp, &claims
}
+302
View File
@@ -0,0 +1,302 @@
package management
import (
"context"
"fmt"
"net"
"net/http"
"os"
"sync"
"time"
"github.com/go-chi/chi/v5"
"github.com/google/uuid"
"github.com/rs/zerolog"
"nhooyr.io/websocket"
)
const (
// In the current state, an invalid command was provided by the client
StatusInvalidCommand websocket.StatusCode = 4001
reasonInvalidCommand = "expected start streaming as first event"
// There are a limited number of available streaming log sessions that cloudflared will service, exceeding this
// value will return this error to incoming requests.
StatusSessionLimitExceeded websocket.StatusCode = 4002
reasonSessionLimitExceeded = "limit exceeded for streaming sessions"
// There is a limited idle time while not actively serving a session for a request before dropping the connection.
StatusIdleLimitExceeded websocket.StatusCode = 4003
reasonIdleLimitExceeded = "session was idle for too long"
)
type ManagementService struct {
// The management tunnel hostname
Hostname string
// Host details related configurations
serviceIP string
clientID uuid.UUID
label string
log *zerolog.Logger
router chi.Router
// streamingMut is a lock to prevent concurrent requests to start streaming. Utilizing the atomic.Bool is not
// sufficient to complete this operation since many other checks during an incoming new request are needed
// to validate this before setting streaming to true.
streamingMut sync.Mutex
logger LoggerListener
}
func New(managementHostname string,
serviceIP string,
clientID uuid.UUID,
label string,
log *zerolog.Logger,
logger LoggerListener,
) *ManagementService {
s := &ManagementService{
Hostname: managementHostname,
log: log,
logger: logger,
serviceIP: serviceIP,
clientID: clientID,
label: label,
}
r := chi.NewRouter()
r.Use(ValidateAccessTokenQueryMiddleware)
r.Get("/ping", ping)
r.Head("/ping", ping)
r.Get("/logs", s.logs)
r.Get("/host_details", s.getHostDetails)
s.router = r
return s
}
func (m *ManagementService) ServeHTTP(w http.ResponseWriter, r *http.Request) {
m.router.ServeHTTP(w, r)
}
// Management Ping handler
func ping(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(200)
}
// The response provided by the /host_details endpoint
type getHostDetailsResponse struct {
ClientID string `json:"connector_id"`
IP string `json:"ip,omitempty"`
HostName string `json:"hostname,omitempty"`
}
func (m *ManagementService) getHostDetails(w http.ResponseWriter, r *http.Request) {
var getHostDetailsResponse = getHostDetailsResponse{
ClientID: m.clientID.String(),
}
if ip, err := getPrivateIP(m.serviceIP); err == nil {
getHostDetailsResponse.IP = ip
}
getHostDetailsResponse.HostName = m.getLabel()
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(200)
json.NewEncoder(w).Encode(getHostDetailsResponse)
}
func (m *ManagementService) getLabel() string {
if m.label != "" {
return fmt.Sprintf("custom:%s", m.label)
}
// If no label is provided we return the system hostname. This is not
// a fqdn hostname.
hostname, err := os.Hostname()
if err != nil {
return "unknown"
}
return hostname
}
// Get preferred private ip of this machine
func getPrivateIP(addr string) (string, error) {
conn, err := net.DialTimeout("tcp", addr, 1*time.Second)
if err != nil {
return "", err
}
defer conn.Close()
localAddr := conn.LocalAddr().String()
host, _, err := net.SplitHostPort(localAddr)
return host, err
}
// readEvents will loop through all incoming websocket messages from a client and marshal them into the
// proper Event structure and pass through to the events channel. Any invalid messages sent will automatically
// terminate the connection.
func (m *ManagementService) readEvents(c *websocket.Conn, ctx context.Context, events chan<- *ClientEvent) {
for {
event, err := ReadClientEvent(c, ctx)
select {
case <-ctx.Done():
return
default:
if err != nil {
// If the client (or the server) already closed the connection, don't attempt to close it again
if !IsClosed(err, m.log) {
m.log.Err(err).Send()
m.log.Err(c.Close(websocket.StatusUnsupportedData, err.Error())).Send()
}
// Any errors when reading the messages from the client will close the connection
return
}
events <- event
}
}
}
// streamLogs will begin the process of reading from the Session listener and write the log events to the client.
func (m *ManagementService) streamLogs(c *websocket.Conn, ctx context.Context, session *session) {
for session.Active() {
select {
case <-ctx.Done():
session.Stop()
return
case event := <-session.listener:
err := WriteEvent(c, ctx, &EventLog{
ServerEvent: ServerEvent{Type: Logs},
Logs: []*Log{event},
})
if err != nil {
// If the client (or the server) already closed the connection, don't attempt to close it again
if !IsClosed(err, m.log) {
m.log.Err(err).Send()
m.log.Err(c.Close(websocket.StatusInternalError, err.Error())).Send()
}
// Any errors when writing the messages to the client will stop streaming and close the connection
session.Stop()
return
}
default:
// No messages to send
}
}
}
// canStartStream will check the conditions of the request and return if the session can begin streaming.
func (m *ManagementService) canStartStream(session *session) bool {
m.streamingMut.Lock()
defer m.streamingMut.Unlock()
// Limits to one actor for streaming logs
if m.logger.ActiveSessions() > 0 {
// Allow the same user to preempt their existing session to disconnect their old session and start streaming
// with this new session instead.
if existingSession := m.logger.ActiveSession(session.actor); existingSession != nil {
m.log.Info().
Msgf("Another management session request for the same actor was requested; the other session will be disconnected to handle the new request.")
existingSession.Stop()
m.logger.Remove(existingSession)
existingSession.cancel()
} else {
m.log.Warn().
Msgf("Another management session request was attempted but one session already being served; there is a limit of streaming log sessions to reduce overall performance impact.")
return false
}
}
return true
}
// parseFilters will check the ClientEvent for start_streaming and assign filters if provided to the session
func (m *ManagementService) parseFilters(c *websocket.Conn, event *ClientEvent, session *session) bool {
// Expect the first incoming request
startEvent, ok := IntoClientEvent[EventStartStreaming](event, StartStreaming)
if !ok {
return false
}
session.Filters(startEvent.Filters)
return true
}
// Management Streaming Logs accept handler
func (m *ManagementService) logs(w http.ResponseWriter, r *http.Request) {
c, err := websocket.Accept(w, r, nil)
if err != nil {
m.log.Debug().Msgf("management handshake: %s", err.Error())
return
}
// Make sure the connection is closed if other go routines fail to close the connection after completing.
defer c.Close(websocket.StatusInternalError, "")
ctx, cancel := context.WithCancel(r.Context())
defer cancel()
events := make(chan *ClientEvent)
go m.readEvents(c, ctx, events)
// Send a heartbeat ping to hold the connection open even if not streaming.
ping := time.NewTicker(15 * time.Second)
defer ping.Stop()
// Close the connection if no operation has occurred after the idle timeout. The timeout is halted
// when streaming logs is active.
idleTimeout := 5 * time.Minute
idle := time.NewTimer(idleTimeout)
defer idle.Stop()
// Fetch the claims from the request context to acquire the actor
claims, ok := ctx.Value(accessClaimsCtxKey).(*managementTokenClaims)
if !ok || claims == nil {
// Typically should never happen as it is provided in the context from the middleware
m.log.Err(c.Close(websocket.StatusInternalError, "missing access_token")).Send()
return
}
session := newSession(logWindow, claims.Actor, cancel)
defer m.logger.Remove(session)
for {
select {
case <-ctx.Done():
m.log.Debug().Msgf("management logs: context cancelled")
c.Close(websocket.StatusNormalClosure, "context closed")
return
case event := <-events:
switch event.Type {
case StartStreaming:
idle.Stop()
// Expect the first incoming request
startEvent, ok := IntoClientEvent[EventStartStreaming](event, StartStreaming)
if !ok {
m.log.Warn().Msgf("expected start_streaming as first recieved event")
m.log.Err(c.Close(StatusInvalidCommand, reasonInvalidCommand)).Send()
return
}
// Make sure the session can start
if !m.canStartStream(session) {
m.log.Err(c.Close(StatusSessionLimitExceeded, reasonSessionLimitExceeded)).Send()
return
}
session.Filters(startEvent.Filters)
m.logger.Listen(session)
m.log.Debug().Msgf("Streaming logs")
go m.streamLogs(c, ctx, session)
continue
case StopStreaming:
idle.Reset(idleTimeout)
// Stop the current session for the current actor who requested it
session.Stop()
m.logger.Remove(session)
case UnknownClientEventType:
fallthrough
default:
// Drop unknown events and close connection
m.log.Debug().Msgf("unexpected management message received: %s", event.Type)
// If the client (or the server) already closed the connection, don't attempt to close it again
if !IsClosed(err, m.log) {
m.log.Err(err).Err(c.Close(websocket.StatusUnsupportedData, err.Error())).Send()
}
return
}
case <-ping.C:
go c.Ping(ctx)
case <-idle.C:
c.Close(StatusIdleLimitExceeded, reasonIdleLimitExceeded)
return
}
}
}
+126
View File
@@ -0,0 +1,126 @@
package management
import (
"context"
"io"
"testing"
"time"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"nhooyr.io/websocket"
"github.com/cloudflare/cloudflared/internal/test"
)
var (
noopLogger = zerolog.New(io.Discard)
)
func TestReadEventsLoop(t *testing.T) {
sentEvent := EventStartStreaming{
ClientEvent: ClientEvent{Type: StartStreaming},
}
client, server := test.WSPipe(nil, nil)
client.CloseRead(context.Background())
defer func() {
client.Close(websocket.StatusInternalError, "")
}()
go func() {
err := WriteEvent(client, context.Background(), &sentEvent)
require.NoError(t, err)
}()
m := ManagementService{
log: &noopLogger,
}
events := make(chan *ClientEvent)
go m.readEvents(server, context.Background(), events)
event := <-events
require.Equal(t, sentEvent.Type, event.Type)
server.Close(websocket.StatusInternalError, "")
}
func TestReadEventsLoop_ContextCancelled(t *testing.T) {
client, server := test.WSPipe(nil, nil)
ctx, cancel := context.WithCancel(context.Background())
client.CloseRead(ctx)
defer func() {
client.Close(websocket.StatusInternalError, "")
}()
m := ManagementService{
log: &noopLogger,
}
events := make(chan *ClientEvent)
go func() {
time.Sleep(time.Second)
cancel()
}()
// Want to make sure this function returns when context is cancelled
m.readEvents(server, ctx, events)
server.Close(websocket.StatusInternalError, "")
}
func TestCanStartStream_NoSessions(t *testing.T) {
m := ManagementService{
log: &noopLogger,
logger: &Logger{
Log: &noopLogger,
},
}
_, cancel := context.WithCancel(context.Background())
session := newSession(0, actor{}, cancel)
assert.True(t, m.canStartStream(session))
}
func TestCanStartStream_ExistingSessionDifferentActor(t *testing.T) {
m := ManagementService{
log: &noopLogger,
logger: &Logger{
Log: &noopLogger,
},
}
_, cancel := context.WithCancel(context.Background())
session1 := newSession(0, actor{ID: "test"}, cancel)
assert.True(t, m.canStartStream(session1))
m.logger.Listen(session1)
assert.True(t, session1.Active())
// Try another session
session2 := newSession(0, actor{ID: "test2"}, cancel)
assert.Equal(t, 1, m.logger.ActiveSessions())
assert.False(t, m.canStartStream(session2))
// Close session1
m.logger.Remove(session1)
assert.True(t, session1.Active()) // Remove doesn't stop a session
session1.Stop()
assert.False(t, session1.Active())
assert.Equal(t, 0, m.logger.ActiveSessions())
// Try session2 again
assert.True(t, m.canStartStream(session2))
}
func TestCanStartStream_ExistingSessionSameActor(t *testing.T) {
m := ManagementService{
log: &noopLogger,
logger: &Logger{
Log: &noopLogger,
},
}
actor := actor{ID: "test"}
_, cancel := context.WithCancel(context.Background())
session1 := newSession(0, actor, cancel)
assert.True(t, m.canStartStream(session1))
m.logger.Listen(session1)
assert.True(t, session1.Active())
// Try another session
session2 := newSession(0, actor, cancel)
assert.Equal(t, 1, m.logger.ActiveSessions())
assert.True(t, m.canStartStream(session2))
// session1 is removed and stopped
assert.Equal(t, 0, m.logger.ActiveSessions())
assert.False(t, session1.Active())
}
+120
View File
@@ -0,0 +1,120 @@
package management
import (
"context"
"math/rand"
"sync/atomic"
)
const (
// Indicates how many log messages the listener will hold before dropping.
// Provides a throttling mechanism to drop latest messages if the sender
// can't keep up with the influx of log messages.
logWindow = 30
)
// session captures a streaming logs session for a connection of an actor.
type session struct {
// Indicates if the session is streaming or not. Modifying this will affect the active session.
active atomic.Bool
// Allows the session to control the context of the underlying connection to close it out when done. Mostly
// used by the LoggerListener to close out and cleanup a session.
cancel context.CancelFunc
// Actor who started the session
actor actor
// Buffered channel that holds the recent log events
listener chan *Log
// Types of log events that this session will provide through the listener
filters *StreamingFilters
// Sampling of the log events this session will send (runs after all other filters if available)
sampler *sampler
}
// NewSession creates a new session.
func newSession(size int, actor actor, cancel context.CancelFunc) *session {
s := &session{
active: atomic.Bool{},
cancel: cancel,
actor: actor,
listener: make(chan *Log, size),
filters: &StreamingFilters{},
}
return s
}
// Filters assigns the StreamingFilters to the session
func (s *session) Filters(filters *StreamingFilters) {
if filters != nil {
s.filters = filters
sampling := filters.Sampling
// clamp the sampling values between 0 and 1
if sampling < 0 {
sampling = 0
}
if sampling > 1 {
sampling = 1
}
s.filters.Sampling = sampling
if sampling > 0 && sampling < 1 {
s.sampler = &sampler{
p: int(sampling * 100),
}
}
} else {
s.filters = &StreamingFilters{}
}
}
// Insert attempts to insert the log to the session. If the log event matches the provided session filters, it
// will be applied to the listener.
func (s *session) Insert(log *Log) {
// Level filters are optional
if s.filters.Level != nil {
if *s.filters.Level > log.Level {
return
}
}
// Event filters are optional
if len(s.filters.Events) != 0 && !contains(s.filters.Events, log.Event) {
return
}
// Sampling is also optional
if s.sampler != nil && !s.sampler.Sample() {
return
}
select {
case s.listener <- log:
default:
// buffer is full, discard
}
}
// Active returns if the session is active
func (s *session) Active() bool {
return s.active.Load()
}
// Stop will halt the session
func (s *session) Stop() {
s.active.Store(false)
}
func contains(array []LogEventType, t LogEventType) bool {
for _, v := range array {
if v == t {
return true
}
}
return false
}
// sampler will send approximately every p percentage log events out of 100.
type sampler struct {
p int
}
// Sample returns true if the event should be part of the sample, false if the event should be dropped.
func (s *sampler) Sample() bool {
return rand.Intn(100) <= s.p
}
+147
View File
@@ -0,0 +1,147 @@
package management
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// Validate the active states of the session
func TestSession_ActiveControl(t *testing.T) {
_, cancel := context.WithCancel(context.Background())
defer cancel()
session := newSession(4, actor{}, cancel)
// session starts out not active
assert.False(t, session.Active())
session.active.Store(true)
assert.True(t, session.Active())
session.Stop()
assert.False(t, session.Active())
}
// Validate that the session filters events
func TestSession_Insert(t *testing.T) {
_, cancel := context.WithCancel(context.Background())
defer cancel()
infoLevel := new(LogLevel)
*infoLevel = Info
warnLevel := new(LogLevel)
*warnLevel = Warn
for _, test := range []struct {
name string
filters StreamingFilters
expectLog bool
}{
{
name: "none",
expectLog: true,
},
{
name: "level",
filters: StreamingFilters{
Level: infoLevel,
},
expectLog: true,
},
{
name: "filtered out level",
filters: StreamingFilters{
Level: warnLevel,
},
expectLog: false,
},
{
name: "events",
filters: StreamingFilters{
Events: []LogEventType{HTTP},
},
expectLog: true,
},
{
name: "filtered out event",
filters: StreamingFilters{
Events: []LogEventType{Cloudflared},
},
expectLog: false,
},
{
name: "sampling",
filters: StreamingFilters{
Sampling: 0.9999999,
},
expectLog: true,
},
{
name: "sampling (invalid negative)",
filters: StreamingFilters{
Sampling: -1.0,
},
expectLog: true,
},
{
name: "sampling (invalid too large)",
filters: StreamingFilters{
Sampling: 2.0,
},
expectLog: true,
},
{
name: "filter and event",
filters: StreamingFilters{
Level: infoLevel,
Events: []LogEventType{HTTP},
},
expectLog: true,
},
} {
t.Run(test.name, func(t *testing.T) {
session := newSession(4, actor{}, cancel)
session.Filters(&test.filters)
log := Log{
Time: time.Now().UTC().Format(time.RFC3339),
Event: HTTP,
Level: Info,
Message: "test",
}
session.Insert(&log)
select {
case <-session.listener:
require.True(t, test.expectLog)
default:
require.False(t, test.expectLog)
}
})
}
}
// Validate that the session has a max amount of events to hold
func TestSession_InsertOverflow(t *testing.T) {
_, cancel := context.WithCancel(context.Background())
defer cancel()
session := newSession(1, actor{}, cancel)
log := Log{
Time: time.Now().UTC().Format(time.RFC3339),
Event: HTTP,
Level: Info,
Message: "test",
}
// Insert 2 but only max channel size for 1
session.Insert(&log)
session.Insert(&log)
select {
case <-session.listener:
// pass
default:
require.Fail(t, "expected one log event")
}
// Second dequeue should fail
select {
case <-session.listener:
require.Fail(t, "expected no more remaining log events")
default:
// pass
}
}
+55
View File
@@ -0,0 +1,55 @@
package management
import (
"fmt"
"github.com/go-jose/go-jose/v3/jwt"
)
type managementTokenClaims struct {
Tunnel tunnel `json:"tun"`
Actor actor `json:"actor"`
}
// VerifyTunnel compares the tun claim isn't empty
func (c *managementTokenClaims) verify() bool {
return c.Tunnel.verify() && c.Actor.verify()
}
type tunnel struct {
ID string `json:"id"`
AccountTag string `json:"account_tag"`
}
// verify compares the tun claim isn't empty
func (t *tunnel) verify() bool {
return t.AccountTag != "" && t.ID != ""
}
type actor struct {
ID string `json:"id"`
Support bool `json:"support"`
}
// verify checks the ID claim isn't empty
func (t *actor) verify() bool {
return t.ID != ""
}
func parseToken(token string) (*managementTokenClaims, error) {
jwt, err := jwt.ParseSigned(token)
if err != nil {
return nil, fmt.Errorf("malformed jwt: %v", err)
}
var claims managementTokenClaims
// This is actually safe because we verify the token in the edge before it reaches cloudflared
err = jwt.UnsafeClaimsWithoutVerification(&claims)
if err != nil {
return nil, fmt.Errorf("malformed jwt: %v", err)
}
if !claims.verify() {
return nil, fmt.Errorf("invalid management token format provided")
}
return &claims, nil
}
+130
View File
@@ -0,0 +1,130 @@
package management
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"errors"
"testing"
"github.com/stretchr/testify/require"
"gopkg.in/square/go-jose.v2"
)
const (
validToken = "eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiIsImtpZCI6IjEifQ.eyJ0dW4iOnsiaWQiOiI3YjA5ODE0OS01MWZlLTRlZTUtYTY4Ny0zZTM3NDQ2NmVmYzciLCJhY2NvdW50X3RhZyI6ImNkMzkxZTljMDYyNmE4Zjc2Y2IxZjY3MGY2NTkxYjA1In0sImFjdG9yIjp7ImlkIjoiZGNhcnJAY2xvdWRmbGFyZS5jb20iLCJzdXBwb3J0IjpmYWxzZX0sInJlcyI6WyJsb2dzIl0sImV4cCI6MTY3NzExNzY5NiwiaWF0IjoxNjc3MTE0MDk2LCJpc3MiOiJ0dW5uZWxzdG9yZSJ9.mKenOdOy3Xi4O-grldFnAAemdlE9WajEpTDC_FwezXQTstWiRTLwU65P5jt4vNsIiZA4OJRq7bH-QYID9wf9NA"
accountTag = "cd391e9c0626a8f76cb1f670f6591b05"
tunnelID = "7b098149-51fe-4ee5-a687-3e374466efc7"
actorID = "45d2751e-6b59-45a9-814d-f630786bd0cd"
)
type invalidManagementTokenClaims struct {
Invalid string `json:"invalid"`
}
func TestParseToken(t *testing.T) {
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
require.NoError(t, err)
for _, test := range []struct {
name string
claims any
err error
}{
{
name: "Valid",
claims: managementTokenClaims{
Tunnel: tunnel{
ID: tunnelID,
AccountTag: accountTag,
},
Actor: actor{
ID: actorID,
},
},
},
{
name: "Invalid claims",
claims: invalidManagementTokenClaims{Invalid: "invalid"},
err: errors.New("invalid management token format provided"),
},
{
name: "Missing Tunnel",
claims: managementTokenClaims{
Actor: actor{
ID: actorID,
},
},
err: errors.New("invalid management token format provided"),
},
{
name: "Missing Tunnel ID",
claims: managementTokenClaims{
Tunnel: tunnel{
AccountTag: accountTag,
},
Actor: actor{
ID: actorID,
},
},
err: errors.New("invalid management token format provided"),
},
{
name: "Missing Account Tag",
claims: managementTokenClaims{
Tunnel: tunnel{
ID: tunnelID,
},
Actor: actor{
ID: actorID,
},
},
err: errors.New("invalid management token format provided"),
},
{
name: "Missing Actor",
claims: managementTokenClaims{
Tunnel: tunnel{
ID: tunnelID,
AccountTag: accountTag,
},
},
err: errors.New("invalid management token format provided"),
},
{
name: "Missing Actor ID",
claims: managementTokenClaims{
Tunnel: tunnel{
ID: tunnelID,
},
Actor: actor{},
},
err: errors.New("invalid management token format provided"),
},
} {
t.Run(test.name, func(t *testing.T) {
jwt := signToken(t, test.claims, key)
claims, err := parseToken(jwt)
if test.err != nil {
require.EqualError(t, err, test.err.Error())
return
}
require.Nil(t, err)
require.Equal(t, test.claims, *claims)
})
}
}
func signToken(t *testing.T, token any, key *ecdsa.PrivateKey) string {
opts := (&jose.SignerOptions{}).WithType("JWT").WithHeader("kid", "1")
signer, err := jose.NewSigner(jose.SigningKey{Algorithm: jose.ES256, Key: key}, opts)
require.NoError(t, err)
payload, err := json.Marshal(token)
require.NoError(t, err)
jws, err := signer.Sign(payload)
require.NoError(t, err)
jwt, err := jws.CompactSerialize()
require.NoError(t, err)
return jwt
}
+15 -14
View File
@@ -28,8 +28,9 @@ type Orchestrator struct {
lock sync.RWMutex
// Underlying value is proxy.Proxy, can be read without the lock, but still needs the lock to update
proxy atomic.Value
// TODO: TUN-6815 Use atomic.Bool once we upgrade to go 1.19. 1 Means enabled and 0 means disabled
warpRoutingEnabled uint32
// Set of local ingress rules defined at cloudflared startup (separate from user-defined ingress rules)
localRules []ingress.Rule
warpRoutingEnabled atomic.Bool
config *Config
tags []tunnelpogs.Tag
log *zerolog.Logger
@@ -40,10 +41,11 @@ type Orchestrator struct {
proxyShutdownC chan<- struct{}
}
func NewOrchestrator(ctx context.Context, config *Config, tags []tunnelpogs.Tag, log *zerolog.Logger) (*Orchestrator, error) {
func NewOrchestrator(ctx context.Context, config *Config, tags []tunnelpogs.Tag, localRules []ingress.Rule, log *zerolog.Logger) (*Orchestrator, error) {
o := &Orchestrator{
// Lowest possible version, any remote configuration will have version higher than this
currentVersion: 0,
localRules: localRules,
config: config,
tags: tags,
log: log,
@@ -112,6 +114,9 @@ func (o *Orchestrator) updateIngress(ingressRules ingress.Ingress, warpRouting i
default:
}
// Assign the local ingress rules to the parsed ingress
ingressRules.LocalRules = o.localRules
// Start new proxy before closing the ones from last version.
// The upside is we don't need to restart proxy from last version, which can fail
// The downside is new version might have ingress rule that require previous version to be shutdown first
@@ -120,14 +125,14 @@ func (o *Orchestrator) updateIngress(ingressRules ingress.Ingress, warpRouting i
if err := ingressRules.StartOrigins(o.log, proxyShutdownC); err != nil {
return errors.Wrap(err, "failed to start origin")
}
newProxy := proxy.NewOriginProxy(ingressRules, warpRouting, o.tags, o.log)
o.proxy.Store(newProxy)
proxy := proxy.NewOriginProxy(ingressRules, warpRouting, o.tags, o.log)
o.proxy.Store(proxy)
o.config.Ingress = &ingressRules
o.config.WarpRouting = warpRouting
if warpRouting.Enabled {
atomic.StoreUint32(&o.warpRoutingEnabled, 1)
o.warpRoutingEnabled.Store(true)
} else {
atomic.StoreUint32(&o.warpRoutingEnabled, 0)
o.warpRoutingEnabled.Store(false)
}
// If proxyShutdownC is nil, there is no previous running proxy
@@ -188,7 +193,7 @@ func (o *Orchestrator) GetOriginProxy() (connection.OriginProxy, error) {
o.log.Error().Msg(err.Error())
return nil, err
}
proxy, ok := val.(*proxy.Proxy)
proxy, ok := val.(connection.OriginProxy)
if !ok {
err := fmt.Errorf("origin proxy has unexpected value %+v", val)
o.log.Error().Msg(err.Error())
@@ -197,12 +202,8 @@ func (o *Orchestrator) GetOriginProxy() (connection.OriginProxy, error) {
return proxy, nil
}
// TODO: TUN-6815 consider storing WarpRouting.Enabled as atomic.Bool once we upgrade to go 1.19
func (o *Orchestrator) WarpRoutingEnabled() (enabled bool) {
if atomic.LoadUint32(&o.warpRoutingEnabled) == 0 {
return false
}
return true
func (o *Orchestrator) WarpRoutingEnabled() bool {
return o.warpRoutingEnabled.Load()
}
func (o *Orchestrator) waitToCloseLastProxy() {
+13 -8
View File
@@ -14,6 +14,7 @@ import (
"time"
"github.com/gobwas/ws/wsutil"
"github.com/google/uuid"
gows "github.com/gorilla/websocket"
"github.com/rs/zerolog"
"github.com/stretchr/testify/require"
@@ -21,7 +22,7 @@ import (
"github.com/cloudflare/cloudflared/config"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/proxy"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/tracing"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
@@ -50,11 +51,11 @@ func TestUpdateConfiguration(t *testing.T) {
initConfig := &Config{
Ingress: &ingress.Ingress{},
}
orchestrator, err := NewOrchestrator(context.Background(), initConfig, testTags, &testLogger)
orchestrator, err := NewOrchestrator(context.Background(), initConfig, testTags, []ingress.Rule{ingress.NewManagementRule(management.New("management.argotunnel.com", "1.1.1.1:80", uuid.Nil, "", &testLogger, nil))}, &testLogger)
require.NoError(t, err)
initOriginProxy, err := orchestrator.GetOriginProxy()
require.NoError(t, err)
require.IsType(t, &proxy.Proxy{}, initOriginProxy)
require.Implements(t, (*connection.OriginProxy)(nil), initOriginProxy)
require.False(t, orchestrator.WarpRoutingEnabled())
configJSONV2 := []byte(`
@@ -95,6 +96,10 @@ func TestUpdateConfiguration(t *testing.T) {
updateWithValidation(t, orchestrator, 2, configJSONV2)
configV2 := orchestrator.config
// Validate local ingress rules
require.Equal(t, "management.argotunnel.com", configV2.Ingress.LocalRules[0].Hostname)
require.True(t, configV2.Ingress.LocalRules[0].Matches("management.argotunnel.com", "/ping"))
require.Equal(t, "management", configV2.Ingress.LocalRules[0].Service.String())
// Validate ingress rule 0
require.Equal(t, "jira.tunnel.org", configV2.Ingress.Rules[0].Hostname)
require.True(t, configV2.Ingress.Rules[0].Matches("jira.tunnel.org", "/login"))
@@ -128,7 +133,7 @@ func TestUpdateConfiguration(t *testing.T) {
originProxyV2, err := orchestrator.GetOriginProxy()
require.NoError(t, err)
require.IsType(t, &proxy.Proxy{}, originProxyV2)
require.Implements(t, (*connection.OriginProxy)(nil), originProxyV2)
require.NotEqual(t, originProxyV2, initOriginProxy)
// Should not downgrade to an older version
@@ -172,7 +177,7 @@ func TestUpdateConfiguration(t *testing.T) {
originProxyV10, err := orchestrator.GetOriginProxy()
require.NoError(t, err)
require.IsType(t, &proxy.Proxy{}, originProxyV10)
require.Implements(t, (*connection.OriginProxy)(nil), originProxyV10)
require.NotEqual(t, originProxyV10, originProxyV2)
}
@@ -257,7 +262,7 @@ func TestConcurrentUpdateAndRead(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
orchestrator, err := NewOrchestrator(ctx, initConfig, testTags, &testLogger)
orchestrator, err := NewOrchestrator(ctx, initConfig, testTags, []ingress.Rule{}, &testLogger)
require.NoError(t, err)
updateWithValidation(t, orchestrator, 1, configJSONV1)
@@ -484,7 +489,7 @@ func TestClosePreviousProxies(t *testing.T) {
)
ctx, cancel := context.WithCancel(context.Background())
orchestrator, err := NewOrchestrator(ctx, initConfig, testTags, &testLogger)
orchestrator, err := NewOrchestrator(ctx, initConfig, testTags, []ingress.Rule{}, &testLogger)
require.NoError(t, err)
updateWithValidation(t, orchestrator, 1, configWithHelloWorld)
@@ -539,7 +544,7 @@ func TestPersistentConnection(t *testing.T) {
initConfig := &Config{
Ingress: &ingress.Ingress{},
}
orchestrator, err := NewOrchestrator(context.Background(), initConfig, testTags, &testLogger)
orchestrator, err := NewOrchestrator(context.Background(), initConfig, testTags, []ingress.Rule{}, &testLogger)
require.NoError(t, err)
wsOrigin := httptest.NewServer(http.HandlerFunc(wsEcho))
+59 -33
View File
@@ -16,6 +16,7 @@ import (
"github.com/cloudflare/cloudflared/cfio"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/stream"
"github.com/cloudflare/cloudflared/tracing"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
@@ -25,10 +26,12 @@ const (
// TagHeaderNamePrefix indicates a Cloudflared Warp Tag prefix that gets appended for warp traffic stream headers.
TagHeaderNamePrefix = "Cf-Warp-Tag-"
LogFieldCFRay = "cfRay"
LogFieldLBProbe = "lbProbe"
LogFieldRule = "ingressRule"
LogFieldOriginService = "originService"
LogFieldFlowID = "flowID"
LogFieldConnIndex = "connIndex"
LogFieldDestAddr = "destAddr"
trailerHeaderName = "Trailer"
)
@@ -37,6 +40,7 @@ const (
type Proxy struct {
ingressRules ingress.Ingress
warpRouting *ingress.WarpRoutingService
management *ingress.ManagementService
tags []tunnelpogs.Tag
log *zerolog.Logger
}
@@ -140,6 +144,9 @@ func (p *Proxy) ProxyHTTP(
return err
}
return nil
case ingress.HTTPLocalProxy:
p.proxyLocalRequest(originProxy, w, req, isWebsocket)
return nil
default:
return fmt.Errorf("Unrecognized service: %s, %t", rule.Service, originProxy)
}
@@ -165,14 +172,24 @@ func (p *Proxy) ProxyTCP(
tracedCtx := tracing.NewTracedContext(serveCtx, req.CfTraceID, p.log)
p.log.Debug().Str(LogFieldFlowID, req.FlowID).Uint8(LogFieldConnIndex, req.ConnIndex).Msg("tcp proxy stream started")
p.log.Debug().
Int(management.EventTypeKey, int(management.TCP)).
Str(LogFieldFlowID, req.FlowID).
Str(LogFieldDestAddr, req.Dest).
Uint8(LogFieldConnIndex, req.ConnIndex).
Msg("tcp proxy stream started")
if err := p.proxyStream(tracedCtx, rwa, req.Dest, p.warpRouting.Proxy); err != nil {
p.logRequestError(err, req.CFRay, req.FlowID, "", ingress.ServiceWarpRouting)
return err
}
p.log.Debug().Str(LogFieldFlowID, req.FlowID).Uint8(LogFieldConnIndex, req.ConnIndex).Msg("tcp proxy stream finished successfully")
p.log.Debug().
Int(management.EventTypeKey, int(management.TCP)).
Str(LogFieldFlowID, req.FlowID).
Str(LogFieldDestAddr, req.Dest).
Uint8(LogFieldConnIndex, req.ConnIndex).
Msg("tcp proxy stream finished successfully")
return nil
}
@@ -302,6 +319,17 @@ func (p *Proxy) proxyStream(
return nil
}
func (p *Proxy) proxyLocalRequest(proxy ingress.HTTPLocalProxy, w connection.ResponseWriter, req *http.Request, isWebsocket bool) {
if isWebsocket {
// These headers are added since they are stripped off during an eyeball request to origintunneld, but they
// are required during the Handshake process of a WebSocket request.
req.Header.Set("Connection", "Upgrade")
req.Header.Set("Upgrade", "websocket")
req.Header.Set("Sec-Websocket-Version", "13")
}
proxy.ServeHTTP(w, req)
}
type bidirectionalStream struct {
reader io.Reader
writer io.Writer
@@ -324,7 +352,7 @@ func (p *Proxy) appendTagHeaders(r *http.Request) {
type logFields struct {
cfRay string
lbProbe bool
rule interface{}
rule int
flowID string
connIndex uint8
}
@@ -338,45 +366,41 @@ func copyTrailers(w connection.ResponseWriter, response *http.Response) {
}
func (p *Proxy) logRequest(r *http.Request, fields logFields) {
log := p.log.With().Int(management.EventTypeKey, int(management.HTTP)).Logger()
event := log.Debug()
if fields.cfRay != "" {
p.log.Debug().Msgf("CF-RAY: %s %s %s %s", fields.cfRay, r.Method, r.URL, r.Proto)
} else if fields.lbProbe {
p.log.Debug().Msgf("CF-RAY: %s Load Balancer health check %s %s %s", fields.cfRay, r.Method, r.URL, r.Proto)
} else {
p.log.Debug().Msgf("All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. %s %s %s ", r.Method, r.URL, r.Proto)
event = event.Str(LogFieldCFRay, fields.cfRay)
}
p.log.Debug().
Str("CF-RAY", fields.cfRay).
Str("Header", fmt.Sprintf("%+v", r.Header)).
if fields.lbProbe {
event = event.Bool(LogFieldLBProbe, fields.lbProbe)
}
if fields.cfRay == "" && !fields.lbProbe {
log.Debug().Msgf("All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. %s %s %s ", r.Method, r.URL, r.Proto)
}
event.
Uint8(LogFieldConnIndex, fields.connIndex).
Str("host", r.Host).
Str("path", r.URL.Path).
Interface("rule", fields.rule).
Uint8(LogFieldConnIndex, fields.connIndex).
Msg("Inbound request")
if contentLen := r.ContentLength; contentLen == -1 {
p.log.Debug().Msgf("CF-RAY: %s Request Content length unknown", fields.cfRay)
} else {
p.log.Debug().Msgf("CF-RAY: %s Request content length %d", fields.cfRay, contentLen)
}
Interface(LogFieldRule, fields.rule).
Interface("headers", r.Header).
Int64("content-length", r.ContentLength).
Msgf("%s %s %s", r.Method, r.URL, r.Proto)
}
func (p *Proxy) logOriginResponse(resp *http.Response, fields logFields) {
responseByCode.WithLabelValues(strconv.Itoa(resp.StatusCode)).Inc()
event := p.log.Debug()
if fields.cfRay != "" {
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("CF-RAY: %s Status: %s served by ingress %d", fields.cfRay, resp.Status, fields.rule)
} else if fields.lbProbe {
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("Response to Load Balancer health check %s", resp.Status)
} else {
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("Status: %s served by ingress %v", resp.Status, fields.rule)
event = event.Str(LogFieldCFRay, fields.cfRay)
}
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("CF-RAY: %s Response Headers %+v", fields.cfRay, resp.Header)
if contentLen := resp.ContentLength; contentLen == -1 {
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("CF-RAY: %s Response content length unknown", fields.cfRay)
} else {
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("CF-RAY: %s Response content length %d", fields.cfRay, contentLen)
if fields.lbProbe {
event = event.Bool(LogFieldLBProbe, fields.lbProbe)
}
event.
Int(management.EventTypeKey, int(management.HTTP)).
Uint8(LogFieldConnIndex, fields.connIndex).
Int64("content-length", resp.ContentLength).
Msgf("%s", resp.Status)
}
func (p *Proxy) logRequestError(err error, cfRay string, flowID string, rule, service string) {
@@ -386,7 +410,9 @@ func (p *Proxy) logRequestError(err error, cfRay string, flowID string, rule, se
log = log.Str(LogFieldCFRay, cfRay)
}
if flowID != "" {
log = log.Str(LogFieldFlowID, flowID)
log = log.Str(LogFieldFlowID, flowID).Int(management.EventTypeKey, int(management.TCP))
} else {
log = log.Int(management.EventTypeKey, int(management.HTTP))
}
if rule != "" {
log = log.Str(LogFieldRule, rule)
@@ -394,7 +420,7 @@ func (p *Proxy) logRequestError(err error, cfRay string, flowID string, rule, se
if service != "" {
log = log.Str(LogFieldOriginService, service)
}
log.Msg("")
log.Send()
}
func getDestFromRule(rule *ingress.Rule, req *http.Request) (string, error) {
+17
View File
@@ -1,6 +1,7 @@
package proxy
import (
"bufio"
"bytes"
"context"
"flag"
@@ -76,6 +77,10 @@ func (w *mockHTTPRespWriter) headers() http.Header {
return w.Header()
}
func (m *mockHTTPRespWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
panic("Hijack not implemented")
}
type mockWSRespWriter struct {
*mockHTTPRespWriter
writeNotification chan []byte
@@ -109,6 +114,10 @@ func (w *mockWSRespWriter) Read(data []byte) (int, error) {
return w.reader.Read(data)
}
func (m *mockWSRespWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
panic("Hijack not implemented")
}
type mockSSERespWriter struct {
*mockHTTPRespWriter
writeNotification chan []byte
@@ -840,6 +849,10 @@ func (w *wsRespWriter) WriteHeader(status int) {
// unused
}
func (m *wsRespWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
panic("Hijack not implemented")
}
type mockTCPRespWriter struct {
w io.Writer
responseHeaders http.Header
@@ -879,6 +892,10 @@ func (m *mockTCPRespWriter) WriteHeader(status int) {
// do nothing
}
func (m *mockTCPRespWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
panic("Hijack not implemented")
}
func createSingleIngressConfig(t *testing.T, service string) ingress.Ingress {
ingressConfig := &config.Configuration{
Ingress: []config.UnvalidatedIngressRule{
+7 -10
View File
@@ -19,8 +19,10 @@ import (
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/edgediscovery"
"github.com/cloudflare/cloudflared/edgediscovery/allregions"
"github.com/cloudflare/cloudflared/features"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/orchestration"
quicpogs "github.com/cloudflare/cloudflared/quic"
"github.com/cloudflare/cloudflared/retry"
@@ -31,13 +33,7 @@ import (
)
const (
dialTimeout = 15 * time.Second
FeatureSerializedHeaders = "serialized_headers"
FeatureQuickReconnects = "quick_reconnects"
FeatureAllowRemoteConfig = "allow_remote_config"
FeatureDatagramV2 = "support_datagram_v2"
FeaturePostQuantum = "postquantum"
FeatureQUICSupportEOF = "support_quic_eof"
dialTimeout = 15 * time.Second
)
type TunnelConfig struct {
@@ -111,11 +107,11 @@ func (c *TunnelConfig) connectionOptions(originLocalAddr string, numPreviousAtte
}
func (c *TunnelConfig) SupportedFeatures() []string {
features := []string{FeatureSerializedHeaders}
supported := []string{features.FeatureSerializedHeaders}
if c.NamedTunnel == nil {
features = append(features, FeatureQuickReconnects)
supported = append(supported, features.FeatureQuickReconnects)
}
return features
return supported
}
func StartTunnelDaemon(
@@ -243,6 +239,7 @@ func (e *EdgeTunnelServer) Serve(ctx context.Context, connIndex uint8, protocolF
}
logger := e.config.Log.With().
Int(management.EventTypeKey, int(management.Cloudflared)).
IPAddr(connection.LogFieldIPAddress, addr.UDP.IP).
Uint8(connection.LogFieldConnIndex, connIndex).
Logger()
+3 -3
View File
@@ -214,19 +214,19 @@ func getToken(appURL *url.URL, appInfo *AppInfo, useHostOnly bool, log *zerolog.
return appToken, nil
}
}
return getTokensFromEdge(appURL, appTokenPath, orgTokenPath, useHostOnly, log)
return getTokensFromEdge(appURL, appInfo.AppAUD, appTokenPath, orgTokenPath, useHostOnly, log)
}
// getTokensFromEdge will attempt to use the transfer service to retrieve an app and org token, save them to disk,
// and return the app token.
func getTokensFromEdge(appURL *url.URL, appTokenPath, orgTokenPath string, useHostOnly bool, log *zerolog.Logger) (string, error) {
func getTokensFromEdge(appURL *url.URL, appAUD, appTokenPath, orgTokenPath string, useHostOnly bool, log *zerolog.Logger) (string, error) {
// If no org token exists or if it couldn't be exchanged for an app token, then run the transfer service flow.
// this weird parameter is the resource name (token) and the key/value
// we want to send to the transfer service. the key is token and the value
// is blank (basically just the id generated in the transfer service)
resourceData, err := RunTransfer(appURL, keyName, keyName, "", true, useHostOnly, log)
resourceData, err := RunTransfer(appURL, appAUD, keyName, keyName, "", true, useHostOnly, log)
if err != nil {
return "", errors.Wrap(err, "failed to run transfer service")
}
+4 -3
View File
@@ -25,12 +25,12 @@ const (
// The "dance" we refer to is building a HTTP request, opening that in a browser waiting for
// the user to complete an action, while it long polls in the background waiting for an
// action to be completed to download the resource.
func RunTransfer(transferURL *url.URL, resourceName, key, value string, shouldEncrypt bool, useHostOnly bool, log *zerolog.Logger) ([]byte, error) {
func RunTransfer(transferURL *url.URL, appAUD, resourceName, key, value string, shouldEncrypt bool, useHostOnly bool, log *zerolog.Logger) ([]byte, error) {
encrypterClient, err := NewEncrypter("cloudflared_priv.pem", "cloudflared_pub.pem")
if err != nil {
return nil, err
}
requestURL, err := buildRequestURL(transferURL, key, value+encrypterClient.PublicKey(), shouldEncrypt, useHostOnly)
requestURL, err := buildRequestURL(transferURL, appAUD, key, value+encrypterClient.PublicKey(), shouldEncrypt, useHostOnly)
if err != nil {
return nil, err
}
@@ -76,9 +76,10 @@ func RunTransfer(transferURL *url.URL, resourceName, key, value string, shouldEn
// BuildRequestURL creates a request suitable for a resource transfer.
// it will return a constructed url based off the base url and query key/value provided.
// cli will build a url for cli transfer request.
func buildRequestURL(baseURL *url.URL, key, value string, cli, useHostOnly bool) (string, error) {
func buildRequestURL(baseURL *url.URL, appAUD string, key, value string, cli, useHostOnly bool) (string, error) {
q := baseURL.Query()
q.Set(key, value)
q.Set("aud", appAUD)
baseURL.RawQuery = q.Encode()
if useHostOnly {
baseURL.Path = ""
+3 -3
View File
@@ -24,17 +24,17 @@ func NewTransportLogger(log *zerolog.Logger, t rpc.Transport) rpc.Transport {
}
func (t *transport) SendMessage(ctx context.Context, msg rpccapnp.Message) error {
t.log.Debug().Msgf("rpcconnect: tx %s", formatMsg(msg))
t.log.Trace().Msgf("rpc tx: %s", formatMsg(msg))
return t.Transport.SendMessage(ctx, msg)
}
func (t *transport) RecvMessage(ctx context.Context) (rpccapnp.Message, error) {
msg, err := t.Transport.RecvMessage(ctx)
if err != nil {
t.log.Debug().Msgf("rpcconnect: rx error: %s", err)
t.log.Debug().Msgf("rpc rx error: %s", err)
return msg, err
}
t.log.Debug().Msgf("rpcconnect: rx %s", formatMsg(msg))
t.log.Trace().Msgf("rpc rx: %s", formatMsg(msg))
return msg, nil
}
+3
View File
@@ -0,0 +1,3 @@
.idea
*.sw?
.vscode
+320
View File
@@ -0,0 +1,320 @@
# Changelog
## v5.0.8 (2022-12-07)
- History of changes: see https://github.com/go-chi/chi/compare/v5.0.7...v5.0.8
## v5.0.7 (2021-11-18)
- History of changes: see https://github.com/go-chi/chi/compare/v5.0.6...v5.0.7
## v5.0.6 (2021-11-15)
- History of changes: see https://github.com/go-chi/chi/compare/v5.0.5...v5.0.6
## v5.0.5 (2021-10-27)
- History of changes: see https://github.com/go-chi/chi/compare/v5.0.4...v5.0.5
## v5.0.4 (2021-08-29)
- History of changes: see https://github.com/go-chi/chi/compare/v5.0.3...v5.0.4
## v5.0.3 (2021-04-29)
- History of changes: see https://github.com/go-chi/chi/compare/v5.0.2...v5.0.3
## v5.0.2 (2021-03-25)
- History of changes: see https://github.com/go-chi/chi/compare/v5.0.1...v5.0.2
## v5.0.1 (2021-03-10)
- Small improvements
- History of changes: see https://github.com/go-chi/chi/compare/v5.0.0...v5.0.1
## v5.0.0 (2021-02-27)
- chi v5, `github.com/go-chi/chi/v5` introduces the adoption of Go's SIV to adhere to the current state-of-the-tools in Go.
- chi v1.5.x did not work out as planned, as the Go tooling is too powerful and chi's adoption is too wide.
The most responsible thing to do for everyone's benefit is to just release v5 with SIV, so I present to you all,
chi v5 at `github.com/go-chi/chi/v5`. I hope someday the developer experience and ergonomics I've been seeking
will still come to fruition in some form, see https://github.com/golang/go/issues/44550
- History of changes: see https://github.com/go-chi/chi/compare/v1.5.4...v5.0.0
## v1.5.4 (2021-02-27)
- Undo prior retraction in v1.5.3 as we prepare for v5.0.0 release
- History of changes: see https://github.com/go-chi/chi/compare/v1.5.3...v1.5.4
## v1.5.3 (2021-02-21)
- Update go.mod to go 1.16 with new retract directive marking all versions without prior go.mod support
- History of changes: see https://github.com/go-chi/chi/compare/v1.5.2...v1.5.3
## v1.5.2 (2021-02-10)
- Reverting allocation optimization as a precaution as go test -race fails.
- Minor improvements, see history below
- History of changes: see https://github.com/go-chi/chi/compare/v1.5.1...v1.5.2
## v1.5.1 (2020-12-06)
- Performance improvement: removing 1 allocation by foregoing context.WithValue, thank you @bouk for
your contribution (https://github.com/go-chi/chi/pull/555). Note: new benchmarks posted in README.
- `middleware.CleanPath`: new middleware that clean's request path of double slashes
- deprecate & remove `chi.ServerBaseContext` in favour of stdlib `http.Server#BaseContext`
- plus other tiny improvements, see full commit history below
- History of changes: see https://github.com/go-chi/chi/compare/v4.1.2...v1.5.1
## v1.5.0 (2020-11-12) - now with go.mod support
`chi` dates back to 2016 with it's original implementation as one of the first routers to adopt the newly introduced
context.Context api to the stdlib -- set out to design a router that is faster, more modular and simpler than anything
else out there -- while not introducing any custom handler types or dependencies. Today, `chi` still has zero dependencies,
and in many ways is future proofed from changes, given it's minimal nature. Between versions, chi's iterations have been very
incremental, with the architecture and api being the same today as it was originally designed in 2016. For this reason it
makes chi a pretty easy project to maintain, as well thanks to the many amazing community contributions over the years
to who all help make chi better (total of 86 contributors to date -- thanks all!).
Chi has been an labour of love, art and engineering, with the goals to offer beautiful ergonomics, flexibility, performance
and simplicity when building HTTP services with Go. I've strived to keep the router very minimal in surface area / code size,
and always improving the code wherever possible -- and as of today the `chi` package is just 1082 lines of code (not counting
middlewares, which are all optional). As well, I don't have the exact metrics, but from my analysis and email exchanges from
companies and developers, chi is used by thousands of projects around the world -- thank you all as there is no better form of
joy for me than to have art I had started be helpful and enjoyed by others. And of course I use chi in all of my own projects too :)
For me, the asthetics of chi's code and usage are very important. With the introduction of Go's module support
(which I'm a big fan of), chi's past versioning scheme choice to v2, v3 and v4 would mean I'd require the import path
of "github.com/go-chi/chi/v4", leading to the lengthy discussion at https://github.com/go-chi/chi/issues/462.
Haha, to some, you may be scratching your head why I've spent > 1 year stalling to adopt "/vXX" convention in the import
path -- which isn't horrible in general -- but for chi, I'm unable to accept it as I strive for perfection in it's API design,
aesthetics and simplicity. It just doesn't feel good to me given chi's simple nature -- I do not foresee a "v5" or "v6",
and upgrading between versions in the future will also be just incremental.
I do understand versioning is a part of the API design as well, which is why the solution for a while has been to "do nothing",
as Go supports both old and new import paths with/out go.mod. However, now that Go module support has had time to iron out kinks and
is adopted everywhere, it's time for chi to get with the times. Luckily, I've discovered a path forward that will make me happy,
while also not breaking anyone's app who adopted a prior versioning from tags in v2/v3/v4. I've made an experimental release of
v1.5.0 with go.mod silently, and tested it with new and old projects, to ensure the developer experience is preserved, and it's
largely unnoticed. Fortunately, Go's toolchain will check the tags of a repo and consider the "latest" tag the one with go.mod.
However, you can still request a specific older tag such as v4.1.2, and everything will "just work". But new users can just
`go get github.com/go-chi/chi` or `go get github.com/go-chi/chi@latest` and they will get the latest version which contains
go.mod support, which is v1.5.0+. `chi` will not change very much over the years, just like it hasn't changed much from 4 years ago.
Therefore, we will stay on v1.x from here on, starting from v1.5.0. Any breaking changes will bump a "minor" release and
backwards-compatible improvements/fixes will bump a "tiny" release.
For existing projects who want to upgrade to the latest go.mod version, run: `go get -u github.com/go-chi/chi@v1.5.0`,
which will get you on the go.mod version line (as Go's mod cache may still remember v4.x). Brand new systems can run
`go get -u github.com/go-chi/chi` or `go get -u github.com/go-chi/chi@latest` to install chi, which will install v1.5.0+
built with go.mod support.
My apologies to the developers who will disagree with the decisions above, but, hope you'll try it and see it's a very
minor request which is backwards compatible and won't break your existing installations.
Cheers all, happy coding!
---
## v4.1.2 (2020-06-02)
- fix that handles MethodNotAllowed with path variables, thank you @caseyhadden for your contribution
- fix to replace nested wildcards correctly in RoutePattern, thank you @@unmultimedio for your contribution
- History of changes: see https://github.com/go-chi/chi/compare/v4.1.1...v4.1.2
## v4.1.1 (2020-04-16)
- fix for issue https://github.com/go-chi/chi/issues/411 which allows for overlapping regexp
route to the correct handler through a recursive tree search, thanks to @Jahaja for the PR/fix!
- new middleware.RouteHeaders as a simple router for request headers with wildcard support
- History of changes: see https://github.com/go-chi/chi/compare/v4.1.0...v4.1.1
## v4.1.0 (2020-04-1)
- middleware.LogEntry: Write method on interface now passes the response header
and an extra interface type useful for custom logger implementations.
- middleware.WrapResponseWriter: minor fix
- middleware.Recoverer: a bit prettier
- History of changes: see https://github.com/go-chi/chi/compare/v4.0.4...v4.1.0
## v4.0.4 (2020-03-24)
- middleware.Recoverer: new pretty stack trace printing (https://github.com/go-chi/chi/pull/496)
- a few minor improvements and fixes
- History of changes: see https://github.com/go-chi/chi/compare/v4.0.3...v4.0.4
## v4.0.3 (2020-01-09)
- core: fix regexp routing to include default value when param is not matched
- middleware: rewrite of middleware.Compress
- middleware: suppress http.ErrAbortHandler in middleware.Recoverer
- History of changes: see https://github.com/go-chi/chi/compare/v4.0.2...v4.0.3
## v4.0.2 (2019-02-26)
- Minor fixes
- History of changes: see https://github.com/go-chi/chi/compare/v4.0.1...v4.0.2
## v4.0.1 (2019-01-21)
- Fixes issue with compress middleware: #382 #385
- History of changes: see https://github.com/go-chi/chi/compare/v4.0.0...v4.0.1
## v4.0.0 (2019-01-10)
- chi v4 requires Go 1.10.3+ (or Go 1.9.7+) - we have deprecated support for Go 1.7 and 1.8
- router: respond with 404 on router with no routes (#362)
- router: additional check to ensure wildcard is at the end of a url pattern (#333)
- middleware: deprecate use of http.CloseNotifier (#347)
- middleware: fix RedirectSlashes to include query params on redirect (#334)
- History of changes: see https://github.com/go-chi/chi/compare/v3.3.4...v4.0.0
## v3.3.4 (2019-01-07)
- Minor middleware improvements. No changes to core library/router. Moving v3 into its
- own branch as a version of chi for Go 1.7, 1.8, 1.9, 1.10, 1.11
- History of changes: see https://github.com/go-chi/chi/compare/v3.3.3...v3.3.4
## v3.3.3 (2018-08-27)
- Minor release
- See https://github.com/go-chi/chi/compare/v3.3.2...v3.3.3
## v3.3.2 (2017-12-22)
- Support to route trailing slashes on mounted sub-routers (#281)
- middleware: new `ContentCharset` to check matching charsets. Thank you
@csucu for your community contribution!
## v3.3.1 (2017-11-20)
- middleware: new `AllowContentType` handler for explicit whitelist of accepted request Content-Types
- middleware: new `SetHeader` handler for short-hand middleware to set a response header key/value
- Minor bug fixes
## v3.3.0 (2017-10-10)
- New chi.RegisterMethod(method) to add support for custom HTTP methods, see _examples/custom-method for usage
- Deprecated LINK and UNLINK methods from the default list, please use `chi.RegisterMethod("LINK")` and `chi.RegisterMethod("UNLINK")` in an `init()` function
## v3.2.1 (2017-08-31)
- Add new `Match(rctx *Context, method, path string) bool` method to `Routes` interface
and `Mux`. Match searches the mux's routing tree for a handler that matches the method/path
- Add new `RouteMethod` to `*Context`
- Add new `Routes` pointer to `*Context`
- Add new `middleware.GetHead` to route missing HEAD requests to GET handler
- Updated benchmarks (see README)
## v3.1.5 (2017-08-02)
- Setup golint and go vet for the project
- As per golint, we've redefined `func ServerBaseContext(h http.Handler, baseCtx context.Context) http.Handler`
to `func ServerBaseContext(baseCtx context.Context, h http.Handler) http.Handler`
## v3.1.0 (2017-07-10)
- Fix a few minor issues after v3 release
- Move `docgen` sub-pkg to https://github.com/go-chi/docgen
- Move `render` sub-pkg to https://github.com/go-chi/render
- Add new `URLFormat` handler to chi/middleware sub-pkg to make working with url mime
suffixes easier, ie. parsing `/articles/1.json` and `/articles/1.xml`. See comments in
https://github.com/go-chi/chi/blob/master/middleware/url_format.go for example usage.
## v3.0.0 (2017-06-21)
- Major update to chi library with many exciting updates, but also some *breaking changes*
- URL parameter syntax changed from `/:id` to `/{id}` for even more flexible routing, such as
`/articles/{month}-{day}-{year}-{slug}`, `/articles/{id}`, and `/articles/{id}.{ext}` on the
same router
- Support for regexp for routing patterns, in the form of `/{paramKey:regExp}` for example:
`r.Get("/articles/{name:[a-z]+}", h)` and `chi.URLParam(r, "name")`
- Add `Method` and `MethodFunc` to `chi.Router` to allow routing definitions such as
`r.Method("GET", "/", h)` which provides a cleaner interface for custom handlers like
in `_examples/custom-handler`
- Deprecating `mux#FileServer` helper function. Instead, we encourage users to create their
own using file handler with the stdlib, see `_examples/fileserver` for an example
- Add support for LINK/UNLINK http methods via `r.Method()` and `r.MethodFunc()`
- Moved the chi project to its own organization, to allow chi-related community packages to
be easily discovered and supported, at: https://github.com/go-chi
- *NOTE:* please update your import paths to `"github.com/go-chi/chi"`
- *NOTE:* chi v2 is still available at https://github.com/go-chi/chi/tree/v2
## v2.1.0 (2017-03-30)
- Minor improvements and update to the chi core library
- Introduced a brand new `chi/render` sub-package to complete the story of building
APIs to offer a pattern for managing well-defined request / response payloads. Please
check out the updated `_examples/rest` example for how it works.
- Added `MethodNotAllowed(h http.HandlerFunc)` to chi.Router interface
## v2.0.0 (2017-01-06)
- After many months of v2 being in an RC state with many companies and users running it in
production, the inclusion of some improvements to the middlewares, we are very pleased to
announce v2.0.0 of chi.
## v2.0.0-rc1 (2016-07-26)
- Huge update! chi v2 is a large refactor targetting Go 1.7+. As of Go 1.7, the popular
community `"net/context"` package has been included in the standard library as `"context"` and
utilized by `"net/http"` and `http.Request` to managing deadlines, cancelation signals and other
request-scoped values. We're very excited about the new context addition and are proud to
introduce chi v2, a minimal and powerful routing package for building large HTTP services,
with zero external dependencies. Chi focuses on idiomatic design and encourages the use of
stdlib HTTP handlers and middlwares.
- chi v2 deprecates its `chi.Handler` interface and requires `http.Handler` or `http.HandlerFunc`
- chi v2 stores URL routing parameters and patterns in the standard request context: `r.Context()`
- chi v2 lower-level routing context is accessible by `chi.RouteContext(r.Context()) *chi.Context`,
which provides direct access to URL routing parameters, the routing path and the matching
routing patterns.
- Users upgrading from chi v1 to v2, need to:
1. Update the old chi.Handler signature, `func(ctx context.Context, w http.ResponseWriter, r *http.Request)` to
the standard http.Handler: `func(w http.ResponseWriter, r *http.Request)`
2. Use `chi.URLParam(r *http.Request, paramKey string) string`
or `URLParamFromCtx(ctx context.Context, paramKey string) string` to access a url parameter value
## v1.0.0 (2016-07-01)
- Released chi v1 stable https://github.com/go-chi/chi/tree/v1.0.0 for Go 1.6 and older.
## v0.9.0 (2016-03-31)
- Reuse context objects via sync.Pool for zero-allocation routing [#33](https://github.com/go-chi/chi/pull/33)
- BREAKING NOTE: due to subtle API changes, previously `chi.URLParams(ctx)["id"]` used to access url parameters
has changed to: `chi.URLParam(ctx, "id")`
+31
View File
@@ -0,0 +1,31 @@
# Contributing
## Prerequisites
1. [Install Go][go-install].
2. Download the sources and switch the working directory:
```bash
go get -u -d github.com/go-chi/chi
cd $GOPATH/src/github.com/go-chi/chi
```
## Submitting a Pull Request
A typical workflow is:
1. [Fork the repository.][fork] [This tip maybe also helpful.][go-fork-tip]
2. [Create a topic branch.][branch]
3. Add tests for your change.
4. Run `go test`. If your tests pass, return to the step 3.
5. Implement the change and ensure the steps from the previous step pass.
6. Run `goimports -w .`, to ensure the new code conforms to Go formatting guideline.
7. [Add, commit and push your changes.][git-help]
8. [Submit a pull request.][pull-req]
[go-install]: https://golang.org/doc/install
[go-fork-tip]: http://blog.campoy.cat/2014/03/github-and-go-forking-pull-requests-and.html
[fork]: https://help.github.com/articles/fork-a-repo
[branch]: http://learn.github.com/p/branching.html
[git-help]: https://guides.github.com
[pull-req]: https://help.github.com/articles/using-pull-requests
+20
View File
@@ -0,0 +1,20 @@
Copyright (c) 2015-present Peter Kieltyka (https://github.com/pkieltyka), Google Inc.
MIT License
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+22
View File
@@ -0,0 +1,22 @@
.PHONY: all
all:
@echo "**********************************************************"
@echo "** chi build tool **"
@echo "**********************************************************"
.PHONY: test
test:
go clean -testcache && $(MAKE) test-router && $(MAKE) test-middleware
.PHONY: test-router
test-router:
go test -race -v .
.PHONY: test-middleware
test-middleware:
go test -race -v ./middleware
.PHONY: docs
docs:
npx docsify-cli serve ./docs
+500
View File
@@ -0,0 +1,500 @@
# <img alt="chi" src="https://cdn.rawgit.com/go-chi/chi/master/_examples/chi.svg" width="220" />
[![GoDoc Widget]][GoDoc] [![Travis Widget]][Travis]
`chi` is a lightweight, idiomatic and composable router for building Go HTTP services. It's
especially good at helping you write large REST API services that are kept maintainable as your
project grows and changes. `chi` is built on the new `context` package introduced in Go 1.7 to
handle signaling, cancelation and request-scoped values across a handler chain.
The focus of the project has been to seek out an elegant and comfortable design for writing
REST API servers, written during the development of the Pressly API service that powers our
public API service, which in turn powers all of our client-side applications.
The key considerations of chi's design are: project structure, maintainability, standard http
handlers (stdlib-only), developer productivity, and deconstructing a large system into many small
parts. The core router `github.com/go-chi/chi` is quite small (less than 1000 LOC), but we've also
included some useful/optional subpackages: [middleware](/middleware), [render](https://github.com/go-chi/render)
and [docgen](https://github.com/go-chi/docgen). We hope you enjoy it too!
## Install
`go get -u github.com/go-chi/chi/v5`
## Features
* **Lightweight** - cloc'd in ~1000 LOC for the chi router
* **Fast** - yes, see [benchmarks](#benchmarks)
* **100% compatible with net/http** - use any http or middleware pkg in the ecosystem that is also compatible with `net/http`
* **Designed for modular/composable APIs** - middlewares, inline middlewares, route groups and sub-router mounting
* **Context control** - built on new `context` package, providing value chaining, cancellations and timeouts
* **Robust** - in production at Pressly, Cloudflare, Heroku, 99Designs, and many others (see [discussion](https://github.com/go-chi/chi/issues/91))
* **Doc generation** - `docgen` auto-generates routing documentation from your source to JSON or Markdown
* **Go.mod support** - as of v5, go.mod support (see [CHANGELOG](https://github.com/go-chi/chi/blob/master/CHANGELOG.md))
* **No external dependencies** - plain ol' Go stdlib + net/http
## Examples
See [_examples/](https://github.com/go-chi/chi/blob/master/_examples/) for a variety of examples.
**As easy as:**
```go
package main
import (
"net/http"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
)
func main() {
r := chi.NewRouter()
r.Use(middleware.Logger)
r.Get("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("welcome"))
})
http.ListenAndServe(":3000", r)
}
```
**REST Preview:**
Here is a little preview of how routing looks like with chi. Also take a look at the generated routing docs
in JSON ([routes.json](https://github.com/go-chi/chi/blob/master/_examples/rest/routes.json)) and in
Markdown ([routes.md](https://github.com/go-chi/chi/blob/master/_examples/rest/routes.md)).
I highly recommend reading the source of the [examples](https://github.com/go-chi/chi/blob/master/_examples/) listed
above, they will show you all the features of chi and serve as a good form of documentation.
```go
import (
//...
"context"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
)
func main() {
r := chi.NewRouter()
// A good base middleware stack
r.Use(middleware.RequestID)
r.Use(middleware.RealIP)
r.Use(middleware.Logger)
r.Use(middleware.Recoverer)
// Set a timeout value on the request context (ctx), that will signal
// through ctx.Done() that the request has timed out and further
// processing should be stopped.
r.Use(middleware.Timeout(60 * time.Second))
r.Get("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("hi"))
})
// RESTy routes for "articles" resource
r.Route("/articles", func(r chi.Router) {
r.With(paginate).Get("/", listArticles) // GET /articles
r.With(paginate).Get("/{month}-{day}-{year}", listArticlesByDate) // GET /articles/01-16-2017
r.Post("/", createArticle) // POST /articles
r.Get("/search", searchArticles) // GET /articles/search
// Regexp url parameters:
r.Get("/{articleSlug:[a-z-]+}", getArticleBySlug) // GET /articles/home-is-toronto
// Subrouters:
r.Route("/{articleID}", func(r chi.Router) {
r.Use(ArticleCtx)
r.Get("/", getArticle) // GET /articles/123
r.Put("/", updateArticle) // PUT /articles/123
r.Delete("/", deleteArticle) // DELETE /articles/123
})
})
// Mount the admin sub-router
r.Mount("/admin", adminRouter())
http.ListenAndServe(":3333", r)
}
func ArticleCtx(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
articleID := chi.URLParam(r, "articleID")
article, err := dbGetArticle(articleID)
if err != nil {
http.Error(w, http.StatusText(404), 404)
return
}
ctx := context.WithValue(r.Context(), "article", article)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
func getArticle(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
article, ok := ctx.Value("article").(*Article)
if !ok {
http.Error(w, http.StatusText(422), 422)
return
}
w.Write([]byte(fmt.Sprintf("title:%s", article.Title)))
}
// A completely separate router for administrator routes
func adminRouter() http.Handler {
r := chi.NewRouter()
r.Use(AdminOnly)
r.Get("/", adminIndex)
r.Get("/accounts", adminListAccounts)
return r
}
func AdminOnly(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
perm, ok := ctx.Value("acl.permission").(YourPermissionType)
if !ok || !perm.IsAdmin() {
http.Error(w, http.StatusText(403), 403)
return
}
next.ServeHTTP(w, r)
})
}
```
## Router interface
chi's router is based on a kind of [Patricia Radix trie](https://en.wikipedia.org/wiki/Radix_tree).
The router is fully compatible with `net/http`.
Built on top of the tree is the `Router` interface:
```go
// Router consisting of the core routing methods used by chi's Mux,
// using only the standard net/http.
type Router interface {
http.Handler
Routes
// Use appends one or more middlewares onto the Router stack.
Use(middlewares ...func(http.Handler) http.Handler)
// With adds inline middlewares for an endpoint handler.
With(middlewares ...func(http.Handler) http.Handler) Router
// Group adds a new inline-Router along the current routing
// path, with a fresh middleware stack for the inline-Router.
Group(fn func(r Router)) Router
// Route mounts a sub-Router along a `pattern`` string.
Route(pattern string, fn func(r Router)) Router
// Mount attaches another http.Handler along ./pattern/*
Mount(pattern string, h http.Handler)
// Handle and HandleFunc adds routes for `pattern` that matches
// all HTTP methods.
Handle(pattern string, h http.Handler)
HandleFunc(pattern string, h http.HandlerFunc)
// Method and MethodFunc adds routes for `pattern` that matches
// the `method` HTTP method.
Method(method, pattern string, h http.Handler)
MethodFunc(method, pattern string, h http.HandlerFunc)
// HTTP-method routing along `pattern`
Connect(pattern string, h http.HandlerFunc)
Delete(pattern string, h http.HandlerFunc)
Get(pattern string, h http.HandlerFunc)
Head(pattern string, h http.HandlerFunc)
Options(pattern string, h http.HandlerFunc)
Patch(pattern string, h http.HandlerFunc)
Post(pattern string, h http.HandlerFunc)
Put(pattern string, h http.HandlerFunc)
Trace(pattern string, h http.HandlerFunc)
// NotFound defines a handler to respond whenever a route could
// not be found.
NotFound(h http.HandlerFunc)
// MethodNotAllowed defines a handler to respond whenever a method is
// not allowed.
MethodNotAllowed(h http.HandlerFunc)
}
// Routes interface adds two methods for router traversal, which is also
// used by the github.com/go-chi/docgen package to generate documentation for Routers.
type Routes interface {
// Routes returns the routing tree in an easily traversable structure.
Routes() []Route
// Middlewares returns the list of middlewares in use by the router.
Middlewares() Middlewares
// Match searches the routing tree for a handler that matches
// the method/path - similar to routing a http request, but without
// executing the handler thereafter.
Match(rctx *Context, method, path string) bool
}
```
Each routing method accepts a URL `pattern` and chain of `handlers`. The URL pattern
supports named params (ie. `/users/{userID}`) and wildcards (ie. `/admin/*`). URL parameters
can be fetched at runtime by calling `chi.URLParam(r, "userID")` for named parameters
and `chi.URLParam(r, "*")` for a wildcard parameter.
### Middleware handlers
chi's middlewares are just stdlib net/http middleware handlers. There is nothing special
about them, which means the router and all the tooling is designed to be compatible and
friendly with any middleware in the community. This offers much better extensibility and reuse
of packages and is at the heart of chi's purpose.
Here is an example of a standard net/http middleware where we assign a context key `"user"`
the value of `"123"`. This middleware sets a hypothetical user identifier on the request
context and calls the next handler in the chain.
```go
// HTTP middleware setting a value on the request context
func MyMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// create new context from `r` request context, and assign key `"user"`
// to value of `"123"`
ctx := context.WithValue(r.Context(), "user", "123")
// call the next handler in the chain, passing the response writer and
// the updated request object with the new context value.
//
// note: context.Context values are nested, so any previously set
// values will be accessible as well, and the new `"user"` key
// will be accessible from this point forward.
next.ServeHTTP(w, r.WithContext(ctx))
})
}
```
### Request handlers
chi uses standard net/http request handlers. This little snippet is an example of a http.Handler
func that reads a user identifier from the request context - hypothetically, identifying
the user sending an authenticated request, validated+set by a previous middleware handler.
```go
// HTTP handler accessing data from the request context.
func MyRequestHandler(w http.ResponseWriter, r *http.Request) {
// here we read from the request context and fetch out `"user"` key set in
// the MyMiddleware example above.
user := r.Context().Value("user").(string)
// respond to the client
w.Write([]byte(fmt.Sprintf("hi %s", user)))
}
```
### URL parameters
chi's router parses and stores URL parameters right onto the request context. Here is
an example of how to access URL params in your net/http handlers. And of course, middlewares
are able to access the same information.
```go
// HTTP handler accessing the url routing parameters.
func MyRequestHandler(w http.ResponseWriter, r *http.Request) {
// fetch the url parameter `"userID"` from the request of a matching
// routing pattern. An example routing pattern could be: /users/{userID}
userID := chi.URLParam(r, "userID")
// fetch `"key"` from the request context
ctx := r.Context()
key := ctx.Value("key").(string)
// respond to the client
w.Write([]byte(fmt.Sprintf("hi %v, %v", userID, key)))
}
```
## Middlewares
chi comes equipped with an optional `middleware` package, providing a suite of standard
`net/http` middlewares. Please note, any middleware in the ecosystem that is also compatible
with `net/http` can be used with chi's mux.
### Core middlewares
----------------------------------------------------------------------------------------------------
| chi/middleware Handler | description |
| :--------------------- | :---------------------------------------------------------------------- |
| [AllowContentEncoding] | Enforces a whitelist of request Content-Encoding headers |
| [AllowContentType] | Explicit whitelist of accepted request Content-Types |
| [BasicAuth] | Basic HTTP authentication |
| [Compress] | Gzip compression for clients that accept compressed responses |
| [ContentCharset] | Ensure charset for Content-Type request headers |
| [CleanPath] | Clean double slashes from request path |
| [GetHead] | Automatically route undefined HEAD requests to GET handlers |
| [Heartbeat] | Monitoring endpoint to check the servers pulse |
| [Logger] | Logs the start and end of each request with the elapsed processing time |
| [NoCache] | Sets response headers to prevent clients from caching |
| [Profiler] | Easily attach net/http/pprof to your routers |
| [RealIP] | Sets a http.Request's RemoteAddr to either X-Real-IP or X-Forwarded-For |
| [Recoverer] | Gracefully absorb panics and prints the stack trace |
| [RequestID] | Injects a request ID into the context of each request |
| [RedirectSlashes] | Redirect slashes on routing paths |
| [RouteHeaders] | Route handling for request headers |
| [SetHeader] | Short-hand middleware to set a response header key/value |
| [StripSlashes] | Strip slashes on routing paths |
| [Throttle] | Puts a ceiling on the number of concurrent requests |
| [Timeout] | Signals to the request context when the timeout deadline is reached |
| [URLFormat] | Parse extension from url and put it on request context |
| [WithValue] | Short-hand middleware to set a key/value on the request context |
----------------------------------------------------------------------------------------------------
[AllowContentEncoding]: https://pkg.go.dev/github.com/go-chi/chi/middleware#AllowContentEncoding
[AllowContentType]: https://pkg.go.dev/github.com/go-chi/chi/middleware#AllowContentType
[BasicAuth]: https://pkg.go.dev/github.com/go-chi/chi/middleware#BasicAuth
[Compress]: https://pkg.go.dev/github.com/go-chi/chi/middleware#Compress
[ContentCharset]: https://pkg.go.dev/github.com/go-chi/chi/middleware#ContentCharset
[CleanPath]: https://pkg.go.dev/github.com/go-chi/chi/middleware#CleanPath
[GetHead]: https://pkg.go.dev/github.com/go-chi/chi/middleware#GetHead
[GetReqID]: https://pkg.go.dev/github.com/go-chi/chi/middleware#GetReqID
[Heartbeat]: https://pkg.go.dev/github.com/go-chi/chi/middleware#Heartbeat
[Logger]: https://pkg.go.dev/github.com/go-chi/chi/middleware#Logger
[NoCache]: https://pkg.go.dev/github.com/go-chi/chi/middleware#NoCache
[Profiler]: https://pkg.go.dev/github.com/go-chi/chi/middleware#Profiler
[RealIP]: https://pkg.go.dev/github.com/go-chi/chi/middleware#RealIP
[Recoverer]: https://pkg.go.dev/github.com/go-chi/chi/middleware#Recoverer
[RedirectSlashes]: https://pkg.go.dev/github.com/go-chi/chi/middleware#RedirectSlashes
[RequestLogger]: https://pkg.go.dev/github.com/go-chi/chi/middleware#RequestLogger
[RequestID]: https://pkg.go.dev/github.com/go-chi/chi/middleware#RequestID
[RouteHeaders]: https://pkg.go.dev/github.com/go-chi/chi/middleware#RouteHeaders
[SetHeader]: https://pkg.go.dev/github.com/go-chi/chi/middleware#SetHeader
[StripSlashes]: https://pkg.go.dev/github.com/go-chi/chi/middleware#StripSlashes
[Throttle]: https://pkg.go.dev/github.com/go-chi/chi/middleware#Throttle
[ThrottleBacklog]: https://pkg.go.dev/github.com/go-chi/chi/middleware#ThrottleBacklog
[ThrottleWithOpts]: https://pkg.go.dev/github.com/go-chi/chi/middleware#ThrottleWithOpts
[Timeout]: https://pkg.go.dev/github.com/go-chi/chi/middleware#Timeout
[URLFormat]: https://pkg.go.dev/github.com/go-chi/chi/middleware#URLFormat
[WithLogEntry]: https://pkg.go.dev/github.com/go-chi/chi/middleware#WithLogEntry
[WithValue]: https://pkg.go.dev/github.com/go-chi/chi/middleware#WithValue
[Compressor]: https://pkg.go.dev/github.com/go-chi/chi/middleware#Compressor
[DefaultLogFormatter]: https://pkg.go.dev/github.com/go-chi/chi/middleware#DefaultLogFormatter
[EncoderFunc]: https://pkg.go.dev/github.com/go-chi/chi/middleware#EncoderFunc
[HeaderRoute]: https://pkg.go.dev/github.com/go-chi/chi/middleware#HeaderRoute
[HeaderRouter]: https://pkg.go.dev/github.com/go-chi/chi/middleware#HeaderRouter
[LogEntry]: https://pkg.go.dev/github.com/go-chi/chi/middleware#LogEntry
[LogFormatter]: https://pkg.go.dev/github.com/go-chi/chi/middleware#LogFormatter
[LoggerInterface]: https://pkg.go.dev/github.com/go-chi/chi/middleware#LoggerInterface
[ThrottleOpts]: https://pkg.go.dev/github.com/go-chi/chi/middleware#ThrottleOpts
[WrapResponseWriter]: https://pkg.go.dev/github.com/go-chi/chi/middleware#WrapResponseWriter
### Extra middlewares & packages
Please see https://github.com/go-chi for additional packages.
--------------------------------------------------------------------------------------------------------------------
| package | description |
|:---------------------------------------------------|:-------------------------------------------------------------
| [cors](https://github.com/go-chi/cors) | Cross-origin resource sharing (CORS) |
| [docgen](https://github.com/go-chi/docgen) | Print chi.Router routes at runtime |
| [jwtauth](https://github.com/go-chi/jwtauth) | JWT authentication |
| [hostrouter](https://github.com/go-chi/hostrouter) | Domain/host based request routing |
| [httplog](https://github.com/go-chi/httplog) | Small but powerful structured HTTP request logging |
| [httprate](https://github.com/go-chi/httprate) | HTTP request rate limiter |
| [httptracer](https://github.com/go-chi/httptracer) | HTTP request performance tracing library |
| [httpvcr](https://github.com/go-chi/httpvcr) | Write deterministic tests for external sources |
| [stampede](https://github.com/go-chi/stampede) | HTTP request coalescer |
--------------------------------------------------------------------------------------------------------------------
## context?
`context` is a tiny pkg that provides simple interface to signal context across call stacks
and goroutines. It was originally written by [Sameer Ajmani](https://github.com/Sajmani)
and is available in stdlib since go1.7.
Learn more at https://blog.golang.org/context
and..
* Docs: https://golang.org/pkg/context
* Source: https://github.com/golang/go/tree/master/src/context
## Benchmarks
The benchmark suite: https://github.com/pkieltyka/go-http-routing-benchmark
Results as of Nov 29, 2020 with Go 1.15.5 on Linux AMD 3950x
```shell
BenchmarkChi_Param 3075895 384 ns/op 400 B/op 2 allocs/op
BenchmarkChi_Param5 2116603 566 ns/op 400 B/op 2 allocs/op
BenchmarkChi_Param20 964117 1227 ns/op 400 B/op 2 allocs/op
BenchmarkChi_ParamWrite 2863413 420 ns/op 400 B/op 2 allocs/op
BenchmarkChi_GithubStatic 3045488 395 ns/op 400 B/op 2 allocs/op
BenchmarkChi_GithubParam 2204115 540 ns/op 400 B/op 2 allocs/op
BenchmarkChi_GithubAll 10000 113811 ns/op 81203 B/op 406 allocs/op
BenchmarkChi_GPlusStatic 3337485 359 ns/op 400 B/op 2 allocs/op
BenchmarkChi_GPlusParam 2825853 423 ns/op 400 B/op 2 allocs/op
BenchmarkChi_GPlus2Params 2471697 483 ns/op 400 B/op 2 allocs/op
BenchmarkChi_GPlusAll 194220 5950 ns/op 5200 B/op 26 allocs/op
BenchmarkChi_ParseStatic 3365324 356 ns/op 400 B/op 2 allocs/op
BenchmarkChi_ParseParam 2976614 404 ns/op 400 B/op 2 allocs/op
BenchmarkChi_Parse2Params 2638084 439 ns/op 400 B/op 2 allocs/op
BenchmarkChi_ParseAll 109567 11295 ns/op 10400 B/op 52 allocs/op
BenchmarkChi_StaticAll 16846 71308 ns/op 62802 B/op 314 allocs/op
```
Comparison with other routers: https://gist.github.com/pkieltyka/123032f12052520aaccab752bd3e78cc
NOTE: the allocs in the benchmark above are from the calls to http.Request's
`WithContext(context.Context)` method that clones the http.Request, sets the `Context()`
on the duplicated (alloc'd) request and returns it the new request object. This is just
how setting context on a request in Go works.
## Credits
* Carl Jackson for https://github.com/zenazn/goji
* Parts of chi's thinking comes from goji, and chi's middleware package
sources from goji.
* Armon Dadgar for https://github.com/armon/go-radix
* Contributions: [@VojtechVitek](https://github.com/VojtechVitek)
We'll be more than happy to see [your contributions](./CONTRIBUTING.md)!
## Beyond REST
chi is just a http router that lets you decompose request handling into many smaller layers.
Many companies use chi to write REST services for their public APIs. But, REST is just a convention
for managing state via HTTP, and there's a lot of other pieces required to write a complete client-server
system or network of microservices.
Looking beyond REST, I also recommend some newer works in the field:
* [webrpc](https://github.com/webrpc/webrpc) - Web-focused RPC client+server framework with code-gen
* [gRPC](https://github.com/grpc/grpc-go) - Google's RPC framework via protobufs
* [graphql](https://github.com/99designs/gqlgen) - Declarative query language
* [NATS](https://nats.io) - lightweight pub-sub
## License
Copyright (c) 2015-present [Peter Kieltyka](https://github.com/pkieltyka)
Licensed under [MIT License](./LICENSE)
[GoDoc]: https://pkg.go.dev/github.com/go-chi/chi?tab=versions
[GoDoc Widget]: https://godoc.org/github.com/go-chi/chi?status.svg
[Travis]: https://travis-ci.org/go-chi/chi
[Travis Widget]: https://travis-ci.org/go-chi/chi.svg?branch=master
+49
View File
@@ -0,0 +1,49 @@
package chi
import "net/http"
// Chain returns a Middlewares type from a slice of middleware handlers.
func Chain(middlewares ...func(http.Handler) http.Handler) Middlewares {
return Middlewares(middlewares)
}
// Handler builds and returns a http.Handler from the chain of middlewares,
// with `h http.Handler` as the final handler.
func (mws Middlewares) Handler(h http.Handler) http.Handler {
return &ChainHandler{h, chain(mws, h), mws}
}
// HandlerFunc builds and returns a http.Handler from the chain of middlewares,
// with `h http.Handler` as the final handler.
func (mws Middlewares) HandlerFunc(h http.HandlerFunc) http.Handler {
return &ChainHandler{h, chain(mws, h), mws}
}
// ChainHandler is a http.Handler with support for handler composition and
// execution.
type ChainHandler struct {
Endpoint http.Handler
chain http.Handler
Middlewares Middlewares
}
func (c *ChainHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
c.chain.ServeHTTP(w, r)
}
// chain builds a http.Handler composed of an inline middleware stack and endpoint
// handler in the order they are passed.
func chain(middlewares []func(http.Handler) http.Handler, endpoint http.Handler) http.Handler {
// Return ahead of time if there aren't any middlewares for the chain
if len(middlewares) == 0 {
return endpoint
}
// Wrap the end handler with the middleware chain
h := middlewares[len(middlewares)-1](endpoint)
for i := len(middlewares) - 2; i >= 0; i-- {
h = middlewares[i](h)
}
return h
}
+134
View File
@@ -0,0 +1,134 @@
// Package chi is a small, idiomatic and composable router for building HTTP services.
//
// chi requires Go 1.14 or newer.
//
// Example:
//
// package main
//
// import (
// "net/http"
//
// "github.com/go-chi/chi/v5"
// "github.com/go-chi/chi/v5/middleware"
// )
//
// func main() {
// r := chi.NewRouter()
// r.Use(middleware.Logger)
// r.Use(middleware.Recoverer)
//
// r.Get("/", func(w http.ResponseWriter, r *http.Request) {
// w.Write([]byte("root."))
// })
//
// http.ListenAndServe(":3333", r)
// }
//
// See github.com/go-chi/chi/_examples/ for more in-depth examples.
//
// URL patterns allow for easy matching of path components in HTTP
// requests. The matching components can then be accessed using
// chi.URLParam(). All patterns must begin with a slash.
//
// A simple named placeholder {name} matches any sequence of characters
// up to the next / or the end of the URL. Trailing slashes on paths must
// be handled explicitly.
//
// A placeholder with a name followed by a colon allows a regular
// expression match, for example {number:\\d+}. The regular expression
// syntax is Go's normal regexp RE2 syntax, except that regular expressions
// including { or } are not supported, and / will never be
// matched. An anonymous regexp pattern is allowed, using an empty string
// before the colon in the placeholder, such as {:\\d+}
//
// The special placeholder of asterisk matches the rest of the requested
// URL. Any trailing characters in the pattern are ignored. This is the only
// placeholder which will match / characters.
//
// Examples:
//
// "/user/{name}" matches "/user/jsmith" but not "/user/jsmith/info" or "/user/jsmith/"
// "/user/{name}/info" matches "/user/jsmith/info"
// "/page/*" matches "/page/intro/latest"
// "/page/{other}/index" also matches "/page/intro/latest"
// "/date/{yyyy:\\d\\d\\d\\d}/{mm:\\d\\d}/{dd:\\d\\d}" matches "/date/2017/04/01"
package chi
import "net/http"
// NewRouter returns a new Mux object that implements the Router interface.
func NewRouter() *Mux {
return NewMux()
}
// Router consisting of the core routing methods used by chi's Mux,
// using only the standard net/http.
type Router interface {
http.Handler
Routes
// Use appends one or more middlewares onto the Router stack.
Use(middlewares ...func(http.Handler) http.Handler)
// With adds inline middlewares for an endpoint handler.
With(middlewares ...func(http.Handler) http.Handler) Router
// Group adds a new inline-Router along the current routing
// path, with a fresh middleware stack for the inline-Router.
Group(fn func(r Router)) Router
// Route mounts a sub-Router along a `pattern`` string.
Route(pattern string, fn func(r Router)) Router
// Mount attaches another http.Handler along ./pattern/*
Mount(pattern string, h http.Handler)
// Handle and HandleFunc adds routes for `pattern` that matches
// all HTTP methods.
Handle(pattern string, h http.Handler)
HandleFunc(pattern string, h http.HandlerFunc)
// Method and MethodFunc adds routes for `pattern` that matches
// the `method` HTTP method.
Method(method, pattern string, h http.Handler)
MethodFunc(method, pattern string, h http.HandlerFunc)
// HTTP-method routing along `pattern`
Connect(pattern string, h http.HandlerFunc)
Delete(pattern string, h http.HandlerFunc)
Get(pattern string, h http.HandlerFunc)
Head(pattern string, h http.HandlerFunc)
Options(pattern string, h http.HandlerFunc)
Patch(pattern string, h http.HandlerFunc)
Post(pattern string, h http.HandlerFunc)
Put(pattern string, h http.HandlerFunc)
Trace(pattern string, h http.HandlerFunc)
// NotFound defines a handler to respond whenever a route could
// not be found.
NotFound(h http.HandlerFunc)
// MethodNotAllowed defines a handler to respond whenever a method is
// not allowed.
MethodNotAllowed(h http.HandlerFunc)
}
// Routes interface adds two methods for router traversal, which is also
// used by the `docgen` subpackage to generation documentation for Routers.
type Routes interface {
// Routes returns the routing tree in an easily traversable structure.
Routes() []Route
// Middlewares returns the list of middlewares in use by the router.
Middlewares() Middlewares
// Match searches the routing tree for a handler that matches
// the method/path - similar to routing a http request, but without
// executing the handler thereafter.
Match(rctx *Context, method, path string) bool
}
// Middlewares type is a slice of standard middleware handlers with methods
// to compose middleware chains and http.Handler's.
type Middlewares []func(http.Handler) http.Handler
+159
View File
@@ -0,0 +1,159 @@
package chi
import (
"context"
"net/http"
"strings"
)
// URLParam returns the url parameter from a http.Request object.
func URLParam(r *http.Request, key string) string {
if rctx := RouteContext(r.Context()); rctx != nil {
return rctx.URLParam(key)
}
return ""
}
// URLParamFromCtx returns the url parameter from a http.Request Context.
func URLParamFromCtx(ctx context.Context, key string) string {
if rctx := RouteContext(ctx); rctx != nil {
return rctx.URLParam(key)
}
return ""
}
// RouteContext returns chi's routing Context object from a
// http.Request Context.
func RouteContext(ctx context.Context) *Context {
val, _ := ctx.Value(RouteCtxKey).(*Context)
return val
}
// NewRouteContext returns a new routing Context object.
func NewRouteContext() *Context {
return &Context{}
}
var (
// RouteCtxKey is the context.Context key to store the request context.
RouteCtxKey = &contextKey{"RouteContext"}
)
// Context is the default routing context set on the root node of a
// request context to track route patterns, URL parameters and
// an optional routing path.
type Context struct {
Routes Routes
// parentCtx is the parent of this one, for using Context as a
// context.Context directly. This is an optimization that saves
// 1 allocation.
parentCtx context.Context
// Routing path/method override used during the route search.
// See Mux#routeHTTP method.
RoutePath string
RouteMethod string
// URLParams are the stack of routeParams captured during the
// routing lifecycle across a stack of sub-routers.
URLParams RouteParams
// Route parameters matched for the current sub-router. It is
// intentionally unexported so it cant be tampered.
routeParams RouteParams
// The endpoint routing pattern that matched the request URI path
// or `RoutePath` of the current sub-router. This value will update
// during the lifecycle of a request passing through a stack of
// sub-routers.
routePattern string
// Routing pattern stack throughout the lifecycle of the request,
// across all connected routers. It is a record of all matching
// patterns across a stack of sub-routers.
RoutePatterns []string
// methodNotAllowed hint
methodNotAllowed bool
}
// Reset a routing context to its initial state.
func (x *Context) Reset() {
x.Routes = nil
x.RoutePath = ""
x.RouteMethod = ""
x.RoutePatterns = x.RoutePatterns[:0]
x.URLParams.Keys = x.URLParams.Keys[:0]
x.URLParams.Values = x.URLParams.Values[:0]
x.routePattern = ""
x.routeParams.Keys = x.routeParams.Keys[:0]
x.routeParams.Values = x.routeParams.Values[:0]
x.methodNotAllowed = false
x.parentCtx = nil
}
// URLParam returns the corresponding URL parameter value from the request
// routing context.
func (x *Context) URLParam(key string) string {
for k := len(x.URLParams.Keys) - 1; k >= 0; k-- {
if x.URLParams.Keys[k] == key {
return x.URLParams.Values[k]
}
}
return ""
}
// RoutePattern builds the routing pattern string for the particular
// request, at the particular point during routing. This means, the value
// will change throughout the execution of a request in a router. That is
// why its advised to only use this value after calling the next handler.
//
// For example,
//
// func Instrument(next http.Handler) http.Handler {
// return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// next.ServeHTTP(w, r)
// routePattern := chi.RouteContext(r.Context()).RoutePattern()
// measure(w, r, routePattern)
// })
// }
func (x *Context) RoutePattern() string {
routePattern := strings.Join(x.RoutePatterns, "")
routePattern = replaceWildcards(routePattern)
routePattern = strings.TrimSuffix(routePattern, "//")
routePattern = strings.TrimSuffix(routePattern, "/")
return routePattern
}
// replaceWildcards takes a route pattern and recursively replaces all
// occurrences of "/*/" to "/".
func replaceWildcards(p string) string {
if strings.Contains(p, "/*/") {
return replaceWildcards(strings.Replace(p, "/*/", "/", -1))
}
return p
}
// RouteParams is a structure to track URL routing parameters efficiently.
type RouteParams struct {
Keys, Values []string
}
// Add will append a URL parameter to the end of the route param
func (s *RouteParams) Add(key, value string) {
s.Keys = append(s.Keys, key)
s.Values = append(s.Values, value)
}
// contextKey is a value for use with context.WithValue. It's used as
// a pointer so it fits in an interface{} without allocation. This technique
// for defining context keys was copied from Go 1.7's new use of context in net/http.
type contextKey struct {
name string
}
func (k *contextKey) String() string {
return "chi context value " + k.name
}
+487
View File
@@ -0,0 +1,487 @@
package chi
import (
"context"
"fmt"
"net/http"
"strings"
"sync"
)
var _ Router = &Mux{}
// Mux is a simple HTTP route multiplexer that parses a request path,
// records any URL params, and executes an end handler. It implements
// the http.Handler interface and is friendly with the standard library.
//
// Mux is designed to be fast, minimal and offer a powerful API for building
// modular and composable HTTP services with a large set of handlers. It's
// particularly useful for writing large REST API services that break a handler
// into many smaller parts composed of middlewares and end handlers.
type Mux struct {
// The computed mux handler made of the chained middleware stack and
// the tree router
handler http.Handler
// The radix trie router
tree *node
// Custom method not allowed handler
methodNotAllowedHandler http.HandlerFunc
// A reference to the parent mux used by subrouters when mounting
// to a parent mux
parent *Mux
// Routing context pool
pool *sync.Pool
// Custom route not found handler
notFoundHandler http.HandlerFunc
// The middleware stack
middlewares []func(http.Handler) http.Handler
// Controls the behaviour of middleware chain generation when a mux
// is registered as an inline group inside another mux.
inline bool
}
// NewMux returns a newly initialized Mux object that implements the Router
// interface.
func NewMux() *Mux {
mux := &Mux{tree: &node{}, pool: &sync.Pool{}}
mux.pool.New = func() interface{} {
return NewRouteContext()
}
return mux
}
// ServeHTTP is the single method of the http.Handler interface that makes
// Mux interoperable with the standard library. It uses a sync.Pool to get and
// reuse routing contexts for each request.
func (mx *Mux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Ensure the mux has some routes defined on the mux
if mx.handler == nil {
mx.NotFoundHandler().ServeHTTP(w, r)
return
}
// Check if a routing context already exists from a parent router.
rctx, _ := r.Context().Value(RouteCtxKey).(*Context)
if rctx != nil {
mx.handler.ServeHTTP(w, r)
return
}
// Fetch a RouteContext object from the sync pool, and call the computed
// mx.handler that is comprised of mx.middlewares + mx.routeHTTP.
// Once the request is finished, reset the routing context and put it back
// into the pool for reuse from another request.
rctx = mx.pool.Get().(*Context)
rctx.Reset()
rctx.Routes = mx
rctx.parentCtx = r.Context()
// NOTE: r.WithContext() causes 2 allocations and context.WithValue() causes 1 allocation
r = r.WithContext(context.WithValue(r.Context(), RouteCtxKey, rctx))
// Serve the request and once its done, put the request context back in the sync pool
mx.handler.ServeHTTP(w, r)
mx.pool.Put(rctx)
}
// Use appends a middleware handler to the Mux middleware stack.
//
// The middleware stack for any Mux will execute before searching for a matching
// route to a specific handler, which provides opportunity to respond early,
// change the course of the request execution, or set request-scoped values for
// the next http.Handler.
func (mx *Mux) Use(middlewares ...func(http.Handler) http.Handler) {
if mx.handler != nil {
panic("chi: all middlewares must be defined before routes on a mux")
}
mx.middlewares = append(mx.middlewares, middlewares...)
}
// Handle adds the route `pattern` that matches any http method to
// execute the `handler` http.Handler.
func (mx *Mux) Handle(pattern string, handler http.Handler) {
mx.handle(mALL, pattern, handler)
}
// HandleFunc adds the route `pattern` that matches any http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) HandleFunc(pattern string, handlerFn http.HandlerFunc) {
mx.handle(mALL, pattern, handlerFn)
}
// Method adds the route `pattern` that matches `method` http method to
// execute the `handler` http.Handler.
func (mx *Mux) Method(method, pattern string, handler http.Handler) {
m, ok := methodMap[strings.ToUpper(method)]
if !ok {
panic(fmt.Sprintf("chi: '%s' http method is not supported.", method))
}
mx.handle(m, pattern, handler)
}
// MethodFunc adds the route `pattern` that matches `method` http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) MethodFunc(method, pattern string, handlerFn http.HandlerFunc) {
mx.Method(method, pattern, handlerFn)
}
// Connect adds the route `pattern` that matches a CONNECT http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) Connect(pattern string, handlerFn http.HandlerFunc) {
mx.handle(mCONNECT, pattern, handlerFn)
}
// Delete adds the route `pattern` that matches a DELETE http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) Delete(pattern string, handlerFn http.HandlerFunc) {
mx.handle(mDELETE, pattern, handlerFn)
}
// Get adds the route `pattern` that matches a GET http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) Get(pattern string, handlerFn http.HandlerFunc) {
mx.handle(mGET, pattern, handlerFn)
}
// Head adds the route `pattern` that matches a HEAD http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) Head(pattern string, handlerFn http.HandlerFunc) {
mx.handle(mHEAD, pattern, handlerFn)
}
// Options adds the route `pattern` that matches a OPTIONS http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) Options(pattern string, handlerFn http.HandlerFunc) {
mx.handle(mOPTIONS, pattern, handlerFn)
}
// Patch adds the route `pattern` that matches a PATCH http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) Patch(pattern string, handlerFn http.HandlerFunc) {
mx.handle(mPATCH, pattern, handlerFn)
}
// Post adds the route `pattern` that matches a POST http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) Post(pattern string, handlerFn http.HandlerFunc) {
mx.handle(mPOST, pattern, handlerFn)
}
// Put adds the route `pattern` that matches a PUT http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) Put(pattern string, handlerFn http.HandlerFunc) {
mx.handle(mPUT, pattern, handlerFn)
}
// Trace adds the route `pattern` that matches a TRACE http method to
// execute the `handlerFn` http.HandlerFunc.
func (mx *Mux) Trace(pattern string, handlerFn http.HandlerFunc) {
mx.handle(mTRACE, pattern, handlerFn)
}
// NotFound sets a custom http.HandlerFunc for routing paths that could
// not be found. The default 404 handler is `http.NotFound`.
func (mx *Mux) NotFound(handlerFn http.HandlerFunc) {
// Build NotFound handler chain
m := mx
hFn := handlerFn
if mx.inline && mx.parent != nil {
m = mx.parent
hFn = Chain(mx.middlewares...).HandlerFunc(hFn).ServeHTTP
}
// Update the notFoundHandler from this point forward
m.notFoundHandler = hFn
m.updateSubRoutes(func(subMux *Mux) {
if subMux.notFoundHandler == nil {
subMux.NotFound(hFn)
}
})
}
// MethodNotAllowed sets a custom http.HandlerFunc for routing paths where the
// method is unresolved. The default handler returns a 405 with an empty body.
func (mx *Mux) MethodNotAllowed(handlerFn http.HandlerFunc) {
// Build MethodNotAllowed handler chain
m := mx
hFn := handlerFn
if mx.inline && mx.parent != nil {
m = mx.parent
hFn = Chain(mx.middlewares...).HandlerFunc(hFn).ServeHTTP
}
// Update the methodNotAllowedHandler from this point forward
m.methodNotAllowedHandler = hFn
m.updateSubRoutes(func(subMux *Mux) {
if subMux.methodNotAllowedHandler == nil {
subMux.MethodNotAllowed(hFn)
}
})
}
// With adds inline middlewares for an endpoint handler.
func (mx *Mux) With(middlewares ...func(http.Handler) http.Handler) Router {
// Similarly as in handle(), we must build the mux handler once additional
// middleware registration isn't allowed for this stack, like now.
if !mx.inline && mx.handler == nil {
mx.updateRouteHandler()
}
// Copy middlewares from parent inline muxs
var mws Middlewares
if mx.inline {
mws = make(Middlewares, len(mx.middlewares))
copy(mws, mx.middlewares)
}
mws = append(mws, middlewares...)
im := &Mux{
pool: mx.pool, inline: true, parent: mx, tree: mx.tree, middlewares: mws,
notFoundHandler: mx.notFoundHandler, methodNotAllowedHandler: mx.methodNotAllowedHandler,
}
return im
}
// Group creates a new inline-Mux with a fresh middleware stack. It's useful
// for a group of handlers along the same routing path that use an additional
// set of middlewares. See _examples/.
func (mx *Mux) Group(fn func(r Router)) Router {
im := mx.With().(*Mux)
if fn != nil {
fn(im)
}
return im
}
// Route creates a new Mux with a fresh middleware stack and mounts it
// along the `pattern` as a subrouter. Effectively, this is a short-hand
// call to Mount. See _examples/.
func (mx *Mux) Route(pattern string, fn func(r Router)) Router {
if fn == nil {
panic(fmt.Sprintf("chi: attempting to Route() a nil subrouter on '%s'", pattern))
}
subRouter := NewRouter()
fn(subRouter)
mx.Mount(pattern, subRouter)
return subRouter
}
// Mount attaches another http.Handler or chi Router as a subrouter along a routing
// path. It's very useful to split up a large API as many independent routers and
// compose them as a single service using Mount. See _examples/.
//
// Note that Mount() simply sets a wildcard along the `pattern` that will continue
// routing at the `handler`, which in most cases is another chi.Router. As a result,
// if you define two Mount() routes on the exact same pattern the mount will panic.
func (mx *Mux) Mount(pattern string, handler http.Handler) {
if handler == nil {
panic(fmt.Sprintf("chi: attempting to Mount() a nil handler on '%s'", pattern))
}
// Provide runtime safety for ensuring a pattern isn't mounted on an existing
// routing pattern.
if mx.tree.findPattern(pattern+"*") || mx.tree.findPattern(pattern+"/*") {
panic(fmt.Sprintf("chi: attempting to Mount() a handler on an existing path, '%s'", pattern))
}
// Assign sub-Router's with the parent not found & method not allowed handler if not specified.
subr, ok := handler.(*Mux)
if ok && subr.notFoundHandler == nil && mx.notFoundHandler != nil {
subr.NotFound(mx.notFoundHandler)
}
if ok && subr.methodNotAllowedHandler == nil && mx.methodNotAllowedHandler != nil {
subr.MethodNotAllowed(mx.methodNotAllowedHandler)
}
mountHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
rctx := RouteContext(r.Context())
// shift the url path past the previous subrouter
rctx.RoutePath = mx.nextRoutePath(rctx)
// reset the wildcard URLParam which connects the subrouter
n := len(rctx.URLParams.Keys) - 1
if n >= 0 && rctx.URLParams.Keys[n] == "*" && len(rctx.URLParams.Values) > n {
rctx.URLParams.Values[n] = ""
}
handler.ServeHTTP(w, r)
})
if pattern == "" || pattern[len(pattern)-1] != '/' {
mx.handle(mALL|mSTUB, pattern, mountHandler)
mx.handle(mALL|mSTUB, pattern+"/", mountHandler)
pattern += "/"
}
method := mALL
subroutes, _ := handler.(Routes)
if subroutes != nil {
method |= mSTUB
}
n := mx.handle(method, pattern+"*", mountHandler)
if subroutes != nil {
n.subroutes = subroutes
}
}
// Routes returns a slice of routing information from the tree,
// useful for traversing available routes of a router.
func (mx *Mux) Routes() []Route {
return mx.tree.routes()
}
// Middlewares returns a slice of middleware handler functions.
func (mx *Mux) Middlewares() Middlewares {
return mx.middlewares
}
// Match searches the routing tree for a handler that matches the method/path.
// It's similar to routing a http request, but without executing the handler
// thereafter.
//
// Note: the *Context state is updated during execution, so manage
// the state carefully or make a NewRouteContext().
func (mx *Mux) Match(rctx *Context, method, path string) bool {
m, ok := methodMap[method]
if !ok {
return false
}
node, _, h := mx.tree.FindRoute(rctx, m, path)
if node != nil && node.subroutes != nil {
rctx.RoutePath = mx.nextRoutePath(rctx)
return node.subroutes.Match(rctx, method, rctx.RoutePath)
}
return h != nil
}
// NotFoundHandler returns the default Mux 404 responder whenever a route
// cannot be found.
func (mx *Mux) NotFoundHandler() http.HandlerFunc {
if mx.notFoundHandler != nil {
return mx.notFoundHandler
}
return http.NotFound
}
// MethodNotAllowedHandler returns the default Mux 405 responder whenever
// a method cannot be resolved for a route.
func (mx *Mux) MethodNotAllowedHandler() http.HandlerFunc {
if mx.methodNotAllowedHandler != nil {
return mx.methodNotAllowedHandler
}
return methodNotAllowedHandler
}
// handle registers a http.Handler in the routing tree for a particular http method
// and routing pattern.
func (mx *Mux) handle(method methodTyp, pattern string, handler http.Handler) *node {
if len(pattern) == 0 || pattern[0] != '/' {
panic(fmt.Sprintf("chi: routing pattern must begin with '/' in '%s'", pattern))
}
// Build the computed routing handler for this routing pattern.
if !mx.inline && mx.handler == nil {
mx.updateRouteHandler()
}
// Build endpoint handler with inline middlewares for the route
var h http.Handler
if mx.inline {
mx.handler = http.HandlerFunc(mx.routeHTTP)
h = Chain(mx.middlewares...).Handler(handler)
} else {
h = handler
}
// Add the endpoint to the tree and return the node
return mx.tree.InsertRoute(method, pattern, h)
}
// routeHTTP routes a http.Request through the Mux routing tree to serve
// the matching handler for a particular http method.
func (mx *Mux) routeHTTP(w http.ResponseWriter, r *http.Request) {
// Grab the route context object
rctx := r.Context().Value(RouteCtxKey).(*Context)
// The request routing path
routePath := rctx.RoutePath
if routePath == "" {
if r.URL.RawPath != "" {
routePath = r.URL.RawPath
} else {
routePath = r.URL.Path
}
if routePath == "" {
routePath = "/"
}
}
// Check if method is supported by chi
if rctx.RouteMethod == "" {
rctx.RouteMethod = r.Method
}
method, ok := methodMap[rctx.RouteMethod]
if !ok {
mx.MethodNotAllowedHandler().ServeHTTP(w, r)
return
}
// Find the route
if _, _, h := mx.tree.FindRoute(rctx, method, routePath); h != nil {
h.ServeHTTP(w, r)
return
}
if rctx.methodNotAllowed {
mx.MethodNotAllowedHandler().ServeHTTP(w, r)
} else {
mx.NotFoundHandler().ServeHTTP(w, r)
}
}
func (mx *Mux) nextRoutePath(rctx *Context) string {
routePath := "/"
nx := len(rctx.routeParams.Keys) - 1 // index of last param in list
if nx >= 0 && rctx.routeParams.Keys[nx] == "*" && len(rctx.routeParams.Values) > nx {
routePath = "/" + rctx.routeParams.Values[nx]
}
return routePath
}
// Recursively update data on child routers.
func (mx *Mux) updateSubRoutes(fn func(subMux *Mux)) {
for _, r := range mx.tree.routes() {
subMux, ok := r.SubRoutes.(*Mux)
if !ok {
continue
}
fn(subMux)
}
}
// updateRouteHandler builds the single mux handler that is a chain of the middleware
// stack, as defined by calls to Use(), and the tree router (Mux) itself. After this
// point, no other middlewares can be registered on this Mux's stack. But you can still
// compose additional middlewares via Group()'s or using a chained middleware handler.
func (mx *Mux) updateRouteHandler() {
mx.handler = chain(mx.middlewares, http.HandlerFunc(mx.routeHTTP))
}
// methodNotAllowedHandler is a helper function to respond with a 405,
// method not allowed.
func methodNotAllowedHandler(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(405)
w.Write(nil)
}
+866
View File
@@ -0,0 +1,866 @@
package chi
// Radix tree implementation below is a based on the original work by
// Armon Dadgar in https://github.com/armon/go-radix/blob/master/radix.go
// (MIT licensed). It's been heavily modified for use as a HTTP routing tree.
import (
"fmt"
"net/http"
"regexp"
"sort"
"strconv"
"strings"
)
type methodTyp uint
const (
mSTUB methodTyp = 1 << iota
mCONNECT
mDELETE
mGET
mHEAD
mOPTIONS
mPATCH
mPOST
mPUT
mTRACE
)
var mALL = mCONNECT | mDELETE | mGET | mHEAD |
mOPTIONS | mPATCH | mPOST | mPUT | mTRACE
var methodMap = map[string]methodTyp{
http.MethodConnect: mCONNECT,
http.MethodDelete: mDELETE,
http.MethodGet: mGET,
http.MethodHead: mHEAD,
http.MethodOptions: mOPTIONS,
http.MethodPatch: mPATCH,
http.MethodPost: mPOST,
http.MethodPut: mPUT,
http.MethodTrace: mTRACE,
}
// RegisterMethod adds support for custom HTTP method handlers, available
// via Router#Method and Router#MethodFunc
func RegisterMethod(method string) {
if method == "" {
return
}
method = strings.ToUpper(method)
if _, ok := methodMap[method]; ok {
return
}
n := len(methodMap)
if n > strconv.IntSize-2 {
panic(fmt.Sprintf("chi: max number of methods reached (%d)", strconv.IntSize))
}
mt := methodTyp(2 << n)
methodMap[method] = mt
mALL |= mt
}
type nodeTyp uint8
const (
ntStatic nodeTyp = iota // /home
ntRegexp // /{id:[0-9]+}
ntParam // /{user}
ntCatchAll // /api/v1/*
)
type node struct {
// subroutes on the leaf node
subroutes Routes
// regexp matcher for regexp nodes
rex *regexp.Regexp
// HTTP handler endpoints on the leaf node
endpoints endpoints
// prefix is the common prefix we ignore
prefix string
// child nodes should be stored in-order for iteration,
// in groups of the node type.
children [ntCatchAll + 1]nodes
// first byte of the child prefix
tail byte
// node type: static, regexp, param, catchAll
typ nodeTyp
// first byte of the prefix
label byte
}
// endpoints is a mapping of http method constants to handlers
// for a given route.
type endpoints map[methodTyp]*endpoint
type endpoint struct {
// endpoint handler
handler http.Handler
// pattern is the routing pattern for handler nodes
pattern string
// parameter keys recorded on handler nodes
paramKeys []string
}
func (s endpoints) Value(method methodTyp) *endpoint {
mh, ok := s[method]
if !ok {
mh = &endpoint{}
s[method] = mh
}
return mh
}
func (n *node) InsertRoute(method methodTyp, pattern string, handler http.Handler) *node {
var parent *node
search := pattern
for {
// Handle key exhaustion
if len(search) == 0 {
// Insert or update the node's leaf handler
n.setEndpoint(method, handler, pattern)
return n
}
// We're going to be searching for a wild node next,
// in this case, we need to get the tail
var label = search[0]
var segTail byte
var segEndIdx int
var segTyp nodeTyp
var segRexpat string
if label == '{' || label == '*' {
segTyp, _, segRexpat, segTail, _, segEndIdx = patNextSegment(search)
}
var prefix string
if segTyp == ntRegexp {
prefix = segRexpat
}
// Look for the edge to attach to
parent = n
n = n.getEdge(segTyp, label, segTail, prefix)
// No edge, create one
if n == nil {
child := &node{label: label, tail: segTail, prefix: search}
hn := parent.addChild(child, search)
hn.setEndpoint(method, handler, pattern)
return hn
}
// Found an edge to match the pattern
if n.typ > ntStatic {
// We found a param node, trim the param from the search path and continue.
// This param/wild pattern segment would already be on the tree from a previous
// call to addChild when creating a new node.
search = search[segEndIdx:]
continue
}
// Static nodes fall below here.
// Determine longest prefix of the search key on match.
commonPrefix := longestPrefix(search, n.prefix)
if commonPrefix == len(n.prefix) {
// the common prefix is as long as the current node's prefix we're attempting to insert.
// keep the search going.
search = search[commonPrefix:]
continue
}
// Split the node
child := &node{
typ: ntStatic,
prefix: search[:commonPrefix],
}
parent.replaceChild(search[0], segTail, child)
// Restore the existing node
n.label = n.prefix[commonPrefix]
n.prefix = n.prefix[commonPrefix:]
child.addChild(n, n.prefix)
// If the new key is a subset, set the method/handler on this node and finish.
search = search[commonPrefix:]
if len(search) == 0 {
child.setEndpoint(method, handler, pattern)
return child
}
// Create a new edge for the node
subchild := &node{
typ: ntStatic,
label: search[0],
prefix: search,
}
hn := child.addChild(subchild, search)
hn.setEndpoint(method, handler, pattern)
return hn
}
}
// addChild appends the new `child` node to the tree using the `pattern` as the trie key.
// For a URL router like chi's, we split the static, param, regexp and wildcard segments
// into different nodes. In addition, addChild will recursively call itself until every
// pattern segment is added to the url pattern tree as individual nodes, depending on type.
func (n *node) addChild(child *node, prefix string) *node {
search := prefix
// handler leaf node added to the tree is the child.
// this may be overridden later down the flow
hn := child
// Parse next segment
segTyp, _, segRexpat, segTail, segStartIdx, segEndIdx := patNextSegment(search)
// Add child depending on next up segment
switch segTyp {
case ntStatic:
// Search prefix is all static (that is, has no params in path)
// noop
default:
// Search prefix contains a param, regexp or wildcard
if segTyp == ntRegexp {
rex, err := regexp.Compile(segRexpat)
if err != nil {
panic(fmt.Sprintf("chi: invalid regexp pattern '%s' in route param", segRexpat))
}
child.prefix = segRexpat
child.rex = rex
}
if segStartIdx == 0 {
// Route starts with a param
child.typ = segTyp
if segTyp == ntCatchAll {
segStartIdx = -1
} else {
segStartIdx = segEndIdx
}
if segStartIdx < 0 {
segStartIdx = len(search)
}
child.tail = segTail // for params, we set the tail
if segStartIdx != len(search) {
// add static edge for the remaining part, split the end.
// its not possible to have adjacent param nodes, so its certainly
// going to be a static node next.
search = search[segStartIdx:] // advance search position
nn := &node{
typ: ntStatic,
label: search[0],
prefix: search,
}
hn = child.addChild(nn, search)
}
} else if segStartIdx > 0 {
// Route has some param
// starts with a static segment
child.typ = ntStatic
child.prefix = search[:segStartIdx]
child.rex = nil
// add the param edge node
search = search[segStartIdx:]
nn := &node{
typ: segTyp,
label: search[0],
tail: segTail,
}
hn = child.addChild(nn, search)
}
}
n.children[child.typ] = append(n.children[child.typ], child)
n.children[child.typ].Sort()
return hn
}
func (n *node) replaceChild(label, tail byte, child *node) {
for i := 0; i < len(n.children[child.typ]); i++ {
if n.children[child.typ][i].label == label && n.children[child.typ][i].tail == tail {
n.children[child.typ][i] = child
n.children[child.typ][i].label = label
n.children[child.typ][i].tail = tail
return
}
}
panic("chi: replacing missing child")
}
func (n *node) getEdge(ntyp nodeTyp, label, tail byte, prefix string) *node {
nds := n.children[ntyp]
for i := 0; i < len(nds); i++ {
if nds[i].label == label && nds[i].tail == tail {
if ntyp == ntRegexp && nds[i].prefix != prefix {
continue
}
return nds[i]
}
}
return nil
}
func (n *node) setEndpoint(method methodTyp, handler http.Handler, pattern string) {
// Set the handler for the method type on the node
if n.endpoints == nil {
n.endpoints = make(endpoints)
}
paramKeys := patParamKeys(pattern)
if method&mSTUB == mSTUB {
n.endpoints.Value(mSTUB).handler = handler
}
if method&mALL == mALL {
h := n.endpoints.Value(mALL)
h.handler = handler
h.pattern = pattern
h.paramKeys = paramKeys
for _, m := range methodMap {
h := n.endpoints.Value(m)
h.handler = handler
h.pattern = pattern
h.paramKeys = paramKeys
}
} else {
h := n.endpoints.Value(method)
h.handler = handler
h.pattern = pattern
h.paramKeys = paramKeys
}
}
func (n *node) FindRoute(rctx *Context, method methodTyp, path string) (*node, endpoints, http.Handler) {
// Reset the context routing pattern and params
rctx.routePattern = ""
rctx.routeParams.Keys = rctx.routeParams.Keys[:0]
rctx.routeParams.Values = rctx.routeParams.Values[:0]
// Find the routing handlers for the path
rn := n.findRoute(rctx, method, path)
if rn == nil {
return nil, nil, nil
}
// Record the routing params in the request lifecycle
rctx.URLParams.Keys = append(rctx.URLParams.Keys, rctx.routeParams.Keys...)
rctx.URLParams.Values = append(rctx.URLParams.Values, rctx.routeParams.Values...)
// Record the routing pattern in the request lifecycle
if rn.endpoints[method].pattern != "" {
rctx.routePattern = rn.endpoints[method].pattern
rctx.RoutePatterns = append(rctx.RoutePatterns, rctx.routePattern)
}
return rn, rn.endpoints, rn.endpoints[method].handler
}
// Recursive edge traversal by checking all nodeTyp groups along the way.
// It's like searching through a multi-dimensional radix trie.
func (n *node) findRoute(rctx *Context, method methodTyp, path string) *node {
nn := n
search := path
for t, nds := range nn.children {
ntyp := nodeTyp(t)
if len(nds) == 0 {
continue
}
var xn *node
xsearch := search
var label byte
if search != "" {
label = search[0]
}
switch ntyp {
case ntStatic:
xn = nds.findEdge(label)
if xn == nil || !strings.HasPrefix(xsearch, xn.prefix) {
continue
}
xsearch = xsearch[len(xn.prefix):]
case ntParam, ntRegexp:
// short-circuit and return no matching route for empty param values
if xsearch == "" {
continue
}
// serially loop through each node grouped by the tail delimiter
for idx := 0; idx < len(nds); idx++ {
xn = nds[idx]
// label for param nodes is the delimiter byte
p := strings.IndexByte(xsearch, xn.tail)
if p < 0 {
if xn.tail == '/' {
p = len(xsearch)
} else {
continue
}
} else if ntyp == ntRegexp && p == 0 {
continue
}
if ntyp == ntRegexp && xn.rex != nil {
if !xn.rex.MatchString(xsearch[:p]) {
continue
}
} else if strings.IndexByte(xsearch[:p], '/') != -1 {
// avoid a match across path segments
continue
}
prevlen := len(rctx.routeParams.Values)
rctx.routeParams.Values = append(rctx.routeParams.Values, xsearch[:p])
xsearch = xsearch[p:]
if len(xsearch) == 0 {
if xn.isLeaf() {
h := xn.endpoints[method]
if h != nil && h.handler != nil {
rctx.routeParams.Keys = append(rctx.routeParams.Keys, h.paramKeys...)
return xn
}
// flag that the routing context found a route, but not a corresponding
// supported method
rctx.methodNotAllowed = true
}
}
// recursively find the next node on this branch
fin := xn.findRoute(rctx, method, xsearch)
if fin != nil {
return fin
}
// not found on this branch, reset vars
rctx.routeParams.Values = rctx.routeParams.Values[:prevlen]
xsearch = search
}
rctx.routeParams.Values = append(rctx.routeParams.Values, "")
default:
// catch-all nodes
rctx.routeParams.Values = append(rctx.routeParams.Values, search)
xn = nds[0]
xsearch = ""
}
if xn == nil {
continue
}
// did we find it yet?
if len(xsearch) == 0 {
if xn.isLeaf() {
h := xn.endpoints[method]
if h != nil && h.handler != nil {
rctx.routeParams.Keys = append(rctx.routeParams.Keys, h.paramKeys...)
return xn
}
// flag that the routing context found a route, but not a corresponding
// supported method
rctx.methodNotAllowed = true
}
}
// recursively find the next node..
fin := xn.findRoute(rctx, method, xsearch)
if fin != nil {
return fin
}
// Did not find final handler, let's remove the param here if it was set
if xn.typ > ntStatic {
if len(rctx.routeParams.Values) > 0 {
rctx.routeParams.Values = rctx.routeParams.Values[:len(rctx.routeParams.Values)-1]
}
}
}
return nil
}
func (n *node) findEdge(ntyp nodeTyp, label byte) *node {
nds := n.children[ntyp]
num := len(nds)
idx := 0
switch ntyp {
case ntStatic, ntParam, ntRegexp:
i, j := 0, num-1
for i <= j {
idx = i + (j-i)/2
if label > nds[idx].label {
i = idx + 1
} else if label < nds[idx].label {
j = idx - 1
} else {
i = num // breaks cond
}
}
if nds[idx].label != label {
return nil
}
return nds[idx]
default: // catch all
return nds[idx]
}
}
func (n *node) isLeaf() bool {
return n.endpoints != nil
}
func (n *node) findPattern(pattern string) bool {
nn := n
for _, nds := range nn.children {
if len(nds) == 0 {
continue
}
n = nn.findEdge(nds[0].typ, pattern[0])
if n == nil {
continue
}
var idx int
var xpattern string
switch n.typ {
case ntStatic:
idx = longestPrefix(pattern, n.prefix)
if idx < len(n.prefix) {
continue
}
case ntParam, ntRegexp:
idx = strings.IndexByte(pattern, '}') + 1
case ntCatchAll:
idx = longestPrefix(pattern, "*")
default:
panic("chi: unknown node type")
}
xpattern = pattern[idx:]
if len(xpattern) == 0 {
return true
}
return n.findPattern(xpattern)
}
return false
}
func (n *node) routes() []Route {
rts := []Route{}
n.walk(func(eps endpoints, subroutes Routes) bool {
if eps[mSTUB] != nil && eps[mSTUB].handler != nil && subroutes == nil {
return false
}
// Group methodHandlers by unique patterns
pats := make(map[string]endpoints)
for mt, h := range eps {
if h.pattern == "" {
continue
}
p, ok := pats[h.pattern]
if !ok {
p = endpoints{}
pats[h.pattern] = p
}
p[mt] = h
}
for p, mh := range pats {
hs := make(map[string]http.Handler)
if mh[mALL] != nil && mh[mALL].handler != nil {
hs["*"] = mh[mALL].handler
}
for mt, h := range mh {
if h.handler == nil {
continue
}
m := methodTypString(mt)
if m == "" {
continue
}
hs[m] = h.handler
}
rt := Route{subroutes, hs, p}
rts = append(rts, rt)
}
return false
})
return rts
}
func (n *node) walk(fn func(eps endpoints, subroutes Routes) bool) bool {
// Visit the leaf values if any
if (n.endpoints != nil || n.subroutes != nil) && fn(n.endpoints, n.subroutes) {
return true
}
// Recurse on the children
for _, ns := range n.children {
for _, cn := range ns {
if cn.walk(fn) {
return true
}
}
}
return false
}
// patNextSegment returns the next segment details from a pattern:
// node type, param key, regexp string, param tail byte, param starting index, param ending index
func patNextSegment(pattern string) (nodeTyp, string, string, byte, int, int) {
ps := strings.Index(pattern, "{")
ws := strings.Index(pattern, "*")
if ps < 0 && ws < 0 {
return ntStatic, "", "", 0, 0, len(pattern) // we return the entire thing
}
// Sanity check
if ps >= 0 && ws >= 0 && ws < ps {
panic("chi: wildcard '*' must be the last pattern in a route, otherwise use a '{param}'")
}
var tail byte = '/' // Default endpoint tail to / byte
if ps >= 0 {
// Param/Regexp pattern is next
nt := ntParam
// Read to closing } taking into account opens and closes in curl count (cc)
cc := 0
pe := ps
for i, c := range pattern[ps:] {
if c == '{' {
cc++
} else if c == '}' {
cc--
if cc == 0 {
pe = ps + i
break
}
}
}
if pe == ps {
panic("chi: route param closing delimiter '}' is missing")
}
key := pattern[ps+1 : pe]
pe++ // set end to next position
if pe < len(pattern) {
tail = pattern[pe]
}
var rexpat string
if idx := strings.Index(key, ":"); idx >= 0 {
nt = ntRegexp
rexpat = key[idx+1:]
key = key[:idx]
}
if len(rexpat) > 0 {
if rexpat[0] != '^' {
rexpat = "^" + rexpat
}
if rexpat[len(rexpat)-1] != '$' {
rexpat += "$"
}
}
return nt, key, rexpat, tail, ps, pe
}
// Wildcard pattern as finale
if ws < len(pattern)-1 {
panic("chi: wildcard '*' must be the last value in a route. trim trailing text or use a '{param}' instead")
}
return ntCatchAll, "*", "", 0, ws, len(pattern)
}
func patParamKeys(pattern string) []string {
pat := pattern
paramKeys := []string{}
for {
ptyp, paramKey, _, _, _, e := patNextSegment(pat)
if ptyp == ntStatic {
return paramKeys
}
for i := 0; i < len(paramKeys); i++ {
if paramKeys[i] == paramKey {
panic(fmt.Sprintf("chi: routing pattern '%s' contains duplicate param key, '%s'", pattern, paramKey))
}
}
paramKeys = append(paramKeys, paramKey)
pat = pat[e:]
}
}
// longestPrefix finds the length of the shared prefix
// of two strings
func longestPrefix(k1, k2 string) int {
max := len(k1)
if l := len(k2); l < max {
max = l
}
var i int
for i = 0; i < max; i++ {
if k1[i] != k2[i] {
break
}
}
return i
}
func methodTypString(method methodTyp) string {
for s, t := range methodMap {
if method == t {
return s
}
}
return ""
}
type nodes []*node
// Sort the list of nodes by label
func (ns nodes) Sort() { sort.Sort(ns); ns.tailSort() }
func (ns nodes) Len() int { return len(ns) }
func (ns nodes) Swap(i, j int) { ns[i], ns[j] = ns[j], ns[i] }
func (ns nodes) Less(i, j int) bool { return ns[i].label < ns[j].label }
// tailSort pushes nodes with '/' as the tail to the end of the list for param nodes.
// The list order determines the traversal order.
func (ns nodes) tailSort() {
for i := len(ns) - 1; i >= 0; i-- {
if ns[i].typ > ntStatic && ns[i].tail == '/' {
ns.Swap(i, len(ns)-1)
return
}
}
}
func (ns nodes) findEdge(label byte) *node {
num := len(ns)
idx := 0
i, j := 0, num-1
for i <= j {
idx = i + (j-i)/2
if label > ns[idx].label {
i = idx + 1
} else if label < ns[idx].label {
j = idx - 1
} else {
i = num // breaks cond
}
}
if ns[idx].label != label {
return nil
}
return ns[idx]
}
// Route describes the details of a routing handler.
// Handlers map key is an HTTP method
type Route struct {
SubRoutes Routes
Handlers map[string]http.Handler
Pattern string
}
// WalkFunc is the type of the function called for each method and route visited by Walk.
type WalkFunc func(method string, route string, handler http.Handler, middlewares ...func(http.Handler) http.Handler) error
// Walk walks any router tree that implements Routes interface.
func Walk(r Routes, walkFn WalkFunc) error {
return walk(r, walkFn, "")
}
func walk(r Routes, walkFn WalkFunc, parentRoute string, parentMw ...func(http.Handler) http.Handler) error {
for _, route := range r.Routes() {
mws := make([]func(http.Handler) http.Handler, len(parentMw))
copy(mws, parentMw)
mws = append(mws, r.Middlewares()...)
if route.SubRoutes != nil {
if err := walk(route.SubRoutes, walkFn, parentRoute+route.Pattern, mws...); err != nil {
return err
}
continue
}
for method, handler := range route.Handlers {
if method == "*" {
// Ignore a "catchAll" method, since we pass down all the specific methods for each route.
continue
}
fullRoute := parentRoute + route.Pattern
fullRoute = strings.Replace(fullRoute, "/*/", "/", -1)
if chain, ok := handler.(*ChainHandler); ok {
if err := walkFn(method, fullRoute, chain.Endpoint, append(mws, chain.Middlewares...)...); err != nil {
return err
}
} else {
if err := walkFn(method, fullRoute, handler, mws...); err != nil {
return err
}
}
}
}
return nil
}
+2
View File
@@ -0,0 +1,2 @@
jose-util/jose-util
jose-util.t.err
+53
View File
@@ -0,0 +1,53 @@
# https://github.com/golangci/golangci-lint
run:
skip-files:
- doc_test.go
modules-download-mode: readonly
linters:
enable-all: true
disable:
- gochecknoglobals
- goconst
- lll
- maligned
- nakedret
- scopelint
- unparam
- funlen # added in 1.18 (requires go-jose changes before it can be enabled)
linters-settings:
gocyclo:
min-complexity: 35
issues:
exclude-rules:
- text: "don't use ALL_CAPS in Go names"
linters:
- golint
- text: "hardcoded credentials"
linters:
- gosec
- text: "weak cryptographic primitive"
linters:
- gosec
- path: json/
linters:
- dupl
- errcheck
- gocritic
- gocyclo
- golint
- govet
- ineffassign
- staticcheck
- structcheck
- stylecheck
- unused
- path: _test\.go
linters:
- scopelint
- path: jwk.go
linters:
- gocyclo
+33
View File
@@ -0,0 +1,33 @@
language: go
matrix:
fast_finish: true
allow_failures:
- go: tip
go:
- "1.13.x"
- "1.14.x"
- tip
before_script:
- export PATH=$HOME/.local/bin:$PATH
before_install:
- go get -u github.com/mattn/goveralls github.com/wadey/gocovmerge
- curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh | sh -s -- -b $(go env GOPATH)/bin v1.18.0
- pip install cram --user
script:
- go test -v -covermode=count -coverprofile=profile.cov .
- go test -v -covermode=count -coverprofile=cryptosigner/profile.cov ./cryptosigner
- go test -v -covermode=count -coverprofile=cipher/profile.cov ./cipher
- go test -v -covermode=count -coverprofile=jwt/profile.cov ./jwt
- go test -v ./json # no coverage for forked encoding/json package
- golangci-lint run
- cd jose-util && go build && PATH=$PWD:$PATH cram -v jose-util.t # cram tests jose-util
- cd ..
after_success:
- gocovmerge *.cov */*.cov > merged.coverprofile
- goveralls -coverprofile merged.coverprofile -service=travis-ci
+10
View File
@@ -0,0 +1,10 @@
Serious about security
======================
Square recognizes the important contributions the security research community
can make. We therefore encourage reporting security issues with the code
contained in this repository.
If you believe you have discovered a security vulnerability, please follow the
guidelines at <https://bugcrowd.com/squareopensource>.
+15
View File
@@ -0,0 +1,15 @@
# Contributing
If you would like to contribute code to go-jose you can do so through GitHub by
forking the repository and sending a pull request.
When submitting code, please make every effort to follow existing conventions
and style in order to keep the code as readable as possible. Please also make
sure all tests pass by running `go test`, and format your code with `go fmt`.
We also recommend using `golint` and `errcheck`.
Before your code can be accepted into the project you must also sign the
Individual Contributor License Agreement. We use [cla-assistant.io][1] and you
will be prompted to sign once a pull request is opened.
[1]: https://cla-assistant.io/
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Some files were not shown because too many files have changed in this diff Show More