Compare commits

..

59 Commits

Author SHA1 Message Date
Nuno Diegues 42cdb557a0 Release 2021.1.3 2021-01-14 13:20:02 +00:00
Nuno Diegues 7c3ceeeaef TUN-3757: Fix legacy Uint flags that are incorrectly handled by ufarve library
The following UInt flags:
 * Uint64 - heartbeat-count, compression-quality
 * Uint - retries, port, proxy-port

were not being correctly picked from the configuration YAML
since the multi origin refactor

This is due to a limitation of the ufarve library, which we
overcome for now with handling those as Int flags.
2021-01-14 13:08:55 +00:00
Nuno Diegues 391facbedf TUN-3756: File logging output must consider the directory 2021-01-14 11:53:35 +00:00
Nuno Diegues 1c9f3ac7d4 Release 2021.1.2 2021-01-14 00:24:44 +00:00
Nuno Diegues 6852047ef1 TUN-3747: Fix logging in Windows 2021-01-13 23:23:31 +00:00
Nuno Diegues a2109e4a78 Release 2021.1.1 2021-01-13 17:59:15 +00:00
Nuno Diegues 01f0d67875 TUN-3744: Fix compilation error in windows service 2021-01-13 16:20:41 +00:00
Nuno Diegues 33d9b4027b Release 2021.1.0 2021-01-13 15:18:31 +00:00
Nuno Diegues 9ed536c990 TUN-3738: Consume UI events even when UI is disabled
Not doing so was causing cloudflared to become stuck after
some time. This would happen because the Observer pattern
was sending events to the UI channel (that has 16 slots) but
no one was consuming those when the UI is not enabled (which
is the default case).

Hence, events (such as connection disconnect / reconnect) would
cause that buffer to be full and cause cloudflared to become
apparently stuck, in the sense that the connections would not be
reconnected.
2021-01-13 13:10:30 +00:00
Adam Chalmers 119d96cdc1 TUN-3670: Update Teamnet API gateway prefixes 2021-01-11 13:40:44 -06:00
Adam Chalmers 94ca4f98dd Review from Igor 2021-01-11 19:36:31 +00:00
Adam Chalmers b601b24f52 Adam's suggestions 2021-01-11 19:36:31 +00:00
TownLake b40d8557cf TUN-3691: Edit Teamnet help text 2021-01-11 19:36:31 +00:00
Areg Harutyunyan a34604cfc7 Merge remote-tracking branch 'upstream/master' 2021-01-08 13:50:11 +00:00
Samuel Rhea 1c1a8f21b2 Update to add deprecated version note (#271) 2021-01-08 11:38:20 +00:00
Adam Chalmers 78ffb1b846 TUN-3688: Subcommand for users to check which route an IP proxies through 2021-01-07 15:31:26 +00:00
Adam Chalmers 92fd039440 TUN-3722: Teamnet API paths include /network 2021-01-06 17:54:51 -06:00
Areg Harutyunyan 55bf904689 TUN-3471: Add structured log context to logs 2021-01-05 20:21:16 +00:00
Adam Chalmers abab78730d TUN-3708: Better info message about system root certpool on Windows 2020-12-31 10:10:47 -06:00
Adam Chalmers b855e33327 TUN-3706: Quit if any origin service fails to start 2020-12-30 13:48:19 -06:00
Adam Chalmers 32336859f8 TUN-3689: Delete routes via cloudflared CLI 2020-12-29 13:53:48 -06:00
Adam Chalmers 94c639d225 TUN-3669: Teamnet commands to add/show Teamnet routes. 2020-12-29 17:39:08 +00:00
Areg Harutyunyan 2ea491b1d0 TUN-3607: Set up single-file logger with zerolog 2020-12-23 14:15:39 -06:00
Areg Harutyunyan 9bc1c0c70b TUN-3472: Set up rolling logger with zerolog and lumberjack 2020-12-23 14:15:37 -06:00
Areg Harutyunyan 870f5fa907 TUN-3470: Replace in-house logger calls with zerolog 2020-12-23 14:15:17 -06:00
Maxime Guerreiro 06404bf3e8 TUN-3650: Remove unused awsuploader package 2020-12-15 18:02:17 +01:00
Adam Chalmers 3b4ed70cf3 Release 2020.12.0 2020-12-08 16:17:24 -06:00
Adam Chalmers d45ca67498 TUN-3612: Upgrade to Go 1.15.6 2020-12-04 23:24:16 +00:00
Sudarsan Reddy 1c0dac77d7 TUN-3599: improved delete if credentials isnt found.
Tunnel delete is successful even if we don't find the credentials
file in the user's filesystem. We no longer "error" indicating this
is a problem. This fix also enables chaining of the delete command
by removing a pre-mature return if the credentials file is not found.
2020-12-04 11:44:13 +00:00
Samuel Rhea 65e70650dc Update to add deprecated version note (#271) 2020-12-03 21:32:23 +04:00
Adam Chalmers 38fb0b28b6 TUN-3593: /ready endpoint for k8s readiness. Move tunnel events out of UI package, into connection package. 2020-12-02 15:22:59 -06:00
cthuang bda8fe2fbe TUN-3594: Log response status at debug level 2020-11-27 12:28:20 +00:00
Adam Chalmers 6a8c7c0727 Release 2020.11.11 2020-11-25 10:09:42 -06:00
Adam Chalmers 69fd502db3 TUN-3581: Tunnels can be run by name using only --credentials-file, no
origin cert necessary.
2020-11-25 09:54:28 -06:00
Michael Borkenstein fcc393e2f0 AUTH-3221: Saves org token to disk and uses it to refresh the app token 2020-11-24 21:38:59 +00:00
Areg Harutyunyan cad58b9b57 TUN-3561: Unified logger configuration 2020-11-23 16:49:07 +00:00
Joe Groocock 78cb60b85f EDGEPLAT-2958 remove deb-compression, defaulting to gzip
dpkg does not support bzip2 compression, so fails to unpack and install
the built package. By omitting the option, fpm defaults to gzip which is
the default supported option by dpkg.

Signed-off-by: Joe Groocock <jgroocock@cloudflare.com>
2020-11-23 16:27:11 +00:00
Adam Chalmers a08a7030d1 TUN-3578: cloudflared tunnel route dns should allow wildcard subdomains 2020-11-23 09:37:46 -06:00
Adam Chalmers 87203bbe25 Release 2020.11.10 2020-11-20 12:25:44 -06:00
Joe Groocock 11acb50cf7 EDGEPLAT-2958 build cloudflared for Bullseye
Signed-off-by: Joe Groocock <jgroocock@cloudflare.com>
2020-11-20 18:24:58 +00:00
Adam Chalmers 23f2a04ed7 TUN-3562: Fix panic when using bastion mode ingress rule 2020-11-20 11:20:39 -06:00
cthuang 1805261263 Release 2020.11.9 2020-11-19 09:29:12 +00:00
Adam Chalmers 53de779a0a TUN-3544: Upgrade to Go 1.15.5 2020-11-18 16:13:54 -06:00
Adam Chalmers b7e91466f5 TUN-3558: cloudflared allows empty config files 2020-11-18 21:13:06 +00:00
Troy Varney 4c1b89576c DEVTOOLS-7936: Remove redundant chgrp from publish
This removes the redundant chgrp command from the publish step when
pushing packages to our public repositories. The directory being pushed
to has the setgid bit set on it, which means that we don't need to force
the group using this command. Further, attempting to do so resulted in
an error as the cfsync user does not have the appropriate permissions to
use the chgrp command.
2020-11-18 19:35:26 +00:00
cthuang a1a554a29d TUN-3559: Share response meta header with other packages 2020-11-18 16:51:03 +00:00
cthuang fdb1f961b3 TUN-3557: Detect SSE if content-type starts with text/event-stream 2020-11-18 15:59:41 +00:00
Adam Chalmers 293b9af4a7 Release 2020.11.8 2020-11-17 17:15:50 -06:00
Adam Chalmers 029f7e0378 TUN-3555: Single origin service should default to localhost:8080 2020-11-17 23:12:32 +00:00
Adam Chalmers 58c5e25b9a Release 2020.11.7 2020-11-16 14:13:04 -06:00
Adam Chalmers 25e72f7760 TUN-3549: Use a separate handler for each websocket proxy 2020-11-16 20:05:35 +00:00
Adam Chalmers 7613410855 TUN-3548, TUN-3547: Bastion mode can be specified as a service, doesn't
require URL.
2020-11-16 20:04:36 +00:00
cthuang c40cb7dc56 TUN-3514: Stop setting --is-autoupdated flag after autoupdate because it can break named tunnel running in k8s 2020-11-16 09:40:38 +00:00
Adam Chalmers 9ae5f306bf Release 2020.11.6 2020-11-14 20:13:18 -06:00
Adam Chalmers 6159cb536f TUN-3546: Fix panic in tlsconfig.LoadOriginCA 2020-11-14 20:03:29 -06:00
Adam Chalmers ae4f687754 Release 2020.11.5 2020-11-13 17:39:51 -06:00
Adam Chalmers ce7d0572fe TUN-3543: ProxyAddress not using default in single-origin mode 2020-11-13 17:27:55 -06:00
Troy Varney 030b768eeb DEVTOOLS-7936: Set permissions on public packages
This updates the public repository upload process to change the group on
the uploaded files to `cf` and adds the write permission for members of
the group. This should allow the `cf` user to properly overwrite the
file when signing it.
2020-11-13 19:02:40 +00:00
Adam Chalmers f36dc6cfd8 TUN-3540: Better copy in ingress rules error messages 2020-11-12 17:57:19 -06:00
451 changed files with 11996 additions and 78587 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
# use a builder image for building cloudflare
ARG TARGET_GOOS
ARG TARGET_GOARCH
FROM golang:1.15.3 as builder
FROM golang:1.15.6 as builder
ENV GO111MODULE=on \
CGO_ENABLED=0 \
TARGET_GOOS=${TARGET_GOOS} \
+3 -2
View File
@@ -89,6 +89,7 @@ define publish_package
for HOST in $(CF_PKG_HOSTS); do \
ssh-keyscan -t rsa $$HOST >> ~/.ssh/known_hosts; \
scp -4 cloudflared*.$(1) cfsync@$$HOST:/state/cf-pkg/staging/$(2)/$(TARGET_PUBLIC_REPO)/cloudflared/; \
ssh cfsync@$$HOST 'chmod g+w /state/cf-pkg/staging/$(2)/$(TARGET_PUBLIC_REPO)/cloudflared/*.$(1)'; \
done
endef
@@ -104,7 +105,7 @@ define build_package
mkdir -p $(PACKAGE_DIR)
cp cloudflared $(PACKAGE_DIR)/cloudflared
cat cloudflared_man_template | sed -e 's/\$${VERSION}/$(VERSION)/; s/\$${DATE}/$(DATE)/' > $(PACKAGE_DIR)/cloudflared.1
fakeroot fpm -C $(PACKAGE_DIR) -s dir -t $(1) --$(1)-compression bzip2 \
fakeroot fpm -C $(PACKAGE_DIR) -s dir -t $(1) \
--description 'Cloudflare Argo tunnel daemon' \
--vendor 'Cloudflare' \
--license 'Cloudflare Service Agreement' \
@@ -225,4 +226,4 @@ vet:
.PHONY: msi
msi: cloudflared
go-msi make --msi cloudflared.msi --version $(MSI_VERSION)
go-msi make --msi cloudflared.msi --version $(MSI_VERSION)
+11
View File
@@ -30,3 +30,14 @@ Once installed, you can authenticate `cloudflared` into your Cloudflare account
## TryCloudflare
Want to test Argo Tunnel before adding a website to Cloudflare? You can do so with TryCloudflare using the documentation [available here](https://developers.cloudflare.com/argo-tunnel/learning/trycloudflare).
## Deprecated versions
Cloudflare currently supports all versions of `cloudflared`. Starting on March 20, 2021, Cloudflare will no longer support versions released prior to 2020.5.1.
All features available in versions released prior to 2020.5.1 are available in current versions. Breaking changes unrelated to feature availability may be introduced that will impact versions released prior to 2020.5.1. You can read more about upgrading `cloudflared` in our [developer documentation](https://developers.cloudflare.com/argo-tunnel/getting-started/installation#updating-cloudflared).
| Version(s) | Deprecation status |
|---|---|
| 2020.5.1 and later | Supported |
| Versions prior to 2020.5.1 | Will no longer be supported starting March 20, 2021 |
+68
View File
@@ -1,3 +1,71 @@
2021.1.3
- 2021-01-14 TUN-3756: File logging output must consider the directory
- 2021-01-14 TUN-3757: Fix legacy Uint flags that are incorrectly handled by ufarve library
2021.1.2
- 2021-01-13 TUN-3747: Fix logging in Windows
2021.1.1
- 2021-01-13 TUN-3744: Fix compilation error in windows service
2021.1.0
- 2021-01-11 TUN-3670: Update Teamnet API gateway prefixes
- 2021-01-13 TUN-3738: Consume UI events even when UI is disabled
- 2021-01-06 TUN-3722: Teamnet API paths include /network
- 2021-01-05 TUN-3688: Subcommand for users to check which route an IP proxies through
- 2021-01-08 TUN-3691: Edit Teamnet help text
- 2020-12-30 TUN-3706: Quit if any origin service fails to start
- 2020-12-31 TUN-3708: Better info message about system root certpool on Windows
- 2020-12-21 TUN-3669: Teamnet commands to add/show Teamnet routes.
- 2020-12-29 TUN-3689: Delete routes via cloudflared CLI
- 2020-12-28 TUN-3471: Add structured log context to logs
- 2020-12-15 TUN-3650: Remove unused awsuploader package
- 2020-12-03 Update to add deprecated version note (#271)
- 2020-12-02 TUN-3472: Set up rolling logger with zerolog and lumberjack
- 2020-12-08 TUN-3607: Set up single-file logger with zerolog
- 2020-12-03 Update to add deprecated version note (#271)
- 2020-11-25 TUN-3470: Replace in-house logger calls with zerolog
2020.12.0
- 2020-12-04 TUN-3599: improved delete if credentials isnt found.
- 2020-12-04 TUN-3612: Upgrade to Go 1.15.6
- 2020-11-30 TUN-3593: /ready endpoint for k8s readiness. Move tunnel events out of UI package, into connection package.
- 2020-11-27 TUN-3594: Log response status at debug level
2020.11.11
- 2020-11-20 TUN-3578: cloudflared tunnel route dns should allow wildcard subdomains
- 2020-11-21 EDGEPLAT-2958 remove deb-compression, defaulting to gzip
- 2020-11-23 TUN-3581: Tunnels can be run by name using only --credentials-file, no origin cert necessary.
- 2020-11-15 TUN-3561: Unified logger configuration
- 2020-11-08 AUTH-3221: Saves org token to disk and uses it to refresh the app token
2020.11.10
- 2020-11-20 TUN-3562: Fix panic when using bastion mode ingress rule
- 2020-11-20 EDGEPLAT-2958 build cloudflared for Bullseye
2020.11.9
- 2020-11-18 TUN-3557: Detect SSE if content-type starts with text/event-stream
- 2020-11-18 TUN-3559: Share response meta header with other packages
- 2020-11-18 DEVTOOLS-7936: Remove redundant chgrp from publish
- 2020-11-18 TUN-3558: cloudflared allows empty config files
- 2020-11-18 TUN-3544: Upgrade to Go 1.15.5
2020.11.8
- 2020-11-17 TUN-3555: Single origin service should default to localhost:8080
2020.11.7
- 2020-11-13 TUN-3514: Stop setting --is-autoupdated flag after autoupdate because it can break named tunnel running in k8s
- 2020-11-15 TUN-3548, TUN-3547: Bastion mode can be specified as a service, doesn't require URL.
- 2020-11-16 TUN-3549: Use a separate handler for each websocket proxy
2020.11.6
- 2020-11-14 TUN-3546: Fix panic in tlsconfig.LoadOriginCA
2020.11.5
- 2020-11-12 TUN-3540: Better copy in ingress rules error messages
- 2020-11-12 DEVTOOLS-7936: Set permissions on public packages
- 2020-11-13 TUN-3543: ProxyAddress not using default in single-origin mode
2020.11.4
- 2020-11-11 TUN-3534: Specific error message when credentials file is a .pem not .json
- 2020-11-02 TUN-3500: Integrate replace h2mux by http2 work with multiple origin support
-106
View File
@@ -1,106 +0,0 @@
package awsuploader
import (
"os"
"path/filepath"
"time"
"github.com/cloudflare/cloudflared/logger"
)
// DirectoryUploadManager is used to manage file uploads on an interval from a directory
type DirectoryUploadManager struct {
logger logger.Service
uploader Uploader
rootDirectory string
sweepInterval time.Duration
ticker *time.Ticker
shutdownC chan struct{}
workQueue chan string
}
// NewDirectoryUploadManager create a new DirectoryUploadManager
// uploader is an Uploader to use as an actual uploading engine
// directory is the directory to sweep for files to upload
// sweepInterval is how often to iterate the directory and upload the files within
func NewDirectoryUploadManager(logger logger.Service, uploader Uploader, directory string, sweepInterval time.Duration, shutdownC chan struct{}) *DirectoryUploadManager {
workerCount := 10
manager := &DirectoryUploadManager{
logger: logger,
uploader: uploader,
rootDirectory: directory,
sweepInterval: sweepInterval,
shutdownC: shutdownC,
workQueue: make(chan string, workerCount),
}
//start workers
for i := 0; i < workerCount; i++ {
go manager.worker()
}
return manager
}
// Upload a file using the uploader
// This is useful for "out of band" uploads that need to be triggered immediately instead of waiting for the sweep
func (m *DirectoryUploadManager) Upload(filepath string) error {
return m.uploader.Upload(filepath)
}
// Start the upload ticker to walk the directories
func (m *DirectoryUploadManager) Start() {
m.ticker = time.NewTicker(m.sweepInterval)
go m.run()
}
func (m *DirectoryUploadManager) run() {
for {
select {
case <-m.shutdownC:
m.ticker.Stop()
return
case <-m.ticker.C:
m.sweep()
}
}
}
// sweep the directory and kick off uploads
func (m *DirectoryUploadManager) sweep() {
filepath.Walk(m.rootDirectory, func(path string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return nil
}
//30 days ago
retentionTime := 30 * (time.Hour * 24)
checkTime := time.Now().Add(-time.Duration(retentionTime))
//delete the file it is stale
if info.ModTime().Before(checkTime) {
os.Remove(path)
return nil
}
//add the upload to the work queue
go func() {
m.workQueue <- path
}()
return nil
})
}
// worker handles upload requests
func (m *DirectoryUploadManager) worker() {
for {
select {
case <-m.shutdownC:
return
case filepath := <-m.workQueue:
if err := m.Upload(filepath); err != nil {
m.logger.Errorf("Cannot upload file to s3 bucket: %s", err)
} else {
os.Remove(filepath)
}
}
}
}
@@ -1,137 +0,0 @@
package awsuploader
import (
"errors"
"io/ioutil"
"math/rand"
"os"
"path/filepath"
"testing"
"time"
"github.com/cloudflare/cloudflared/logger"
)
type MockUploader struct {
shouldFail bool
}
func (m *MockUploader) Upload(filepath string) error {
if m.shouldFail {
return errors.New("upload set to fail")
}
return nil
}
func NewMockUploader(shouldFail bool) Uploader {
return &MockUploader{shouldFail: shouldFail}
}
func getDirectoryPath(t *testing.T) string {
dir, err := os.Getwd()
if err != nil {
t.Fatal("couldn't create the test directory!", err)
}
return filepath.Join(dir, "uploads")
}
func setupTestDirectory(t *testing.T) string {
path := getDirectoryPath(t)
os.RemoveAll(path)
time.Sleep(100 * time.Millisecond) //short way to wait for the OS to delete the folder
err := os.MkdirAll(path, os.ModePerm)
if err != nil {
t.Fatal("couldn't create the test directory!", err)
}
return path
}
func createUploadManager(t *testing.T, shouldFailUpload bool) *DirectoryUploadManager {
rootDirectory := setupTestDirectory(t)
uploader := NewMockUploader(shouldFailUpload)
logger := logger.NewOutputWriter(logger.NewMockWriteManager())
shutdownC := make(chan struct{})
return NewDirectoryUploadManager(logger, uploader, rootDirectory, 1*time.Second, shutdownC)
}
func createFile(t *testing.T, fileName string) (*os.File, string) {
path := filepath.Join(getDirectoryPath(t), fileName)
f, err := os.Create(path)
if err != nil {
t.Fatal("upload to create file for sweep test", err)
}
return f, path
}
func TestUploadSuccess(t *testing.T) {
manager := createUploadManager(t, false)
path := filepath.Join(getDirectoryPath(t), "test_file")
if err := manager.Upload(path); err != nil {
t.Fatal("the upload request method failed", err)
}
}
func TestUploadFailure(t *testing.T) {
manager := createUploadManager(t, true)
path := filepath.Join(getDirectoryPath(t), "test_file")
if err := manager.Upload(path); err == nil {
t.Fatal("the upload request method should have failed and didn't", err)
}
}
func TestSweepSuccess(t *testing.T) {
manager := createUploadManager(t, false)
f, path := createFile(t, "test_file")
defer f.Close()
manager.Start()
time.Sleep(2 * time.Second)
if _, err := os.Stat(path); os.IsExist(err) {
//the file should have been deleted
t.Fatal("the manager failed to delete the file", err)
}
}
func TestSweepFailure(t *testing.T) {
manager := createUploadManager(t, true)
f, path := createFile(t, "test_file")
defer f.Close()
manager.Start()
time.Sleep(2 * time.Second)
_, serr := f.Stat()
if serr != nil {
//the file should still exist
os.Remove(path)
t.Fatal("the manager failed to delete the file", serr)
}
}
func TestHighLoad(t *testing.T) {
manager := createUploadManager(t, false)
for i := 0; i < 30; i++ {
f, _ := createFile(t, randomString(6))
defer f.Close()
}
manager.Start()
time.Sleep(4 * time.Second)
directory := getDirectoryPath(t)
files, err := ioutil.ReadDir(directory)
if err != nil || len(files) > 0 {
t.Fatalf("the manager failed to upload all the files: %s files left: %d", err, len(files))
}
}
// LowerCase [a-z]
const randSet = "abcdefghijklmnopqrstuvwxyz"
// String returns a string of length 'n' from a set of letters 'lset'
func randomString(n int) string {
b := make([]byte, n)
lsetLen := len(randSet)
for i := range b {
b[i] = randSet[rand.Intn(lsetLen)]
}
return string(b)
}
-62
View File
@@ -1,62 +0,0 @@
package awsuploader
import (
"context"
"os"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
)
//FileUploader aws compliant bucket upload
type FileUploader struct {
storage *s3.S3
bucketName string
clientID string
secretID string
}
// NewFileUploader creates a new S3 compliant bucket uploader
func NewFileUploader(bucketName, region, accessKeyID, secretID, token, s3Host string) (*FileUploader, error) {
sess, err := session.NewSession(&aws.Config{
Region: aws.String(region),
Credentials: credentials.NewStaticCredentials(accessKeyID, secretID, token),
})
if err != nil {
return nil, err
}
var storage *s3.S3
if s3Host != "" {
storage = s3.New(sess, &aws.Config{Endpoint: aws.String(s3Host)})
} else {
storage = s3.New(sess)
}
return &FileUploader{
storage: storage,
bucketName: bucketName,
}, nil
}
// Upload a file to the bucket
func (u *FileUploader) Upload(filepath string) error {
info, err := os.Stat(filepath)
if err != nil {
return err
}
file, err := os.Open(filepath)
if err != nil {
return err
}
defer file.Close()
_, serr := u.storage.PutObjectWithContext(context.Background(), &s3.PutObjectInput{
Bucket: aws.String(u.bucketName),
Key: aws.String(info.Name()),
Body: file,
})
return serr
}
-7
View File
@@ -1,7 +0,0 @@
package awsuploader
// UploadManager is used to manage file uploads on an interval
type UploadManager interface {
Upload(string) error
Start()
}
-7
View File
@@ -1,7 +0,0 @@
package awsuploader
// Uploader the functions required to upload to a bucket
type Uploader interface {
//Upload a file to the bucket
Upload(string) error
}
+8 -5
View File
@@ -12,10 +12,13 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/token"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/logger"
"github.com/pkg/errors"
"github.com/rs/zerolog"
)
const LogFieldOriginURL = "originURL"
type StartOptions struct {
OriginURL string
Headers http.Header
@@ -49,7 +52,7 @@ func (c *StdinoutStream) Write(p []byte) (int, error) {
// Helper to allow defering the response close with a check that the resp is not nil
func closeRespBody(resp *http.Response) {
if resp != nil {
resp.Body.Close()
_ = resp.Body.Close()
}
}
@@ -103,7 +106,7 @@ func Serve(remoteConn Connection, listener net.Listener, shutdownC <-chan struct
// serveConnection handles connections for the Serve() call
func serveConnection(remoteConn Connection, c net.Conn, options *StartOptions) {
defer c.Close()
remoteConn.ServeStream(options, c)
_ = remoteConn.ServeStream(options, c)
}
// IsAccessResponse checks the http Response to see if the url location
@@ -125,13 +128,13 @@ func IsAccessResponse(resp *http.Response) bool {
}
// BuildAccessRequest builds an HTTP request with the Access token set
func BuildAccessRequest(options *StartOptions, logger logger.Service) (*http.Request, error) {
func BuildAccessRequest(options *StartOptions, log *zerolog.Logger) (*http.Request, error) {
req, err := http.NewRequest(http.MethodGet, options.OriginURL, nil)
if err != nil {
return nil, err
}
token, err := token.FetchTokenWithRedirect(req.URL, logger)
token, err := token.FetchTokenWithRedirect(req.URL, log)
if err != nil {
return nil, err
}
+9 -9
View File
@@ -9,8 +9,8 @@ import (
"sync"
"testing"
"github.com/cloudflare/cloudflared/logger"
ws "github.com/gorilla/websocket"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
)
@@ -43,8 +43,8 @@ func (s *testStreamer) Write(p []byte) (int, error) {
func TestStartClient(t *testing.T) {
message := "Good morning Austin! Time for another sunny day in the great state of Texas."
logger := logger.NewOutputWriter(logger.NewMockWriteManager())
wsConn := NewWSConnection(logger, false)
log := zerolog.Nop()
wsConn := NewWSConnection(&log, false)
ts := newTestWebSocketServer()
defer ts.Close()
@@ -55,10 +55,10 @@ func TestStartClient(t *testing.T) {
}
err := StartClient(wsConn, buf, options)
assert.NoError(t, err)
buf.Write([]byte(message))
_, _ = buf.Write([]byte(message))
readBuffer := make([]byte, len(message))
buf.Read(readBuffer)
_, _ = buf.Read(readBuffer)
assert.Equal(t, message, string(readBuffer))
}
@@ -68,9 +68,9 @@ func TestStartServer(t *testing.T) {
t.Fatalf("Error starting listener: %v", err)
}
message := "Good morning Austin! Time for another sunny day in the great state of Texas."
logger := logger.NewOutputWriter(logger.NewMockWriteManager())
log := zerolog.Nop()
shutdownC := make(chan struct{})
wsConn := NewWSConnection(logger, false)
wsConn := NewWSConnection(&log, false)
ts := newTestWebSocketServer()
defer ts.Close()
options := &StartOptions{
@@ -86,10 +86,10 @@ func TestStartServer(t *testing.T) {
}()
conn, err := net.Dial("tcp", listener.Addr().String())
conn.Write([]byte(message))
_, _ = conn.Write([]byte(message))
readBuffer := make([]byte, len(message))
conn.Read(readBuffer)
_, _ = conn.Read(readBuffer)
assert.Equal(t, string(readBuffer), message)
}
+20 -19
View File
@@ -8,16 +8,17 @@ import (
"net/http/httputil"
"github.com/cloudflare/cloudflared/cmd/cloudflared/token"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/socks"
cfwebsocket "github.com/cloudflare/cloudflared/websocket"
"github.com/gorilla/websocket"
"github.com/rs/zerolog"
)
// Websocket is used to carry data via WS binary frames over the tunnel from client to the origin
// This implements the functions for glider proxy (sock5) and the carrier interface
type Websocket struct {
logger logger.Service
log *zerolog.Logger
isSocks bool
}
@@ -36,9 +37,9 @@ func (d *wsdialer) Dial(address string) (io.ReadWriteCloser, *socks.AddrSpec, er
}
// NewWSConnection returns a new connection object
func NewWSConnection(logger logger.Service, isSocks bool) Connection {
func NewWSConnection(log *zerolog.Logger, isSocks bool) Connection {
return &Websocket{
logger: logger,
log: log,
isSocks: isSocks,
}
}
@@ -46,9 +47,9 @@ func NewWSConnection(logger logger.Service, isSocks bool) Connection {
// ServeStream will create a Websocket client stream connection to the edge
// it blocks and writes the raw data from conn over the tunnel
func (ws *Websocket) ServeStream(options *StartOptions, conn io.ReadWriter) error {
wsConn, err := createWebsocketStream(options, ws.logger)
wsConn, err := createWebsocketStream(options, ws.log)
if err != nil {
ws.logger.Errorf("failed to connect to %s with error: %s", options.OriginURL, err)
ws.log.Err(err).Str(LogFieldOriginURL, options.OriginURL).Msg("failed to connect to origin")
return err
}
defer wsConn.Close()
@@ -58,7 +59,7 @@ func (ws *Websocket) ServeStream(options *StartOptions, conn io.ReadWriter) erro
requestHandler := socks.NewRequestHandler(dialer)
socksServer := socks.NewConnectionHandler(requestHandler)
socksServer.Serve(conn)
_ = socksServer.Serve(conn)
} else {
cfwebsocket.Stream(wsConn, conn)
}
@@ -68,13 +69,13 @@ func (ws *Websocket) ServeStream(options *StartOptions, conn io.ReadWriter) erro
// StartServer creates a Websocket server to listen for connections.
// This is used on the origin (tunnel) side to take data from the muxer and send it to the origin
func (ws *Websocket) StartServer(listener net.Listener, remote string, shutdownC <-chan struct{}) error {
return cfwebsocket.StartProxyServer(ws.logger, listener, remote, shutdownC, cfwebsocket.DefaultStreamHandler)
return cfwebsocket.StartProxyServer(ws.log, listener, remote, shutdownC, cfwebsocket.DefaultStreamHandler)
}
// createWebsocketStream will create a WebSocket connection to stream data over
// It also handles redirects from Access and will present that flow if
// the token is not present on the request
func createWebsocketStream(options *StartOptions, logger logger.Service) (*cfwebsocket.Conn, error) {
func createWebsocketStream(options *StartOptions, log *zerolog.Logger) (*cfwebsocket.Conn, error) {
req, err := http.NewRequest(http.MethodGet, options.OriginURL, nil)
if err != nil {
return nil, err
@@ -82,13 +83,13 @@ func createWebsocketStream(options *StartOptions, logger logger.Service) (*cfweb
req.Header = options.Headers
dump, err := httputil.DumpRequest(req, false)
logger.Debugf("Websocket request: %s", string(dump))
log.Debug().Msgf("Websocket request: %s", string(dump))
wsConn, resp, err := cfwebsocket.ClientConnect(req, nil)
defer closeRespBody(resp)
if err != nil && IsAccessResponse(resp) {
wsConn, err = createAccessAuthenticatedStream(options, logger)
wsConn, err = createAccessAuthenticatedStream(options, log)
if err != nil {
return nil, err
}
@@ -104,8 +105,8 @@ func createWebsocketStream(options *StartOptions, logger logger.Service) (*cfweb
// this probably means the token in storage is invalid (expired/revoked). If that
// happens it deletes the token and runs the connection again, so the user can
// login again and generate a new one.
func createAccessAuthenticatedStream(options *StartOptions, logger logger.Service) (*websocket.Conn, error) {
wsConn, resp, err := createAccessWebSocketStream(options, logger)
func createAccessAuthenticatedStream(options *StartOptions, log *zerolog.Logger) (*websocket.Conn, error) {
wsConn, resp, err := createAccessWebSocketStream(options, log)
defer closeRespBody(resp)
if err == nil {
return wsConn, nil
@@ -123,7 +124,7 @@ func createAccessAuthenticatedStream(options *StartOptions, logger logger.Servic
if err := token.RemoveTokenIfExists(originReq.URL); err != nil {
return nil, err
}
wsConn, resp, err = createAccessWebSocketStream(options, logger)
wsConn, resp, err = createAccessWebSocketStream(options, log)
defer closeRespBody(resp)
if err != nil {
return nil, err
@@ -133,23 +134,23 @@ func createAccessAuthenticatedStream(options *StartOptions, logger logger.Servic
}
// createAccessWebSocketStream builds an Access request and makes a connection
func createAccessWebSocketStream(options *StartOptions, logger logger.Service) (*websocket.Conn, *http.Response, error) {
req, err := BuildAccessRequest(options, logger)
func createAccessWebSocketStream(options *StartOptions, log *zerolog.Logger) (*websocket.Conn, *http.Response, error) {
req, err := BuildAccessRequest(options, log)
if err != nil {
return nil, nil, err
}
dump, err := httputil.DumpRequest(req, false)
logger.Debugf("Access Websocket request: %s", string(dump))
log.Debug().Msgf("Access Websocket request: %s", string(dump))
conn, resp, err := cfwebsocket.ClientConnect(req, nil)
if resp != nil {
r, err := httputil.DumpResponse(resp, true)
if r != nil {
logger.Debugf("Websocket response: %q", r)
log.Debug().Msgf("Websocket response: %q", r)
} else if err != nil {
logger.Debugf("Websocket response error: %v", err)
log.Debug().Msgf("Websocket response error: %v", err)
}
}
+7 -9
View File
@@ -1,6 +1,6 @@
pinned_go: &pinned_go go=1.15.3-1
pinned_go: &pinned_go go=1.15.6-1
build_dir: &build_dir /cfsetup_build
default-flavor: stretch
default-flavor: buster
stretch: &stretch
build:
build_dir: *build_dir
@@ -239,10 +239,8 @@ stretch: &stretch
- export GOARCH=amd64
- make publish-cloudflared-junos
jessie: *stretch
buster: *stretch
bullseye: *stretch
centos-7:
publish-rpm:
build_dir: *build_dir
@@ -250,8 +248,8 @@ centos-7:
- https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
pre-cache:
- yum install -y fakeroot
- wget https://golang.org/dl/go1.15.3.linux-amd64.tar.gz -P /tmp/
- tar -C /usr/local -xzf /tmp/go1.15.3.linux-amd64.tar.gz
- wget https://golang.org/dl/go1.15.6.linux-amd64.tar.gz -P /tmp/
- tar -C /usr/local -xzf /tmp/go1.15.6.linux-amd64.tar.gz
post-cache:
- export PATH=$PATH:/usr/local/go/bin
- export GOOS=linux
@@ -262,8 +260,8 @@ centos-7:
builddeps: *el7_builddeps
pre-cache:
- yum install -y fakeroot
- wget https://golang.org/dl/go1.15.3.linux-amd64.tar.gz -P /tmp/
- tar -C /usr/local -xzf /tmp/go1.15.3.linux-amd64.tar.gz
- wget https://golang.org/dl/go1.15.6.linux-amd64.tar.gz -P /tmp/
- tar -C /usr/local -xzf /tmp/go1.15.6.linux-amd64.tar.gz
post-cache:
- export PATH=$PATH:/usr/local/go/bin
- export GOOS=linux
+15 -25
View File
@@ -5,17 +5,22 @@ import (
"strings"
"github.com/cloudflare/cloudflared/carrier"
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/validation"
"github.com/pkg/errors"
cli "github.com/urfave/cli/v2"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
)
const (
LogFieldHost = "host"
)
// StartForwarder starts a client side websocket forward
func StartForwarder(forwarder config.Forwarder, shutdown <-chan struct{}, logger logger.Service) error {
func StartForwarder(forwarder config.Forwarder, shutdown <-chan struct{}, log *zerolog.Logger) error {
validURL, err := validation.ValidateUrl(forwarder.Listener)
if err != nil {
return errors.Wrap(err, "error validating origin URL")
@@ -41,9 +46,9 @@ func StartForwarder(forwarder config.Forwarder, shutdown <-chan struct{}, logger
}
// we could add a cmd line variable for this bool if we want the SOCK5 server to be on the client side
wsConn := carrier.NewWSConnection(logger, false)
wsConn := carrier.NewWSConnection(log, false)
logger.Infof("Start Websocket listener on: %s", validURL.Host)
log.Info().Str(LogFieldHost, validURL.Host).Msg("Start Websocket listener")
return carrier.StartForwarder(wsConn, validURL.Host, shutdown, options)
}
@@ -52,22 +57,7 @@ func StartForwarder(forwarder config.Forwarder, shutdown <-chan struct{}, logger
// useful for proxying other protocols (like ssh) over websockets
// (which you can put Access in front of)
func ssh(c *cli.Context) error {
logDirectory, logLevel := config.FindLogSettings()
flagLogDirectory := c.String(sshLogDirectoryFlag)
if flagLogDirectory != "" {
logDirectory = flagLogDirectory
}
flagLogLevel := c.String(sshLogLevelFlag)
if flagLogLevel != "" {
logLevel = flagLogLevel
}
logger, err := logger.New(logger.DefaultFile(logDirectory), logger.LogLevelString(logLevel))
if err != nil {
return cliutil.PrintLoggerSetupError("error setting up logger", err)
}
log := logger.CreateSSHLoggerFromContext(c, logger.EnableTerminalLog)
// get the hostname from the cmdline and error out if its not provided
rawHostName := c.String(sshHostnameFlag)
@@ -97,19 +87,19 @@ func ssh(c *cli.Context) error {
}
// we could add a cmd line variable for this bool if we want the SOCK5 server to be on the client side
wsConn := carrier.NewWSConnection(logger, false)
wsConn := carrier.NewWSConnection(log, false)
if c.NArg() > 0 || c.IsSet(sshURLFlag) {
forwarder, err := config.ValidateUrl(c, true)
if err != nil {
logger.Errorf("Error validating origin URL: %s", err)
log.Err(err).Msg("Error validating origin URL")
return errors.Wrap(err, "error validating origin URL")
}
logger.Infof("Start Websocket listener on: %s", forwarder.Host)
log.Info().Str(LogFieldHost, forwarder.Host).Msg("Start Websocket listener")
err = carrier.StartForwarder(wsConn, forwarder.Host, shutdownC, options)
if err != nil {
logger.Errorf("Error on Websocket listener: %s", err)
log.Err(err).Msg("Error on Websocket listener")
}
return err
}
+36 -46
View File
@@ -17,24 +17,23 @@ import (
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/sshgen"
"github.com/cloudflare/cloudflared/validation"
"github.com/pkg/errors"
"golang.org/x/net/idna"
"github.com/getsentry/raven-go"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
"golang.org/x/net/idna"
)
const (
sshHostnameFlag = "hostname"
sshDestinationFlag = "destination"
sshURLFlag = "url"
sshHeaderFlag = "header"
sshTokenIDFlag = "service-token-id"
sshTokenSecretFlag = "service-token-secret"
sshGenCertFlag = "short-lived-cert"
sshLogDirectoryFlag = "log-directory"
sshLogLevelFlag = "log-level"
sshConfigTemplate = `
sshHostnameFlag = "hostname"
sshDestinationFlag = "destination"
sshURLFlag = "url"
sshHeaderFlag = "header"
sshTokenIDFlag = "service-token-id"
sshTokenSecretFlag = "service-token-secret"
sshGenCertFlag = "short-lived-cert"
sshConfigTemplate = `
Add to your {{.Home}}/.ssh/config:
Host {{.Hostname}}
@@ -157,12 +156,12 @@ func Commands() []*cli.Command {
Usage: "specify an Access service token secret you wish to use.",
},
&cli.StringFlag{
Name: sshLogDirectoryFlag,
Name: logger.LogSSHDirectoryFlag,
Aliases: []string{"logfile"}, //added to match the tunnel side
Usage: "Save application log to this directory for reporting issues.",
},
&cli.StringFlag{
Name: sshLogLevelFlag,
Name: logger.LogSSHLevelFlag,
Aliases: []string{"loglevel"}, //added to match the tunnel side
Usage: "Application logging level {fatal, error, info, debug}. ",
},
@@ -207,24 +206,21 @@ func login(c *cli.Context) error {
return err
}
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
args := c.Args()
rawURL := ensureURLScheme(args.First())
appURL, err := url.Parse(rawURL)
if args.Len() < 1 || err != nil {
logger.Errorf("Please provide the url of the Access application\n")
log.Error().Msg("Please provide the url of the Access application")
return err
}
if err := verifyTokenAtEdge(appURL, c, logger); err != nil {
logger.Errorf("Could not verify token: %s", err)
if err := verifyTokenAtEdge(appURL, c, log); err != nil {
log.Err(err).Msg("Could not verify token")
return err
}
cfdToken, err := token.GetTokenIfExists(appURL)
cfdToken, err := token.GetAppTokenIfExists(appURL)
if err != nil {
fmt.Fprintln(os.Stderr, "Unable to find token for provided application.")
return err
@@ -252,32 +248,29 @@ func curl(c *cli.Context) error {
if err := raven.SetDSN(sentryDSN); err != nil {
return err
}
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
args := c.Args()
if args.Len() < 1 {
logger.Error("Please provide the access app and command you wish to run.")
log.Error().Msg("Please provide the access app and command you wish to run.")
return errors.New("incorrect args")
}
cmdArgs, allowRequest := parseAllowRequest(args.Slice())
appURL, err := getAppURL(cmdArgs, logger)
appURL, err := getAppURL(cmdArgs, log)
if err != nil {
return err
}
tok, err := token.GetTokenIfExists(appURL)
tok, err := token.GetAppTokenIfExists(appURL)
if err != nil || tok == "" {
if allowRequest {
logger.Info("You don't have an Access token set. Please run access token <access application> to fetch one.")
log.Info().Msg("You don't have an Access token set. Please run access token <access application> to fetch one.")
return shell.Run("curl", cmdArgs...)
}
tok, err = token.FetchToken(appURL, logger)
tok, err = token.FetchToken(appURL, log)
if err != nil {
logger.Errorf("Failed to refresh token: %s", err)
log.Err(err).Msg("Failed to refresh token")
return err
}
}
@@ -297,7 +290,7 @@ func generateToken(c *cli.Context) error {
fmt.Fprintln(os.Stderr, "Please provide a url.")
return err
}
tok, err := token.GetTokenIfExists(appURL)
tok, err := token.GetAppTokenIfExists(appURL)
if err != nil || tok == "" {
fmt.Fprintln(os.Stderr, "Unable to find token for provided application. Please run token command to generate token.")
return err
@@ -331,10 +324,7 @@ func sshConfig(c *cli.Context) error {
// sshGen generates a short lived certificate for provided hostname
func sshGen(c *cli.Context) error {
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
// get the hostname from the cmdline and error out if its not provided
rawHostName := c.String(sshHostnameFlag)
@@ -351,7 +341,7 @@ func sshGen(c *cli.Context) error {
// this fetchToken function mutates the appURL param. We should refactor that
fetchTokenURL := &url.URL{}
*fetchTokenURL = *originURL
cfdToken, err := token.FetchTokenWithRedirect(fetchTokenURL, logger)
cfdToken, err := token.FetchTokenWithRedirect(fetchTokenURL, log)
if err != nil {
return err
}
@@ -364,15 +354,15 @@ func sshGen(c *cli.Context) error {
}
// getAppURL will pull the appURL needed for fetching a user's Access token
func getAppURL(cmdArgs []string, logger logger.Service) (*url.URL, error) {
func getAppURL(cmdArgs []string, log *zerolog.Logger) (*url.URL, error) {
if len(cmdArgs) < 1 {
logger.Error("Please provide a valid URL as the first argument to curl.")
log.Error().Msg("Please provide a valid URL as the first argument to curl.")
return nil, errors.New("not a valid url")
}
u, err := processURL(cmdArgs[0])
if err != nil {
logger.Error("Please provide a valid URL as the first argument to curl.")
log.Error().Msg("Please provide a valid URL as the first argument to curl.")
return nil, err
}
@@ -438,7 +428,7 @@ func isFileThere(candidate string) bool {
// verifyTokenAtEdge checks for a token on disk, or generates a new one.
// Then makes a request to to the origin with the token to ensure it is valid.
// Returns nil if token is valid.
func verifyTokenAtEdge(appUrl *url.URL, c *cli.Context, logger logger.Service) error {
func verifyTokenAtEdge(appUrl *url.URL, c *cli.Context, log *zerolog.Logger) error {
headers := buildRequestHeaders(c.StringSlice(sshHeaderFlag))
if c.IsSet(sshTokenIDFlag) {
headers.Add(h2mux.CFAccessClientIDHeader, c.String(sshTokenIDFlag))
@@ -448,7 +438,7 @@ func verifyTokenAtEdge(appUrl *url.URL, c *cli.Context, logger logger.Service) e
}
options := &carrier.StartOptions{OriginURL: appUrl.String(), Headers: headers}
if valid, err := isTokenValid(options, logger); err != nil {
if valid, err := isTokenValid(options, log); err != nil {
return err
} else if valid {
return nil
@@ -458,7 +448,7 @@ func verifyTokenAtEdge(appUrl *url.URL, c *cli.Context, logger logger.Service) e
return err
}
if valid, err := isTokenValid(options, logger); err != nil {
if valid, err := isTokenValid(options, log); err != nil {
return err
} else if !valid {
return errors.New("failed to verify token")
@@ -468,8 +458,8 @@ func verifyTokenAtEdge(appUrl *url.URL, c *cli.Context, logger logger.Service) e
}
// isTokenValid makes a request to the origin and returns true if the response was not a 302.
func isTokenValid(options *carrier.StartOptions, logger logger.Service) (bool, error) {
req, err := carrier.BuildAccessRequest(options, logger)
func isTokenValid(options *carrier.StartOptions, log *zerolog.Logger) (bool, error) {
req, err := carrier.BuildAccessRequest(options, log)
if err != nil {
return false, errors.Wrap(err, "Could not create access request")
}
+6 -5
View File
@@ -3,7 +3,8 @@ package main
import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/access"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/logger"
"github.com/rs/zerolog"
)
// ForwardServiceType is used to identify what kind of overwatch service this is
@@ -15,12 +16,12 @@ const ForwardServiceType = "forward"
type ForwarderService struct {
forwarder config.Forwarder
shutdown chan struct{}
logger logger.Service
log *zerolog.Logger
}
// NewForwardService creates a new forwarder service
func NewForwardService(f config.Forwarder, logger logger.Service) *ForwarderService {
return &ForwarderService{forwarder: f, shutdown: make(chan struct{}, 1), logger: logger}
func NewForwardService(f config.Forwarder, log *zerolog.Logger) *ForwarderService {
return &ForwarderService{forwarder: f, shutdown: make(chan struct{}, 1), log: log}
}
// Name is used to figure out this service is related to the others (normally the addr it binds to)
@@ -46,5 +47,5 @@ func (s *ForwarderService) Shutdown() {
// Run is the run loop that is started by the overwatch service
func (s *ForwarderService) Run() error {
return access.StartForwarder(s.forwarder, s.shutdown, s.logger)
return access.StartForwarder(s.forwarder, s.shutdown, s.log)
}
+22 -10
View File
@@ -2,12 +2,18 @@ package main
import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/tunneldns"
"github.com/rs/zerolog"
)
// ResolverServiceType is used to identify what kind of overwatch service this is
const ResolverServiceType = "resolver"
const (
// ResolverServiceType is used to identify what kind of overwatch service this is
ResolverServiceType = "resolver"
LogFieldResolverAddress = "resolverAddress"
LogFieldResolverPort = "resolverPort"
)
// ResolverService is used to wrap the tunneldns package's DNS over HTTP
// into a service model for the overwatch package.
@@ -15,14 +21,14 @@ const ResolverServiceType = "resolver"
type ResolverService struct {
resolver config.DNSResolver
shutdown chan struct{}
logger logger.Service
log *zerolog.Logger
}
// NewResolverService creates a new resolver service
func NewResolverService(r config.DNSResolver, logger logger.Service) *ResolverService {
func NewResolverService(r config.DNSResolver, log *zerolog.Logger) *ResolverService {
return &ResolverService{resolver: r,
shutdown: make(chan struct{}),
logger: logger,
log: log,
}
}
@@ -51,7 +57,7 @@ func (s *ResolverService) Shutdown() {
func (s *ResolverService) Run() error {
// create a listener
l, err := tunneldns.CreateListener(s.resolver.AddressOrDefault(), s.resolver.PortOrDefault(),
s.resolver.UpstreamsOrDefault(), s.resolver.BootstrapsOrDefault(), s.logger)
s.resolver.UpstreamsOrDefault(), s.resolver.BootstrapsOrDefault(), s.log)
if err != nil {
return err
}
@@ -60,14 +66,20 @@ func (s *ResolverService) Run() error {
readySignal := make(chan struct{})
err = l.Start(readySignal)
if err != nil {
l.Stop()
_ = l.Stop()
return err
}
<-readySignal
s.logger.Infof("start resolver on: %s:%d", s.resolver.AddressOrDefault(), s.resolver.PortOrDefault())
resolverLog := s.log.With().
Str(LogFieldResolverAddress, s.resolver.AddressOrDefault()).
Uint16(LogFieldResolverPort, s.resolver.PortOrDefault()).
Logger()
resolverLog.Info().Msg("Starting resolver")
// wait for shutdown signal
<-s.shutdown
s.logger.Infof("shutdown on: %s:%d", s.resolver.AddressOrDefault(), s.resolver.PortOrDefault())
resolverLog.Info().Msg("Shutting down resolver")
return l.Stop()
}
+8 -6
View File
@@ -2,8 +2,9 @@ package main
import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/overwatch"
"github.com/rs/zerolog"
)
// AppService is the main service that runs when no command lines flags are passed to cloudflared
@@ -13,17 +14,17 @@ type AppService struct {
serviceManager overwatch.Manager
shutdownC chan struct{}
configUpdateChan chan config.Root
logger logger.Service
log *zerolog.Logger
}
// NewAppService creates a new AppService with needed supporting services
func NewAppService(configManager config.Manager, serviceManager overwatch.Manager, shutdownC chan struct{}, logger logger.Service) *AppService {
func NewAppService(configManager config.Manager, serviceManager overwatch.Manager, shutdownC chan struct{}, log *zerolog.Logger) *AppService {
return &AppService{
configManager: configManager,
serviceManager: serviceManager,
shutdownC: shutdownC,
configUpdateChan: make(chan config.Root),
logger: logger,
log: log,
}
}
@@ -37,6 +38,7 @@ func (s *AppService) Run() error {
func (s *AppService) Shutdown() error {
s.configManager.Shutdown()
s.shutdownC <- struct{}{}
return nil
}
@@ -66,14 +68,14 @@ func (s *AppService) handleConfigUpdate(c config.Root) {
// handle the client forward listeners
activeServices := map[string]struct{}{}
for _, f := range c.Forwarders {
service := NewForwardService(f, s.logger)
service := NewForwardService(f, s.log)
s.serviceManager.Add(service)
activeServices[service.Name()] = struct{}{}
}
// handle resolver changes
if c.Resolver.Enabled {
service := NewResolverService(c.Resolver, s.logger)
service := NewResolverService(c.Resolver, s.log)
s.serviceManager.Add(service)
activeServices[service.Name()] = struct{}{}
+4 -5
View File
@@ -1,9 +1,8 @@
package buildinfo
import (
"github.com/rs/zerolog"
"runtime"
"github.com/cloudflare/cloudflared/logger"
)
type BuildInfo struct {
@@ -22,7 +21,7 @@ func GetBuildInfo(cloudflaredVersion string) *BuildInfo {
}
}
func (bi *BuildInfo) Log(logger logger.Service) {
logger.Infof("Version %s", bi.CloudflaredVersion)
logger.Infof("GOOS: %s, GOVersion: %s, GoArch: %s", bi.GoOS, bi.GoVersion, bi.GoArch)
func (bi *BuildInfo) Log(log *zerolog.Logger) {
log.Info().Msgf("Version %s", bi.CloudflaredVersion)
log.Info().Msgf("GOOS: %s, GOVersion: %s, GoArch: %s", bi.GoOS, bi.GoVersion, bi.GoArch)
}
-18
View File
@@ -2,10 +2,6 @@ package cliutil
import (
"fmt"
"log"
"github.com/cloudflare/cloudflared/logger"
"github.com/pkg/errors"
"github.com/urfave/cli/v2"
)
@@ -27,8 +23,6 @@ func UsageError(format string, args ...interface{}) error {
// Ensures exit with error code if actionFunc returns an error
func ErrorHandler(actionFunc cli.ActionFunc) cli.ActionFunc {
return func(ctx *cli.Context) error {
defer logger.SharedWriteManager.Shutdown()
err := actionFunc(ctx)
if err != nil {
if _, ok := err.(usageError); ok {
@@ -41,15 +35,3 @@ func ErrorHandler(actionFunc cli.ActionFunc) cli.ActionFunc {
return err
}
}
// PrintLoggerSetupError returns an error to stdout to notify when a logger can't start
func PrintLoggerSetupError(msg string, err error) error {
l, le := logger.New()
if le != nil {
log.Printf("%s: %s", msg, err)
} else {
l.Errorf("%s: %s", msg, err)
}
return errors.Wrap(err, msg)
}
+11 -35
View File
@@ -2,19 +2,20 @@ package config
import (
"fmt"
"io"
"net/url"
"os"
"path/filepath"
"runtime"
"time"
homedir "github.com/mitchellh/go-homedir"
"github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
"github.com/urfave/cli/v2"
"gopkg.in/yaml.v2"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/validation"
"github.com/rs/zerolog"
)
var (
@@ -94,7 +95,7 @@ func FileExists(path string) (bool, error) {
}
return false, err
}
f.Close()
_ = f.Close()
return true, nil
}
@@ -137,7 +138,7 @@ func FindOrCreateConfigPath() string {
defer file.Close()
logDir := DefaultLogDirectory()
os.MkdirAll(logDir, os.ModePerm) //try and create it. Doesn't matter if it succeed or not, only byproduct will be no logs
_ = os.MkdirAll(logDir, os.ModePerm) //try and create it. Doesn't matter if it succeed or not, only byproduct will be no logs
c := Root{
LogDirectory: logDir,
@@ -150,35 +151,6 @@ func FindOrCreateConfigPath() string {
return path
}
// FindLogSettings gets the log directory and level from the config file
func FindLogSettings() (string, string) {
configPath := FindOrCreateConfigPath()
defaultDirectory := DefaultLogDirectory()
defaultLevel := "info"
file, err := os.Open(configPath)
if err != nil {
return defaultDirectory, defaultLevel
}
defer file.Close()
var config Root
if err := yaml.NewDecoder(file).Decode(&config); err != nil {
return defaultDirectory, defaultLevel
}
directory := defaultDirectory
if config.LogDirectory != "" {
directory = config.LogDirectory
}
level := defaultLevel
if config.LogLevel != "" {
level = config.LogLevel
}
return directory, level
}
// ValidateUnixSocket ensures --unix-socket param is used exclusively
// i.e. it fails if a user specifies both --url and --unix-socket
func ValidateUnixSocket(c *cli.Context) (string, error) {
@@ -373,7 +345,7 @@ func GetConfiguration() *Configuration {
// ReadConfigFile returns InputSourceContext initialized from the configuration file.
// On repeat calls returns with the same file, returns without reading the file again; however,
// if value of "config" flag changes, will read the new config file
func ReadConfigFile(c *cli.Context, log logger.Service) (*configFileSettings, error) {
func ReadConfigFile(c *cli.Context, log *zerolog.Logger) (*configFileSettings, error) {
configFile := c.String("config")
if configuration.Source() == configFile || configFile == "" {
if configuration.Source() == "" {
@@ -382,7 +354,7 @@ func ReadConfigFile(c *cli.Context, log logger.Service) (*configFileSettings, er
return &configuration, nil
}
log.Debugf("Loading configuration from %s", configFile)
log.Debug().Msgf("Loading configuration from %s", configFile)
file, err := os.Open(configFile)
if err != nil {
if os.IsNotExist(err) {
@@ -392,6 +364,10 @@ func ReadConfigFile(c *cli.Context, log logger.Service) (*configFileSettings, er
}
defer file.Close()
if err := yaml.NewDecoder(file).Decode(&configuration); err != nil {
if err == io.EOF {
log.Error().Msgf("Configuration file %s was empty", configFile)
return &configuration, nil
}
return nil, errors.Wrap(err, "error parsing YAML in config file at "+configFile)
}
configuration.sourceFile = configFile
+16 -10
View File
@@ -1,11 +1,13 @@
package config
import (
"io"
"os"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/watcher"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"gopkg.in/yaml.v2"
)
@@ -26,16 +28,16 @@ type FileManager struct {
watcher watcher.Notifier
notifier Notifier
configPath string
logger logger.Service
ReadConfig func(string) (Root, error)
log *zerolog.Logger
ReadConfig func(string, *zerolog.Logger) (Root, error)
}
// NewFileManager creates a config manager
func NewFileManager(watcher watcher.Notifier, configPath string, logger logger.Service) (*FileManager, error) {
func NewFileManager(watcher watcher.Notifier, configPath string, log *zerolog.Logger) (*FileManager, error) {
m := &FileManager{
watcher: watcher,
configPath: configPath,
logger: logger,
log: log,
ReadConfig: readConfigFromPath,
}
err := watcher.Add(configPath)
@@ -59,7 +61,7 @@ func (m *FileManager) Start(notifier Notifier) error {
// GetConfig reads the yaml file from the disk
func (m *FileManager) GetConfig() (Root, error) {
return m.ReadConfig(m.configPath)
return m.ReadConfig(m.configPath, m.log)
}
// Shutdown stops the watcher
@@ -67,7 +69,7 @@ func (m *FileManager) Shutdown() {
m.watcher.Shutdown()
}
func readConfigFromPath(configPath string) (Root, error) {
func readConfigFromPath(configPath string, log *zerolog.Logger) (Root, error) {
if configPath == "" {
return Root{}, errors.New("unable to find config file")
}
@@ -80,6 +82,10 @@ func readConfigFromPath(configPath string) (Root, error) {
var config Root
if err := yaml.NewDecoder(file).Decode(&config); err != nil {
if err == io.EOF {
log.Error().Msgf("Configuration file %s was empty", configPath)
return Root{}, nil
}
return Root{}, errors.Wrap(err, "error parsing YAML in config file at "+configPath)
}
@@ -93,14 +99,14 @@ func readConfigFromPath(configPath string) (Root, error) {
func (m *FileManager) WatcherItemDidChange(filepath string) {
config, err := m.GetConfig()
if err != nil {
m.logger.Errorf("Failed to read new config: %s", err)
m.log.Err(err).Msg("Failed to read new config")
return
}
m.logger.Info("Config file has been updated")
m.log.Info().Msg("Config file has been updated")
m.notifier.ConfigDidUpdate(config)
}
// WatcherDidError notifies of errors with the file watcher
func (m *FileManager) WatcherDidError(err error) {
m.logger.Errorf("Config watcher encountered an error: %s", err)
m.log.Err(err).Msg("Config watcher encountered an error")
}
+8 -8
View File
@@ -4,10 +4,10 @@ import (
"os"
"testing"
"github.com/stretchr/testify/assert"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/watcher"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
)
type mockNotifier struct {
@@ -46,8 +46,8 @@ func TestConfigChanged(t *testing.T) {
f, err := os.Create(filePath)
assert.NoError(t, err)
defer func() {
f.Close()
os.Remove(filePath)
_ = f.Close()
_ = os.Remove(filePath)
}()
c := &Root{
Forwarders: []Forwarder{
@@ -57,15 +57,15 @@ func TestConfigChanged(t *testing.T) {
},
},
}
configRead := func(configPath string) (Root, error) {
configRead := func(configPath string, log *zerolog.Logger) (Root, error) {
return *c, nil
}
wait := make(chan struct{})
w := &mockFileWatcher{path: filePath, ready: wait}
logger := logger.NewOutputWriter(logger.NewMockWriteManager())
log := zerolog.Nop()
service, err := NewFileManager(w, filePath, logger)
service, err := NewFileManager(w, filePath, &log)
service.ReadConfig = configRead
assert.NoError(t, err)
+38 -43
View File
@@ -7,12 +7,13 @@ import (
"os"
"path/filepath"
"github.com/pkg/errors"
cli "github.com/urfave/cli/v2"
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/cmd/cloudflared/tunnel"
"github.com/cloudflare/cloudflared/logger"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
)
func runApp(app *cli.App, shutdownC, graceShutdownC chan struct{}) {
@@ -20,7 +21,7 @@ func runApp(app *cli.App, shutdownC, graceShutdownC chan struct{}) {
Name: "service",
Usage: "Manages the Argo Tunnel system service",
Subcommands: []*cli.Command{
&cli.Command{
{
Name: "install",
Usage: "Install Argo Tunnel as a system service",
Action: cliutil.ErrorHandler(installLinuxService),
@@ -31,7 +32,7 @@ func runApp(app *cli.App, shutdownC, graceShutdownC chan struct{}) {
},
},
},
&cli.Command{
{
Name: "uninstall",
Usage: "Uninstall the Argo Tunnel service",
Action: cliutil.ErrorHandler(uninstallLinuxService),
@@ -189,7 +190,7 @@ func isSystemd() bool {
return false
}
func copyUserConfiguration(userConfigDir, userConfigFile, userCredentialFile string, logger logger.Service) error {
func copyUserConfiguration(userConfigDir, userConfigFile, userCredentialFile string, log *zerolog.Logger) error {
srcCredentialPath := filepath.Join(userConfigDir, userCredentialFile)
destCredentialPath := filepath.Join(serviceConfigDir, serviceCredentialFile)
if srcCredentialPath != destCredentialPath {
@@ -204,17 +205,14 @@ func copyUserConfiguration(userConfigDir, userConfigFile, userCredentialFile str
if err := copyConfig(srcConfigPath, destConfigPath); err != nil {
return err
}
logger.Infof("Copied %s to %s", srcConfigPath, destConfigPath)
log.Info().Msgf("Copied %s to %s", srcConfigPath, destConfigPath)
}
return nil
}
func installLinuxService(c *cli.Context) error {
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
etPath, err := os.Executable()
if err != nil {
@@ -231,8 +229,8 @@ func installLinuxService(c *cli.Context) error {
userConfigDir := filepath.Dir(c.String("config"))
userConfigFile := filepath.Base(c.String("config"))
userCredentialFile := config.DefaultCredentialFile
if err = copyUserConfiguration(userConfigDir, userConfigFile, userCredentialFile, logger); err != nil {
logger.Errorf("Failed to copy user configuration: %s. Before running the service, ensure that %s contains two files, %s and %s", err,
if err = copyUserConfiguration(userConfigDir, userConfigFile, userCredentialFile, log); err != nil {
log.Err(err).Msgf("Failed to copy user configuration. Before running the service, ensure that %s contains two files, %s and %s",
serviceConfigDir, serviceCredentialFile, serviceConfigFile)
return err
}
@@ -240,7 +238,7 @@ func installLinuxService(c *cli.Context) error {
"--origincert", serviceConfigDir + "/" + serviceCredentialFile,
}
} else {
src, err := config.ReadConfigFile(c, logger)
src, err := config.ReadConfigFile(c, log)
if err != nil {
return err
}
@@ -250,7 +248,7 @@ func installLinuxService(c *cli.Context) error {
val, err := src.String(s)
return err == nil && val != ""
}
if src.TunnelID == "" || !configPresent("credentials-file") {
if src.TunnelID == "" || !configPresent(tunnel.CredFileFlag) {
return fmt.Errorf(`Configuration file %s must contain entries for the tunnel to run and its associated credentials:
tunnel: TUNNEL-UUID
credentials-file: CREDENTIALS-FILE
@@ -273,42 +271,42 @@ credentials-file: CREDENTIALS-FILE
switch {
case isSystemd():
logger.Infof("Using Systemd")
return installSystemd(&templateArgs, logger)
log.Info().Msgf("Using Systemd")
return installSystemd(&templateArgs, log)
default:
logger.Infof("Using SysV")
return installSysv(&templateArgs, logger)
log.Info().Msgf("Using SysV")
return installSysv(&templateArgs, log)
}
}
func installSystemd(templateArgs *ServiceTemplateArgs, logger logger.Service) error {
func installSystemd(templateArgs *ServiceTemplateArgs, log *zerolog.Logger) error {
for _, serviceTemplate := range systemdTemplates {
err := serviceTemplate.Generate(templateArgs)
if err != nil {
logger.Errorf("error generating service template: %s", err)
log.Err(err).Msg("error generating service template")
return err
}
}
if err := runCommand("systemctl", "enable", "cloudflared.service"); err != nil {
logger.Errorf("systemctl enable cloudflared.service error: %s", err)
log.Err(err).Msg("systemctl enable cloudflared.service error")
return err
}
if err := runCommand("systemctl", "start", "cloudflared-update.timer"); err != nil {
logger.Errorf("systemctl start cloudflared-update.timer error: %s", err)
log.Err(err).Msg("systemctl start cloudflared-update.timer error")
return err
}
logger.Infof("systemctl daemon-reload")
log.Info().Msg("systemctl daemon-reload")
return runCommand("systemctl", "daemon-reload")
}
func installSysv(templateArgs *ServiceTemplateArgs, logger logger.Service) error {
func installSysv(templateArgs *ServiceTemplateArgs, log *zerolog.Logger) error {
confPath, err := sysvTemplate.ResolvePath()
if err != nil {
logger.Errorf("error resolving system path: %s", err)
log.Err(err).Msg("error resolving system path")
return err
}
if err := sysvTemplate.Generate(templateArgs); err != nil {
logger.Errorf("error generating system template: %s", err)
log.Err(err).Msg("error generating system template")
return err
}
for _, i := range [...]string{"2", "3", "4", "5"} {
@@ -325,43 +323,40 @@ func installSysv(templateArgs *ServiceTemplateArgs, logger logger.Service) error
}
func uninstallLinuxService(c *cli.Context) error {
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
switch {
case isSystemd():
logger.Infof("Using Systemd")
return uninstallSystemd(logger)
log.Info().Msg("Using Systemd")
return uninstallSystemd(log)
default:
logger.Infof("Using SysV")
return uninstallSysv(logger)
log.Info().Msg("Using SysV")
return uninstallSysv(log)
}
}
func uninstallSystemd(logger logger.Service) error {
func uninstallSystemd(log *zerolog.Logger) error {
if err := runCommand("systemctl", "disable", "cloudflared.service"); err != nil {
logger.Errorf("systemctl disable cloudflared.service error: %s", err)
log.Err(err).Msg("systemctl disable cloudflared.service error")
return err
}
if err := runCommand("systemctl", "stop", "cloudflared-update.timer"); err != nil {
logger.Errorf("systemctl stop cloudflared-update.timer error: %s", err)
log.Err(err).Msg("systemctl stop cloudflared-update.timer error")
return err
}
for _, serviceTemplate := range systemdTemplates {
if err := serviceTemplate.Remove(); err != nil {
logger.Errorf("error removing service template: %s", err)
log.Err(err).Msg("error removing service template")
return err
}
}
logger.Infof("Successfully uninstall cloudflared service")
log.Info().Msgf("Successfully uninstalled cloudflared service from systemd")
return nil
}
func uninstallSysv(logger logger.Service) error {
func uninstallSysv(log *zerolog.Logger) error {
if err := sysvTemplate.Remove(); err != nil {
logger.Errorf("error removing service template: %s", err)
log.Err(err).Msg("error removing service template")
return err
}
for _, i := range [...]string{"2", "3", "4", "5"} {
@@ -374,6 +369,6 @@ func uninstallSysv(logger logger.Service) error {
continue
}
}
logger.Infof("Successfully uninstall cloudflared service")
log.Info().Msgf("Successfully uninstalled cloudflared service from sysv")
return nil
}
+19 -32
View File
@@ -6,11 +6,11 @@ import (
"fmt"
"os"
"github.com/urfave/cli/v2"
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/logger"
"github.com/pkg/errors"
"github.com/urfave/cli/v2"
)
const (
@@ -34,7 +34,7 @@ func runApp(app *cli.App, shutdownC, graceShutdownC chan struct{}) {
},
},
})
app.Run(os.Args)
_ = app.Run(os.Args)
}
func newLaunchdTemplate(installPath, stdoutPath, stderrPath string) *ServiceTemplate {
@@ -107,71 +107,61 @@ func stderrPath() (string, error) {
}
func installLaunchd(c *cli.Context) error {
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
if isRootUser() {
logger.Infof("Installing Argo Tunnel client as a system launch daemon. " +
log.Info().Msg("Installing Argo Tunnel client as a system launch daemon. " +
"Argo Tunnel client will run at boot")
} else {
logger.Infof("Installing Argo Tunnel client as an user launch agent. " +
log.Info().Msg("Installing Argo Tunnel client as an user launch agent. " +
"Note that Argo Tunnel client will only run when the user is logged in. " +
"If you want to run Argo Tunnel client at boot, install with root permission. " +
"For more information, visit https://developers.cloudflare.com/argo-tunnel/reference/service/")
}
etPath, err := os.Executable()
if err != nil {
logger.Errorf("Error determining executable path: %s", err)
log.Err(err).Msg("Error determining executable path")
return fmt.Errorf("Error determining executable path: %v", err)
}
installPath, err := installPath()
if err != nil {
logger.Errorf("Error determining install path: %s", err)
log.Err(err).Msg("Error determining install path")
return errors.Wrap(err, "Error determining install path")
}
stdoutPath, err := stdoutPath()
if err != nil {
logger.Errorf("error determining stdout path: %s", err)
log.Err(err).Msg("error determining stdout path")
return errors.Wrap(err, "error determining stdout path")
}
stderrPath, err := stderrPath()
if err != nil {
logger.Errorf("error determining stderr path: %s", err)
log.Err(err).Msg("error determining stderr path")
return errors.Wrap(err, "error determining stderr path")
}
launchdTemplate := newLaunchdTemplate(installPath, stdoutPath, stderrPath)
if err != nil {
logger.Errorf("error creating launchd template: %s", err)
return errors.Wrap(err, "error creating launchd template")
}
templateArgs := ServiceTemplateArgs{Path: etPath}
err = launchdTemplate.Generate(&templateArgs)
if err != nil {
logger.Errorf("error generating launchd template: %s", err)
log.Err(err).Msg("error generating launchd template")
return err
}
plistPath, err := launchdTemplate.ResolvePath()
if err != nil {
logger.Errorf("error resolving launchd template path: %s", err)
log.Err(err).Msg("error resolving launchd template path")
return err
}
logger.Infof("Outputs are logged to %s and %s", stderrPath, stdoutPath)
log.Info().Msgf("Outputs are logged to %s and %s", stderrPath, stdoutPath)
return runCommand("launchctl", "load", plistPath)
}
func uninstallLaunchd(c *cli.Context) error {
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
if isRootUser() {
logger.Infof("Uninstalling Argo Tunnel as a system launch daemon")
log.Info().Msg("Uninstalling Argo Tunnel as a system launch daemon")
} else {
logger.Infof("Uninstalling Argo Tunnel as an user launch agent")
log.Info().Msg("Uninstalling Argo Tunnel as an user launch agent")
}
installPath, err := installPath()
if err != nil {
@@ -186,20 +176,17 @@ func uninstallLaunchd(c *cli.Context) error {
return errors.Wrap(err, "error determining stderr path")
}
launchdTemplate := newLaunchdTemplate(installPath, stdoutPath, stderrPath)
if err != nil {
return errors.Wrap(err, "error creating launchd template")
}
plistPath, err := launchdTemplate.ResolvePath()
if err != nil {
logger.Errorf("error resolving launchd template path: %s", err)
log.Err(err).Msg("error resolving launchd template path")
return err
}
err = runCommand("launchctl", "unload", plistPath)
if err != nil {
logger.Errorf("error unloading: %s", err)
log.Err(err).Msg("error unloading")
return err
}
logger.Infof("Outputs are logged to %s and %s", stderrPath, stdoutPath)
log.Info().Msgf("Outputs are logged to %s and %s", stderrPath, stdoutPath)
return launchdTemplate.Remove()
}
+14 -22
View File
@@ -10,17 +10,16 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/cmd/cloudflared/tunnel"
"github.com/cloudflare/cloudflared/cmd/cloudflared/updater"
log "github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/metrics"
"github.com/cloudflare/cloudflared/overwatch"
"github.com/cloudflare/cloudflared/tunneldns"
"github.com/cloudflare/cloudflared/watcher"
raven "github.com/getsentry/raven-go"
homedir "github.com/mitchellh/go-homedir"
cli "github.com/urfave/cli/v2"
"github.com/getsentry/raven-go"
"github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
"github.com/urfave/cli/v2"
)
const (
@@ -147,7 +146,7 @@ func isEmptyInvocation(c *cli.Context) bool {
func action(version string, shutdownC, graceShutdownC chan struct{}) cli.ActionFunc {
return cliutil.ErrorHandler(func(c *cli.Context) (err error) {
if isEmptyInvocation(c) {
return handleServiceMode(shutdownC)
return handleServiceMode(c, shutdownC)
}
tags := make(map[string]string)
tags["hostname"] = c.String("hostname")
@@ -184,41 +183,34 @@ func captureError(err error) {
}
// cloudflared was started without any flags
func handleServiceMode(shutdownC chan struct{}) error {
defer log.SharedWriteManager.Shutdown()
logDirectory, logLevel := config.FindLogSettings()
logger, err := log.New(log.DefaultFile(logDirectory), log.LogLevelString(logLevel))
if err != nil {
return cliutil.PrintLoggerSetupError("error setting up logger", err)
}
logger.Infof("logging to directory: %s", logDirectory)
func handleServiceMode(c *cli.Context, shutdownC chan struct{}) error {
log := logger.CreateLoggerFromContext(c, logger.DisableTerminalLog)
// start the main run loop that reads from the config file
f, err := watcher.NewFile()
if err != nil {
logger.Errorf("Cannot load config file: %s", err)
log.Err(err).Msg("Cannot load config file")
return err
}
configPath := config.FindOrCreateConfigPath()
configManager, err := config.NewFileManager(f, configPath, logger)
configManager, err := config.NewFileManager(f, configPath, log)
if err != nil {
logger.Errorf("Cannot setup config file for monitoring: %s", err)
log.Err(err).Msg("Cannot setup config file for monitoring")
return err
}
logger.Infof("monitoring config file at: %s", configPath)
log.Info().Msgf("monitoring config file at: %s", configPath)
serviceCallback := func(t string, name string, err error) {
if err != nil {
logger.Errorf("%s service: %s encountered an error: %s", t, name, err)
log.Err(err).Msgf("%s service: %s encountered an error", t, name)
}
}
serviceManager := overwatch.NewAppManager(serviceCallback)
appService := NewAppService(configManager, serviceManager, shutdownC, logger)
appService := NewAppService(configManager, serviceManager, shutdownC, log)
if err := appService.Run(); err != nil {
logger.Errorf("Failed to start app service: %s", err)
log.Err(err).Msg("Failed to start app service")
return err
}
return nil
+22 -7
View File
@@ -11,8 +11,27 @@ import (
"github.com/mitchellh/go-homedir"
)
// GenerateFilePathFromURL will return a filepath for given access application url
func GenerateFilePathFromURL(url *url.URL, suffix string) (string, error) {
// GenerateAppTokenFilePathFromURL will return a filepath for given Access org token
func GenerateAppTokenFilePathFromURL(url *url.URL, suffix string) (string, error) {
configPath, err := getConfigPath()
if err != nil {
return "", err
}
name := strings.Replace(fmt.Sprintf("%s%s-%s", url.Hostname(), url.EscapedPath(), suffix), "/", "-", -1)
return filepath.Join(configPath, name), nil
}
// GenerateOrgTokenFilePathFromURL will return a filepath for given Access application token
func GenerateOrgTokenFilePathFromURL(authDomain string) (string, error) {
configPath, err := getConfigPath()
if err != nil {
return "", err
}
name := strings.Replace(fmt.Sprintf("%s-org-token", authDomain), "/", "-", -1)
return filepath.Join(configPath, name), nil
}
func getConfigPath() (string, error) {
configPath, err := homedir.Expand(config.DefaultConfigSearchDirectories()[0])
if err != nil {
return "", err
@@ -22,9 +41,5 @@ func GenerateFilePathFromURL(url *url.URL, suffix string) (string, error) {
// create config directory if doesn't already exist
err = os.Mkdir(configPath, 0700)
}
if err != nil {
return "", err
}
name := strings.Replace(fmt.Sprintf("%s%s-%s", url.Hostname(), url.EscapedPath(), suffix), "/", "-", -1)
return filepath.Join(configPath, name), nil
return configPath, err
}
+208 -39
View File
@@ -5,22 +5,27 @@ import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"os"
"os/signal"
"strings"
"syscall"
"time"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/cmd/cloudflared/path"
"github.com/cloudflare/cloudflared/cmd/cloudflared/transfer"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/origin"
"github.com/coreos/go-oidc/jose"
"github.com/pkg/errors"
"github.com/rs/zerolog"
)
const (
keyName = "token"
keyName = "token"
tokenHeader = "CF_Authorization"
)
type lock struct {
@@ -34,7 +39,7 @@ type signalHandler struct {
signals []os.Signal
}
type jwtPayload struct {
type appJWTPayload struct {
Aud []string `json:"aud"`
Email string `json:"email"`
Exp int `json:"exp"`
@@ -45,7 +50,17 @@ type jwtPayload struct {
Subt string `json:"sub"`
}
func (p jwtPayload) isExpired() bool {
type orgJWTPayload struct {
appJWTPayload
Aud string `json:"aud"`
}
type transferServiceResponse struct {
AppToken string `json:"app_token"`
OrgToken string `json:"org_token"`
}
func (p appJWTPayload) isExpired() bool {
return int(time.Now().Unix()) > p.Exp
}
@@ -83,7 +98,7 @@ func newLock(path string) *lock {
func (l *lock) Acquire() error {
// Intercept SIGINT and SIGTERM to release lock before exiting
l.sigHandler.register(func() {
l.deleteLockFile()
_ = l.deleteLockFile()
os.Exit(0)
})
@@ -129,67 +144,185 @@ func isTokenLocked(lockFilePath string) bool {
// FetchTokenWithRedirect will either load a stored token or generate a new one
// it appends the full url as the redirect URL to the access cli request if opening the browser
func FetchTokenWithRedirect(appURL *url.URL, logger logger.Service) (string, error) {
return getToken(appURL, false, logger)
func FetchTokenWithRedirect(appURL *url.URL, log *zerolog.Logger) (string, error) {
return getToken(appURL, false, log)
}
// FetchToken will either load a stored token or generate a new one
// it appends the host of the appURL as the redirect URL to the access cli request if opening the browser
func FetchToken(appURL *url.URL, logger logger.Service) (string, error) {
return getToken(appURL, true, logger)
func FetchToken(appURL *url.URL, log *zerolog.Logger) (string, error) {
return getToken(appURL, true, log)
}
// getToken will either load a stored token or generate a new one
func getToken(appURL *url.URL, useHostOnly bool, logger logger.Service) (string, error) {
if token, err := GetTokenIfExists(appURL); token != "" && err == nil {
func getToken(appURL *url.URL, useHostOnly bool, log *zerolog.Logger) (string, error) {
if token, err := GetAppTokenIfExists(appURL); token != "" && err == nil {
return token, nil
}
path, err := path.GenerateFilePathFromURL(appURL, keyName)
appTokenPath, err := path.GenerateAppTokenFilePathFromURL(appURL, keyName)
if err != nil {
return "", err
return "", errors.Wrap(err, "failed to generate app token file path")
}
fileLock := newLock(path)
err = fileLock.Acquire()
if err != nil {
return "", err
fileLockAppToken := newLock(appTokenPath)
if err = fileLockAppToken.Acquire(); err != nil {
return "", errors.Wrap(err, "failed to acquire app token lock")
}
defer fileLock.Release()
defer fileLockAppToken.Release()
// check to see if another process has gotten a token while we waited for the lock
if token, err := GetTokenIfExists(appURL); token != "" && err == nil {
if token, err := GetAppTokenIfExists(appURL); token != "" && err == nil {
return token, nil
}
// If an app token couldnt be found on disk, check for an org token and attempt to exchange it for an app token.
var orgTokenPath string
// Get auth domain to format into org token file path
if authDomain, err := getAuthDomain(appURL); err != nil {
log.Error().Msgf("failed to get auth domain: %s", err)
} else {
orgToken, err := GetOrgTokenIfExists(authDomain)
if err != nil {
orgTokenPath, err = path.GenerateOrgTokenFilePathFromURL(authDomain)
if err != nil {
return "", errors.Wrap(err, "failed to generate org token file path")
}
fileLockOrgToken := newLock(orgTokenPath)
if err = fileLockOrgToken.Acquire(); err != nil {
return "", errors.Wrap(err, "failed to acquire org token lock")
}
defer fileLockOrgToken.Release()
// check if an org token has been created since the lock was acquired
orgToken, err = GetOrgTokenIfExists(authDomain)
}
if err == nil {
if appToken, err := exchangeOrgToken(appURL, orgToken); err != nil {
log.Debug().Msgf("failed to exchange org token for app token: %s", err)
} else {
if err := ioutil.WriteFile(appTokenPath, []byte(appToken), 0600); err != nil {
return "", errors.Wrap(err, "failed to write app token to disk")
}
return appToken, nil
}
}
}
return getTokensFromEdge(appURL, appTokenPath, orgTokenPath, useHostOnly, log)
}
// getTokensFromEdge will attempt to use the transfer service to retrieve an app and org token, save them to disk,
// and return the app token.
func getTokensFromEdge(appURL *url.URL, appTokenPath, orgTokenPath string, useHostOnly bool, log *zerolog.Logger) (string, error) {
// If no org token exists or if it couldnt be exchanged for an app token, then run the transfer service flow.
// this weird parameter is the resource name (token) and the key/value
// we want to send to the transfer service. the key is token and the value
// is blank (basically just the id generated in the transfer service)
token, err := transfer.Run(appURL, keyName, keyName, "", path, true, useHostOnly, logger)
resourceData, err := transfer.Run(appURL, keyName, keyName, "", true, useHostOnly, log)
if err != nil {
return "", err
return "", errors.Wrap(err, "failed to run transfer service")
}
var resp transferServiceResponse
if err = json.Unmarshal(resourceData, &resp); err != nil {
return "", errors.Wrap(err, "failed to marshal transfer service response")
}
return string(token), nil
// If we were able to get the auth domain and generate an org token path, lets write it to disk.
if orgTokenPath != "" {
if err := ioutil.WriteFile(orgTokenPath, []byte(resp.OrgToken), 0600); err != nil {
return "", errors.Wrap(err, "failed to write org token to disk")
}
}
if err := ioutil.WriteFile(appTokenPath, []byte(resp.AppToken), 0600); err != nil {
return "", errors.Wrap(err, "failed to write app token to disk")
}
return resp.AppToken, nil
}
// GetTokenIfExists will return the token from local storage if it exists and not expired
func GetTokenIfExists(url *url.URL) (string, error) {
path, err := path.GenerateFilePathFromURL(url, keyName)
if err != nil {
return "", err
}
content, err := ioutil.ReadFile(path)
if err != nil {
return "", err
}
token, err := jose.ParseJWT(string(content))
if err != nil {
return "", err
// getAuthDomain makes a request to the appURL and stops at the first redirect. The 302 location header will contain the
// auth domain
func getAuthDomain(appURL *url.URL) (string, error) {
client := &http.Client{
// do not follow redirects
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
Timeout: time.Second * 7,
}
var payload jwtPayload
authDomainReq, err := http.NewRequest("HEAD", appURL.String(), nil)
if err != nil {
return "", errors.Wrap(err, "failed to create auth domain request")
}
resp, err := client.Do(authDomainReq)
if err != nil {
return "", errors.Wrap(err, "failed to get auth domain")
}
resp.Body.Close()
location, err := resp.Location()
if err != nil {
return "", fmt.Errorf("failed to get auth domain. Received status code %d from %s", resp.StatusCode, appURL.String())
}
return location.Hostname(), nil
}
// exchangeOrgToken attaches an org token to a request to the appURL and returns an app token. This uses the Access SSO
// flow to automatically generate and return an app token without the login page.
func exchangeOrgToken(appURL *url.URL, orgToken string) (string, error) {
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
// attach org token to login request
if strings.Contains(req.URL.Path, "cdn-cgi/access/login") {
req.AddCookie(&http.Cookie{Name: tokenHeader, Value: orgToken})
}
// stop after hitting authorized endpoint since it will contain the app token
if strings.Contains(via[len(via)-1].URL.Path, "cdn-cgi/access/authorized") {
return http.ErrUseLastResponse
}
return nil
},
Timeout: time.Second * 7,
}
appTokenRequest, err := http.NewRequest("HEAD", appURL.String(), nil)
if err != nil {
return "", errors.Wrap(err, "failed to create app token request")
}
resp, err := client.Do(appTokenRequest)
if err != nil {
return "", errors.Wrap(err, "failed to get app token")
}
resp.Body.Close()
var appToken string
for _, c := range resp.Cookies() {
if c.Name == tokenHeader {
appToken = c.Value
break
}
}
if len(appToken) > 0 {
return appToken, nil
}
return "", fmt.Errorf("response from %s did not contain app token", resp.Request.URL.String())
}
func GetOrgTokenIfExists(authDomain string) (string, error) {
path, err := path.GenerateOrgTokenFilePathFromURL(authDomain)
if err != nil {
return "", err
}
token, err := getTokenIfExists(path)
if err != nil {
return "", err
}
var payload orgJWTPayload
err = json.Unmarshal(token.Payload, &payload)
if err != nil {
return "", err
@@ -199,13 +332,49 @@ func GetTokenIfExists(url *url.URL) (string, error) {
err := os.Remove(path)
return "", err
}
return token.Encode(), nil
}
func GetAppTokenIfExists(url *url.URL) (string, error) {
path, err := path.GenerateAppTokenFilePathFromURL(url, keyName)
if err != nil {
return "", err
}
token, err := getTokenIfExists(path)
if err != nil {
return "", err
}
var payload appJWTPayload
err = json.Unmarshal(token.Payload, &payload)
if err != nil {
return "", err
}
if payload.isExpired() {
err := os.Remove(path)
return "", err
}
return token.Encode(), nil
}
// GetTokenIfExists will return the token from local storage if it exists and not expired
func getTokenIfExists(path string) (*jose.JWT, error) {
content, err := ioutil.ReadFile(path)
if err != nil {
return nil, err
}
token, err := jose.ParseJWT(string(content))
if err != nil {
return nil, err
}
return &token, nil
}
// RemoveTokenIfExists removes the a token from local storage if it exists
func RemoveTokenIfExists(url *url.URL) error {
path, err := path.GenerateFilePathFromURL(url, keyName)
path, err := path.GenerateAppTokenFilePathFromURL(url, keyName)
if err != nil {
return err
}
+12 -15
View File
@@ -3,10 +3,8 @@ package transfer
import (
"bytes"
"encoding/base64"
"errors"
"fmt"
"io"
"io/ioutil"
"net/http"
"net/url"
"os"
@@ -14,7 +12,8 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/encrypter"
"github.com/cloudflare/cloudflared/cmd/cloudflared/shell"
"github.com/cloudflare/cloudflared/logger"
"github.com/pkg/errors"
"github.com/rs/zerolog"
)
const (
@@ -28,7 +27,7 @@ const (
// The "dance" we refer to is building a HTTP request, opening that in a browser waiting for
// the user to complete an action, while it long polls in the background waiting for an
// action to be completed to download the resource.
func Run(transferURL *url.URL, resourceName, key, value, path string, shouldEncrypt bool, useHostOnly bool, logger logger.Service) ([]byte, error) {
func Run(transferURL *url.URL, resourceName, key, value string, shouldEncrypt bool, useHostOnly bool, log *zerolog.Logger) ([]byte, error) {
encrypterClient, err := encrypter.New("cloudflared_priv.pem", "cloudflared_pub.pem")
if err != nil {
return nil, err
@@ -49,7 +48,7 @@ func Run(transferURL *url.URL, resourceName, key, value, path string, shouldEncr
var resourceData []byte
if shouldEncrypt {
buf, key, err := transferRequest(baseStoreURL+"transfer/"+encrypterClient.PublicKey(), logger)
buf, key, err := transferRequest(baseStoreURL+"transfer/"+encrypterClient.PublicKey(), log)
if err != nil {
return nil, err
}
@@ -65,18 +64,15 @@ func Run(transferURL *url.URL, resourceName, key, value, path string, shouldEncr
resourceData = decrypted
} else {
buf, _, err := transferRequest(baseStoreURL+encrypterClient.PublicKey(), logger)
buf, _, err := transferRequest(baseStoreURL+encrypterClient.PublicKey(), log)
if err != nil {
return nil, err
}
resourceData = buf
}
if err := ioutil.WriteFile(path, resourceData, 0600); err != nil {
return nil, err
}
return resourceData, nil
}
// BuildRequestURL creates a request suitable for a resource transfer.
@@ -93,23 +89,24 @@ func buildRequestURL(baseURL *url.URL, key, value string, cli, useHostOnly bool)
return baseURL.String(), nil
}
q.Set("redirect_url", baseURL.String()) // we add the token as a query param on both the redirect_url and the main url
q.Set("send_org_token", "true") // indicates that the cli endpoint should return both the org and app token
baseURL.RawQuery = q.Encode() // and this actual baseURL.
baseURL.Path = "cdn-cgi/access/cli"
return baseURL.String(), nil
}
// transferRequest downloads the requested resource from the request URL
func transferRequest(requestURL string, logger logger.Service) ([]byte, string, error) {
func transferRequest(requestURL string, log *zerolog.Logger) ([]byte, string, error) {
client := &http.Client{Timeout: clientTimeout}
const pollAttempts = 10
// we do "long polling" on the endpoint to get the resource.
for i := 0; i < pollAttempts; i++ {
buf, key, err := poll(client, requestURL, logger)
buf, key, err := poll(client, requestURL, log)
if err != nil {
return nil, "", err
} else if len(buf) > 0 {
if err := putSuccess(client, requestURL); err != nil {
logger.Errorf("Failed to update resource success: %s", err)
log.Err(err).Msg("Failed to update resource success")
}
return buf, key, nil
}
@@ -118,7 +115,7 @@ func transferRequest(requestURL string, logger logger.Service) ([]byte, string,
}
// poll the endpoint for the request resource, waiting for the user interaction
func poll(client *http.Client, requestURL string, logger logger.Service) ([]byte, string, error) {
func poll(client *http.Client, requestURL string, log *zerolog.Logger) ([]byte, string, error) {
resp, err := client.Get(requestURL)
if err != nil {
return nil, "", err
@@ -131,7 +128,7 @@ func poll(client *http.Client, requestURL string, logger logger.Service) ([]byte
return nil, "", fmt.Errorf("error on request %d", resp.StatusCode)
}
if resp.StatusCode != 200 {
logger.Info("Waiting for login...")
log.Info().Msg("Waiting for login...")
return nil, "", nil
}
+104 -123
View File
@@ -35,6 +35,7 @@ import (
"github.com/google/uuid"
"github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
"github.com/urfave/cli/v2/altsrc"
)
@@ -42,8 +43,6 @@ import (
const (
sentryDSN = "https://56a9c9fa5c364ab28f34b14f35ea0f1b:3e8827f6f9f740738eb11138f7bebb68@sentry.io/189878"
sshLogFileDirectory = "/usr/local/var/log/cloudflared/"
// sshPortFlag is the port on localhost the cloudflared ssh server will run on
sshPortFlag = "local-ssh-port"
@@ -77,10 +76,14 @@ const (
// uiFlag is to enable launching cloudflared in interactive UI mode
uiFlag = "ui"
logDirectoryFlag = "log-directory"
debugLevelWarning = "At debug level, request URL, method, protocol, content legnth and header will be logged. " +
"Response status, content length and header will also be logged in debug level."
LogFieldCommand = "command"
LogFieldExpandedPath = "expandedPath"
LogFieldPIDPathname = "pidPathname"
LogFieldTmpTraceFilename = "tmpTraceFilename"
LogFieldTraceOutputFilepath = "traceOutputFilepath"
)
var (
@@ -176,14 +179,14 @@ func runAdhocNamedTunnel(sc *subcommandContext, name string) error {
return errors.Wrap(err, "failed to create tunnel")
}
} else {
sc.logger.Infof("Tunnel already created with ID %s", tunnel.ID)
sc.log.Info().Str(LogFieldTunnelID, tunnel.ID.String()).Msg("Reusing existing tunnel with this name")
}
if r, ok := routeFromFlag(sc.c); ok {
if res, err := sc.route(tunnel.ID, r); err != nil {
sc.logger.Errorf("failed to create route, please create it manually. err: %v.", err)
sc.log.Err(err).Msg("failed to create route, please create it manually")
} else {
sc.logger.Infof(res.SuccessSummary())
sc.log.Info().Msg(res.SuccessSummary())
}
}
@@ -196,7 +199,7 @@ func runAdhocNamedTunnel(sc *subcommandContext, name string) error {
// runClassicTunnel creates a "classic" non-named tunnel
func runClassicTunnel(sc *subcommandContext) error {
return StartServer(sc.c, version, shutdownC, graceShutdownC, nil, sc.logger, sc.isUIEnabled)
return StartServer(sc.c, version, shutdownC, graceShutdownC, nil, sc.log, sc.isUIEnabled)
}
func routeFromFlag(c *cli.Context) (tunnelstore.Route, bool) {
@@ -209,47 +212,13 @@ func routeFromFlag(c *cli.Context) (tunnelstore.Route, bool) {
return nil, false
}
func createLogger(c *cli.Context, isTransport bool, disableTerminal bool) (*logger.OutputWriter, error) {
var loggerOpts []logger.Option
logPath := c.String("logfile")
if logPath == "" {
logPath = c.String(logDirectoryFlag)
}
if logPath != "" {
loggerOpts = append(loggerOpts, logger.DefaultFile(logPath))
}
logLevel := c.String("loglevel")
if isTransport {
logLevel = c.String("transport-loglevel")
if logLevel == "" {
logLevel = "fatal"
}
}
loggerOpts = append(loggerOpts, logger.LogLevelString(logLevel))
if disableTerminal {
disableOption := logger.DisableTerminal(true)
loggerOpts = append(loggerOpts, disableOption)
}
l, err := logger.New(loggerOpts...)
if err != nil {
return nil, err
}
return l, nil
}
func StartServer(
c *cli.Context,
version string,
shutdownC,
graceShutdownC chan struct{},
namedTunnel *connection.NamedTunnelConfig,
generalLogger logger.Service,
log *zerolog.Logger,
isUIEnabled bool,
) error {
_ = raven.SetDSN(sentryDSN)
@@ -260,45 +229,51 @@ func StartServer(
dnsReadySignal := make(chan struct{})
if config.GetConfiguration().Source() == "" {
generalLogger.Infof(config.ErrNoConfigFile.Error())
log.Info().Msg(config.ErrNoConfigFile.Error())
}
if c.IsSet("trace-output") {
tmpTraceFile, err := ioutil.TempFile("", "trace")
if err != nil {
generalLogger.Errorf("Failed to create new temporary file to save trace output: %s", err)
log.Err(err).Msg("Failed to create new temporary file to save trace output")
}
traceLog := log.With().Str(LogFieldTmpTraceFilename, tmpTraceFile.Name()).Logger()
defer func() {
if err := tmpTraceFile.Close(); err != nil {
generalLogger.Errorf("Failed to close trace output file %s with error: %s", tmpTraceFile.Name(), err)
traceLog.Err(err).Msg("Failed to close temporary trace output file")
}
if err := os.Rename(tmpTraceFile.Name(), c.String("trace-output")); err != nil {
generalLogger.Errorf("Failed to rename temporary trace output file %s to %s with error: %s", tmpTraceFile.Name(), c.String("trace-output"), err)
traceOutputFilepath := c.String("trace-output")
if err := os.Rename(tmpTraceFile.Name(), traceOutputFilepath); err != nil {
traceLog.
Err(err).
Str(LogFieldTraceOutputFilepath, traceOutputFilepath).
Msg("Failed to rename temporary trace output file")
} else {
err := os.Remove(tmpTraceFile.Name())
if err != nil {
generalLogger.Errorf("Failed to remove the temporary trace file %s with error: %s", tmpTraceFile.Name(), err)
traceLog.Err(err).Msg("Failed to remove the temporary trace file")
}
}
}()
if err := trace.Start(tmpTraceFile); err != nil {
generalLogger.Errorf("Failed to start trace: %s", err)
traceLog.Err(err).Msg("Failed to start trace")
return errors.Wrap(err, "Error starting tracing")
}
defer trace.Stop()
}
buildInfo := buildinfo.GetBuildInfo(version)
buildInfo.Log(generalLogger)
logClientOptions(c, generalLogger)
buildInfo.Log(log)
logClientOptions(c, log)
if c.IsSet("proxy-dns") {
wg.Add(1)
go func() {
defer wg.Done()
errC <- runDNSProxyServer(c, dnsReadySignal, shutdownC, generalLogger)
errC <- runDNSProxyServer(c, dnsReadySignal, shutdownC, log)
}()
} else {
close(dnsReadySignal)
@@ -307,26 +282,14 @@ func StartServer(
// Wait for proxy-dns to come up (if used)
<-dnsReadySignal
metricsListener, err := listeners.Listen("tcp", c.String("metrics"))
if err != nil {
generalLogger.Errorf("Error opening metrics server listener: %s", err)
return errors.Wrap(err, "Error opening metrics server listener")
}
defer metricsListener.Close()
wg.Add(1)
go func() {
defer wg.Done()
errC <- metrics.ServeMetrics(metricsListener, shutdownC, generalLogger)
}()
go notifySystemd(connectedSignal)
if c.IsSet("pidfile") {
go writePidFile(connectedSignal, c.String("pidfile"), generalLogger)
go writePidFile(connectedSignal, c.String("pidfile"), log)
}
cloudflaredID, err := uuid.NewRandom()
if err != nil {
generalLogger.Errorf("Cannot generate cloudflared ID: %s", err)
log.Err(err).Msg("Cannot generate cloudflared ID")
return err
}
@@ -337,12 +300,13 @@ func StartServer(
}()
// update needs to be after DNS proxy is up to resolve equinox server address
if updater.IsAutoupdateEnabled(c, generalLogger) {
generalLogger.Infof("Autoupdate frequency is set to %v", c.Duration("autoupdate-freq"))
if updater.IsAutoupdateEnabled(c, log) {
autoupdateFreq := c.Duration("autoupdate-freq")
log.Info().Dur("autoupdateFreq", autoupdateFreq).Msg("Autoupdate frequency is set")
wg.Add(1)
go func() {
defer wg.Done()
autoupdater := updater.NewAutoUpdater(c.Duration("autoupdate-freq"), &listeners, generalLogger)
autoupdater := updater.NewAutoUpdater(c.Duration("autoupdate-freq"), &listeners, log)
errC <- autoupdater.Run(ctx)
}()
}
@@ -351,33 +315,51 @@ func StartServer(
if dnsProxyStandAlone(c) {
connectedSignal.Notify()
// no grace period, handle SIGINT/SIGTERM immediately
return waitToShutdown(&wg, errC, shutdownC, graceShutdownC, 0, generalLogger)
return waitToShutdown(&wg, errC, shutdownC, graceShutdownC, 0, log)
}
url := c.String("url")
hostname := c.String("hostname")
if url == hostname && url != "" && hostname != "" {
errText := "hostname and url shouldn't match. See --help for more information"
generalLogger.Error(errText)
log.Error().Msg(errText)
return fmt.Errorf(errText)
}
transportLogger, err := createLogger(c, true, isUIEnabled)
if err != nil {
return errors.Wrap(err, "error setting up transport logger")
}
transportLog := logger.CreateTransportLoggerFromContext(c, isUIEnabled)
tunnelConfig, ingressRules, err := prepareTunnelConfig(c, buildInfo, version, generalLogger, transportLogger, namedTunnel, isUIEnabled)
readinessCh := make(chan connection.Event, 16)
uiCh := make(chan connection.Event, 16)
eventChannels := []chan connection.Event{
readinessCh,
uiCh,
}
tunnelConfig, ingressRules, err := prepareTunnelConfig(c, buildInfo, version, log, transportLog, namedTunnel, isUIEnabled, eventChannels)
if err != nil {
log.Err(err).Msg("Couldn't start tunnel")
return err
}
ingressRules.StartOrigins(&wg, generalLogger, shutdownC, errC)
metricsListener, err := listeners.Listen("tcp", c.String("metrics"))
if err != nil {
log.Err(err).Msg("Error opening metrics server listener")
return errors.Wrap(err, "Error opening metrics server listener")
}
defer metricsListener.Close()
wg.Add(1)
go func() {
defer wg.Done()
errC <- metrics.ServeMetrics(metricsListener, shutdownC, readinessCh, log)
}()
if err := ingressRules.StartOrigins(&wg, log, shutdownC, errC); err != nil {
return err
}
reconnectCh := make(chan origin.ReconnectSignal, 1)
if c.IsSet("stdin-control") {
generalLogger.Info("Enabling control through stdin")
go stdinControl(reconnectCh, generalLogger)
log.Info().Msg("Enabling control through stdin")
go stdinControl(reconnectCh, log)
}
wg.Add(1)
@@ -394,31 +376,21 @@ func StartServer(
&ingressRules,
tunnelConfig.HAConnections,
)
logLevels, err := logger.ParseLevelString(c.String("loglevel"))
if err != nil {
return err
}
tunnelInfo.LaunchUI(ctx, generalLogger, transportLogger, logLevels, tunnelConfig.TunnelEventChan)
tunnelInfo.LaunchUI(ctx, log, transportLog, uiCh)
} else {
go func() {
for range uiCh {
// Consume UI events into a noop
}
}()
}
return waitToShutdown(&wg, errC, shutdownC, graceShutdownC, c.Duration("grace-period"), generalLogger)
}
// forceSetFlag attempts to set the given flag value in the closest context that has it defined
func forceSetFlag(c *cli.Context, name, value string) {
for _, ctx := range c.Lineage() {
if err := ctx.Set(name, value); err == nil {
break
}
}
return waitToShutdown(&wg, errC, shutdownC, graceShutdownC, c.Duration("grace-period"), log)
}
func SetFlagsFromConfigFile(c *cli.Context) error {
const exitCode = 1
log, err := createLogger(c, false, false)
if err != nil {
return cliutil.PrintLoggerSetupError("error setting up logger", err)
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
inputSource, err := config.ReadConfigFile(c, log)
if err != nil {
if err == config.ErrNoConfigFile {
@@ -440,20 +412,20 @@ func waitToShutdown(wg *sync.WaitGroup,
errC chan error,
shutdownC, graceShutdownC chan struct{},
gracePeriod time.Duration,
logger logger.Service,
log *zerolog.Logger,
) error {
var err error
if gracePeriod > 0 {
err = waitForSignalWithGraceShutdown(errC, shutdownC, graceShutdownC, gracePeriod, logger)
err = waitForSignalWithGraceShutdown(errC, shutdownC, graceShutdownC, gracePeriod, log)
} else {
err = waitForSignal(errC, shutdownC, logger)
err = waitForSignal(errC, shutdownC, log)
close(graceShutdownC)
}
if err != nil {
logger.Errorf("Quitting due to error: %s", err)
log.Err(err).Msg("Quitting due to error")
} else {
logger.Info("Quitting...")
log.Info().Msg("Quitting...")
}
// Wait for clean exit, discarding all errors
go func() {
@@ -469,16 +441,16 @@ func notifySystemd(waitForSignal *signal.Signal) {
daemon.SdNotify(false, "READY=1")
}
func writePidFile(waitForSignal *signal.Signal, pidFile string, logger logger.Service) {
func writePidFile(waitForSignal *signal.Signal, pidPathname string, log *zerolog.Logger) {
<-waitForSignal.Wait()
expandedPath, err := homedir.Expand(pidFile)
expandedPath, err := homedir.Expand(pidPathname)
if err != nil {
logger.Errorf("Unable to expand %s, try to use absolute path in --pidfile: %s", pidFile, err)
log.Err(err).Str(LogFieldPIDPathname, pidPathname).Msg("Unable to expand the path, try to use absolute path in --pidfile")
return
}
file, err := os.Create(expandedPath)
if err != nil {
logger.Errorf("Unable to write pid to %s: %s", expandedPath, err)
log.Err(err).Str(LogFieldExpandedPath, expandedPath).Msg("Unable to write pid")
return
}
defer file.Close()
@@ -650,13 +622,15 @@ func tunnelFlags(shouldHide bool) []cli.Flag {
Value: time.Second * 5,
Hidden: true,
}),
altsrc.NewUint64Flag(&cli.Uint64Flag{
// Note TUN-3758 , we use Int because UInt is not supported with altsrc
altsrc.NewIntFlag(&cli.IntFlag{
Name: "heartbeat-count",
Usage: "Minimum number of unacked heartbeats to send before closing the connection.",
Value: 5,
Hidden: true,
}),
altsrc.NewUintFlag(&cli.UintFlag{
// Note TUN-3758 , we use Int because UInt is not supported with altsrc
altsrc.NewIntFlag(&cli.IntFlag{
Name: "retries",
Value: 5,
Usage: "Maximum number of retries for connection/protocol errors.",
@@ -675,7 +649,8 @@ func tunnelFlags(shouldHide bool) []cli.Flag {
EnvVars: []string{"TUNNEL_GRACE_PERIOD"},
Hidden: true,
}),
altsrc.NewUintFlag(&cli.UintFlag{
// Note TUN-3758 , we use Int because UInt is not supported with altsrc
altsrc.NewIntFlag(&cli.IntFlag{
Name: "compression-quality",
Value: 0,
Usage: "(beta) Use cross-stream compression instead HTTP compression. 0-off, 1-low, 2-medium, >=3-high.",
@@ -960,6 +935,7 @@ func sshFlags(shouldHide bool) []cli.Flag {
EnvVars: []string{"TUNNEL_PROXY_ADDRESS"},
Hidden: shouldHide,
}),
// Note TUN-3758 , we use Int because UInt is not supported with altsrc
altsrc.NewIntFlag(&cli.IntFlag{
Name: ingress.ProxyPortFlag,
Usage: "Listen port for the proxy.",
@@ -973,14 +949,14 @@ func sshFlags(shouldHide bool) []cli.Flag {
func configureLoggingFlags(shouldHide bool) []cli.Flag {
return []cli.Flag{
altsrc.NewStringFlag(&cli.StringFlag{
Name: "loglevel",
Name: logger.LogLevelFlag,
Value: "info",
Usage: "Application logging level {fatal, error, info, debug}. " + debugLevelWarning,
EnvVars: []string{"TUNNEL_LOGLEVEL"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "transport-loglevel",
Name: logger.LogTransportLevelFlag,
Aliases: []string{"proto-loglevel"}, // This flag used to be called proto-loglevel
Value: "info",
Usage: "Transport logging level(previously called protocol logging level) {fatal, error, info, debug}",
@@ -988,13 +964,13 @@ func configureLoggingFlags(shouldHide bool) []cli.Flag {
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "logfile",
Name: logger.LogFileFlag,
Usage: "Save application log to this file for reporting issues.",
EnvVars: []string{"TUNNEL_LOGFILE"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: logDirectoryFlag,
Name: logger.LogDirectoryFlag,
Usage: "Save application log to this directory for reporting issues.",
EnvVars: []string{"TUNNEL_LOGDIRECTORY"},
Hidden: shouldHide,
@@ -1038,16 +1014,21 @@ func configureProxyDNSFlags(shouldHide bool) []cli.Flag {
Hidden: shouldHide,
}),
altsrc.NewStringSliceFlag(&cli.StringSliceFlag{
Name: "proxy-dns-bootstrap",
Usage: "bootstrap endpoint URL, you can specify multiple endpoints for redundancy.",
Value: cli.NewStringSlice("https://162.159.36.1/dns-query", "https://162.159.46.1/dns-query", "https://[2606:4700:4700::1111]/dns-query", "https://[2606:4700:4700::1001]/dns-query"),
Name: "proxy-dns-bootstrap",
Usage: "bootstrap endpoint URL, you can specify multiple endpoints for redundancy.",
Value: cli.NewStringSlice(
"https://162.159.36.1/dns-query",
"https://162.159.46.1/dns-query",
"https://[2606:4700:4700::1111]/dns-query",
"https://[2606:4700:4700::1001]/dns-query",
),
EnvVars: []string{"TUNNEL_DNS_BOOTSTRAP"},
Hidden: shouldHide,
}),
}
}
func stdinControl(reconnectCh chan origin.ReconnectSignal, logger logger.Service) {
func stdinControl(reconnectCh chan origin.ReconnectSignal, log *zerolog.Logger) {
for {
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
@@ -1062,17 +1043,17 @@ func stdinControl(reconnectCh chan origin.ReconnectSignal, logger logger.Service
if len(parts) > 1 {
var err error
if reconnect.Delay, err = time.ParseDuration(parts[1]); err != nil {
logger.Error(err.Error())
log.Error().Msg(err.Error())
continue
}
}
logger.Infof("Sending reconnect signal %+v", reconnect)
log.Info().Msgf("Sending reconnect signal %+v", reconnect)
reconnectCh <- reconnect
default:
logger.Infof("Unknown command: %s", command)
log.Info().Str(LogFieldCommand, command).Msg("Unknown command")
fallthrough
case "help":
logger.Info(`Supported command:
log.Info().Msg(`Supported command:
reconnect [delay]
- restarts one randomly chosen connection with optional delay before reconnect`)
}
+55 -51
View File
@@ -10,12 +10,10 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/buildinfo"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/cmd/cloudflared/ui"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/edgediscovery"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/origin"
"github.com/cloudflare/cloudflared/tlsconfig"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
@@ -24,15 +22,20 @@ import (
"github.com/google/uuid"
"github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
"golang.org/x/crypto/ssh/terminal"
)
const LogFieldOriginCertPath = "originCertPath"
var (
developerPortal = "https://developers.cloudflare.com/argo-tunnel"
quickStartUrl = developerPortal + "/quickstart/quickstart/"
serviceUrl = developerPortal + "/reference/service/"
argumentsUrl = developerPortal + "/reference/arguments/"
LogFieldHostname = "hostname"
)
// returns the first path that contains a cert.pem file. If none of the DefaultConfigSearchDirectories
@@ -47,16 +50,16 @@ func findDefaultOriginCertPath() string {
return ""
}
func generateRandomClientID(logger logger.Service) (string, error) {
func generateRandomClientID(log *zerolog.Logger) (string, error) {
u, err := uuid.NewRandom()
if err != nil {
logger.Errorf("couldn't create UUID for client ID %s", err)
log.Error().Msgf("couldn't create UUID for client ID %s", err)
return "", err
}
return u.String(), nil
}
func logClientOptions(c *cli.Context, logger logger.Service) {
func logClientOptions(c *cli.Context, log *zerolog.Logger) {
flags := make(map[string]interface{})
for _, flag := range c.LocalFlagNames() {
flags[flag] = c.Generic(flag)
@@ -70,7 +73,7 @@ func logClientOptions(c *cli.Context, logger logger.Service) {
}
if len(flags) > 0 {
logger.Infof("Environment variables %v", flags)
log.Info().Msgf("Environment variables %v", flags)
}
envs := make(map[string]string)
@@ -85,7 +88,7 @@ func logClientOptions(c *cli.Context, logger logger.Service) {
}
}
if len(envs) > 0 {
logger.Infof("Environmental variables %v", envs)
log.Info().Msgf("Environmental variables %v", envs)
}
}
@@ -93,32 +96,31 @@ func dnsProxyStandAlone(c *cli.Context) bool {
return c.IsSet("proxy-dns") && (!c.IsSet("hostname") && !c.IsSet("tag") && !c.IsSet("hello-world"))
}
func findOriginCert(c *cli.Context, logger logger.Service) (string, error) {
originCertPath := c.String("origincert")
func findOriginCert(originCertPath string, log *zerolog.Logger) (string, error) {
if originCertPath == "" {
logger.Infof("Cannot determine default origin certificate path. No file %s in %v", config.DefaultCredentialFile, config.DefaultConfigSearchDirectories())
log.Info().Msgf("Cannot determine default origin certificate path. No file %s in %v", config.DefaultCredentialFile, config.DefaultConfigSearchDirectories())
if isRunningFromTerminal() {
logger.Errorf("You need to specify the origin certificate path with --origincert option, or set TUNNEL_ORIGIN_CERT environment variable. See %s for more information.", argumentsUrl)
return "", fmt.Errorf("Client didn't specify origincert path when running from terminal")
log.Error().Msgf("You need to specify the origin certificate path with --origincert option, or set TUNNEL_ORIGIN_CERT environment variable. See %s for more information.", argumentsUrl)
return "", fmt.Errorf("client didn't specify origincert path when running from terminal")
} else {
logger.Errorf("You need to specify the origin certificate path by specifying the origincert option in the configuration file, or set TUNNEL_ORIGIN_CERT environment variable. See %s for more information.", serviceUrl)
return "", fmt.Errorf("Client didn't specify origincert path")
log.Error().Msgf("You need to specify the origin certificate path by specifying the origincert option in the configuration file, or set TUNNEL_ORIGIN_CERT environment variable. See %s for more information.", serviceUrl)
return "", fmt.Errorf("client didn't specify origincert path")
}
}
var err error
originCertPath, err = homedir.Expand(originCertPath)
if err != nil {
logger.Errorf("Cannot resolve path %s: %s", originCertPath, err)
return "", fmt.Errorf("Cannot resolve path %s", originCertPath)
log.Err(err).Msgf("Cannot resolve origin certificate path")
return "", fmt.Errorf("cannot resolve path %s", originCertPath)
}
// Check that the user has acquired a certificate using the login command
ok, err := config.FileExists(originCertPath)
if err != nil {
logger.Errorf("Cannot check if origin cert exists at path %s", originCertPath)
return "", fmt.Errorf("Cannot check if origin cert exists at path %s", originCertPath)
log.Error().Msgf("Cannot check if origin cert exists at path %s", originCertPath)
return "", fmt.Errorf("cannot check if origin cert exists at path %s", originCertPath)
}
if !ok {
logger.Errorf(`Cannot find a valid certificate for your origin at the path:
log.Error().Msgf(`Cannot find a valid certificate for your origin at the path:
%s
@@ -127,29 +129,26 @@ If you don't have a certificate signed by Cloudflare, run the command:
%s login
`, originCertPath, os.Args[0])
return "", fmt.Errorf("Cannot find a valid certificate at the path %s", originCertPath)
return "", fmt.Errorf("cannot find a valid certificate at the path %s", originCertPath)
}
return originCertPath, nil
}
func readOriginCert(originCertPath string, logger logger.Service) ([]byte, error) {
logger.Debugf("Reading origin cert from %s", originCertPath)
func readOriginCert(originCertPath string) ([]byte, error) {
// Easier to send the certificate as []byte via RPC than decoding it at this point
originCert, err := ioutil.ReadFile(originCertPath)
if err != nil {
logger.Errorf("Cannot read %s to load origin certificate: %s", originCertPath, err)
return nil, fmt.Errorf("Cannot read %s to load origin certificate", originCertPath)
return nil, fmt.Errorf("cannot read %s to load origin certificate", originCertPath)
}
return originCert, nil
}
func getOriginCert(c *cli.Context, logger logger.Service) ([]byte, error) {
if originCertPath, err := findOriginCert(c, logger); err != nil {
func getOriginCert(originCertPath string, log *zerolog.Logger) ([]byte, error) {
if originCertPath, err := findOriginCert(originCertPath, log); err != nil {
return nil, err
} else {
return readOriginCert(originCertPath, logger)
return readOriginCert(originCertPath)
}
}
@@ -157,22 +156,24 @@ func prepareTunnelConfig(
c *cli.Context,
buildInfo *buildinfo.BuildInfo,
version string,
logger logger.Service,
transportLogger logger.Service,
log *zerolog.Logger,
transportLogger *zerolog.Logger,
namedTunnel *connection.NamedTunnelConfig,
uiIsEnabled bool,
isUIEnabled bool,
eventChans []chan connection.Event,
) (*origin.TunnelConfig, ingress.Ingress, error) {
isNamedTunnel := namedTunnel != nil
hostname, err := validation.ValidateHostname(c.String("hostname"))
configHostname := c.String("hostname")
hostname, err := validation.ValidateHostname(configHostname)
if err != nil {
logger.Errorf("Invalid hostname: %s", err)
log.Err(err).Str(LogFieldHostname, configHostname).Msg("Invalid hostname")
return nil, ingress.Ingress{}, errors.Wrap(err, "Invalid hostname")
}
isFreeTunnel := hostname == ""
clientID := c.String("id")
if !c.IsSet("id") {
clientID, err = generateRandomClientID(logger)
clientID, err = generateRandomClientID(log)
if err != nil {
return nil, ingress.Ingress{}, err
}
@@ -180,7 +181,7 @@ func prepareTunnelConfig(
tags, err := NewTagSliceFromCLI(c.StringSlice("tag"))
if err != nil {
logger.Errorf("Tag parse failure: %s", err)
log.Err(err).Msg("Tag parse failure")
return nil, ingress.Ingress{}, errors.Wrap(err, "Tag parse failure")
}
@@ -188,7 +189,12 @@ func prepareTunnelConfig(
var originCert []byte
if !isFreeTunnel {
originCert, err = getOriginCert(c, logger)
originCertPath := c.String("origincert")
originCertLog := log.With().
Str(LogFieldOriginCertPath, originCertPath).
Logger()
originCert, err = getOriginCert(originCertPath, &originCertLog)
if err != nil {
return nil, ingress.Ingress{}, errors.Wrap(err, "Error getting origin cert")
}
@@ -227,17 +233,17 @@ func prepareTunnelConfig(
// Convert single-origin configuration into multi-origin configuration.
if ingressRules.IsEmpty() {
ingressRules, err = ingress.NewSingleOrigin(c, !isNamedTunnel, logger)
ingressRules, err = ingress.NewSingleOrigin(c, !isNamedTunnel)
if err != nil {
return nil, ingress.Ingress{}, err
}
}
protocolSelector, err := connection.NewProtocolSelector(c.String("protocol"), namedTunnel, edgediscovery.HTTP2Percentage, origin.ResolveTTL, logger)
protocolSelector, err := connection.NewProtocolSelector(c.String("protocol"), namedTunnel, edgediscovery.HTTP2Percentage, origin.ResolveTTL, log)
if err != nil {
return nil, ingress.Ingress{}, err
}
logger.Infof("Initial protocol %s", protocolSelector.Current())
log.Info().Msgf("Initial protocol %s", protocolSelector.Current())
edgeTLSConfigs := make(map[connection.Protocol]*tls.Config, len(connection.ProtocolList))
for _, p := range connection.ProtocolList {
@@ -248,7 +254,7 @@ func prepareTunnelConfig(
edgeTLSConfigs[p] = edgeTLSConfig
}
originClient := origin.NewClient(ingressRules, tags, logger)
originClient := origin.NewClient(ingressRules, tags, log)
connectionConfig := &connection.Config{
OriginClient: originClient,
GracePeriod: c.Duration("grace-period"),
@@ -256,16 +262,13 @@ func prepareTunnelConfig(
}
muxerConfig := &connection.MuxerConfig{
HeartbeatInterval: c.Duration("heartbeat-interval"),
MaxHeartbeats: c.Uint64("heartbeat-count"),
CompressionSetting: h2mux.CompressionSetting(c.Uint64("compression-quality")),
// Note TUN-3758 , we use Int because UInt is not supported with altsrc
MaxHeartbeats: uint64(c.Int("heartbeat-count")),
// Note TUN-3758 , we use Int because UInt is not supported with altsrc
CompressionSetting: h2mux.CompressionSetting(uint64(c.Int("compression-quality"))),
MetricsUpdateFreq: c.Duration("metrics-update-freq"),
}
var tunnelEventChan chan ui.TunnelEvent
if uiIsEnabled {
tunnelEventChan = make(chan ui.TunnelEvent, 16)
}
return &origin.TunnelConfig{
ConnectionConfig: connectionConfig,
BuildInfo: buildInfo,
@@ -277,15 +280,16 @@ func prepareTunnelConfig(
IsFreeTunnel: isFreeTunnel,
LBPool: c.String("lb-pool"),
Tags: tags,
Logger: logger,
Observer: connection.NewObserver(transportLogger, tunnelEventChan),
Log: log,
Observer: connection.NewObserver(transportLogger, eventChans, isUIEnabled),
ReportedVersion: version,
Retries: c.Uint("retries"),
// Note TUN-3758 , we use Int because UInt is not supported with altsrc
Retries: uint(c.Int("retries")),
RunFromTerminal: isRunningFromTerminal(),
NamedTunnel: namedTunnel,
ClassicTunnel: classicTunnel,
MuxerConfig: muxerConfig,
TunnelEventChan: tunnelEventChan,
TunnelEventChans: eventChans,
ProtocolSelector: protocolSelector,
EdgeTLSConfigs: edgeTLSConfigs,
}, ingressRules, nil
@@ -0,0 +1,83 @@
package tunnel
import (
"fmt"
"path/filepath"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/google/uuid"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
)
// CredFinder can find the tunnel credentials file.
type CredFinder interface {
Path() (string, error)
}
// Implements CredFinder and looks for the credentials file at the given
// filepath.
type staticPath struct {
filePath string
fs fileSystem
}
func newStaticPath(filePath string, fs fileSystem) CredFinder {
return staticPath{
filePath: filePath,
fs: fs,
}
}
func (a staticPath) Path() (string, error) {
if a.filePath != "" && a.fs.validFilePath(a.filePath) {
return a.filePath, nil
}
return "", fmt.Errorf("Tunnel credentials file '%s' doesn't exist or is not a file", a.filePath)
}
// Implements CredFinder and looks for the credentials file in several directories
// searching for a file named <id>.json
type searchByID struct {
id uuid.UUID
c *cli.Context
log *zerolog.Logger
fs fileSystem
}
func newSearchByID(id uuid.UUID, c *cli.Context, log *zerolog.Logger, fs fileSystem) CredFinder {
return searchByID{
id: id,
c: c,
log: log,
fs: fs,
}
}
func (s searchByID) Path() (string, error) {
originCertPath := s.c.String("origincert")
originCertLog := s.log.With().
Str(LogFieldOriginCertPath, originCertPath).
Logger()
// Fallback to look for tunnel credentials in the origin cert directory
if originCertPath, err := findOriginCert(originCertPath, &originCertLog); err == nil {
originCertDir := filepath.Dir(originCertPath)
if filePath, err := tunnelFilePath(s.id, originCertDir); err == nil {
if s.fs.validFilePath(filePath) {
return filePath, nil
}
}
}
// Last resort look under default config directories
for _, configDir := range config.DefaultConfigSearchDirectories() {
if filePath, err := tunnelFilePath(s.id, configDir); err == nil {
if s.fs.validFilePath(filePath) {
return filePath, nil
}
}
}
return "", fmt.Errorf("tunnel credentials file not found")
}
+27
View File
@@ -0,0 +1,27 @@
package tunnel
import (
"io/ioutil"
"os"
)
// Abstract away details of reading files, so that SubcommandContext can read
// from either the real filesystem, or a mock (when running unit tests).
type fileSystem interface {
readFile(filePath string) ([]byte, error)
validFilePath(path string) bool
}
type realFileSystem struct{}
func (fs realFileSystem) validFilePath(path string) bool {
fileStat, err := os.Stat(path)
if err != nil {
return false
}
return !fileStat.IsDir()
}
func (fs realFileSystem) readFile(filePath string) ([]byte, error) {
return ioutil.ReadFile(filePath)
}
@@ -4,12 +4,12 @@ import (
"fmt"
"net/url"
"github.com/pkg/errors"
"github.com/urfave/cli/v2"
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/ingress"
"github.com/pkg/errors"
"github.com/urfave/cli/v2"
)
func buildIngressSubcommand() *cli.Command {
+17 -7
View File
@@ -2,14 +2,15 @@ package tunnel
import (
"fmt"
"io/ioutil"
"net/url"
"os"
"path/filepath"
"syscall"
homedir "github.com/mitchellh/go-homedir"
"github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
cli "github.com/urfave/cli/v2"
"github.com/urfave/cli/v2"
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
@@ -39,10 +40,7 @@ func buildLoginSubcommand(hidden bool) *cli.Command {
}
func login(c *cli.Context) error {
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
path, ok, err := checkForExistingCert()
if ok {
@@ -58,12 +56,24 @@ func login(c *cli.Context) error {
return err
}
_, err = transfer.Run(loginURL, "cert", "callback", callbackStoreURL, path, false, false, logger)
resourceData, err := transfer.Run(
loginURL,
"cert",
"callback",
callbackStoreURL,
false,
false,
log,
)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to write the certificate due to the following error:\n%v\n\nYour browser will download the certificate instead. You will have to manually\ncopy it to the following path:\n\n%s\n", err, path)
return err
}
if err := ioutil.WriteFile(path, resourceData, 0600); err != nil {
return errors.Wrap(err, fmt.Sprintf("error writing cert to %s", path))
}
fmt.Fprintf(os.Stdout, "You have successfully logged in.\nIf you wish to copy your credentials to a server, they have been saved to:\n%s\n", path)
return nil
}
+5 -6
View File
@@ -1,20 +1,19 @@
package tunnel
import (
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/tunneldns"
"github.com/urfave/cli/v2"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
)
func runDNSProxyServer(c *cli.Context, dnsReadySignal, shutdownC chan struct{}, logger logger.Service) error {
func runDNSProxyServer(c *cli.Context, dnsReadySignal, shutdownC chan struct{}, log *zerolog.Logger) error {
port := c.Int("proxy-dns-port")
if port <= 0 || port > 65535 {
return errors.New("The 'proxy-dns-port' must be a valid port number in <1, 65535> range.")
}
listener, err := tunneldns.CreateListener(c.String("proxy-dns-address"), uint16(port), c.StringSlice("proxy-dns-upstream"), c.StringSlice("proxy-dns-bootstrap"), logger)
listener, err := tunneldns.CreateListener(c.String("proxy-dns-address"), uint16(port), c.StringSlice("proxy-dns-upstream"), c.StringSlice("proxy-dns-bootstrap"), log)
if err != nil {
close(dnsReadySignal)
listener.Stop()
@@ -26,6 +25,6 @@ func runDNSProxyServer(c *cli.Context, dnsReadySignal, shutdownC chan struct{},
return errors.Wrap(err, "Cannot start the DNS over HTTPS proxy server")
}
<-shutdownC
listener.Stop()
_ = listener.Stop()
return nil
}
+11 -10
View File
@@ -6,24 +6,26 @@ import (
"syscall"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/rs/zerolog"
)
const LogFieldSignal = "signal"
// waitForSignal notifies all routines to shutdownC immediately by closing the
// shutdownC when one of the routines in main exits, or when this process receives
// SIGTERM/SIGINT
func waitForSignal(errC chan error, shutdownC chan struct{}, logger logger.Service) error {
func waitForSignal(errC chan error, shutdownC chan struct{}, log *zerolog.Logger) error {
signals := make(chan os.Signal, 10)
signal.Notify(signals, syscall.SIGTERM, syscall.SIGINT)
defer signal.Stop(signals)
select {
case err := <-errC:
logger.Infof("terminating due to error: %v", err)
log.Err(err).Msg("terminating due to error")
close(shutdownC)
return err
case s := <-signals:
logger.Infof("terminating due to signal %s", s)
log.Info().Str(LogFieldSignal, s.String()).Msg("terminating due to signal")
close(shutdownC)
case <-shutdownC:
}
@@ -41,7 +43,7 @@ func waitForSignal(errC chan error, shutdownC chan struct{}, logger logger.Servi
func waitForSignalWithGraceShutdown(errC chan error,
shutdownC, graceShutdownC chan struct{},
gracePeriod time.Duration,
logger logger.Service,
logger *zerolog.Logger,
) error {
signals := make(chan os.Signal, 10)
signal.Notify(signals, syscall.SIGTERM, syscall.SIGINT)
@@ -49,16 +51,16 @@ func waitForSignalWithGraceShutdown(errC chan error,
select {
case err := <-errC:
logger.Infof("Initiating graceful shutdown due to %v ...", err)
logger.Info().Msgf("Initiating graceful shutdown due to %v ...", err)
close(graceShutdownC)
close(shutdownC)
return err
case s := <-signals:
logger.Infof("Initiating graceful shutdown due to signal %s ...", s)
logger.Info().Msgf("Initiating graceful shutdown due to signal %s ...", s)
close(graceShutdownC)
waitForGracePeriod(signals, errC, shutdownC, gracePeriod, logger)
waitForGracePeriod(signals, errC, shutdownC, gracePeriod)
case <-graceShutdownC:
waitForGracePeriod(signals, errC, shutdownC, gracePeriod, logger)
waitForGracePeriod(signals, errC, shutdownC, gracePeriod)
case <-shutdownC:
close(graceShutdownC)
}
@@ -70,7 +72,6 @@ func waitForGracePeriod(signals chan os.Signal,
errC chan error,
shutdownC chan struct{},
gracePeriod time.Duration,
logger logger.Service,
) {
// Unregister signal handler early, so the client can send a second SIGTERM/SIGINT
// to force shutdown cloudflared
+13 -13
View File
@@ -2,11 +2,11 @@ package tunnel
import (
"fmt"
"github.com/rs/zerolog"
"syscall"
"testing"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/stretchr/testify/assert"
)
@@ -28,7 +28,7 @@ func testChannelClosed(t *testing.T, c chan struct{}) {
}
func TestWaitForSignal(t *testing.T) {
logger := logger.NewOutputWriter(logger.NewMockWriteManager())
log := zerolog.Nop()
// Test handling server error
errC := make(chan error)
@@ -39,7 +39,7 @@ func TestWaitForSignal(t *testing.T) {
}()
// received error, shutdownC should be closed
err := waitForSignal(errC, shutdownC, logger)
err := waitForSignal(errC, shutdownC, &log)
assert.Equal(t, serverErr, err)
testChannelClosed(t, shutdownC)
@@ -56,10 +56,10 @@ func TestWaitForSignal(t *testing.T) {
go func(sig syscall.Signal) {
// sleep for a tick to prevent sending signal before calling waitForSignal
time.Sleep(tick)
syscall.Kill(syscall.Getpid(), sig)
_ = syscall.Kill(syscall.Getpid(), sig)
}(sig)
err = waitForSignal(errC, shutdownC, logger)
err = waitForSignal(errC, shutdownC, &log)
assert.Equal(t, nil, err)
assert.Equal(t, shutdownErr, <-errC)
testChannelClosed(t, shutdownC)
@@ -76,10 +76,10 @@ func TestWaitForSignalWithGraceShutdown(t *testing.T) {
errC <- serverErr
}()
logger := logger.NewOutputWriter(logger.NewMockWriteManager())
log := zerolog.Nop()
// received error, both shutdownC and graceshutdownC should be closed
err := waitForSignalWithGraceShutdown(errC, shutdownC, graceshutdownC, tick, logger)
err := waitForSignalWithGraceShutdown(errC, shutdownC, graceshutdownC, tick, &log)
assert.Equal(t, serverErr, err)
testChannelClosed(t, shutdownC)
testChannelClosed(t, graceshutdownC)
@@ -89,7 +89,7 @@ func TestWaitForSignalWithGraceShutdown(t *testing.T) {
shutdownC = make(chan struct{})
graceshutdownC = make(chan struct{})
close(shutdownC)
err = waitForSignalWithGraceShutdown(errC, shutdownC, graceshutdownC, tick, logger)
err = waitForSignalWithGraceShutdown(errC, shutdownC, graceshutdownC, tick, &log)
assert.NoError(t, err)
testChannelClosed(t, shutdownC)
testChannelClosed(t, graceshutdownC)
@@ -99,7 +99,7 @@ func TestWaitForSignalWithGraceShutdown(t *testing.T) {
shutdownC = make(chan struct{})
graceshutdownC = make(chan struct{})
close(graceshutdownC)
err = waitForSignalWithGraceShutdown(errC, shutdownC, graceshutdownC, tick, logger)
err = waitForSignalWithGraceShutdown(errC, shutdownC, graceshutdownC, tick, &log)
assert.NoError(t, err)
testChannelClosed(t, shutdownC)
testChannelClosed(t, graceshutdownC)
@@ -119,10 +119,10 @@ func TestWaitForSignalWithGraceShutdown(t *testing.T) {
go func(sig syscall.Signal) {
// sleep for a tick to prevent sending signal before calling waitForSignalWithGraceShutdown
time.Sleep(tick)
syscall.Kill(syscall.Getpid(), sig)
_ = syscall.Kill(syscall.Getpid(), sig)
}(sig)
err = waitForSignalWithGraceShutdown(errC, shutdownC, graceshutdownC, tick, logger)
err = waitForSignalWithGraceShutdown(errC, shutdownC, graceshutdownC, tick, &log)
assert.Equal(t, nil, err)
assert.Equal(t, graceShutdownErr, <-errC)
testChannelClosed(t, shutdownC)
@@ -145,10 +145,10 @@ func TestWaitForSignalWithGraceShutdown(t *testing.T) {
go func(sig syscall.Signal) {
// sleep for a tick to prevent sending signal before calling waitForSignalWithGraceShutdown
time.Sleep(tick)
syscall.Kill(syscall.Getpid(), sig)
_ = syscall.Kill(syscall.Getpid(), sig)
}(sig)
err = waitForSignalWithGraceShutdown(errC, shutdownC, graceshutdownC, tick, logger)
err = waitForSignalWithGraceShutdown(errC, shutdownC, graceshutdownC, tick, &log)
assert.Equal(t, nil, err)
assert.Equal(t, shutdownErr, <-errC)
testChannelClosed(t, shutdownC)
+90 -75
View File
@@ -3,20 +3,17 @@ package tunnel
import (
"encoding/json"
"fmt"
"io/ioutil"
"os"
"path/filepath"
"strings"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
"github.com/cloudflare/cloudflared/certutil"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/cloudflare/cloudflared/tunnelstore"
)
@@ -32,32 +29,38 @@ func (e errInvalidJSONCredential) Error() string {
// subcommandContext carries structs shared between subcommands, to reduce number of arguments needed to
// pass between subcommands, and make sure they are only initialized once
type subcommandContext struct {
c *cli.Context
logger logger.Service
c *cli.Context
log *zerolog.Logger
isUIEnabled bool
fs fileSystem
// These fields should be accessed using their respective Getter
tunnelstoreClient tunnelstore.Client
userCredential *userCredential
isUIEnabled bool
}
func newSubcommandContext(c *cli.Context) (*subcommandContext, error) {
isUIEnabled := c.IsSet(uiFlag) && c.String("name") != ""
// If UI is enabled, terminal log output should be disabled -- log should be written into a UI log window instead
logger, err := createLogger(c, false, isUIEnabled)
if err != nil {
return nil, errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, isUIEnabled)
return &subcommandContext{
c: c,
logger: logger,
log: log,
isUIEnabled: isUIEnabled,
fs: realFileSystem{},
}, nil
}
// Returns something that can find the given tunnel's credentials file.
func (sc *subcommandContext) credentialFinder(tunnelID uuid.UUID) CredFinder {
if path := sc.c.String(CredFileFlag); path != "" {
return newStaticPath(path, sc.fs)
}
return newSearchByID(tunnelID, sc.c, sc.log, sc.fs)
}
type userCredential struct {
cert *certutil.OriginCert
certPath string
@@ -72,7 +75,15 @@ func (sc *subcommandContext) client() (tunnelstore.Client, error) {
return nil, err
}
userAgent := fmt.Sprintf("cloudflared/%s", version)
client, err := tunnelstore.NewRESTClient(sc.c.String("api-url"), credential.cert.AccountID, credential.cert.ZoneID, credential.cert.ServiceKey, userAgent, sc.logger)
client, err := tunnelstore.NewRESTClient(
sc.c.String("api-url"),
credential.cert.AccountID,
credential.cert.ZoneID,
credential.cert.ServiceKey,
userAgent,
sc.log,
)
if err != nil {
return nil, err
}
@@ -82,11 +93,16 @@ func (sc *subcommandContext) client() (tunnelstore.Client, error) {
func (sc *subcommandContext) credential() (*userCredential, error) {
if sc.userCredential == nil {
originCertPath, err := findOriginCert(sc.c, sc.logger)
originCertPath := sc.c.String("origincert")
originCertLog := sc.log.With().
Str(LogFieldOriginCertPath, originCertPath).
Logger()
originCertPath, err := findOriginCert(originCertPath, &originCertLog)
if err != nil {
return nil, errors.Wrap(err, "Error locating origin cert")
}
blocks, err := readOriginCert(originCertPath, sc.logger)
blocks, err := readOriginCert(originCertPath)
if err != nil {
return nil, errors.Wrapf(err, "Can't read origin cert from %s", originCertPath)
}
@@ -108,56 +124,27 @@ func (sc *subcommandContext) credential() (*userCredential, error) {
return sc.userCredential, nil
}
func (sc *subcommandContext) readTunnelCredentials(tunnelID uuid.UUID) (*pogs.TunnelAuth, error) {
filePath, err := sc.tunnelCredentialsPath(tunnelID)
func (sc *subcommandContext) readTunnelCredentials(credFinder CredFinder) (connection.Credentials, error) {
filePath, err := credFinder.Path()
if err != nil {
return nil, err
return connection.Credentials{}, err
}
body, err := ioutil.ReadFile(filePath)
body, err := sc.fs.readFile(filePath)
if err != nil {
return nil, errors.Wrapf(err, "couldn't read tunnel credentials from %v", filePath)
return connection.Credentials{}, errors.Wrapf(err, "couldn't read tunnel credentials from %v", filePath)
}
var auth pogs.TunnelAuth
if err = json.Unmarshal(body, &auth); err != nil {
var credentials connection.Credentials
if err = json.Unmarshal(body, &credentials); err != nil {
if strings.HasSuffix(filePath, ".pem") {
return nil, fmt.Errorf("The tunnel credentials file should be .json but you gave a .pem. "+
"The tunnel credentials file was originally created by `cloudflared tunnel create` and named %s.json."+
"You may have accidentally used the filepath to cert.pem, which is generated by `cloudflared tunnel "+
"login`.", tunnelID)
return connection.Credentials{}, fmt.Errorf("The tunnel credentials file should be .json but you gave a .pem. " +
"The tunnel credentials file was originally created by `cloudflared tunnel create`. " +
"You may have accidentally used the filepath to cert.pem, which is generated by `cloudflared tunnel " +
"login`.")
}
return nil, errInvalidJSONCredential{path: filePath, err: err}
return connection.Credentials{}, errInvalidJSONCredential{path: filePath, err: err}
}
return &auth, nil
}
func (sc *subcommandContext) tunnelCredentialsPath(tunnelID uuid.UUID) (string, error) {
if filePath := sc.c.String("credentials-file"); filePath != "" {
if validFilePath(filePath) {
return filePath, nil
}
return "", fmt.Errorf("Tunnel credentials file %s doesn't exist or is not a file", filePath)
}
// Fallback to look for tunnel credentials in the origin cert directory
if originCertPath, err := findOriginCert(sc.c, sc.logger); err == nil {
originCertDir := filepath.Dir(originCertPath)
if filePath, err := tunnelFilePath(tunnelID, originCertDir); err == nil {
if validFilePath(filePath) {
return filePath, nil
}
}
}
// Last resort look under default config directories
for _, configDir := range config.DefaultConfigSearchDirectories() {
if filePath, err := tunnelFilePath(tunnelID, configDir); err == nil {
if validFilePath(filePath) {
return filePath, nil
}
}
}
return "", fmt.Errorf("Tunnel credentials file not found")
return credentials, nil
}
func (sc *subcommandContext) create(name string) (*tunnelstore.Tunnel, error) {
@@ -180,7 +167,14 @@ func (sc *subcommandContext) create(name string) (*tunnelstore.Tunnel, error) {
if err != nil {
return nil, err
}
if writeFileErr := writeTunnelCredentials(tunnel.ID, credential.cert.AccountID, credential.certPath, tunnelSecret, sc.logger); err != nil {
tunnelCredentials := connection.Credentials{
AccountTag: credential.cert.AccountID,
TunnelSecret: tunnelSecret,
TunnelID: tunnel.ID,
TunnelName: name,
}
filePath, writeFileErr := writeTunnelCredentials(credential.certPath, &tunnelCredentials)
if writeFileErr != nil {
var errorLines []string
errorLines = append(errorLines, fmt.Sprintf("Your tunnel '%v' was created with ID %v. However, cloudflared couldn't write to the tunnel credentials file at %v.json.", tunnel.Name, tunnel.ID, tunnel.ID))
errorLines = append(errorLines, fmt.Sprintf("The file-writing error is: %v", writeFileErr))
@@ -193,12 +187,13 @@ func (sc *subcommandContext) create(name string) (*tunnelstore.Tunnel, error) {
errorMsg := strings.Join(errorLines, "\n")
return nil, errors.New(errorMsg)
}
sc.log.Info().Msgf("Tunnel credentials written to %v. cloudflared chose this file based on where your origin certificate was found. Keep this file secret. To revoke these credentials, delete the tunnel.", filePath)
if outputFormat := sc.c.String(outputFormatFlag.Name); outputFormat != "" {
return nil, renderOutput(outputFormat, &tunnel)
}
sc.logger.Infof("Created tunnel %s with id %s", tunnel.Name, tunnel.ID)
sc.log.Info().Msgf("Created tunnel %s with id %s", tunnel.Name, tunnel.ID)
return tunnel, nil
}
@@ -243,25 +238,35 @@ func (sc *subcommandContext) delete(tunnelIDs []uuid.UUID) error {
return errors.Wrapf(err, "Error deleting tunnel %s", tunnel.ID)
}
tunnelCredentialsPath, err := sc.tunnelCredentialsPath(tunnel.ID)
if err != nil {
sc.logger.Infof("Cannot locate tunnel credentials to delete, error: %v. Please delete the file manually", err)
return nil
}
if err = os.Remove(tunnelCredentialsPath); err != nil {
sc.logger.Infof("Cannot delete tunnel credentials, error: %v. Please delete the file manually", err)
credFinder := sc.credentialFinder(id)
if tunnelCredentialsPath, err := credFinder.Path(); err == nil {
if err = os.Remove(tunnelCredentialsPath); err != nil {
sc.log.Info().Msgf("Tunnel %v was deleted, but we could not remove its credentials file %s: %s. Consider deleting this file manually.", id, tunnelCredentialsPath, err)
}
}
}
return nil
}
// findCredentials will choose the right way to find the credentials file, find it,
// and add the TunnelID into any old credentials (generated before TUN-3581 added the `TunnelID`
// field to credentials files)
func (sc *subcommandContext) findCredentials(tunnelID uuid.UUID) (connection.Credentials, error) {
credFinder := sc.credentialFinder(tunnelID)
credentials, err := sc.readTunnelCredentials(credFinder)
// This line ensures backwards compatibility with credentials files generated before
// TUN-3581. Those old credentials files don't have a TunnelID field, so we enrich the struct
// with the ID, which we have already resolved from the user input.
credentials.TunnelID = tunnelID
return credentials, err
}
func (sc *subcommandContext) run(tunnelID uuid.UUID) error {
credentials, err := sc.readTunnelCredentials(tunnelID)
credentials, err := sc.findCredentials(tunnelID)
if err != nil {
if e, ok := err.(errInvalidJSONCredential); ok {
sc.logger.Errorf("The credentials file at %s contained invalid JSON. This is probably caused by passing the wrong filepath. Reminder: the credentials file is a .json file created via `cloudflared tunnel create`.", e.path)
sc.logger.Errorf("Invalid JSON when parsing credentials file: %s", e.err.Error())
sc.log.Error().Msgf("The credentials file at %s contained invalid JSON. This is probably caused by passing the wrong filepath. Reminder: the credentials file is a .json file created via `cloudflared tunnel create`.", e.path)
sc.log.Error().Msgf("Invalid JSON when parsing credentials file: %s", e.err.Error())
}
return err
}
@@ -271,8 +276,8 @@ func (sc *subcommandContext) run(tunnelID uuid.UUID) error {
version,
shutdownC,
graceShutdownC,
&connection.NamedTunnelConfig{Auth: *credentials, ID: tunnelID},
sc.logger,
&connection.NamedTunnelConfig{Credentials: credentials},
sc.log,
sc.isUIEnabled,
)
}
@@ -283,9 +288,9 @@ func (sc *subcommandContext) cleanupConnections(tunnelIDs []uuid.UUID) error {
return err
}
for _, tunnelID := range tunnelIDs {
sc.logger.Infof("Cleanup connection for tunnel %s", tunnelID)
sc.log.Info().Msgf("Cleanup connection for tunnel %s", tunnelID)
if err := client.CleanupConnections(tunnelID); err != nil {
sc.logger.Errorf("Error cleaning up connections for tunnel %v, error :%v", tunnelID, err)
sc.log.Error().Msgf("Error cleaning up connections for tunnel %v, error :%v", tunnelID, err)
}
}
return nil
@@ -300,6 +305,7 @@ func (sc *subcommandContext) route(tunnelID uuid.UUID, r tunnelstore.Route) (tun
return client.RouteTunnel(tunnelID, r)
}
// Query Tunnelstore to find the active tunnel with the given name.
func (sc *subcommandContext) tunnelActive(name string) (*tunnelstore.Tunnel, bool, error) {
filter := tunnelstore.NewFilter()
filter.NoDeleted()
@@ -322,6 +328,15 @@ func (sc *subcommandContext) findID(input string) (uuid.UUID, error) {
return u, nil
}
// Look up name in the credentials file.
credFinder := newStaticPath(sc.c.String(CredFileFlag), sc.fs)
if credentials, err := sc.readTunnelCredentials(credFinder); err == nil {
if credentials.TunnelID != uuid.Nil && input == credentials.TunnelName {
return credentials.TunnelID, nil
}
}
// Fall back to querying Tunnelstore.
if tunnel, found, err := sc.tunnelActive(input); err != nil {
return uuid.Nil, err
} else if found {
@@ -0,0 +1,42 @@
package tunnel
import (
"net"
"github.com/cloudflare/cloudflared/teamnet"
"github.com/pkg/errors"
)
const noClientMsg = "error while creating backend client"
func (sc *subcommandContext) listRoutes(filter *teamnet.Filter) ([]*teamnet.DetailedRoute, error) {
client, err := sc.client()
if err != nil {
return nil, errors.Wrap(err, noClientMsg)
}
return client.ListRoutes(filter)
}
func (sc *subcommandContext) addRoute(newRoute teamnet.NewRoute) (teamnet.Route, error) {
client, err := sc.client()
if err != nil {
return teamnet.Route{}, errors.Wrap(err, noClientMsg)
}
return client.AddRoute(newRoute)
}
func (sc *subcommandContext) deleteRoute(network net.IPNet) error {
client, err := sc.client()
if err != nil {
return errors.Wrap(err, noClientMsg)
}
return client.DeleteRoute(network)
}
func (sc *subcommandContext) getRouteByIP(ip net.IP) (teamnet.DetailedRoute, error) {
client, err := sc.client()
if err != nil {
return teamnet.DetailedRoute{}, errors.Wrap(err, noClientMsg)
}
return client.GetByIP(ip)
}
@@ -1,11 +1,19 @@
package tunnel
import (
"encoding/base64"
"flag"
"fmt"
"github.com/rs/zerolog"
"reflect"
"testing"
"time"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/tunnelstore"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/urfave/cli/v2"
)
func Test_findIDs(t *testing.T) {
@@ -80,3 +88,264 @@ func Test_findIDs(t *testing.T) {
})
}
}
type mockFileSystem struct {
rf func(string) ([]byte, error)
vfp func(string) bool
}
func (fs mockFileSystem) validFilePath(path string) bool {
return fs.vfp(path)
}
func (fs mockFileSystem) readFile(filePath string) ([]byte, error) {
return fs.rf(filePath)
}
func Test_subcommandContext_findCredentials(t *testing.T) {
type fields struct {
c *cli.Context
log *zerolog.Logger
isUIEnabled bool
fs fileSystem
tunnelstoreClient tunnelstore.Client
userCredential *userCredential
}
type args struct {
tunnelID uuid.UUID
}
oldCertPath := "old_cert.json"
newCertPath := "new_cert.json"
accountTag := "0000d4d14e84bd4ae5a6a02e0000ac63"
secret := []byte{211, 79, 177, 245, 179, 194, 152, 127, 140, 71, 18, 46, 183, 209, 10, 24, 192, 150, 55, 249, 211, 16, 167, 30, 113, 51, 152, 168, 72, 100, 205, 144}
secretB64 := base64.StdEncoding.EncodeToString(secret)
tunnelID := uuid.MustParse("df5ed608-b8b4-4109-89f3-9f2cf199df64")
name := "mytunnel"
fs := mockFileSystem{
rf: func(filePath string) ([]byte, error) {
if filePath == oldCertPath {
// An old credentials file created before TUN-3581 added the new fields
return []byte(fmt.Sprintf(`{"AccountTag":"%s","TunnelSecret":"%s"}`, accountTag, secretB64)), nil
}
if filePath == newCertPath {
// A new credentials file created after TUN-3581 with its new fields.
return []byte(fmt.Sprintf(`{"AccountTag":"%s","TunnelSecret":"%s","TunnelID":"%s","TunnelName":"%s"}`, accountTag, secretB64, tunnelID, name)), nil
}
return nil, errors.New("file not found")
},
vfp: func(string) bool { return true },
}
log := zerolog.Nop()
tests := []struct {
name string
fields fields
args args
want connection.Credentials
wantErr bool
}{
{
name: "Filepath given leads to old credentials file",
fields: fields{
log: &log,
fs: fs,
c: func() *cli.Context {
flagSet := flag.NewFlagSet("test0", flag.PanicOnError)
flagSet.String(CredFileFlag, oldCertPath, "")
c := cli.NewContext(cli.NewApp(), flagSet, nil)
_ = c.Set(CredFileFlag, oldCertPath)
return c
}(),
},
args: args{
tunnelID: tunnelID,
},
want: connection.Credentials{
AccountTag: accountTag,
TunnelID: tunnelID,
TunnelSecret: secret,
},
},
{
name: "Filepath given leads to new credentials file",
fields: fields{
log: &log,
fs: fs,
c: func() *cli.Context {
flagSet := flag.NewFlagSet("test0", flag.PanicOnError)
flagSet.String(CredFileFlag, newCertPath, "")
c := cli.NewContext(cli.NewApp(), flagSet, nil)
_ = c.Set(CredFileFlag, newCertPath)
return c
}(),
},
args: args{
tunnelID: tunnelID,
},
want: connection.Credentials{
AccountTag: accountTag,
TunnelID: tunnelID,
TunnelSecret: secret,
TunnelName: name,
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sc := &subcommandContext{
c: tt.fields.c,
log: tt.fields.log,
isUIEnabled: tt.fields.isUIEnabled,
fs: tt.fields.fs,
tunnelstoreClient: tt.fields.tunnelstoreClient,
userCredential: tt.fields.userCredential,
}
got, err := sc.findCredentials(tt.args.tunnelID)
if (err != nil) != tt.wantErr {
t.Errorf("subcommandContext.findCredentials() error = %v, wantErr %v", err, tt.wantErr)
return
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("subcommandContext.findCredentials() = %v, want %v", got, tt.want)
}
})
}
}
type deleteMockTunnelStore struct {
tunnelstore.Client
mockTunnels map[uuid.UUID]mockTunnelBehaviour
deletedTunnelIDs []uuid.UUID
}
type mockTunnelBehaviour struct {
tunnel tunnelstore.Tunnel
deleteErr error
cleanupErr error
}
func newDeleteMockTunnelStore(tunnels ...mockTunnelBehaviour) *deleteMockTunnelStore {
mockTunnels := make(map[uuid.UUID]mockTunnelBehaviour)
for _, tunnel := range tunnels {
mockTunnels[tunnel.tunnel.ID] = tunnel
}
return &deleteMockTunnelStore{
mockTunnels: mockTunnels,
deletedTunnelIDs: make([]uuid.UUID, 0),
}
}
func (d *deleteMockTunnelStore) GetTunnel(tunnelID uuid.UUID) (*tunnelstore.Tunnel, error) {
tunnel, ok := d.mockTunnels[tunnelID]
if !ok {
return nil, fmt.Errorf("Couldn't find tunnel: %v", tunnelID)
}
return &tunnel.tunnel, nil
}
func (d *deleteMockTunnelStore) DeleteTunnel(tunnelID uuid.UUID) error {
tunnel, ok := d.mockTunnels[tunnelID]
if !ok {
return fmt.Errorf("Couldn't find tunnel: %v", tunnelID)
}
if tunnel.deleteErr != nil {
return tunnel.deleteErr
}
d.deletedTunnelIDs = append(d.deletedTunnelIDs, tunnelID)
tunnel.tunnel.DeletedAt = time.Now()
delete(d.mockTunnels, tunnelID)
return nil
}
func (d *deleteMockTunnelStore) CleanupConnections(tunnelID uuid.UUID) error {
tunnel, ok := d.mockTunnels[tunnelID]
if !ok {
return fmt.Errorf("Couldn't find tunnel: %v", tunnelID)
}
return tunnel.cleanupErr
}
func Test_subcommandContext_Delete(t *testing.T) {
type fields struct {
c *cli.Context
log *zerolog.Logger
isUIEnabled bool
fs fileSystem
tunnelstoreClient *deleteMockTunnelStore
userCredential *userCredential
}
type args struct {
tunnelIDs []uuid.UUID
}
newCertPath := "new_cert.json"
tunnelID1 := uuid.MustParse("df5ed608-b8b4-4109-89f3-9f2cf199df64")
tunnelID2 := uuid.MustParse("af5ed608-b8b4-4109-89f3-9f2cf199df64")
log := zerolog.Nop()
var tests = []struct {
name string
fields fields
args args
want []uuid.UUID
wantErr bool
}{
{
name: "clean up continues if credentials are not found",
fields: fields{
log: &log,
fs: mockFileSystem{
rf: func(filePath string) ([]byte, error) {
return nil, errors.New("file not found")
},
vfp: func(string) bool { return true },
},
c: func() *cli.Context {
flagSet := flag.NewFlagSet("test0", flag.PanicOnError)
flagSet.String(CredFileFlag, newCertPath, "")
c := cli.NewContext(cli.NewApp(), flagSet, nil)
_ = c.Set(CredFileFlag, newCertPath)
return c
}(),
tunnelstoreClient: newDeleteMockTunnelStore(
mockTunnelBehaviour{
tunnel: tunnelstore.Tunnel{ID: tunnelID1},
},
mockTunnelBehaviour{
tunnel: tunnelstore.Tunnel{ID: tunnelID2},
},
),
},
args: args{
tunnelIDs: []uuid.UUID{tunnelID1, tunnelID2},
},
want: []uuid.UUID{tunnelID1, tunnelID2},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sc := &subcommandContext{
c: tt.fields.c,
log: tt.fields.log,
isUIEnabled: tt.fields.isUIEnabled,
fs: tt.fields.fs,
tunnelstoreClient: tt.fields.tunnelstoreClient,
userCredential: tt.fields.userCredential,
}
err := sc.delete(tt.args.tunnelIDs)
if (err != nil) != tt.wantErr {
t.Errorf("subcommandContext.findCredentials() error = %v, wantErr %v", err, tt.wantErr)
return
}
got := tt.fields.tunnelstoreClient.deletedTunnelIDs
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("subcommandContext.findCredentials() = %v, want %v", got, tt.want)
return
}
})
}
}
+42 -43
View File
@@ -24,13 +24,14 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/cloudflare/cloudflared/tunnelstore"
)
const (
credFileFlagAlias = "cred-file"
CredFileFlagAlias = "cred-file"
CredFileFlag = "credentials-file"
LogFieldTunnelID = "tunnelID"
)
var (
@@ -75,8 +76,8 @@ var (
"tunnels, you can do so with Cloudflare's Load Balancer product.",
})
credentialsFileFlag = altsrc.NewStringFlag(&cli.StringFlag{
Name: "credentials-file",
Aliases: []string{credFileFlagAlias},
Name: CredFileFlag,
Aliases: []string{CredFileFlagAlias},
Usage: "File path of tunnel credentials",
EnvVars: []string{"TUNNEL_CRED_FILE"},
})
@@ -103,7 +104,7 @@ func buildCreateCommand() *cli.Command {
Usage: "Create a new tunnel with given name",
UsageText: "cloudflared tunnel [tunnel command options] create [subcommand options] NAME",
Description: `Creates a tunnel, registers it with Cloudflare edge and generates credential file used to run this tunnel.
Use "cloudflared tunnel route" subcommand to map a DNS name to this tunnel and "cloudflared tunnel run" to start the connection.
Use "cloudflared tunnel route" subcommand to map a DNS name to this tunnel and "cloudflared tunnel run" to start the connection.
For example, to create a tunnel named 'my-tunnel' run:
@@ -141,30 +142,21 @@ func tunnelFilePath(tunnelID uuid.UUID, directory string) (string, error) {
return homedir.Expand(filePath)
}
func writeTunnelCredentials(tunnelID uuid.UUID, accountID, originCertPath string, tunnelSecret []byte, logger logger.Service) error {
func writeTunnelCredentials(
originCertPath string,
credentials *connection.Credentials,
) (filePath string, err error) {
originCertDir := filepath.Dir(originCertPath)
filePath, err := tunnelFilePath(tunnelID, originCertDir)
filePath, err = tunnelFilePath(credentials.TunnelID, originCertDir)
if err != nil {
return err
return "", err
}
body, err := json.Marshal(pogs.TunnelAuth{
AccountTag: accountID,
TunnelSecret: tunnelSecret,
})
// Write the name and ID to the file too
body, err := json.Marshal(credentials)
if err != nil {
return errors.Wrap(err, "Unable to marshal tunnel credentials to JSON")
return "", errors.Wrap(err, "Unable to marshal tunnel credentials to JSON")
}
logger.Infof("Writing tunnel credentials to %v. cloudflared chose this file based on where your origin certificate was found.", filePath)
logger.Infof("Keep this file secret. To revoke these credentials, delete the tunnel.")
return ioutil.WriteFile(filePath, body, 400)
}
func validFilePath(path string) bool {
fileStat, err := os.Stat(path)
if err != nil {
return false
}
return !fileStat.IsDir()
return filePath, ioutil.WriteFile(filePath, body, 400)
}
func buildListCommand() *cli.Command {
@@ -213,14 +205,14 @@ func listCommand(c *cli.Context) error {
}
if len(tunnels) > 0 {
fmtAndPrintTunnelList(tunnels, c.Bool("show-recently-disconnected"))
formatAndPrintTunnelList(tunnels, c.Bool("show-recently-disconnected"))
} else {
fmt.Println("You have no tunnels, use 'cloudflared tunnel create' to define a new tunnel")
}
return nil
}
func fmtAndPrintTunnelList(tunnels []*tunnelstore.Tunnel, showRecentlyDisconnected bool) {
func formatAndPrintTunnelList(tunnels []*tunnelstore.Tunnel, showRecentlyDisconnected bool) {
const (
minWidth = 0
tabWidth = 8
@@ -233,7 +225,7 @@ func fmtAndPrintTunnelList(tunnels []*tunnelstore.Tunnel, showRecentlyDisconnect
defer writer.Flush()
// Print column headers with tabbed columns
fmt.Fprintln(writer, "ID\tNAME\tCREATED\tCONNECTIONS\t")
_, _ = fmt.Fprintln(writer, "ID\tNAME\tCREATED\tCONNECTIONS\t")
// Loop through tunnels, create formatted string for each, and print using tabwriter
for _, t := range tunnels {
@@ -244,7 +236,7 @@ func fmtAndPrintTunnelList(tunnels []*tunnelstore.Tunnel, showRecentlyDisconnect
t.CreatedAt.Format(time.RFC3339),
fmtConnections(t.Connections, showRecentlyDisconnected),
)
fmt.Fprintln(writer, formattedStr)
_, _ = fmt.Fprintln(writer, formattedStr)
}
}
@@ -329,13 +321,13 @@ func buildRunCommand() *cli.Command {
Before: SetFlagsFromConfigFile,
Usage: "Proxy a local web server by running the given tunnel",
UsageText: "cloudflared tunnel [tunnel command options] run [subcommand options] [TUNNEL]",
Description: `Runs the tunnel identified by name or UUUD, creating highly available connections
Description: `Runs the tunnel identified by name or UUUD, creating highly available connections
between your server and the Cloudflare edge. You can provide name or UUID of tunnel to run either as the
last command line argument or in the configuration file using "tunnel: TUNNEL".
This command requires the tunnel credentials file created when "cloudflared tunnel create" was run,
This command requires the tunnel credentials file created when "cloudflared tunnel create" was run,
however it does not need access to cert.pem from "cloudflared login" if you identify the tunnel by UUID.
If you experience other problems running the tunnel, "cloudflared tunnel cleanup" may help by removing
If you experience other problems running the tunnel, "cloudflared tunnel cleanup" may help by removing
any old connection records.
`,
Flags: flags,
@@ -370,7 +362,7 @@ func runNamedTunnel(sc *subcommandContext, tunnelRef string) error {
return errors.Wrap(err, "error parsing tunnel ID")
}
sc.logger.Infof("Starting tunnel %s", tunnelID.String())
sc.log.Info().Str(LogFieldTunnelID, tunnelID.String()).Msg("Starting tunnel")
return sc.run(tunnelID)
}
@@ -417,6 +409,9 @@ func buildRouteCommand() *cli.Command {
To use this tunnel as a load balancer origin, creating pool and load balancer if necessary:
cloudflared tunnel route lb <tunnel ID> <load balancer name> <load balancer pool>`,
CustomHelpTemplate: commandHelpTemplate(),
Subcommands: []*cli.Command{
buildRouteIPSubcommand(),
},
}
}
@@ -431,7 +426,7 @@ func dnsRouteFromArg(c *cli.Context) (tunnelstore.Route, error) {
userHostname := c.Args().Get(userHostnameIndex)
if userHostname == "" {
return nil, cliutil.UsageError("The third argument should be the hostname")
} else if !validateHostname(userHostname) {
} else if !validateHostname(userHostname, true) {
return nil, errors.Errorf("%s is not a valid hostname", userHostname)
}
return tunnelstore.NewDNSRoute(userHostname), nil
@@ -449,14 +444,14 @@ func lbRouteFromArg(c *cli.Context) (tunnelstore.Route, error) {
lbName := c.Args().Get(lbNameIndex)
if lbName == "" {
return nil, cliutil.UsageError("The third argument should be the load balancer name")
} else if !validateHostname(lbName) {
} else if !validateHostname(lbName, true) {
return nil, errors.Errorf("%s is not a valid load balancer name", lbName)
}
lbPool := c.Args().Get(lbPoolIndex)
if lbPool == "" {
return nil, cliutil.UsageError("The fourth argument should be the pool name")
} else if !validateName(lbPool) {
} else if !validateName(lbPool, false) {
return nil, errors.Errorf("%s is not a valid pool name", lbPool)
}
@@ -464,19 +459,23 @@ func lbRouteFromArg(c *cli.Context) (tunnelstore.Route, error) {
}
var nameRegex = regexp.MustCompile("^[_a-zA-Z0-9][-_.a-zA-Z0-9]*$")
var hostNameRegex = regexp.MustCompile("^[*_a-zA-Z0-9][-_.a-zA-Z0-9]*$")
func validateName(s string) bool {
func validateName(s string, allowWildcardSubdomain bool) bool {
if allowWildcardSubdomain {
return hostNameRegex.MatchString(s)
}
return nameRegex.MatchString(s)
}
func validateHostname(s string) bool {
func validateHostname(s string, allowWildcardSubdomain bool) bool {
// Slightly stricter than PunyCodeProfile
idnaProfile := idna.New(
idna.ValidateLabels(true),
idna.VerifyDNSLength(true))
puny, err := idnaProfile.ToASCII(s)
return err == nil && validateName(puny)
return err == nil && validateName(puny, allowWildcardSubdomain)
}
func routeCommand(c *cli.Context) error {
@@ -521,7 +520,7 @@ func routeCommand(c *cli.Context) error {
return err
}
sc.logger.Infof(res.SuccessSummary())
sc.log.Info().Str(LogFieldTunnelID, tunnelID.String()).Msg(res.SuccessSummary())
return nil
}
@@ -535,13 +534,13 @@ func commandHelpTemplate() string {
}
const template = `NAME:
{{.HelpName}} - {{.Usage}}
USAGE:
{{.UsageText}}
DESCRIPTION:
{{.Description}}
TUNNEL COMMAND OPTIONS:
%s
SUBCOMMAND OPTIONS:
+59 -4
View File
@@ -6,9 +6,8 @@ import (
"testing"
"github.com/cloudflare/cloudflared/tunnelstore"
"github.com/mitchellh/go-homedir"
"github.com/google/uuid"
"github.com/mitchellh/go-homedir"
"github.com/stretchr/testify/assert"
)
@@ -117,8 +116,64 @@ func TestValidateName(t *testing.T) {
{name: "_ab_c.-d-ef", want: true},
}
for _, tt := range tests {
if got := validateName(tt.name); got != tt.want {
if got := validateName(tt.name, false); got != tt.want {
t.Errorf("validateName() = %v, want %v", got, tt.want)
}
}
}
}
func Test_validateHostname(t *testing.T) {
type args struct {
s string
allowWildcardSubdomain bool
}
tests := []struct {
name string
args args
want bool
}{
{
name: "Normal",
args: args{
s: "example.com",
allowWildcardSubdomain: true,
},
want: true,
},
{
name: "wildcard subdomain for TUN-358",
args: args{
s: "*.ehrig.io",
allowWildcardSubdomain: true,
},
want: true,
},
{
name: "Misplaced wildcard",
args: args{
s: "subdomain.*.ehrig.io",
allowWildcardSubdomain: true,
},
},
{
name: "Invalid domain",
args: args{
s: "..",
allowWildcardSubdomain: true,
},
},
{
name: "Invalid domain",
args: args{
s: "..",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := validateHostname(tt.args.s, tt.args.allowWildcardSubdomain); got != tt.want {
t.Errorf("validateHostname() = %v, want %v", got, tt.want)
}
})
}
}
@@ -0,0 +1,198 @@
package tunnel
import (
"fmt"
"net"
"os"
"text/tabwriter"
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/teamnet"
"github.com/pkg/errors"
"github.com/urfave/cli/v2"
)
func buildRouteIPSubcommand() *cli.Command {
return &cli.Command{
Name: "ip",
Category: "Tunnel",
Usage: "Configure and query private routes",
UsageText: "cloudflared tunnel [--config FILEPATH] route COMMAND [arguments...]",
Hidden: true,
Description: `cloudflared can provision private routes from your private IP space to origins
in your corporate network. Users enrolled in your Cloudflare for Teams organization can reach
those routes through the Cloudflare Warp client. You can also build rules to determine who
can reach certain routes.
`,
Subcommands: []*cli.Command{
{
Name: "add",
Action: cliutil.ErrorHandler(addRouteCommand),
Usage: "Add a new Teamnet route to the table",
UsageText: "cloudflared tunnel [--config FILEPATH] route ip add [CIDR] [TUNNEL] [COMMENT?]",
Description: `Adds a private route to a CIDR in your private IP space. Requests will
be sent through the Cloudflare Warp client running on a user's machine, proxied
through the specified tunnel, and reach an IP in the given CIDR.`,
},
{
Name: "show",
Action: cliutil.ErrorHandler(showRoutesCommand),
Usage: "Show the routing table",
UsageText: "cloudflared tunnel [--config FILEPATH] route ip show [flags]",
Description: `Shows your organization's private route table. You can use flags to filter the results.`,
Flags: teamnet.FilterFlags,
},
{
Name: "delete",
Action: cliutil.ErrorHandler(deleteRouteCommand),
Usage: "Delete a row from your organization's private routing table",
UsageText: "cloudflared tunnel [--config FILEPATH] route ip delete [CIDR]",
Description: `Deletes the row for a given CIDR from your routing table`,
},
{
Name: "get",
Action: cliutil.ErrorHandler(getRouteByIPCommand),
Usage: "Check which row of the routing table matches a given IP",
UsageText: "cloudflared tunnel [--config FILEPATH] route ip get [IP]",
Description: `Checks which row of the routing table will be used to proxy a given IP.
This helps check and validate your config.`,
},
},
}
}
func showRoutesCommand(c *cli.Context) error {
sc, err := newSubcommandContext(c)
if err != nil {
return err
}
filter, err := teamnet.NewFromCLI(c)
if err != nil {
return errors.Wrap(err, "invalid config for routing filters")
}
routes, err := sc.listRoutes(filter)
if err != nil {
return err
}
if outputFormat := c.String(outputFormatFlag.Name); outputFormat != "" {
return renderOutput(outputFormat, routes)
}
if len(routes) > 0 {
formatAndPrintRouteList(routes)
} else {
fmt.Println("You have no routes, use 'cloudflared tunnel route ip add' to add a route")
}
return nil
}
func addRouteCommand(c *cli.Context) error {
sc, err := newSubcommandContext(c)
if err != nil {
return err
}
if c.NArg() < 2 {
return errors.New("You must supply at least 2 arguments, first the network you wish to route (in CIDR form e.g. 1.2.3.4/32) and then the tunnel ID to proxy with")
}
args := c.Args()
_, network, err := net.ParseCIDR(args.Get(0))
if err != nil {
return errors.Wrap(err, "Invalid network CIDR")
}
if network == nil {
return errors.New("Invalid network CIDR")
}
tunnelRef := args.Get(1)
tunnelID, err := sc.findID(tunnelRef)
if err != nil {
return errors.Wrap(err, "Invalid tunnel")
}
comment := ""
if c.NArg() >= 3 {
comment = args.Get(2)
}
_, err = sc.addRoute(teamnet.NewRoute{
Comment: comment,
Network: *network,
TunnelID: tunnelID,
})
if err != nil {
return errors.Wrap(err, "API error")
}
fmt.Printf("Successfully added route for %s over tunnel %s\n", network, tunnelID)
return nil
}
func deleteRouteCommand(c *cli.Context) error {
sc, err := newSubcommandContext(c)
if err != nil {
return err
}
if c.NArg() != 1 {
return errors.New("You must supply exactly one argument, the network whose route you want to delete (in CIDR form e.g. 1.2.3.4/32)")
}
_, network, err := net.ParseCIDR(c.Args().First())
if err != nil {
return errors.Wrap(err, "Invalid network CIDR")
}
if network == nil {
return errors.New("Invalid network CIDR")
}
if err := sc.deleteRoute(*network); err != nil {
return errors.Wrap(err, "API error")
}
fmt.Printf("Successfully deleted route for %s\n", network)
return nil
}
func getRouteByIPCommand(c *cli.Context) error {
sc, err := newSubcommandContext(c)
if err != nil {
return err
}
if c.NArg() != 1 {
return errors.New("You must supply exactly one argument, an IP whose route will be queried (e.g. 1.2.3.4 or 2001:0db8:::7334)")
}
ipInput := c.Args().First()
ip := net.ParseIP(ipInput)
if ip == nil {
return fmt.Errorf("Invalid IP %s", ipInput)
}
route, err := sc.getRouteByIP(ip)
if err != nil {
return errors.Wrap(err, "API error")
}
if route.IsZero() {
fmt.Printf("No route matches the IP %s\n", ip)
} else {
formatAndPrintRouteList([]*teamnet.DetailedRoute{&route})
}
return nil
}
func formatAndPrintRouteList(routes []*teamnet.DetailedRoute) {
const (
minWidth = 0
tabWidth = 8
padding = 1
padChar = ' '
flags = 0
)
writer := tabwriter.NewWriter(os.Stdout, minWidth, tabWidth, padding, padChar, flags)
defer writer.Flush()
// Print column headers with tabbed columns
_, _ = fmt.Fprintln(writer, "NETWORK\tCOMMENT\tTUNNEL ID\tTUNNEL NAME\tCREATED\tDELETED\t")
// Loop through routes, create formatted string for each, and print using tabwriter
for _, route := range routes {
formattedStr := route.TableString()
_, _ = fmt.Fprintln(writer, formattedStr)
}
}
+28 -45
View File
@@ -4,35 +4,18 @@ import (
"context"
"fmt"
"strings"
"time"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/logger"
"github.com/gdamore/tcell"
"github.com/rivo/tview"
"github.com/rs/zerolog"
)
type connState struct {
location string
state status
}
type status int
const (
Disconnected status = iota
Connected
Reconnecting
SetUrl
RegisteringTunnel
)
type TunnelEvent struct {
Index uint8
EventType status
Location string
Url string
state connection.Status
}
type uiModel struct {
@@ -67,16 +50,16 @@ func NewUIModel(version, hostname, metricsURL string, ing *ingress.Ingress, haCo
func (data *uiModel) LaunchUI(
ctx context.Context,
generalLogger, transportLogger logger.Service,
logLevels []logger.Level,
tunnelEventChan <-chan TunnelEvent,
log, transportLog *zerolog.Logger,
tunnelEventChan <-chan connection.Event,
) {
// Configure the logger to stream logs into the textview
// Add TextView as a group to write output to
logTextView := NewDynamicColorTextView()
generalLogger.Add(logTextView, logger.NewUIFormatter(time.RFC3339), logLevels...)
transportLogger.Add(logTextView, logger.NewUIFormatter(time.RFC3339), logLevels...)
// TODO: Format log for UI
//log.Add(logTextView, logger.NewUIFormatter(time.RFC3339), logLevels...)
//transportLog.Add(logTextView, logger.NewUIFormatter(time.RFC3339), logLevels...)
// Construct the UI
palette := palette{
@@ -138,14 +121,14 @@ func (data *uiModel) LaunchUI(
return
case event := <-tunnelEventChan:
switch event.EventType {
case Connected:
case connection.Connected:
data.setConnTableCell(event, connTable, palette)
case Disconnected, Reconnecting:
data.changeConnStatus(event, connTable, generalLogger, palette)
case SetUrl:
tunnelHostText.SetText(event.Url)
data.edgeURL = event.Url
case RegisteringTunnel:
case connection.Disconnected, connection.Reconnecting:
data.changeConnStatus(event, connTable, log, palette)
case connection.SetURL:
tunnelHostText.SetText(event.URL)
data.edgeURL = event.URL
case connection.RegisteringTunnel:
if data.edgeURL == "" {
tunnelHostText.SetText("Registering tunnel...")
}
@@ -157,7 +140,7 @@ func (data *uiModel) LaunchUI(
go func() {
if err := app.SetRoot(frame, true).Run(); err != nil {
generalLogger.Errorf("Error launching UI: %s", err)
log.Error().Msgf("Error launching UI: %s", err)
}
}()
}
@@ -175,22 +158,22 @@ func handleNewText(app *tview.Application, logTextView *tview.TextView) func() {
}
}
func (data *uiModel) changeConnStatus(event TunnelEvent, table *tview.Table, logger logger.Service, palette palette) {
func (data *uiModel) changeConnStatus(event connection.Event, table *tview.Table, log *zerolog.Logger, palette palette) {
index := int(event.Index)
// Get connection location and state
connState := data.getConnState(index)
// Check if connection is already displayed in UI
if connState == nil {
logger.Info("Connection is not in the UI table")
log.Info().Msg("Connection is not in the UI table")
return
}
locationState := event.Location
if event.EventType == Disconnected {
connState.state = Disconnected
} else if event.EventType == Reconnecting {
connState.state = Reconnecting
if event.EventType == connection.Disconnected {
connState.state = connection.Disconnected
} else if event.EventType == connection.Reconnecting {
connState.state = connection.Reconnecting
locationState = "Reconnecting..."
}
@@ -211,12 +194,12 @@ func (data *uiModel) getConnState(connID int) *connState {
return nil
}
func (data *uiModel) setConnTableCell(event TunnelEvent, table *tview.Table, palette palette) {
func (data *uiModel) setConnTableCell(event connection.Event, table *tview.Table, palette palette) {
index := int(event.Index)
connectionNum := index + 1
// Update slice to keep track of connection location and state in UI table
data.connections[index].state = Connected
data.connections[index].state = connection.Connected
data.connections[index].location = event.Location
// Update text in table cell to show disconnected state
@@ -225,18 +208,18 @@ func (data *uiModel) setConnTableCell(event TunnelEvent, table *tview.Table, pal
table.SetCell(index, 0, cell)
}
func newCellText(palette palette, connectionNum int, location string, connectedStatus status) string {
func newCellText(palette palette, connectionNum int, location string, connectedStatus connection.Status) string {
// HA connection indicator formatted as: "• #<CONNECTION_INDEX>: <COLO>",
// where the left middle dot's color depends on the status of the connection
const connFmtString = "[%s]\u2022[%s] #%d: %s"
var dotColor string
switch connectedStatus {
case Connected:
case connection.Connected:
dotColor = palette.connected
case Disconnected:
case connection.Disconnected:
dotColor = palette.disconnected
case Reconnecting:
case connection.Reconnecting:
dotColor = palette.reconnecting
}
+26 -29
View File
@@ -3,6 +3,7 @@ package updater
import (
"context"
"fmt"
"github.com/rs/zerolog"
"os"
"path/filepath"
"runtime"
@@ -14,18 +15,18 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/logger"
"github.com/facebookgo/grace/gracenet"
"github.com/pkg/errors"
)
const (
DefaultCheckUpdateFreq = time.Hour * 24
appID = "app_idCzgxYerVD"
noUpdateInShellMessage = "cloudflared will not automatically update when run from the shell. To enable auto-updates, run cloudflared as a service: https://developers.cloudflare.com/argo-tunnel/reference/service/"
noUpdateOnWindowsMessage = "cloudflared will not automatically update on Windows systems."
noUpdateManagedPackageMessage = "cloudflared will not automatically update if installed by a package manager."
isManagedInstallFile = ".installedFromPackageManager"
UpdateURL = "https://update.argotunnel.com"
StagingUpdateURL = "https://staging-update.argotunnel.com"
LogFieldVersion = "version"
)
var (
@@ -114,38 +115,35 @@ func checkForUpdateAndApply(options updateOptions) UpdateOutcome {
// Update is the handler for the update command from the command line
func Update(c *cli.Context) error {
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
if wasInstalledFromPackageManager() {
logger.Error("cloudflared was installed by a package manager. Please update using the same method.")
log.Error().Msg("cloudflared was installed by a package manager. Please update using the same method.")
return nil
}
isBeta := c.Bool("beta")
if isBeta {
logger.Info("cloudflared is set to update to the latest beta version")
log.Info().Msg("cloudflared is set to update to the latest beta version")
}
isStaging := c.Bool("staging")
if isStaging {
logger.Info("cloudflared is set to update from staging")
log.Info().Msg("cloudflared is set to update from staging")
}
isForced := c.Bool("force")
if isForced {
logger.Info("cloudflared is set to upgrade to the latest publish version regardless of the current version")
log.Info().Msg("cloudflared is set to upgrade to the latest publish version regardless of the current version")
}
updateOutcome := loggedUpdate(logger, updateOptions{isBeta: isBeta, isStaging: isStaging, isForced: isForced, version: c.String("version")})
updateOutcome := loggedUpdate(log, updateOptions{isBeta: isBeta, isStaging: isStaging, isForced: isForced, version: c.String("version")})
if updateOutcome.Error != nil {
return &statusErr{updateOutcome.Error}
}
if updateOutcome.noUpdate() {
logger.Infof("cloudflared is up to date (%s)", updateOutcome.Version)
log.Info().Str(LogFieldVersion, updateOutcome.Version).Msg("cloudflared is up to date")
return nil
}
@@ -153,13 +151,13 @@ func Update(c *cli.Context) error {
}
// Checks for an update and applies it if one is available
func loggedUpdate(logger logger.Service, options updateOptions) UpdateOutcome {
func loggedUpdate(log *zerolog.Logger, options updateOptions) UpdateOutcome {
updateOutcome := checkForUpdateAndApply(options)
if updateOutcome.Updated {
logger.Infof("cloudflared has been updated to version %s", updateOutcome.Version)
log.Info().Str(LogFieldVersion, updateOutcome.Version).Msg("cloudflared has been updated")
}
if updateOutcome.Error != nil {
logger.Errorf("update check failed: %s", updateOutcome.Error)
log.Err(updateOutcome.Error).Msg("update check failed: %s")
}
return updateOutcome
@@ -170,7 +168,7 @@ type AutoUpdater struct {
configurable *configurable
listeners *gracenet.Net
updateConfigChan chan *configurable
logger logger.Service
log *zerolog.Logger
}
// AutoUpdaterConfigurable is the attributes of AutoUpdater that can be reconfigured during runtime
@@ -179,7 +177,7 @@ type configurable struct {
freq time.Duration
}
func NewAutoUpdater(freq time.Duration, listeners *gracenet.Net, logger logger.Service) *AutoUpdater {
func NewAutoUpdater(freq time.Duration, listeners *gracenet.Net, log *zerolog.Logger) *AutoUpdater {
updaterConfigurable := &configurable{
enabled: true,
freq: freq,
@@ -192,7 +190,7 @@ func NewAutoUpdater(freq time.Duration, listeners *gracenet.Net, logger logger.S
configurable: updaterConfigurable,
listeners: listeners,
updateConfigChan: make(chan *configurable),
logger: logger,
log: log,
}
}
@@ -200,20 +198,19 @@ func (a *AutoUpdater) Run(ctx context.Context) error {
ticker := time.NewTicker(a.configurable.freq)
for {
if a.configurable.enabled {
updateOutcome := loggedUpdate(a.logger, updateOptions{})
updateOutcome := loggedUpdate(a.log, updateOptions{})
if updateOutcome.Updated {
os.Args = append(os.Args, "--is-autoupdated=true")
if IsSysV() {
// SysV doesn't have a mechanism to keep service alive, we have to restart the process
a.logger.Info("Restarting service managed by SysV...")
a.log.Info().Msg("Restarting service managed by SysV...")
pid, err := a.listeners.StartProcess()
if err != nil {
a.logger.Errorf("Unable to restart server automatically: %s", err)
a.log.Err(err).Msg("Unable to restart server automatically")
return &statusErr{err: err}
}
// stop old process after autoupdate. Otherwise we create a new process
// after each update
a.logger.Infof("PID of the new process is %d", pid)
a.log.Info().Msgf("PID of the new process is %d", pid)
}
return &statusSuccess{newVersion: updateOutcome.Version}
}
@@ -245,26 +242,26 @@ func (a *AutoUpdater) Update(newFreq time.Duration) {
a.updateConfigChan <- newConfigurable
}
func IsAutoupdateEnabled(c *cli.Context, l logger.Service) bool {
if !SupportAutoUpdate(l) {
func IsAutoupdateEnabled(c *cli.Context, log *zerolog.Logger) bool {
if !SupportAutoUpdate(log) {
return false
}
return !c.Bool("no-autoupdate") && c.Duration("autoupdate-freq") != 0
}
func SupportAutoUpdate(logger logger.Service) bool {
func SupportAutoUpdate(log *zerolog.Logger) bool {
if runtime.GOOS == "windows" {
logger.Info(noUpdateOnWindowsMessage)
log.Info().Msg(noUpdateOnWindowsMessage)
return false
}
if wasInstalledFromPackageManager() {
logger.Info(noUpdateManagedPackageMessage)
log.Info().Msg(noUpdateManagedPackageMessage)
return false
}
if isRunningFromTerminal() {
logger.Info(noUpdateInShellMessage)
log.Info().Msg(noUpdateInShellMessage)
return false
}
return true
+3 -3
View File
@@ -4,15 +4,15 @@ import (
"context"
"testing"
"github.com/cloudflare/cloudflared/logger"
"github.com/facebookgo/grace/gracenet"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
)
func TestDisabledAutoUpdater(t *testing.T) {
listeners := &gracenet.Net{}
logger := logger.NewOutputWriter(logger.NewMockWriteManager())
autoupdater := NewAutoUpdater(0, listeners, logger)
log := zerolog.Nop()
autoupdater := NewAutoUpdater(0, listeners, &log)
ctx, cancel := context.WithCancel(context.Background())
errC := make(chan error)
go func() {
+28 -39
View File
@@ -13,9 +13,8 @@ import (
"unsafe"
"github.com/cloudflare/cloudflared/logger"
"github.com/pkg/errors"
cli "github.com/urfave/cli/v2"
"github.com/urfave/cli/v2"
"golang.org/x/sys/windows"
"golang.org/x/sys/windows/svc"
"golang.org/x/sys/windows/svc/eventlog"
@@ -37,6 +36,8 @@ const (
// ERROR_FAILED_SERVICE_CONTROLLER_CONNECT
// https://docs.microsoft.com/en-us/windows/desktop/debug/system-error-codes--1000-1299-
serviceControllerConnectionFailure = 1063
LogFieldWindowsServiceName = "windowsServiceName"
)
func runApp(app *cli.App, shutdownC, graceShutdownC chan struct{}) {
@@ -67,15 +68,11 @@ func runApp(app *cli.App, shutdownC, graceShutdownC chan struct{}) {
// 2. get ERROR_FAILED_SERVICE_CONTROLLER_CONNECT
// This involves actually trying to start the service.
logger, err := logger.New()
if err != nil {
os.Exit(1)
return
}
log := logger.Create(nil)
isIntSess, err := svc.IsAnInteractiveSession()
if err != nil {
logger.Fatalf("failed to determine if we are running in an interactive session: %v", err)
log.Fatal().Err(err).Msg("failed to determine if we are running in an interactive session")
}
if isIntSess {
app.Run(os.Args)
@@ -93,7 +90,7 @@ func runApp(app *cli.App, shutdownC, graceShutdownC chan struct{}) {
app.Run(os.Args)
return
}
logger.Fatalf("%s service failed: %v", windowsServiceName, err)
log.Fatal().Err(err).Msgf("%s service failed", windowsServiceName)
}
}
@@ -105,15 +102,10 @@ type windowsService struct {
// called by the package code at the start of the service
func (s *windowsService) Execute(serviceArgs []string, r <-chan svc.ChangeRequest, statusChan chan<- svc.Status) (ssec bool, errno uint32) {
logger, err := logger.New()
if err != nil {
os.Exit(1)
return
}
log := logger.Create(nil)
elog, err := eventlog.Open(windowsServiceName)
if err != nil {
logger.Errorf("Cannot open event log for %s with error: %s", windowsServiceName, err)
log.Err(err).Msgf("Cannot open event log for %s", windowsServiceName)
return
}
defer elog.Close()
@@ -173,79 +165,76 @@ func (s *windowsService) Execute(serviceArgs []string, r <-chan svc.ChangeReques
}
func installWindowsService(c *cli.Context) error {
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
zeroLogger := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog)
logger.Infof("Installing Argo Tunnel Windows service")
zeroLogger.Info().Msg("Installing Argo Tunnel Windows service")
exepath, err := os.Executable()
if err != nil {
logger.Errorf("Cannot find path name that start the process")
zeroLogger.Err(err).Msg("Cannot find path name that start the process")
return err
}
m, err := mgr.Connect()
if err != nil {
logger.Errorf("Cannot establish a connection to the service control manager: %s", err)
zeroLogger.Err(err).Msg("Cannot establish a connection to the service control manager")
return err
}
defer m.Disconnect()
s, err := m.OpenService(windowsServiceName)
log := zeroLogger.With().Str(LogFieldWindowsServiceName, windowsServiceName).Logger()
if err == nil {
s.Close()
logger.Errorf("service %s already exists", windowsServiceName)
log.Err(err).Msg("service already exists")
return fmt.Errorf("service %s already exists", windowsServiceName)
}
config := mgr.Config{StartType: mgr.StartAutomatic, DisplayName: windowsServiceDescription}
s, err = m.CreateService(windowsServiceName, exepath, config)
if err != nil {
logger.Errorf("Cannot install service %s", windowsServiceName)
log.Error().Msg("Cannot install service")
return err
}
defer s.Close()
logger.Infof("Argo Tunnel agent service is installed")
log.Info().Msg("Argo Tunnel agent service is installed")
err = eventlog.InstallAsEventCreate(windowsServiceName, eventlog.Error|eventlog.Warning|eventlog.Info)
if err != nil {
s.Delete()
logger.Errorf("Cannot install event logger: %s", err)
log.Err(err).Msg("Cannot install event logger")
return fmt.Errorf("SetupEventLogSource() failed: %s", err)
}
err = configRecoveryOption(s.Handle)
if err != nil {
logger.Errorf("Cannot set service recovery actions: %s", err)
logger.Infof("See %s to manually configure service recovery actions", windowsServiceUrl)
log.Err(err).Msg("Cannot set service recovery actions")
log.Info().Msgf("See %s to manually configure service recovery actions", windowsServiceUrl)
}
return nil
}
func uninstallWindowsService(c *cli.Context) error {
logger, err := logger.New()
if err != nil {
return errors.Wrap(err, "error setting up logger")
}
log := logger.CreateLoggerFromContext(c, logger.EnableTerminalLog).
With().
Str(LogFieldWindowsServiceName, windowsServiceName).Logger()
logger.Infof("Uninstalling Argo Tunnel Windows Service")
log.Info().Msg("Uninstalling Argo Tunnel Windows Service")
m, err := mgr.Connect()
if err != nil {
logger.Errorf("Cannot establish a connection to the service control manager")
log.Error().Msg("Cannot establish a connection to the service control manager")
return err
}
defer m.Disconnect()
s, err := m.OpenService(windowsServiceName)
if err != nil {
logger.Errorf("service %s is not installed", windowsServiceName)
log.Error().Msg("service is not installed")
return fmt.Errorf("service %s is not installed", windowsServiceName)
}
defer s.Close()
err = s.Delete()
if err != nil {
logger.Errorf("Cannot delete service %s", windowsServiceName)
log.Error().Msg("Cannot delete service")
return err
}
logger.Infof("Argo Tunnel agent service is uninstalled")
log.Info().Msg("Argo Tunnel agent service is uninstalled")
err = eventlog.Remove(windowsServiceName)
if err != nil {
logger.Errorf("Cannot remove event logger")
log.Error().Msg("Cannot remove event logger")
return fmt.Errorf("RemoveEventLogSource() failed: %s", err)
}
return nil
+25 -10
View File
@@ -11,9 +11,7 @@ import (
"github.com/google/uuid"
)
const (
lbProbeUserAgentPrefix = "Mozilla/5.0 (compatible; Cloudflare-Traffic-Manager/1.0; +https://www.cloudflare.com/traffic-manager/;"
)
const LogFieldConnIndex = "connIndex"
type Config struct {
OriginClient OriginClient
@@ -22,9 +20,23 @@ type Config struct {
}
type NamedTunnelConfig struct {
Auth pogs.TunnelAuth
ID uuid.UUID
Client pogs.ClientInfo
Credentials Credentials
Client pogs.ClientInfo
}
// Credentials are stored in the credentials file and contain all info needed to run a tunnel.
type Credentials struct {
AccountTag string
TunnelSecret []byte
TunnelID uuid.UUID
TunnelName string
}
func (c *Credentials) Auth() pogs.TunnelAuth {
return pogs.TunnelAuth{
AccountTag: c.AccountTag,
TunnelSecret: c.TunnelSecret,
}
}
type ClassicTunnelConfig struct {
@@ -53,10 +65,13 @@ type ConnectedFuse interface {
IsConnected() bool
}
func IsServerSentEvent(headers http.Header) bool {
if contentType := headers.Get("content-type"); contentType != "" {
return strings.HasPrefix(strings.ToLower(contentType), "text/event-stream")
}
return false
}
func uint8ToString(input uint8) string {
return strconv.FormatUint(uint64(input), 10)
}
func isServerSentEvent(headers http.Header) bool {
return strings.ToLower(headers.Get("content-type")) == "text/event-stream"
}
+48 -9
View File
@@ -5,11 +5,12 @@ import (
"io"
"net/http"
"net/url"
"testing"
"time"
"github.com/cloudflare/cloudflared/cmd/cloudflared/ui"
"github.com/cloudflare/cloudflared/logger"
"github.com/gobwas/ws/wsutil"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
)
const (
@@ -21,16 +22,17 @@ var (
OriginClient: &mockOriginClient{},
GracePeriod: time.Millisecond * 100,
}
testLogger, _ = logger.New()
log = zerolog.Nop()
testOriginURL = &url.URL{
Scheme: "https",
Host: "connectiontest.argotunnel.com",
}
testTunnelEventChan = make(chan ui.TunnelEvent)
testTunnelEventChan = make(chan Event)
testObserver = &Observer{
testLogger,
&log,
m,
testTunnelEventChan,
[]chan Event{testTunnelEventChan},
false,
}
testLargeResp = make([]byte, largeFileSize)
)
@@ -79,7 +81,7 @@ func wsEndpoint(w ResponseWriter, r *http.Request) error {
resp := &http.Response{
StatusCode: http.StatusSwitchingProtocols,
}
w.WriteRespHeaders(resp)
_ = w.WriteRespHeaders(resp)
clientReader := nowriter{r.Body}
go func() {
for {
@@ -100,8 +102,8 @@ func originRespEndpoint(w ResponseWriter, status int, data []byte) {
resp := &http.Response{
StatusCode: status,
}
w.WriteRespHeaders(resp)
w.Write(data)
_ = w.WriteRespHeaders(resp)
_, _ = w.Write(data)
}
type mockConnectedFuse struct{}
@@ -111,3 +113,40 @@ func (mcf mockConnectedFuse) Connected() {}
func (mcf mockConnectedFuse) IsConnected() bool {
return true
}
func TestIsEventStream(t *testing.T) {
tests := []struct {
headers http.Header
isEventStream bool
}{
{
headers: newHeader("Content-Type", "text/event-stream"),
isEventStream: true,
},
{
headers: newHeader("content-type", "text/event-stream"),
isEventStream: true,
},
{
headers: newHeader("Content-Type", "text/event-stream; charset=utf-8"),
isEventStream: true,
},
{
headers: newHeader("Content-Type", "application/json"),
isEventStream: false,
},
{
headers: http.Header{},
isEventStream: false,
},
}
for _, test := range tests {
assert.Equal(t, test.isEventStream, IsServerSentEvent(test.headers))
}
}
func newHeader(key, value string) http.Header {
header := http.Header{}
header.Add(key, value)
return header
}
+8 -3
View File
@@ -63,13 +63,18 @@ func (e muxerShutdownError) Error() string {
}
func isHandshakeErrRecoverable(err error, connIndex uint8, observer *Observer) bool {
log := observer.log.With().
Uint8(LogFieldConnIndex, connIndex).
Err(err).
Logger()
switch err.(type) {
case edgediscovery.DialError:
observer.Errorf("Connection %d unable to dial edge: %s", connIndex, err)
log.Error().Msg("Connection unable to dial edge")
case h2mux.MuxerHandshakeError:
observer.Errorf("Connection %d handshake with edge server failed: %s", connIndex, err)
log.Error().Msg("Connection handshake with edge server failed")
default:
observer.Errorf("Connection %d failed: %s", connIndex, err)
log.Error().Msg("Connection failed")
return false
}
return true
+25
View File
@@ -0,0 +1,25 @@
package connection
// Event is something that happened to a connection, e.g. disconnection or registration.
type Event struct {
Index uint8
EventType Status
Location string
URL string
}
// Status is the status of a connection.
type Status int
const (
// Disconnected means the connection to the edge was broken.
Disconnected Status = iota
// Connected means the connection to the edge was successfully established.
Connected
// Reconnecting means the connection to the edge is being re-established.
Reconnecting
// SetURL means this connection's tunnel was given a URL by the edge. Used for free tunnels.
SetURL
// RegisteringTunnel means the non-named tunnel is registering its connection.
RegisteringTunnel
)
+14 -11
View File
@@ -7,10 +7,11 @@ import (
"time"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/logger"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/cloudflare/cloudflared/websocket"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"golang.org/x/sync/errgroup"
)
@@ -37,20 +38,20 @@ type MuxerConfig struct {
MetricsUpdateFreq time.Duration
}
func (mc *MuxerConfig) H2MuxerConfig(h h2mux.MuxedStreamHandler, logger logger.Service) *h2mux.MuxerConfig {
func (mc *MuxerConfig) H2MuxerConfig(h h2mux.MuxedStreamHandler, log *zerolog.Logger) *h2mux.MuxerConfig {
return &h2mux.MuxerConfig{
Timeout: muxerTimeout,
Handler: h,
IsClient: true,
HeartbeatInterval: mc.HeartbeatInterval,
MaxHeartbeats: mc.MaxHeartbeats,
Logger: logger,
Log: log,
CompressionQuality: mc.CompressionSetting,
}
}
// NewTunnelHandler returns a TunnelHandler, origin LAN IP and error
func NewH2muxConnection(ctx context.Context,
func NewH2muxConnection(
config *Config,
muxerConfig *MuxerConfig,
edgeConn net.Conn,
@@ -67,7 +68,7 @@ func NewH2muxConnection(ctx context.Context,
// Establish a muxed connection with the edge
// Client mux handshake with agent server
muxer, err := h2mux.Handshake(edgeConn, edgeConn, *muxerConfig.H2MuxerConfig(h, observer), h2mux.ActiveStreams)
muxer, err := h2mux.Handshake(edgeConn, edgeConn, *muxerConfig.H2MuxerConfig(h, observer.log), h2mux.ActiveStreams)
if err != nil {
recoverable := isHandshakeErrRecoverable(err, connIndex, observer)
return nil, err, recoverable
@@ -87,7 +88,7 @@ func (h *h2muxConnection) ServeNamedTunnel(ctx context.Context, namedTunnel *Nam
if err != nil {
return err
}
rpcClient := newRegistrationRPCClient(ctx, stream, h.observer)
rpcClient := newRegistrationRPCClient(ctx, stream, h.observer.log)
defer rpcClient.Close()
if err = rpcClient.RegisterConnection(serveCtx, namedTunnel, connOptions, h.connIndex, h.observer); err != nil {
@@ -122,7 +123,9 @@ func (h *h2muxConnection) ServeClassicTunnel(ctx context.Context, classicTunnel
return nil
}
// log errors and proceed to RegisterTunnel
h.observer.Errorf("Couldn't reconnect connection %d. Reregistering it instead. Error was: %v", h.connIndex, err)
h.observer.log.Err(err).
Uint8(LogFieldConnIndex, h.connIndex).
Msg("Couldn't reconnect connection. Re-registering it instead.")
}
return h.registerTunnel(ctx, credentialManager, classicTunnel, registrationOptions)
})
@@ -207,14 +210,14 @@ type h2muxRespWriter struct {
func (rp *h2muxRespWriter) WriteRespHeaders(resp *http.Response) error {
headers := h2mux.H1ResponseToH2ResponseHeaders(resp)
headers = append(headers, h2mux.Header{Name: responseMetaHeaderField, Value: responseMetaHeaderOrigin})
headers = append(headers, h2mux.Header{Name: ResponseMetaHeaderField, Value: responseMetaHeaderOrigin})
return rp.WriteHeaders(headers)
}
func (rp *h2muxRespWriter) WriteErrorResponse() {
rp.WriteHeaders([]h2mux.Header{
_ = rp.WriteHeaders([]h2mux.Header{
{Name: ":status", Value: "502"},
{Name: responseMetaHeaderField, Value: responseMetaHeaderCfd},
{Name: ResponseMetaHeaderField, Value: responseMetaHeaderCfd},
})
rp.Write([]byte("502 Bad Gateway"))
_, _ = rp.Write([]byte("502 Bad Gateway"))
}
+11 -11
View File
@@ -26,19 +26,19 @@ var (
}
)
func newH2MuxConnection(ctx context.Context, t require.TestingT) (*h2muxConnection, *h2mux.Muxer) {
func newH2MuxConnection(t require.TestingT) (*h2muxConnection, *h2mux.Muxer) {
edgeConn, originConn := net.Pipe()
edgeMuxChan := make(chan *h2mux.Muxer)
go func() {
edgeMuxConfig := h2mux.MuxerConfig{
Logger: testObserver,
Log: testObserver.log,
}
edgeMux, err := h2mux.Handshake(edgeConn, edgeConn, edgeMuxConfig, h2mux.ActiveStreams)
require.NoError(t, err)
edgeMuxChan <- edgeMux
}()
var connIndex = uint8(0)
h2muxConn, err, _ := NewH2muxConnection(ctx, testConfig, testMuxerConfig, originConn, connIndex, testObserver)
h2muxConn, err, _ := NewH2muxConnection(testConfig, testMuxerConfig, originConn, connIndex, testObserver)
require.NoError(t, err)
return h2muxConn, <-edgeMuxChan
}
@@ -79,13 +79,13 @@ func TestServeStreamHTTP(t *testing.T) {
}
ctx, cancel := context.WithCancel(context.Background())
h2muxConn, edgeMux := newH2MuxConnection(ctx, t)
h2muxConn, edgeMux := newH2MuxConnection(t)
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
edgeMux.Serve(ctx)
_ = edgeMux.Serve(ctx)
}()
go func() {
defer wg.Done()
@@ -105,9 +105,9 @@ func TestServeStreamHTTP(t *testing.T) {
require.True(t, hasHeader(stream, ":status", strconv.Itoa(test.expectedStatus)))
if test.isProxyError {
assert.True(t, hasHeader(stream, responseMetaHeaderField, responseMetaHeaderCfd))
assert.True(t, hasHeader(stream, ResponseMetaHeaderField, responseMetaHeaderCfd))
} else {
assert.True(t, hasHeader(stream, responseMetaHeaderField, responseMetaHeaderOrigin))
assert.True(t, hasHeader(stream, ResponseMetaHeaderField, responseMetaHeaderOrigin))
body := make([]byte, len(test.expectedBody))
_, err = stream.Read(body)
require.NoError(t, err)
@@ -120,7 +120,7 @@ func TestServeStreamHTTP(t *testing.T) {
func TestServeStreamWS(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
h2muxConn, edgeMux := newH2MuxConnection(ctx, t)
h2muxConn, edgeMux := newH2MuxConnection(t)
var wg sync.WaitGroup
wg.Add(2)
@@ -154,7 +154,7 @@ func TestServeStreamWS(t *testing.T) {
require.NoError(t, err)
require.True(t, hasHeader(stream, ":status", strconv.Itoa(http.StatusSwitchingProtocols)))
assert.True(t, hasHeader(stream, responseMetaHeaderField, responseMetaHeaderOrigin))
assert.True(t, hasHeader(stream, ResponseMetaHeaderField, responseMetaHeaderOrigin))
data := []byte("test websocket")
err = wsutil.WriteClientText(writePipe, data)
@@ -179,7 +179,7 @@ func hasHeader(stream *h2mux.MuxedStream, name, val string) bool {
func benchmarkServeStreamHTTPSimple(b *testing.B, test testRequest) {
ctx, cancel := context.WithCancel(context.Background())
h2muxConn, edgeMux := newH2MuxConnection(ctx, b)
h2muxConn, edgeMux := newH2MuxConnection(b)
var wg sync.WaitGroup
wg.Add(2)
@@ -209,7 +209,7 @@ func benchmarkServeStreamHTTPSimple(b *testing.B, test testRequest) {
b.StopTimer()
require.NoError(b, openstreamErr)
assert.True(b, hasHeader(stream, responseMetaHeaderField, responseMetaHeaderOrigin))
assert.True(b, hasHeader(stream, ResponseMetaHeaderField, responseMetaHeaderOrigin))
require.True(b, hasHeader(stream, ":status", strconv.Itoa(http.StatusOK)))
require.NoError(b, readBodyErr)
require.Equal(b, test.expectedBody, body)
+2 -2
View File
@@ -8,12 +8,12 @@ import (
)
const (
responseMetaHeaderField = "cf-cloudflared-response-meta"
ResponseMetaHeaderField = "cf-cloudflared-response-meta"
)
var (
canonicalResponseUserHeadersField = http.CanonicalHeaderKey(h2mux.ResponseUserHeadersField)
canonicalResponseMetaHeaderField = http.CanonicalHeaderKey(responseMetaHeaderField)
canonicalResponseMetaHeaderField = http.CanonicalHeaderKey(ResponseMetaHeaderField)
responseMetaHeaderCfd = mustInitRespMetaHeader("cloudflared")
responseMetaHeaderOrigin = mustInitRespMetaHeader("origin")
)
+5 -10
View File
@@ -2,7 +2,6 @@ package connection
import (
"context"
"errors"
"io"
"math"
"net"
@@ -11,9 +10,9 @@ import (
"sync"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/logger"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/rs/zerolog"
"golang.org/x/net/http2"
)
@@ -23,10 +22,6 @@ const (
controlStreamUpgrade = "control-stream"
)
var (
errNotFlusher = errors.New("ResponseWriter doesn't implement http.Flusher")
)
type http2Connection struct {
conn net.Conn
server *http2.Server
@@ -38,7 +33,7 @@ type http2Connection struct {
connIndex uint8
wg *sync.WaitGroup
// newRPCClientFunc allows us to mock RPCs during testing
newRPCClientFunc func(context.Context, io.ReadWriteCloser, logger.Service) NamedTunnelRPCClient
newRPCClientFunc func(context.Context, io.ReadWriteCloser, *zerolog.Logger) NamedTunnelRPCClient
connectedFuse ConnectedFuse
}
@@ -89,7 +84,7 @@ func (c *http2Connection) ServeHTTP(w http.ResponseWriter, r *http.Request) {
}
flusher, isFlusher := w.(http.Flusher)
if !isFlusher {
c.observer.Errorf("%T doesn't implement http.Flusher", w)
c.observer.log.Error().Msgf("%T doesn't implement http.Flusher", w)
respWriter.WriteErrorResponse()
return
}
@@ -112,7 +107,7 @@ func (c *http2Connection) ServeHTTP(w http.ResponseWriter, r *http.Request) {
}
func (c *http2Connection) serveControlStream(ctx context.Context, respWriter *http2RespWriter) error {
rpcClient := c.newRPCClientFunc(ctx, respWriter, c.observer)
rpcClient := c.newRPCClientFunc(ctx, respWriter, c.observer.log)
defer rpcClient.Close()
if err := rpcClient.RegisterConnection(ctx, c.namedTunnel, c.connOptions, c.connIndex, c.observer); err != nil {
@@ -167,7 +162,7 @@ func (rp *http2RespWriter) WriteRespHeaders(resp *http.Response) error {
status = http.StatusOK
}
rp.w.WriteHeader(status)
if isServerSentEvent(resp.Header) {
if IsServerSentEvent(resp.Header) {
rp.shouldFlush = true
}
if rp.shouldFlush {
+6 -5
View File
@@ -12,10 +12,11 @@ import (
"testing"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/gobwas/ws/wsutil"
"github.com/rs/zerolog"
"github.com/stretchr/testify/require"
"golang.org/x/net/http2"
)
@@ -100,9 +101,9 @@ func TestServeHTTP(t *testing.T) {
require.Equal(t, test.expectedBody, respBody)
}
if test.isProxyError {
require.Equal(t, responseMetaHeaderCfd, resp.Header.Get(responseMetaHeaderField))
require.Equal(t, responseMetaHeaderCfd, resp.Header.Get(ResponseMetaHeaderField))
} else {
require.Equal(t, responseMetaHeaderOrigin, resp.Header.Get(responseMetaHeaderField))
require.Equal(t, responseMetaHeaderOrigin, resp.Header.Get(ResponseMetaHeaderField))
}
}
cancel()
@@ -136,7 +137,7 @@ type mockRPCClientFactory struct {
unregistered chan struct{}
}
func (mf *mockRPCClientFactory) newMockRPCClient(context.Context, io.ReadWriteCloser, logger.Service) NamedTunnelRPCClient {
func (mf *mockRPCClientFactory) newMockRPCClient(context.Context, io.ReadWriteCloser, *zerolog.Logger) NamedTunnelRPCClient {
return mockNamedTunnelRPCClient{
registered: mf.registered,
unregistered: mf.unregistered,
@@ -202,7 +203,7 @@ func TestServeWS(t *testing.T) {
resp := respWriter.Result()
// http2RespWriter should rewrite status 101 to 200
require.Equal(t, http.StatusOK, resp.StatusCode)
require.Equal(t, responseMetaHeaderOrigin, resp.Header.Get(responseMetaHeaderField))
require.Equal(t, responseMetaHeaderOrigin, resp.Header.Get(ResponseMetaHeaderField))
wg.Wait()
}
+1 -1
View File
@@ -380,7 +380,7 @@ func newTunnelMetrics() *tunnelMetrics {
serverLocations: serverLocations,
oldServerLocations: make(map[string]string),
muxerMetrics: newMuxerMetrics(),
tunnelsHA: NewTunnelsForHA(),
tunnelsHA: newTunnelsForHA(),
regSuccess: registerSuccess,
regFail: registerFail,
rpcFail: rpcFail,
+36 -24
View File
@@ -5,43 +5,47 @@ import (
"net/url"
"strings"
"github.com/cloudflare/cloudflared/cmd/cloudflared/ui"
"github.com/cloudflare/cloudflared/logger"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/rs/zerolog"
)
const LogFieldLocation = "location"
type Observer struct {
logger.Service
metrics *tunnelMetrics
tunnelEventChan chan<- ui.TunnelEvent
log *zerolog.Logger
metrics *tunnelMetrics
tunnelEventChans []chan Event
uiEnabled bool
}
func NewObserver(logger logger.Service, tunnelEventChan chan<- ui.TunnelEvent) *Observer {
func NewObserver(log *zerolog.Logger, tunnelEventChans []chan Event, uiEnabled bool) *Observer {
return &Observer{
logger,
log,
newTunnelMetrics(),
tunnelEventChan,
tunnelEventChans,
uiEnabled,
}
}
func (o *Observer) logServerInfo(connIndex uint8, location, msg string) {
// If launch-ui flag is set, send connect msg
if o.tunnelEventChan != nil {
o.tunnelEventChan <- ui.TunnelEvent{Index: connIndex, EventType: ui.Connected, Location: location}
}
o.Infof(msg)
o.sendEvent(Event{Index: connIndex, EventType: Connected, Location: location})
o.log.Info().
Uint8(LogFieldConnIndex, connIndex).
Str(LogFieldLocation, location).
Msg(msg)
o.metrics.registerServerLocation(uint8ToString(connIndex), location)
}
func (o *Observer) logTrialHostname(registration *tunnelpogs.TunnelRegistration) error {
// Print out the user's trial zone URL in a nice box (if they requested and got one and UI flag is not set)
if o.tunnelEventChan == nil {
if !o.uiEnabled {
if registrationURL, err := url.Parse(registration.Url); err == nil {
for _, line := range asciiBox(trialZoneMsg(registrationURL.String()), 2) {
o.Info(line)
o.log.Info().Msg(line)
}
} else {
o.Error("Failed to connect tunnel, please try again.")
o.log.Error().Msg("Failed to connect tunnel, please try again.")
return fmt.Errorf("empty URL in response from Cloudflare edge")
}
}
@@ -81,19 +85,27 @@ func trialZoneMsg(url string) []string {
}
func (o *Observer) sendRegisteringEvent() {
if o.tunnelEventChan != nil {
o.tunnelEventChan <- ui.TunnelEvent{EventType: ui.RegisteringTunnel}
}
o.sendEvent(Event{EventType: RegisteringTunnel})
}
func (o *Observer) sendConnectedEvent(connIndex uint8, location string) {
if o.tunnelEventChan != nil {
o.tunnelEventChan <- ui.TunnelEvent{Index: connIndex, EventType: ui.Connected, Location: location}
}
o.sendEvent(Event{Index: connIndex, EventType: Connected, Location: location})
}
func (o *Observer) sendURL(url string) {
if o.tunnelEventChan != nil {
o.tunnelEventChan <- ui.TunnelEvent{EventType: ui.SetUrl, Url: url}
o.sendEvent(Event{EventType: SetURL, URL: url})
}
func (o *Observer) SendReconnect(connIndex uint8) {
o.sendEvent(Event{Index: connIndex, EventType: Reconnecting})
}
func (o *Observer) SendDisconnect(connIndex uint8) {
o.sendEvent(Event{Index: connIndex, EventType: Disconnected})
}
func (o *Observer) sendEvent(e Event) {
for _, ch := range o.tunnelEventChans {
ch <- e
}
}
+18 -12
View File
@@ -6,7 +6,7 @@ import (
"sync"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/rs/zerolog"
)
const (
@@ -89,7 +89,7 @@ type autoProtocolSelector struct {
fetchFunc PercentageFetcher
refreshAfter time.Time
ttl time.Duration
logger logger.Service
log *zerolog.Logger
}
func newAutoProtocolSelector(
@@ -97,7 +97,7 @@ func newAutoProtocolSelector(
switchThrehold int32,
fetchFunc PercentageFetcher,
ttl time.Duration,
logger logger.Service,
log *zerolog.Logger,
) *autoProtocolSelector {
return &autoProtocolSelector{
current: current,
@@ -105,7 +105,7 @@ func newAutoProtocolSelector(
fetchFunc: fetchFunc,
refreshAfter: time.Now().Add(ttl),
ttl: ttl,
logger: logger,
log: log,
}
}
@@ -118,7 +118,7 @@ func (s *autoProtocolSelector) Current() Protocol {
percentage, err := s.fetchFunc()
if err != nil {
s.logger.Errorf("Failed to refresh protocol, err: %v", err)
s.log.Err(err).Msg("Failed to refresh protocol")
return s.current
}
@@ -139,7 +139,13 @@ func (s *autoProtocolSelector) Fallback() (Protocol, bool) {
type PercentageFetcher func() (int32, error)
func NewProtocolSelector(protocolFlag string, namedTunnel *NamedTunnelConfig, fetchFunc PercentageFetcher, ttl time.Duration, logger logger.Service) (ProtocolSelector, error) {
func NewProtocolSelector(
protocolFlag string,
namedTunnel *NamedTunnelConfig,
fetchFunc PercentageFetcher,
ttl time.Duration,
log *zerolog.Logger,
) (ProtocolSelector, error) {
if namedTunnel == nil {
return &staticProtocolSelector{
current: H2mux,
@@ -157,23 +163,23 @@ func NewProtocolSelector(protocolFlag string, namedTunnel *NamedTunnelConfig, fe
}
if protocolFlag == HTTP2.String() {
if http2Percentage < 0 {
return newAutoProtocolSelector(H2mux, explicitHTTP2FallbackThreshold, fetchFunc, ttl, logger), nil
return newAutoProtocolSelector(H2mux, explicitHTTP2FallbackThreshold, fetchFunc, ttl, log), nil
}
return newAutoProtocolSelector(HTTP2, explicitHTTP2FallbackThreshold, fetchFunc, ttl, logger), nil
return newAutoProtocolSelector(HTTP2, explicitHTTP2FallbackThreshold, fetchFunc, ttl, log), nil
}
if protocolFlag != autoSelectFlag {
return nil, fmt.Errorf("Unknown protocol %s, %s", protocolFlag, AvailableProtocolFlagMessage)
}
threshold := switchThreshold(namedTunnel.Auth.AccountTag)
threshold := switchThreshold(namedTunnel.Credentials.AccountTag)
if threshold < http2Percentage {
return newAutoProtocolSelector(HTTP2, threshold, fetchFunc, ttl, logger), nil
return newAutoProtocolSelector(HTTP2, threshold, fetchFunc, ttl, log), nil
}
return newAutoProtocolSelector(H2mux, threshold, fetchFunc, ttl, logger), nil
return newAutoProtocolSelector(H2mux, threshold, fetchFunc, ttl, log), nil
}
func switchThreshold(accountTag string) int32 {
h := fnv.New32a()
h.Write([]byte(accountTag))
_, _ = h.Write([]byte(accountTag))
return int32(h.Sum32() % 100)
}
+6 -11
View File
@@ -5,8 +5,6 @@ import (
"testing"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/stretchr/testify/assert"
)
@@ -16,7 +14,7 @@ const (
var (
testNamedTunnelConfig = &NamedTunnelConfig{
Auth: pogs.TunnelAuth{
Credentials: Credentials{
AccountTag: "testAccountTag",
},
}
@@ -131,9 +129,9 @@ func TestNewProtocolSelector(t *testing.T) {
wantErr: true,
},
}
logger, _ := logger.New()
for _, test := range tests {
selector, err := NewProtocolSelector(test.protocol, test.namedTunnelConfig, test.fetchFunc, testNoTTL, logger)
selector, err := NewProtocolSelector(test.protocol, test.namedTunnelConfig, test.fetchFunc, testNoTTL, &log)
if test.wantErr {
assert.Error(t, err, fmt.Sprintf("test %s failed", test.name))
} else {
@@ -149,9 +147,8 @@ func TestNewProtocolSelector(t *testing.T) {
}
func TestAutoProtocolSelectorRefresh(t *testing.T) {
logger, _ := logger.New()
fetcher := dynamicMockFetcher{}
selector, err := NewProtocolSelector("auto", testNamedTunnelConfig, fetcher.fetch(), testNoTTL, logger)
selector, err := NewProtocolSelector("auto", testNamedTunnelConfig, fetcher.fetch(), testNoTTL, &log)
assert.NoError(t, err)
assert.Equal(t, H2mux, selector.Current())
@@ -179,9 +176,8 @@ func TestAutoProtocolSelectorRefresh(t *testing.T) {
}
func TestHTTP2ProtocolSelectorRefresh(t *testing.T) {
logger, _ := logger.New()
fetcher := dynamicMockFetcher{}
selector, err := NewProtocolSelector("http2", testNamedTunnelConfig, fetcher.fetch(), testNoTTL, logger)
selector, err := NewProtocolSelector("http2", testNamedTunnelConfig, fetcher.fetch(), testNoTTL, &log)
assert.NoError(t, err)
assert.Equal(t, HTTP2, selector.Current())
@@ -209,9 +205,8 @@ func TestHTTP2ProtocolSelectorRefresh(t *testing.T) {
}
func TestProtocolSelectorRefreshTTL(t *testing.T) {
logger, _ := logger.New()
fetcher := dynamicMockFetcher{percentage: 100}
selector, err := NewProtocolSelector("auto", testNamedTunnelConfig, fetcher.fetch(), time.Hour, logger)
selector, err := NewProtocolSelector("auto", testNamedTunnelConfig, fetcher.fetch(), time.Hour, &log)
assert.NoError(t, err)
assert.Equal(t, HTTP2, selector.Current())
+30 -29
View File
@@ -6,9 +6,10 @@ import (
"io"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/tunnelrpc"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/rs/zerolog"
"zombiezen.com/go/capnproto2/rpc"
)
@@ -22,12 +23,12 @@ type tunnelServerClient struct {
func NewTunnelServerClient(
ctx context.Context,
stream io.ReadWriteCloser,
logger logger.Service,
log *zerolog.Logger,
) *tunnelServerClient {
transport := tunnelrpc.NewTransportLogger(logger, rpc.StreamTransport(stream))
transport := tunnelrpc.NewTransportLogger(log, rpc.StreamTransport(stream))
conn := rpc.NewConn(
transport,
tunnelrpc.ConnLog(logger),
tunnelrpc.ConnLog(log),
)
registrationClient := tunnelpogs.RegistrationServer_PogsClient{Client: conn.Bootstrap(ctx), Conn: conn}
return &tunnelServerClient{
@@ -46,8 +47,8 @@ func (tsc *tunnelServerClient) Authenticate(ctx context.Context, classicTunnel *
func (tsc *tunnelServerClient) Close() {
// Closing the client will also close the connection
tsc.client.Close()
tsc.transport.Close()
_ = tsc.client.Close()
_ = tsc.transport.Close()
}
type NamedTunnelRPCClient interface {
@@ -70,12 +71,12 @@ type registrationServerClient struct {
func newRegistrationRPCClient(
ctx context.Context,
stream io.ReadWriteCloser,
logger logger.Service,
log *zerolog.Logger,
) NamedTunnelRPCClient {
transport := tunnelrpc.NewTransportLogger(logger, rpc.StreamTransport(stream))
transport := tunnelrpc.NewTransportLogger(log, rpc.StreamTransport(stream))
conn := rpc.NewConn(
transport,
tunnelrpc.ConnLog(logger),
tunnelrpc.ConnLog(log),
)
return &registrationServerClient{
client: tunnelpogs.RegistrationServer_PogsClient{Client: conn.Bootstrap(ctx), Conn: conn},
@@ -92,8 +93,8 @@ func (rsc *registrationServerClient) RegisterConnection(
) error {
conn, err := rsc.client.RegisterConnection(
ctx,
config.Auth,
config.ID,
config.Credentials.Auth(),
config.Credentials.TunnelID,
connIndex,
options,
)
@@ -108,7 +109,7 @@ func (rsc *registrationServerClient) RegisterConnection(
observer.metrics.regSuccess.WithLabelValues("registerConnection").Inc()
observer.logServerInfo(connIndex, conn.Location, fmt.Sprintf("Connection %d registered with %s using ID %s", connIndex, conn.Location, conn.UUID))
observer.logServerInfo(connIndex, conn.Location, fmt.Sprintf("Connection %s registered", conn.UUID))
observer.sendConnectedEvent(connIndex, conn.Location)
return nil
@@ -117,14 +118,14 @@ func (rsc *registrationServerClient) RegisterConnection(
func (rsc *registrationServerClient) GracefulShutdown(ctx context.Context, gracePeriod time.Duration) {
ctx, cancel := context.WithTimeout(ctx, gracePeriod)
defer cancel()
rsc.client.UnregisterConnection(ctx)
_ = rsc.client.UnregisterConnection(ctx)
}
func (rsc *registrationServerClient) Close() {
// Closing the client will also close the connection
rsc.client.Close()
_ = rsc.client.Close()
// Closing the transport also closes the stream
rsc.transport.Close()
_ = rsc.transport.Close()
}
type rpcName string
@@ -143,10 +144,10 @@ func (h *h2muxConnection) registerTunnel(ctx context.Context, credentialSetter C
if err != nil {
return err
}
rpcClient := NewTunnelServerClient(ctx, stream, h.observer)
rpcClient := NewTunnelServerClient(ctx, stream, h.observer.log)
defer rpcClient.Close()
h.logServerInfo(ctx, rpcClient)
_ = h.logServerInfo(ctx, rpcClient)
registration := rpcClient.client.RegisterTunnel(
ctx,
classicTunnel.OriginCert,
@@ -178,12 +179,12 @@ func (h *h2muxConnection) processRegistrationSuccess(
credentialManager CredentialManager, classicTunnel *ClassicTunnelConfig,
) error {
for _, logLine := range registration.LogLines {
h.observer.Info(logLine)
h.observer.log.Info().Msg(logLine)
}
if registration.TunnelID != "" {
h.observer.metrics.tunnelsHA.AddTunnelID(h.connIndex, registration.TunnelID)
h.observer.Infof("Each HA connection's tunnel IDs: %v", h.observer.metrics.tunnelsHA.String())
h.observer.log.Info().Msgf("Each HA connection's tunnel IDs: %v", h.observer.metrics.tunnelsHA.String())
}
// Print out the user's trial zone URL in a nice box (if they requested and got one and UI flag is not set)
@@ -197,7 +198,7 @@ func (h *h2muxConnection) processRegistrationSuccess(
credentialManager.SetConnDigest(h.connIndex, registration.ConnDigest)
h.observer.metrics.userHostnamesCounts.WithLabelValues(registration.Url).Inc()
h.observer.Infof("Route propagating, it may take up to 1 minute for your new route to become functional")
h.observer.log.Info().Msgf("Route propagating, it may take up to 1 minute for your new route to become functional")
h.observer.metrics.regSuccess.WithLabelValues(string(name)).Inc()
return nil
}
@@ -228,15 +229,15 @@ func (h *h2muxConnection) reconnectTunnel(ctx context.Context, credentialManager
return err
}
h.observer.Debug("initiating RPC stream to reconnect")
h.observer.log.Debug().Msg("initiating RPC stream to reconnect")
stream, err := h.newRPCStream(ctx, register)
if err != nil {
return err
}
rpcClient := NewTunnelServerClient(ctx, stream, h.observer)
rpcClient := NewTunnelServerClient(ctx, stream, h.observer.log)
defer rpcClient.Close()
h.logServerInfo(ctx, rpcClient)
_ = h.logServerInfo(ctx, rpcClient)
registration := rpcClient.client.ReconnectTunnel(
ctx,
token,
@@ -259,15 +260,15 @@ func (h *h2muxConnection) logServerInfo(ctx context.Context, rpcClient *tunnelSe
})
serverInfoMessage, err := serverInfoPromise.Result().Struct()
if err != nil {
h.observer.Errorf("Failed to retrieve server information: %s", err)
h.observer.log.Err(err).Msg("Failed to retrieve server information")
return err
}
serverInfo, err := tunnelpogs.UnmarshalServerInfo(serverInfoMessage)
if err != nil {
h.observer.Errorf("Failed to retrieve server information: %s", err)
h.observer.log.Err(err).Msg("Failed to retrieve server information")
return err
}
h.observer.logServerInfo(h.connIndex, serverInfo.LocationName, fmt.Sprintf("Connnection %d connected to %s", h.connIndex, serverInfo.LocationName))
h.observer.logServerInfo(h.connIndex, serverInfo.LocationName, "Connection established")
return nil
}
@@ -281,15 +282,15 @@ func (h *h2muxConnection) unregister(isNamedTunnel bool) {
}
if isNamedTunnel {
rpcClient := newRegistrationRPCClient(unregisterCtx, stream, h.observer)
rpcClient := newRegistrationRPCClient(unregisterCtx, stream, h.observer.log)
defer rpcClient.Close()
rpcClient.GracefulShutdown(unregisterCtx, h.config.GracePeriod)
} else {
rpcClient := NewTunnelServerClient(unregisterCtx, stream, h.observer)
rpcClient := NewTunnelServerClient(unregisterCtx, stream, h.observer.log)
defer rpcClient.Close()
// gracePeriod is encoded in int64 using capnproto
rpcClient.client.UnregisterTunnel(unregisterCtx, h.config.GracePeriod.Nanoseconds())
_ = rpcClient.client.UnregisterTunnel(unregisterCtx, h.config.GracePeriod.Nanoseconds())
}
}
+1 -1
View File
@@ -15,7 +15,7 @@ type tunnelsForHA struct {
}
// NewTunnelsForHA initializes the Prometheus metrics etc for a tunnelsForHA.
func NewTunnelsForHA() tunnelsForHA {
func newTunnelsForHA() tunnelsForHA {
metrics := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "tunnel_ids",
+13 -12
View File
@@ -8,20 +8,25 @@ import (
"net"
"net/http"
"net/url"
"os"
"time"
"github.com/cloudflare/cloudflared/hello"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/validation"
"github.com/gorilla/mux"
"github.com/pkg/errors"
"github.com/rs/zerolog"
)
const (
LogFieldCFRay = "cfRay"
)
// Proxy is an HTTP server that proxies requests to a Client.
type Proxy struct {
client Client
accessValidator *validation.Access
logger logger.Service
log *zerolog.Logger
}
// NewInsecureProxy creates a Proxy that talks to a Client at an origin.
@@ -43,12 +48,9 @@ func NewInsecureProxy(ctx context.Context, origin string) (*Proxy, error) {
return nil, errors.Wrap(err, "could not connect to the database")
}
logger, err := logger.New()
if err != nil {
return nil, errors.Wrap(err, "error setting up logger")
}
log := zerolog.New(os.Stderr).With().Logger() // TODO: Does not obey log configuration
return &Proxy{client, nil, logger}, nil
return &Proxy{client, nil, &log}, nil
}
// NewSecureProxy creates a Proxy that talks to a Client at an origin.
@@ -95,8 +97,7 @@ func (proxy *Proxy) IsAllowed(r *http.Request, verbose ...bool) bool {
// Warn administrators that invalid JWTs are being rejected. This is indicative
// of either a misconfiguration of the CLI or a massive failure of upstream systems.
if len(verbose) > 0 {
cfRay := proxy.getRayHeader(r)
proxy.logger.Infof("dbproxy: Failed JWT authentication: cf-ray: %s %s", cfRay, err)
proxy.log.Err(err).Str(LogFieldCFRay, proxy.getRayHeader(r)).Msg("dbproxy: Failed JWT authentication")
}
return false
@@ -151,8 +152,8 @@ func (proxy *Proxy) httpListen(ctx context.Context, listener net.Listener) error
go func() {
<-ctx.Done()
httpServer.Close()
listener.Close()
_ = httpServer.Close()
_ = listener.Close()
}()
return httpServer.Serve(listener)
@@ -241,7 +242,7 @@ func (proxy *Proxy) httpRespondErr(w http.ResponseWriter, r *http.Request, defau
proxy.httpRespond(w, r, status, err.Error())
if len(err.Error()) > 0 {
cfRay := proxy.getRayHeader(r)
proxy.logger.Infof("dbproxy: Database proxy error: cf-ray: %s %s", cfRay, err)
proxy.log.Err(err).Str(LogFieldCFRay, cfRay).Msg("dbproxy: Database proxy error")
}
}
+2 -2
View File
@@ -46,7 +46,7 @@ func NewSQLClient(ctx context.Context, originURL *url.URL) (Client, error) {
// Closes the driver, will occur when the context finishes.
go func() {
<-ctx.Done()
driver.Close()
_ = driver.Close()
}()
return &SQLClient{driver.DriverName(), driver}, nil
@@ -260,7 +260,7 @@ func sqlRows(rows *sql.Rows) ([]map[string]interface{}, error) {
for i := range columns {
pointers[i] = &values[i]
}
rows.Scan(pointers...)
_ = rows.Scan(pointers...)
// Convert a row, an array of values, into an object where
// each key is the name of its respective column.
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.15.3 as builder
FROM golang:1.15.6 as builder
ENV GO111MODULE=on \
CGO_ENABLED=0
WORKDIR /go/src/github.com/cloudflare/cloudflared/
+7 -12
View File
@@ -7,8 +7,8 @@ import (
"net"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/pkg/errors"
"github.com/rs/zerolog"
)
const (
@@ -21,12 +21,7 @@ const (
// discover HA origintunneld servers (GitHub issue #75).
dotServerName = "cloudflare-dns.com"
dotServerAddr = "1.1.1.1:853"
dotTimeout = time.Duration(15 * time.Second)
// SRV record resolution TTL
resolveEdgeAddrTTL = 1 * time.Hour
subsystemEdgeAddrResolver = "edgeAddrResolver"
dotTimeout = 15 * time.Second
)
// Redeclare network functions so they can be overridden in tests.
@@ -58,15 +53,15 @@ var friendlyDNSErrorLines = []string{
}
// EdgeDiscovery implements HA service discovery lookup.
func edgeDiscovery(logger logger.Service) ([][]*net.TCPAddr, error) {
func edgeDiscovery(log *zerolog.Logger) ([][]*net.TCPAddr, error) {
_, addrs, err := netLookupSRV(srvService, srvProto, srvName)
if err != nil {
_, fallbackAddrs, fallbackErr := fallbackLookupSRV(srvService, srvProto, srvName)
if fallbackErr != nil || len(fallbackAddrs) == 0 {
// use the original DNS error `err` in messages, not `fallbackErr`
logger.Errorf("Error looking up Cloudflare edge IPs: the DNS query failed: %s", err)
log.Err(err).Msg("Error looking up Cloudflare edge IPs: the DNS query failed")
for _, s := range friendlyDNSErrorLines {
logger.Error(s)
log.Error().Msg(s)
}
return nil, errors.Wrapf(err, "Could not lookup srv records on _%v._%v.%v", srvService, srvProto, srvName)
}
@@ -122,11 +117,11 @@ func resolveSRVToTCP(srv *net.SRV) ([]*net.TCPAddr, error) {
// ResolveAddrs resolves TCP address given a list of addresses. Address can be a hostname, however, it will return at most one
// of the hostname's IP addresses.
func ResolveAddrs(addrs []string, logger logger.Service) (resolved []*net.TCPAddr) {
func ResolveAddrs(addrs []string, log *zerolog.Logger) (resolved []*net.TCPAddr) {
for _, addr := range addrs {
tcpAddr, err := net.ResolveTCPAddr("tcp", addr)
if err != nil {
logger.Errorf("Failed to resolve %s, err: %v", addr, err)
log.Err(err).Msgf("Failed to resolve %s", addr)
} else {
resolved = append(resolved, tcpAddr)
}
+3 -3
View File
@@ -3,7 +3,7 @@ package allregions
import (
"testing"
"github.com/cloudflare/cloudflared/logger"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
)
@@ -19,8 +19,8 @@ func TestEdgeDiscovery(t *testing.T) {
}
}
l := logger.NewOutputWriter(logger.NewMockWriteManager())
addrLists, err := edgeDiscovery(l)
l := zerolog.Nop()
addrLists, err := edgeDiscovery(&l)
assert.NoError(t, err)
actualAddrSet := map[string]bool{}
for _, addrs := range addrLists {
-1
View File
@@ -57,7 +57,6 @@ func (r Region) GetUnusedIP(excluding *net.TCPAddr) *net.TCPAddr {
// Use the address, assigning it to a proxy connection.
func (r Region) Use(addr *net.TCPAddr, connID int) {
if addr == nil {
//logrus.Errorf("Attempted to use nil address for connection %d", connID)
return
}
r.connFor[addr] = InUse(connID)
+5 -5
View File
@@ -4,7 +4,7 @@ import (
"fmt"
"net"
"github.com/cloudflare/cloudflared/logger"
"github.com/rs/zerolog"
)
// Regions stores Cloudflare edge network IPs, partitioned into two regions.
@@ -19,8 +19,8 @@ type Regions struct {
// ------------------------------------
// ResolveEdge resolves the Cloudflare edge, returning all regions discovered.
func ResolveEdge(logger logger.Service) (*Regions, error) {
addrLists, err := edgeDiscovery(logger)
func ResolveEdge(log *zerolog.Logger) (*Regions, error) {
addrLists, err := edgeDiscovery(log)
if err != nil {
return nil, err
}
@@ -35,8 +35,8 @@ func ResolveEdge(logger logger.Service) (*Regions, error) {
// StaticEdge creates a list of edge addresses from the list of hostnames.
// Mainly used for testing connectivity.
func StaticEdge(hostnames []string, logger logger.Service) (*Regions, error) {
resolved := ResolveAddrs(hostnames, logger)
func StaticEdge(hostnames []string, log *zerolog.Logger) (*Regions, error) {
resolved := ResolveAddrs(hostnames, log)
if len(resolved) == 0 {
return nil, fmt.Errorf("failed to resolve any edge address")
}
+29 -24
View File
@@ -6,20 +6,21 @@ import (
"sync"
"github.com/cloudflare/cloudflared/edgediscovery/allregions"
"github.com/cloudflare/cloudflared/logger"
"github.com/rs/zerolog"
)
const (
subsystem = "edgediscovery"
LogFieldAddress = "address"
LogFieldConnIndex = "connIndex"
)
var errNoAddressesLeft = fmt.Errorf("There are no free edge addresses left")
var errNoAddressesLeft = fmt.Errorf("there are no free edge addresses left")
// Edge finds addresses on the Cloudflare edge and hands them out to connections.
type Edge struct {
regions *allregions.Regions
sync.Mutex
logger logger.Service
log *zerolog.Logger
}
// ------------------------------------
@@ -28,34 +29,34 @@ type Edge struct {
// ResolveEdge runs the initial discovery of the Cloudflare edge, finding Addrs that can be allocated
// to connections.
func ResolveEdge(l logger.Service) (*Edge, error) {
regions, err := allregions.ResolveEdge(l)
func ResolveEdge(log *zerolog.Logger) (*Edge, error) {
regions, err := allregions.ResolveEdge(log)
if err != nil {
return new(Edge), err
}
return &Edge{
logger: l,
log: log,
regions: regions,
}, nil
}
// StaticEdge creates a list of edge addresses from the list of hostnames. Mainly used for testing connectivity.
func StaticEdge(l logger.Service, hostnames []string) (*Edge, error) {
regions, err := allregions.StaticEdge(hostnames, l)
func StaticEdge(log *zerolog.Logger, hostnames []string) (*Edge, error) {
regions, err := allregions.StaticEdge(hostnames, log)
if err != nil {
return new(Edge), err
}
return &Edge{
logger: l,
log: log,
regions: regions,
}, nil
}
// MockEdge creates a Cloudflare Edge from arbitrary TCP addresses. Used for testing.
func MockEdge(l logger.Service, addrs []*net.TCPAddr) *Edge {
func MockEdge(log *zerolog.Logger, addrs []*net.TCPAddr) *Edge {
regions := allregions.NewNoResolve(addrs)
return &Edge{
logger: l,
log: log,
regions: regions,
}
}
@@ -76,42 +77,46 @@ func (ed *Edge) GetAddrForRPC() (*net.TCPAddr, error) {
}
// GetAddr gives this proxy connection an edge Addr. Prefer Addrs this connection has already used.
func (ed *Edge) GetAddr(connID int) (*net.TCPAddr, error) {
func (ed *Edge) GetAddr(connIndex int) (*net.TCPAddr, error) {
log := ed.log.With().Int(LogFieldConnIndex, connIndex).Logger()
ed.Lock()
defer ed.Unlock()
// If this connection has already used an edge addr, return it.
if addr := ed.regions.AddrUsedBy(connID); addr != nil {
ed.logger.Debugf("edgediscovery - GetAddr: Returning same address back to proxy connection: connID: %d", connID)
if addr := ed.regions.AddrUsedBy(connIndex); addr != nil {
log.Debug().Msg("edgediscovery - GetAddr: Returning same address back to proxy connection")
return addr, nil
}
// Otherwise, give it an unused one
addr := ed.regions.GetUnusedAddr(nil, connID)
addr := ed.regions.GetUnusedAddr(nil, connIndex)
if addr == nil {
ed.logger.Debugf("edgediscovery - GetAddr: No addresses left to give proxy connection: connID: %d", connID)
log.Debug().Msg("edgediscovery - GetAddr: No addresses left to give proxy connection")
return nil, errNoAddressesLeft
}
ed.logger.Debugf("edgediscovery - GetAddr: Giving connection its new address %s: connID: %d", addr, connID)
log.Debug().Str(LogFieldAddress, addr.String()).Msg("edgediscovery - GetAddr: Giving connection its new address")
return addr, nil
}
// GetDifferentAddr gives back the proxy connection's edge Addr and uses a new one.
func (ed *Edge) GetDifferentAddr(connID int) (*net.TCPAddr, error) {
func (ed *Edge) GetDifferentAddr(connIndex int) (*net.TCPAddr, error) {
log := ed.log.With().Int(LogFieldConnIndex, connIndex).Logger()
ed.Lock()
defer ed.Unlock()
oldAddr := ed.regions.AddrUsedBy(connID)
oldAddr := ed.regions.AddrUsedBy(connIndex)
if oldAddr != nil {
ed.regions.GiveBack(oldAddr)
}
addr := ed.regions.GetUnusedAddr(oldAddr, connID)
addr := ed.regions.GetUnusedAddr(oldAddr, connIndex)
if addr == nil {
ed.logger.Debugf("edgediscovery - GetDifferentAddr: No addresses left to give proxy connection: connID: %d", connID)
log.Debug().Msg("edgediscovery - GetDifferentAddr: No addresses left to give proxy connection")
// note: if oldAddr were not nil, it will become available on the next iteration
return nil, errNoAddressesLeft
}
ed.logger.Debugf("edgediscovery - GetDifferentAddr: Giving connection its new address %s: connID: %d", addr, connID)
log.Debug().Str(LogFieldAddress, addr.String()).Msg("edgediscovery - GetDifferentAddr: Giving connection its new address")
return addr, nil
}
@@ -127,6 +132,6 @@ func (ed *Edge) AvailableAddrs() int {
func (ed *Edge) GiveBack(addr *net.TCPAddr) bool {
ed.Lock()
defer ed.Unlock()
ed.logger.Debug("edgediscovery - GiveBack: Address now unused")
ed.log.Debug().Msg("edgediscovery - GiveBack: Address now unused")
return ed.regions.GiveBack(addr)
}
+11 -21
View File
@@ -4,7 +4,7 @@ import (
"net"
"testing"
"github.com/cloudflare/cloudflared/logger"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
)
@@ -29,11 +29,12 @@ var (
Port: 8000,
Zone: "",
}
log = zerolog.Nop()
)
func TestGiveBack(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
edge := MockEdge(l, []*net.TCPAddr{&addr0, &addr1, &addr2, &addr3})
edge := MockEdge(&log, []*net.TCPAddr{&addr0, &addr1, &addr2, &addr3})
// Give this connection an address
assert.Equal(t, 4, edge.AvailableAddrs())
@@ -49,10 +50,8 @@ func TestGiveBack(t *testing.T) {
}
func TestRPCAndProxyShareSingleEdgeIP(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
// Make an edge with a single IP
edge := MockEdge(l, []*net.TCPAddr{&addr0})
edge := MockEdge(&log, []*net.TCPAddr{&addr0})
tunnelConnID := 0
// Use the IP for a tunnel
@@ -66,8 +65,7 @@ func TestRPCAndProxyShareSingleEdgeIP(t *testing.T) {
}
func TestGetAddrForRPC(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
edge := MockEdge(l, []*net.TCPAddr{&addr0, &addr1, &addr2, &addr3})
edge := MockEdge(&log, []*net.TCPAddr{&addr0, &addr1, &addr2, &addr3})
// Get a connection
assert.Equal(t, 4, edge.AvailableAddrs())
@@ -84,10 +82,8 @@ func TestGetAddrForRPC(t *testing.T) {
}
func TestOnePerRegion(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
// Make an edge with only one address
edge := MockEdge(l, []*net.TCPAddr{&addr0, &addr1})
edge := MockEdge(&log, []*net.TCPAddr{&addr0, &addr1})
// Use the only address
const connID = 0
@@ -108,10 +104,8 @@ func TestOnePerRegion(t *testing.T) {
}
func TestOnlyOneAddrLeft(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
// Make an edge with only one address
edge := MockEdge(l, []*net.TCPAddr{&addr0})
edge := MockEdge(&log, []*net.TCPAddr{&addr0})
// Use the only address
const connID = 0
@@ -130,10 +124,8 @@ func TestOnlyOneAddrLeft(t *testing.T) {
}
func TestNoAddrsLeft(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
// Make an edge with no addresses
edge := MockEdge(l, []*net.TCPAddr{})
edge := MockEdge(&log, []*net.TCPAddr{})
_, err := edge.GetAddr(2)
assert.Error(t, err)
@@ -142,8 +134,7 @@ func TestNoAddrsLeft(t *testing.T) {
}
func TestGetAddr(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
edge := MockEdge(l, []*net.TCPAddr{&addr0, &addr1, &addr2, &addr3})
edge := MockEdge(&log, []*net.TCPAddr{&addr0, &addr1, &addr2, &addr3})
// Give this connection an address
const connID = 0
@@ -158,8 +149,7 @@ func TestGetAddr(t *testing.T) {
}
func TestGetDifferentAddr(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
edge := MockEdge(l, []*net.TCPAddr{&addr0, &addr1, &addr2, &addr3})
edge := MockEdge(&log, []*net.TCPAddr{&addr0, &addr1, &addr2, &addr3})
// Give this connection an address
assert.Equal(t, 4, edge.AvailableAddrs())
+7 -6
View File
@@ -4,10 +4,10 @@ go 1.15
require (
github.com/DATA-DOG/go-sqlmock v1.3.3
github.com/acmacalister/skittles v0.0.0-20160609003031-7423546701e1
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d
github.com/acmacalister/skittles v0.0.0-20160609003031-7423546701e1 // indirect
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d // indirect
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239 // indirect
github.com/aws/aws-sdk-go v1.34.19
github.com/aws/aws-sdk-go v1.34.19 // indirect
github.com/certifi/gocertifi v0.0.0-20200211180108-c7c1fbc02894 // indirect
github.com/cloudflare/brotli-go v0.0.0-20191101163834-d34379f7ff93
github.com/cloudflare/golibs v0.0.0-20170913112048-333127dbecfc
@@ -15,7 +15,6 @@ require (
github.com/coreos/go-oidc v0.0.0-20171002155002-a93f71fdfe73
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf
github.com/denisenkom/go-mssqldb v0.0.0-20191001013358-cfbb681360f0
github.com/equinox-io/equinox v1.2.0 // indirect
github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51 // indirect
github.com/facebookgo/freeport v0.0.0-20150612182905-d4adf43b75b9 // indirect
github.com/facebookgo/grace v0.0.0-20180706040059-75cf19382434
@@ -25,7 +24,7 @@ require (
github.com/fsnotify/fsnotify v1.4.9
github.com/gdamore/tcell v1.3.0
github.com/getsentry/raven-go v0.0.0-20180517221441-ed7bcb39ff10
github.com/gliderlabs/ssh v0.0.0-20191009160644-63518b5243e0
github.com/gliderlabs/ssh v0.0.0-20191009160644-63518b5243e0 // indirect
github.com/go-sql-driver/mysql v1.5.0
github.com/gobwas/httphead v0.0.0-20200921212729-da3d93bc3c58 // indirect
github.com/gobwas/pool v0.2.1 // indirect
@@ -41,6 +40,7 @@ require (
github.com/kshvakov/clickhouse v1.3.11
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/lib/pq v1.2.0
github.com/mattn/go-colorable v0.1.8
github.com/mattn/go-sqlite3 v1.11.0
github.com/miekg/dns v1.1.31
github.com/mitchellh/go-homedir v1.1.0
@@ -51,8 +51,8 @@ require (
github.com/pquerna/cachecontrol v0.0.0-20180517163645-1555304b9b35 // indirect
github.com/prometheus/client_golang v1.7.1
github.com/prometheus/common v0.13.0 // indirect
github.com/rifflock/lfshook v0.0.0-20180920164130-b9218ef580f5 // indirect
github.com/rivo/tview v0.0.0-20200712113419-c65badfc3d92
github.com/rs/zerolog v1.20.0
github.com/stretchr/testify v1.6.0
github.com/urfave/cli/v2 v2.2.0
github.com/xo/dburl v0.0.0-20191005012637-293c3298d6c0
@@ -65,6 +65,7 @@ require (
google.golang.org/grpc v1.32.0 // indirect
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f // indirect
gopkg.in/coreos/go-oidc.v2 v2.1.0
gopkg.in/natefinch/lumberjack.v2 v2.0.0
gopkg.in/square/go-jose.v2 v2.4.0 // indirect
gopkg.in/yaml.v2 v2.3.0
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776 // indirect
+27 -6
View File
@@ -100,7 +100,6 @@ github.com/aws/aws-lambda-go v1.13.3/go.mod h1:4UKl9IzQMoD+QF79YdCuzCwp8VbmG4VAQ
github.com/aws/aws-sdk-go v1.23.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-sdk-go v1.32.1/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
github.com/aws/aws-sdk-go v1.34.19 h1:x3MMvAJ1nfWviixEduchBSs65DgY5Y2pA2/NAcxVGOo=
github.com/aws/aws-sdk-go v1.34.19/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
github.com/aws/aws-sdk-go-v2 v0.18.0/go.mod h1:JWVYvqSMppoMJC0x5wdwiImzgXTI9FuZwxzkQq9wy+g=
github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f/go.mod h1:AuiFmCCPBSrqvVMvuqFuk0qogytodnVFVSN5CeJB8Gc=
@@ -109,6 +108,7 @@ github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+Ce
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=
github.com/bkaradzic/go-lz4 v1.0.0 h1:RXc4wYsyz985CkXXeX04y4VnZFGG8Rd43pRaHsOXAKk=
github.com/bkaradzic/go-lz4 v1.0.0/go.mod h1:0YdlkowM3VswSROI7qDxhRvJ3sLhlFrRRwjwegp5jy4=
github.com/caddyserver/caddy v1.0.5 h1:5B1Hs0UF2x2tggr2X9jL2qOZtDXbIWQb9YLbmlxHSuM=
github.com/caddyserver/caddy v1.0.5/go.mod h1:AnFHB+/MrgRC+mJAvuAgQ38ePzw+wKeW0wzENpdQQKY=
@@ -144,6 +144,7 @@ github.com/coreos/go-oidc v0.0.0-20171002155002-a93f71fdfe73 h1:7CNPV0LWRCa1FNmq
github.com/coreos/go-oidc v0.0.0-20171002155002-a93f71fdfe73/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf h1:iW4rZ826su+pqaw19uhpSCzhj44qo35pNgKFGqzDKkU=
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd/v22 v22.0.0/go.mod h1:xO0FLkIi5MaZafQlIrOotqXZ90ih+1atmu1JpKERPPk=
@@ -179,15 +180,17 @@ github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymF
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/equinox-io/equinox v1.2.0 h1:bBS7Ou+Y7Jwgmy8TWSYxEh85WctuFn7FPlgbUzX4DBA=
github.com/equinox-io/equinox v1.2.0/go.mod h1:6s3HJB0PYUNgs0mxmI8fHdfVl3TQ25ieA/PVfr+eyVo=
github.com/evanphx/json-patch v4.2.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/exoscale/egoscale v0.18.1/go.mod h1:Z7OOdzzTOz1Q1PjQXumlz9Wn/CddH0zSYdCF3rnBKXE=
github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51 h1:0JZ+dUmQeA8IIVUMzysrX4/AKuQwWhV2dYQuPZdvdSQ=
github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51/go.mod h1:Yg+htXGokKKdzcwhuNDwVvN+uBxDGXJ7G/VN1d8fa64=
github.com/facebookgo/freeport v0.0.0-20150612182905-d4adf43b75b9 h1:wWke/RUCl7VRjQhwPlR/v0glZXNYzBHdNUzf/Am2Nmg=
github.com/facebookgo/freeport v0.0.0-20150612182905-d4adf43b75b9/go.mod h1:uPmAp6Sws4L7+Q/OokbWDAK1ibXYhB3PXFP1kol5hPg=
github.com/facebookgo/grace v0.0.0-20180706040059-75cf19382434 h1:mOp33BLbcbJ8fvTAmZacbBiOASfxN+MLcLxymZCIrGE=
github.com/facebookgo/grace v0.0.0-20180706040059-75cf19382434/go.mod h1:KigFdumBXUPSwzLDbeuzyt0elrL7+CP7TKuhrhT4bcU=
github.com/facebookgo/stack v0.0.0-20160209184415-751773369052 h1:JWuenKqqX8nojtoVVWjGfOF9635RETekkoH6Cc9SX0A=
github.com/facebookgo/stack v0.0.0-20160209184415-751773369052/go.mod h1:UbMTZqLaRiH3MsBH8va0n7s1pQYcu3uTb8G4tygF4Zg=
github.com/facebookgo/subset v0.0.0-20150612182917-8dac2c3c4870 h1:E2s37DuLxFhQDg5gKsWoLBOB0n+ZW8s599zru8FJ2/Y=
github.com/facebookgo/subset v0.0.0-20150612182917-8dac2c3c4870/go.mod h1:5tD+neXqOorC30/tWg0LCSkrqj/AR6gu8yY8/fpw1q0=
github.com/farsightsec/golang-framestream v0.0.0-20190425193708-fa4b164d59b8/go.mod h1:eNde4IQyEiA5br02AouhEHCu3p3UzrCdFR4LuQHklMI=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
@@ -196,6 +199,7 @@ github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568 h1:BHsljHzVlRcyQhjr
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
github.com/franela/goblin v0.0.0-20200105215937-c9ffbefa60db/go.mod h1:7dvUGVsVBjqR7JHJk0brhHOZYGmfBYOrK0ZhYMEtBr4=
github.com/franela/goreq v0.0.0-20171204163338-bcd34c9993f8/go.mod h1:ZhphrRTfi2rbfLwlschooIH4+wKKDR4Pdxhh+TRoA20=
github.com/frankban/quicktest v1.10.0 h1:Gfh+GAJZOAoKZsIZeZbdn2JF10kN1XHNvjsvQK8gVkE=
github.com/frankban/quicktest v1.10.0/go.mod h1:ui7WezCLWMWxVWr1GETZY3smRy0G4KWq9vcPtJmFl7Y=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
@@ -289,6 +293,7 @@ github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2 h1:X2ev0eStA3AbceY54o37/0PQ/UWqKEiiO2dKL5OPaFM=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
@@ -392,13 +397,16 @@ github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxv
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.2.0 h1:s5hAObm+yFO5uHYt5dYjxi2rXrsnmRpJx4OYvIWUaQs=
github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kshvakov/clickhouse v1.3.11 h1:dtzTJY0fCA+MWkLyuKZaNPkmSwdX4gh8+Klic9NB1Lw=
github.com/kshvakov/clickhouse v1.3.11/go.mod h1:/SVBAcqF3u7rxQ9sTWCZwf8jzzvxiZGeQvtmSF2BBEc=
github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348/go.mod h1:B69LEHPfb2qLo0BaaOLcbitczOKLWTsrBG9LczfCD4k=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/labbsr0x/bindman-dns-webhook v1.0.2/go.mod h1:p6b+VCXIR8NYKpDr8/dg1HKfQoRHCdcsROXKvmoehKA=
github.com/labbsr0x/goh v1.0.1/go.mod h1:8K2UhVoaWXcCU7Lxoa2omWnC8gyW8px7/lmO61c027w=
@@ -419,8 +427,12 @@ github.com/marten-seemann/chacha20 v0.2.0/go.mod h1:HSdjFau7GzYRj+ahFNwsO3ouVJr1
github.com/marten-seemann/qpack v0.1.0/go.mod h1:LFt1NU/Ptjip0C2CPkhimBz5CGE3WGDAUWqna+CNTrI=
github.com/marten-seemann/qtls v0.4.1/go.mod h1:pxVXcHHw1pNIt8Qo0pwSYQEoZ8yYOOPXTCZLQQunvRc=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-colorable v0.1.8 h1:c1ghPdyEDarC70ftn0y+A/Ee++9zz8ljHG1b13eJ0s8=
github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-runewidth v0.0.8 h1:3tS41NlGYSmhhe/8fhGRzc+z3AYCw1Fe1WAyLuujKs0=
@@ -468,6 +480,7 @@ github.com/nats-io/nkeys v0.1.0/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxzi
github.com/nats-io/nkeys v0.1.3/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/nbio/st v0.0.0-20140626010706-e9e8d9816f32/go.mod h1:9wM+0iRr9ahx58uYLpLIr5fm8diHn0JbqRycJi6w0Ms=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/nrdcg/auroradns v1.0.0/go.mod h1:6JPXKzIRzZzMqtTDgueIhTi6rFf1QvYE/HzqidhOhjw=
github.com/nrdcg/goinwx v0.6.1/go.mod h1:XPiut7enlbEdntAqalBIqcYcTEVhpv/dKWgDCX2SwKQ=
@@ -502,9 +515,11 @@ github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FI
github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k=
github.com/performancecopilot/speed v3.0.0+incompatible/go.mod h1:/CLtqpZ5gBg1M9iaPbIdPPGyKcA8hKdoy6hAWba7Yac=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
github.com/philhofer/fwd v1.0.0 h1:UbZqGr5Y38ApvM/V/jEljVxwocdweyH+vmYvRPBnbqQ=
github.com/philhofer/fwd v1.0.0/go.mod h1:gk3iGcWd9+svBvR0sR+KPcfE+RNWozjowpeBVG3ZVNU=
github.com/pierrec/lz4 v1.0.2-0.20190131084431-473cd7ce01a1/go.mod h1:3/3N9NVKO0jef7pBehbT1qWhCMrIgbYNnFAZCqQ5LRc=
github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
github.com/pierrec/lz4 v2.5.2+incompatible h1:WCjObylUIOlKy/+7Abdn34TLIkXiA4UWUMhxq9m9ZXI=
github.com/pierrec/lz4 v2.5.2+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -543,7 +558,6 @@ github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R
github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/procfs v0.0.3/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ=
github.com/prometheus/procfs v0.0.5/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ=
github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
github.com/prometheus/procfs v0.0.11/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
github.com/prometheus/procfs v0.1.3 h1:F0+tqvhOksq22sc6iCHF5WGlWjdwj92p0udFh1VFBS8=
@@ -551,13 +565,15 @@ github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4O
github.com/prometheus/prometheus v2.5.0+incompatible/go.mod h1:oAIUtOny2rjMX0OWN5vPR5/q/twIROJvdqnQKDdil/s=
github.com/rainycape/memcache v0.0.0-20150622160815-1031fa0ce2f2/go.mod h1:7tZKcyumwBO6qip7RNQ5r77yrssm9bfCowcLEBcU5IA=
github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
github.com/rifflock/lfshook v0.0.0-20180920164130-b9218ef580f5/go.mod h1:GEXHk5HgEKCvEIIrSpFI3ozzG5xOKA2DVlEX/gGnewM=
github.com/rivo/tview v0.0.0-20200712113419-c65badfc3d92 h1:rqaqSUdaW+OBbjnsrOoiaJv43mSRARuvsAuirmdxu7E=
github.com/rivo/tview v0.0.0-20200712113419-c65badfc3d92/go.mod h1:6lkG1x+13OShEf0EaOCaTQYyB7d5nSbb181KtjlS+84=
github.com/rivo/uniseg v0.1.0 h1:+2KBaVoUmb9XzDsrx/Ct0W/EYOSFf/nWTauy++DprtY=
github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rs/xid v1.2.1/go.mod h1:+uKXf+4Djp6Md1KODXJxgGQPKngRmWyn10oCKFzNHOQ=
github.com/rs/zerolog v1.20.0 h1:38k9hgtUBdxFwE34yS8rTHmHBa4eN16E4DJlv177LNs=
github.com/rs/zerolog v1.20.0/go.mod h1:IzD0RJ65iWH0w97OQQebJEvTZYvsCUm9WVLWBQrJRjo=
github.com/russross/blackfriday v0.0.0-20170610170232-067529f716f4 h1:S9YlS71UNJIyS61OqGAmLXv3w5zclSidN+qwr80XxKs=
github.com/russross/blackfriday v0.0.0-20170610170232-067529f716f4/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
github.com/russross/blackfriday/v2 v2.0.1 h1:lPqVAte+HuHNfhJ/0LC98ESWRz8afy9tM/0RK8m9o+Q=
@@ -595,7 +611,7 @@ github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5
github.com/stretchr/testify v1.6.0 h1:jlIyCplCJFULU/01vCkhKuTyc3OorI3bJFuw6obfgho=
github.com/stretchr/testify v1.6.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/timewasted/linode v0.0.0-20160829202747-37e84520dcf7/go.mod h1:imsgLplxEC/etjIhdr3dNzV3JeT27LbVu5pYWm0JCBY=
github.com/tinylib/msgp v1.1.0/go.mod h1:+d+yLhGm8mzTaHzB+wgMYrodPfmZrzkirds8fDWklFE=
github.com/tinylib/msgp v1.1.2 h1:gWmO7n0Ys2RBEb7GPYB9Ujq8Mk5p2U08lRnmMcGy6BQ=
github.com/tinylib/msgp v1.1.2/go.mod h1:+d+yLhGm8mzTaHzB+wgMYrodPfmZrzkirds8fDWklFE=
github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/transip/gotransip v0.0.0-20190812104329-6d8d9179b66f/go.mod h1:i0f4R4o2HM0m3DZYQWsj6/MEowD57VzoH0v3d7igeFY=
@@ -783,6 +799,7 @@ golang.org/x/sys v0.0.0-20191220142924-d4481acd189f/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -830,6 +847,7 @@ golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgw
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20190828213141-aed303cbaa74/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
@@ -865,6 +883,7 @@ golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.3.1/go.mod h1:6wY9I6uQWHQ8EM57III9mq/AjF+i8G65rmVagqKMtkk=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
@@ -961,6 +980,7 @@ gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLks
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU=
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw=
gopkg.in/coreos/go-oidc.v2 v2.1.0 h1:E8PjVFdj/SLDKB0hvb70KTbMbYVHjqztiQdSkIg8E+I=
@@ -973,6 +993,7 @@ gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/ini.v1 v1.42.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/ini.v1 v1.44.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/mcuadros/go-syslog.v2 v2.2.1/go.mod h1:l5LPIyOOyIdQquNg+oU6Z3524YwrcqEm0aKH+5zpt2U=
gopkg.in/natefinch/lumberjack.v2 v2.0.0 h1:1Lc07Kr7qY4U2YPouBjpCLxpiyxIVoxqXgkXLknAOE8=
gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k=
gopkg.in/ns1/ns1-go.v2 v2.0.0-20190730140822-b51389932cbc/go.mod h1:VV+3haRsgDiVLxyifmMBrBIuCWFBPYKbRssXB9z67Hw=
gopkg.in/resty.v1 v1.9.1/go.mod h1:vo52Hzryw9PnPHcJfPsBiFW62XhNx5OczbV9y+IMpgc=
+9 -10
View File
@@ -8,11 +8,10 @@ import (
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/rs/zerolog"
"golang.org/x/net/http2"
"golang.org/x/net/http2/hpack"
"golang.org/x/sync/errgroup"
"github.com/cloudflare/cloudflared/logger"
)
const (
@@ -50,7 +49,7 @@ type MuxerConfig struct {
// The minimum number of heartbeats to send before terminating the connection.
MaxHeartbeats uint64
// Logger to use
Logger logger.Service
Log *zerolog.Logger
CompressionQuality CompressionSetting
// Initial size for HTTP2 flow control windows
DefaultWindowSize uint32
@@ -138,10 +137,10 @@ func Handshake(
handshakeSetting := http2.Setting{ID: SettingMuxerMagic, Val: MuxerMagicEdge}
compressionSetting := http2.Setting{ID: SettingCompression, Val: config.CompressionQuality.toH2Setting()}
if CompressionIsSupported() {
config.Logger.Debug("muxer: Compression is supported")
config.Log.Debug().Msg("muxer: Compression is supported")
m.compressionQuality = config.CompressionQuality.getPreset()
} else {
config.Logger.Debug("muxer: Compression is not supported")
config.Log.Debug().Msg("muxer: Compression is not supported")
compressionSetting = http2.Setting{ID: SettingCompression, Val: 0}
}
@@ -178,12 +177,12 @@ func Handshake(
// Sanity check to enusre idelDuration is sane
if idleDuration == 0 || idleDuration < defaultTimeout {
idleDuration = defaultTimeout
config.Logger.Infof("muxer: Minimum idle time has been adjusted to %d", defaultTimeout)
config.Log.Info().Msgf("muxer: Minimum idle time has been adjusted to %d", defaultTimeout)
}
maxRetries := config.MaxHeartbeats
if maxRetries == 0 {
maxRetries = defaultRetries
config.Logger.Infof("muxer: Minimum number of unacked heartbeats to send before closing the connection has been adjusted to %d", maxRetries)
config.Log.Info().Msgf("muxer: Minimum number of unacked heartbeats to send before closing the connection has been adjusted to %d", maxRetries)
}
compBytesBefore, compBytesAfter := NewAtomicCounter(0), NewAtomicCounter(0)
@@ -325,7 +324,7 @@ func (m *Muxer) Serve(ctx context.Context) error {
errGroup.Go(func() error {
ch := make(chan error)
go func() {
err := m.muxReader.run(m.config.Logger)
err := m.muxReader.run(m.config.Log)
m.explicitShutdown.Fuse(false)
m.r.Close()
m.abort()
@@ -346,7 +345,7 @@ func (m *Muxer) Serve(ctx context.Context) error {
errGroup.Go(func() error {
ch := make(chan error)
go func() {
err := m.muxWriter.run(m.config.Logger)
err := m.muxWriter.run(m.config.Log)
m.explicitShutdown.Fuse(false)
m.w.Close()
m.abort()
@@ -367,7 +366,7 @@ func (m *Muxer) Serve(ctx context.Context) error {
errGroup.Go(func() error {
ch := make(chan error)
go func() {
err := m.muxMetricsUpdater.run(m.config.Logger)
err := m.muxMetricsUpdater.run(m.config.Log)
// don't block if parent goroutine quit early
select {
case ch <- err:
+55 -59
View File
@@ -16,10 +16,9 @@ import (
"time"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
"golang.org/x/sync/errgroup"
"github.com/cloudflare/cloudflared/logger"
)
const (
@@ -27,6 +26,8 @@ const (
testHandshakeTimeout = time.Millisecond * 1000
)
var log = zerolog.Nop()
func TestMain(m *testing.M) {
if os.Getenv("VERBOSE") == "1" {
//TODO: set log level
@@ -52,7 +53,7 @@ func NewDefaultMuxerPair(t assert.TestingT, testName string, f MuxedStreamFunc)
Handler: f,
IsClient: true,
Name: "origin",
Logger: logger.NewOutputWriter(logger.NewMockWriteManager()),
Log: &log,
DefaultWindowSize: (1 << 8) - 1,
MaxWindowSize: (1 << 15) - 1,
StreamWriteBufferMaxLen: 1024,
@@ -64,7 +65,7 @@ func NewDefaultMuxerPair(t assert.TestingT, testName string, f MuxedStreamFunc)
Timeout: testHandshakeTimeout,
IsClient: false,
Name: "edge",
Logger: logger.NewOutputWriter(logger.NewMockWriteManager()),
Log: &log,
DefaultWindowSize: (1 << 8) - 1,
MaxWindowSize: (1 << 15) - 1,
StreamWriteBufferMaxLen: 1024,
@@ -87,7 +88,7 @@ func NewCompressedMuxerPair(t assert.TestingT, testName string, quality Compress
IsClient: true,
Name: "origin",
CompressionQuality: quality,
Logger: logger.NewOutputWriter(logger.NewMockWriteManager()),
Log: &log,
HeartbeatInterval: defaultTimeout,
MaxHeartbeats: defaultRetries,
},
@@ -97,7 +98,7 @@ func NewCompressedMuxerPair(t assert.TestingT, testName string, quality Compress
IsClient: false,
Name: "edge",
CompressionQuality: quality,
Logger: logger.NewOutputWriter(logger.NewMockWriteManager()),
Log: &log,
HeartbeatInterval: defaultTimeout,
MaxHeartbeats: defaultRetries,
},
@@ -186,11 +187,11 @@ func TestSingleStream(t *testing.T) {
if stream.Headers[0].Value != "headerValue" {
t.Fatalf("expected header value %s, got %s", "headerValue", stream.Headers[0].Value)
}
stream.WriteHeaders([]Header{
_ = stream.WriteHeaders([]Header{
{Name: "response-header", Value: "responseValue"},
})
buf := []byte("Hello world")
stream.Write(buf)
_, _ = stream.Write(buf)
n, err := io.ReadFull(stream, buf)
if n > 0 {
t.Fatalf("read %d bytes after EOF", n)
@@ -230,7 +231,7 @@ func TestSingleStream(t *testing.T) {
if string(responseBody) != "Hello world" {
t.Fatalf("expected response body %s, got %s", "Hello world", responseBody)
}
stream.Close()
_ = stream.Close()
n, err = stream.Write([]byte("aaaaa"))
if n > 0 {
t.Fatalf("wrote %d bytes after EOF", n)
@@ -252,7 +253,7 @@ func TestSingleStreamLargeResponseBody(t *testing.T) {
if stream.Headers[0].Value != "headerValue" {
t.Fatalf("expected header value %s, got %s", "headerValue", stream.Headers[0].Value)
}
stream.WriteHeaders([]Header{
_ = stream.WriteHeaders([]Header{
{Name: "response-header", Value: "responseValue"},
})
payload := make([]byte, bodySize)
@@ -302,7 +303,6 @@ func TestSingleStreamLargeResponseBody(t *testing.T) {
}
func TestMultipleStreams(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
f := MuxedStreamFunc(func(stream *MuxedStream) error {
if len(stream.Headers) != 1 {
t.Fatalf("expected %d headers, got %d", 1, len(stream.Headers))
@@ -310,13 +310,13 @@ func TestMultipleStreams(t *testing.T) {
if stream.Headers[0].Name != "client-token" {
t.Fatalf("expected header name %s, got %s", "client-token", stream.Headers[0].Name)
}
l.Debugf("Got request for stream %s", stream.Headers[0].Value)
stream.WriteHeaders([]Header{
log.Debug().Msgf("Got request for stream %s", stream.Headers[0].Value)
_ = stream.WriteHeaders([]Header{
{Name: "response-token", Value: stream.Headers[0].Value},
})
l.Debugf("Wrote headers for stream %s", stream.Headers[0].Value)
stream.Write([]byte("OK"))
l.Debugf("Wrote body for stream %s", stream.Headers[0].Value)
log.Debug().Msgf("Wrote headers for stream %s", stream.Headers[0].Value)
_, _ = stream.Write([]byte("OK"))
log.Debug().Msgf("Wrote body for stream %s", stream.Headers[0].Value)
return nil
})
muxPair := NewDefaultMuxerPair(t, t.Name(), f)
@@ -334,7 +334,7 @@ func TestMultipleStreams(t *testing.T) {
[]Header{{Name: "client-token", Value: tokenString}},
nil,
)
l.Debugf("Got headers for stream %d", tokenId)
log.Debug().Msgf("Got headers for stream %d", tokenId)
if err != nil {
errorsC <- err
return
@@ -372,7 +372,7 @@ func TestMultipleStreams(t *testing.T) {
testFail := false
for err := range errorsC {
testFail = true
l.Errorf("%s", err)
log.Error().Msgf("%s", err)
}
if testFail {
t.Fatalf("TestMultipleStreams failed")
@@ -396,7 +396,7 @@ func TestMultipleStreamsFlowControl(t *testing.T) {
if stream.Headers[0].Value != "headerValue" {
t.Fatalf("expected header value %s, got %s", "headerValue", stream.Headers[0].Value)
}
stream.WriteHeaders([]Header{
_ = stream.WriteHeaders([]Header{
{Name: "response-header", Value: "responseValue"},
})
payload := make([]byte, responseSizes[(stream.streamID-2)/2])
@@ -450,27 +450,25 @@ func TestMultipleStreamsFlowControl(t *testing.T) {
}
func TestGracefulShutdown(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
sendC := make(chan struct{})
responseBuf := bytes.Repeat([]byte("Hello world"), 65536)
f := MuxedStreamFunc(func(stream *MuxedStream) error {
stream.WriteHeaders([]Header{
_ = stream.WriteHeaders([]Header{
{Name: "response-header", Value: "responseValue"},
})
<-sendC
l.Debugf("Writing %d bytes", len(responseBuf))
stream.Write(responseBuf)
stream.CloseWrite()
l.Debugf("Wrote %d bytes", len(responseBuf))
log.Debug().Msgf("Writing %d bytes", len(responseBuf))
_, _ = stream.Write(responseBuf)
_ = stream.CloseWrite()
log.Debug().Msgf("Wrote %d bytes", len(responseBuf))
// Reading from the stream will block until the edge closes its end of the stream.
// Otherwise, we'll close the whole connection before receiving the 'stream closed'
// message from the edge.
// Graceful shutdown works if you omit this, it just gives spurious errors for now -
// TODO ignore errors when writing 'stream closed' and we're shutting down.
stream.Read([]byte{0})
l.Debugf("Handler ends")
_, _ = stream.Read([]byte{0})
log.Debug().Msgf("Handler ends")
return nil
})
muxPair := NewDefaultMuxerPair(t, t.Name(), f)
@@ -487,7 +485,7 @@ func TestGracefulShutdown(t *testing.T) {
muxPair.EdgeMux.Shutdown()
close(sendC)
responseBody := make([]byte, len(responseBuf))
l.Debugf("Waiting for %d bytes", len(responseBuf))
log.Debug().Msgf("Waiting for %d bytes", len(responseBuf))
n, err := io.ReadFull(stream, responseBody)
if err != nil {
t.Fatalf("error from (*MuxedStream).Read with %d bytes read: %s", n, err)
@@ -498,7 +496,7 @@ func TestGracefulShutdown(t *testing.T) {
if !bytes.Equal(responseBuf, responseBody) {
t.Fatalf("response body mismatch")
}
stream.Close()
_ = stream.Close()
muxPair.Wait(t)
}
@@ -509,7 +507,7 @@ func TestUnexpectedShutdown(t *testing.T) {
f := MuxedStreamFunc(func(stream *MuxedStream) error {
defer close(handlerFinishC)
stream.WriteHeaders([]Header{
_ = stream.WriteHeaders([]Header{
{Name: "response-header", Value: "responseValue"},
})
<-sendC
@@ -536,7 +534,7 @@ func TestUnexpectedShutdown(t *testing.T) {
nil,
)
// Close the underlying connection before telling the origin to write.
muxPair.EdgeConn.Close()
_ = muxPair.EdgeConn.Close()
close(sendC)
if err != nil {
t.Fatalf("error in OpenStream: %s", err)
@@ -559,18 +557,18 @@ func TestUnexpectedShutdown(t *testing.T) {
func EchoHandler(stream *MuxedStream) error {
var buf bytes.Buffer
fmt.Fprintf(&buf, "Hello, world!\n\n# REQUEST HEADERS:\n\n")
_, _ = fmt.Fprintf(&buf, "Hello, world!\n\n# REQUEST HEADERS:\n\n")
for _, header := range stream.Headers {
fmt.Fprintf(&buf, "[%s] = %s\n", header.Name, header.Value)
_, _ = fmt.Fprintf(&buf, "[%s] = %s\n", header.Name, header.Value)
}
stream.WriteHeaders([]Header{
_ = stream.WriteHeaders([]Header{
{Name: ":status", Value: "200"},
{Name: "server", Value: "Echo-server/1.0"},
{Name: "date", Value: time.Now().Format(time.RFC850)},
{Name: "content-type", Value: "text/html; charset=utf-8"},
{Name: "content-length", Value: strconv.Itoa(buf.Len())},
})
buf.WriteTo(stream)
_, _ = buf.WriteTo(stream)
return nil
}
@@ -582,14 +580,14 @@ func TestOpenAfterDisconnect(t *testing.T) {
switch i {
case 0:
// Close both directions of the connection to cause EOF on both peers.
muxPair.OriginConn.Close()
muxPair.EdgeConn.Close()
_ = muxPair.OriginConn.Close()
_ = muxPair.EdgeConn.Close()
case 1:
// Close origin conn to cause EOF on origin first.
muxPair.OriginConn.Close()
_ = muxPair.OriginConn.Close()
case 2:
// Close edge conn to cause EOF on edge first.
muxPair.EdgeConn.Close()
_ = muxPair.EdgeConn.Close()
}
_, err := muxPair.OpenEdgeMuxStream(
@@ -617,7 +615,7 @@ func TestHPACK(t *testing.T) {
if err != nil {
t.Fatalf("error in OpenStream: %s", err)
}
stream.Close()
_ = stream.Close()
for i := 0; i < 3; i++ {
stream, err := muxPair.OpenEdgeMuxStream(
@@ -654,8 +652,8 @@ func TestHPACK(t *testing.T) {
if stream.Headers[0].Value != "200" {
t.Fatalf("expected status 200, got %s", stream.Headers[0].Value)
}
ioutil.ReadAll(stream)
stream.Close()
_, _ = ioutil.ReadAll(stream)
_ = stream.Close()
}
}
@@ -680,7 +678,7 @@ func AssertIfPipeReadable(t *testing.T, pipe io.ReadCloser) {
}
func TestMultipleStreamsWithDictionaries(t *testing.T) {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
l := zerolog.Nop()
for q := CompressionNone; q <= CompressionMax; q++ {
htmlBody := `<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"` +
@@ -730,10 +728,10 @@ func TestMultipleStreamsWithDictionaries(t *testing.T) {
contentType = "img/gif"
}
stream.WriteHeaders([]Header{
Header{Name: "content-type", Value: contentType},
_ = stream.WriteHeaders([]Header{
{Name: "content-type", Value: contentType},
})
stream.Write([]byte(strings.Replace(htmlBody, "paragraph", pathHeader.Value, 1) + stream.Headers[5].Value))
_, _ = stream.Write([]byte(strings.Replace(htmlBody, "paragraph", pathHeader.Value, 1) + stream.Headers[5].Value))
return nil
})
@@ -817,7 +815,7 @@ func TestMultipleStreamsWithDictionaries(t *testing.T) {
testFail := false
for err := range errorsC {
testFail = true
l.Errorf("%s", err)
l.Error().Msgf("%s", err)
}
if testFail {
t.Fatalf("TestMultipleStreams failed")
@@ -831,8 +829,6 @@ func TestMultipleStreamsWithDictionaries(t *testing.T) {
}
func sampleSiteHandler(files map[string][]byte) MuxedStreamFunc {
l := logger.NewOutputWriter(logger.NewMockWriteManager())
return func(stream *MuxedStream) error {
var contentType string
var pathHeader Header
@@ -857,16 +853,16 @@ func sampleSiteHandler(files map[string][]byte) MuxedStreamFunc {
} else {
contentType = "img/gif"
}
stream.WriteHeaders([]Header{
Header{Name: "content-type", Value: contentType},
_ = stream.WriteHeaders([]Header{
{Name: "content-type", Value: contentType},
})
l.Debugf("Wrote headers for stream %s", pathHeader.Value)
log.Debug().Msgf("Wrote headers for stream %s", pathHeader.Value)
file, ok := files[pathHeader.Value]
if !ok {
return fmt.Errorf("%s content is not preloaded", pathHeader.Value)
}
stream.Write(file)
l.Debugf("Wrote body for stream %s", pathHeader.Value)
_, _ = stream.Write(file)
log.Debug().Msgf("Wrote body for stream %s", pathHeader.Value)
return nil
}
}
@@ -1008,7 +1004,7 @@ func BenchmarkOpenStream(b *testing.B) {
if stream.Headers[0].Value != "headerValue" {
b.Fatalf("expected header value %s, got %s", "headerValue", stream.Headers[0].Value)
}
stream.WriteHeaders([]Header{
_ = stream.WriteHeaders([]Header{
{Name: "response-header", Value: "responseValue"},
})
return nil
@@ -1058,7 +1054,7 @@ func BenchmarkSingleStreamLargeResponseBody(b *testing.B) {
if stream.Headers[0].Value != "headerValue" {
b.Fatalf("expected header value %s, got %s", "headerValue", stream.Headers[0].Value)
}
stream.WriteHeaders([]Header{
_ = stream.WriteHeaders([]Header{
{Name: "response-header", Value: "responseValue"},
})
for i := 0; i < writeN; i++ {
@@ -1083,7 +1079,7 @@ func BenchmarkSingleStreamLargeResponseBody(b *testing.B) {
Handler: f,
IsClient: true,
Name: "origin",
Logger: logger.NewOutputWriter(logger.NewMockWriteManager()),
Log: &log,
DefaultWindowSize: defaultWindowSize,
MaxWindowSize: maxWindowSize,
StreamWriteBufferMaxLen: defaultWriteBufferMaxLen,
@@ -1095,7 +1091,7 @@ func BenchmarkSingleStreamLargeResponseBody(b *testing.B) {
Timeout: testHandshakeTimeout,
IsClient: false,
Name: "edge",
Logger: logger.NewOutputWriter(logger.NewMockWriteManager()),
Log: &log,
DefaultWindowSize: defaultWindowSize,
MaxWindowSize: maxWindowSize,
StreamWriteBufferMaxLen: defaultWriteBufferMaxLen,
+10 -10
View File
@@ -4,8 +4,8 @@ import (
"sync"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/golang-collections/collections/queue"
"github.com/rs/zerolog"
)
// data points used to compute average receive window and send window size
@@ -20,7 +20,7 @@ type muxMetricsUpdater interface {
// metrics returns the latest metrics
metrics() *MuxerMetrics
// run is a blocking call to start the event loop
run(logger logger.Service) error
run(log *zerolog.Logger) error
// updateRTTChan is called by muxReader to report new RTT measurements
updateRTT(rtt *roundTripMeasurement)
//updateReceiveWindowChan is called by muxReader and muxWriter when receiveWindow size is updated
@@ -137,30 +137,30 @@ func (updater *muxMetricsUpdaterImpl) metrics() *MuxerMetrics {
return m
}
func (updater *muxMetricsUpdaterImpl) run(logger logger.Service) error {
defer logger.Debug("mux - metrics: event loop finished")
func (updater *muxMetricsUpdaterImpl) run(log *zerolog.Logger) error {
defer log.Debug().Msg("mux - metrics: event loop finished")
for {
select {
case <-updater.abortChan:
logger.Infof("mux - metrics: Stopping mux metrics updater")
log.Info().Msgf("mux - metrics: Stopping mux metrics updater")
return nil
case roundTripMeasurement := <-updater.updateRTTChan:
go updater.rttData.update(roundTripMeasurement)
logger.Debug("mux - metrics: Update rtt")
log.Debug().Msg("mux - metrics: Update rtt")
case receiveWindow := <-updater.updateReceiveWindowChan:
go updater.receiveWindowData.update(receiveWindow)
logger.Debug("mux - metrics: Update receive window")
log.Debug().Msg("mux - metrics: Update receive window")
case sendWindow := <-updater.updateSendWindowChan:
go updater.sendWindowData.update(sendWindow)
logger.Debug("mux - metrics: Update send window")
log.Debug().Msg("mux - metrics: Update send window")
case inBoundBytes := <-updater.updateInBoundBytesChan:
// inBoundBytes is bytes/sec because the update interval is 1 sec
go updater.inBoundRate.update(inBoundBytes)
logger.Debugf("mux - metrics: Inbound bytes %d", inBoundBytes)
log.Debug().Msgf("mux - metrics: Inbound bytes %d", inBoundBytes)
case outBoundBytes := <-updater.updateOutBoundBytesChan:
// outBoundBytes is bytes/sec because the update interval is 1 sec
go updater.outBoundRate.update(outBoundBytes)
logger.Debugf("mux - metrics: Outbound bytes %d", outBoundBytes)
log.Debug().Msgf("mux - metrics: Outbound bytes %d", outBoundBytes)
}
}
}
+3 -3
View File
@@ -5,7 +5,7 @@ import (
"testing"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
)
@@ -91,10 +91,10 @@ func TestMuxMetricsUpdater(t *testing.T) {
abortChan := make(chan struct{})
compBefore, compAfter := NewAtomicCounter(0), NewAtomicCounter(0)
m := newMuxMetricsUpdater(abortChan, compBefore, compAfter)
logger := logger.NewOutputWriter(logger.NewMockWriteManager())
log := zerolog.Nop()
go func() {
errChan <- m.run(logger)
errChan <- m.run(&log)
}()
var wg sync.WaitGroup
+14 -14
View File
@@ -8,7 +8,7 @@ import (
"net/url"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/rs/zerolog"
"golang.org/x/net/http2"
)
@@ -68,8 +68,8 @@ func (r *MuxReader) Shutdown() <-chan struct{} {
return done
}
func (r *MuxReader) run(logger logger.Service) error {
defer logger.Debug("mux - read: event loop finished")
func (r *MuxReader) run(log *zerolog.Logger) error {
defer log.Debug().Msg("mux - read: event loop finished")
// routine to periodically update bytesRead
go func() {
@@ -93,7 +93,7 @@ func (r *MuxReader) run(logger logger.Service) error {
}
switch e := err.(type) {
case http2.StreamError:
logger.Infof("%s: stream error", errorString)
log.Info().Msgf("%s: stream error", errorString)
// Ideally we wouldn't return here, since that aborts the muxer.
// We should communicate the error to the relevant MuxedStream
// data structure, so that callers of MuxedStream.Read() and
@@ -101,28 +101,28 @@ func (r *MuxReader) run(logger logger.Service) error {
// and keep the muxer going.
return r.streamError(e.StreamID, e.Code)
case http2.ConnectionError:
logger.Infof("%s: stream error", errorString)
log.Info().Msgf("%s: stream error", errorString)
return r.connectionError(err)
default:
if isConnectionClosedError(err) {
if r.streams.Len() == 0 {
// don't log the error here -- that would just be extra noise
logger.Debug("mux - read: shutting down")
log.Debug().Msg("mux - read: shutting down")
return nil
}
logger.Infof("%s: connection closed unexpectedly", errorString)
log.Info().Msgf("%s: connection closed unexpectedly", errorString)
return err
} else {
logger.Infof("%s: frame read error", errorString)
log.Info().Msgf("%s: frame read error", errorString)
return r.connectionError(err)
}
}
}
r.connActive.Signal()
logger.Debugf("mux - read: read frame: data %v", frame)
log.Debug().Msgf("mux - read: read frame: data %v", frame)
switch f := frame.(type) {
case *http2.DataFrame:
err = r.receiveFrameData(f, logger)
err = r.receiveFrameData(f, log)
case *http2.MetaHeadersFrame:
err = r.receiveHeaderData(f)
case *http2.RSTStreamFrame:
@@ -155,7 +155,7 @@ func (r *MuxReader) run(logger logger.Service) error {
err = ErrUnexpectedFrameType
}
if err != nil {
logger.Debugf("mux - read: read error: data %v", frame)
log.Debug().Msgf("mux - read: read error: data %v", frame)
return r.connectionError(err)
}
}
@@ -276,7 +276,7 @@ func (r *MuxReader) handleStream(stream *MuxedStream) {
}
// Receives a data frame from a stream. A non-nil error is a connection error.
func (r *MuxReader) receiveFrameData(frame *http2.DataFrame, logger logger.Service) error {
func (r *MuxReader) receiveFrameData(frame *http2.DataFrame, log *zerolog.Logger) error {
stream, err := r.getStreamForFrame(frame)
if err != nil {
return r.defaultStreamErrorHandler(err, frame.Header())
@@ -292,9 +292,9 @@ func (r *MuxReader) receiveFrameData(frame *http2.DataFrame, logger logger.Servi
if frame.Header().Flags.Has(http2.FlagDataEndStream) {
if stream.receiveEOF() {
r.streams.Delete(stream.streamID)
logger.Debugf("mux - read: stream closed: streamID: %d", frame.Header().StreamID)
log.Debug().Msgf("mux - read: stream closed: streamID: %d", frame.Header().StreamID)
} else {
logger.Debugf("mux - read: shutdown receive side: streamID: %d", frame.Header().StreamID)
log.Debug().Msgf("mux - read: shutdown receive side: streamID: %d", frame.Header().StreamID)
}
return nil
}
+22 -22
View File
@@ -3,10 +3,10 @@ package h2mux
import (
"bytes"
"encoding/binary"
"github.com/rs/zerolog"
"io"
"time"
"github.com/cloudflare/cloudflared/logger"
"golang.org/x/net/http2"
"golang.org/x/net/http2/hpack"
)
@@ -72,8 +72,8 @@ func tsToPingData(ts int64) [8]byte {
return pingData
}
func (w *MuxWriter) run(logger logger.Service) error {
defer logger.Debug("mux - write: event loop finished")
func (w *MuxWriter) run(log *zerolog.Logger) error {
defer log.Debug().Msg("mux - write: event loop finished")
// routine to periodically communicate bytesWrote
go func() {
@@ -91,17 +91,17 @@ func (w *MuxWriter) run(logger logger.Service) error {
for {
select {
case <-w.abortChan:
logger.Debug("mux - write: aborting writer thread")
log.Debug().Msg("mux - write: aborting writer thread")
return nil
case errCode := <-w.goAwayChan:
logger.Debugf("mux - write: sending GOAWAY code %v", errCode)
log.Debug().Msgf("mux - write: sending GOAWAY code %v", errCode)
err := w.f.WriteGoAway(w.streams.LastPeerStreamID(), errCode, []byte{})
if err != nil {
return err
}
w.idleTimer.MarkActive()
case <-w.pingTimestamp.GetUpdateChan():
logger.Debug("mux - write: sending PING ACK")
log.Debug().Msg("mux - write: sending PING ACK")
err := w.f.WritePing(true, tsToPingData(w.pingTimestamp.Get()))
if err != nil {
return err
@@ -111,7 +111,7 @@ func (w *MuxWriter) run(logger logger.Service) error {
if !w.idleTimer.Retry() {
return ErrConnectionDropped
}
logger.Debug("mux - write: sending PING")
log.Debug().Msg("mux - write: sending PING")
err := w.f.WritePing(false, tsToPingData(time.Now().UnixNano()))
if err != nil {
return err
@@ -121,7 +121,7 @@ func (w *MuxWriter) run(logger logger.Service) error {
w.idleTimer.MarkActive()
case <-w.streamErrors.GetSignalChan():
for streamID, errCode := range w.streamErrors.GetErrors() {
logger.Debugf("mux - write: resetting stream with code: %v streamID: %d", errCode, streamID)
log.Debug().Msgf("mux - write: resetting stream with code: %v streamID: %d", errCode, streamID)
err := w.f.WriteRSTStream(streamID, errCode)
if err != nil {
return err
@@ -141,7 +141,7 @@ func (w *MuxWriter) run(logger logger.Service) error {
if streamRequest.body != nil {
go streamRequest.flushBody()
}
err := w.writeStreamData(streamRequest.stream, logger)
err := w.writeStreamData(streamRequest.stream, log)
if err != nil {
return err
}
@@ -151,7 +151,7 @@ func (w *MuxWriter) run(logger logger.Service) error {
if !ok {
continue
}
err := w.writeStreamData(stream, logger)
err := w.writeStreamData(stream, log)
if err != nil {
return err
}
@@ -159,7 +159,7 @@ func (w *MuxWriter) run(logger logger.Service) error {
case useDict := <-w.useDictChan:
err := w.writeUseDictionary(useDict)
if err != nil {
logger.Errorf("mux - write: error writing use dictionary: %s", err)
log.Error().Msgf("mux - write: error writing use dictionary: %s", err)
return err
}
w.idleTimer.MarkActive()
@@ -167,18 +167,18 @@ func (w *MuxWriter) run(logger logger.Service) error {
}
}
func (w *MuxWriter) writeStreamData(stream *MuxedStream, logger logger.Service) error {
logger.Debugf("mux - write: writable: streamID: %d", stream.streamID)
func (w *MuxWriter) writeStreamData(stream *MuxedStream, log *zerolog.Logger) error {
log.Debug().Msgf("mux - write: writable: streamID: %d", stream.streamID)
chunk := stream.getChunk()
w.metricsUpdater.updateReceiveWindow(stream.getReceiveWindow())
w.metricsUpdater.updateSendWindow(stream.getSendWindow())
if chunk.sendHeadersFrame() {
err := w.writeHeaders(chunk.streamID, chunk.headers)
if err != nil {
logger.Errorf("mux - write: error writing headers: %s: streamID: %d", err, stream.streamID)
log.Error().Msgf("mux - write: error writing headers: %s: streamID: %d", err, stream.streamID)
return err
}
logger.Debugf("mux - write: output headers: streamID: %d", stream.streamID)
log.Debug().Msgf("mux - write: output headers: streamID: %d", stream.streamID)
}
if chunk.sendWindowUpdateFrame() {
@@ -189,22 +189,22 @@ func (w *MuxWriter) writeStreamData(stream *MuxedStream, logger logger.Service)
// window, unless the receiver treats this as a connection error"
err := w.f.WriteWindowUpdate(chunk.streamID, chunk.windowUpdate)
if err != nil {
logger.Errorf("mux - write: error writing window update: %s: streamID: %d", err, stream.streamID)
log.Error().Msgf("mux - write: error writing window update: %s: streamID: %d", err, stream.streamID)
return err
}
logger.Debugf("mux - write: increment receive window by %d streamID: %d", chunk.windowUpdate, stream.streamID)
log.Debug().Msgf("mux - write: increment receive window by %d streamID: %d", chunk.windowUpdate, stream.streamID)
}
for chunk.sendDataFrame() {
payload, sentEOF := chunk.nextDataFrame(int(w.maxFrameSize))
err := w.f.WriteData(chunk.streamID, sentEOF, payload)
if err != nil {
logger.Errorf("mux - write: error writing data: %s: streamID: %d", err, stream.streamID)
log.Error().Msgf("mux - write: error writing data: %s: streamID: %d", err, stream.streamID)
return err
}
// update the amount of data wrote
w.bytesWrote.IncrementBy(uint64(len(payload)))
logger.Debugf("mux - write: output data: %d: streamID: %d", len(payload), stream.streamID)
log.Debug().Msgf("mux - write: output data: %d: streamID: %d", len(payload), stream.streamID)
if sentEOF {
if stream.readBuffer.Closed() {
@@ -212,15 +212,15 @@ func (w *MuxWriter) writeStreamData(stream *MuxedStream, logger logger.Service)
if !stream.gotReceiveEOF() {
// the peer may send data that we no longer want to receive. Force them into the
// closed state.
logger.Debugf("mux - write: resetting stream: streamID: %d", stream.streamID)
log.Debug().Msgf("mux - write: resetting stream: streamID: %d", stream.streamID)
w.f.WriteRSTStream(chunk.streamID, http2.ErrCodeNo)
} else {
// Half-open stream transitioned into closed
logger.Debugf("mux - write: closing stream: streamID: %d", stream.streamID)
log.Debug().Msgf("mux - write: closing stream: streamID: %d", stream.streamID)
}
w.streams.Delete(chunk.streamID)
} else {
logger.Debugf("mux - write: closing stream write side: streamID: %d", stream.streamID)
log.Debug().Msgf("mux - write: closing stream write side: streamID: %d", stream.streamID)
}
}
}
+17 -17
View File
@@ -12,8 +12,8 @@ import (
"os"
"time"
"github.com/cloudflare/cloudflared/logger"
"github.com/gorilla/websocket"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/tlsconfig"
)
@@ -99,8 +99,8 @@ const indexTemplate = `
</html>
`
func StartHelloWorldServer(logger logger.Service, listener net.Listener, shutdownC <-chan struct{}) error {
logger.Infof("Starting Hello World server at %s", listener.Addr())
func StartHelloWorldServer(log *zerolog.Logger, listener net.Listener, shutdownC <-chan struct{}) error {
log.Info().Msgf("Starting Hello World server at %s", listener.Addr())
serverName := defaultServerName
if hostname, err := os.Hostname(); err == nil {
serverName = hostname
@@ -113,14 +113,14 @@ func StartHelloWorldServer(logger logger.Service, listener net.Listener, shutdow
muxer := http.NewServeMux()
muxer.HandleFunc(UptimeRoute, uptimeHandler(time.Now()))
muxer.HandleFunc(WSRoute, websocketHandler(logger, upgrader))
muxer.HandleFunc(SSERoute, sseHandler(logger))
muxer.HandleFunc(WSRoute, websocketHandler(log, upgrader))
muxer.HandleFunc(SSERoute, sseHandler(log))
muxer.HandleFunc(HealthRoute, healthHandler())
muxer.HandleFunc("/", rootHandler(serverName))
httpServer := &http.Server{Addr: listener.Addr().String(), Handler: muxer}
go func() {
<-shutdownC
httpServer.Close()
_ = httpServer.Close()
}()
err := httpServer.Serve(listener)
@@ -152,13 +152,13 @@ func uptimeHandler(startTime time.Time) http.HandlerFunc {
w.WriteHeader(http.StatusInternalServerError)
} else {
w.Header().Set("Content-Type", "application/json")
w.Write(respJson)
_, _ = w.Write(respJson)
}
}
}
// This handler will echo message
func websocketHandler(logger logger.Service, upgrader websocket.Upgrader) http.HandlerFunc {
func websocketHandler(log *zerolog.Logger, upgrader websocket.Upgrader) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// This addresses the issue of r.Host includes port but origin header doesn't
host, _, err := net.SplitHostPort(r.Host)
@@ -168,32 +168,32 @@ func websocketHandler(logger logger.Service, upgrader websocket.Upgrader) http.H
conn, err := upgrader.Upgrade(w, r, nil)
if err != nil {
logger.Errorf("failed to upgrade to websocket connection, error: %s", err)
log.Err(err).Msg("failed to upgrade to websocket connection")
return
}
defer conn.Close()
for {
mt, message, err := conn.ReadMessage()
if err != nil {
logger.Errorf("websocket read message error: %s", err)
log.Err(err).Msg("websocket read message error")
break
}
if err := conn.WriteMessage(mt, message); err != nil {
logger.Errorf("websocket write message error: %s", err)
log.Err(err).Msg("websocket write message error")
break
}
}
}
}
func sseHandler(logger logger.Service) http.HandlerFunc {
func sseHandler(log *zerolog.Logger) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Content-Type", "text/event-stream; charset=utf-8")
flusher, ok := w.(http.Flusher)
if !ok {
w.WriteHeader(http.StatusInternalServerError)
logger.Errorf("Can't support SSE. ResponseWriter %T doesn't implement http.Flusher interface", w)
log.Error().Msgf("Can't support SSE. ResponseWriter %T doesn't implement http.Flusher interface", w)
return
}
@@ -204,7 +204,7 @@ func sseHandler(logger logger.Service) http.HandlerFunc {
freq = parsedFreq
}
}
logger.Infof("Server Sent Events every %s", freq)
log.Info().Msgf("Server Sent Events every %s", freq)
ticker := time.NewTicker(freq)
counter := 0
for {
@@ -247,9 +247,9 @@ func rootHandler(serverName string) http.HandlerFunc {
})
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, "error: %v", err)
_, _ = fmt.Fprintf(w, "error: %v", err)
} else {
buffer.WriteTo(w)
_, _ = buffer.WriteTo(w)
}
}
}
+28 -14
View File
@@ -9,18 +9,18 @@ import (
"strings"
"sync"
"github.com/pkg/errors"
"github.com/urfave/cli/v2"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/logger"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
)
var (
ErrNoIngressRules = errors.New("No ingress rules were specified in the config file")
errLastRuleNotCatchAll = errors.New("The last ingress rule must match all hostnames (i.e. it must be missing, or must be \"*\")")
ErrNoIngressRules = errors.New("The config file doesn't contain any ingress rules")
errLastRuleNotCatchAll = errors.New("The last ingress rule must match all URLs (i.e. it should not have a hostname or path filter)")
errBadWildcard = errors.New("Hostname patterns can have at most one wildcard character (\"*\") and it can only be used for subdomains, e.g. \"*.example.com\"")
errHostnameContainsPort = errors.New("Hostname cannot contain port")
errHostnameContainsPort = errors.New("Hostname cannot contain a port")
ErrURLIncompatibleWithIngress = errors.New("You can't set the --url flag (or $TUNNEL_URL) when using multiple-origin ingress rules")
)
@@ -63,7 +63,7 @@ type Ingress struct {
// NewSingleOrigin constructs an Ingress set with only one rule, constructed from
// legacy CLI parameters like --url or --no-chunked-encoding.
func NewSingleOrigin(c *cli.Context, allowURLFromArgs bool, logger logger.Service) (Ingress, error) {
func NewSingleOrigin(c *cli.Context, allowURLFromArgs bool) (Ingress, error) {
service, err := parseSingleOriginService(c, allowURLFromArgs)
if err != nil {
@@ -89,7 +89,7 @@ func parseSingleOriginService(c *cli.Context, allowURLFromArgs bool) (OriginServ
if c.IsSet("hello-world") {
return new(helloWorld), nil
}
if c.IsSet("url") {
if c.IsSet("url") || c.IsSet(config.BastionFlag) {
originURL, err := config.ValidateUrl(c, allowURLFromArgs)
if err != nil {
return nil, errors.Wrap(err, "Error validating origin URL")
@@ -103,7 +103,8 @@ func parseSingleOriginService(c *cli.Context, allowURLFromArgs bool) (OriginServ
}
return &unixSocketPath{path: path}, nil
}
return nil, errors.New("You must either set ingress rules in your config file, or use --url or use --unix-socket")
u, err := url.Parse("http://localhost:8080")
return &localService{URL: u, RootURL: u}, err
}
// IsEmpty checks if there are any ingress rules.
@@ -112,12 +113,18 @@ func (ing Ingress) IsEmpty() bool {
}
// StartOrigins will start any origin services managed by cloudflared, e.g. proxy servers or Hello World.
func (ing Ingress) StartOrigins(wg *sync.WaitGroup, log logger.Service, shutdownC <-chan struct{}, errC chan error) {
func (ing Ingress) StartOrigins(
wg *sync.WaitGroup,
log *zerolog.Logger,
shutdownC <-chan struct{},
errC chan error,
) error {
for _, rule := range ing.Rules {
if err := rule.Service.start(wg, log, shutdownC, errC, rule.Config); err != nil {
log.Errorf("Error starting local service %s: %s", rule.Service, err)
return errors.Wrapf(err, "Error starting local service %s", rule.Service)
}
}
return nil
}
// CatchAll returns the catch-all rule (i.e. the last rule)
@@ -128,6 +135,7 @@ func (ing Ingress) CatchAll() *Rule {
func validate(ingress []config.UnvalidatedIngressRule, defaults OriginRequestConfig) (Ingress, error) {
rules := make([]Rule, len(ingress))
for i, r := range ingress {
cfg := setConfig(defaults, r.OriginRequest)
var service OriginService
if prefix := "unix:"; strings.HasPrefix(r.Service, prefix) {
@@ -143,6 +151,12 @@ func validate(ingress []config.UnvalidatedIngressRule, defaults OriginRequestCon
service = &srv
} else if r.Service == "hello_world" || r.Service == "hello-world" || r.Service == "helloworld" {
service = new(helloWorld)
} else if r.Service == "bastion" || cfg.BastionMode {
// Bastion mode will always start a Websocket proxy server, which will
// overwrite the localService.URL field when `start` is called. So,
// leave the URL field empty for now.
cfg.BastionMode = true
service = new(localService)
} else {
// Validate URL services
u, err := url.Parse(r.Service)
@@ -151,7 +165,7 @@ func validate(ingress []config.UnvalidatedIngressRule, defaults OriginRequestCon
}
if u.Scheme == "" || u.Hostname() == "" {
return Ingress{}, fmt.Errorf("The service %s must have a scheme and a hostname", r.Service)
return Ingress{}, fmt.Errorf("%s is an invalid address, please make sure it has a scheme and a hostname", r.Service)
}
if u.Path != "" {
@@ -178,7 +192,7 @@ func validate(ingress []config.UnvalidatedIngressRule, defaults OriginRequestCon
Hostname: r.Hostname,
Service: service,
Path: pathRegex,
Config: setConfig(defaults, r.OriginRequest),
Config: cfg,
}
}
return Ingress{Rules: rules, defaults: defaults}, nil
+43 -3
View File
@@ -14,7 +14,6 @@ import (
"gopkg.in/yaml.v2"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/tlsconfig"
)
@@ -35,6 +34,7 @@ func Test_parseIngress(t *testing.T) {
fourOhFour := newStatusCode(404)
defaultConfig := setConfig(originRequestFromYAML(config.OriginRequestConfig{}), config.OriginRequestConfig{})
require.Equal(t, defaultKeepAliveConnections, defaultConfig.KeepAliveConnections)
tr := true
type args struct {
rawYAML string
}
@@ -209,6 +209,47 @@ ingress:
},
},
},
{
name: "URL isn't necessary if using bastion",
args: args{rawYAML: `
ingress:
- hostname: bastion.foo.com
originRequest:
bastionMode: true
- service: http_status:404
`},
want: []Rule{
{
Hostname: "bastion.foo.com",
Service: &localService{},
Config: setConfig(originRequestFromYAML(config.OriginRequestConfig{}), config.OriginRequestConfig{BastionMode: &tr}),
},
{
Service: &fourOhFour,
Config: defaultConfig,
},
},
},
{
name: "Bastion service",
args: args{rawYAML: `
ingress:
- hostname: bastion.foo.com
service: bastion
- service: http_status:404
`},
want: []Rule{
{
Hostname: "bastion.foo.com",
Service: &localService{},
Config: setConfig(originRequestFromYAML(config.OriginRequestConfig{}), config.OriginRequestConfig{BastionMode: &tr}),
},
{
Service: &fourOhFour,
Config: defaultConfig,
},
},
},
{
name: "Hostname contains port",
args: args{rawYAML: `
@@ -287,9 +328,8 @@ func TestSingleOriginSetsConfig(t *testing.T) {
require.NoError(t, err)
allowURLFromArgs := false
logger, err := logger.New()
require.NoError(t, err)
ingress, err := NewSingleOrigin(cliCtx, allowURLFromArgs, logger)
ingress, err := NewSingleOrigin(cliCtx, allowURLFromArgs)
require.NoError(t, err)
assert.Equal(t, time.Second, ingress.Rules[0].Config.ConnectTimeout)
+3 -2
View File
@@ -49,7 +49,7 @@ func originRequestFromSingeRule(c *cli.Context) OriginRequestConfig {
var noTLSVerify bool
var disableChunkedEncoding bool
var bastionMode bool
var proxyAddress string
var proxyAddress = defaultProxyAddress
var proxyPort uint
var proxyType string
if flag := ProxyConnectTimeoutFlag; c.IsSet(flag) {
@@ -92,7 +92,8 @@ func originRequestFromSingeRule(c *cli.Context) OriginRequestConfig {
proxyAddress = c.String(flag)
}
if flag := ProxyPortFlag; c.IsSet(flag) {
proxyPort = c.Uint(flag)
// Note TUN-3758 , we use Int because UInt is not supported with altsrc
proxyPort = uint(c.Int(flag))
}
if c.IsSet(Socks5Flag) {
proxyType = socksProxy
+18
View File
@@ -1,11 +1,13 @@
package ingress
import (
"flag"
"testing"
"time"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/stretchr/testify/require"
"github.com/urfave/cli/v2"
"gopkg.in/yaml.v2"
)
@@ -182,3 +184,19 @@ ingress:
}
require.Equal(t, expected1, actual1)
}
func TestDefaultConfigFromCLI(t *testing.T) {
set := flag.NewFlagSet("contrive", 0)
c := cli.NewContext(nil, set, nil)
expected := OriginRequestConfig{
ConnectTimeout: defaultConnectTimeout,
TLSTimeout: defaultTLSTimeout,
TCPKeepAlive: defaultTCPKeepAlive,
KeepAliveConnections: defaultKeepAliveConnections,
KeepAliveTimeout: defaultKeepAliveTimeout,
ProxyAddress: defaultProxyAddress,
}
actual := originRequestFromSingeRule(c)
require.Equal(t, expected, actual)
}
+43 -25
View File
@@ -13,12 +13,12 @@ import (
"time"
"github.com/cloudflare/cloudflared/hello"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/socks"
"github.com/cloudflare/cloudflared/tlsconfig"
"github.com/cloudflare/cloudflared/websocket"
gws "github.com/gorilla/websocket"
"github.com/pkg/errors"
"github.com/rs/zerolog"
)
// OriginService is something a tunnel can proxy traffic to.
@@ -29,7 +29,7 @@ type OriginService interface {
// Start the origin service if it's managed by cloudflared, e.g. proxy servers or Hello World.
// If it's not managed by cloudflared, this is a no-op because the user is responsible for
// starting the origin service.
start(wg *sync.WaitGroup, log logger.Service, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error
start(wg *sync.WaitGroup, log *zerolog.Logger, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error
}
// unixSocketPath is an OriginService representing a unix socket (which accepts HTTP)
@@ -42,8 +42,8 @@ func (o *unixSocketPath) String() string {
return "unix socket: " + o.path
}
func (o *unixSocketPath) start(wg *sync.WaitGroup, log logger.Service, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error {
transport, err := newHTTPTransport(o, cfg)
func (o *unixSocketPath) start(wg *sync.WaitGroup, log *zerolog.Logger, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error {
transport, err := newHTTPTransport(o, cfg, log)
if err != nil {
return err
}
@@ -84,20 +84,15 @@ func (o *localService) Dial(reqURL *url.URL, headers http.Header) (*gws.Conn, *h
return d.Dial(reqURL.String(), headers)
}
func (o *localService) address() string {
return o.URL.String()
}
func (o *localService) start(wg *sync.WaitGroup, log logger.Service, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error {
transport, err := newHTTPTransport(o, cfg)
func (o *localService) start(wg *sync.WaitGroup, log *zerolog.Logger, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error {
transport, err := newHTTPTransport(o, cfg, log)
if err != nil {
return err
}
o.transport = transport
// Start a proxy if one is needed
staticHost := o.staticHost()
if originRequiresProxy(staticHost, cfg) {
if staticHost := o.staticHost(); originRequiresProxy(staticHost, cfg) {
if err := o.startProxy(staticHost, wg, log, shutdownC, errC, cfg); err != nil {
return err
}
@@ -106,13 +101,13 @@ func (o *localService) start(wg *sync.WaitGroup, log logger.Service, shutdownC <
return nil
}
func (o *localService) startProxy(staticHost string, wg *sync.WaitGroup, log logger.Service, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error {
func (o *localService) startProxy(staticHost string, wg *sync.WaitGroup, log *zerolog.Logger, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error {
// Start a listener for the proxy
proxyAddress := net.JoinHostPort(cfg.ProxyAddress, strconv.Itoa(int(cfg.ProxyPort)))
listener, err := net.Listen("tcp", proxyAddress)
if err != nil {
log.Errorf("Cannot start Websocket Proxy Server: %s", err)
log.Error().Msgf("Cannot start Websocket Proxy Server: %s", err)
return errors.Wrap(err, "Cannot start Websocket Proxy Server")
}
@@ -124,18 +119,18 @@ func (o *localService) startProxy(staticHost string, wg *sync.WaitGroup, log log
// This origin's config specifies what type of proxy to start.
switch cfg.ProxyType {
case socksProxy:
log.Info("SOCKS5 server started")
log.Info().Msg("SOCKS5 server started")
streamHandler = func(wsConn *websocket.Conn, remoteConn net.Conn, _ http.Header) {
dialer := socks.NewConnDialer(remoteConn)
requestHandler := socks.NewRequestHandler(dialer)
socksServer := socks.NewConnectionHandler(requestHandler)
socksServer.Serve(wsConn)
_ = socksServer.Serve(wsConn)
}
case "":
log.Debug("Not starting any websocket proxy")
log.Debug().Msg("Not starting any websocket proxy")
default:
log.Errorf("%s isn't a valid proxy (valid options are {%s})", cfg.ProxyType, socksProxy)
log.Error().Msgf("%s isn't a valid proxy (valid options are {%s})", cfg.ProxyType, socksProxy)
}
errC <- websocket.StartProxyServer(log, listener, staticHost, shutdownC, streamHandler)
@@ -152,7 +147,14 @@ func (o *localService) startProxy(staticHost string, wg *sync.WaitGroup, log log
}
func (o *localService) String() string {
return o.address()
if o.isBastion() {
return "Bastion"
}
return o.URL.String()
}
func (o *localService) isBastion() bool {
return o.URL == nil
}
func (o *localService) RoundTrip(req *http.Request) (*http.Response, error) {
@@ -164,6 +166,10 @@ func (o *localService) RoundTrip(req *http.Request) (*http.Response, error) {
func (o *localService) staticHost() string {
if o.URL == nil {
return ""
}
addPortIfMissing := func(uri *url.URL, port int) string {
if uri.Port() != "" {
return uri.Host
@@ -197,8 +203,14 @@ func (o *helloWorld) String() string {
}
// Start starts a HelloWorld server and stores its address in the Service receiver.
func (o *helloWorld) start(wg *sync.WaitGroup, log logger.Service, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error {
transport, err := newHTTPTransport(o, cfg)
func (o *helloWorld) start(
wg *sync.WaitGroup,
log *zerolog.Logger,
shutdownC <-chan struct{},
errC chan error,
cfg OriginRequestConfig,
) error {
transport, err := newHTTPTransport(o, cfg, log)
if err != nil {
return err
}
@@ -255,7 +267,13 @@ func (o *statusCode) String() string {
return fmt.Sprintf("HTTP %d", o.resp.StatusCode)
}
func (o *statusCode) start(wg *sync.WaitGroup, log logger.Service, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error {
func (o *statusCode) start(
wg *sync.WaitGroup,
log *zerolog.Logger,
shutdownC <-chan struct{},
errC chan error,
cfg OriginRequestConfig,
) error {
return nil
}
@@ -274,8 +292,8 @@ func (nrc *NopReadCloser) Close() error {
return nil
}
func newHTTPTransport(service OriginService, cfg OriginRequestConfig) (*http.Transport, error) {
originCertPool, err := tlsconfig.LoadOriginCA(cfg.CAPool, nil)
func newHTTPTransport(service OriginService, cfg OriginRequestConfig, log *zerolog.Logger) (*http.Transport, error) {
originCertPool, err := tlsconfig.LoadOriginCA(cfg.CAPool, log)
if err != nil {
return nil, errors.Wrap(err, "Error loading cert pool")
}
@@ -332,6 +350,6 @@ func (mos MockOriginService) String() string {
return "MockOriginService"
}
func (mos MockOriginService) start(wg *sync.WaitGroup, log logger.Service, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error {
func (mos MockOriginService) start(wg *sync.WaitGroup, log *zerolog.Logger, shutdownC <-chan struct{}, errC chan error, cfg OriginRequestConfig) error {
return nil
}
+129
View File
@@ -0,0 +1,129 @@
package logger
import (
"path/filepath"
)
var defaultConfig = createDefaultConfig()
// Logging configuration
type Config struct {
ConsoleConfig *ConsoleConfig // If nil, the logger will not log into the console
FileConfig *FileConfig // If nil, the logger will not use an individual log file
RollingConfig *RollingConfig // If nil, the logger will not use a rolling log
MinLevel string // debug | info | error | fatal
}
type ConsoleConfig struct {
noColor bool
}
type FileConfig struct {
Dirname string
Filename string
}
func (fc *FileConfig) Fullpath() string {
return filepath.Join(fc.Dirname, fc.Filename)
}
type RollingConfig struct {
Dirname string
Filename string
maxSize int // megabytes
maxBackups int // files
maxAge int // days
}
func createDefaultConfig() Config {
const minLevel = "info"
const RollingMaxSize = 1 // Mb
const RollingMaxBackups = 5 // files
const RollingMaxAge = 0 // Keep forever
const defaultLogFilename = "cloudflared.log"
return Config{
ConsoleConfig: &ConsoleConfig{
noColor: false,
},
FileConfig: &FileConfig{
Dirname: "",
Filename: defaultLogFilename,
},
RollingConfig: &RollingConfig{
Dirname: "",
Filename: defaultLogFilename,
maxSize: RollingMaxSize,
maxBackups: RollingMaxBackups,
maxAge: RollingMaxAge,
},
MinLevel: minLevel,
}
}
func CreateConfig(
minLevel string,
disableTerminal bool,
rollingLogPath, nonRollingLogFilePath string,
) *Config {
var console *ConsoleConfig
if !disableTerminal {
console = createConsoleConfig()
}
var file *FileConfig
var rolling *RollingConfig
if rollingLogPath != "" {
rolling = createRollingConfig(rollingLogPath)
} else if nonRollingLogFilePath != "" {
file = createFileConfig(nonRollingLogFilePath)
}
if minLevel == "" {
minLevel = defaultConfig.MinLevel
}
return &Config{
ConsoleConfig: console,
FileConfig: file,
RollingConfig: rolling,
MinLevel: minLevel,
}
}
func createConsoleConfig() *ConsoleConfig {
return &ConsoleConfig{
noColor: false,
}
}
func createFileConfig(fullpath string) *FileConfig {
if fullpath == "" {
return defaultConfig.FileConfig
}
dirname, filename := filepath.Split(fullpath)
return &FileConfig{
Dirname: dirname,
Filename: filename,
}
}
func createRollingConfig(directory string) *RollingConfig {
if directory == "" {
directory = defaultConfig.RollingConfig.Dirname
}
return &RollingConfig{
Dirname: directory,
Filename: defaultConfig.RollingConfig.Filename,
maxSize: defaultConfig.RollingConfig.maxSize,
maxBackups: defaultConfig.RollingConfig.maxBackups,
maxAge: defaultConfig.RollingConfig.maxAge,
}
}
+145 -115
View File
@@ -2,152 +2,182 @@ package logger
import (
"fmt"
"io"
"os"
"path"
"path/filepath"
"strings"
"time"
"github.com/alecthomas/units"
"github.com/mattn/go-colorable"
"github.com/rs/zerolog"
fallbacklog "github.com/rs/zerolog/log"
"github.com/urfave/cli/v2"
"gopkg.in/natefinch/lumberjack.v2"
)
// Option is to encaspulate actions that will be called by Parse and run later to build an Options struct
type Option func(*Options) error
const (
EnableTerminalLog = false
DisableTerminalLog = true
// Options is use to set logging configuration data
type Options struct {
logFileDirectory string
maxFileSize units.Base2Bytes
maxFileCount uint
terminalOutputDisabled bool
supportedFileLevels []Level
supportedTerminalLevels []Level
LogLevelFlag = "loglevel"
LogFileFlag = "logfile"
LogDirectoryFlag = "log-directory"
LogTransportLevelFlag = "transport-loglevel"
LogSSHDirectoryFlag = "log-directory"
LogSSHLevelFlag = "log-level"
dirPermMode = 0744 // rwxr--r--
filePermMode = 0644 // rw-r--r--
)
func fallbackLogger(err error) *zerolog.Logger {
failLog := fallbacklog.With().Logger()
fallbacklog.Error().Msgf("Falling back to a default logger due to logger setup failure: %s", err)
return &failLog
}
// DisableTerminal stops terminal output for the logger
func DisableTerminal(disable bool) Option {
return func(c *Options) error {
c.terminalOutputDisabled = disable
return nil
func newZerolog(loggerConfig *Config) *zerolog.Logger {
var writers []io.Writer
if loggerConfig.ConsoleConfig != nil {
writers = append(writers, createConsoleLogger(*loggerConfig.ConsoleConfig))
}
}
// File sets a custom file to log events
func File(path string, size units.Base2Bytes, count uint) Option {
return func(c *Options) error {
c.logFileDirectory = path
c.maxFileSize = size
c.maxFileCount = count
return nil
}
}
// DefaultFile configures the log options will the defaults
func DefaultFile(directoryPath string) Option {
return func(c *Options) error {
size, err := units.ParseBase2Bytes("1MB")
if loggerConfig.FileConfig != nil {
fileLogger, err := createFileLogger(*loggerConfig.FileConfig)
if err != nil {
return err
return fallbackLogger(err)
}
c.logFileDirectory = directoryPath
c.maxFileSize = size
c.maxFileCount = 5
return nil
writers = append(writers, fileLogger)
}
}
// SupportedFileLevels sets the supported logging levels for the log file
func SupportedFileLevels(supported []Level) Option {
return func(c *Options) error {
c.supportedFileLevels = supported
return nil
}
}
// SupportedTerminalevels sets the supported logging levels for the terminal output
func SupportedTerminalevels(supported []Level) Option {
return func(c *Options) error {
c.supportedTerminalLevels = supported
return nil
}
}
// LogLevelString sets the supported logging levels from a command line flag
func LogLevelString(level string) Option {
return func(c *Options) error {
supported, err := ParseLevelString(level)
if loggerConfig.RollingConfig != nil {
rollingLogger, err := createRollingLogger(*loggerConfig.RollingConfig)
if err != nil {
return err
return fallbackLogger(err)
}
c.supportedFileLevels = supported
c.supportedTerminalLevels = supported
return nil
writers = append(writers, rollingLogger)
}
multi := zerolog.MultiLevelWriter(writers...)
level, err := zerolog.ParseLevel(loggerConfig.MinLevel)
if err != nil {
return fallbackLogger(err)
}
log := zerolog.New(multi).With().Timestamp().Logger().Level(level)
return &log
}
func CreateTransportLoggerFromContext(c *cli.Context, disableTerminal bool) *zerolog.Logger {
return createFromContext(c, LogTransportLevelFlag, LogDirectoryFlag, disableTerminal)
}
func CreateLoggerFromContext(c *cli.Context, disableTerminal bool) *zerolog.Logger {
return createFromContext(c, LogLevelFlag, LogDirectoryFlag, disableTerminal)
}
func CreateSSHLoggerFromContext(c *cli.Context, disableTerminal bool) *zerolog.Logger {
return createFromContext(c, LogSSHLevelFlag, LogSSHDirectoryFlag, disableTerminal)
}
func createFromContext(
c *cli.Context,
logLevelFlagName,
logDirectoryFlagName string,
disableTerminal bool,
) *zerolog.Logger {
logLevel := c.String(logLevelFlagName)
logFile := c.String(LogFileFlag)
logDirectory := c.String(logDirectoryFlagName)
loggerConfig := CreateConfig(
logLevel,
disableTerminal,
logDirectory,
logFile,
)
log := newZerolog(loggerConfig)
if incompatibleFlagsSet := logFile != "" && logDirectory != ""; incompatibleFlagsSet {
log.Error().Msgf("Your config includes values for both %s and %s, but they are incompatible. %s takes precedence.", LogFileFlag, logDirectoryFlagName, LogFileFlag)
}
return log
}
func Create(loggerConfig *Config) *zerolog.Logger {
if loggerConfig == nil {
loggerConfig = &Config{
defaultConfig.ConsoleConfig,
nil,
nil,
defaultConfig.MinLevel,
}
}
return newZerolog(loggerConfig)
}
func createConsoleLogger(config ConsoleConfig) io.Writer {
return zerolog.ConsoleWriter{
Out: colorable.NewColorableStderr(),
NoColor: config.noColor,
}
}
// Parse builds the Options struct so the caller knows what actions should be run
func Parse(opts ...Option) (*Options, error) {
options := &Options{}
for _, opt := range opts {
if err := opt(options); err != nil {
func createFileLogger(config FileConfig) (io.Writer, error) {
var logFile io.Writer
fullpath := config.Fullpath()
// Try to open the existing file
logFile, err := os.OpenFile(fullpath, os.O_APPEND|os.O_WRONLY, filePermMode)
if err != nil {
// If the existing file wasn't found, or couldn't be opened, just ignore
// it and recreate a new one.
logFile, err = createLogFile(config)
// If creating a new logfile fails, then we have no choice but to error out.
if err != nil {
return nil, err
}
}
return options, nil
fileLogger := zerolog.New(logFile).With().Logger()
return fileLogger, nil
}
// New setups a new logger based on the options.
// The default behavior is to write to standard out
func New(opts ...Option) (*OutputWriter, error) {
options, err := Parse(opts...)
func createLogFile(config FileConfig) (io.Writer, error) {
if config.Dirname != "" {
err := os.MkdirAll(config.Dirname, dirPermMode)
if err != nil {
return nil, err
}
l := NewOutputWriter(SharedWriteManager)
if options.logFileDirectory != "" {
l.Add(NewFileRollingWriter(SanitizeLogPath(options.logFileDirectory),
"cloudflared",
int64(options.maxFileSize),
options.maxFileCount),
NewDefaultFormatter(time.RFC3339Nano), options.supportedFileLevels...)
}
if !options.terminalOutputDisabled {
terminalFormatter := NewTerminalFormatter(time.RFC3339)
if len(options.supportedTerminalLevels) == 0 {
l.Add(os.Stderr, terminalFormatter, InfoLevel, ErrorLevel, FatalLevel)
} else {
l.Add(os.Stderr, terminalFormatter, options.supportedTerminalLevels...)
if err != nil {
return nil, fmt.Errorf("unable to create directories for new logfile: %s", err)
}
}
return l, nil
mode := os.FileMode(filePermMode)
fullPath := filepath.Join(config.Dirname, config.Filename)
logFile, err := os.OpenFile(fullPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, mode)
if err != nil {
return nil, fmt.Errorf("unable to create a new logfile: %s", err)
}
return logFile, nil
}
// ParseLevelString returns the expected log levels based on the cmd flag
func ParseLevelString(lvl string) ([]Level, error) {
switch strings.ToLower(lvl) {
case "fatal":
return []Level{FatalLevel}, nil
case "error":
return []Level{FatalLevel, ErrorLevel}, nil
case "info", "warn":
return []Level{FatalLevel, ErrorLevel, InfoLevel}, nil
case "debug":
return []Level{FatalLevel, ErrorLevel, InfoLevel, DebugLevel}, nil
func createRollingLogger(config RollingConfig) (io.Writer, error) {
if err := os.MkdirAll(config.Dirname, dirPermMode); err != nil {
return nil, err
}
return []Level{}, fmt.Errorf("not a valid log level: %q", lvl)
}
// SanitizeLogPath checks that the logger log path
func SanitizeLogPath(path string) string {
newPath := strings.TrimSpace(path)
// make sure it has a log file extension and is not a directory
if filepath.Ext(newPath) != ".log" && !(isDirectory(newPath) || strings.HasSuffix(newPath, "/")) {
newPath = newPath + ".log"
}
return newPath
return &lumberjack.Logger{
Filename: path.Join(config.Dirname, config.Filename),
MaxBackups: config.maxBackups,
MaxSize: config.maxSize,
MaxAge: config.maxAge,
}, nil
}
-46
View File
@@ -1,46 +0,0 @@
package logger
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestLogLevelParse(t *testing.T) {
lvls, err := ParseLevelString("fatal")
assert.NoError(t, err)
assert.Equal(t, []Level{FatalLevel}, lvls)
lvls, err = ParseLevelString("error")
assert.NoError(t, err)
assert.Equal(t, []Level{FatalLevel, ErrorLevel}, lvls)
lvls, err = ParseLevelString("info")
assert.NoError(t, err)
assert.Equal(t, []Level{FatalLevel, ErrorLevel, InfoLevel}, lvls)
lvls, err = ParseLevelString("info")
assert.NoError(t, err)
assert.Equal(t, []Level{FatalLevel, ErrorLevel, InfoLevel}, lvls)
lvls, err = ParseLevelString("warn")
assert.NoError(t, err)
assert.Equal(t, []Level{FatalLevel, ErrorLevel, InfoLevel}, lvls)
lvls, err = ParseLevelString("debug")
assert.NoError(t, err)
assert.Equal(t, []Level{FatalLevel, ErrorLevel, InfoLevel, DebugLevel}, lvls)
_, err = ParseLevelString("blah")
assert.Error(t, err)
_, err = ParseLevelString("")
assert.Error(t, err)
}
func TestPathSanitizer(t *testing.T) {
assert.Equal(t, "somebad/path/log.bat.log", SanitizeLogPath("\t somebad/path/log.bat\n\n"))
assert.Equal(t, "proper/path/cloudflared.log", SanitizeLogPath("proper/path/cloudflared.log"))
assert.Equal(t, "proper/path/", SanitizeLogPath("proper/path/"))
assert.Equal(t, "proper/path/cloudflared.log", SanitizeLogPath("\tproper/path/cloudflared\n\n"))
}
-125
View File
@@ -1,125 +0,0 @@
package logger
import (
"fmt"
"os"
"path/filepath"
)
// FileRollingWriter maintains a set of log files numbered in order
// to keep a subset of log data to ensure it doesn't grow pass defined limits
type FileRollingWriter struct {
baseFileName string
directory string
maxFileSize int64
maxFileCount uint
fileHandle *os.File
}
// NewFileRollingWriter creates a new rolling file writer.
// directory is the working directory for the files
// baseFileName is the log file name. This writer appends .log to the name for the file name
// maxFileSize is the size in bytes of how large each file can be. Not a hard limit, general limit based after each write
// maxFileCount is the number of rolled files to keep.
func NewFileRollingWriter(directory, baseFileName string, maxFileSize int64, maxFileCount uint) *FileRollingWriter {
return &FileRollingWriter{
directory: directory,
baseFileName: baseFileName,
maxFileSize: maxFileSize,
maxFileCount: maxFileCount,
}
}
// Write is an implementation of io.writer the rolls the file once it reaches its max size
// It is expected the caller to Write is doing so in a thread safe manner (as WriteManager does).
func (w *FileRollingWriter) Write(p []byte) (n int, err error) {
logFile, isSingleFile := buildPath(w.directory, w.baseFileName)
if w.fileHandle == nil {
h, err := os.OpenFile(logFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0664)
if err != nil {
return 0, err
}
w.fileHandle = h
}
// get size for rolling check
info, err := w.fileHandle.Stat()
if err != nil {
// failed to stat the file. Close the file handle and attempt to open a new handle on the next write
w.Close()
w.fileHandle = nil
return 0, err
}
// write to the file
written, err := w.fileHandle.Write(p)
// check if the file needs to be rolled
if err == nil && info.Size()+int64(written) > w.maxFileSize && !isSingleFile {
// close the file handle than do the renaming. A new one will be opened on the next write
w.Close()
w.rename(logFile, 1)
}
return written, err
}
// Close closes the file handle if it is open
func (w *FileRollingWriter) Close() {
if w.fileHandle != nil {
w.fileHandle.Close()
w.fileHandle = nil
}
}
// rename is how the files are rolled. It works recursively to move the base log file to the rolled ones
// e.g. cloudflared.log -> cloudflared-1.log,
// but if cloudflared-1.log already exists, it is renamed to cloudflared-2.log,
// then the other files move in to their postion
func (w *FileRollingWriter) rename(sourcePath string, index uint) {
destinationPath, isSingleFile := buildPath(w.directory, fmt.Sprintf("%s-%d", w.baseFileName, index))
if isSingleFile {
return //don't need to rename anything, it is a single file
}
// rolled to the max amount of files allowed on disk
if index >= w.maxFileCount {
os.Remove(destinationPath)
}
// if the rolled path already exist, rename it to cloudflared-2.log, then do this one.
// recursive call since the oldest one needs to be renamed, before the newer ones can be moved
if exists(destinationPath) {
w.rename(destinationPath, index+1)
}
os.Rename(sourcePath, destinationPath)
}
// return the path to the log file and if it is a single file or not.
// true means a single file. false means a rolled file
func buildPath(directory, fileName string) (string, bool) {
if !isDirectory(directory) { // not a directory, so try and treat it as a single file for backwards compatibility sake
return directory, true
}
return filepath.Join(directory, fileName+".log"), false
}
func exists(filePath string) bool {
if _, err := os.Stat(filePath); os.IsNotExist(err) {
return false
}
return true
}
func isDirectory(path string) bool {
if path == "" {
return true
}
fileInfo, err := os.Stat(path)
if err != nil {
return false
}
return fileInfo.IsDir()
}
-108
View File
@@ -1,108 +0,0 @@
package logger
import (
"fmt"
"io/ioutil"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
)
func TestFileWrite(t *testing.T) {
fileName := "test_file"
fileLog := fileName + ".log"
testData := []byte(string("hello Dalton, how are you doing?"))
defer func() {
os.Remove(fileLog)
}()
w := NewFileRollingWriter("", fileName, 1000, 2)
defer w.Close()
l, err := w.Write(testData)
assert.NoError(t, err)
assert.Equal(t, l, len(testData), "expected write length and data length to match")
d, err := ioutil.ReadFile(fileLog)
assert.FileExists(t, fileLog, "file doesn't exist at expected path")
assert.Equal(t, d, testData, "expected data in file to match test data")
}
func TestRolling(t *testing.T) {
dirName := "testdir"
err := os.Mkdir(dirName, 0755)
assert.NoError(t, err)
fileName := "test_file"
firstFile := filepath.Join(dirName, fileName+".log")
secondFile := filepath.Join(dirName, fileName+"-1.log")
thirdFile := filepath.Join(dirName, fileName+"-2.log")
defer func() {
os.RemoveAll(dirName)
os.Remove(firstFile)
os.Remove(secondFile)
os.Remove(thirdFile)
}()
w := NewFileRollingWriter(dirName, fileName, 1000, 2)
defer w.Close()
for i := 99; i >= 1; i-- {
testData := []byte(fmt.Sprintf("%d bottles of beer on the wall...", i))
w.Write(testData)
}
assert.FileExists(t, firstFile, "first file doesn't exist as expected")
assert.FileExists(t, secondFile, "second file doesn't exist as expected")
assert.FileExists(t, thirdFile, "third file doesn't exist as expected")
assert.False(t, exists(filepath.Join(dirName, fileName+"-3.log")), "limited to two files and there is more")
}
func TestSingleFile(t *testing.T) {
fileName := "test_file"
testData := []byte(string("hello Dalton, how are you doing?"))
defer func() {
os.Remove(fileName)
}()
w := NewFileRollingWriter(fileName, fileName, 1000, 2)
defer w.Close()
l, err := w.Write(testData)
assert.NoError(t, err)
assert.Equal(t, l, len(testData), "expected write length and data length to match")
d, err := ioutil.ReadFile(fileName)
assert.FileExists(t, fileName, "file doesn't exist at expected path")
assert.Equal(t, d, testData, "expected data in file to match test data")
}
func TestSingleFileInDirectory(t *testing.T) {
dirName := "testdir"
err := os.Mkdir(dirName, 0755)
assert.NoError(t, err)
fileName := "test_file"
fullPath := filepath.Join(dirName, fileName+".log")
testData := []byte(string("hello Dalton, how are you doing?"))
defer func() {
os.Remove(fullPath)
os.RemoveAll(dirName)
}()
w := NewFileRollingWriter(fullPath, fileName, 1000, 2)
defer w.Close()
l, err := w.Write(testData)
assert.NoError(t, err)
assert.Equal(t, l, len(testData), "expected write length and data length to match")
d, err := ioutil.ReadFile(fullPath)
assert.FileExists(t, fullPath, "file doesn't exist at expected path")
assert.Equal(t, d, testData, "expected data in file to match test data")
}
-138
View File
@@ -1,138 +0,0 @@
package logger
import (
"fmt"
"runtime"
"time"
"github.com/acmacalister/skittles"
)
// Level of logging, lower number means more verbose logging, higher more terse
type Level int
const (
// DebugLevel is for messages that are intended for purposes debugging only
DebugLevel Level = iota
// InfoLevel is for standard log messages
InfoLevel
// ErrorLevel is for error message to indicate something has gone wrong
ErrorLevel
// FatalLevel is for error message that log and kill the program with an os.exit(1)
FatalLevel
)
// Formatter is the base interface for formatting logging messages before writing them out
type Formatter interface {
Timestamp(Level, time.Time) string // format the timestamp string
Content(Level, string) string // format content string (color for terminal, etc)
}
// DefaultFormatter writes a simple structure timestamp and the message per log line
type DefaultFormatter struct {
format string
}
// NewDefaultFormatter creates the standard log formatter
// format is the time format to use for timestamp formatting
func NewDefaultFormatter(format string) Formatter {
return &DefaultFormatter{
format: format,
}
}
// Timestamp formats a log line timestamp with a brackets around them
func (f *DefaultFormatter) Timestamp(l Level, d time.Time) string {
if f.format == "" {
return ""
}
return fmt.Sprintf("[%s]: ", d.Format(f.format))
}
// Content just writes the log line straight to the sources
func (f *DefaultFormatter) Content(l Level, c string) string {
return c
}
// TerminalFormatter is setup for colored output
type TerminalFormatter struct {
format string
supportsColor bool
}
// UIFormatter is used for streaming logs to UI
type UIFormatter struct {
format string
supportsColor bool
}
// NewTerminalFormatter creates a Terminal formatter for colored output
// format is the time format to use for timestamp formatting
func NewTerminalFormatter(format string) Formatter {
supportsColor := (runtime.GOOS != "windows")
return &TerminalFormatter{
format: format,
supportsColor: supportsColor,
}
}
func NewUIFormatter(format string) Formatter {
supportsColor := (runtime.GOOS != "windows")
return &UIFormatter{
format: format,
supportsColor: supportsColor,
}
}
// Timestamp uses formatting that is tview-specific for UI
func (f *UIFormatter) Timestamp(l Level, d time.Time) string {
t := ""
dateStr := "[" + d.Format(f.format) + "] "
switch l {
case InfoLevel:
t = "[#00ffff]INFO[white]"
case ErrorLevel:
t = "[red]ERROR[white]"
case DebugLevel:
t = "[yellow]DEBUG[white]"
case FatalLevel:
t = "[red]FATAL[white]"
}
return t + dateStr
}
func (f *UIFormatter) Content(l Level, c string) string {
return c
}
// Timestamp returns the log level with a matching color to the log type
func (f *TerminalFormatter) Timestamp(l Level, d time.Time) string {
t := ""
dateStr := "[" + d.Format(f.format) + "] "
switch l {
case InfoLevel:
t = f.output("INFO", skittles.Cyan)
case ErrorLevel:
t = f.output("ERROR", skittles.Red)
case DebugLevel:
t = f.output("DEBUG", skittles.Yellow)
case FatalLevel:
t = f.output("FATAL", skittles.Red)
}
return t + dateStr
}
// Content just writes the log line straight to the sources
func (f *TerminalFormatter) Content(l Level, c string) string {
return c
}
func (f *TerminalFormatter) output(msg string, colorFunc func(interface{}) string) string {
if f.supportsColor {
return colorFunc(msg)
}
return msg
}
-59
View File
@@ -1,59 +0,0 @@
package logger
import "sync"
// SharedWriteManager is a package level variable to allows multiple loggers to use the same write manager.
// This is useful when multiple loggers will write to the same file to ensure they don't clobber each other.
var SharedWriteManager = NewWriteManager()
type writeData struct {
target LogOutput
data []byte
}
// WriteManager is a logging service that handles managing multiple writing streams
type WriteManager struct {
shutdown chan struct{}
writeChan chan writeData
writers map[string]Service
wg sync.WaitGroup
}
// NewWriteManager creates a write manager that implements OutputManager
func NewWriteManager() OutputManager {
m := &WriteManager{
shutdown: make(chan struct{}),
writeChan: make(chan writeData, 1000),
}
go m.run()
return m
}
// Append adds a message to the writer runloop
func (m *WriteManager) Append(data []byte, target LogOutput) {
m.wg.Add(1)
m.writeChan <- writeData{data: data, target: target}
}
// Shutdown stops the sync manager service
func (m *WriteManager) Shutdown() {
m.wg.Wait()
close(m.shutdown)
close(m.writeChan)
}
// run is the main runloop that schedules log messages
func (m *WriteManager) run() {
for {
select {
case event, ok := <-m.writeChan:
if ok {
event.target.WriteLogLine(event.data)
m.wg.Done()
}
case <-m.shutdown:
return
}
}
}
-24
View File
@@ -1,24 +0,0 @@
package logger
import (
"testing"
"github.com/stretchr/testify/assert"
)
type outputFunc func(b []byte)
func (f outputFunc) WriteLogLine(data []byte) {
f(data)
}
func TestWriteManger(t *testing.T) {
testData := []byte(string("hello Austin, how are you doing?"))
waitChan := make(chan []byte)
m := NewWriteManager()
m.Append(testData, outputFunc(func(b []byte) {
waitChan <- b
}))
resp := <-waitChan
assert.Equal(t, testData, resp)
}
-18
View File
@@ -1,18 +0,0 @@
package logger
// MockWriteManager does nothing and is provided for testing purposes
type MockWriteManager struct {
}
// NewMockWriteManager creates an OutputManager that does nothing for testing purposes
func NewMockWriteManager() OutputManager {
return &MockWriteManager{}
}
// Append is a mock stub
func (m *MockWriteManager) Append(data []byte, target LogOutput) {
}
// Shutdown is a mock stub
func (m *MockWriteManager) Shutdown() {
}
-157
View File
@@ -1,157 +0,0 @@
package logger
import (
"fmt"
"io"
"os"
"time"
)
// provided for testing
var osExit = os.Exit
type LogOutput interface {
WriteLogLine([]byte)
}
// OutputManager is used to sync data of Output
type OutputManager interface {
Append([]byte, LogOutput)
Shutdown()
}
// Service is the logging service that is either a group or single log writer
type Service interface {
Error(message string)
Info(message string)
Debug(message string)
Fatal(message string)
Errorf(format string, args ...interface{})
Infof(format string, args ...interface{})
Debugf(format string, args ...interface{})
Fatalf(format string, args ...interface{})
Add(writer io.Writer, formatter Formatter, levels ...Level)
}
type sourceGroup struct {
writer io.Writer
formatter Formatter
levelsSupported []Level
}
func (s *sourceGroup) WriteLogLine(data []byte) {
_, _ = s.writer.Write(data)
}
func (s *sourceGroup) supportsLevel(l Level) bool {
for _, level := range s.levelsSupported {
if l == level {
return true
}
}
return false
}
// OutputWriter is the standard logging implementation
type OutputWriter struct {
groups []*sourceGroup
syncWriter OutputManager
minLevel Level
}
// NewOutputWriter creates a new logger
func NewOutputWriter(syncWriter OutputManager) *OutputWriter {
return &OutputWriter{
syncWriter: syncWriter,
groups: nil,
minLevel: FatalLevel,
}
}
// Add a writer and formatter to output to
func (s *OutputWriter) Add(writer io.Writer, formatter Formatter, levels ...Level) {
s.groups = append(s.groups, &sourceGroup{writer: writer, formatter: formatter, levelsSupported: levels})
// track most verbose (lowest) level we need to output
for _, level := range levels {
if level < s.minLevel {
s.minLevel = level
}
}
}
// Error writes an error to the logging sources
func (s *OutputWriter) Error(message string) {
if s.minLevel <= ErrorLevel {
s.output(ErrorLevel, message)
}
}
// Info writes an info string to the logging sources
func (s *OutputWriter) Info(message string) {
if s.minLevel <= InfoLevel {
s.output(InfoLevel, message)
}
}
// Debug writes a debug string to the logging sources
func (s *OutputWriter) Debug(message string) {
if s.minLevel <= DebugLevel {
s.output(DebugLevel, message)
}
}
// Fatal writes a error string to the logging sources and runs does an os.exit()
func (s *OutputWriter) Fatal(message string) {
s.output(FatalLevel, message)
s.syncWriter.Shutdown() // waits for the pending logging to finish
osExit(1)
}
// Errorf writes a formatted error to the logging sources
func (s *OutputWriter) Errorf(format string, args ...interface{}) {
if s.minLevel <= ErrorLevel {
s.output(ErrorLevel, fmt.Sprintf(format, args...))
}
}
// Infof writes a formatted info statement to the logging sources
func (s *OutputWriter) Infof(format string, args ...interface{}) {
if s.minLevel <= InfoLevel {
s.output(InfoLevel, fmt.Sprintf(format, args...))
}
}
// Debugf writes a formatted debug statement to the logging sources
func (s *OutputWriter) Debugf(format string, args ...interface{}) {
if s.minLevel <= DebugLevel {
s.output(DebugLevel, fmt.Sprintf(format, args...))
}
}
// Fatalf writes a writes a formatted error statement and runs does an os.exit()
func (s *OutputWriter) Fatalf(format string, args ...interface{}) {
s.output(FatalLevel, fmt.Sprintf(format, args...))
s.syncWriter.Shutdown() // waits for the pending logging to finish
osExit(1)
}
// output does the actual write to the sync manager
func (s *OutputWriter) output(l Level, content string) {
now := time.Now()
for _, group := range s.groups {
if group.supportsLevel(l) {
logLine := fmt.Sprintf("%s%s\n", group.formatter.Timestamp(l, now),
group.formatter.Content(l, content))
s.syncWriter.Append([]byte(logLine), group)
}
}
}
// Write implements io.Writer to support SetOutput of the log package
func (s *OutputWriter) Write(p []byte) (n int, err error) {
s.Info(string(p))
return len(p), nil
}
-106
View File
@@ -1,106 +0,0 @@
package logger
import (
"bufio"
"bytes"
"fmt"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestLogLevel(t *testing.T) {
timeFormat := "2006-01-02"
f := NewDefaultFormatter(timeFormat)
m := NewWriteManager()
var testBuffer bytes.Buffer
logger := NewOutputWriter(m)
logger.Add(&testBuffer, f, InfoLevel, DebugLevel)
testTime := f.Timestamp(InfoLevel, time.Now())
testInfo := "hello Dalton, how are you doing?"
logger.Info(testInfo)
tesErr := "hello Austin, how did it break today?"
logger.Error(tesErr)
testDebug := "hello Bill, who are you?"
logger.Debug(testDebug)
m.Shutdown()
lines := strings.Split(testBuffer.String(), "\n")
assert.Len(t, lines, 3, "only expected two strings in the buffer")
infoLine := lines[0]
debugLine := lines[1]
compareInfo := fmt.Sprintf("%s%s", testTime, testInfo)
assert.Equal(t, compareInfo, infoLine, "expect the strings to match")
compareDebug := fmt.Sprintf("%s%s", testTime, testDebug)
assert.Equal(t, compareDebug, debugLine, "expect the strings to match")
}
func TestOutputWrite(t *testing.T) {
timeFormat := "2006-01-02"
f := NewDefaultFormatter(timeFormat)
m := NewWriteManager()
var testBuffer bytes.Buffer
logger := NewOutputWriter(m)
logger.Add(&testBuffer, f, InfoLevel)
logger.Debugf("debug message not logged here")
testData := "hello Bob Bork, how are you doing?"
logger.Info(testData)
testTime := f.Timestamp(InfoLevel, time.Now())
m.Shutdown()
scanner := bufio.NewScanner(&testBuffer)
scanner.Scan()
line := scanner.Text()
assert.NoError(t, scanner.Err())
compareLine := fmt.Sprintf("%s%s", testTime, testData)
assert.Equal(t, compareLine, line, "expect the strings to match")
}
func TestFatalWrite(t *testing.T) {
timeFormat := "2006-01-02"
f := NewDefaultFormatter(timeFormat)
m := NewWriteManager()
var testBuffer bytes.Buffer
logger := NewOutputWriter(m)
logger.Add(&testBuffer, f, FatalLevel)
oldOsExit := osExit
defer func() { osExit = oldOsExit }()
var got int
myExit := func(code int) {
got = code
}
osExit = myExit
testData := "so long y'all"
logger.Fatal(testData)
testTime := f.Timestamp(FatalLevel, time.Now())
scanner := bufio.NewScanner(&testBuffer)
scanner.Scan()
line := scanner.Text()
assert.NoError(t, scanner.Err())
compareLine := fmt.Sprintf("%s%s", testTime, testData)
assert.Equal(t, compareLine, line, "expect the strings to match")
assert.Equal(t, got, 1, "exit code should be one for a fatal log")
}
+26 -12
View File
@@ -10,11 +10,12 @@ import (
"sync"
"time"
"golang.org/x/net/trace"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/logger"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
"github.com/rs/zerolog"
"golang.org/x/net/trace"
)
const (
@@ -22,43 +23,56 @@ const (
startupTime = time.Millisecond * 500
)
func ServeMetrics(l net.Listener, shutdownC <-chan struct{}, logger logger.Service) (err error) {
func newMetricsHandler(connectionEvents <-chan connection.Event, log *zerolog.Logger) *http.ServeMux {
readyServer := NewReadyServer(connectionEvents, log)
mux := http.NewServeMux()
mux.Handle("/metrics", promhttp.Handler())
mux.HandleFunc("/healthcheck", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprintf(w, "OK\n")
})
mux.Handle("/ready", readyServer)
return mux
}
func ServeMetrics(
l net.Listener,
shutdownC <-chan struct{},
connectionEvents <-chan connection.Event,
log *zerolog.Logger,
) (err error) {
var wg sync.WaitGroup
// Metrics port is privileged, so no need for further access control
trace.AuthRequest = func(*http.Request) (bool, bool) { return true, true }
// TODO: parameterize ReadTimeout and WriteTimeout. The maximum time we can
// profile CPU usage depends on WriteTimeout
h := newMetricsHandler(connectionEvents, log)
server := &http.Server{
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
Handler: h,
}
http.Handle("/metrics", promhttp.Handler())
http.Handle("/healthcheck", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "OK\n")
}))
wg.Add(1)
go func() {
defer wg.Done()
err = server.Serve(l)
}()
logger.Infof("Starting metrics server on %s", fmt.Sprintf("%v/metrics", l.Addr()))
log.Info().Msgf("Starting metrics server on %s", fmt.Sprintf("%v/metrics", l.Addr()))
// server.Serve will hang if server.Shutdown is called before the server is
// fully started up. So add artificial delay.
time.Sleep(startupTime)
<-shutdownC
ctx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
server.Shutdown(ctx)
_ = server.Shutdown(ctx)
cancel()
wg.Wait()
if err == http.ErrServerClosed {
logger.Info("Metrics server stopped")
log.Info().Msg("Metrics server stopped")
return nil
}
logger.Errorf("Metrics server quit with error: %s", err)
log.Err(err).Msg("Metrics server failed")
return err
}
+81
View File
@@ -0,0 +1,81 @@
package metrics
import (
"encoding/json"
"fmt"
"net/http"
"sync"
conn "github.com/cloudflare/cloudflared/connection"
"github.com/rs/zerolog"
)
// ReadyServer serves HTTP 200 if the tunnel can serve traffic. Intended for k8s readiness checks.
type ReadyServer struct {
sync.RWMutex
isConnected map[int]bool
log *zerolog.Logger
}
// NewReadyServer initializes a ReadyServer and starts listening for dis/connection events.
func NewReadyServer(connectionEvents <-chan conn.Event, log *zerolog.Logger) *ReadyServer {
rs := ReadyServer{
isConnected: make(map[int]bool, 0),
log: log,
}
go func() {
for c := range connectionEvents {
switch c.EventType {
case conn.Connected:
rs.Lock()
rs.isConnected[int(c.Index)] = true
rs.Unlock()
case conn.Disconnected, conn.Reconnecting, conn.RegisteringTunnel:
rs.Lock()
rs.isConnected[int(c.Index)] = false
rs.Unlock()
case conn.SetURL:
continue
default:
rs.log.Error().Msgf("Unknown connection event case %v", c)
}
}
}()
return &rs
}
type body struct {
Status int `json:"status"`
ReadyConnections int `json:"readyConnections"`
}
// ServeHTTP responds with HTTP 200 if the tunnel is connected to the edge.
func (rs *ReadyServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
statusCode, readyConnections := rs.makeResponse()
w.WriteHeader(statusCode)
body := body{
Status: statusCode,
ReadyConnections: readyConnections,
}
msg, err := json.Marshal(body)
if err != nil {
_, _ = fmt.Fprintf(w, `{"error": "%s"}`, err)
}
_, _ = w.Write(msg)
}
// This is the bulk of the logic for ServeHTTP, broken into its own pure function
// to make unit testing easy.
func (rs *ReadyServer) makeResponse() (statusCode, readyConnections int) {
statusCode = http.StatusServiceUnavailable
rs.RLock()
defer rs.RUnlock()
for _, connected := range rs.isConnected {
if connected {
statusCode = http.StatusOK
readyConnections++
}
}
return statusCode, readyConnections
}
+58
View File
@@ -0,0 +1,58 @@
package metrics
import (
"net/http"
"testing"
)
func TestReadyServer_makeResponse(t *testing.T) {
type fields struct {
isConnected map[int]bool
}
tests := []struct {
name string
fields fields
wantOK bool
wantReadyConnections int
}{
{
name: "One connection online => HTTP 200",
fields: fields{
isConnected: map[int]bool{
0: false,
1: false,
2: true,
3: false,
},
},
wantOK: true,
wantReadyConnections: 1,
},
{
name: "No connections online => no HTTP 200",
fields: fields{
isConnected: map[int]bool{
0: false,
1: false,
2: false,
3: false,
},
},
wantReadyConnections: 0,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
rs := &ReadyServer{
isConnected: tt.fields.isConnected,
}
gotStatusCode, gotReadyConnections := rs.makeResponse()
if tt.wantOK && gotStatusCode != http.StatusOK {
t.Errorf("ReadyServer.makeResponse() gotStatusCode = %v, want ok = %v", gotStatusCode, tt.wantOK)
}
if gotReadyConnections != tt.wantReadyConnections {
t.Errorf("ReadyServer.makeResponse() gotReadyConnections = %v, want %v", gotReadyConnections, tt.wantReadyConnections)
}
})
}
}

Some files were not shown because too many files have changed in this diff Show More