Proxying with Hashicorp Boundary

For a while, I’ve been trying to get away from my previous VPN - OpenVPN. It was clunky and, somewhat, unreliable. I’d start it up, give it a username, password, OTP and it would connect after about 30 seconds.

Then I’d leave my laptop for 2 seconds, the screen would go blank and as soon as I shake the mouse, OpenVPN would have disconnected, removed my routes and be asking for re-authentication.

Not only is supplying a OTP frequently painful, all of my sessions (SSH etc.) are lost every time.

There were some other downsides as well - it didn’t use the central IDP (as this is not publicly accessible), but the VPN itself was not accessible over the public internet.

New age

I spent some time finding, trialing and implementing a new VPN solution and it has all of the bells and whistles. However, more critically, it’s “always-on”. That is, if my laptop is running and connected to the internet, the VPN is connected.

This is both wonderfully useful, but disasterously dangerous if how the VPN is treated is not re-evaluated.

Can you trust always-on-VPNs?

So to conquor this, I changed my methology, the VPN was no longer treated as sacred as the previous - it would just be a stepping stone to accessing services.

The VPN would provide access to some crucial services, preparing the user to obtain deeper access - DNS, IDP etc.

So the hunt for a candidate to provide access to resources started - there were two main candidates:

• Teleport
• Hashicorp Boundary

Feature-wise they were mostly similar, except that Teleport does not support SSO (which is an awful deal-breaker - and, no, I don’t count only supporting Github as “SSO support”). However, I’ll note that I commend the work performed by aderumier for adding and maintaining OIDC in a fork.

The Biggest problem

On the hole, Boundary works great, but I faced two issues, one of which I’ll tackle in this post:

• Lack of proxying
• Lack of connection to groups of targets

To understand the first problem, in case you’re not aware of how Boundary works, I’ll quickly walk through it.

Connecting to a resource (target):

1. User selects target in Boundary
2. Boundary connects to worker
3. Boundary starts a local listener (listening on localohost and, by default, random port)
4. Requests to the local listener are forwarded to the target, via the worker.

To be able to connect to the service - for example for HTTP applications - the user must access https://localhost:.

The biggest problem here is that this can affect any and/or all of:

• SSL Certificate validation
• Cookies
• CORS
• Any application that perform redirects of non-relative URLs
• IDP authentication
• URLs, such as those from emails, bookmarks etc. won’t work

The worst non-starter here is IDP authentication, since OIDC is configured with known redirect URLs. This means that not only does the IDP need to know the URL of the URL of the application that will be redirect to after authentication, but nearly always the application itself will be configured with it’s own domain/URL.

As a result, accessing an application via this localhost forwarder won’t work for many many cases.

Workaround

I needed to create a workaround that was not only secure, it had to be frictionless - this whole journey was because I was annoyed at the disconnects and re-authentication with OpenVPN.

So this solution must just work.

Design

So I created a simple proxy (yes, a proxy for a proxy 😉). From this point on, I’ll refer to the HTTP proxy as “proxy” and Boundary proxy as “listener”.

The traffic flow looks like this:

Browser –HTTP Proxy–> local proxy —> local Boundary listener –> target

Note that when I say “HTTP proxy”, I’m not meaning the reverse-proxy type of proxy, I’m refering to the good ol' Squid type - so utilising the Proxy configuration inside the browser.

Given I didn’t want to deal with PAC files and such to handle determining what would/wouldn’t be sent through the proxy, the proxy just handles all traffic and modify what it cares about.

Hooking in

So how does it know what/where to send traffic? The proxy is injected into the flow when connecting to targets in Boundary.

The basic flow for connecting to a target using Boundary Desktop is:

• User selects target inside Boundary desktop
• Boundary desktop forks to a bundled Boundary CLI
• Boundary CLI connects to server, starts local listener

The proxy maintains state of active targets: the domain/port of the target and the port of the listener.

To get the two to communicate, the proxy has an API, which allows for the registration/deregistration of targets.

The Boundary CLI code was modified, adding two basic functions:

func sendTransparentProxyRequest(method string, data map[string]string) error {
	jsonData, err := json.Marshal(data)
	if err != nil {
		return err
	}
	request, err := http.NewRequest(method, "http://localhost:9999/redirects", bytes.NewBuffer(jsonData))
	if err != nil {
		return err
	}
	request.Header.Set("Content-Type", "application/json; charset=UTF-8")

	client := &http.Client{}
	_, err = client.Do(request)
	return err
}

func setupTransparentProxyRedirect(destinationHost string, clientProxyHost string, clientProxyPort string) error {
	data := map[string]string{
		"Host":   destinationHost,
		"Target": fmt.Sprintf("%s:%s", clientProxyHost, clientProxyPort),
	}

	return sendTransparentProxyRequest("POST", data)
}

func teardownTransparentProxyRedirect(destinationHost string) error {
	data := map[string]string{
		"Host": destinationHost,
	}

	return sendTransparentProxyRequest("DELETE", data)
}

and then, fairly basically, at the point in the CLI connect method where the port of the local listener has been generated, the host:port of the target and the generated port for the local listener are used to call the above method, which calls out the proxy API:

		err := setupTransparentProxyRedirect(destinationHost, clientProxyHost, clientProxyPort)
		if err != nil {
			c.PrintCliError(fmt.Errorf("error registering with proxy: %w", err))
			return base.CommandCliError
		}

When the proxy receives a request, it looks up the host/port of the connection, if it matches one of the registered targets, it modifies the request to send it to the listener. If not, the request is carried out like normal.

Then, during the connection closing functionality of boundary, the second method is called to remove the redirect.

How this fits in

Now, the proxy can be left running (with or without the Boundary desktop application running), just proxying requests, meaning that when it’s needed, it’s always there - adding little-to-no friction to my workflow.

The patched Boundary CLI can be injected into the Boundary Desktop application (on OS X, it’s simple a file inside the App bundle). I can utlisie Golang’s cross-complilation ability in the pipeline, to allow a custom build of the CLI, inject it into the Boundary App. Fortunately, OS X doesn’t seem to care at all about this and the application runs absolutely fine. (Maybe I’ll spend some time learning how OS X application bundling and signing works one day)

Implementation and gotchas

For the proxy, I used the goproxy package by elazarl - it’s incredibly simple - providing a fully working proxy out of the box with optional methods to hook onto parts of the request lifecycle. It was interesting to note that I had to hook into the method HandleConnect, which is at the lowest level of a connection. This is because the proxy does NOT MITM requests - SSL connections are left completely intact, meaning the SSL handshake is still performed with the target.

The function itself was incredibly simple:

    // Create handler for new connection
	proxy.OnRequest().HandleConnectFunc(func(host string, ctx *goproxy.ProxyCtx) (*goproxy.ConnectAction, string) {
        // Lock the redirectMap, lookup if the host:port exists
        // and unlock again
		mapLock.RLock()
		localTarget, exists := redirectMap[host]
		mapLock.RUnlock()

        // If the host exists, return the Boundary listener ip:port
		if exists {
			return goproxy.OkConnect, localTarget
		}
        // Otherwise, return the original host:ip
		return goproxy.OkConnect, host
	})

The API was also very simple, with just an endpoint with POST/DELETE, which adds/removes a given target.

Summary

This little proxy has transformed Hashicorp Boundary from being uterly useless to a fantastically transparent workflow and took little more than an hour or so of design and coding.