Spokes

Spokes is a fully-featued self-hosted version of our edge-server.

Running Spokes will provide administraters more control for tunnels and exposes configuration options that are helpful if you intend to build or connect applications using Packetriot tunnels. This option also provides users the ability to audit and manage network traffic to tunnels more rigidily and may suit those serving an enterprise.

The Packetriot client works identically with a Spokes server. The only changes are the configuration steps which we will highlight below. Otherwise all of the functions of client are available.

In addition to managing the server, administrators will also be required to manage client registration and authentication tokens. We have a section below to review this. In lieu of the user management perform by Packetriot, administrators will need to manage tokens to perform the same function.

One important missing feature to note are the assigned hostnames. Hostnames and their DNS records are generated and managed by packetriot.com. Since Spokes is an independently running server we can't provide this functionality, however, we have suggestions you can use to implement this feature.

Prior to installing and configuring Spokes you will need the following:

  • A Spokes License
  • A domain name
  • A server or virtual machine that is Internet accessible

Please contact us if this guide does not provide answers to your questions or if any issues arise. Follow us on Twitter @packetriot for announcements on new releases or updates for Spokes.

Need help planning a large deployment? Please reach out to us, we're glad to help!

Installation

Spokes is compiled and packaged to run on CentOS 7 x86-64bit. The latest RPM can be downloaded on our software page. There are no dependencies for Spokes. All that's required is to install the RPM. See the example command below to download Spokes.

Note, this example may not be latest revsion of Spokes.

[user@host] curl -o spokes-0.9.10-1.x86_64.rpm https://pktriot-dl-bucket.sfo2.digitaloceanspaces.com/releases/linux/spokes-0.9.10-1.x86_64.rpm 

[user@host] sudo rpm -Uvh spokes-0.9.10-1.x86_64.rpm

Typically you will want Spokes to run 24/7 and start up on system reboots. It comes with a service unit for Systemd. Use the following commands to enable the service for continuous opeations:

[user@host] sudo systemctl enable spokes

Before we start up Spokes, we'll need to edit the configuration file.

Configuration

After installation you'll need to configure several values in the Spokes configuration file. By default the RPM will install a template - /etc/spokes/config.json.example. Copy this template to the path expected by Spokes.

[user@host] sudo -u spokes cp /etc/spokes/config.json.example /etc/spokes/config.json

Below is the template that is provided after the RPM is installed.

{
    "domain": "example.com",
    "dataPath": "/var/lib/spokes",
    "database": {
        "path": "/var/lib/spokes/spokes.db"
    },
    "token": "REPLACE WITH LICENSE AUTH TOKEN",
    "portRange": {
        "begin": 22000,
        "end": 22999
    },
    "tunOptions": {
        "maxPoolSize": 10,
        "maxPorts": 20
    }
}

The values that are absolutely required to be customized are:

  • The domain name
  • The authentication token

Read the sections below to understand what steps need to be taken to setup a domain name for your Spokes server.

The authentication token is available on the Packetriot website. You'll need to login and visit the License page. Any licenses you have purchased will be listed there.

Licenses

Copy the authentication token. Note, the token is very long, please use the Copy button to place the token in your clipboard.

Setup a Domain

You'll need to assign and setup a hostname for the Spokes server. We'll use the domain example.com for our instructions in this section. We will assume that example.com is being used to serve a website, so we'll create a subdomain called spokes.example.com for our server.

A DNS A record needs to be created for this subdomain with the value set as the IP of the server. We'll use the IP 2.3.4.5. You'll need to perform these steps using the tools available with the domain registrar you used to purchase your domain or another service you use to manage the DNS records.

Setup a Wildcard Domain

Setting up a wildcard domain is highly encouraged. It will allow you to utilize hostnames that can be used by clients for tunneling traffic and will work similarly to the assigned hostnames that the managed Packetriot Tunnel service provides.

In this configuration we will utilize the subdomain we created earlier for our spokes server to create a wildcard domain record. This DNS record will be a CNAME with the name *.spokes with a value spokes.example.com.

This might be presented differently across different DNS management tools. Simply put, we want *.spokes.example.com to resolve to spokes.example.com. Using a CNAME here will also make it easy to change the IP of example.com later and not have to also update this wildcard CNAME record.

The Packetriot client can now request any subdomain from spokes.example.com and it will resolve to spokes.example.com and when received the server, Spokes will correctly route to the tunnel requesting traffic. The example below will illustrate how this works:

[user@host] pktriot tunnel http add --domain tun-01.spokes.example.com --destination 127.0.0.1 --http 8080
[user@host] pktriot tunnel http add --domain tun-02.spokes.example.com --destination 127.0.0.1 --http 8081

In this example, we have a tunnel running on a PC that will request the Spokes server to route all traffic for the two domains to it. Since tun-01.spokes.example.com will resolve to spokes.example.com for all clients on the Internet, the traffic will be sent to the Spokes server. The Spokes server will then route the traffic to this client.

This provides a very similar feature to the assigned hostnames that Packetriot provides. In fact, if you configure a wildcard domain like this it will be even more flexible.

Tunnel Options

The configuration file includes some options that will change the the parameters sent to the Packetriot client when establishing a tunnel to a Spokes server. These parameters are the maximum number of data-plane connections a tunnel can establish and the number of TCP ports that can be reserved.

{
    "tunOptions": {
        "maxPoolSize": 10,
        "maxPorts": 20
    }
}

The defaults in Spokes are more generous then what's used by the managed Packetriot servers. A max "pool size" of 5 is whats used on Packetriot.com and the max ports are based on the users' plan. These values can be changed based on the limits you want to enforce.

If you don't want any reserved ports then set that value to 0.

Increasing the pool size will not necessarily improve performance of serving client connections. The function of the pool is similar to a connection pool setup in an HTTP reverse proxy like nginx or Apache.

Incoming traffic will be relayed across one connection in the tunnel pool but having more will prevent saturation of that connection.

Port Ranges

By default the port ranges used are 22,000-22,999. This is the range used on Packetriot edge-servers. The range begins at 22,000 since most users want to tunnel SSH and that services port is 22. You can change this range to any value you want.

To ensure that the range you choose will be available for listening on, e.g. not selected for other network connections the OS is serving, you will need to update the /etc/sysctl.d/spokes-sysctl.conf. This file includes one lines that reserves the port range for selective use by applications.

net.ipv4.ip_local_reserved_ports = 22000,22999

Be sure to update this line if you make any changes to the port ranges in the Spokes configuration file /etc/spokes/config.json. After making changes and saving them use the command sysctl -p to make the changes active. Note, since these are reserved ports, you may want to reboot the server to cautiously ensure that any port in that range hasn't already been used by some process.

Starting the Service

Once all changes to the configuration file is complete you can start the service using systemctl. Spokes will not function as intended without a domain. Spokes uses Let's Encrypt to generate a TLS certificate for HTTPS support, so it's important that a domain and DNS records have been setup correctly.

To start the service run the following command:

[user@host] sudo systemctl start spokes

Administration

In this section we will discuss the mechanisms for managing clients (Tunnels) for Spokes. Since user management and authentication is not performed by Packetriot, Spokes includes commands in the CLI program to provide this functionality.

There are two tokens that you can manage with Spokes that serve different use-cases.

  • Registration
  • Authentication

Registration Token

A registration token is scoped such it can be used to generate an authentication token by clients. A registration token cannot be used by clients to establish a tunnel session. This token type is useful for fleets of clients where the registration token is used to generate an authentication token that is used for some life-time.

With the Spokes CLI tool you can generate and remove any number of registration tokens. It's important to note that these operations do modify the Spokes database, you will need to execute these commands as the Spokes user.

In the commands below we will first switch to the spokes user on the system and then perform our commands.

[user@host] sudo su spokes
[spokes@host] spokes registration ls

[spokes@host] spokes registration create --description "provide a note to describe use-case for token"

Registration token:
5b72e92d2c5e863e3a21685e....

# or

[user@host] sudo -u spokes spokes registration create --description "provide a note to describe use-case for token"

In this command we used the --description flag to include a short note about how this registration token will be used. It's discussed in more detail in the Client Configuration section how this token is used, but this example below will provide some helpful context of how it's used by clients.

[user@host] pktriot unmanaged configuration --hostname spokes.example.com --registration <token...>

This is most simplest version of using a registration token with the Packetriot client to generate an authentication token such that the client can begin requesting traffic.

Administrators can list registration tokens. This example demonstrates how helpful leaving a description or note about the token makes management and tracking easier.

[user@host] spokes registration ls

+----------------------------------------------------------------------------------------------------------------------------------+
| ID                                 | Created                 | Description                     | Value                | Active   |
+----------------------------------------------------------------------------------------------------------------------------------+
| e93dc13a2b6a4a82862e35e5ee80292c   | Feb 23, 2020 18:27:11   | beta-keys                       | e7c5bfc68fb6e1....   | true     |
+----------------------------------------------------------------------------------------------------------------------------------+
| 64183c82433a4e53b85cfc29338543ac   | Feb 24, 2020 00:22:06   | integration testing             | cc55c793cba732....   | true     |
+----------------------------------------------------------------------------------------------------------------------------------+
| c28959b384b24b168008eb7a91195741   | Mar 1, 2020 16:28:04    | production registration token   | 5b72e92d2c5e86....   | true     |
+----------------------------------------------------------------------------------------------------------------------------------+

When an administrator wants to terminate the use of a token it's simply disabled using the rm sub-command in the registation command. The token is left in the database but it's disabled for future use. You can use the --id flag to remove a specific key.

There is also the --all flag which will remove use for all registration tokens. Use with caution.

[user@host] spokes registration rm --id e93dc13a2b6a4a82862e35e5ee80292c

Authentication Tokens

An authentication token is the key that allows clients to immediately authenticate with a Spokes server and begin requesting traffic to the client. It functions similarly to registration tokens and has an identical command structure.

[user@host] spokes token create 
Client auth-token:
c6e15c70e9a6504dd1fe036d23b...

Since Spokes does not automatically provide a hostname like the managed Packetriot service, it provides a flag --hostname that can be used to specify a hostname for the tunnel associated to the generated token.

If you're using wildcard domains to manage and access your tunnels this flag can useful. However, you can input any value, it's not used by Spokes for anything functional.

[user@host] spokes token create --hostname tun-01.spokes.example.com
Client auth-token:
c6e15c70e9a6504dd1fe036d23b...

Use this token to configure a Packetriot client so that it can establish authenticated sessions with Spokes and begin requesting traffic. Below is a quick example of thow this token is used by the client.

[user@host] pktriot unmanaged configure --hostname spokes.example.com --key <token...>

Managing Tokens

Authentication tokens can be listed as well. One thing to notice is that some tokens have values in the user sections and others do not. We'll quickly explore why here.

When a client uses a registration token to generate an authentication token, they can supply an id field. This id field is basically treated as unique ID. An underlying "user" is created to represent this id and can be looked up to associate more tokens (and tunnels) to it.

Here is a quick example:

[user@host] pktriot --config spokes.json unmanaged configure --id 'jack' --tunnel-hostname "tun-10.spokes.example.com" --hostname unmanaged.borak.co --registration 5b72e92d2c5e863e3a21685ef7c4c...

This allows users to "group" a set of tunnels and identify their users or use-cases. It's not necessary to do this but it can be helpful if you're managing many tunnels some of which are created/used by the same user.

You will see a mixture of tokens and tunnels below. The tokens that have a value in the user column were created with a variety of the pktriot unmanaged configure command above. The other tokens with an empty user value were create using the spokes CLI commanf spokes token create.

+-------------------------------------------------------------------------------------------------------------------------------------+
| ID                                 | Created              | User   | Tunnel                      | Value                 | Active   |
+-------------------------------------------------------------------------------------------------------------------------------------+
| c5f6cd3f76124f7e959959e76de7df16   | Mar 20, 2020 17:28   |        | tun-01.spokes.example.com   | 6b6098da12b0f62d...   | true     |
+-------------------------------------------------------------------------------------------------------------------------------------+
| d6065c44dee446cfacb1d666b482875f   | Mar 20, 2020 17:28   |        | tun-02.spokes.example.com   | 196299cdd15e5e38...   | true     |
+-------------------------------------------------------------------------------------------------------------------------------------+
| c95bebd87cd34e60b8e43e2d1543a995   | Mar 20, 2020 17:29   |        | tun-03.spokes.example.com   | 24b78f0f3f63372d...   | true     |
+-------------------------------------------------------------------------------------------------------------------------------------+
| 1dc94bcb47454b3b954c57deb28fbdc0   | Mar 20, 2020 17:29   |        | testing-non-functional      | 3bce91dd9c3c7d7a...   | true     |
+-------------------------------------------------------------------------------------------------------------------------------------+
| 03e0a1fa2ba24f369470ebb9c167bcda   | Mar 20, 2020 17:31   | jack   | tun-10.spokes.example.com   | 973a97e234218963...   | true     |
+-------------------------------------------------------------------------------------------------------------------------------------+
| 7a0451f45f9c49a9bcf6f556a169778c   | Mar 20, 2020 17:31   | jack   | tun-11.spokes.example.com   | e656e259bfe9802c...   | true     |
+-------------------------------------------------------------------------------------------------------------------------------------+
| 1b7ff3a246554f918fdac4a2894f296f   | Mar 20, 2020 17:31   | jack   | tun-13.spokes.example.com   | 6c33045d3077d638...   | true     |
+-------------------------------------------------------------------------------------------------------------------------------------+

7 tokens

Need to recover the entire token value? You can use the flag --expand and it will print the entire token value along with the basic information.

Authentication tokens can be deleted as well by using the rm subcommand. This will delete tokens generation by the administrator using the CLI (in our examples directly above) or ones created through a registration token.

[user@host] spokes tokens rm --id e239ad1556654d58a540114632b00055

There is also a --all flag that can be used this command and it will delete all tokens. Please use with caution. If you make any mistakes with this command you can attempt to recover the token by use sqlite3 and opening up the Spokes database located in /var/lib/spokes/spokes.db. You will need to update the value of the active column in the apikeys table.

Listing Tunnels

A list of the tunnels and some information on their state, origin (IP) of the clients and their bandwidth consumption, can be present by using the spokes tunnel ls command in the CLI.

The bandwidth values and uptime are not updated in real-time by Spokes. These values are cached so that the accumulated data-points can be stored in a single transaction. However, when connects and disconnect, the online state is updated in real-time.

[user@host] spokes tunnel ls

+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ID                                 | User   | Hostname                    | Online   | Address         | Uptime   | Daily / Monthly   | Created              |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 8cf14c7e48fd4addb5f6daa5af7da969   |        | tun-01.spokes.example.com   | false    |                 | 0 secs   | 0.00 B / 0.00 B   | Mar 20, 2020 17:28   |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 4a2954d4eff04fe9bf881ea546f827ba   |        | tun-02.spokes.example.com   | false    |                 | 0 secs   | 0.00 B / 0.00 B   | Mar 20, 2020 17:28   |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 5d906dcf26ab4e67909ad3ffc282b770   |        | tun-03.spokes.example.com   | false    |                 | 0 secs   | 0.00 B / 0.00 B   | Mar 20, 2020 17:29   |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 399b236fdfa84c58b46acf7db7d00562   |        | testing-non-functional      | false    |                 | 0 secs   | 0.00 B / 0.00 B   | Mar 20, 2020 17:29   |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 810b2453b97c44f8bdb1369126e99489   | jack   | tun-10.spokes.example.com   | false    |                 | 0 secs   | 0.00 B / 0.00 B   | Mar 20, 2020 17:31   |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 6bb481ae9e614686a23d12470ae69d76   | jack   | tun-11.spokes.example.com   | false    |                 | 0 secs   | 0.00 B / 0.00 B   | Mar 20, 2020 17:31   |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 70fddfb497874f1e8e99a4140f3cad0d   | jack   | tun-13.spokes.example.com   | false    | 123.45.67.89    | 0 secs   | 0.00 B / 0.00 B   | Mar 20, 2020 17:31   |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------+

7 tunnels

You can use the --online flag to list only those tunnels that are currently connected to the Spokes server. The --all flag will list all tunnels including those that have been shutdown (deleted), which happens when the associated authentication token was deleted earlier spokes token rm --id <token-id>.

Listing Ports

Ports limited resources, to help manage and understand which tunnel are using allocated ports, the spokes port ls can provide that information.

[user@host] spokes port ls

+------------------------------------------------------------------------------------------------------------------+
| Port No.   | User   | Tunnel ID                          | Tunnel Online   | Tunnel Uptime   | Daily / Monthly   |
+------------------------------------------------------------------------------------------------------------------+
| 22817      | jack   | 70fddfb497874f1e8e99a4140f3cad0d   | false           | 0 secs          | 0.00 B / 0.00 B   |
+------------------------------------------------------------------------------------------------------------------+
| 22708      | jack   | 70fddfb497874f1e8e99a4140f3cad0d   | false           | 0 secs          | 0.00 B / 0.00 B   |
+------------------------------------------------------------------------------------------------------------------+
| 22459      | jack   | 70fddfb497874f1e8e99a4140f3cad0d   | false           | 0 secs          | 0.00 B / 0.00 B   |
+------------------------------------------------------------------------------------------------------------------+
| 22212      | jack   | 70fddfb497874f1e8e99a4140f3cad0d   | false           | 0 secs          | 0.00 B / 0.00 B   |
+------------------------------------------------------------------------------------------------------------------+

4 port allocations

The tunnel ID is included in the output and can be used to filter the tunnel using a combination of ID, spokes tunnel ls and grep

[user@host] spokes tunnel ls | grep 70fddfb497874f1e8e99a4140f3cad0d

| 70fddfb497874f1e8e99a4140f3cad0d   | jack   | tun-13.spokes.example.com   | false    | 123.45.67.89    | 0 secs   | 0.00 B / 0.00 B   | Mar 20, 2020 17:31   |

House Keeping

The Spokes command provide a simple command for performing some house-keeping. In the case of a server crash or some other unforeseen issues, admins can run the command spokes clean that will review the database and fix any issues that don't look right.

The potential issues that may be resolved are collecting orphaned ports or correcting the states of tunnels when their associated auth tokens have been deleted (de-activated).

[user@host] spokes clean

Issues Fixed (tunnels + ports): 0

The clean command can also be run with the --check flag that will allow you to see the issues and not automatically correct them.

House-keeping is performed while the Spokes server is run. It will log issues that are found and then correct them.

Updating

Keeping your Spokes server up-to-date is easy. Follow us on Twitter @packetriot for announcement of new releases for Spokes. Visit our downloads to get the latest versions. Then use the following commands to perform the update on your server.

[user@host] sudo systemctl stop spokes
[user@host] sudo rpm -Uvh spokes-<new version>-1.x86_64.rpm
[user@host] sudo systemctl start spokes

Client Configuration

When using the managed service provided by Packetriot users will configure their client using the command pktriot configure. Authenticating through your credentials or an auth-url to begin the configuration workflow. By default the Packetriot client will prompt you to choose a path from a set of defaults to store the configuration file it will generate.

In addition, users can manually specify a path to a configuration file using the --config flag as well.

Here is an example of the basic client configuration work-flow using a registration token.

[user@host] pktriot unmanaged configure --hostname unmanaged.borak.co --registration 5b72e92d2c5e863e...
Choose a path to the configuration file:
[1] /etc/pktriot/config.json
[2] /data/config.json
[3] /home/user/.pktriot/config.json

The first path [1] is the system-wide Packetriot client configuration path. If you're not running the configure command as the pktriot user, then you'll need to run that the command using sudo: sudo -u pktriot pktriot unmanaged configure....

The second path [2] is the path that used by default in our containerized client. This is done with the intent, and assumption, that you may want to preserve and have access to the data generated by the Packetriot client: certificates (custom or from Let's Encrypt) and the configuration file.

Also, you may be generating or creating static content that you serve through a tunnel and having a mount-point into the container that is host accessible makes this common use-case easier to integrate.

The third path [3] is for clients that are used by a user. If you're developing in your home directory and want to use Packetriot for serving out a project you're developing, this is the option you want to choose.

Note, if clients are being initialized for fleets of devices or users, it would be useful to plan how these clients are started on reboot and if the initial configuration is intended to be automatic. You will want manually specify a path but perhaps choose one of the defaults, e.g. the system-wide path. That would enable your deployment to reuse the Systemd service unit that is included without changes.

Registration Token

We touched a bit already on how the Packetriot client uses the registration token but we will expand on it more in this section. The --d flag was mentioned earlier in the authentication tokens section to explain how "user" are indicated for a particular authentication token,

An authentication token is associated to an underlying user in Spokes. So if you want to "group" one or more clients together then using the --id flag is how you do that. In the case of users, you may want to use emails, system usernames or a full name. If you're managing devices or automated software, you may want to use a hardware address like a mac address, CPU ID, or another unique value that would also help you identify the device.

Use double or single quotes if the value you use choose is using any whitespace or special characters that will be evaluted the shell your using. Example: an awesome tunnel!

[user@host] pktriot unmanaged configuration --id <value> --hostname spokes.example.com --registration <token...>

The --hostname is used to lookup the Spokes server and will be saved to the configuration file. It should be the domain you setup for your Spokes server. In addition, the --id is an optional flag.

Another flag can be used to customize the hostname of the tunnel --tunnel-hostname. This flag can be used to set the hostname when the tunnel is created with the registration token. Here is an example of the command above using this flag.

[user@host] pktriot unmanaged configuration --id "jack" \
    --tunnel-hostname "tun-13.spokes.example.com" \
    --hostname spokes.example.com --registration <token...>

Note, tunnel hostnames are not functional in Spokes. However, if you setup a wildcard domain with your Spokes server, as we did throughout these sections, then it would be functional in practice.

Authentication Token

An authentication token is a key that was generated on the Spokes server and can be used immediately for connecting and creating an authenticated session by a Packetriot client.
It's use is similar to the registration and it will also provide the same default behaviors regarding the prompt to select a configuration path or when you specify a path manually using --config.

Here is an example of the command. You can still use the --id flag to specify a "user" and the --key flag to specify the authentication token generate for this client.

Note that --id is optional.

[user@host] pktriot unmanaged configuration --id <value> --hostname spokes.example.com --key <token...>

De-Authenticating

There may times when you want or need to deauthenticate a client. What this means is that you want the authentication token to be deleted and no longer useful for creating a session.

In cases where you have a fleet of Packetriot clients creating tunnels and there are workflows for "resetting" the system or device, this can be useful and it will help manage the number of clients so that the quota of simulatenous tunnels is not reached because of stale or unneeded authentication tokens.

The command for de-authenticating is simple. Note, if you specified a manual configuration path you will need to specify it with this command.

[user@host] pktriot unmanaged deauth

# or...

[user@host] pktriot --config /path/to/config.json unmanaged deauth

The de-authentication feature is intended to assist use-cases where reconfiguring a client from scratch is necessary and disabling the previous authentication would help achieve greater security.

It's not intended to be used often or widely.