201 lines
8.2 KiB
Markdown
201 lines
8.2 KiB
Markdown
# Gateway
|
|
|
|
This document acts as the spec for a general purpose gateway system built into
|
|
the isle daemon.
|
|
|
|
The high level goal of the gateway is to remove significant barriers to entry
|
|
for individuals hosting data and services at their own domain names:
|
|
|
|
* Maintaining a public IP or dDNS
|
|
|
|
* Keeping a server always online
|
|
|
|
If a network operator has configured their host as a lighthouse for their
|
|
network they will have already solved these problems. By providing gateway
|
|
functionality they can share that work with their community.
|
|
|
|
The gateway functions by serving requests for a user's DNS domain on their
|
|
behalf. Requests are served by publicly facing hosts of an isle network using
|
|
a backend configured by the user.
|
|
|
|
## Terminology
|
|
|
|
* "gateway process" refers to the the system process running on a single host
|
|
which serves requests for an isle network's gateway.
|
|
|
|
* "gateway" refers to the collection of all gateway processes which are
|
|
configured and running for an isle network. All gateway processes exhibit
|
|
the same behavior, and serve the same domain names, and so are conceived as
|
|
being a single unit.
|
|
|
|
* "gateway DNS name" refers to the single DNS domain or subdomain which contains
|
|
a A/AAAA record for each of the gateway processes of an isle network.
|
|
|
|
## Gateway DNS Setup
|
|
|
|
In order to support the gateway feature, the operators of a network will need to
|
|
agree on a single public DNS name (which could be any arbitrary domain or
|
|
sub-domain) which will be used by the gateway. The operators will need to agree
|
|
on who is able to edit this name, taking into consideration "bus factor"
|
|
concerns.
|
|
|
|
For an operator to enable their host's gateway process they need to:
|
|
|
|
* Enable the option within their `daemon.yml`, and include the appropriate
|
|
public IP or DNS name for their host.
|
|
|
|
* Make ports 80/443 publicly available on their host.
|
|
|
|
* Have that same public IP/DNS name added as an A/AAAA record on the gateway DNS
|
|
name.
|
|
|
|
## Backends
|
|
|
|
Within the context of the gateway, a backend is any method used by the gateway
|
|
to serve requests at a domain. Backends could include:
|
|
|
|
- Public git repositories (like how github pages works)
|
|
- S3 buckets on the private garage cluster
|
|
- IPFS files or IPNS names
|
|
- Services which are available privately over an isle network
|
|
|
|
A backend is uniquely described and identified by an object containing a `type`
|
|
field, and then further fields which depend on the type. The object for a git
|
|
repository backend would look like:
|
|
|
|
```json
|
|
{
|
|
"type":"git",
|
|
"origin":"URL",
|
|
"branch":"main"
|
|
}
|
|
```
|
|
|
|
All backends essentially act as a static filesystem, from which requests are
|
|
served by the gateway. For example, given a domain `example.com` which has been
|
|
configured with an isle network gateway to serve a git repo, if
|
|
`example.com/foo/bar` is requested then the file at `/foo/bar` within the
|
|
current tip of the git repo is served.
|
|
|
|
## User Experience
|
|
|
|
It should be noted up-front that a gateway user should not need to have any
|
|
hosts registered on an isle network, or have any other material connection to
|
|
the network, in order to use the gateway.
|
|
|
|
### Setup a Domain
|
|
|
|
The user navigates to a webpage hosted at the gateway DNS name, and selects the
|
|
option to "setup domain". This brings them to a new webpage.
|
|
|
|
This page presents the user with a dropdown input containing options for each
|
|
supported backend type. When the user selects a backend type, the page updates
|
|
to present further fields depending on the backend type. The presented fields
|
|
are all those required to construct a backend object of the selected type.
|
|
|
|
Once the user has input all necessary information, they click a button to
|
|
continue to the next step.
|
|
|
|
At the next step they are prompted for a secret passphrase that is only known by
|
|
the network operators. The user must get this passphrase from an operator, if
|
|
they haven't already. Once successfully input, they continue to the next step.
|
|
|
|
At the next step they are presented with a unique, opaque string, along with
|
|
instructions. The instructions prompt the user to:
|
|
|
|
* Add a CNAME record to their DNS domain, pointing to the gateway DNS name.
|
|
|
|
* Add a TXT record to their DNS domain containing `isle_gateway=<opaque
|
|
string>`.
|
|
|
|
The page provides the user with an input box to input their domain name into,
|
|
and a "Check" button to click once they have completed these instructions. If
|
|
they have not successfully completed the instructions then clicking "Check" will
|
|
display an appropriate error to them. If they have successfully completed the
|
|
instructions then clicking "Check" will result in a "Success" screen.
|
|
|
|
At this point navigating to their domain should result in their chosen backend
|
|
being served.
|
|
|
|
### Checking a Domain
|
|
|
|
The user navigates to a webpage hosted at the gateway DNS name, and selects the
|
|
option to "check existing domain". This brings them to a new webpage.
|
|
|
|
This page presents the user with a text box to input a domain name into, and a
|
|
"Check" button. When the button is clicked and a domain without a backend
|
|
already set up on it is given then an error is displayed. When the button is
|
|
clicked and a domain with a backend set up is given then the information for
|
|
that backend is displayed in a read-only mode.
|
|
|
|
### Updating a Domain
|
|
|
|
The user navigates to a webpage hosted at the gateway DNS name, and selects the
|
|
option to "update existing domain". This brings them to a new webpage.
|
|
|
|
This webpage works almost the same as the "Checking a Domain" page. The
|
|
difference comes at the final step, when the "Check" button is clicked and a
|
|
domain with a backend set up is given. At this point the existing backend
|
|
information for that domain is displayed in a _read-write_ mode, i.e. in a form
|
|
similar to that used in the "Setup a Domain" step for the same backend. From
|
|
here the user can edit any information they want, and the flow continues exactly
|
|
as if they were in the "Setup a Domain" flow.
|
|
|
|
## Implementation Notes
|
|
|
|
The following are general notes and ideas about how this can be implemented.
|
|
They are not meant to be set in stone.
|
|
|
|
* A backend object can be deterministically hashed to form a unique ID for that
|
|
backend. The "opaque string" which is added to a user's domain's TXT record is
|
|
this ID.
|
|
|
|
* The gateway processes use a shared garage S3 bucket for persistence of
|
|
domain/backend mappings.
|
|
|
|
* Each process periodically re-reads all mapping data from S3 and uses that as
|
|
its currently active set of mappings. A pubsub system like NATS can be added
|
|
later to ensure that re-syncing happens at the same time.
|
|
|
|
* Each file in the bucket corresponds to a domain name which has been
|
|
successfully setup. The value contains the backend which is set up for the
|
|
domain.
|
|
|
|
* When the "Check" button is clicked during the "Setup/Update a Domain" flows,
|
|
the gateway process which handles the call does the following:
|
|
|
|
* Checks that a TXT record with the expected value exists on the domain being
|
|
added.
|
|
|
|
* Checks that a CNAME record with the expected value exists on the domain
|
|
being added.
|
|
|
|
* Performs ACME challenge for SSL cert. It will do this via an
|
|
[HTTP-01][http01] challenge. The challenge token will be put in the shared
|
|
garage S3 bucket, so that all gateway processes can potentially serve it
|
|
when requested.
|
|
|
|
* Depending on the backend, it may be required to sync some data into the
|
|
shared garage S3 bucket. For example, if the git backend is being used, then
|
|
the tip of the repo+branch will be synced into the S3 bucket.
|
|
|
|
* Adds a file to the shared garage S3 bucket corresponding to the new
|
|
domain/backend mapping.
|
|
|
|
* Each gateway process is responsible for keeping its own SSL certificates up to
|
|
date via periodic ACME challenges.
|
|
|
|
* Backend data in the shared garage S3 bucket will need to be periodically kept
|
|
up to date. A mechanism needs to be devised to do this reasonably reliably,
|
|
but without _all_ gateway processes constantly doing it. A communication layer
|
|
like NATS might come in handy here.
|
|
|
|
[http01]: https://letsencrypt.org/docs/challenge-types/#http-01-challenge
|
|
|
|
## Future Work
|
|
|
|
It should be possible for someone accessing a private domain on the isle
|
|
network to access the gateway web panel and use it _without_ requiring a secret
|
|
passphrase. This way community members who have already been vetted (by being
|
|
added to the VPN) can have even less friction.
|