Brian Picciano
8 months ago
parent
5ee80b1b7d
commit
745f7786e8
@ -1,200 +0,0 @@ |
||||
# Gateway |
||||
|
||||
This document acts as the spec for a general purpose gateway system built into |
||||
the isle daemon. |
||||
|
||||
The high level goal of the gateway is to remove significant barriers to entry |
||||
for individuals hosting data and services at their own domain names: |
||||
|
||||
* Maintaining a public IP or dDNS |
||||
|
||||
* Keeping a server always online |
||||
|
||||
If a network operator has configured their host as a lighthouse for their |
||||
network they will have already solved these problems. By providing gateway |
||||
functionality they can share that work with their community. |
||||
|
||||
The gateway functions by serving requests for a user's DNS domain on their |
||||
behalf. Requests are served by publicly facing hosts of an isle network using |
||||
a backend configured by the user. |
||||
|
||||
## Terminology |
||||
|
||||
* "gateway process" refers to the the system process running on a single host |
||||
which serves requests for an isle network's gateway. |
||||
|
||||
* "gateway" refers to the collection of all gateway processes which are |
||||
configured and running for an isle network. All gateway processes exhibit |
||||
the same behavior, and serve the same domain names, and so are conceived as |
||||
being a single unit. |
||||
|
||||
* "gateway DNS name" refers to the single DNS domain or subdomain which contains |
||||
a A/AAAA record for each of the gateway processes of an isle network. |
||||
|
||||
## Gateway DNS Setup |
||||
|
||||
In order to support the gateway feature, the operators of a network will need to |
||||
agree on a single public DNS name (which could be any arbitrary domain or |
||||
sub-domain) which will be used by the gateway. The operators will need to agree |
||||
on who is able to edit this name, taking into consideration "bus factor" |
||||
concerns. |
||||
|
||||
For an operator to enable their host's gateway process they need to: |
||||
|
||||
* Enable the option within their `daemon.yml`, and include the appropriate |
||||
public IP or DNS name for their host. |
||||
|
||||
* Make ports 80/443 publicly available on their host. |
||||
|
||||
* Have that same public IP/DNS name added as an A/AAAA record on the gateway DNS |
||||
name. |
||||
|
||||
## Backends |
||||
|
||||
Within the context of the gateway, a backend is any method used by the gateway |
||||
to serve requests at a domain. Backends could include: |
||||
|
||||
- Public git repositories (like how github pages works) |
||||
- S3 buckets on the private garage cluster |
||||
- IPFS files or IPNS names |
||||
- Services which are available privately over an isle network |
||||
|
||||
A backend is uniquely described and identified by an object containing a `type` |
||||
field, and then further fields which depend on the type. The object for a git |
||||
repository backend would look like: |
||||
|
||||
```json |
||||
{ |
||||
"type":"git", |
||||
"origin":"URL", |
||||
"branch":"main" |
||||
} |
||||
``` |
||||
|
||||
All backends essentially act as a static filesystem, from which requests are |
||||
served by the gateway. For example, given a domain `example.com` which has been |
||||
configured with an isle network gateway to serve a git repo, if |
||||
`example.com/foo/bar` is requested then the file at `/foo/bar` within the |
||||
current tip of the git repo is served. |
||||
|
||||
## User Experience |
||||
|
||||
It should be noted up-front that a gateway user should not need to have any |
||||
hosts registered on an isle network, or have any other material connection to |
||||
the network, in order to use the gateway. |
||||
|
||||
### Setup a Domain |
||||
|
||||
The user navigates to a webpage hosted at the gateway DNS name, and selects the |
||||
option to "setup domain". This brings them to a new webpage. |
||||
|
||||
This page presents the user with a dropdown input containing options for each |
||||
supported backend type. When the user selects a backend type, the page updates |
||||
to present further fields depending on the backend type. The presented fields |
||||
are all those required to construct a backend object of the selected type. |
||||
|
||||
Once the user has input all necessary information, they click a button to |
||||
continue to the next step. |
||||
|
||||
At the next step they are prompted for a secret passphrase that is only known by |
||||
the network operators. The user must get this passphrase from an operator, if |
||||
they haven't already. Once successfully input, they continue to the next step. |
||||
|
||||
At the next step they are presented with a unique, opaque string, along with |
||||
instructions. The instructions prompt the user to: |
||||
|
||||
* Add a CNAME record to their DNS domain, pointing to the gateway DNS name. |
||||
|
||||
* Add a TXT record to their DNS domain containing `isle_gateway=<opaque |
||||
string>`. |
||||
|
||||
The page provides the user with an input box to input their domain name into, |
||||
and a "Check" button to click once they have completed these instructions. If |
||||
they have not successfully completed the instructions then clicking "Check" will |
||||
display an appropriate error to them. If they have successfully completed the |
||||
instructions then clicking "Check" will result in a "Success" screen. |
||||
|
||||
At this point navigating to their domain should result in their chosen backend |
||||
being served. |
||||
|
||||
### Checking a Domain |
||||
|
||||
The user navigates to a webpage hosted at the gateway DNS name, and selects the |
||||
option to "check existing domain". This brings them to a new webpage. |
||||
|
||||
This page presents the user with a text box to input a domain name into, and a |
||||
"Check" button. When the button is clicked and a domain without a backend |
||||
already set up on it is given then an error is displayed. When the button is |
||||
clicked and a domain with a backend set up is given then the information for |
||||
that backend is displayed in a read-only mode. |
||||
|
||||
### Updating a Domain |
||||
|
||||
The user navigates to a webpage hosted at the gateway DNS name, and selects the |
||||
option to "update existing domain". This brings them to a new webpage. |
||||
|
||||
This webpage works almost the same as the "Checking a Domain" page. The |
||||
difference comes at the final step, when the "Check" button is clicked and a |
||||
domain with a backend set up is given. At this point the existing backend |
||||
information for that domain is displayed in a _read-write_ mode, i.e. in a form |
||||
similar to that used in the "Setup a Domain" step for the same backend. From |
||||
here the user can edit any information they want, and the flow continues exactly |
||||
as if they were in the "Setup a Domain" flow. |
||||
|
||||
## Implementation Notes |
||||
|
||||
The following are general notes and ideas about how this can be implemented. |
||||
They are not meant to be set in stone. |
||||
|
||||
* A backend object can be deterministically hashed to form a unique ID for that |
||||
backend. The "opaque string" which is added to a user's domain's TXT record is |
||||
this ID. |
||||
|
||||
* The gateway processes use a shared garage S3 bucket for persistence of |
||||
domain/backend mappings. |
||||
|
||||
* Each process periodically re-reads all mapping data from S3 and uses that as |
||||
its currently active set of mappings. A pubsub system like NATS can be added |
||||
later to ensure that re-syncing happens at the same time. |
||||
|
||||
* Each file in the bucket corresponds to a domain name which has been |
||||
successfully setup. The value contains the backend which is set up for the |
||||
domain. |
||||
|
||||
* When the "Check" button is clicked during the "Setup/Update a Domain" flows, |
||||
the gateway process which handles the call does the following: |
||||
|
||||
* Checks that a TXT record with the expected value exists on the domain being |
||||
added. |
||||
|
||||
* Checks that a CNAME record with the expected value exists on the domain |
||||
being added. |
||||
|
||||
* Performs ACME challenge for SSL cert. It will do this via an |
||||
[HTTP-01][http01] challenge. The challenge token will be put in the shared |
||||
garage S3 bucket, so that all gateway processes can potentially serve it |
||||
when requested. |
||||
|
||||
* Depending on the backend, it may be required to sync some data into the |
||||
shared garage S3 bucket. For example, if the git backend is being used, then |
||||
the tip of the repo+branch will be synced into the S3 bucket. |
||||
|
||||
* Adds a file to the shared garage S3 bucket corresponding to the new |
||||
domain/backend mapping. |
||||
|
||||
* Each gateway process is responsible for keeping its own SSL certificates up to |
||||
date via periodic ACME challenges. |
||||
|
||||
* Backend data in the shared garage S3 bucket will need to be periodically kept |
||||
up to date. A mechanism needs to be devised to do this reasonably reliably, |
||||
but without _all_ gateway processes constantly doing it. A communication layer |
||||
like NATS might come in handy here. |
||||
|
||||
[http01]: https://letsencrypt.org/docs/challenge-types/#http-01-challenge |
||||
|
||||
## Future Work |
||||
|
||||
It should be possible for someone accessing a private domain on the isle |
||||
network to access the gateway web panel and use it _without_ requiring a secret |
||||
passphrase. This way community members who have already been vetted (by being |
||||
added to the VPN) can have even less friction. |
||||
Loading…
Reference in new issue