parent
5ee80b1b7d
commit
745f7786e8
@ -1,200 +0,0 @@ |
|||||||
# Gateway |
|
||||||
|
|
||||||
This document acts as the spec for a general purpose gateway system built into |
|
||||||
the isle daemon. |
|
||||||
|
|
||||||
The high level goal of the gateway is to remove significant barriers to entry |
|
||||||
for individuals hosting data and services at their own domain names: |
|
||||||
|
|
||||||
* Maintaining a public IP or dDNS |
|
||||||
|
|
||||||
* Keeping a server always online |
|
||||||
|
|
||||||
If a network operator has configured their host as a lighthouse for their |
|
||||||
network they will have already solved these problems. By providing gateway |
|
||||||
functionality they can share that work with their community. |
|
||||||
|
|
||||||
The gateway functions by serving requests for a user's DNS domain on their |
|
||||||
behalf. Requests are served by publicly facing hosts of an isle network using |
|
||||||
a backend configured by the user. |
|
||||||
|
|
||||||
## Terminology |
|
||||||
|
|
||||||
* "gateway process" refers to the the system process running on a single host |
|
||||||
which serves requests for an isle network's gateway. |
|
||||||
|
|
||||||
* "gateway" refers to the collection of all gateway processes which are |
|
||||||
configured and running for an isle network. All gateway processes exhibit |
|
||||||
the same behavior, and serve the same domain names, and so are conceived as |
|
||||||
being a single unit. |
|
||||||
|
|
||||||
* "gateway DNS name" refers to the single DNS domain or subdomain which contains |
|
||||||
a A/AAAA record for each of the gateway processes of an isle network. |
|
||||||
|
|
||||||
## Gateway DNS Setup |
|
||||||
|
|
||||||
In order to support the gateway feature, the operators of a network will need to |
|
||||||
agree on a single public DNS name (which could be any arbitrary domain or |
|
||||||
sub-domain) which will be used by the gateway. The operators will need to agree |
|
||||||
on who is able to edit this name, taking into consideration "bus factor" |
|
||||||
concerns. |
|
||||||
|
|
||||||
For an operator to enable their host's gateway process they need to: |
|
||||||
|
|
||||||
* Enable the option within their `daemon.yml`, and include the appropriate |
|
||||||
public IP or DNS name for their host. |
|
||||||
|
|
||||||
* Make ports 80/443 publicly available on their host. |
|
||||||
|
|
||||||
* Have that same public IP/DNS name added as an A/AAAA record on the gateway DNS |
|
||||||
name. |
|
||||||
|
|
||||||
## Backends |
|
||||||
|
|
||||||
Within the context of the gateway, a backend is any method used by the gateway |
|
||||||
to serve requests at a domain. Backends could include: |
|
||||||
|
|
||||||
- Public git repositories (like how github pages works) |
|
||||||
- S3 buckets on the private garage cluster |
|
||||||
- IPFS files or IPNS names |
|
||||||
- Services which are available privately over an isle network |
|
||||||
|
|
||||||
A backend is uniquely described and identified by an object containing a `type` |
|
||||||
field, and then further fields which depend on the type. The object for a git |
|
||||||
repository backend would look like: |
|
||||||
|
|
||||||
```json |
|
||||||
{ |
|
||||||
"type":"git", |
|
||||||
"origin":"URL", |
|
||||||
"branch":"main" |
|
||||||
} |
|
||||||
``` |
|
||||||
|
|
||||||
All backends essentially act as a static filesystem, from which requests are |
|
||||||
served by the gateway. For example, given a domain `example.com` which has been |
|
||||||
configured with an isle network gateway to serve a git repo, if |
|
||||||
`example.com/foo/bar` is requested then the file at `/foo/bar` within the |
|
||||||
current tip of the git repo is served. |
|
||||||
|
|
||||||
## User Experience |
|
||||||
|
|
||||||
It should be noted up-front that a gateway user should not need to have any |
|
||||||
hosts registered on an isle network, or have any other material connection to |
|
||||||
the network, in order to use the gateway. |
|
||||||
|
|
||||||
### Setup a Domain |
|
||||||
|
|
||||||
The user navigates to a webpage hosted at the gateway DNS name, and selects the |
|
||||||
option to "setup domain". This brings them to a new webpage. |
|
||||||
|
|
||||||
This page presents the user with a dropdown input containing options for each |
|
||||||
supported backend type. When the user selects a backend type, the page updates |
|
||||||
to present further fields depending on the backend type. The presented fields |
|
||||||
are all those required to construct a backend object of the selected type. |
|
||||||
|
|
||||||
Once the user has input all necessary information, they click a button to |
|
||||||
continue to the next step. |
|
||||||
|
|
||||||
At the next step they are prompted for a secret passphrase that is only known by |
|
||||||
the network operators. The user must get this passphrase from an operator, if |
|
||||||
they haven't already. Once successfully input, they continue to the next step. |
|
||||||
|
|
||||||
At the next step they are presented with a unique, opaque string, along with |
|
||||||
instructions. The instructions prompt the user to: |
|
||||||
|
|
||||||
* Add a CNAME record to their DNS domain, pointing to the gateway DNS name. |
|
||||||
|
|
||||||
* Add a TXT record to their DNS domain containing `isle_gateway=<opaque |
|
||||||
string>`. |
|
||||||
|
|
||||||
The page provides the user with an input box to input their domain name into, |
|
||||||
and a "Check" button to click once they have completed these instructions. If |
|
||||||
they have not successfully completed the instructions then clicking "Check" will |
|
||||||
display an appropriate error to them. If they have successfully completed the |
|
||||||
instructions then clicking "Check" will result in a "Success" screen. |
|
||||||
|
|
||||||
At this point navigating to their domain should result in their chosen backend |
|
||||||
being served. |
|
||||||
|
|
||||||
### Checking a Domain |
|
||||||
|
|
||||||
The user navigates to a webpage hosted at the gateway DNS name, and selects the |
|
||||||
option to "check existing domain". This brings them to a new webpage. |
|
||||||
|
|
||||||
This page presents the user with a text box to input a domain name into, and a |
|
||||||
"Check" button. When the button is clicked and a domain without a backend |
|
||||||
already set up on it is given then an error is displayed. When the button is |
|
||||||
clicked and a domain with a backend set up is given then the information for |
|
||||||
that backend is displayed in a read-only mode. |
|
||||||
|
|
||||||
### Updating a Domain |
|
||||||
|
|
||||||
The user navigates to a webpage hosted at the gateway DNS name, and selects the |
|
||||||
option to "update existing domain". This brings them to a new webpage. |
|
||||||
|
|
||||||
This webpage works almost the same as the "Checking a Domain" page. The |
|
||||||
difference comes at the final step, when the "Check" button is clicked and a |
|
||||||
domain with a backend set up is given. At this point the existing backend |
|
||||||
information for that domain is displayed in a _read-write_ mode, i.e. in a form |
|
||||||
similar to that used in the "Setup a Domain" step for the same backend. From |
|
||||||
here the user can edit any information they want, and the flow continues exactly |
|
||||||
as if they were in the "Setup a Domain" flow. |
|
||||||
|
|
||||||
## Implementation Notes |
|
||||||
|
|
||||||
The following are general notes and ideas about how this can be implemented. |
|
||||||
They are not meant to be set in stone. |
|
||||||
|
|
||||||
* A backend object can be deterministically hashed to form a unique ID for that |
|
||||||
backend. The "opaque string" which is added to a user's domain's TXT record is |
|
||||||
this ID. |
|
||||||
|
|
||||||
* The gateway processes use a shared garage S3 bucket for persistence of |
|
||||||
domain/backend mappings. |
|
||||||
|
|
||||||
* Each process periodically re-reads all mapping data from S3 and uses that as |
|
||||||
its currently active set of mappings. A pubsub system like NATS can be added |
|
||||||
later to ensure that re-syncing happens at the same time. |
|
||||||
|
|
||||||
* Each file in the bucket corresponds to a domain name which has been |
|
||||||
successfully setup. The value contains the backend which is set up for the |
|
||||||
domain. |
|
||||||
|
|
||||||
* When the "Check" button is clicked during the "Setup/Update a Domain" flows, |
|
||||||
the gateway process which handles the call does the following: |
|
||||||
|
|
||||||
* Checks that a TXT record with the expected value exists on the domain being |
|
||||||
added. |
|
||||||
|
|
||||||
* Checks that a CNAME record with the expected value exists on the domain |
|
||||||
being added. |
|
||||||
|
|
||||||
* Performs ACME challenge for SSL cert. It will do this via an |
|
||||||
[HTTP-01][http01] challenge. The challenge token will be put in the shared |
|
||||||
garage S3 bucket, so that all gateway processes can potentially serve it |
|
||||||
when requested. |
|
||||||
|
|
||||||
* Depending on the backend, it may be required to sync some data into the |
|
||||||
shared garage S3 bucket. For example, if the git backend is being used, then |
|
||||||
the tip of the repo+branch will be synced into the S3 bucket. |
|
||||||
|
|
||||||
* Adds a file to the shared garage S3 bucket corresponding to the new |
|
||||||
domain/backend mapping. |
|
||||||
|
|
||||||
* Each gateway process is responsible for keeping its own SSL certificates up to |
|
||||||
date via periodic ACME challenges. |
|
||||||
|
|
||||||
* Backend data in the shared garage S3 bucket will need to be periodically kept |
|
||||||
up to date. A mechanism needs to be devised to do this reasonably reliably, |
|
||||||
but without _all_ gateway processes constantly doing it. A communication layer |
|
||||||
like NATS might come in handy here. |
|
||||||
|
|
||||||
[http01]: https://letsencrypt.org/docs/challenge-types/#http-01-challenge |
|
||||||
|
|
||||||
## Future Work |
|
||||||
|
|
||||||
It should be possible for someone accessing a private domain on the isle |
|
||||||
network to access the gateway web panel and use it _without_ requiring a secret |
|
||||||
passphrase. This way community members who have already been vetted (by being |
|
||||||
added to the VPN) can have even less friction. |
|
Loading…
Reference in new issue