micropelago/isle
4
1
Fork 0
Code Issues 1 Pull Requests Projects Releases 5 Activity

Gateway doc

Browse Source
This commit is contained in:
Brian Picciano 2023-04-24 21:31:59 +02:00
parent a1b3ff71b3
commit 257b961459
3 changed files with 203 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -101,6 +101,8 @@ Documentation for devs:
created by `cryptic-net daemon` at runtime.
* [Rebuilding Documentation](docs/dev/rebuilding-documentation.md)
* [Releases](docs/dev/releases.md)
* [Gateway](docs/dev/gateway.md): Tentative spec for gateway functionality in
cryptic-net
## Misc

200
docs/dev/gateway.md Normal file
View File

@ -0,0 +1,200 @@
# Gateway
This document acts as the spec for a general purpose gateway system built into
the cryptic-net daemon.
The high level goal of the gateway is to remove significant barriers to entry
for individuals hosting data and services at their own domain names:
* Maintaining a public IP or dDNS
* Keeping a server always online
If a network operator has configured their host as a lighthouse for their
network they will have already solved these problems. By providing gateway
functionality they can share that work with their community.
The gateway functions by serving requests for a user's DNS domain on their
behalf. Requests are served by publicly facing hosts of a cryptic network using
a backend configured by the user.
## Terminology
* "gateway process" refers to the the system process running on a single host
which serves requests for a cryptic network's gateway.
* "gateway" refers to the collection of all gateway processes which are
configured and running for a cryptic network. All gateway processes exhibit
the same behavior, and serve the same domain names, and so are conceived as
being a single unit.
* "gateway DNS name" refers to the single DNS domain or subdomain which contains
a A/AAAA record for each of the gateway processes of a cryptic network.
## Gateway DNS Setup
In order to support the gateway feature, the operators of a network will need to
agree on a single public DNS name (which could be any arbitrary domain or
sub-domain) which will be used by the gateway. The operators will need to agree
on who is able to edit this name, taking into consideration "bus factor"
concerns.
For an operator to enable their host's gateway process they need to:
* Enable the option within their `daemon.yml`, and include the appropriate
public IP or DNS name for their host.
* Make ports 80/443 publicly available on their host.
* Have that same public IP/DNS name added as an A/AAAA record on the gateway DNS
name.
## Backends
Within the context of the gateway, a backend is any method used by the gateway
to serve requests at a domain. Backends could include:
- Public git repositories (like how github pages works)
- S3 buckets on the private garage cluster
- IPFS files or IPNS names
- Services which are available privately over cryptic network
A backend is uniquely described and identified by an object containing a `type`
field, and then further fields which depend on the type. The object for a git
repository backend would look like:
```json
{
"type":"git",
"origin":"URL",
"branch":"main"
}
```
All backends essentially act as a static filesystem, from which requests are
served by the gateway. For example, given a domain `example.com` which has been
configured with a cryptic network gateway to serve a git repo, if
`example.com/foo/bar` is requested then the file at `/foo/bar` within the
current tip of the git repo is served.
## User Experience
It should be noted up-front that a gateway user should not need to have any
hosts registered on a cryptic network, or have any other material connection to
the network, in order to use the gateway.
### Setup a Domain
The user navigates to a webpage hosted at the gateway DNS name, and selects the
option to "setup domain". This brings them to a new webpage.
This page presents the user with a dropdown input containing options for each
supported backend type. When the user selects a backend type, the page updates
to present further fields depending on the backend type. The presented fields
are all those required to construct a backend object of the selected type.
Once the user has input all necessary information, they click a button to
continue to the next step.
At the next step they are prompted for a secret passphrase that is only known by
the network operators. The user must get this passphrase from an operator, if
they haven't already. Once successfully input, they continue to the next step.
At the next step they are presented with a unique, opaque string, along with
instructions. The instructions prompt the user to:
* Add a CNAME record to their DNS domain, pointing to the gateway DNS name.
* Add a TXT record to their DNS domain containing `cryptic_net_gateway=<opaque
string>`.
The page provides the user with an input box to input their domain name into,
and a "Check" button to click once they have completed these instructions. If
they have not successfully completed the instructions then clicking "Check" will
display an appropriate error to them. If they have successfully completed the
instructions then clicking "Check" will result in a "Success" screen.
At this point navigating to their domain should result in their chosen backend
being served.
### Checking a Domain
The user navigates to a webpage hosted at the gateway DNS name, and selects the
option to "check existing domain". This brings them to a new webpage.
This page presents the user with a text box to input a domain name into, and a
"Check" button. When the button is clicked and a domain without a backend
already set up on it is given then an error is displayed. When the button is
clicked and a domain with a backend set up is given then the information for
that backend is displayed in a read-only mode.
### Updating a Domain
The user navigates to a webpage hosted at the gateway DNS name, and selects the
option to "update existing domain". This brings them to a new webpage.
This webpage works almost the same as the "Checking a Domain" page. The
difference comes at the final step, when the "Check" button is clicked and a
domain with a backend set up is given. At this point the existing backend
information for that domain is displayed in a _read-write_ mode, i.e. in a form
similar to that used in the "Setup a Domain" step for the same backend. From
here the user can edit any information they want, and the flow continues exactly
as if they were in the "Setup a Domain" flow.
## Implementation Notes
The following are general notes and ideas about how this can be implemented.
They are not meant to be set in stone.
* A backend object can be deterministically hashed to form a unique ID for that
backend. The "opaque string" which is added to a user's domain's TXT record is
this ID.
* The gateway processes use a shared garage S3 bucket for persistence of
domain/backend mappings.
* Each process periodically re-reads all mapping data from S3 and uses that as
its currently active set of mappings. A pubsub system like NATS can be added
later to ensure that re-syncing happens at the same time.
* Each file in the bucket corresponds to a domain name which has been
successfully setup. The value contains the backend which is set up for the
domain.
* When the "Check" button is clicked during the "Setup/Update a Domain" flows,
the gateway process which handles the call does the following:
* Checks that a TXT record with the expected value exists on the domain being
added.
* Checks that a CNAME record with the expected value exists on the domain
being added.
* Performs ACME challenge for SSL cert. It will do this via an
[HTTP-01][http01] challenge. The challenge token will be put in the shared
garage S3 bucket, so that all gateway processes can potentially serve it
when requested.
* Depending on the backend, it may be required to sync some data into the
shared garage S3 bucket. For example, if the git backend is being used, then
the tip of the repo+branch will be synced into the S3 bucket.
* Adds a file to the shared garage S3 bucket corresponding to the new
domain/backend mapping.
* Each gateway process is responsible for keeping its own SSL certificates up to
date via periodic ACME challenges.
* Backend data in the shared garage S3 bucket will need to be periodically kept
up to date. A mechanism needs to be devised to do this reasonably reliably,
but without _all_ gateway processes constantly doing it. A communication layer
like NATS might come in handy here.
[http01]: https://letsencrypt.org/docs/challenge-types/#http-01-challenge
## Future Work
It should be possible for someone accessing a private domain on the cryptic
network to access the gateway web panel and use it _without_ requiring a secret
passphrase. This way community members who have already been vetted (by being
added to the VPN) can have even less friction.

11
docs/roadmap.md
View File

@ -39,16 +39,7 @@ auto-detect the installed operating system and install the daemon automatically.
One idea is to have every `cryptic-net daemon` run a webserver as one of its
sub-processes. This server could serve multiple functions:
- Possible public gateway, if the host is configured to be public, into internal
cryptic-net services. This will require some sort of service discovery
mechanism that other hosts in the network can easily hook into via their
`daemon.yml`s. Ideally this mechanism would involve little beyond updating
entries in garage, which the public gateways then pick up on.
One wrinkle here will be TLS certificates. Ideally internal hosts could host
websites publicly via the gateways on their network, using arbitrary domains
that the users own. The gateways will need some way of obtaining certs for
these domains automatically (or as automatically as possible).
- [Gateway service](./dev/gateway.md).
- Local interface for the `cryptic-net daemon` process. For example, status and
connectivity information for the local host could be provided via a simple web