I had previously made the mistake of thinking that the Curve25519 key
which is generated for each host to use in nebula communication could
also be used for signing. This is not the case, Ed25519 is used for
signing and is different thant Curve25519.
Rather than figuring out how to convert the Curve25519 key into an
Ed25519 key, which there is no apparent support for in the standard
library, I opted to instead ship a separate key just for signing with
each host. Doing this required a bit of refactoring in order to keep all
the different keys straight and ensure all data which needs a signature
still has it.
This allows each host to verify the cert against the CA cert. We also
now have each host sign the yaml file that it posts to garage, to ensure
that a host can't arbitrarily overwrite another host's file.
As part of this all "wait" constraints have been migrated to pure-go
implementations, taking advantage of pmux's `StartAfterFunc` argument.
nebula-entrypoint was the final main process besides the entrypoint
itself, allowing us to get rid of cryptic-net-main.
The new code runs the equivalent functionality within the daemon go
code. It was required to make Env be immutable in order to prevent race
conditions (this really should have been done from the beginning
anyway).
This required a lot of re-implementation of how garage gets interacted
with, including updating cluster layout using the admin API and
initialization of the global bucket key.
Previously if the `daemon.yml` of a host was changed it would first
start up, load that new daemon.yml in, persist the new configuration for
the host to garage using `update-garage-host`, pull that config back
down and persist it to the bootstrap in `runDaemonPmuxOnce`, and restart
all child processes so they get the new config.
Now, once `daemon.yml` is loaded in we immediately merge it into and
persist this host's bootstrap file, prior to ever starting child
processes. This removes the necessity of restarting those process at
start.
This change also allows the bootstrap file to be the sole repository of
information required to pick a garage node to connect to, since it is
presumably always as up-to-date as it can possibly be. This allows for
removing some more logic from `Env`.
The `bootstrap/creator` package is gone, almost as quickly as it
arrived. The `Bootstrap` type is now able to write its own tgz file, and
the two places where bootstrap files are being created pull the data
down to do so and create the `Bootstrap` structs directly.
The structure of the bootstrap file itself has been changed, now there's
just a single `hosts` directory which contains files which are yaml
encodings of the `Host` type, rather than having it be split into
`nebula` and `garage` directories. This makes creating bootstrap files a
lot easier.
Host types have been moved within the `bootstrap` package.
Refactored how boostrap FS is interacted with. There is now a
`Bootstrap` struct which has pre-loaded all data within the bootstrap
FS. This helps centralize the logic around reading the data (though not
yet completely).
Choosing of the garage node to interact with no longer occurs in like
three different places. It occurs at the environment level now, and is
aided by the new `garage.Peer` type.
The new code makes it a lot clearer what the sources of each
file/directory is, and makes it more difficult to forget to add a file
or directory. This will be helpful when it comes to bootstrapping an
entire network, which will require yet a third way of generating
bootstrap files.
