See discussion in
https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/414. Apparently the
capacity is only relative to that of other instances in the layout, it
has no absolute value of its own. We can say it corresponds to GB for
ease-of-use, but garage doesn't see it that way.
I switched to using mlog for logging, as opposed to writing directly to
Stderr. This gives us control over log levels, as well as coordination
so that we don't have multiple go-routines writing to stderr at the same
time.
Putting bootstrap host data into garage, and applying garage layout
diff, no longer happen simultaneously in the background. This was
causing some weird non-determinism in the startup which wasn't really
breaking anything, but made the logs harder to debug.
This also potentially fixes `waitForGarageAndNebula`, which was
neglecting to wait for nebula if there were allocations defined.
I had previously made the mistake of thinking that the Curve25519 key
which is generated for each host to use in nebula communication could
also be used for signing. This is not the case, Ed25519 is used for
signing and is different thant Curve25519.
Rather than figuring out how to convert the Curve25519 key into an
Ed25519 key, which there is no apparent support for in the standard
library, I opted to instead ship a separate key just for signing with
each host. Doing this required a bit of refactoring in order to keep all
the different keys straight and ensure all data which needs a signature
still has it.
This allows each host to verify the cert against the CA cert. We also
now have each host sign the yaml file that it posts to garage, to ensure
that a host can't arbitrarily overwrite another host's file.
As part of this all "wait" constraints have been migrated to pure-go
implementations, taking advantage of pmux's `StartAfterFunc` argument.
nebula-entrypoint was the final main process besides the entrypoint
itself, allowing us to get rid of cryptic-net-main.
The new code runs the equivalent functionality within the daemon go
code. It was required to make Env be immutable in order to prevent race
conditions (this really should have been done from the beginning
anyway).
This required a lot of re-implementation of how garage gets interacted
with, including updating cluster layout using the admin API and
initialization of the global bucket key.
