Compare commits

..

7 Commits

42 changed files with 1056 additions and 601 deletions

View File

@ -173,6 +173,7 @@ in rec {
pkgs.yq-go pkgs.yq-go
pkgs.jq pkgs.jq
pkgs.dig pkgs.dig
pkgs.nebula
]} ]}
export SHELL=${pkgs.bash}/bin/bash export SHELL=${pkgs.bash}/bin/bash
exec ${pkgs.bash}/bin/bash ${./tests}/entrypoint.sh "$@" exec ${pkgs.bash}/bin/bash ${./tests}/entrypoint.sh "$@"

View File

@ -23,16 +23,13 @@ was configured when creating the network.
## Step 3: Create a `bootstrap.json` File ## Step 3: Create a `bootstrap.json` File
Access to an `admin.json` file is required for this step.
To create a `bootstrap.json` file for the new host, the admin should perform the To create a `bootstrap.json` file for the new host, the admin should perform the
following command from their own host: following command from their own host:
``` ```
isle admin create-bootstrap \ isle hosts create \
--hostname <name> \ --hostname <name> \
--ip <ip> \ --ip <ip> \
--admin-path <path to admin.json> \
> bootstrap.json > bootstrap.json
``` ```
@ -44,21 +41,3 @@ The user can now proceed with calling `isle network join`, as described in the
[Getting Started][getting-started] document. [Getting Started][getting-started] document.
[getting-started]: ../user/getting-started.md [getting-started]: ../user/getting-started.md
### Encrypted `admin.json`
If `admin.json` is kept in an encrypted format on disk (it should be!) then the
decrypted form can be piped into `create-bootstrap` over stdin. For example, if
GPG is being used to secure `admin.json` then the following could be used to
generate a `bootstrap.json`:
```
gpg -d <path to admin.json.gpg> | isle admin create-bootstrap \
--hostname <name> \
--ip <ip> \
--admin-path - \
> bootstrap.json
```
Note that the value of `--admin-path` is `-`, indicating that `admin.json`
should be read from stdin.

View File

@ -82,44 +82,18 @@ be chosen with care.
* IP: The IP of your host, which will be the first host in the network. This IP * IP: The IP of your host, which will be the first host in the network. This IP
must be within the chosen subnet range. must be within the chosen subnet range.
## Step 3: Prepare to Encrypt `admin.json` ## Step 3: Create the Network
The `admin.json` file (which will be created in the next step) is the most To create the network, run:
sensitive part of an isle network. If it falls into the wrong hands it can be
used to completely compromise your network, impersonate hosts on the network,
and will likely lead to someone stealing or deleting all of your data.
Therefore it is important that the file remains encrypted when it is not being
used, and that it is never stored to disk in its decrypted form.
This guide assumes that you have GPG already set up with your own secret key,
and that you are familiar with how it works. There is no requirement to use GPG,
if you care to use a different method.
## Step 4: Create the `admin.json` File
To create the network, and the `admin.json` file in the process, run:
``` ```
sudo isle network create \ sudo isle network create \
--name <name> \ --name <name> \
--ip-net <subnet> \ --ip-net <subnet> \
--domain <domain> \ --domain <domain> \
--hostname <hostname> \ --hostname <hostname>
| gpg -e -r <my gpg email> \
> admin.json.gpg
``` ```
A couple of notes here:
* Only one gpg recipient is specified. If you intend on including other users as
network administrators you can add them to the recipients list at this step,
so they will be able to use the `admin.json` file as well. You can also
manually add them as recipients later.
The `isle network create` command may take up to a minute to complete. Once
completed you should have an `admin.json.gpg` file in your current directory.
At this point your host, and your network, are ready to go! To add other hosts At this point your host, and your network, are ready to go! To add other hosts
to the network you can reference the [Adding a Host to the Network][add-host] to the network you can reference the [Adding a Host to the Network][add-host]
document. document.

View File

@ -8,6 +8,14 @@ order they will be implemented.
These items are listed more or less in the order they need to be completed, as These items are listed more or less in the order they need to be completed, as
they generally depend on the items previous to them. they generally depend on the items previous to them.
### Windows Support + GUI
Support for Windows is a must. This requirement also includes a simple GUI,
which would essentially act as a thin layer on top of `daemon.yml` to start
with.
Depending on difficulty level, OSX support might be added at this stage as well.
### NATS ### NATS
Garage is currently used to handle eventually-consistent persistent storage, but Garage is currently used to handle eventually-consistent persistent storage, but
@ -36,14 +44,6 @@ files. The bootstrap file would be stored, encrypted, in garage, with the invite
code being able to both identify and decrypt it. To instantiate a host, the user code being able to both identify and decrypt it. To instantiate a host, the user
only needs to input the network domain name and the invite code. only needs to input the network domain name and the invite code.
### Windows Support + GUI
Support for Windows is a must. This requirement also includes a simple GUI,
which would essentially act as a thin layer on top of `daemon.yml` to start
with.
Depending on difficulty level, OSX support might be added at this stage as well.
### FUSE Mount ### FUSE Mount
KBFS style. Every user should be able to mount virtual directories to their host KBFS style. Every user should be able to mount virtual directories to their host

View File

@ -1,43 +0,0 @@
// Package admin deals with the parsing and creation of admin.json files.
package admin
import (
"encoding/json"
"io"
"isle/garage"
"isle/nebula"
)
// CreationParams are general parameters used when creating a new network. These
// are available to all hosts within the network via their bootstrap files.
type CreationParams struct {
ID string
Name string
Domain string
}
// Admin is used for accessing all information contained within an admin.json.
type Admin struct {
CreationParams CreationParams
Nebula struct {
CACredentials nebula.CACredentials
}
Garage struct {
RPCSecret string
GlobalBucketS3APICredentials garage.S3APICredentials
}
}
// FromReader reads an admin.json from the given io.Reader.
func FromReader(r io.Reader) (Admin, error) {
var a Admin
err := json.NewDecoder(r).Decode(&a)
return a, err
}
// WriteTo writes the Admin as an admin.json to the given io.Writer.
func (a Admin) WriteTo(into io.Writer) error {
return json.NewEncoder(into).Encode(a)
}

View File

@ -6,12 +6,8 @@ import (
"crypto/sha512" "crypto/sha512"
"encoding/json" "encoding/json"
"fmt" "fmt"
"io"
"isle/admin"
"isle/garage"
"isle/nebula" "isle/nebula"
"net/netip" "net/netip"
"os"
"path/filepath" "path/filepath"
"sort" "sort"
) )
@ -28,24 +24,19 @@ func AppDirPath(appDirPath string) string {
return filepath.Join(appDirPath, "share/bootstrap.json") return filepath.Join(appDirPath, "share/bootstrap.json")
} }
// Garage contains parameters needed to connect to and use the garage cluster. // CreationParams are general parameters used when creating a new network. These
type Garage struct { // are available to all hosts within the network via their bootstrap files.
// TODO this should be part of some new configuration section related to type CreationParams struct {
// secrets which may or may not be granted to this host ID string
RPCSecret string Name string
Domain string
AdminToken string
// TODO this should be part of admin.CreationParams
GlobalBucketS3APICredentials garage.S3APICredentials
} }
// Bootstrap is used for accessing all information contained within a // Bootstrap contains all information which is needed by a host daemon to join a
// bootstrap.json file. // network on boot.
type Bootstrap struct { type Bootstrap struct {
AdminCreationParams admin.CreationParams NetworkCreationParams CreationParams
CAPublicCredentials nebula.CAPublicCredentials CAPublicCredentials nebula.CAPublicCredentials
Garage Garage
PrivateCredentials nebula.HostPrivateCredentials PrivateCredentials nebula.HostPrivateCredentials
HostAssigned `json:"-"` HostAssigned `json:"-"`
@ -58,8 +49,7 @@ type Bootstrap struct {
// function assigns Hosts an empty map. // function assigns Hosts an empty map.
func New( func New(
caCreds nebula.CACredentials, caCreds nebula.CACredentials,
adminCreationParams admin.CreationParams, adminCreationParams CreationParams,
garage Garage,
name nebula.HostName, name nebula.HostName,
ip netip.Addr, ip netip.Addr,
) ( ) (
@ -83,33 +73,18 @@ func New(
} }
return Bootstrap{ return Bootstrap{
AdminCreationParams: adminCreationParams, NetworkCreationParams: adminCreationParams,
CAPublicCredentials: caCreds.Public, CAPublicCredentials: caCreds.Public,
Garage: garage, PrivateCredentials: hostPrivCreds,
PrivateCredentials: hostPrivCreds, HostAssigned: assigned,
HostAssigned: assigned, SignedHostAssigned: signedAssigned,
SignedHostAssigned: signedAssigned, Hosts: map[nebula.HostName]Host{},
Hosts: map[nebula.HostName]Host{},
}, nil }, nil
} }
// FromFile reads a bootstrap from a file at the given path. The HostAssigned // UnmarshalJSON implements the json.Unmarshaler interface. It will
// field will automatically be unwrapped. // automatically populate the HostAssigned field by unwrapping the
func FromFile(path string) (Bootstrap, error) { // SignedHostAssigned field.
f, err := os.Open(path)
if err != nil {
return Bootstrap{}, fmt.Errorf("opening file: %w", err)
}
defer f.Close()
var b Bootstrap
if err := json.NewDecoder(f).Decode(&b); err != nil {
return Bootstrap{}, fmt.Errorf("decoding json: %w", err)
}
return b, nil
}
func (b *Bootstrap) UnmarshalJSON(data []byte) error { func (b *Bootstrap) UnmarshalJSON(data []byte) error {
type inner Bootstrap type inner Bootstrap
@ -128,11 +103,6 @@ func (b *Bootstrap) UnmarshalJSON(data []byte) error {
return nil return nil
} }
// WriteTo writes the Bootstrap as a new bootstrap to the given io.Writer.
func (b Bootstrap) WriteTo(into io.Writer) error {
return json.NewEncoder(into).Encode(b)
}
// ThisHost is a shortcut for b.Hosts[b.HostName], but will panic if the // ThisHost is a shortcut for b.Hosts[b.HostName], but will panic if the
// HostName isn't found in the Hosts map. // HostName isn't found in the Hosts map.
func (b Bootstrap) ThisHost() Host { func (b Bootstrap) ThisHost() Host {

View File

@ -4,24 +4,6 @@ import (
"isle/garage" "isle/garage"
) )
// GarageClientParams contains all the data needed to instantiate garage
// clients.
type GarageClientParams struct {
Peer garage.RemotePeer
GlobalBucketS3APICredentials garage.S3APICredentials
RPCSecret string
}
// GlobalBucketS3APIClient returns an S3 client pre-configured with access to
// the global bucket.
func (p GarageClientParams) GlobalBucketS3APIClient() garage.S3APIClient {
var (
addr = p.Peer.S3APIAddr()
creds = p.GlobalBucketS3APICredentials
)
return garage.NewS3APIClient(addr, creds)
}
// GaragePeers returns a Peer for each known garage instance in the network. // GaragePeers returns a Peer for each known garage instance in the network.
func (b Bootstrap) GaragePeers() []garage.RemotePeer { func (b Bootstrap) GaragePeers() []garage.RemotePeer {
@ -69,12 +51,3 @@ func (b Bootstrap) ChooseGaragePeer() garage.RemotePeer {
panic("no garage instances configured") panic("no garage instances configured")
} }
// GarageClientParams returns a GarageClientParams.
func (b Bootstrap) GarageClientParams() GarageClientParams {
return GarageClientParams{
Peer: b.ChooseGaragePeer(),
GlobalBucketS3APICredentials: b.Garage.GlobalBucketS3APICredentials,
RPCSecret: b.Garage.RPCSecret,
}
}

View File

@ -3,7 +3,7 @@ package bootstrap
import ( import (
"fmt" "fmt"
"isle/nebula" "isle/nebula"
"net" "net/netip"
) )
// NebulaHost describes the nebula configuration of a Host which is relevant for // NebulaHost describes the nebula configuration of a Host which is relevant for
@ -77,10 +77,17 @@ type Host struct {
// //
// This assumes that the Host and its data has already been verified against the // This assumes that the Host and its data has already been verified against the
// CA signing key. // CA signing key.
func (h Host) IP() net.IP { func (h Host) IP() netip.Addr {
cert := h.PublicCredentials.Cert.Unwrap() cert := h.PublicCredentials.Cert.Unwrap()
if len(cert.Details.Ips) == 0 { if len(cert.Details.Ips) == 0 {
panic(fmt.Sprintf("host %q not configured with any ips: %+v", h.Name, h)) panic(fmt.Sprintf("host %q not configured with any ips: %+v", h.Name, h))
} }
return cert.Details.Ips[0].IP
ip := cert.Details.Ips[0].IP
addr, ok := netip.AddrFromSlice(ip)
if !ok {
panic(fmt.Sprintf("ip %q (%#v) is not valid, somehow", ip, ip))
}
return addr
} }

View File

@ -1,199 +0,0 @@
package main
import (
"crypto/rand"
"encoding/hex"
"errors"
"fmt"
"isle/admin"
"isle/bootstrap"
"isle/nebula"
"net/netip"
"os"
)
func randStr(l int) string {
b := make([]byte, l)
if _, err := rand.Read(b); err != nil {
panic(err)
}
return hex.EncodeToString(b)
}
func readAdmin(path string) (admin.Admin, error) {
if path == "-" {
adm, err := admin.FromReader(os.Stdin)
if err != nil {
return admin.Admin{}, fmt.Errorf("parsing admin.json from stdin: %w", err)
}
return adm, nil
}
f, err := os.Open(path)
if err != nil {
return admin.Admin{}, fmt.Errorf("opening file: %w", err)
}
defer f.Close()
return admin.FromReader(f)
}
var subCmdAdminCreateBootstrap = subCmd{
name: "create-bootstrap",
descr: "Creates a new bootstrap.json file for a particular host and writes it to stdout",
do: func(subCmdCtx subCmdCtx) error {
var (
flags = subCmdCtx.flagSet(false)
hostName nebula.HostName
ip netip.Addr
)
hostNameF := flags.VarPF(
textUnmarshalerFlag{&hostName},
"hostname", "h",
"Name of the host to generate bootstrap.json for",
)
ipF := flags.VarPF(
textUnmarshalerFlag{&ip}, "ip", "i", "IP of the new host",
)
adminPath := flags.StringP(
"admin-path", "a", "",
`Path to admin.json file. If the given path is "-" then stdin is used.`,
)
if err := flags.Parse(subCmdCtx.args); err != nil {
return fmt.Errorf("parsing flags: %w", err)
}
if !hostNameF.Changed ||
!ipF.Changed ||
*adminPath == "" {
return errors.New("--hostname, --ip, and --admin-path are required")
}
adm, err := readAdmin(*adminPath)
if err != nil {
return fmt.Errorf("reading admin.json with --admin-path of %q: %w", *adminPath, err)
}
garageBootstrap := bootstrap.Garage{
RPCSecret: adm.Garage.RPCSecret,
AdminToken: randStr(32),
GlobalBucketS3APICredentials: adm.Garage.GlobalBucketS3APICredentials,
}
newHostBootstrap, err := bootstrap.New(
adm.Nebula.CACredentials,
adm.CreationParams,
garageBootstrap,
hostName,
ip,
)
if err != nil {
return fmt.Errorf("initializing bootstrap data: %w", err)
}
hostsRes, err := subCmdCtx.getHosts()
if err != nil {
return fmt.Errorf("getting hosts: %w", err)
}
for _, host := range hostsRes.Hosts {
newHostBootstrap.Hosts[host.Name] = host
}
return newHostBootstrap.WriteTo(os.Stdout)
},
}
var subCmdAdminCreateNebulaCert = subCmd{
name: "create-nebula-cert",
descr: "Creates a signed nebula certificate file and writes it to stdout",
do: func(subCmdCtx subCmdCtx) error {
var (
flags = subCmdCtx.flagSet(false)
hostName nebula.HostName
ip netip.Addr
)
hostNameF := flags.VarPF(
textUnmarshalerFlag{&hostName},
"hostname", "h",
"Name of the host to generate a certificate for",
)
ipF := flags.VarPF(
textUnmarshalerFlag{&ip}, "ip", "i", "IP of the new host",
)
adminPath := flags.StringP(
"admin-path", "a", "",
`Path to admin.json file. If the given path is "-" then stdin is used.`,
)
pubKeyPath := flags.StringP(
"public-key-path", "p", "",
`Path to PEM file containing public key which will be embedded in the cert.`,
)
if err := flags.Parse(subCmdCtx.args); err != nil {
return fmt.Errorf("parsing flags: %w", err)
}
if !hostNameF.Changed ||
!ipF.Changed ||
*adminPath == "" ||
*pubKeyPath == "" {
return errors.New("--hostname, --ip, --admin-path, and --pub-key-path are required")
}
adm, err := readAdmin(*adminPath)
if err != nil {
return fmt.Errorf("reading admin.json with --admin-path of %q: %w", *adminPath, err)
}
hostPubPEM, err := os.ReadFile(*pubKeyPath)
if err != nil {
return fmt.Errorf("reading public key from %q: %w", *pubKeyPath, err)
}
var hostPub nebula.EncryptingPublicKey
if err := hostPub.UnmarshalNebulaPEM(hostPubPEM); err != nil {
return fmt.Errorf("unmarshaling public key as PEM: %w", err)
}
nebulaHostCert, err := nebula.NewHostCert(
adm.Nebula.CACredentials, hostPub, hostName, ip,
)
if err != nil {
return fmt.Errorf("creating cert: %w", err)
}
nebulaHostCertPEM, err := nebulaHostCert.Unwrap().MarshalToPEM()
if err != nil {
return fmt.Errorf("marshaling cert to PEM: %w", err)
}
if _, err := os.Stdout.Write([]byte(nebulaHostCertPEM)); err != nil {
return fmt.Errorf("writing to stdout: %w", err)
}
return nil
},
}
var subCmdAdmin = subCmd{
name: "admin",
descr: "Sub-commands which only admins can run",
do: func(subCmdCtx subCmdCtx) error {
return subCmdCtx.doSubCmd(
subCmdAdminCreateBootstrap,
subCmdAdminCreateNebulaCert,
)
},
}

View File

@ -1,12 +1,13 @@
package main package main
import ( import (
"errors"
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
"syscall" "syscall"
"isle/bootstrap" "isle/daemon"
) )
// minio-client keeps a configuration directory which contains various pieces of // minio-client keeps a configuration directory which contains various pieces of
@ -52,7 +53,7 @@ var subCmdGarageMC = subCmd{
return fmt.Errorf("parsing flags: %w", err) return fmt.Errorf("parsing flags: %w", err)
} }
var clientParams bootstrap.GarageClientParams var clientParams daemon.GarageClientParams
err := subCmdCtx.daemonRCPClient.Call( err := subCmdCtx.daemonRCPClient.Call(
subCmdCtx.ctx, &clientParams, "GetGarageClientParams", nil, subCmdCtx.ctx, &clientParams, "GetGarageClientParams", nil,
) )
@ -119,7 +120,7 @@ var subCmdGarageCLI = subCmd{
descr: "Runs the garage binary, automatically configured to point to the garage sub-process of a running isle daemon", descr: "Runs the garage binary, automatically configured to point to the garage sub-process of a running isle daemon",
do: func(subCmdCtx subCmdCtx) error { do: func(subCmdCtx subCmdCtx) error {
var clientParams bootstrap.GarageClientParams var clientParams daemon.GarageClientParams
err := subCmdCtx.daemonRCPClient.Call( err := subCmdCtx.daemonRCPClient.Call(
subCmdCtx.ctx, &clientParams, "GetGarageClientParams", nil, subCmdCtx.ctx, &clientParams, "GetGarageClientParams", nil,
) )
@ -127,6 +128,10 @@ var subCmdGarageCLI = subCmd{
return fmt.Errorf("calling GetGarageClientParams: %w", err) return fmt.Errorf("calling GetGarageClientParams: %w", err)
} }
if clientParams.RPCSecret == "" {
return errors.New("this host does not have the garage RPC secret")
}
var ( var (
binPath = binPath("garage") binPath = binPath("garage")
args = append([]string{"garage"}, subCmdCtx.args...) args = append([]string{"garage"}, subCmdCtx.args...)

View File

@ -1,16 +1,73 @@
package main package main
import ( import (
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"isle/bootstrap" "isle/bootstrap"
"isle/daemon" "isle/daemon"
"isle/jsonutil" "isle/jsonutil"
"isle/nebula" "isle/nebula"
"net/netip"
"os" "os"
"sort" "sort"
) )
var subCmdHostsCreate = subCmd{
name: "create",
descr: "Creates a new host in the network, writing its new bootstrap.json to stdout",
do: func(subCmdCtx subCmdCtx) error {
var (
flags = subCmdCtx.flagSet(false)
hostName nebula.HostName
ip netip.Addr
)
hostNameF := flags.VarPF(
textUnmarshalerFlag{&hostName},
"hostname", "h",
"Name of the host to generate bootstrap.json for",
)
ipF := flags.VarPF(
textUnmarshalerFlag{&ip}, "ip", "i", "IP of the new host",
)
canCreateHosts := flags.Bool(
"can-create-hosts",
false,
"The new host should have the ability to create hosts too",
)
if err := flags.Parse(subCmdCtx.args); err != nil {
return fmt.Errorf("parsing flags: %w", err)
}
if !hostNameF.Changed || !ipF.Changed {
return errors.New("--hostname and --ip are required")
}
var res daemon.CreateHostResult
err := subCmdCtx.daemonRCPClient.Call(
subCmdCtx.ctx,
&res,
"CreateHost",
daemon.CreateHostRequest{
HostName: hostName,
IP: ip,
Opts: daemon.CreateHostOpts{
CanCreateHosts: *canCreateHosts,
},
},
)
if err != nil {
return fmt.Errorf("calling CreateHost: %w", err)
}
return json.NewEncoder(os.Stdout).Encode(res.JoiningBootstrap)
},
}
var subCmdHostsList = subCmd{ var subCmdHostsList = subCmd{
name: "list", name: "list",
descr: "Lists all hosts in the network, and their IPs", descr: "Lists all hosts in the network, and their IPs",
@ -88,6 +145,7 @@ var subCmdHosts = subCmd{
descr: "Sub-commands having to do with configuration of hosts in the network", descr: "Sub-commands having to do with configuration of hosts in the network",
do: func(subCmdCtx subCmdCtx) error { do: func(subCmdCtx subCmdCtx) error {
return subCmdCtx.doSubCmd( return subCmdCtx.doSubCmd(
subCmdHostsCreate,
subCmdHostsRemove, subCmdHostsRemove,
subCmdHostsList, subCmdHostsList,
) )

View File

@ -61,7 +61,6 @@ func main() {
ctx: ctx, ctx: ctx,
logger: logger, logger: logger,
}.doSubCmd( }.doSubCmd(
subCmdAdmin,
subCmdDaemon, subCmdDaemon,
subCmdGarage, subCmdGarage,
subCmdHosts, subCmdHosts,

View File

@ -1,12 +1,79 @@
package main package main
import ( import (
"errors"
"fmt" "fmt"
"isle/daemon"
"isle/jsonutil" "isle/jsonutil"
"isle/nebula" "isle/nebula"
"os" "os"
) )
var subCmdNebulaCreateCert = subCmd{
name: "create-cert",
descr: "Creates a signed nebula certificate file for an existing host and writes it to stdout",
do: func(subCmdCtx subCmdCtx) error {
var (
flags = subCmdCtx.flagSet(false)
hostName nebula.HostName
)
hostNameF := flags.VarPF(
textUnmarshalerFlag{&hostName},
"hostname", "h",
"Name of the host to generate a certificate for",
)
pubKeyPath := flags.StringP(
"public-key-path", "p", "",
`Path to PEM file containing public key which will be embedded in the cert.`,
)
if err := flags.Parse(subCmdCtx.args); err != nil {
return fmt.Errorf("parsing flags: %w", err)
}
if !hostNameF.Changed || *pubKeyPath == "" {
return errors.New("--hostname, --admin-path, and --pub-key-path are required")
}
hostPubPEM, err := os.ReadFile(*pubKeyPath)
if err != nil {
return fmt.Errorf("reading public key from %q: %w", *pubKeyPath, err)
}
var hostPub nebula.EncryptingPublicKey
if err := hostPub.UnmarshalNebulaPEM(hostPubPEM); err != nil {
return fmt.Errorf("unmarshaling public key as PEM: %w", err)
}
var res daemon.CreateNebulaCertificateResult
err = subCmdCtx.daemonRCPClient.Call(
subCmdCtx.ctx,
&res,
"CreateNebulaCertificate",
daemon.CreateNebulaCertificateRequest{
HostName: hostName,
HostEncryptingPublicKey: hostPub,
},
)
if err != nil {
return fmt.Errorf("calling CreateNebulaCertificate: %w", err)
}
nebulaHostCertPEM, err := res.HostNebulaCertifcate.Unwrap().MarshalToPEM()
if err != nil {
return fmt.Errorf("marshaling cert to PEM: %w", err)
}
if _, err := os.Stdout.Write([]byte(nebulaHostCertPEM)); err != nil {
return fmt.Errorf("writing to stdout: %w", err)
}
return nil
},
}
var subCmdNebulaShow = subCmd{ var subCmdNebulaShow = subCmd{
name: "show", name: "show",
descr: "Writes nebula network information to stdout in JSON format", descr: "Writes nebula network information to stdout in JSON format",
@ -30,20 +97,17 @@ var subCmdNebulaShow = subCmd{
return fmt.Errorf("calling GetNebulaCAPublicCredentials: %w", err) return fmt.Errorf("calling GetNebulaCAPublicCredentials: %w", err)
} }
caCert := caPublicCreds.Cert.Unwrap() caCert := caPublicCreds.Cert
caCertPEM, err := caCert.MarshalToPEM() caCertDetails := caCert.Unwrap().Details
if err != nil {
return fmt.Errorf("marshaling CA cert to PEM: %w", err)
}
if len(caCert.Details.Subnets) != 1 { if len(caCertDetails.Subnets) != 1 {
return fmt.Errorf( return fmt.Errorf(
"malformed ca.crt, contains unexpected subnets %#v", "malformed ca.crt, contains unexpected subnets %#v",
caCert.Details.Subnets, caCertDetails.Subnets,
) )
} }
subnet := caCert.Details.Subnets[0] subnet := caCertDetails.Subnets[0]
type outLighthouse struct { type outLighthouse struct {
PublicAddr string PublicAddr string
@ -51,11 +115,11 @@ var subCmdNebulaShow = subCmd{
} }
out := struct { out := struct {
CACert string CACert nebula.Certificate
SubnetCIDR string SubnetCIDR string
Lighthouses []outLighthouse Lighthouses []outLighthouse
}{ }{
CACert: string(caCertPEM), CACert: caCert,
SubnetCIDR: subnet.String(), SubnetCIDR: subnet.String(),
} }
@ -83,6 +147,7 @@ var subCmdNebula = subCmd{
descr: "Sub-commands related to the nebula VPN", descr: "Sub-commands related to the nebula VPN",
do: func(subCmdCtx subCmdCtx) error { do: func(subCmdCtx subCmdCtx) error {
return subCmdCtx.doSubCmd( return subCmdCtx.doSubCmd(
subCmdNebulaCreateCert,
subCmdNebulaShow, subCmdNebulaShow,
) )
}, },

View File

@ -3,15 +3,13 @@ package main
import ( import (
"errors" "errors"
"fmt" "fmt"
"isle/admin"
"isle/bootstrap"
"isle/daemon" "isle/daemon"
"os" "isle/jsonutil"
) )
var subCmdNetworkCreate = subCmd{ var subCmdNetworkCreate = subCmd{
name: "create", name: "create",
descr: "Create's a new network, with this host being the first host in that network. The resulting admin.json is output to stdout.", descr: "Create's a new network, with this host being the first host in that network.",
do: func(subCmdCtx subCmdCtx) error { do: func(subCmdCtx subCmdCtx) error {
var ( var (
ctx = subCmdCtx.ctx ctx = subCmdCtx.ctx
@ -53,16 +51,11 @@ var subCmdNetworkCreate = subCmd{
return errors.New("--name, --domain, --ip-net, and --hostname are required") return errors.New("--name, --domain, --ip-net, and --hostname are required")
} }
var adm admin.Admin err := subCmdCtx.daemonRCPClient.Call(ctx, nil, "CreateNetwork", req)
err := subCmdCtx.daemonRCPClient.Call(ctx, &adm, "CreateNetwork", req)
if err != nil { if err != nil {
return fmt.Errorf("creating network: %w", err) return fmt.Errorf("creating network: %w", err)
} }
if err := adm.WriteTo(os.Stdout); err != nil {
return fmt.Errorf("writing admin.json to stdout")
}
return nil return nil
}, },
} }
@ -88,8 +81,8 @@ var subCmdNetworkJoin = subCmd{
return errors.New("--bootstrap-path is required") return errors.New("--bootstrap-path is required")
} }
newBootstrap, err := bootstrap.FromFile(*bootstrapPath) var newBootstrap daemon.JoiningBootstrap
if err != nil { if err := jsonutil.LoadFile(&newBootstrap, *bootstrapPath); err != nil {
return fmt.Errorf( return fmt.Errorf(
"loading bootstrap from %q: %w", *bootstrapPath, err, "loading bootstrap from %q: %w", *bootstrapPath, err,
) )

View File

@ -1,14 +1,24 @@
package daemon package daemon
import ( import (
"encoding/json"
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
"isle/bootstrap" "isle/bootstrap"
"isle/garage/garagesrv" "isle/garage/garagesrv"
"isle/jsonutil"
"isle/secrets"
) )
// JoiningBootstrap wraps a normal Bootstrap to include extra data which a host
// might need while joining a network.
type JoiningBootstrap struct {
Bootstrap bootstrap.Bootstrap
Secrets map[secrets.ID]json.RawMessage
}
func writeBootstrapToStateDir( func writeBootstrapToStateDir(
stateDirPath string, hostBootstrap bootstrap.Bootstrap, stateDirPath string, hostBootstrap bootstrap.Bootstrap,
) error { ) error {
@ -21,14 +31,11 @@ func writeBootstrapToStateDir(
return fmt.Errorf("creating directory %q: %w", dirPath, err) return fmt.Errorf("creating directory %q: %w", dirPath, err)
} }
f, err := os.Create(path) if err := jsonutil.WriteFile(hostBootstrap, path, 0700); err != nil {
if err != nil { return fmt.Errorf("writing bootstrap to %q: %w", path, err)
return fmt.Errorf("creating file %q: %w", path, err)
} }
defer f.Close() return nil
return hostBootstrap.WriteTo(f)
} }
func coalesceDaemonConfigAndBootstrap( func coalesceDaemonConfigAndBootstrap(

View File

@ -33,7 +33,7 @@ func dnsmasqPmuxProcConfig(
confData := dnsmasq.ConfData{ confData := dnsmasq.ConfData{
Resolvers: daemonConfig.DNS.Resolvers, Resolvers: daemonConfig.DNS.Resolvers,
Domain: hostBootstrap.AdminCreationParams.Domain, Domain: hostBootstrap.NetworkCreationParams.Domain,
IP: hostBootstrap.ThisHost().IP().String(), IP: hostBootstrap.ThisHost().IP().String(),
Hosts: hostsSlice, Hosts: hostsSlice,
} }

View File

@ -19,11 +19,12 @@ func garageAdminClientLogger(logger *mlog.Logger) *mlog.Logger {
return logger.WithNamespace("garageAdminClient") return logger.WithNamespace("garageAdminClient")
} }
// NewGarageAdminClient will return an AdminClient for a local garage instance, // newGarageAdminClient will return an AdminClient for a local garage instance,
// or it will _panic_ if there is no local instance configured. // or it will _panic_ if there is no local instance configured.
func NewGarageAdminClient( func newGarageAdminClient(
logger *mlog.Logger, logger *mlog.Logger,
daemonConfig Config, daemonConfig Config,
adminToken string,
hostBootstrap bootstrap.Bootstrap, hostBootstrap bootstrap.Bootstrap,
) *garage.AdminClient { ) *garage.AdminClient {
@ -35,7 +36,7 @@ func NewGarageAdminClient(
thisHost.IP().String(), thisHost.IP().String(),
strconv.Itoa(daemonConfig.Storage.Allocations[0].AdminPort), strconv.Itoa(daemonConfig.Storage.Allocations[0].AdminPort),
), ),
hostBootstrap.Garage.AdminToken, adminToken,
) )
} }
@ -43,6 +44,7 @@ func waitForGarage(
ctx context.Context, ctx context.Context,
logger *mlog.Logger, logger *mlog.Logger,
daemonConfig Config, daemonConfig Config,
adminToken string,
hostBootstrap bootstrap.Bootstrap, hostBootstrap bootstrap.Bootstrap,
) error { ) error {
@ -64,9 +66,7 @@ func waitForGarage(
) )
adminClient := garage.NewAdminClient( adminClient := garage.NewAdminClient(
adminClientLogger, adminClientLogger, adminAddr, adminToken,
adminAddr,
hostBootstrap.Garage.AdminToken,
) )
ctx := mctx.Annotate(ctx, "garageAdminAddr", adminAddr) ctx := mctx.Annotate(ctx, "garageAdminAddr", adminAddr)
@ -101,7 +101,7 @@ func bootstrapGarageHostForAlloc(
} }
func garageWriteChildConfig( func garageWriteChildConfig(
runtimeDirPath string, rpcSecret, runtimeDirPath, adminToken string,
hostBootstrap bootstrap.Bootstrap, hostBootstrap bootstrap.Bootstrap,
alloc ConfigStorageAllocation, alloc ConfigStorageAllocation,
) ( ) (
@ -129,8 +129,8 @@ func garageWriteChildConfig(
MetaPath: alloc.MetaPath, MetaPath: alloc.MetaPath,
DataPath: alloc.DataPath, DataPath: alloc.DataPath,
RPCSecret: hostBootstrap.Garage.RPCSecret, RPCSecret: rpcSecret,
AdminToken: hostBootstrap.Garage.AdminToken, AdminToken: adminToken,
LocalPeer: peer, LocalPeer: peer,
BootstrapPeers: hostBootstrap.GaragePeers(), BootstrapPeers: hostBootstrap.GaragePeers(),
@ -144,20 +144,29 @@ func garageWriteChildConfig(
} }
func garagePmuxProcConfigs( func garagePmuxProcConfigs(
ctx context.Context,
logger *mlog.Logger, logger *mlog.Logger,
runtimeDirPath, binDirPath string, rpcSecret, runtimeDirPath, binDirPath string,
daemonConfig Config, daemonConfig Config,
adminToken string,
hostBootstrap bootstrap.Bootstrap, hostBootstrap bootstrap.Bootstrap,
) ( ) (
[]pmuxlib.ProcessConfig, error, []pmuxlib.ProcessConfig, error,
) { ) {
var (
pmuxProcConfigs []pmuxlib.ProcessConfig
allocs = daemonConfig.Storage.Allocations
)
var pmuxProcConfigs []pmuxlib.ProcessConfig if len(allocs) > 0 && rpcSecret == "" {
logger.WarnString(ctx, "Not starting garage instances for storage allocations, missing garage RPC secret")
return nil, nil
}
for _, alloc := range daemonConfig.Storage.Allocations { for _, alloc := range allocs {
childConfigPath, err := garageWriteChildConfig( childConfigPath, err := garageWriteChildConfig(
runtimeDirPath, hostBootstrap, alloc, rpcSecret, runtimeDirPath, adminToken, hostBootstrap, alloc,
) )
if err != nil { if err != nil {
return nil, fmt.Errorf("writing child config file for alloc %+v: %w", alloc, err) return nil, fmt.Errorf("writing child config file for alloc %+v: %w", alloc, err)
@ -176,21 +185,22 @@ func garagePmuxProcConfigs(
return pmuxProcConfigs, nil return pmuxProcConfigs, nil
} }
// TODO don't expose this publicly once cluster creation is done via Daemon func garageApplyLayout(
// interface.
func GarageApplyLayout(
ctx context.Context, ctx context.Context,
logger *mlog.Logger, logger *mlog.Logger,
daemonConfig Config, daemonConfig Config,
adminToken string,
hostBootstrap bootstrap.Bootstrap, hostBootstrap bootstrap.Bootstrap,
) error { ) error {
var ( var (
adminClient = NewGarageAdminClient(logger, daemonConfig, hostBootstrap) adminClient = newGarageAdminClient(
thisHost = hostBootstrap.ThisHost() logger, daemonConfig, adminToken, hostBootstrap,
hostName = thisHost.Name )
allocs = daemonConfig.Storage.Allocations thisHost = hostBootstrap.ThisHost()
peers = make([]garage.PeerLayout, len(allocs)) hostName = thisHost.Name
allocs = daemonConfig.Storage.Allocations
peers = make([]garage.PeerLayout, len(allocs))
) )
for i, alloc := range allocs { for i, alloc := range allocs {

View File

@ -22,7 +22,7 @@ func waitForNebula(
ctx context.Context, logger *mlog.Logger, hostBootstrap bootstrap.Bootstrap, ctx context.Context, logger *mlog.Logger, hostBootstrap bootstrap.Bootstrap,
) error { ) error {
var ( var (
ip = hostBootstrap.ThisHost().IP() ip = net.IP(hostBootstrap.ThisHost().IP().AsSlice())
lUDPAddr = &net.UDPAddr{IP: ip, Port: 0} lUDPAddr = &net.UDPAddr{IP: ip, Port: 0}
rUDPAddr = &net.UDPAddr{IP: ip, Port: 45535} rUDPAddr = &net.UDPAddr{IP: ip, Port: 45535}
) )

View File

@ -9,8 +9,14 @@ import (
) )
func (c *Children) newPmuxConfig( func (c *Children) newPmuxConfig(
binDirPath string, daemonConfig Config, hostBootstrap bootstrap.Bootstrap, ctx context.Context,
) (pmuxlib.Config, error) { garageRPCSecret, binDirPath string,
daemonConfig Config,
garageAdminToken string,
hostBootstrap bootstrap.Bootstrap,
) (
pmuxlib.Config, error,
) {
nebulaPmuxProcConfig, err := nebulaPmuxProcConfig( nebulaPmuxProcConfig, err := nebulaPmuxProcConfig(
c.opts.EnvVars.RuntimeDirPath, c.opts.EnvVars.RuntimeDirPath,
binDirPath, binDirPath,
@ -34,10 +40,13 @@ func (c *Children) newPmuxConfig(
} }
garagePmuxProcConfigs, err := garagePmuxProcConfigs( garagePmuxProcConfigs, err := garagePmuxProcConfigs(
ctx,
c.logger, c.logger,
garageRPCSecret,
c.opts.EnvVars.RuntimeDirPath, c.opts.EnvVars.RuntimeDirPath,
binDirPath, binDirPath,
daemonConfig, daemonConfig,
garageAdminToken,
hostBootstrap, hostBootstrap,
) )
if err != nil { if err != nil {
@ -60,6 +69,7 @@ func (c *Children) newPmuxConfig(
func (c *Children) postPmuxInit( func (c *Children) postPmuxInit(
ctx context.Context, ctx context.Context,
daemonConfig Config, daemonConfig Config,
garageAdminToken string,
hostBootstrap bootstrap.Bootstrap, hostBootstrap bootstrap.Bootstrap,
) error { ) error {
c.logger.Info(ctx, "Waiting for nebula VPN to come online") c.logger.Info(ctx, "Waiting for nebula VPN to come online")
@ -68,7 +78,9 @@ func (c *Children) postPmuxInit(
} }
c.logger.Info(ctx, "Waiting for garage instances to come online") c.logger.Info(ctx, "Waiting for garage instances to come online")
err := waitForGarage(ctx, c.logger, daemonConfig, hostBootstrap) err := waitForGarage(
ctx, c.logger, daemonConfig, garageAdminToken, hostBootstrap,
)
if err != nil { if err != nil {
return fmt.Errorf("waiting for garage to start: %w", err) return fmt.Errorf("waiting for garage to start: %w", err)
} }

View File

@ -2,8 +2,10 @@ package daemon
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"isle/bootstrap" "isle/bootstrap"
"isle/secrets"
"code.betamike.com/micropelago/pmux/pmuxlib" "code.betamike.com/micropelago/pmux/pmuxlib"
"dev.mediocregopher.com/mediocre-go-lib.git/mlog" "dev.mediocregopher.com/mediocre-go-lib.git/mlog"
@ -27,15 +29,23 @@ type Children struct {
func NewChildren( func NewChildren(
ctx context.Context, ctx context.Context,
logger *mlog.Logger, logger *mlog.Logger,
daemonConfig Config,
hostBootstrap bootstrap.Bootstrap,
binDirPath string, binDirPath string,
secretsStore secrets.Store,
daemonConfig Config,
garageAdminToken string,
hostBootstrap bootstrap.Bootstrap,
opts *Opts, opts *Opts,
) ( ) (
*Children, error, *Children, error,
) { ) {
opts = opts.withDefaults() opts = opts.withDefaults()
logger.Info(ctx, "Loading secrets")
garageRPCSecret, err := getGarageRPCSecret(ctx, secretsStore)
if err != nil && !errors.Is(err, secrets.ErrNotFound) {
return nil, fmt.Errorf("loading garage RPC secret: %w", err)
}
pmuxCtx, pmuxCancelFn := context.WithCancel(context.Background()) pmuxCtx, pmuxCancelFn := context.WithCancel(context.Background())
c := &Children{ c := &Children{
@ -45,7 +55,14 @@ func NewChildren(
pmuxStoppedCh: make(chan struct{}), pmuxStoppedCh: make(chan struct{}),
} }
pmuxConfig, err := c.newPmuxConfig(binDirPath, daemonConfig, hostBootstrap) pmuxConfig, err := c.newPmuxConfig(
ctx,
garageRPCSecret,
binDirPath,
daemonConfig,
garageAdminToken,
hostBootstrap,
)
if err != nil { if err != nil {
return nil, fmt.Errorf("generating pmux config: %w", err) return nil, fmt.Errorf("generating pmux config: %w", err)
} }
@ -56,7 +73,9 @@ func NewChildren(
c.logger.Debug(pmuxCtx, "pmux stopped") c.logger.Debug(pmuxCtx, "pmux stopped")
}() }()
initErr := c.postPmuxInit(ctx, daemonConfig, hostBootstrap) initErr := c.postPmuxInit(
ctx, daemonConfig, garageAdminToken, hostBootstrap,
)
if initErr != nil { if initErr != nil {
logger.Warn(ctx, "failed to initialize Children, shutting down child processes", err) logger.Warn(ctx, "failed to initialize Children, shutting down child processes", err)
if err := c.Shutdown(); err != nil { if err := c.Shutdown(); err != nil {

View File

@ -9,17 +9,26 @@ import (
"fmt" "fmt"
"io" "io"
"io/fs" "io/fs"
"isle/admin"
"isle/bootstrap" "isle/bootstrap"
"isle/garage" "isle/jsonutil"
"isle/nebula" "isle/nebula"
"isle/secrets"
"net/netip"
"os" "os"
"path/filepath"
"sync" "sync"
"time" "time"
"dev.mediocregopher.com/mediocre-go-lib.git/mlog" "dev.mediocregopher.com/mediocre-go-lib.git/mlog"
) )
// CreateHostOpts are optional parameters to the CreateHost method.
type CreateHostOpts struct {
// CanCreateHosts indicates that the bootstrap produced by CreateHost should
// give the new host the ability to create new hosts as well.
CanCreateHosts bool
}
// Daemon presents all functionality required for client frontends to interact // Daemon presents all functionality required for client frontends to interact
// with isle, typically via the unix socket. // with isle, typically via the unix socket.
type Daemon interface { type Daemon interface {
@ -32,29 +41,56 @@ type Daemon interface {
// become this first host's IP. // become this first host's IP.
// - hostName: The name of this first host in the network. // - hostName: The name of this first host in the network.
// //
// An Admin instance is returned, which is necessary to perform admin // The daemon on which this is called will become the first host in the
// actions in the future. // network, and will have full administrative privileges.
CreateNetwork( CreateNetwork(
ctx context.Context, name, domain string, ctx context.Context, name, domain string,
ipNet nebula.IPNet, ipNet nebula.IPNet,
hostName nebula.HostName, hostName nebula.HostName,
) ( ) error
admin.Admin, error,
)
// JoinNetwork joins the Daemon to an existing network using the given // JoinNetwork joins the Daemon to an existing network using the given
// Bootstrap. // Bootstrap.
// //
// Errors: // Errors:
// - ErrAlreadyJoined // - ErrAlreadyJoined
JoinNetwork(context.Context, bootstrap.Bootstrap) error JoinNetwork(context.Context, JoiningBootstrap) error
// GetBootstraps returns the currently active Bootstrap. // GetBootstraps returns the currently active Bootstrap.
GetBootstrap(context.Context) (bootstrap.Bootstrap, error) GetBootstrap(context.Context) (bootstrap.Bootstrap, error)
// GetGarageClientParams returns a GarageClientParams for the current
// network state.
GetGarageClientParams(context.Context) (GarageClientParams, error)
// RemoveHost removes the host of the given name from the network. // RemoveHost removes the host of the given name from the network.
RemoveHost(context.Context, nebula.HostName) error RemoveHost(context.Context, nebula.HostName) error
// CreateHost creates a bootstrap for a new host with the given name and IP
// address.
CreateHost(
ctx context.Context,
hostName nebula.HostName,
ip netip.Addr, // TODO automatically choose IP address
opts CreateHostOpts,
) (
JoiningBootstrap, error,
)
// CreateNebulaCertificate creates and signs a new nebula certficate for an
// existing host, given the public key for that host. This is currently
// mostly useful for creating certs for mobile devices.
//
// Errors:
// - ErrHostNotFound
CreateNebulaCertificate(
ctx context.Context,
hostName nebula.HostName,
hostPubKey nebula.EncryptingPublicKey,
) (
nebula.Certificate, error,
)
// Shutdown blocks until all resources held or created by the daemon, // Shutdown blocks until all resources held or created by the daemon,
// including child processes it has started, have been cleaned up. // including child processes it has started, have been cleaned up.
// //
@ -107,6 +143,9 @@ type daemon struct {
envBinDirPath string envBinDirPath string
opts *Opts opts *Opts
secretsStore secrets.Store
garageAdminToken string
l sync.RWMutex l sync.RWMutex
state int state int
children *Children children *Children
@ -141,11 +180,12 @@ func NewDaemon(
) { ) {
var ( var (
d = &daemon{ d = &daemon{
logger: logger, logger: logger,
daemonConfig: daemonConfig, daemonConfig: daemonConfig,
envBinDirPath: envBinDirPath, envBinDirPath: envBinDirPath,
opts: opts.withDefaults(), opts: opts.withDefaults(),
shutdownCh: make(chan struct{}), garageAdminToken: randStr(32),
shutdownCh: make(chan struct{}),
} }
bootstrapFilePath = bootstrap.StateDirPath(d.opts.EnvVars.StateDirPath) bootstrapFilePath = bootstrap.StateDirPath(d.opts.EnvVars.StateDirPath)
) )
@ -154,7 +194,19 @@ func NewDaemon(
return nil, fmt.Errorf("initializing daemon directories: %w", err) return nil, fmt.Errorf("initializing daemon directories: %w", err)
} }
currBootstrap, err := bootstrap.FromFile(bootstrapFilePath) var (
secretsPath = filepath.Join(d.opts.EnvVars.StateDirPath, "secrets")
err error
)
if d.secretsStore, err = secrets.NewFSStore(secretsPath); err != nil {
return nil, fmt.Errorf(
"initializing secrets store at %q: %w", secretsPath, err,
)
}
var currBootstrap bootstrap.Bootstrap
err = jsonutil.LoadFile(&currBootstrap, bootstrapFilePath)
if errors.Is(err, fs.ErrNotExist) { if errors.Is(err, fs.ErrNotExist) {
// daemon has never had a network created or joined // daemon has never had a network created or joined
} else if err != nil { } else if err != nil {
@ -249,7 +301,7 @@ func (d *daemon) checkBootstrap(
thisHost := hostBootstrap.ThisHost() thisHost := hostBootstrap.ThisHost()
newHosts, err := getGarageBootstrapHosts(ctx, d.logger, hostBootstrap) newHosts, err := d.getGarageBootstrapHosts(ctx, d.logger, hostBootstrap)
if err != nil { if err != nil {
return bootstrap.Bootstrap{}, false, fmt.Errorf("getting hosts from garage: %w", err) return bootstrap.Bootstrap{}, false, fmt.Errorf("getting hosts from garage: %w", err)
} }
@ -321,8 +373,12 @@ func (d *daemon) postInit(ctx context.Context) bool {
d.logger, d.logger,
"Applying garage layout", "Applying garage layout",
func(ctx context.Context) error { func(ctx context.Context) error {
return GarageApplyLayout( return garageApplyLayout(
ctx, d.logger, d.daemonConfig, d.currBootstrap, ctx,
d.logger,
d.daemonConfig,
d.garageAdminToken,
d.currBootstrap,
) )
}, },
) { ) {
@ -335,28 +391,29 @@ func (d *daemon) postInit(ctx context.Context) bool {
// //
// TODO this is pretty hacky, but there doesn't seem to be a better way to // TODO this is pretty hacky, but there doesn't seem to be a better way to
// manage it at the moment. // manage it at the moment.
if d.currBootstrap.Garage.GlobalBucketS3APICredentials == (garage.S3APICredentials{}) { _, err := getGarageS3APIGlobalBucketCredentials(ctx, d.secretsStore)
currBootstrap := d.currBootstrap if errors.Is(err, secrets.ErrNotFound) {
if !until( if !until(
ctx, ctx,
d.logger, d.logger,
"Initializing garage shared global bucket", "Initializing garage shared global bucket",
func(ctx context.Context) error { func(ctx context.Context) error {
garageGlobalBucketCreds, err := garageInitializeGlobalBucket( garageGlobalBucketCreds, err := garageInitializeGlobalBucket(
ctx, d.logger, d.daemonConfig, d.currBootstrap, ctx,
d.logger,
d.daemonConfig,
d.garageAdminToken,
d.currBootstrap,
) )
if err != nil { if err != nil {
return fmt.Errorf("initializing global bucket: %w", err) return fmt.Errorf("initializing global bucket: %w", err)
} }
currBootstrap.Garage.GlobalBucketS3APICredentials = garageGlobalBucketCreds err = setGarageS3APIGlobalBucketCredentials(
ctx, d.secretsStore, garageGlobalBucketCreds,
d.logger.Info(ctx, "Writing bootstrap to state directory")
err = writeBootstrapToStateDir(
d.opts.EnvVars.StateDirPath, currBootstrap,
) )
if err != nil { if err != nil {
return fmt.Errorf("writing bootstrap to state dir: %w", err) return fmt.Errorf("storing global bucket creds: %w", err)
} }
return nil return nil
@ -364,10 +421,6 @@ func (d *daemon) postInit(ctx context.Context) bool {
) { ) {
return false return false
} }
d.l.Lock()
d.currBootstrap = currBootstrap
d.l.Unlock()
} }
if !until( if !until(
@ -375,7 +428,7 @@ func (d *daemon) postInit(ctx context.Context) bool {
d.logger, d.logger,
"Updating host info in garage", "Updating host info in garage",
func(ctx context.Context) error { func(ctx context.Context) error {
return putGarageBoostrapHost(ctx, d.logger, d.currBootstrap) return d.putGarageBoostrapHost(ctx, d.logger, d.currBootstrap)
}, },
) { ) {
return false return false
@ -410,15 +463,17 @@ func (d *daemon) restartLoop(ctx context.Context, readyCh chan<- struct{}) {
children, err := NewChildren( children, err := NewChildren(
ctx, ctx,
d.logger.WithNamespace("children"), d.logger.WithNamespace("children"),
d.daemonConfig,
d.currBootstrap,
d.envBinDirPath, d.envBinDirPath,
d.secretsStore,
d.daemonConfig,
d.garageAdminToken,
d.currBootstrap,
d.opts, d.opts,
) )
if errors.Is(err, context.Canceled) { if errors.Is(err, context.Canceled) {
return return
} else if err != nil { } else if err != nil {
d.logger.Error(ctx, "failed to initialize daemon", err) d.logger.Error(ctx, "failed to initialize child processes", err)
if !wait(1 * time.Second) { if !wait(1 * time.Second) {
return return
} }
@ -467,48 +522,52 @@ func (d *daemon) restartLoop(ctx context.Context, readyCh chan<- struct{}) {
func (d *daemon) CreateNetwork( func (d *daemon) CreateNetwork(
ctx context.Context, ctx context.Context,
name, domain string, ipNet nebula.IPNet, hostName nebula.HostName, name, domain string, ipNet nebula.IPNet, hostName nebula.HostName,
) ( ) error {
admin.Admin, error,
) {
nebulaCACreds, err := nebula.NewCACredentials(domain, ipNet) nebulaCACreds, err := nebula.NewCACredentials(domain, ipNet)
if err != nil { if err != nil {
return admin.Admin{}, fmt.Errorf("creating nebula CA cert: %w", err) return fmt.Errorf("creating nebula CA cert: %w", err)
} }
adm := admin.Admin{ var (
CreationParams: admin.CreationParams{ creationParams = bootstrap.CreationParams{
ID: randStr(32), ID: randStr(32),
Name: name, Name: name,
Domain: domain, Domain: domain,
}, }
garageRPCSecret = randStr(32)
)
err = setGarageRPCSecret(ctx, d.secretsStore, garageRPCSecret)
if err != nil {
return fmt.Errorf("setting garage RPC secret: %w", err)
} }
garageBootstrap := bootstrap.Garage{ err = setNebulaCASigningPrivateKey(ctx, d.secretsStore, nebulaCACreds.SigningPrivateKey)
RPCSecret: randStr(32), if err != nil {
AdminToken: randStr(32), return fmt.Errorf("setting nebula CA signing key secret: %w", err)
} }
hostBootstrap, err := bootstrap.New( hostBootstrap, err := bootstrap.New(
nebulaCACreds, nebulaCACreds,
adm.CreationParams, creationParams,
garageBootstrap,
hostName, hostName,
ipNet.FirstAddr(), ipNet.FirstAddr(),
) )
if err != nil { if err != nil {
return adm, fmt.Errorf("initializing bootstrap data: %w", err) return fmt.Errorf("initializing bootstrap data: %w", err)
} }
d.l.Lock() d.l.Lock()
if d.state != daemonStateNoNetwork { if d.state != daemonStateNoNetwork {
d.l.Unlock() d.l.Unlock()
return adm, ErrAlreadyJoined return ErrAlreadyJoined
} }
if len(d.daemonConfig.Storage.Allocations) < 3 { if len(d.daemonConfig.Storage.Allocations) < 3 {
d.l.Unlock() d.l.Unlock()
return adm, ErrInvalidConfig.WithData( return ErrInvalidConfig.WithData(
"At least three storage allocations are required.", "At least three storage allocations are required.",
) )
} }
@ -518,32 +577,20 @@ func (d *daemon) CreateNetwork(
err = d.initialize(hostBootstrap, readyCh) err = d.initialize(hostBootstrap, readyCh)
d.l.Unlock() d.l.Unlock()
if err != nil { if err != nil {
return adm, fmt.Errorf("initializing daemon: %w", err) return fmt.Errorf("initializing daemon: %w", err)
} }
select { select {
case <-readyCh: case <-readyCh:
case <-ctx.Done(): case <-ctx.Done():
return adm, ctx.Err() return ctx.Err()
} }
// As part of postInit, which is called prior to ready(), the restartLoop return nil
// will check if the global bucket creds have been created yet or not, and
// create them if so. So once ready() is called we can get the created creds
// from the currBootstrap
d.l.RLock()
garageGlobalBucketCreds := d.currBootstrap.Garage.GlobalBucketS3APICredentials
d.l.RUnlock()
adm.Nebula.CACredentials = nebulaCACreds
adm.Garage.RPCSecret = hostBootstrap.Garage.RPCSecret
adm.Garage.GlobalBucketS3APICredentials = garageGlobalBucketCreds
return adm, nil
} }
func (d *daemon) JoinNetwork( func (d *daemon) JoinNetwork(
ctx context.Context, newBootstrap bootstrap.Bootstrap, ctx context.Context, newBootstrap JoiningBootstrap,
) error { ) error {
d.l.Lock() d.l.Lock()
@ -554,7 +601,13 @@ func (d *daemon) JoinNetwork(
readyCh := make(chan struct{}, 1) readyCh := make(chan struct{}, 1)
err := d.initialize(newBootstrap, readyCh) err := secrets.Import(ctx, d.secretsStore, newBootstrap.Secrets)
if err != nil {
d.l.Unlock()
return fmt.Errorf("importing secrets: %w", err)
}
err = d.initialize(newBootstrap.Bootstrap, readyCh)
d.l.Unlock() d.l.Unlock()
if err != nil { if err != nil {
return fmt.Errorf("initializing daemon: %w", err) return fmt.Errorf("initializing daemon: %w", err)
@ -578,6 +631,20 @@ func (d *daemon) GetBootstrap(ctx context.Context) (bootstrap.Bootstrap, error)
}) })
} }
func (d *daemon) GetGarageClientParams(
ctx context.Context,
) (
GarageClientParams, error,
) {
return withCurrBootstrap(d, func(
currBootstrap bootstrap.Bootstrap,
) (
GarageClientParams, error,
) {
return d.getGarageClientParams(ctx, currBootstrap)
})
}
func (d *daemon) RemoveHost(ctx context.Context, hostName nebula.HostName) error { func (d *daemon) RemoveHost(ctx context.Context, hostName nebula.HostName) error {
// TODO RemoveHost should publish a certificate revocation for the host // TODO RemoveHost should publish a certificate revocation for the host
// being removed. // being removed.
@ -586,12 +653,114 @@ func (d *daemon) RemoveHost(ctx context.Context, hostName nebula.HostName) error
) ( ) (
struct{}, error, struct{}, error,
) { ) {
client := currBootstrap.GarageClientParams().GlobalBucketS3APIClient() garageClientParams, err := d.getGarageClientParams(ctx, currBootstrap)
if err != nil {
return struct{}{}, fmt.Errorf("get garage client params: %w", err)
}
client := garageClientParams.GlobalBucketS3APIClient()
return struct{}{}, removeGarageBootstrapHost(ctx, client, hostName) return struct{}{}, removeGarageBootstrapHost(ctx, client, hostName)
}) })
return err return err
} }
func makeCACreds(
currBootstrap bootstrap.Bootstrap,
caSigningPrivateKey nebula.SigningPrivateKey,
) nebula.CACredentials {
return nebula.CACredentials{
Public: currBootstrap.CAPublicCredentials,
SigningPrivateKey: caSigningPrivateKey,
}
}
func (d *daemon) CreateHost(
ctx context.Context,
hostName nebula.HostName,
ip netip.Addr,
opts CreateHostOpts,
) (
JoiningBootstrap, error,
) {
return withCurrBootstrap(d, func(
currBootstrap bootstrap.Bootstrap,
) (
JoiningBootstrap, error,
) {
caSigningPrivateKey, err := getNebulaCASigningPrivateKey(
ctx, d.secretsStore,
)
if err != nil {
return JoiningBootstrap{}, fmt.Errorf("getting CA signing key: %w", err)
}
var joiningBootstrap JoiningBootstrap
joiningBootstrap.Bootstrap, err = bootstrap.New(
makeCACreds(currBootstrap, caSigningPrivateKey),
currBootstrap.NetworkCreationParams,
hostName,
ip,
)
if err != nil {
return JoiningBootstrap{}, fmt.Errorf(
"initializing bootstrap data: %w", err,
)
}
joiningBootstrap.Bootstrap.Hosts = currBootstrap.Hosts
secretsIDs := []secrets.ID{
garageRPCSecretSecretID,
garageS3APIGlobalBucketCredentialsSecretID,
}
if opts.CanCreateHosts {
secretsIDs = append(secretsIDs, nebulaCASigningPrivateKeySecretID)
}
if joiningBootstrap.Secrets, err = secrets.Export(
ctx, d.secretsStore, secretsIDs,
); err != nil {
return JoiningBootstrap{}, fmt.Errorf("exporting secrets: %w", err)
}
// TODO persist new bootstrap to garage. Requires making the daemon
// config change watching logic smarter, so only dnsmasq gets restarted.
return joiningBootstrap, nil
})
}
func (d *daemon) CreateNebulaCertificate(
ctx context.Context,
hostName nebula.HostName,
hostPubKey nebula.EncryptingPublicKey,
) (
nebula.Certificate, error,
) {
return withCurrBootstrap(d, func(
currBootstrap bootstrap.Bootstrap,
) (
nebula.Certificate, error,
) {
host, ok := currBootstrap.Hosts[hostName]
if !ok {
return nebula.Certificate{}, ErrHostNotFound
}
caSigningPrivateKey, err := getNebulaCASigningPrivateKey(
ctx, d.secretsStore,
)
if err != nil {
return nebula.Certificate{}, fmt.Errorf("getting CA signing key: %w", err)
}
caCreds := makeCACreds(currBootstrap, caSigningPrivateKey)
return nebula.NewHostCert(caCreds, hostPubKey, hostName, host.IP())
})
}
func (d *daemon) Shutdown() error { func (d *daemon) Shutdown() error {
d.l.Lock() d.l.Lock()
defer d.l.Unlock() defer d.l.Unlock()

View File

@ -22,34 +22,6 @@ type EnvVars struct {
func (e EnvVars) init() error { func (e EnvVars) init() error {
var errs []error var errs []error
mkDir := func(path string) error {
{
parentPath := filepath.Dir(path)
parentInfo, err := os.Stat(parentPath)
if err != nil {
return fmt.Errorf("checking parent path %q: %w", parentPath, err)
} else if !parentInfo.IsDir() {
return fmt.Errorf("%q is not a directory", parentPath)
}
}
info, err := os.Stat(path)
if errors.Is(err, fs.ErrNotExist) {
// fine
} else if err != nil {
return fmt.Errorf("checking path: %w", err)
} else if !info.IsDir() {
return fmt.Errorf("path is not a directory")
} else {
return nil
}
if err := os.Mkdir(path, 0700); err != nil {
return fmt.Errorf("creating directory: %w", err)
}
return nil
}
if err := mkDir(e.RuntimeDirPath); err != nil { if err := mkDir(e.RuntimeDirPath); err != nil {
errs = append(errs, fmt.Errorf( errs = append(errs, fmt.Errorf(
"creating runtime directory %q: %w", "creating runtime directory %q: %w",

View File

@ -24,4 +24,8 @@ var (
// //
// The Data field will be a string containing further details. // The Data field will be a string containing further details.
ErrInvalidConfig = jsonrpc2.NewError(5, "Invalid daemon config") ErrInvalidConfig = jsonrpc2.NewError(5, "Invalid daemon config")
// ErrHostNotFound is returned when performing an operation which expected a
// host to exist in the network, but that host wasn't found.
ErrHostNotFound = jsonrpc2.NewError(6, "Host not found")
) )

View File

@ -0,0 +1,52 @@
package daemon
import (
"context"
"errors"
"fmt"
"isle/bootstrap"
"isle/garage"
"isle/secrets"
)
// GarageClientParams contains all the data needed to instantiate garage
// clients.
type GarageClientParams struct {
Peer garage.RemotePeer
GlobalBucketS3APICredentials garage.S3APICredentials
// RPCSecret may be empty, if the secret is not available on the host.
RPCSecret string
}
func (d *daemon) getGarageClientParams(
ctx context.Context, currBootstrap bootstrap.Bootstrap,
) (
GarageClientParams, error,
) {
creds, err := getGarageS3APIGlobalBucketCredentials(ctx, d.secretsStore)
if err != nil {
return GarageClientParams{}, fmt.Errorf("getting garage global bucket creds: %w", err)
}
rpcSecret, err := getGarageRPCSecret(ctx, d.secretsStore)
if err != nil && !errors.Is(err, secrets.ErrNotFound) {
return GarageClientParams{}, fmt.Errorf("getting garage rpc secret: %w", err)
}
return GarageClientParams{
Peer: currBootstrap.ChooseGaragePeer(),
GlobalBucketS3APICredentials: creds,
RPCSecret: rpcSecret,
}, nil
}
// GlobalBucketS3APIClient returns an S3 client pre-configured with access to
// the global bucket.
func (p GarageClientParams) GlobalBucketS3APIClient() garage.S3APIClient {
var (
addr = p.Peer.S3APIAddr()
creds = p.GlobalBucketS3APICredentials
)
return garage.NewS3APIClient(addr, creds)
}

View File

@ -24,12 +24,13 @@ func garageInitializeGlobalBucket(
ctx context.Context, ctx context.Context,
logger *mlog.Logger, logger *mlog.Logger,
daemonConfig Config, daemonConfig Config,
adminToken string,
hostBootstrap bootstrap.Bootstrap, hostBootstrap bootstrap.Bootstrap,
) ( ) (
garage.S3APICredentials, error, garage.S3APICredentials, error,
) { ) {
adminClient := NewGarageAdminClient( adminClient := newGarageAdminClient(
logger, daemonConfig, hostBootstrap, logger, daemonConfig, adminToken, hostBootstrap,
) )
creds, err := adminClient.CreateS3APICredentials( creds, err := adminClient.CreateS3APICredentials(
@ -62,24 +63,28 @@ func garageInitializeGlobalBucket(
// putGarageBoostrapHost places the <hostname>.json.signed file for this host // putGarageBoostrapHost places the <hostname>.json.signed file for this host
// into garage so that other hosts are able to see relevant configuration for // into garage so that other hosts are able to see relevant configuration for
// it. // it.
func putGarageBoostrapHost( func (d *daemon) putGarageBoostrapHost(
ctx context.Context, logger *mlog.Logger, currBootstrap bootstrap.Bootstrap, ctx context.Context, logger *mlog.Logger, currBootstrap bootstrap.Bootstrap,
) error { ) error {
garageClientParams, err := d.getGarageClientParams(ctx, currBootstrap)
if err != nil {
return fmt.Errorf("getting garage client params: %w", err)
}
var ( var (
b = currBootstrap host = currBootstrap.ThisHost()
host = b.ThisHost() client = garageClientParams.GlobalBucketS3APIClient()
client = b.GarageClientParams().GlobalBucketS3APIClient()
) )
configured, err := nebula.Sign( configured, err := nebula.Sign(
host.HostConfigured, b.PrivateCredentials.SigningPrivateKey, host.HostConfigured, currBootstrap.PrivateCredentials.SigningPrivateKey,
) )
if err != nil { if err != nil {
return fmt.Errorf("signing host configured data: %w", err) return fmt.Errorf("signing host configured data: %w", err)
} }
hostB, err := json.Marshal(bootstrap.AuthenticatedHost{ hostB, err := json.Marshal(bootstrap.AuthenticatedHost{
Assigned: b.SignedHostAssigned, Assigned: currBootstrap.SignedHostAssigned,
Configured: configured, Configured: configured,
}) })
if err != nil { if err != nil {
@ -107,14 +112,18 @@ func putGarageBoostrapHost(
return nil return nil
} }
func getGarageBootstrapHosts( func (d *daemon) getGarageBootstrapHosts(
ctx context.Context, logger *mlog.Logger, currBootstrap bootstrap.Bootstrap, ctx context.Context, logger *mlog.Logger, currBootstrap bootstrap.Bootstrap,
) ( ) (
map[nebula.HostName]bootstrap.Host, error, map[nebula.HostName]bootstrap.Host, error,
) { ) {
garageClientParams, err := d.getGarageClientParams(ctx, currBootstrap)
if err != nil {
return nil, fmt.Errorf("getting garage client params: %w", err)
}
var ( var (
b = currBootstrap client = garageClientParams.GlobalBucketS3APIClient()
client = b.GarageClientParams().GlobalBucketS3APIClient()
hosts = map[nebula.HostName]bootstrap.Host{} hosts = map[nebula.HostName]bootstrap.Host{}
objInfoCh = client.ListObjects( objInfoCh = client.ListObjects(
@ -152,7 +161,7 @@ func getGarageBootstrapHosts(
continue continue
} }
host, err := authedHost.Unwrap(b.CAPublicCredentials) host, err := authedHost.Unwrap(currBootstrap.CAPublicCredentials)
if err != nil { if err != nil {
logger.Warn(ctx, "Host could not be authenticated", err) logger.Warn(ctx, "Host could not be authenticated", err)
} }

View File

@ -4,6 +4,11 @@ import (
"context" "context"
"crypto/rand" "crypto/rand"
"encoding/hex" "encoding/hex"
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"time" "time"
"dev.mediocregopher.com/mediocre-go-lib.git/mlog" "dev.mediocregopher.com/mediocre-go-lib.git/mlog"
@ -38,3 +43,34 @@ func randStr(l int) string {
} }
return hex.EncodeToString(b) return hex.EncodeToString(b)
} }
// mkDir is like os.Mkdir but it returns better error messages. If the directory
// already exists then nil is returned.
func mkDir(path string) error {
{
parentPath := filepath.Dir(path)
parentInfo, err := os.Stat(parentPath)
if err != nil {
return fmt.Errorf("checking fs node of parent %q: %w", parentPath, err)
} else if !parentInfo.IsDir() {
return fmt.Errorf("%q is not a directory", parentPath)
}
}
info, err := os.Stat(path)
if errors.Is(err, fs.ErrNotExist) {
// fine
} else if err != nil {
return fmt.Errorf("checking fs node: %w", err)
} else if !info.IsDir() {
return fmt.Errorf("exists but is not a directory")
} else {
return nil
}
if err := os.Mkdir(path, 0700); err != nil {
return fmt.Errorf("creating directory: %w", err)
}
return nil
}

View File

@ -4,9 +4,9 @@ import (
"cmp" "cmp"
"context" "context"
"fmt" "fmt"
"isle/admin"
"isle/bootstrap" "isle/bootstrap"
"isle/nebula" "isle/nebula"
"net/netip"
"slices" "slices"
"golang.org/x/exp/maps" "golang.org/x/exp/maps"
@ -46,16 +46,16 @@ type CreateNetworkRequest struct {
func (r *RPC) CreateNetwork( func (r *RPC) CreateNetwork(
ctx context.Context, req CreateNetworkRequest, ctx context.Context, req CreateNetworkRequest,
) ( ) (
admin.Admin, error, struct{}, error,
) { ) {
return r.daemon.CreateNetwork( return struct{}{}, r.daemon.CreateNetwork(
ctx, req.Name, req.Domain, req.IPNet, req.HostName, ctx, req.Name, req.Domain, req.IPNet, req.HostName,
) )
} }
// JoinNetwork passes through to the Daemon method of the same name. // JoinNetwork passes through to the Daemon method of the same name.
func (r *RPC) JoinNetwork( func (r *RPC) JoinNetwork(
ctx context.Context, req bootstrap.Bootstrap, ctx context.Context, req JoiningBootstrap,
) ( ) (
struct{}, error, struct{}, error,
) { ) {
@ -86,21 +86,14 @@ func (r *RPC) GetHosts(
return GetHostsResult{hosts}, nil return GetHostsResult{hosts}, nil
} }
// GetGarageClientParams returns a GarageClientParams which can be used to // GetGarageClientParams passes the call through to the Daemon method of the
// interact with garage. // same name.
func (r *RPC) GetGarageClientParams( func (r *RPC) GetGarageClientParams(
ctx context.Context, req struct{}, ctx context.Context, req struct{},
) ( ) (
bootstrap.GarageClientParams, error, GarageClientParams, error,
) { ) {
b, err := r.daemon.GetBootstrap(ctx) return r.daemon.GetGarageClientParams(ctx)
if err != nil {
return bootstrap.GarageClientParams{}, fmt.Errorf(
"retrieving bootstrap: %w", err,
)
}
return b.GarageClientParams(), nil
} }
// GetNebulaCAPublicCredentials returns the CAPublicCredentials for the network. // GetNebulaCAPublicCredentials returns the CAPublicCredentials for the network.
@ -119,7 +112,7 @@ func (r *RPC) GetNebulaCAPublicCredentials(
return b.CAPublicCredentials, nil return b.CAPublicCredentials, nil
} }
// RemoveHostRequest contains the arguments to the RemoveHost method. // RemoveHostRequest contains the arguments to the RemoveHost RPC method.
// //
// All fields are required. // All fields are required.
type RemoveHostRequest struct { type RemoveHostRequest struct {
@ -130,3 +123,69 @@ type RemoveHostRequest struct {
func (r *RPC) RemoveHost(ctx context.Context, req RemoveHostRequest) (struct{}, error) { func (r *RPC) RemoveHost(ctx context.Context, req RemoveHostRequest) (struct{}, error) {
return struct{}{}, r.daemon.RemoveHost(ctx, req.HostName) return struct{}{}, r.daemon.RemoveHost(ctx, req.HostName)
} }
// CreateHostRequest contains the arguments to the
// CreateHost RPC method.
//
// All fields are required.
type CreateHostRequest struct {
HostName nebula.HostName
IP netip.Addr
Opts CreateHostOpts
}
// CreateHostResult wraps the results from the CreateHost RPC method.
type CreateHostResult struct {
JoiningBootstrap JoiningBootstrap
}
// CreateHost passes the call through to the Daemon method of the
// same name.
func (r *RPC) CreateHost(
ctx context.Context, req CreateHostRequest,
) (
CreateHostResult, error,
) {
joiningBootstrap, err := r.daemon.CreateHost(
ctx, req.HostName, req.IP, req.Opts,
)
if err != nil {
return CreateHostResult{}, err
}
return CreateHostResult{JoiningBootstrap: joiningBootstrap}, nil
}
// CreateNebulaCertificateRequest contains the arguments to the
// CreateNebulaCertificate RPC method.
//
// All fields are required.
type CreateNebulaCertificateRequest struct {
HostName nebula.HostName
HostEncryptingPublicKey nebula.EncryptingPublicKey
}
// CreateNebulaCertificateResult wraps the results from the
// CreateNebulaCertificate RPC method.
type CreateNebulaCertificateResult struct {
HostNebulaCertifcate nebula.Certificate
}
// CreateNebulaCertificate passes the call through to the Daemon method of the
// same name.
func (r *RPC) CreateNebulaCertificate(
ctx context.Context, req CreateNebulaCertificateRequest,
) (
CreateNebulaCertificateResult, error,
) {
cert, err := r.daemon.CreateNebulaCertificate(
ctx, req.HostName, req.HostEncryptingPublicKey,
)
if err != nil {
return CreateNebulaCertificateResult{}, err
}
return CreateNebulaCertificateResult{
HostNebulaCertifcate: cert,
}, nil
}

52
go/daemon/secrets.go Normal file
View File

@ -0,0 +1,52 @@
package daemon
import (
"fmt"
"isle/garage"
"isle/nebula"
"isle/secrets"
)
const (
secretsNSNebula = "nebula"
secretsNSGarage = "garage"
)
////////////////////////////////////////////////////////////////////////////////
// Nebula-related secrets
var (
nebulaCASigningPrivateKeySecretID = secrets.NewID(secretsNSNebula, "ca-signing-private-key")
)
var getNebulaCASigningPrivateKey, setNebulaCASigningPrivateKey = secrets.GetSetFunctions[nebula.SigningPrivateKey](
nebulaCASigningPrivateKeySecretID,
)
////////////////////////////////////////////////////////////////////////////////
// Garage-related secrets
func garageS3APIBucketCredentialsSecretID(credsName string) secrets.ID {
return secrets.NewID(
secretsNSGarage, fmt.Sprintf("s3-api-bucket-credentials-%s", credsName),
)
}
var (
garageRPCSecretSecretID = secrets.NewID(secretsNSGarage, "rpc-secret")
garageS3APIGlobalBucketCredentialsSecretID = garageS3APIBucketCredentialsSecretID(
garage.GlobalBucketS3APICredentialsName,
)
)
// Get/Set functions for garage-related secrets.
var (
getGarageRPCSecret, setGarageRPCSecret = secrets.GetSetFunctions[string](
garageRPCSecretSecretID,
)
getGarageS3APIGlobalBucketCredentials,
setGarageS3APIGlobalBucketCredentials = secrets.GetSetFunctions[garage.S3APICredentials](
garageS3APIGlobalBucketCredentialsSecretID,
)
)

View File

@ -11,6 +11,8 @@ const (
// accessible to all hosts in the network. // accessible to all hosts in the network.
GlobalBucket = "global-shared" GlobalBucket = "global-shared"
// GlobalBucketS3APICredentialsName is the main alias of the shared API key
// used to write to the global bucket.
GlobalBucketS3APICredentialsName = "global-shared-key" GlobalBucketS3APICredentialsName = "global-shared-key"
// ReplicationFactor indicates the replication factor set on the garage // ReplicationFactor indicates the replication factor set on the garage

View File

@ -46,7 +46,7 @@ func (pk *EncryptingPublicKey) UnmarshalText(b []byte) error {
// UnmarshalNebulaPEM unmarshals the EncryptingPublicKey as a nebula host public // UnmarshalNebulaPEM unmarshals the EncryptingPublicKey as a nebula host public
// key PEM. // key PEM.
func (pk *EncryptingPublicKey) UnmarshalNebulaPEM(b []byte) error { func (pk *EncryptingPublicKey) UnmarshalNebulaPEM(b []byte) error {
b, _, err := cert.UnmarshalEd25519PublicKey(b) b, _, err := cert.UnmarshalX25519PublicKey(b)
if err != nil { if err != nil {
return fmt.Errorf("unmarshaling: %w", err) return fmt.Errorf("unmarshaling: %w", err)
} }

13
go/secrets/secrets.go Normal file
View File

@ -0,0 +1,13 @@
// Package secrets manages the storage and distributions of secret values that
// hosts need to perform various actions.
package secrets
import "fmt"
// ID is a unique identifier for a Secret.
type ID string
// NewID returns a new ID within the given namespace.
func NewID(namespace, id string) ID {
return ID(fmt.Sprintf("%s-%s", namespace, id))
}

105
go/secrets/store.go Normal file
View File

@ -0,0 +1,105 @@
package secrets
import (
"context"
"encoding/json"
"errors"
"fmt"
)
// ErrNotFound is returned when an ID could not be found.
var ErrNotFound = errors.New("not found")
// Store is used to persist and retrieve secrets. If a Store serializes a
// payload it will do so using JSON.
type Store interface {
// Set stores the secret payload of the given ID.
Set(context.Context, ID, any) error
// Get retrieves the secret of the given ID, setting it into the given
// pointer value, or returns ErrNotFound.
Get(context.Context, any, ID) error
}
// GetSetFunctions returns a Get/Set function pair for the given ID and payload
// type.
func GetSetFunctions[T any](
id ID,
) (
func(context.Context, Store) (T, error), // Get
func(context.Context, Store, T) error, // Set
) {
var (
get = func(ctx context.Context, store Store) (T, error) {
var v T
err := store.Get(ctx, &v, id)
return v, err
}
set = func(ctx context.Context, store Store, v T) error {
return store.Set(ctx, id, v)
}
)
return get, set
}
// MultiSet will call Set on the given Store for every key-value pair in the
// given map.
func MultiSet(ctx context.Context, s Store, m map[ID]any) error {
var errs []error
for id, payload := range m {
if err := s.Set(ctx, id, payload); err != nil {
errs = append(errs, fmt.Errorf("setting payload for %q: %w", id, err))
}
}
return errors.Join(errs...)
}
// MultiGet will call Get on the given Store for every key-value pair in the
// given map. Each value in the map must be a pointer receiver.
func MultiGet(ctx context.Context, s Store, m map[ID]any) error {
var errs []error
for id, into := range m {
if err := s.Get(ctx, into, id); err != nil {
errs = append(errs, fmt.Errorf("getting payload for %q: %w", id, err))
}
}
return errors.Join(errs...)
}
// Export returns a map of ID to raw payload for each ID given. An error is
// returned for _each_ ID which could not be exported, wrapped using
// `errors.Join`, alongside whatever keys could be exported.
func Export(
ctx context.Context, s Store, ids []ID,
) (
map[ID]json.RawMessage, error,
) {
var (
m = map[ID]json.RawMessage{}
errs []error
)
for _, id := range ids {
var into json.RawMessage
if err := s.Get(ctx, &into, id); err != nil {
errs = append(errs, fmt.Errorf("exporting %q: %w", id, err))
continue
}
m[id] = into
}
return m, errors.Join(errs...)
}
// Import sets all given ID/payload pairs into the Store.
func Import(
ctx context.Context, s Store, m map[ID]json.RawMessage,
) error {
var errs []error
for id, payload := range m {
if err := s.Set(ctx, id, payload); err != nil {
errs = append(errs, fmt.Errorf("importing %q: %w", id, err))
}
}
return errors.Join(errs...)
}

83
go/secrets/store_fs.go Normal file
View File

@ -0,0 +1,83 @@
package secrets
import (
"context"
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
)
type fsStore struct {
dirPath string
}
type fsStorePayload[Body any] struct {
Version int
Body Body
}
// NewFSStore returns a Store which will store secrets to the given directory.
func NewFSStore(dirPath string) (Store, error) {
err := os.Mkdir(dirPath, 0700)
if err != nil && !errors.Is(err, fs.ErrExist) {
return nil, fmt.Errorf("making directory: %w", err)
}
return &fsStore{dirPath}, nil
}
func (s *fsStore) path(id ID) string {
return filepath.Join(s.dirPath, string(id))
}
func (s *fsStore) Set(_ context.Context, id ID, payload any) error {
path := s.path(id)
f, err := os.Create(path)
if err != nil {
return fmt.Errorf("creating file %q: %w", path, err)
}
defer f.Close()
if err := json.NewEncoder(f).Encode(fsStorePayload[any]{
Version: 1,
Body: payload,
}); err != nil {
return fmt.Errorf("writing JSON encoded payload to %q: %w", path, err)
}
return nil
}
func (s *fsStore) Get(_ context.Context, into any, id ID) error {
path := s.path(id)
f, err := os.Open(path)
if errors.Is(err, fs.ErrNotExist) {
return ErrNotFound
} else if err != nil {
return fmt.Errorf("creating file %q: %w", path, err)
}
defer f.Close()
var fullPayload fsStorePayload[json.RawMessage]
if err := json.NewDecoder(f).Decode(&fullPayload); err != nil {
return fmt.Errorf("decoding JSON payload from %q: %w", path, err)
}
if fullPayload.Version != 1 {
return fmt.Errorf(
"unexpected JSON payload version %d", fullPayload.Version,
)
}
if err := json.Unmarshal(fullPayload.Body, into); err != nil {
return fmt.Errorf(
"decoding JSON payload body from %q into %T: %w", path, into, err,
)
}
return nil
}

View File

@ -0,0 +1,42 @@
package secrets
import (
"context"
"errors"
"testing"
)
func Test_fsStore(t *testing.T) {
type payload struct {
Foo int
}
var (
ctx = context.Background()
dir = t.TempDir()
id = NewID("testing", "a")
)
store, err := NewFSStore(dir)
if err != nil {
t.Fatal(err)
}
var got payload
if err := store.Get(ctx, &got, id); !errors.Is(err, ErrNotFound) {
t.Fatalf("expected %v, got: %v", ErrNotFound, err)
}
want := payload{Foo: 5}
if err := store.Set(ctx, id, want); err != nil {
t.Fatal(err)
}
if err := store.Get(ctx, &got, id); err != nil {
t.Fatal(err)
}
if want != got {
t.Fatalf("wanted %+v, got: %+v", want, got)
}
}

View File

@ -1,16 +0,0 @@
# shellcheck source=../../utils/with-1-data-1-empty-node-network.sh
source "$UTILS"/with-1-data-1-empty-node-network.sh
[ "$(cat a/meta/isle/rpc_port)" = "3900" ]
[ "$(cat b/meta/isle/rpc_port)" = "3910" ]
[ "$(cat c/meta/isle/rpc_port)" = "3920" ]
[ "$(jq -r <admin.json '.CreationParams.ID')" != "" ]
[ "$(jq -r <admin.json '.CreationParams.Name')" = "testing" ]
[ "$(jq -r <admin.json '.CreationParams.Domain')" = "shared.test" ]
bootstrap_file="$XDG_STATE_HOME/isle/bootstrap.json"
[ "$(jq -rc <"$bootstrap_file" '.AdminCreationParams')" = "$(jq -rc <admin.json '.CreationParams')" ]
[ "$(jq -rc <"$bootstrap_file" '.CAPublicCredentials')" = "$(jq -rc <admin.json '.Nebula.CACredentials.Public')" ]
[ "$(jq -r <"$bootstrap_file" '.SignedHostAssigned.Body.Name')" = "primus" ]

View File

@ -1,14 +0,0 @@
# shellcheck source=../../utils/with-1-data-1-empty-node-network.sh
source "$UTILS"/with-1-data-1-empty-node-network.sh
adminBS="$XDG_STATE_HOME"/isle/bootstrap.json
bs="$secondus_bootstrap" # set in with-1-data-1-empty-node-network.sh
[ "$(jq -r <"$bs" '.AdminCreationParams')" = "$(jq -r <admin.json '.CreationParams')" ]
[ "$(jq -r <"$bs" '.SignedHostAssigned.Body.Name')" = "secondus" ]
[ "$(jq -r <"$bs" '.Hosts.primus.PublicCredentials')" \
= "$(jq -r <"$adminBS" '.SignedHostAssigned.Body.PublicCredentials')" ]
[ "$(jq <"$bs" '.Hosts.primus.Garage.Instances|length')" = "3" ]

View File

@ -0,0 +1,17 @@
# shellcheck source=../../utils/with-1-data-1-empty-node-network.sh
source "$UTILS"/with-1-data-1-empty-node-network.sh
adminBS="$XDG_STATE_HOME"/isle/bootstrap.json
bs="$secondus_bootstrap" # set in with-1-data-1-empty-node-network.sh
[ "$(jq -r <"$bs" '.Bootstrap.NetworkCreationParams.Domain')" = "shared.test" ]
[ "$(jq -r <"$bs" '.Bootstrap.NetworkCreationParams.Name')" = "testing" ]
[ "$(jq -r <"$bs" '.Bootstrap.SignedHostAssigned.Body.Name')" = "secondus" ]
[ "$(jq -r <"$bs" '.Bootstrap.Hosts.primus.PublicCredentials')" \
= "$(jq -r <"$adminBS" '.SignedHostAssigned.Body.PublicCredentials')" ]
[ "$(jq <"$bs" '.Bootstrap.Hosts.primus.Garage.Instances|length')" = "3" ]
[ "$(jq <"$bs" '.Secrets["garage-rpc-secret"]')" != "null" ]

View File

@ -0,0 +1,12 @@
# shellcheck source=../../utils/with-1-data-1-empty-node-network.sh
source "$UTILS"/with-1-data-1-empty-node-network.sh
info="$(isle nebula show)"
[ "$(echo "$info" | jq -r '.CACert')" \
= "$(jq -r <"$BOOTSTRAP_FILE" '.CAPublicCredentials.Cert')" ]
[ "$(echo "$info" | jq -r '.SubnetCIDR')" = "10.6.9.0/24" ]
[ "$(echo "$info" | jq -r '.Lighthouses|length')" = "1" ]
[ "$(echo "$info" | jq -r '.Lighthouses[0].PublicAddr')" = "127.0.0.1:60000" ]
[ "$(echo "$info" | jq -r '.Lighthouses[0].IP')" = "10.6.9.1" ]

View File

@ -0,0 +1,17 @@
# shellcheck source=../../utils/with-1-data-1-empty-node-network.sh
source "$UTILS"/with-1-data-1-empty-node-network.sh
nebula-cert keygen -out-key /dev/null -out-pub pubkey
cat pubkey
(
isle nebula create-cert \
--hostname non-esiste \
--public-key-path pubkey \
2>&1 || true \
) | grep '\[6\] Host not found'
isle nebula create-cert \
--hostname primus \
--public-key-path pubkey \
| grep -- '-----BEGIN NEBULA CERTIFICATE-----'

View File

@ -0,0 +1,12 @@
# shellcheck source=../../utils/with-1-data-1-empty-node-network.sh
source "$UTILS"/with-1-data-1-empty-node-network.sh
[ "$(cat a/meta/isle/rpc_port)" = "3900" ]
[ "$(cat b/meta/isle/rpc_port)" = "3910" ]
[ "$(cat c/meta/isle/rpc_port)" = "3920" ]
[ "$(jq -r <"$BOOTSTRAP_FILE" '.NetworkCreationParams.ID')" != "" ]
[ "$(jq -r <"$BOOTSTRAP_FILE" '.NetworkCreationParams.Name')" = "testing" ]
[ "$(jq -r <"$BOOTSTRAP_FILE" '.NetworkCreationParams.Domain')" = "shared.test" ]
[ "$(jq -r <"$BOOTSTRAP_FILE" '.SignedHostAssigned.Body.Name')" = "primus" ]

View File

@ -13,5 +13,6 @@ export TMPDIR="$TMPDIR"
export XDG_RUNTIME_DIR="$XDG_RUNTIME_DIR" export XDG_RUNTIME_DIR="$XDG_RUNTIME_DIR"
export XDG_STATE_HOME="$XDG_STATE_HOME" export XDG_STATE_HOME="$XDG_STATE_HOME"
export ISLE_DAEMON_HTTP_SOCKET_PATH="$ROOT_TMPDIR/$base-daemon.sock" export ISLE_DAEMON_HTTP_SOCKET_PATH="$ROOT_TMPDIR/$base-daemon.sock"
BOOTSTRAP_FILE="$XDG_STATE_HOME/isle/bootstrap.json"
cd "$TMPDIR" cd "$TMPDIR"
EOF EOF

View File

@ -63,12 +63,10 @@ EOF
--domain shared.test \ --domain shared.test \
--hostname primus \ --hostname primus \
--ip-net "$ipNet" \ --ip-net "$ipNet" \
--name "testing" \ --name "testing"
> admin.json
echo "Creating secondus bootstrap" echo "Creating secondus bootstrap"
isle admin create-bootstrap \ isle hosts create \
--admin-path admin.json \
--hostname secondus \ --hostname secondus \
--ip "$secondus_ip" \ --ip "$secondus_ip" \
> "$secondus_bootstrap" > "$secondus_bootstrap"