Use yaml instead of tgz for bootstrap file

This commit is contained in:
Brian Picciano 2022-11-02 14:34:40 +01:00
parent 7d95825f97
commit 3ac86e07cf
18 changed files with 81 additions and 389 deletions

View File

@ -17,7 +17,7 @@ in rec {
builder = builtins.toFile "builder.sh" ''
source $stdenv/setup
mkdir -p "$out"/share
cp "$src" "$out"/share/bootstrap.tgz
cp "$src" "$out"/share/bootstrap.yml
'';
};

View File

@ -7,12 +7,12 @@ wishes to add.
There are two ways for a user to add a host to the cryptic-net network.
- If the user is savy enough to obtain their own `cryptic-net` binary, they can
do so. The admin can then generate a `bootstrap.tgz` file for their host,
do so. The admin can then generate a `bootstrap.yml` file for their host,
give that to the user, and the user can run `cryptic-net daemon` using that
bootstrap file.
- If the user is not so savy, the admin can generate a custom `cryptic-net`
binary with the `bootstrap.tgz` embedded into it. The user can be given this
binary with the `bootstrap.yml` embedded into it. The user can be given this
binary and run `cryptic-net daemon` without any configuration on their end.
From the admin's perspective the only difference between these cases is one
@ -35,11 +35,11 @@ The admin should choose an IP for the host. The IP you choose for the new host
should be one which is not yet used by any other host, and which is in subnet
which was configured when creating the network.
## Step 3: Create a `bootstrap.tgz` File
## Step 3: Create a `bootstrap.yml` File
Access to an `admin.yml` file is required for this step.
To create a `bootstrap.tgz` file for the new host, the admin should perform the
To create a `bootstrap.yml` file for the new host, the admin should perform the
following command from their own host:
```
@ -47,15 +47,15 @@ cryptic-net hosts make-bootstrap \
--name <name> \
--ip <ip> \
--admin-path <path to admin.yml> \
> bootstrap.tgz
> bootstrap.yml
```
The resulting `bootstrap.tgz` file should be treated as a secret file that is
shared only with the user it was generated for. The `bootstrap.tgz` file should
The resulting `bootstrap.yml` file should be treated as a secret file that is
shared only with the user it was generated for. The `bootstrap.yml` file should
not be re-used between hosts either.
If the user already has access to a `cryptic-net` binary then the new
`bootstrap.tgz` file can be given to them as-is, and they can proceed with
`bootstrap.yml` file can be given to them as-is, and they can proceed with
running their host's `cryptic-net daemon`.
### Encrypted `admin.yml`
@ -63,14 +63,14 @@ running their host's `cryptic-net daemon`.
If `admin.yml` is kept in an encrypted format on disk (it should be!) then the
decrypted form can be piped into `make-bootstrap` over stdin. For example, if
GPG is being used to secure `admin.yml` then the following could be used to
generate a `bootstrap.tgz`:
generate a `bootstrap.yml`:
```
gpg -d <path to admin.yml.gpg> | cryptic-net hosts make-boostrap \
--name <name> \
--ip <ip> \
--admin-path - \
> bootstrap.tgz
> bootstrap.yml
```
Note that the value of `--admin-path` is `-`, indicating that `admin.yml` should
@ -78,14 +78,14 @@ be read from stdin.
## Step 4: Optionally, Build Binary
If you wish to embed the `bootstrap.tgz` into a custom binary for the user (to
If you wish to embed the `bootstrap.yml` into a custom binary for the user (to
make installation _extremely_ easy for them) then you can run the following:
```
nix-build --arg bootstrap <path to bootstrap.tgz> -A appImage
nix-build --arg bootstrap <path to bootstrap.yml> -A appImage
```
The resulting binary can be found in the `result` directory which is created.
This binary should be treated like a `bootstrap.tgz` in terms of its uniqueness
This binary should be treated like a `bootstrap.yml` in terms of its uniqueness
and sensitivity.

View File

@ -15,8 +15,8 @@ state AppDir {
entrypoint : * Create runtime dir at $_RUNTIME_DIR_PATH
entrypoint : * Lock runtime dir
entrypoint : * Merge given and default daemon.yml files
entrypoint : * Copy bootstrap.tgz into $_DATA_DIR_PATH, if it's not there
entrypoint : * Merge daemon.yml config into bootstrap.tgz
entrypoint : * Copy bootstrap.yml into $_DATA_DIR_PATH, if it's not there
entrypoint : * Merge daemon.yml config into bootstrap.yml
entrypoint : * Create $_RUNTIME_DIR_PATH/dnsmasq.conf
entrypoint : * Create $_RUNTIME_DIR_PATH/nebula.yml
entrypoint : * Create $_RUNTIME_DIR_PATH/garage-N.toml\n (one per storage allocation)

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

View File

@ -63,7 +63,7 @@ would be great. Such a mobile app could be based on the existing
[mobile_nebula](https://github.com/DefinedNet/mobile_nebula). The main changes
needed would be:
- Allow importing a `bootstrap.tgz` file, rather than requiring manual setup by
- Allow importing a `bootstrap.yml` file, rather than requiring manual setup by
users.
- Set device's DNS settings. There is an [open

View File

@ -37,7 +37,7 @@ variable for `nix-daemon` (see [this github issue][tmpdir-gh].))
The resulting binary can be found in the `result` directory which is created.
In this case you will need an admin to provide you with a `bootstrap.tgz` for
In this case you will need an admin to provide you with a `bootstrap.yml` for
your host, rather than a custom binary. When running the daemon in the following
steps you will need to provide the `--bootstrap-path` CLI argument to the daemon
process.

View File

@ -1,4 +1,4 @@
// Package bootstrap deals with the parsing and creation of bootstrap.tgz files.
// Package bootstrap deals with the parsing and creation of bootstrap.yml files.
// It also contains some helpers which rely on bootstrap data.
package bootstrap
@ -6,12 +6,9 @@ import (
"cryptic-net/admin"
"cryptic-net/garage"
"cryptic-net/nebula"
"cryptic-net/tarutil"
"cryptic-net/yamlutil"
"crypto/sha512"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"sort"
@ -19,101 +16,45 @@ import (
"gopkg.in/yaml.v3"
)
// Paths within the bootstrap FS which for general data.
const (
adminCreationParamsPath = "admin/creation-params.yml"
hostNamePath = "hostname"
)
// DataDirPath returns the path within the user's data directory where the
// bootstrap file is stored.
func DataDirPath(dataDirPath string) string {
return filepath.Join(dataDirPath, "bootstrap.tgz")
return filepath.Join(dataDirPath, "bootstrap.yml")
}
// AppDirPath returns the path within the AppDir where an embedded bootstrap
// file might be found.
func AppDirPath(appDirPath string) string {
return filepath.Join(appDirPath, "share/bootstrap.tgz")
return filepath.Join(appDirPath, "share/bootstrap.yml")
}
// Bootstrap is used for accessing all information contained within a
// bootstrap.tgz file.
// bootstrap.yml file.
type Bootstrap struct {
AdminCreationParams admin.CreationParams
AdminCreationParams admin.CreationParams `yaml:"admin_creation_params"`
Hosts map[string]Host
HostName string
Hosts map[string]Host `yaml:"hosts"`
HostName string `yaml:"hostname"`
NebulaHostCredentials nebula.HostCredentials
Nebula struct {
HostCredentials nebula.HostCredentials `yaml:"host_credentials"`
} `yaml:"nebula"`
GarageRPCSecret string
GarageAdminToken string
GarageGlobalBucketS3APICredentials garage.S3APICredentials
Garage struct {
RPCSecret string `yaml:"rpc_secret"`
AdminToken string `yaml:"admin_token"`
GlobalBucketS3APICredentials garage.S3APICredentials `yaml:"global_bucket_s3_api_credentials"`
} `yaml:"garage"`
}
// FromFS loads a Boostrap instance from the given fs.FS, which presumably
// represents the file structure of a bootstrap.tgz file.
func FromFS(bootstrapFS fs.FS) (Bootstrap, error) {
var (
b Bootstrap
err error
)
if b.Hosts, err = loadHosts(bootstrapFS); err != nil {
return Bootstrap{}, fmt.Errorf("loading hosts info from fs: %w", err)
}
filesToLoadAsYAML := []struct {
into interface{}
path string
}{
{&b.AdminCreationParams, adminCreationParamsPath},
{&b.GarageGlobalBucketS3APICredentials, garageGlobalBucketKeyYmlPath},
}
for _, f := range filesToLoadAsYAML {
if err := yamlutil.LoadYamlFSFile(f.into, bootstrapFS, f.path); err != nil {
return Bootstrap{}, fmt.Errorf("loading %q from fs: %w", f.path, err)
}
}
filesToLoadAsString := []struct {
into *string
path string
}{
{&b.HostName, hostNamePath},
{&b.NebulaHostCredentials.CACertPEM, nebulaCertsCACertPath},
{&b.NebulaHostCredentials.HostCertPEM, nebulaCertsHostCertPath},
{&b.NebulaHostCredentials.HostKeyPEM, nebulaCertsHostKeyPath},
{&b.GarageRPCSecret, garageRPCSecretPath},
{&b.GarageAdminToken, garageAdminTokenPath},
}
for _, f := range filesToLoadAsString {
body, err := fs.ReadFile(bootstrapFS, f.path)
if err != nil {
return Bootstrap{}, fmt.Errorf("loading %q from fs: %w", f.path, err)
}
*f.into = string(body)
}
return b, nil
}
// FromReader reads a bootstrap.tgz file from the given io.Reader.
// FromReader reads a bootstrap.yml file from the given io.Reader.
func FromReader(r io.Reader) (Bootstrap, error) {
fs, err := tarutil.FSFromReader(r)
if err != nil {
return Bootstrap{}, fmt.Errorf("reading bootstrap.tgz: %w", err)
}
return FromFS(fs)
var b Bootstrap
err := yaml.NewDecoder(r).Decode(&b)
return b, err
}
// FromFile reads a bootstrap.tgz from a file at the given path.
// FromFile reads a bootstrap.yml from a file at the given path.
func FromFile(path string) (Bootstrap, error) {
f, err := os.Open(path)
@ -125,58 +66,9 @@ func FromFile(path string) (Bootstrap, error) {
return FromReader(f)
}
// WriteTo writes the Bootstrap as a new bootstrap.tgz to the given io.Writer.
// WriteTo writes the Bootstrap as a new bootstrap.yml to the given io.Writer.
func (b Bootstrap) WriteTo(into io.Writer) error {
w := tarutil.NewTGZWriter(into)
for _, host := range b.Hosts {
hostB, err := yaml.Marshal(host)
if err != nil {
return fmt.Errorf("yaml encoding host %#v: %w", host, err)
}
path := filepath.Join(hostsDirPath, host.Name+".yml")
w.WriteFileBytes(path, hostB)
}
filesToWriteAsYAML := []struct {
value interface{}
path string
}{
{b.AdminCreationParams, adminCreationParamsPath},
{b.GarageGlobalBucketS3APICredentials, garageGlobalBucketKeyYmlPath},
}
for _, f := range filesToWriteAsYAML {
b, err := yaml.Marshal(f.value)
if err != nil {
return fmt.Errorf("yaml encoding data for %q: %w", f.path, err)
}
w.WriteFileBytes(f.path, b)
}
filesToWriteAsString := []struct {
value string
path string
}{
{b.HostName, hostNamePath},
{b.NebulaHostCredentials.CACertPEM, nebulaCertsCACertPath},
{b.NebulaHostCredentials.HostCertPEM, nebulaCertsHostCertPath},
{b.NebulaHostCredentials.HostKeyPEM, nebulaCertsHostKeyPath},
{b.GarageRPCSecret, garageRPCSecretPath},
{b.GarageAdminToken, garageAdminTokenPath},
}
for _, f := range filesToWriteAsString {
w.WriteFileBytes(f.path, []byte(f.value))
}
return w.Close()
return yaml.NewEncoder(into).Encode(b)
}
// ThisHost is a shortcut for b.Hosts[b.HostName], but will panic if the

View File

@ -4,13 +4,6 @@ import (
"cryptic-net/garage"
)
// Paths within the bootstrap FS related to garage.
const (
garageRPCSecretPath = "garage/rpc-secret.txt"
garageAdminTokenPath = "garage/admin-token.txt"
garageGlobalBucketKeyYmlPath = "garage/cryptic-net-global-bucket-key.yml"
)
// GaragePeers returns a Peer for each known garage instance in the network.
func (b Bootstrap) GaragePeers() []garage.RemotePeer {
@ -77,6 +70,6 @@ func (b Bootstrap) ChooseGaragePeer() garage.RemotePeer {
// the global bucket.
func (b Bootstrap) GlobalBucketS3APIClient() garage.S3APIClient {
addr := b.ChooseGaragePeer().S3APIAddr()
creds := b.GarageGlobalBucketS3APICredentials
creds := b.Garage.GlobalBucketS3APICredentials
return garage.NewS3APIClient(addr, creds)
}

View File

@ -33,7 +33,7 @@ func (b Bootstrap) PutGarageBoostrapHost(ctx context.Context) error {
buf := new(bytes.Buffer)
err = nebula.SignAndWrap(buf, b.NebulaHostCredentials.HostKeyPEM, hostB)
err = nebula.SignAndWrap(buf, b.Nebula.HostCredentials.HostKeyPEM, hostB)
if err != nil {
return fmt.Errorf("signing encoded host data: %w", err)
}
@ -82,7 +82,7 @@ func (b Bootstrap) GetGarageBootstrapHosts(
map[string]Host, error,
) {
caCertPEM := b.NebulaHostCredentials.CACertPEM
caCertPEM := b.Nebula.HostCredentials.CACertPEM
client := b.GlobalBucketS3APIClient()
hosts := map[string]Host{}
@ -124,12 +124,12 @@ func (b Bootstrap) GetGarageBootstrapHosts(
hostCertPEM := host.Nebula.CertPEM
if err := nebula.ValidateSignature(hostCertPEM, hostB, sig); err != nil {
fmt.Fprintf(os.Stderr, "invalid host data for %q: %w\n", objInfo.Key, err)
fmt.Fprintf(os.Stderr, "invalid host data for %q: %v\n", objInfo.Key, err)
continue
}
if err := nebula.ValidateHostCertPEM(caCertPEM, hostCertPEM); err != nil {
fmt.Fprintf(os.Stderr, "invalid nebula cert for %q: %w\n", objInfo.Key, err)
fmt.Fprintf(os.Stderr, "invalid nebula cert for %q: %v\n", objInfo.Key, err)
continue
}

View File

@ -3,16 +3,7 @@ package bootstrap
import (
"cryptic-net/nebula"
"fmt"
"io/fs"
"net"
"path/filepath"
"strings"
"gopkg.in/yaml.v3"
)
const (
hostsDirPath = "hosts"
)
// NebulaHost describes the nebula configuration of a Host which is relevant for
@ -54,43 +45,3 @@ func (h Host) IP() net.IP {
return ip
}
func loadHosts(bootstrapFS fs.FS) (map[string]Host, error) {
hosts := map[string]Host{}
readAsYaml := func(into interface{}, path string) error {
b, err := fs.ReadFile(bootstrapFS, path)
if err != nil {
return fmt.Errorf("reading file from fs: %w", err)
}
return yaml.Unmarshal(b, into)
}
globPath := filepath.Join(hostsDirPath, "*.yml")
hostPaths, err := fs.Glob(bootstrapFS, globPath)
if err != nil {
return nil, fmt.Errorf("listing host files at %q in fs: %w", globPath, err)
}
for _, hostPath := range hostPaths {
hostName := filepath.Base(hostPath)
hostName = strings.TrimSuffix(hostName, filepath.Ext(hostName))
var host Host
if err := readAsYaml(&host, hostPath); err != nil {
return nil, fmt.Errorf("reading %q as yaml: %w", hostPath, err)
}
hosts[hostName] = host
}
if len(hosts) == 0 {
return nil, fmt.Errorf("failed to load any hosts from fs")
}
return hosts, nil
}

View File

@ -1,8 +0,0 @@
package bootstrap
// Paths within the bootstrap FS related to nebula.
const (
nebulaCertsCACertPath = "nebula/certs/ca.crt"
nebulaCertsHostCertPath = "nebula/certs/host.crt"
nebulaCertsHostKeyPath = "nebula/certs/host.key"
)

View File

@ -142,13 +142,14 @@ var subCmdAdminCreateNetwork = subCmd{
},
},
},
HostName: *hostName,
NebulaHostCredentials: nebulaHostCreds,
GarageRPCSecret: randStr(32),
GarageAdminToken: randStr(32),
GarageGlobalBucketS3APICredentials: garage.NewS3APICredentials(),
HostName: *hostName,
}
hostBootstrap.Nebula.HostCredentials = nebulaHostCreds
hostBootstrap.Garage.RPCSecret = randStr(32)
hostBootstrap.Garage.AdminToken = randStr(32)
hostBootstrap.Garage.GlobalBucketS3APICredentials = garage.NewS3APICredentials()
if hostBootstrap, err = mergeDaemonConfigIntoBootstrap(hostBootstrap, daemonConfig); err != nil {
return fmt.Errorf("merging daemon config into bootstrap data: %w", err)
}
@ -215,8 +216,8 @@ var subCmdAdminCreateNetwork = subCmd{
CreationParams: adminCreationParams,
}
adm.Nebula.CACredentials = nebulaCACreds
adm.Garage.RPCSecret = hostBootstrap.GarageRPCSecret
adm.Garage.GlobalBucketS3APICredentials = hostBootstrap.GarageGlobalBucketS3APICredentials
adm.Garage.RPCSecret = hostBootstrap.Garage.RPCSecret
adm.Garage.GlobalBucketS3APICredentials = hostBootstrap.Garage.GlobalBucketS3APICredentials
if err := adm.WriteTo(os.Stdout); err != nil {
return fmt.Errorf("writing admin.yml to stdout")
@ -228,7 +229,7 @@ var subCmdAdminCreateNetwork = subCmd{
var subCmdAdminMakeBootstrap = subCmd{
name: "make-bootstrap",
descr: "Creates a new bootstrap.tgz file for a particular host and writes it to stdout",
descr: "Creates a new bootstrap.yml file for a particular host and writes it to stdout",
checkLock: true,
do: func(subCmdCtx subCmdCtx) error {
@ -236,7 +237,7 @@ var subCmdAdminMakeBootstrap = subCmd{
name := flags.StringP(
"name", "n", "",
"Name of the host to generate bootstrap.tgz for",
"Name of the host to generate bootstrap.yml for",
)
ipStr := flags.StringP(
@ -287,14 +288,13 @@ var subCmdAdminMakeBootstrap = subCmd{
Hosts: hostBootstrap.Hosts,
HostName: *name,
NebulaHostCredentials: nebulaHostCreds,
GarageRPCSecret: adm.Garage.RPCSecret,
GarageAdminToken: randStr(32),
GarageGlobalBucketS3APICredentials: adm.Garage.GlobalBucketS3APICredentials,
}
newHostBootstrap.Nebula.HostCredentials = nebulaHostCreds
newHostBootstrap.Garage.RPCSecret = adm.Garage.RPCSecret
newHostBootstrap.Garage.AdminToken = randStr(32)
newHostBootstrap.Garage.GlobalBucketS3APICredentials = adm.Garage.GlobalBucketS3APICredentials
return newHostBootstrap.WriteTo(os.Stdout)
},
}

View File

@ -213,7 +213,7 @@ var subCmdDaemon = subCmd{
bootstrapPath := flags.StringP(
"bootstrap-path", "b", "",
`Path to a bootstrap.tgz file. This only needs to be provided the first time the daemon is started, after that it is ignored. If the cryptic-net binary has a bootstrap built into it then this argument is always optional.`,
`Path to a bootstrap.yml file. This only needs to be provided the first time the daemon is started, after that it is ignored. If the cryptic-net binary has a bootstrap built into it then this argument is always optional.`,
)
if err := flags.Parse(subCmdCtx.args); err != nil {
@ -249,7 +249,7 @@ var subCmdDaemon = subCmd{
return false
} else if err != nil {
err = fmt.Errorf("parsing bootstrap.tgz at %q: %w", path, err)
err = fmt.Errorf("parsing bootstrap.yml at %q: %w", path, err)
return false
}
@ -262,17 +262,17 @@ var subCmdDaemon = subCmd{
foundHostBootstrap = !foundHostBootstrap && tryLoadBootstrap(bootstrapAppDirPath)
if err != nil {
return fmt.Errorf("attempting to load bootstrap.tgz file: %w", err)
return fmt.Errorf("attempting to load bootstrap.yml file: %w", err)
} else if !foundHostBootstrap {
return errors.New("No bootstrap.tgz file could be found, and one is not provided with --bootstrap-path")
return errors.New("No bootstrap.yml file could be found, and one is not provided with --bootstrap-path")
} else if hostBootstrapPath != bootstrapDataDirPath {
// If the bootstrap file is not being stored in the data dir, copy
// it there, so it can be loaded from there next time.
if err := writeBootstrapToDataDir(hostBootstrap); err != nil {
return fmt.Errorf("writing bootstrap.tgz to data dir: %w", err)
return fmt.Errorf("writing bootstrap.yml to data dir: %w", err)
}
}

View File

@ -38,11 +38,11 @@ var subCmdGarageMC = subCmd{
if *keyID == "" || *keySecret == "" {
if *keyID == "" {
*keyID = hostBootstrap.GarageGlobalBucketS3APICredentials.ID
*keyID = hostBootstrap.Garage.GlobalBucketS3APICredentials.ID
}
if *keySecret == "" {
*keyID = hostBootstrap.GarageGlobalBucketS3APICredentials.Secret
*keyID = hostBootstrap.Garage.GlobalBucketS3APICredentials.Secret
}
}
@ -97,7 +97,7 @@ var subCmdGarageCLI = subCmd{
cliEnv = append(
os.Environ(),
"GARAGE_RPC_HOST="+hostBootstrap.ChooseGaragePeer().RPCAddr(),
"GARAGE_RPC_SECRET="+hostBootstrap.GarageRPCSecret,
"GARAGE_RPC_SECRET="+hostBootstrap.Garage.RPCSecret,
)
)

View File

@ -26,7 +26,7 @@ func newGarageAdminClient(
thisHost.IP().String(),
strconv.Itoa(daemonConfig.Storage.Allocations[0].AdminPort),
),
hostBootstrap.GarageAdminToken,
hostBootstrap.Garage.AdminToken,
)
}
@ -53,7 +53,7 @@ func waitForGarageAndNebula(
adminClient := garage.NewAdminClient(
adminAddr,
hostBootstrap.GarageAdminToken,
hostBootstrap.Garage.AdminToken,
)
if err := adminClient.Wait(ctx); err != nil {
@ -112,8 +112,8 @@ func garageWriteChildConfig(
MetaPath: alloc.MetaPath,
DataPath: alloc.DataPath,
RPCSecret: hostBootstrap.GarageRPCSecret,
AdminToken: hostBootstrap.GarageAdminToken,
RPCSecret: hostBootstrap.Garage.RPCSecret,
AdminToken: hostBootstrap.Garage.AdminToken,
RPCAddr: peer.RPCAddr(),
S3APIAddr: peer.S3APIAddr(),
@ -167,7 +167,7 @@ func garageInitializeGlobalBucket(
var (
adminClient = newGarageAdminClient(hostBootstrap, daemonConfig)
globalBucketCreds = hostBootstrap.GarageGlobalBucketS3APICredentials
globalBucketCreds = hostBootstrap.Garage.GlobalBucketS3APICredentials
)
// first attempt to import the key

View File

@ -58,9 +58,9 @@ func nebulaPmuxProcConfig(
config := map[string]interface{}{
"pki": map[string]string{
"ca": hostBootstrap.NebulaHostCredentials.CACertPEM,
"cert": hostBootstrap.NebulaHostCredentials.HostCertPEM,
"key": hostBootstrap.NebulaHostCredentials.HostKeyPEM,
"ca": hostBootstrap.Nebula.HostCredentials.CACertPEM,
"cert": hostBootstrap.Nebula.HostCredentials.HostCertPEM,
"key": hostBootstrap.Nebula.HostCredentials.HostKeyPEM,
},
"static_host_map": staticHostMap,
"punchy": map[string]bool{

View File

@ -1,24 +0,0 @@
// Package tarutil implements utilities which are useful for interacting with
// tar and tgz files.
package tarutil
import (
"compress/gzip"
"fmt"
"io"
"io/fs"
"github.com/nlepage/go-tarfs"
)
// FSFromReader returns a FS instance which will read the contents of a tgz
// file from the given Reader.
func FSFromReader(r io.Reader) (fs.FS, error) {
gf, err := gzip.NewReader(r)
if err != nil {
return nil, fmt.Errorf("un-gziping: %w", err)
}
defer gf.Close()
return tarfs.New(gf)
}

View File

@ -1,112 +0,0 @@
package tarutil
import (
"archive/tar"
"bytes"
"compress/gzip"
"fmt"
"io"
"path/filepath"
"strings"
)
// TGZWriter is a utility for writing tgz files. If an internal error is
// encountered by any method then all subsequent methods will be no-ops, and
// Close() will return that error (after closing out resources).
type TGZWriter struct {
gzipW *gzip.Writer
tarW *tar.Writer
err error
dirsWritten map[string]bool
}
// NewTGZWriter initializes and returns a new instance of TGZWriter which will
// write all data to the given io.Writer.
func NewTGZWriter(w io.Writer) *TGZWriter {
gzipW := gzip.NewWriter(w)
tarW := tar.NewWriter(gzipW)
return &TGZWriter{
gzipW: gzipW,
tarW: tarW,
dirsWritten: map[string]bool{},
}
}
// Close cleans up all open resources being held by TGZWriter, and returns the
// first internal error which was encountered during its operation (if any).
func (w *TGZWriter) Close() error {
w.tarW.Close()
w.gzipW.Close()
return w.err
}
func (w *TGZWriter) writeDir(path string) {
if w.err != nil {
return
} else if path != "." {
w.writeDir(filepath.Dir(path))
}
if path == "." {
path = "./"
} else {
path = "./" + strings.TrimPrefix(path, "./")
path = path + "/"
}
if w.dirsWritten[path] {
return
}
err := w.tarW.WriteHeader(&tar.Header{
Name: path,
Mode: 0700,
})
if err != nil {
w.err = fmt.Errorf("writing header for directory %q: %w", path, err)
return
}
w.dirsWritten[path] = true
}
// WriteFile writes a file to the tgz archive. The file will automatically be
// rooted to the "." directory, and any sub-directories the file exists in
// should have already been created.
func (w *TGZWriter) WriteFile(path string, size int64, body io.Reader) {
if w.err != nil {
return
}
path = "./" + strings.TrimPrefix(path, "./")
w.writeDir(filepath.Dir(path))
err := w.tarW.WriteHeader(&tar.Header{
Name: path,
Size: size,
Mode: 0400,
})
if err != nil {
w.err = fmt.Errorf("writing header for file %q: %w", path, err)
return
}
if _, err := io.Copy(w.tarW, body); err != nil {
w.err = fmt.Errorf("writing file body of file %q: %w", path, err)
return
}
}
// WriteFileBytes is a shortcut for calling WriteFile with the given byte slice
// being used as the file body.
func (w *TGZWriter) WriteFileBytes(path string, body []byte) {
bodyR := bytes.NewReader(body)
w.WriteFile(path, bodyR.Size(), bodyR)
}