diff --git a/README.md b/README.md index 941a062..4126cf1 100644 --- a/README.md +++ b/README.md @@ -9,11 +9,12 @@ they do, and fine-grained control over which users can do what. **Pull requests and issues** - Facilitation of discussion via comment commits, and fine-grained (down to the file level) sign-off requirements. -**Tags and releases**\* - Mark releases in the repo itself, and provide -immutable and verifiable git tags so there's never any funny business. +**Tags and releases** - Mark releases in the repo itself, and provide +immutable and verifiable git tags so there's never any funny business. (Not yet +implemented) -**Plugins**\*: Extend all aspects of dehub functionality via executables managed -in the repo itself (in the same style as git hooks). +**Plugins**: Extend all aspects of dehub functionality via executables managed +in the repo itself (in the same style as git hooks). (Not yet implemented) ## Key Concepts @@ -26,10 +27,10 @@ to the configuration of its parent in order to be considered _verifiable_. The configuration file is committed to the repo like any other file would be, and so is even able to define the access controls on itself. -Second, the commit message of every dehub commit contains YAML encoded metadata, -which allows dehub to extend git and provide multiple commit types, each with -its own capabilities and restrictions. Some example dehub commit types are -`change` commits, `comment` commits, and `credential` commits. +Second, the commit message of every dehub commit contains a YAML encoded +payload, which allows dehub to extend git and provide multiple commit types, +each with its own capabilities and restrictions. Some example dehub commit types +are `change` commits, `comment` commits, and `credential` commits. ## Infrastructure (or lack thereof) @@ -40,17 +41,20 @@ infrastructure can be used to host any dehub project: * The most barebones [git daemon](https://git-scm.com/book/en/v2/Git-on-the-Server-Git-Daemon) server (with a simple pre-receive hook set up). + * A remote SSH endpoint. + * A mailing list (aka the old school way). + * Network file syncing utilities such as dropbox, [syncthing](https://github.com/syncthing/syncthing), or [NFS](https://en.wikipedia.org/wiki/Network_File_System). + * Existing git project hosts like GitHub, Bitbucket, or Keybase. -* Decentralized filesystems such as IPFS\*. -_\* Planned feature, but not yet implemented._ +* Decentralized filesystems such as IPFS. (Not yet implemented) -# Getting Started +## Getting Started {#getting-started} The dehub project itself can be found by cloning `https://dehub.dev/src/dehub.git`. @@ -58,7 +62,7 @@ The dehub project itself can be found by cloning Installation of the dehub tool is currently done via the `go get` command: ``` -go get dehub.dev/src/dehub.git/cmd/dehub +go get -u -v dehub.dev/src/dehub.git/cmd/dehub ``` This will install the binary to your `$GOBIN` path, which you'll want to put in @@ -68,28 +72,33 @@ Once installed, running `dehub -h` should show you the help output of the command. You can continue on to the tutorials if you're not sure where to go from here. -## Tutorials +### Tutorials {#tutorials} -The following tutorials will guide you through the basic usage of dehub. As -dehub is still very much in development a high level of git and PGP profiency is -still required in order to use dehub effectively. +The following tutorials will guide you through the basic usage of dehub. Note +that dehub is in the infancy of its development, and so a certain level of +profiency with git and PGP is required in order to follow these tutorials. -TODO +* [Tutorial 0: Say Hello!](/docs/tut0.html) +* [Tutorial 1: Create Your Own Project](/docs/tut1.html) +* [Tutorial 2: Access Controls](/docs/tut2.html) +* [Tutorial 3: Commit Sign-Off](/docs/tut3.html) -## Documentation +### Documentation -The [SPEC](/SPEC.html) is the best place to see every possible nitty-gritty +The [SPEC](/docs/SPEC.html) is the best place to see every possible nitty-gritty detail of how dehub works. It attempts to be both human-readable and exhaustive in its coverage. -## Other links +### Other links -[ROADMAP](/ROADMAP.html) documents upcoming features and other work required on -the project. If you're looking to contribute, this is a great place to start. +[ROADMAP](/docs/ROADMAP.html) documents upcoming features and other work +required on the project. If you're looking to contribute, this is a great place +to start. [dehub-remote](/cmd/dehub-remote/) is a simple docker image which can be used to host a remote dehub project over http(s). The endpoint will automatically verify -all pushed commits. +all pushed commits. [Tutorial 4](#tutorials) provides a brief walkthrough on +using it. [git-http-server](/cmd/git-http-server/) is a small server which makes a git repo's file tree available via http. It will automatically render markdown files diff --git a/ROADMAP.md b/docs/ROADMAP.md similarity index 99% rename from ROADMAP.md rename to docs/ROADMAP.md index 71b3da5..9802adb 100644 --- a/ROADMAP.md +++ b/docs/ROADMAP.md @@ -10,7 +10,6 @@ Must be able to feel good about showing the project publicly, as well as be able to accept help from people asking to help. * Figure out commit range syntax, use that everywhere. -* Tutorials * Maybe move external host? ## Milestone: IPFS support diff --git a/SPEC.md b/docs/SPEC.md similarity index 61% rename from SPEC.md rename to docs/SPEC.md index b2ecdb1..f917d1f 100644 --- a/SPEC.md +++ b/docs/SPEC.md @@ -7,12 +7,14 @@ and in practical use of the git tool. All references to a git-specific concept retain their meaning; dehub concepts build upon git concepts, but do not override them. -## dehub Project +## Project {#project} A dehub project is comprised of: * A collection of files and directories. + * Meta actions related to those files, e.g. discussion, proposed changes, etc. + * Configuration defining which meta actions are allowed under which circumstances. @@ -20,11 +22,11 @@ All of these components are housed in a git repository. A dehub project does not require a central repository location (a "remote"), though it may use one if desired. -## Commit Payload +## Commit Payload {#payload} -All commits in a dehub project contain a payload. The payload is encoded into -the commit message as a YAML object. Here is the general structure of a commit -message containing a payload: +All commits in a dehub [project](#project) contain a payload. The payload is +encoded into the commit message as a YAML object. Here is the general structure +of a commit message containing a payload: ``` Human readable message head @@ -44,18 +46,18 @@ The message head is a human readable description of what is being committed, and is terminated at the first newline. Everything after the message head must be valid YAML which encodes the payload. -### Fingerprint +### Fingerprint {#fingerprint} -Each payload object contains a `fingerprint` field. The fingerprint is an opaque -byte string encoded using standard base-64. The algorithm used to generate the -fingerprint will depend on the payload type, and can be found in each type's -sub-section in this document. +Each [payload](#payload) object contains a `fingerprint` field. The fingerprint +is an opaque byte string encoded using standard base-64. The algorithm used to +generate the fingerprint will depend on the payload type, and can be found in +each type's sub-section in this document. -### Credential +### Credential {#credential} -The `credentials` field is not requires, but in practice will be found on almost -every payload. The field's value will be an array of credential objects. Only -one credential object is currently supported, `pgp_signature`: +The `credentials` field is not required, but in practice will be found on almost +every [payload](#payload). The field's value will be an array of credential +objects. Only one credential object is currently supported, `pgp_signature`: ```yaml type: pgp_signature @@ -75,13 +77,13 @@ pub_key_id: XXX body: std-base-64 signature ``` -### Payload Types +### Payload Types {#payload-types} -#### Change Payload +#### Change Payload {#change-payload} -A change payload encompasses a set of changes to the files in the project. To -construct the change payload one must reference the file tree of the commit -which houses the payload as well as the file tree of its parent commit; +A change [payload](#payload) encompasses a set of changes to the files in the +project. To construct the change payload one must reference the file tree of the +commit which houses the payload as well as the file tree of its parent commit; specifically one must take the difference between them. A change payload looks like this: @@ -98,9 +100,10 @@ description: |- made to the project's files. ``` -##### Change Payload Fingerprint +##### Change Payload Fingerprint {#change-payload-fingerprint} -The unencoded fingerprint of a change payload is calculated as follows: +The unencoded [fingerprint](#fingerprint) of a [change payload](#change-payload) +is calculated as follows: * Concatenate the following: * A uvarint indicating the number of bytes in the description string. @@ -124,12 +127,12 @@ The unencoded fingerprint of a change payload is calculated as follows: * Prepend a 0 byte to the result of the SHA-256 hash. This unencoded fingerprint is then standard base-64 encoded, and that is used as -the value of the fingerprint field. +the value of the `fingerprint` field. -#### Comment Payload +#### Comment Payload {#comment-payload} -A comment payload encompasses no file changes, and is used only to contain a -comment made by a single user. +A comment [payload](#payload) encompasses no file changes, and is used only to +contain a comment made by a single user. A comment payload looks like this: @@ -146,9 +149,10 @@ comment: |- The message head of a comment payload will generally be a truncated form of the comment itself. -##### Comment Payload Fingerprint +##### Comment Payload Fingerprint {#comment-payload-fingerprint} -The unencoded fingerprint of a comment payload is calculated as follows: +The unencoded [fingerprint](#fingerprint) of a [comment +payload](#comment-payload) is calculated as follows: * Concatenate the following: * A uvarint indicating the number of bytes in the comment string. @@ -157,13 +161,14 @@ The unencoded fingerprint of a comment payload is calculated as follows: * Prepend a 0 byte to the result of the SHA-256 hash. This unencoded fingerprint is then standard base-64 encoded, and that is used as -the value of the fingerprint field. +the value of the `fingerprint` field. #### Credential Payload -A credential payload contains only one or more credentials for an arbitrary -fingerprint. Credential payloads can be combined with other payloads of the same -fingerprint to create a new payload with many credentials. +A credential [payload](#payload) contains only one or more credentials for an +arbitrary [fingerprint](#fingerprint). Credential payloads can be combined with +other payloads of the same fingerprint to create a new payload with many +credentials. A credential payload looks like this: @@ -184,13 +189,13 @@ commits: change_description: blah blah blah ``` -## Project Configuration +## Project Configuration {#project-configuration} The `.dehub` directory contains all meta information related to the dehub -project. All files within `.dehub` are tracked by the git repo like any other -files in the project. +[project](#project). All files within `.dehub` are tracked by the git repo like +any other files in the project. -### config.yml +### config.yml {#config-yml} The `.dehub/config.yml` file contains a yaml encoded configuration object: @@ -201,10 +206,10 @@ access_controls: [...] Both fields are described in their own sub-section below. -#### Account +#### Account {#account} -An account defines a specific user of the project. Every account has an ID; no -two accounts within a project may share the same ID. +An account defines a specific user of a [project](#project). Every account has +an ID; no two accounts within a project may share the same ID. An account looks like this: @@ -213,11 +218,12 @@ id: some_string signifiers: [...] ``` -##### Signifier +##### Signifier {#signifier} -A signifier is used to signify that an account has taken some action. The most -common use-case is to prove that an account created a particular credential. An -account may have more than one signifier. +A signifier is used to signify that an [account](#account) has taken some +action. The most common use-case is to prove that an account created a +particular [credential](#credential). An account may have more than one +signifier. Currently there is only one signifier type, `pgp_public_key`: @@ -235,11 +241,11 @@ type: pgp_public_key body: inlined ASCII-armored pgp public key ``` -#### Access Control +#### Access Control {#access-control} An access control allows or denies a particular commit from becoming a part of -the project. Each action control has an action (allow or deny) and a set of -filters: +a [project](#project). Each access control has an action (allow or deny) and a +set of filters (filters are described in the next section): ```yaml action: allow # or deny @@ -253,29 +259,31 @@ and its action is taken. An access control with no filters matches all commits. -##### Filters +##### Filter {#filter} -There are many kinds of access control filters. Any filter can be applied to a -commit, with no other input, and produce a boolean value. All filters have a -`type` field which indicates their type. +There are many kinds of [access control](#access-control) filters. Any filter +can be applied to a commit, with no other input, and produce a boolean value. +All filters have a `type` field which indicates their type. -###### Signature Filter +###### Signature Filter {#signature-filter} -A filter of type `signature` asserts that a commit's payload contains signature -credentials with certain properties. A signature filter must have one of these -fields, which define the set of users or accounts whose signatures are -applicable. +A [filter](#filter) of type `signature` asserts that a commit's +[payload](#payload) contains [signature credentials](#credential) with certain +properties. A signature filter must have one of these fields, which define the +set of users or [accounts](#account) whose signatures are applicable. + +* `account_ids: [...]` - an array of account IDs, each having been defined in + the accounts section of the [configuration](#config-yml). -* `account_ids: [...]` - an array of account IDs, each having been defined in the - accounts section of the configuration. * `any_account: true` - matches any account defined in the accounts section of the configuration. + * `any: true` - matches any signature, whether or not its signifier has been defined in the configuration. -A `count` field may also be included. Its value may be a number or a string -indicating a percent (e.g. `"50%"`). If not included it will be assumed to be -`1`. +A `count` field may also be included. Its value may be an absolute number (e.g. +`5`) or it may be a string indicating a percent (e.g. `"50%"`). If not included +it will be assumed to be `1`. The count indicates how many accounts from the specified set must have a signature included. If a percent is given then that will be multiplied against @@ -308,11 +316,11 @@ type: signature any: true ``` -###### Branch Filter +###### Branch Filter {#branch-filter} -A filter of type `branch` matches the commit based on which branch in the repo -it is being or has been committed to. Matching is performed on the short name -of the branch, using globstar pattern matching. +A [filter](#filter) of type `branch` matches the commit based on which branch in +the repo it is being or has been committed to. Matching is performed on the +short name of the branch, using globstar pattern matching. A branch filter can have one or multiple patterns defined. The filter will match if at least one defined pattern matches the short form of the branch name. @@ -334,11 +342,12 @@ patterns: - amy/** ``` -###### Files Changed Filter +###### Files Changed Filter {#files-changed-filter} -A filter of type `files_changed` matches the commit based on which files were -changed between the tree of the commit's parent and the commit's tree. Matching -is performed on the paths of the changed files, relative to the repo root. +A [filter](#filter) of type `files_changed` matches the commit based on which +files were changed between the tree of the commit's parent and the commit's +tree. Matching is performed on the paths of the changed files, relative to the +repo root. A files changed filter can have one or multiple patterns defined. The filter will match if any of the changed files matches at least one defined pattern. @@ -360,12 +369,12 @@ patterns: - **.jpg ``` -###### Payload Type Filter +###### Payload Type Filter {#payload-type-filter} -A filter of type `payload_type` matches a commit based on the type of its -payload. A payload type filter can have one or more types defined. The filter -will match if the commit's payload type matches at least one of the defined -types. +A [filter](#filter) of type `payload_type` matches a commit based on the type of +its [payload](#payload). A payload type filter can have one or more types +defined. The filter will match if the commit's payload type matches at least one +of the defined types. A payload type filter with only one matching type can be defined like this: @@ -383,13 +392,13 @@ payload_types: - change ``` -###### Commit Attributes Filter +###### Commit Attributes Filter {#commit-attributes-filter} -A filter of type `commit_attributes` matches a commit based on certain -attributes it has. A commit attributes filter may have one or more fields -defined, each corresponding to a different attribute the commit may have. If -more than one field is defined then all corresponding attributes on the commit -must match for the filter to match. +A [filter](#filter) of type `commit_attributes` matches a commit based on +certain attributes it has. A commit attributes filter may have one or more +fields defined, each corresponding to a different attribute the commit may have. +If more than one field is defined then all corresponding attributes on the +commit must match for the filter to match. Currently the only possible attribute is `non_fast_forward: true`, which matches a commit which is not an ancestor of the HEAD of the branch it's being pushed @@ -402,11 +411,11 @@ type: commit_attributes non_fast_forward: true ``` -###### Not Filter +###### Not Filter {#not-filter} -A filter of type `not` matches a commit using the negation of a sub-filter, -defined within the not filter. If the sub-filter returns true for the commit, -then the not filter returns false, and vice-versa. +A [filter](#filter) of type `not` matches a commit using the negation of a +sub-filter, defined within the not filter. If the sub-filter returns true for +the commit, then the not filter returns false, and vice-versa. A not filter looks like this: @@ -418,10 +427,10 @@ filter: pattern: main ``` -##### Default Access Controls +##### Default Access Controls {#default-access-controls} -These access controls will be implicitly appended to the list defined in the -configuration: +These [access controls](#access-control) will be implicitly appended to the list +defined in the [configuration](#config-yml): ```yaml # Any account may add any commit to any non-main branch, provided there is at @@ -460,30 +469,32 @@ configuration: ``` These default access controls provide a useful baseline of requirements that all -projects will (hopefully) find useful in their infancy. +[projects](#project) will (hopefully) find useful in their infancy. -## Commit Verification +## Commit Verification {#commit-verification} The dehub protocol is designed such that every commit is "verifiable". A verifiable commit has the following properties: -* Its fingerprint is correctly formed. -* All of its credentials are correctly formed. +* Its [fingerprint](#fingerprint) is correctly formed. +* All of its [credentials](#credential) are correctly formed. * If they are signatures, they are valid signatures of the commit's unencoded fingerprint. -* The project's access controls allow the commit. +* The project's [access controls](#access-control) allow the commit. -The project's configuration is referenced frequently when verifying a commit, -such as when determining which access controls to apply and discovering -signifiers of accounts. In all cases the configuration as defined in the -commit's _parent_ is used when verifying that commit. The exception is the prime -commit, which uses its own configuration. +The [project's configuration](#config-yml) is referenced frequently when +verifying a commit, such as when determining which access controls to apply and +discovering [signifiers](#signifier) of [accounts](#account). In all cases the +configuration as defined in the commit's _parent_ is used when verifying that +commit. The exception is the [prime commit](#prime-commit), which uses its own +configuration. -### Prime Commit +### Prime Commit {#prime-commit} -The prime commit is the trusted seed of the project. When a user clones and -verifies a dehub project they must, implicitly or explicitly, trust the contents -of the prime commit. All other commits must be ancestors of the prime commit. +The prime commit is the trusted seed of the [project](#project). When a user +clones and verifies a dehub project they must, implicitly or explicitly, trust +the contents of the prime commit. All other commits must be ancestors of the +prime commit. Manually specifying a prime commit is not currently spec'd, but it will be. diff --git a/docs/tut0.md b/docs/tut0.md new file mode 100644 index 0000000..2f17f9b --- /dev/null +++ b/docs/tut0.md @@ -0,0 +1,128 @@ +# Tutorial 0: Say Hello! + +This tutorial will guide you through cloning a dehub project locally, creating a +comment, and pushing that comment back up to the remote. The project in +question: dehub itself! + +This tutorial assumes you have [dehub installed](/index.html#getting-started), +you have git and gpg installed, and you have a gpg key already created. + +## Step 0: Clone the Project + +Cloning the dehub project is as simple as cloning its git repo: + +``` +git clone https://dehub.dev/src/dehub.git +cd dehub +``` + +Once cloned, feel free to look around the project. You should initially find +yourself on the `main` branch, the primary branch of most dehub projects +(analogous to the `master` branch of most git repos). + +Calling `git log` will show the commits messages for all commits in the branch. +You will notice the commit messages aren't formatted in the familiar way, for +example: + +``` +commit 351048e9aabef7dc0f99b00f02547e409859a33f +Author: mediocregopher <> +Date: Sat Apr 25 15:17:21 2020 -0600 + + Completely rewrite SPEC + + --- + type: change + description: |- + Completely rewrite SPEC + + It's good this time, and complete. After this rewrite it will be necessary to + update a lot of the code, since quite a few things got renamed. + fingerprint: AG0s3yILU+0uIZltVY7A9/cgxr/pXk2MzGwExsY/hbIc + credentials: + - type: pgp_signature + pub_key_id: 95C46FA6A41148AC + body: BIG LONG STRING + account: mediocregopher +``` + +Instead of just being a human-readable description they are YAML encoded payload +objects. We will dive into these payload objects more throughout this tutorial +series. + +## Step 1: Checkout the Welcome Branch + +Next you're going to checkout the public welcome branch. This is done through a +normal git checkout command: + +``` +git checkout public/welcome +``` + +You can do `git log` to see all the comments people have been leaving in this +branch. The `public/welcome` branch is differentiated from the `main` branch in +two ways: + +* It has been configured to allow comment commits from anonymous users to be + pushed to it. Project configuration is covered in a future tutorial. + +* It has no code files tracked, its only purpose is for comments. + +## Step 2: Create Your Comment + +Now that you've poked around the welcome branch a bit, it's time to leave a +comment of your own! This is as easy as doing: + +``` +dehub commit --anon-pgp-key=KEY_NAME comment +``` + +(`KEY_NAME` should be replaced with any selector which will match your pgp key, +such as the key ID, the name on the key, or the email.) + +Your default text editor (defined by the EDITOR environment variable) will pop +up and you can then write down your comment. When you save and close your editor +dehub will sign the comment with your pgp key and create a commit with it. + +If you're having trouble thinking of something to say, here's some prompts to +get you going: + +* Introduce yourself; say where you're from and what your interests are. + +* How did you find dehub? Why is it interesting to you? + +* If you're using dehub for a project, shill your project! + +* If you'd like to get involved in dehub's development, let us know what your + skills are and how you can help. Remember, it takes more than expert + programmers to make a project successful. + +Once you've created your commit you can call `git log` to verify that it's been +created to your liking. If there's anything about the comment you'd like to +change you can amend the commit like so: + +``` +dehub commit --anon-pgp-key=KEY_NAME comment --amend +``` + +## Step 3: Push Your Commit + +As of now your comment commit only exists on your local machine. For everyone +else to see it you'll need to push it to the dehub server, exactly like with a +normal git commit. Pushing is done in the same way as a normal git commit as +well: `git push`. + +If you receive an error that's like `Updates were rejected because the tip of +your current branch is behind` then someone else has pushed to the branch in +between the last time you pulled and now. Do a `git pull --rebase` to pull in +those new changes, and try pushing again. + +## Step 4: Follow the Conversation + +In order to see other people's responses to your comment, and all other parts of +the conversation, all you need to do is call `git pull` with the +`public/welcome` branch checked out. + +You now have all the tools needed to participate in a dehub discussion thread! +Continue on to [Tutorial 1](tut1.html) to set up your own dehub project and +learn about credentials and their verification. diff --git a/docs/tut1.md b/docs/tut1.md new file mode 100644 index 0000000..120db9f --- /dev/null +++ b/docs/tut1.md @@ -0,0 +1,178 @@ +# Tutorial 1: Create Your Own Project + +This tutorial will guide you through starting a dehub project of your own, as +well as introducing some basic concepts regarding how commit payloads work. You +will use an example hello world project to do this. + +This tutorial assumes you have already completed [Tutorial 0](tut0.html). + +## Step 0: Init the Project + +A dehub project is initialized in the same way as a git project. An empty +directory is created, and `dehub init` is run within that directory. + +``` +mkdir hello-world +cd hello-world +dehub init +``` + +`dehub init` does nearly exactly the same thing as `git init`, with the primary +difference being that it sets the initial branch to be `main` instead of +`master`. dehub makes a distinction between `main` and `master` in order to help +prevent confusion between dehub and vanilla git projects, as well as to avoid +conflicts when migrating vanilla git projects to dehub. + +## Step 1: Add the First Account + +A dehub project is not fully initialized until it has an account defined for it. +dehub accounts refer to a specific user who has some kind of access to the +project. Each account can have specific permissions for it, as well as multiple +ways of signifying itself. + +For now, you'll add a basic account `tut` with a pgp key signifier. First, +create the `.dehub` directory, which is where all dehub project configuration +goes, and put your pgp key there: + +``` +mkdir .dehub +gpg -a --export KEY_ID > .dehub/tut.asc +``` + +Next you'll create the `.dehub/config.yml` file, which is where accounts are +actually defined (amongst many other things). The file should have the following +contents: + +```yaml +# contents of .dehub/config.yml +--- +accounts: + - id: tut + signifiers: + - type: pgp_public_key_file + path: ".dehub/tut.asc" +``` + +Finally, you'll commit these changes and the project will have its first commit! +Committing changes works very similarly to committing comments (as you did in +[Tutorial 0](tut0.html)). Where a comment commit merely carries a user's +comment, a change commit describes a set of changes to the tracked files in the +git repo. + +``` +git add --all +dehub commit --as tut change +``` + +Like when you made a comment commit this will pop up with your editor asking for +a description of the changes. Fill it in with something like `Initialize the +project` and save/close the editor. Depending on your pgp key settings you'll +likely be prompted for your pgp key password at this point. After that the +commit has been created! + +## Step 2: Inspect the Payload + +In this step you're going to look at the commit you just created and learn about +the contents of the payload. To view the commit do `git show`. Something similar +to the following should be output as the commit message: + +``` +commit 3cdcbc19546d4e6d817ebfba3e18afbc23283ec0 +Author: username <> +Date: Sat Apr 25 15:17:21 2020 -0600 + + Initialize the project + + --- + type: change + description: Initialize the project + fingerprint: AG0s3yILU+0uIZltVY7A9/cgxr/pXk2MzGwExsY/hbIc + credentials: + - type: pgp_signature + pub_key_id: 95C46FA6A41148AC + body: BIG LONG STRING + account: tut +``` + +All commits in a dehub project will contain a similar looking message. The first +line (the head) is always a human readable description of the commit. In this +case our commit description itself, `Initialize the project`, was used. + +After the head comes the payload, which is always a YAML encoded object. All +payloads have a `type` field indicating what type of payload they are. That type +will determine what other fields the payload is expected to have. The other +fields in this payload object are: + +* `description`: This is the description which was input into the editor when + creating the change commit. + +* `fingerprint`: A unique descriptor for this set of changes. It is computed + using both `description` and the files changed. + +* `credentials`: A set of credentials for this commit, each one declaring + that this commit has been given approval by a user. This commit has one + `pgp_signature` credential, created by the `tut` account. The `body` is a + signature of the `fingerprint` created by the `tut`'s pgp key. + +## Step 3: Create Another Commit + +Now that the initial commit is created, and configuration has been added to the +dehub project, you can continue on to use the project for what it was intended +for: greeting the world! + +Add a simple "hello world" script to the project by doing: + +``` +echo 'echo "hello world"' > hello.sh +git add hello.sh +dehub commit --as tut change --descr 'add hello.sh' +``` + +You'll notice that this time around you used the `--descr` flag to declare the +change's description, rather than opening up the editor + +Once again you can inspect the payload you just created using `git show`, if +you'd like, or continue on to the next step to learn about commit verification. + +## Step 4: Verify Your Commits + +All this work to create YAML encoded payloads has been done for one primary +purpose: to make commits verifiable. A verifiable commit is one which follows +the access controls defined by its parent. + +Your dehub project doesn't have any explicitly defined access controls (that +will be covered in a future tutorial), and so the defaults are used. By default, +dehub requires that all commits in `main` are change commits which have been +signed by at least one account. + +In order to verify the HEAD commit you can do: + +``` +dehub verify +``` + +This command looks at the project configuration defined in the parent of HEAD +and verifies that HEAD conforms to it. The HEAD of your project is a change +commit signed by the account `tut`, and so should be verifiable. + +Arbitrary commits can be verified using the `--rev` flag. This command will +verify the parent of HEAD, i.e. the initial commit: + +``` +dehub verify --rev HEAD^ +``` + +The initial commit doesn't have a parent, and so is a special case for +verification. The initial commit uses the configuration defined within itself in +order to verify itself. This creates an exploit opportunity: if you clone a +remote dehub project and an attacker intercepts that request they will be able +to send you back a project with a different initial commit than what you +expected. The whole project will still be verifiable, even though it's been +compromised. For this reason it's important to manually verify that the initial +commit of projects you clone are configured correctly, using the expected +signifiers for the expected accounts. + +You are now able to initialize a project, configure accounts within it, commit +changes to its files, and verify those commits. Well done! Continue on to +[Tutorial 2](tut2.html), where you will learn how to configure dehub's access +controls. diff --git a/docs/tut2.md b/docs/tut2.md new file mode 100644 index 0000000..45e6a12 --- /dev/null +++ b/docs/tut2.md @@ -0,0 +1,259 @@ +# Tutorial 2: Access Controls + +Access controls, in the context of a dehub project, refer to configuration +defining who is allowed to do what. These controls are defined within the dehub +project itself, within the `.dehub/config.yml` file. This tutorial will guide +you through the basics of how access controls work, how to define them, and some +examples of what can be done with them. + +This tutorial assumes you have already completed [Tutorial 1](tut1.html), and +builds on top of the project which was started there. + +## Step 0: Create a Restricted Account + +Inside the project you started in [Tutorial 1](tut1.html) you're going to add +another account to the project, called `tot`. Initially, `tot` will have all the +same permissions as `tut`, except being allowed to modify the project +configuration. + +First, export your gpg key into the project for `tot` to use, the same key used +for `tut`: + +``` +gpg -a --export KEY_ID > .dehub/tot.asc +``` + +(For the purposes of a tutorial it's fine for two accounts to share a +key, but it's not something which generally makes sense to do.) + +Now, modify the `.dehub/config.yml` to have the following contents: + +```yaml +# contents of .dehub/config.yml +--- +accounts: + - id: tut + signifiers: + - type: pgp_public_key_file + path: ".dehub/tut.asc" + - id: tot + signifiers: + - type: pgp_public_key_file + path: ".dehub/tot.asc" + +access_controls: + - action: allow + filters: + - type: signature + account_ids: + - tut + - type: files_changed + pattern: .dehub/* + + - action: deny + filters: + - type: files_changed + pattern: .dehub/* +``` + +The `accounts` section has been modified to add the `tot` account, but the +primary change here has been to add the `access_controls` section. The next +sub-sections will explain what exactly is being done here, but for now go ahead +and commit these changes: + +``` +git add --all +dehub commit --as tut change --descr 'add new restricted tot account' +``` + +### Access Controls + +Each access control is an action/filters pair. For any commit being verified, +the access controls defined in its parent commit are iterated through, in order, +until one is found whose filters all match the commit being verified. The action +for that access control, either `allow` or `deny`, is then taken. + +If no access controls are defined, or none match, then the default access +controls are used. These are explicitly defined in the [SPEC](SPEC.html), but +the general effect of them is to require that all commits have one signature +from any of the project's accounts. + +### Access Control Filters + +There are many different filter types, so only the ones used in the tutorial +will be explained. An exhaustive listing can be found in the [SPEC](SPEC.html). + +The `signature` filter matches commits which have a signature credential created +by any one of the specified accounts. The `files_changed` filter matches commits +which have changed files whose paths match the specified patterns (relative to +the project's root). + +### Putting it Together + +The first of the new actions controls you've defined is: + +``` +- action: allow + filters: + - type: signature + account_ids: + - tut + - type: files_changed + pattern: .dehub/* +``` + +This allows any commits which have been signed by `tut` and which modify any of +the files in `.dehub/*`. The second access control is: + +``` +- action: deny + filters: + - type: files_changed + pattern: .dehub/* +``` + +This denies any commits which modify any of the files in `.dehub/*`. If a commit +does not match the first access control, but does match this second access +control, it can be assumed that the commit does _not_ have a signature from +`tut` (because that's the only difference between them). Therefore, the effect +of these two controls put together is to only allow `tut` to make changes to the +`.dehub` directory's files. + +## Step 1: Test the Restrictions + +Let's say that your new user `tot` is having a bit of rebellious phase, and +wants to kick `tut` out of the project. Change `.dehub/config.yml` to have the +following contents (note that `accounts` has been left the same and so is mostly +elided): + +``` +# abbreviated contents of .dehub/config.yml +--- +accounts: + ... + +access_controls: + - action: deny + filters: + - type: signature + account_ids: + - tut +``` + +So edgy. Make the commit for `tot`, being sure that the value for the `--as` +flag indicates you're committing _as_ `tot`: + +``` +git add --all +dehub commit --as tot change --descr 'tut is a butt' +``` + +Somewhat unexpectedly, the commit has been created! You can see it by doing `git +show`. Is dehub broken? + +The fact is that, regardless of whether or not the `dehub` tool allows one to +create this commit, `tut` can create this commit. The important thing is that +`tot` is able to notice that it's been created and do something about it. In a +real-world situation, both `tot` and `tut` would be using different computers, +and when `tot` (or anyone else) receives the commit from `tut` they will try to +verify it, fail to do so, and ignore it. + +If you perform `dehub verify` you will be greeted with the following error: + +``` +exiting: blah blah blah: commit matched and denied by this access control: +action: deny +filters: +- type: files_changed + pattern: .dehub/* +``` + +Because the parent of this commit's config disallows this commit (via the given +access control) it is not verifiable. Go ahead and delete the commit by doing: + +``` +git reset --hard "$(git rev-list HEAD | tail -3 | head -n1)" +``` + +## Step 2: Different Restrictions + +In light of `tot`'s recent actions it might be prudent to pull back their +permissions a bit. Go ahead and change the `.dehub/config.yml` to: + + +``` +# abbreviated contents of .dehub/config.yml +--- +accounts: + ... + +access_controls: + - action: allow + filters: + - type: signature + account_ids: + - tot + - type: branch + pattern: tot/* + + - action: deny + filters: + - type: signature + account_ids: + - tot +``` + +and commit the change: + +``` +git add --all +dehub commit --as tut change --descr 'restrict tot to non-main branches' +``` + +After this, `tot` will still be able to interact with the project, but only +within branches whose names have the prefix `tot/`; the `main` branch remains +open to other accounts, such as `tut`, due to the default access controls. + +### Check the New Restrictions + +`tot` has decided to do something constructive and wants to make a shell script +which wraps the `echo` command. So helpful. Make a new branch for `tot` to use, +and create a commit on it: + +``` +git checkout -b tot/echo-script +echo 'echo "$@"' > echo.sh +git add echo.sh +dehub commit --as tot change --descr "added echo.sh script" +``` + +Check that the commit verifies (it should, since it's on a branch with the +prefix `tot/`): + +``` +dehub verify +``` + +Now, as a final sanity check, you'll cherry-pick the commit onto `main` and +ensure that it does _not_ verify there. + +``` +git checkout main +git cherry-pick tot/echo-script +``` + +Running `dehub verify` now should fail, even though the commit remains the same. +The only difference is the branch name; the commit is allowed in branches with +the prefix `tot/`, and disallowed otherwise. + +Finally, reverse that cherry-pick to make main verifiable again: + +``` +git reset --hard "$(git rev-list HEAD | tail -4 | head -n1)" +``` + +You now have an understanding of how dehub's access controls work. Access +controls are extremely flexible and can be formulated to fit a wide-variety of +use-cases. In [Tutorial 3](tut3.html) we'll see how access controls can be +formulated to allow for commit sign-offs, where multiple accounts must accredit +a commit before it can be verified, and how such a commit can be created. diff --git a/docs/tut3.md b/docs/tut3.md new file mode 100644 index 0000000..24e494e --- /dev/null +++ b/docs/tut3.md @@ -0,0 +1,247 @@ +# Tutorial 3: Commit Sign-Off + +Commit sign-off is a common pattern in vanilla git projects, where a commit must +be approved by one or more people (besides the commit author themselves) in +order to be allowed into the primary branch. + +dehub is able to accomplish this same pattern using only the access controls +which have already been covered in this tutorial series and a command which has +not: `dehub combine`. This tutorial will guide you through using `dehub combine` +to facilitate commit sign-off. + +This tutorial assumes you have already completed [Tutorial 2](tut2.html), and +builds on top of the project which was started there. + +## Step 0: Loosen the Previous Restrictions + +In the [previous tutorial](tut2.html) you took an existing project, added a new +user `tot` to it, and then restricted `tot` to only be allowed to make commits +in a certain subset of branches which excluded the `main` branch. + +As seen in that tutorial, `tot` is not able to create commits for the `main` +branch _at all_. In this tutorial we're going to open `main` back up to `tot`, +but only with a very important caveat: `tot`'s commits must be approved by +someone else. + +In the `hello-world` project which was used for previous tutorials, with the +`main` branch checked out, go ahead and modify `.dehub/config.yml` to have the +following contents: + +``` +# contents of .dehub/config.yml +--- +accounts: + - id: tut + signifiers: + - type: pgp_public_key_file + path: ".dehub/tut.asc" + - id: tot + signifiers: + - type: pgp_public_key_file + path: ".dehub/tot.asc" + +access_controls: + - action: allow + filters: + - type: signature + account_ids: + - tot + - type: branch + pattern: tot/* + + - action: deny + filters: + - type: branch + pattern: main + - type: not + filter: + type: signature + any_account: true + count: 2 +``` + +and commit the changes: + +``` +git add .dehub/config.yml +dehub commit --as tut change --descr 'require commit sign-offs in main' +``` + +The primary change was to replace the old access control denying `tot` the +ability to commit to anything (outside of `tot/*` branches) with this one: + +``` +- action: deny + filters: + - type: branch + pattern: main + - type: not + filter: + type: signature + any_account: true + count: 2 +``` + +There are two new things here. The first is the new fields on the `signature` +filter: `any_account` replaces the `account_ids` field, and refers to any +account which is defined in the `accounts` section; `count` declares how many +accounts must have a signature on the commit for the filter to match (if not +specified it defaults to 1). + +The second new thing is the `not` filter: `not` wraps any other filter, and +reverses whether or not it matches. In this case, it's wrapping our `signature` +filter, such that this access control will match only if the commit _does not_ +have signature credentials from 2 different accounts. + +The total effect of this access control is to deny any commits to `main` which +do not have signature credentials from 2 different accounts. In effect, commit +sign-off. + +## Step 1: Some Changes to Merge + +In the previous tutorial `tot` created a new script, `echo.sh`, in a new branch +called `tot/echo-script`. Check that branch out, rebase it on `main` (this will +help in later steps), and add another script to it: + +``` +git checkout tot/echo-script +git rebase main +echo 'echo "$@" | awk "{ print toupper(\$0) }"' > echo-upper.sh +git add echo-upper.sh +dehub commit --as tot change --descr 'echo-upper.sh' +``` + +Now the `tot/echo-script` branch contains two commits which aren't on `main`, +both of them signed by `tot`. What will happen next is that the branch's commits +will be combined into a single commit, be given accreditation by both `tut` and +`tot`, and added to the `main` branch. + +## Step 2: Accreditation + +First, `tot` will accredit both commits, and unify the two descriptions in the +process. To do this, you will create your first `credential` commit: + +``` +dehub commit --as tot credential --start HEAD^^ --descr 'add echo.sh and echo-upper.sh' +``` + +A `credential` commit, at its core, contains nothing except credentials for any +arbitrary fingerprint. To view the credential commit you just made +do: `git show`. You should see a commit message like: + +``` +Credential of AO3dn4Se61hq6OWy4Lm6m3MxdT2ru6TrIobuHaWJJidt + +--- +type: credential +commits: +- f085f13fa839ece122476601d970460ac249dc69 # these will be different +- 40a81ffb4f52dc4149570672f7f7fc053f12226a +change_description: add echo.sh and echo-upper.sh +fingerprint: AO3dn4Se61hq6OWy4Lm6m3MxdT2ru6TrIobuHaWJJidt +credentials: +- type: pgp_signature + pub_key_id: XXX + body: BIG LONG STRING + account: tot +``` + +You'll notice that the credential commit's fingerprint is different than either +of the two commits it accredits. This is the fingerprint is based on the +_combination_ of the two commits; it is based on the total of the file changes +and the description provided by the user. The two commits are enumerated in the +`commits` field of the payload, and the description provided by the user is +stored in the `change_description` field. + +The combined commits have now been accredited by `tot`, but not `tut`, and so +they still lack a necessary credential. Have `tut` make a credential now: + +``` +dehub commit --as tut credential --rev HEAD +``` + +This form of the `credential` sub-command only accredits a single commit. When a +single commit is accredited and it itself is a credential commit then the new +commit which is created is merely a copy of the specified credential commit with +the caller's own credential appended to the `credentials` list. You can see this +with `git show`, which should look like: + +``` +Credential of AO3dn4Se61hq6OWy4Lm6m3MxdT2ru6TrIobuHaWJJidt + +--- +type: credential +commits: +- f085f13fa839ece122476601d970460ac249dc69 # these will be different +- 40a81ffb4f52dc4149570672f7f7fc053f12226a +change_description: add echo.sh and echo-upper.sh +fingerprint: AO3dn4Se61hq6OWy4Lm6m3MxdT2ru6TrIobuHaWJJidt +credentials: +- type: pgp_signature + pub_key_id: XXX + body: BIG LONG STRING + account: tot +- type: pgp_signature + pub_key_id: XXX + body: BIG LONG STRING + account: tut +``` + +There is now enough credentials to combine both commits in the `tot/echo-script` +branch into a single commit on the `main` branch. + +## Step 3: Combination + +At this point the `tot/echo-script` branch has the following elements in place: + +* Two change commits, which we want to combine and bring over to `main`. +* A credential commit made by `tot` for the combined changes. +* A credential commit made by `tut` for the combined changes, which includes + `tot`'s credentials. + +Combining the commits and placing them on `main` is done with a single command: + +``` +dehub combine --start HEAD^^^^ --end HEAD --onto main +``` + +This `combine` command combines all changes made within the given commit range, +the last change description found in that range (in this case it will be from +`tut`'s credential commit), and all credentials for that set of changes. The +command combines them into a single commit which it places on the `main` branch. +You can see the commit you've just created by doing: + +``` +git checkout main +git show +``` + +The commit should contain both of the new files, and the message should look +something like: + +``` + add echo.sh and echo-upper.sh + + --- + type: change + description: add echo.sh and echo-upper.sh + fingerprint: ALOcEuKJkgIdz27z0fjF1NEbK6Y9cEh2RH4/sL3uf3oa + credentials: + - type: pgp_signature + pub_key_id: XXX + body: BIG LONG BODY + account: tot + - type: pgp_signature + pub_key_id: XXX + body: BIG LONG BODY + account: tut +``` + +The commit is accredited by two different accounts, and so is allowed to be on +the `main` branch. This can be verified by doing `dehub verify`. + +You now are able to require commit sign-off and create signed-off commits! The +access control settings surrounding commit sign-offs are entirely up to you and +your project's needs. You can require sign-off from specific accounts, any +accounts, only on specific files, only in certain branches, etc... all using the +same basic access control building blocks.