diff --git a/INTRODUCTION.md b/INTRODUCTION.md index 824129a..52ff2a7 100644 --- a/INTRODUCTION.md +++ b/INTRODUCTION.md @@ -37,6 +37,12 @@ By embedding project meta-information into git messages, as yaml encoded data structures, dehub is able to incept all the features generally provided by git platforms into the git history itself, including dehub's own configuration. +By doing this, the server-side git component can be reduced to a mere +pre-receive hook (if anything at all). This opens the door for much more +lightweight and flexible hosting of git projects, and even more radical +solutions like hosting git projects on completely decentralized platforms like +IPFS. + ### Example MyProject wants to ensure that at least 2 of the 3 maintainers sign off on a @@ -64,6 +70,7 @@ A commit in the `trunk` branch would have a message with the following form: ``` This is the first line of the commit message. It remains human readable +--- type: change message: | This is the first line of the commit message. It remains human readable @@ -93,23 +100,25 @@ The `credentials` contains signatures of both the commit message and its changes, allowing it to be added to the `trunk`. A simple git hook is all that's needed to verify commits in `trunk` when they are pushed or pulled. -## dehub Branches +## dehub Thread Branches The `trunk` branch is the project's source-of-truth; all commits in it must have -dehub encoded message bodies with acceptable credentials. Other branches are -used to coordinate new changes, and then coalesce those changes into a commit -suitable for `trunk`. +dehub encoded message bodies with acceptable credentials. Other branches, called +threads, are used to coordinate new changes, and then coalesce those changes +into a commit suitable for `trunk`. ### Example -Alice creates and pushes a branch on the git repo called `featureBranch`, and -pushes to it a commit with the following commit message: +Alice creates and pushes a thread branch on the git repo called `featureBranch`, +and pushes to it a commit with the following commit message: ``` This commit adds some really cool features +--- type: change message: This commit adds some really cool features +change_hash: SOMECHANGEHASH credentials: - type: pgp_signature pub_key_id: 01234 @@ -120,12 +129,13 @@ credentials: # trunk branch. ``` -Bob sees the new branch and looks through it. He pushes the following commit -(with no file changes): +Bob sees the new thread branch and looks through it. He pushes the following +commit (with no file changes): ``` A small comment +--- type: comment message: | A small comment @@ -143,18 +153,20 @@ credentials: ``` Alice sees Bob's comment, and agrees with his suggestion. She pushes a new -commit, which contains a slight modification of the original commit message plus -the suggested changes: +commit to the thread, which contains a slight modification of the original +commit message plus the suggested changes: ``` This commit adds some really cool features +--- type: change message: | This commit adds some really cool features The pattern used at file:line was suggested by Bob. Thanks Bob! +change_hash: NEWCHANGEHASH credentials: - type: pgp_signature pub_key_id: 01234 @@ -165,25 +177,29 @@ credentials: # trunk branch. ``` -Bob, happy with these changes, pushes a commit to `featureBranch` which adds his -own signature for the latest commit message and all file changes in the branch: +Bob, happy with these changes, pushes a commit to the thread which adds his own +signature for the latest commit message and all file changes in the branch: ``` bob's signature for this branch's changes -type: pgp_signature -pub_key_id: 01234 -body: SIGNATUREBODY -account: bob +--- +type: credential +change_hash: NEWCHANGEHASH +credentials: + - type: pgp_signature + pub_key_id: 56789 + body: SIGNATUREBODY + account: bob ``` -_Finally_ the feature branch is ready to be coalesced, which is a step anyone +_Finally_ the thread branch is ready to be coalesced, which is a step anyone can do once all the required credentials are available. -To coalesce, the following is done: All changes in the branch are squashed into -a single commit, using the latest commit message which was pushed by Alice. -Bob's signature is added to the commit structure as a credential. The commit can -then be pushed to `trunk` (because it now has two credentials) and +To coalesce, the following is done: All file changes in the branch are squashed +into a single change commit, using the latest commit message which was pushed by Alice. +Bob's signature is added to the change commit message as a credential. The +commit can then be pushed to `trunk` (because it now has two credentials) and `featureBranch` can be deleted. ## Pre-emptively Answered Questions diff --git a/SPEC.md b/SPEC.md index 5d3f844..d6b1ce7 100644 --- a/SPEC.md +++ b/SPEC.md @@ -65,34 +65,90 @@ access_controls: count: 1 ``` -# Trunk Commit +# Change Hash -All new commits being appended to the HEAD of the `trunk` branch are subject to -the following requirements: +When a change commit (see Commits section) is being signed by a signifier there +is an expected data format for the data to be signed. The format is a SHA-256 +hash of the following pieces of data concatenated together (the change hash): -* Must conform to all requirements defined by the `access_controls` section of - the `config.yml`, as found in the HEAD. If the commit is the initial commit of - the branch then it instead uses the `config.yml` found in itself. +* A uvarint indicating the number of bytes in the commit message. +* The message. +* A uvarint indicating the number of files changed. +* For each file changed in the commit, ordered lexographically-ascending based + on its full relative path within the repo, the following is then written: + * A uvarint indicating the length of the full relative path of the file + within the repo. + * The full relative path of the file within the repo. + * A little-endian uint32 representing the previous file mode of the file (or 0 + if the file is being inserted). + * The 20-byte SHA1 hash of the previous version of the file's contents (or 20 + 0 bytes if the file is being inserted). + * A little-endian uint32 representing the new file mode of the file (or 0 + if the file is being deleted). + * The 20-byte SHA1 hash of the new version of the file's contents (or 20 + 0 bytes if the file is being deleted). -* Must not be a merge commit (this may be amended later, but at present it - simplifies implementation). +The raw output from the SHA-256 is then prepended with a `0` byte (for forward +compatibility). The result is the raw change hash. -* The commit message must conform to the format and semantics defined below. +# Credentials -## Trunk Commit Message +All file changes need to have some kind of credential to be accepted into the +`trunk` branch (see Trunk Branch section). Each credential is encoded as a yaml +object with a `type` field. -The commit message for a commit being appended to the HEAD of the `trunk` -branch must conform to the following format: a single line (the message head) -giving a short description of the change, then two newlines, then a body which -is a yaml formatted string: +All credentials contain enough information to correspond them to a specific +signifier in the `config.yml`, so as to be able to verify them. -```yaml -This is the message head. It will be re-iterated within the yaml body. +## PGP Signature Credential + +Currently there is only a single credential type, the `pgp_signature`, which +signs a raw change hash (which is communicated out-of-band of the object): + +``` +type: pgp_signature +account_id: some_user_id +pub_key_id: XXX +body: "base-64 signature body" +``` + +# Commits + +All commit messages in dehub repositories are expected to follow the following +template (newlines included, yaml comments start with `#` and are only for +informational purposes): + +``` +Human readable message head -# Now the yaml body begins --- +# Three dashes indicate the start of the yaml body. Everything after must be +# valid yaml. + +type: type of the commit # Always required +fieldA: valueA +fieldB: valueB +``` + +## Change Commits + +Commits of type `change` correspond to the standard git commit; they encompass a +set of file changes as well as a message describing the changes which occurred. +They extend the standard git commit with a few dehub specific features, such as +the change hash and credentials. + +`change` commits are, currently, the _only_ commit type which are allowed to +have file changes. + +Example change commit message: + +``` +This is the message head. It will be re-iterated within the message field + +--- +type: change message: > - This is the message head. It will be re-iterated within the yaml body. + This is the message head. It will be re-iterated within the message field The rest of this field is for the message body, which corresponds to the body of a normal commit message which might give a more long-form @@ -103,91 +159,71 @@ message: > duplicated. Otherwise the exact bytes of the message would be ambiguous. This situation is ugly, but not unbearable. -# See the Commit Signatures section below for how this is computed. The -# change_hash is always recomputed when verifying a commit, but is reproduced in -# the commit message itself for cases of forward compatibility, e.g. if the -algorithm to compute the hash changes. +# The change_hash is able to be computed from the commit's message and changed +# files, but is reproduced in the commit message for forward compatibility, e.g. +# if the algorithm to compute the hash changes. change_hash: XXX -# Credentials are the set of credentials which count towards requirements -# specified in the `access_controls` section of the `config.yml` file. +# Credentials are the set of credentials which indicate approval of the change credentials: - - type: pgp_signature account_id: some_user_id pub_key_id: XXX body: "base-64 signature body" ``` -## Commit Signatures +## Credential Commits -When a commit is being signed by a signifier there is an expected data format -for the data to be signed. The format is a SHA-256 hash of the following pieces -of data concatenated together (the "change_hash"): +Commits of type `credential` contain one or more credentials for some set of +changes, and the change hash to which those credentials apply. The commit +message head is not spec'd, but should be a human-readable description of "who +is crediting what, and how". -* A uvarint indicating the number of bytes in the commit message. -* The message. -* A uvarint indicating the number of files changed. -* For each file changed in the commit, ordered lexographically-ascending based - on its full relative path within the repo, the following is then written: - * A uvarint indicating the length of the full relative path of the file - within the repo. - * The full relative path of the file within the repo. - * A little-endian uint32 representing the previous file mode of the file (or 0 - if the file is being inserted). - * The 20-byte SHA1 hash of the previous version of the file's contents (or 20 - 0 bytes if the file is being inserted). - * A little-endian uint32 representing the new file mode of the file (or 0 - if the file is being deleted). - * The 20-byte SHA1 hash of the new version of the file's contents (or 20 - 0 bytes if the file is being deleted). +Example credential commit message: -The raw output from the SHA-256 is then prepended with a `0` byte (for forward -compatibility) and signed, and the result used as the signature body. +``` +some_user_id pgp sig of commits AAA..BBB with key CCC + +--- +change_hash: XXX +credentials: + - type: pgp_signature + account_id: some_user_id + pub_key_id: CCC + body: "base-64 signature body" +``` -# Merge Requests +# Branches -A merge request (MR) may be pushed to the repository as a new branch at any -time. All MR branch names follow the naming convention `DHMR-short-description`. -An MR branch has the following qualities: +dehub branches correspond 1-to-1 with branches in the underlying git repo. All +commits in a dehub branch should contain an encoded message as specified in the +Commits section of this document, and possibly file changes as appropriate. -* Meta commits (see sub-section) will only contain a commit message head/body, - but no file changes. +## Trunk Branch -* The most recent substantial commit (as opposed to meta commits) should always - contain the full commit message head and body. +The "primary" branch of a dehub repo is the `trunk` branch. All new commits +being appended to the HEAD of the `trunk` branch are subject to the following +requirements: -## Meta Commits +* Must be `change` commits. -Meta commits are those which add information about the changes being requested, -but do not modify the changes themselves. +* Must conform to all requirements defined by the `access_controls` section of + the `config.yml`, as found in the current HEAD. If the commit is the initial + commit of the branch then it instead uses the `config.yml` found in itself. -### Signature Commits +* Must be a "fast-forward" commit (this may be amended later, but at present it + simplifies implementation). -Signature commits sign the changes requested in order to count towards their -access control requirements. The message head of these are arbitrary, but the -body must be formatted as such: -```yaml -# This object matches the one found in the `credentials` section of the trunk -# commit message. -type: pgp_signature -account_id: some_user_id ``` -pub_key_id: XXX -body: "base-64 signature body" # see Commit Signatures sub-section. -``` +## Thread Branches -If a signature commit is added to a MR branch, and a substantial commit is -added after it, then that signature commit will no longer be valid, as it was -only signing the the prior changeset. The signer will need to create and push a -new signature commit, if they agree with the new changes. +Branches which are not the `trunk` branch are referred to as "threads", and have +much less stringent requirements than the `trunk` branch: -## Merging MRs +* They can contain commits of any type, as long as the commits come from those + with an account defined in the `config.yml`. -When an MR has accumulated enough meta commits to fulfill access control -requirements it may be coalesced into a single commit destined for the `trunk` -branch. See the Trunk Commit Message sub-section for details on how commit -messages in the `trunk` branch must be formatted. +* `change` commits are not subject `access_controls` requirements. # TODO