mediocregopher/dehub
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Refactor all documentation

Browse Source 
---
type: change
message: |-
  Refactor all documentation

  * INTRODUCTION.md has been changed into README.md, and been essentially
  rewritten.

  * ROADMAP has been updated significantly.

  * SPEC has been slightly updated, mostly to add a header section indicating that
  it is an incomplete document.
change_hash: ALqHmxfJSBVUIHrI4B9bGjeQpKvBUwL6GPmkDfjhu9uf
credentials:
- type: pgp_signature
  pub_key_id: 95C46FA6A41148AC
  body: iQIzBAABAgAdFiEEJ6tQKp6olvZKJ0lwlcRvpqQRSKwFAl6BIBQACgkQlcRvpqQRSKxguw//YGGDA/UJTVMPAkW8FeX+Pw8GzyW/nnF+f8HOo2/z56GA+llXVDwAyTXPJO3MIUBwPfnN1mLxANDClpofgKuy5I/uXD0ZEN8P4C8zyhv/OQIrgbkEa3iCnHgKFXmdn8URoeb8Bpcb5za/6VN/EwhkQlXGdwqppWcBMCuBx555AJRCIuC0LmOaVVSulbWTr4dAPU0OGodKqXxB5Je+mVzMJ43JQPw7nT5AxnN/CuEG6KsmtfRJISg6BwIVaVipoBGCNPA8Qsf6MC7RK2bCaQgxlQk3kVC46hNaGHlCbfOJgSMHwz4ivhk6tSCmfqzDQOfSQN4FQ0y9LghvFiuBI2PszytEVnYQZlX2vvXBCVa3k0BMNktCs1BKMUAxfZKJIg6YsfWlGt7hlbEgd/YvypC0JCLe3AEh7pILf2UQnxD9VWQhcpHiivXapSKgsoN1qgLgfRqQvFBAz9Wy1PMe3ySnUsBQ3hEdwqWXtQILtHRrhe9TzvMF+zQYQb0bQPM09tR0MicQD7bjzo3F0OOvudRBAAA+5W9uOEpThUVi6hJa2pTcH38ZAHcISENwqOahran4TPZ2eGyl6afmkzjjrMe3THo6rZg+U+P1MQe/2K8JItSPH98fD/eeO74QUAgq/SLIa24MqaCSFIiJ3I1xgnB5o1b8/xhAcodOUFKdgwGtdXE=
  account: mediocregopher
This commit is contained in:
mediocregopher 2020-03-29 16:24:44 -06:00
parent 4d56716fe8
commit d5f172875e
4 changed files with 182 additions and 318 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

230
INTRODUCTION.md
View File

@ -1,230 +0,0 @@
# dehub
**Embed project coordination into your git history.**
## Gettin Started
```
git clone https://dehub.mediocregopher.com/dehub.git
```
and check out the project! dehub is still very very alpha, but it will be
"eating its own dogfood" from the start.
Check out the `cmd/http-server` directory if you'd like to host your own.
## Motivation
Any active git project has a set of requirements which are not met by the git
protocol directly, for example:
* Authenticating committers
* Some kind of ticket system for bugs and proposals
* Change reviews
* Signoff of changes by one or more maintainers
* Release management (git tags are mutable and therefore generally ineffective)
To solve these requirements developers generally turn to centralized services
like GitHub or Bitbucket, or self-hosted server solutions like Gitlab or gogs.
These platforms become a point of hindrance; their sheer size makes developers
dependent on the platform developers to implement features and fix bugs, as well
as making developers dependent on non-trivial amounts of devops (whether
provided by the service, or self-hosted) in order to function.
## Enter dehub
By embedding project meta-information into git messages, as yaml encoded data
structures, dehub is able to incept all the features generally provided by git
platforms into the git history itself, including dehub's own configuration.
By doing this, the server-side git component can be reduced to a mere
pre-receive hook (if anything at all). This opens the door for much more
lightweight and flexible hosting of git projects, as well as even more radical
solutions; dehub can enable hosting git projects on completely decentralized
platforms like IPFS.
### Example
MyProject wants to ensure that at least 2 of the 3 maintainers sign off on a
commit which changes files before the commit can be placed into the `main`
branch (dehub's equivalent of the `master` branch). MyProject's repo would
contain a `.dehub/config.yml` file with the following access controls set:
```
# ...
access_controls:
- action: allow
filters:
- type: branch
pattern: main
- type: commit_type
commit_type: change
- type: signature
account_ids:
- alice
- bob
- carol
count: 2
- action: deny
filters:
- type: branch
branch: main
```
A commit in the `main` branch would have a message with the following form:
```
This is the first line of the commit message. It remains human readable
---
type: change
message: |
This is the first line of the commit message. It remains human readable
The rest of the message body is a yaml encoded object. The message field of
that object repeats the first line of the commit message, followed by the
rest of the commit message (if there is one). The first line is duplicated
so that commands like `git log` are more usable, while at the same time
allowing the full commit message to be signed off on.
# A hash of the diff between the previous commit and this one.
change_hash: ABCDEFGHIJKLMNOPQRSTUVWXYZ
credentials:
- type: pgp_signature
pub_key_id: 01234
body: SIGNATUREBODY
account: alice
- type: pgp_signature
pub_key_id: 56789
body: SIGNATUREBODY
account: carol
```
The `credentials` contains signatures of both the commit message and its
changes, allowing it to be added to the `main`. A simple git hook is all that's
needed to verify commits in `main` when they are pushed or pulled.
## dehub Thread Branches
The `main` branch is the project's source-of-truth. Other branches, called
threads, are used to coordinate new changes, and then coalesce those changes
into a commit suitable for `main`.
### Example
Alice creates and pushes a thread branch on the git repo called `featureBranch`,
and pushes to it a commit with the following commit message:
```
This commit adds some really cool features
---
type: change
message: This commit adds some really cool features
change_hash: SOMECHANGEHASH
credentials:
- type: pgp_signature
pub_key_id: 01234
body: SIGNATUREBODY
account: alice
# Note that this commit does not have enough credentials to be allowed in the
# main branch.
```
Bob sees the new thread branch and looks through it. He pushes the following
commit (with no file changes):
```
A small comment
---
type: comment
message: |
A small comment
I think you should change the code at file:line to be more like the code at
otherFile:otherLine
# Comment credentials sign the comment itself, so you can be sure of its
# authenticity.
credentials:
- type: pgp_signature
pub_key_id: 01234
body: SIGNATUREBODY
account: bob
```
Alice sees Bob's comment, and agrees with his suggestion. She pushes a new
commit to the thread, which contains a slight modification of the original
commit message plus the suggested changes:
```
This commit adds some really cool features
---
type: change
message: |
This commit adds some really cool features
The pattern used at file:line was suggested by Bob. Thanks Bob!
change_hash: NEWCHANGEHASH
credentials:
- type: pgp_signature
pub_key_id: 01234
body: NEWSIGNATUREBODY
account: alice
# Note that this commit does not have enough credentials to be allowed in the
# main branch.
```
Bob, happy with these changes, pushes a commit to the thread which adds his own
signature for the latest commit message and all file changes in the branch:
```
bob's signature for this branch's changes
---
type: credential
change_hash: NEWCHANGEHASH
credentials:
- type: pgp_signature
pub_key_id: 56789
body: SIGNATUREBODY
account: bob
```
_Finally_ the thread branch is ready to be coalesced, which is a step anyone
can do once all the required credentials are available.
To coalesce, the following is done: All file changes in the branch are squashed
into a single change commit, using the latest commit message which was pushed by
Alice. Bob's signature is added to the change commit message as a credential.
The commit can then be pushed to `main` (because it now has two credentials) and
`featureBranch` can be deleted.
## Pre-emptively Answered Questions
**How can I trust that the git history I've received is legitimate?**
Each commit in `main` can have its credentials verified locally. Credentials
are currently provided by pgp signatures, so your trust in the git chain can be
as strong as your trust in those signatures. Support for other kinds of
credentials (e.g. keybase signatures) will increase the number of options for
trust the user has.
**Why `main`?**
The primary branch in most git projects is called `master`. It makes sense to
use a different one, `main`, for dehub, since the commits on it will be
following a specific protocol which is not compatible with most `master`
branches. By having a different primary branch convention we can prevent undue
conflict, as well as make it easy to tell at a glance what kind of project is
being worked with.

97
README.md Normal file
View File

@ -0,0 +1,97 @@
# dehub
dehub aims to provide all the features of a git hosting platform, but without
the hosting part. These features include:
**User management** - Authentication that commits come from the user they say
they do, and fine-grained control over which users can do what.
**Pull requests and issues** - Facilitation of discussion via comment commits,
and fine-grained (down to the file level) sign-off requirements.
**Tags and releases**\* - Mark releases in the repo itself, and provide
immutable and verifiable git tags so there's never any funny business.
**Plugins**\*: Extend all aspects of dehub functionality via executables managed
in the repo itself (in the same style as git hooks).
## Key Concepts
To implement these features, dehub combines two key concepts:
First, repo configuration is defined in the repo itself. A file called
`.dehub/config.yml` contains all information related to user accounts, their pgp
keys, branch and file level access controls, and more. Every commit must adhere
to the configuration of its parent in order to be considered _verifiable_. The
configuration file is committed to the repo like any other file would be, and so
is even able to define the access controls on itself.
Second, the commit message of every dehub commit contains YAML encoded metadata,
which allows dehub to extend git and provide multiple commit types, each with
its own capabilities and restrictions. Some example dehub commit types are
`change` commits, `comment` commits, and `credential` commits.
## Infrastructure (or lack thereof)
Because a dehub project is entirely housed within a traditional git project,
which is merely a collection of files, any existing git or network filesystem
infrastructure can be used to host any dehub project:
* The most barebones [git
daemon](https://git-scm.com/book/en/v2/Git-on-the-Server-Git-Daemon) server
(with a simple pre-receive hook set up).
* A remote SSH endpoint.
* A mailing list (aka the old school way).
* Network file syncing utilities such as dropbox,
[syncthing](https://github.com/syncthing/syncthing), or
[NFS](https://en.wikipedia.org/wiki/Network_File_System).
* Existing git project hosts like GitHub, Bitbucket, or Keybase.
* Decentralized filesystems such as IPFS\*.
_\* Planned feature, but not yet implemented._
# Getting Started
The dehub project itself can be found by cloning
`https://dehub.dev/src/dehub.git`.
Installation of the dehub tool is currently done via the `go get` command:
```
go get dehub.dev/src/dehub.git/cmd/dehub
```
This will install the binary to your `$GOBIN` path, which you'll want to put in
your `$PATH`. Run `go env` if you're not sure where your `$GOBIN` is.
Once installed, running `dehub -h` should show you the help output of the
command. You can continue on to the tutorials if you're not sure where to go
from here.
## Tutorials
The following tutorials will guide you through the basic usage of dehub. As
dehub is still very much in development a high level of git and PGP profiency is
still required in order to use dehub effectively.
TODO
## Documentation
The [SPEC](/SPEC.html) is the best place to see every possible nitty-gritty
detail of how dehub works. It attempts to be both human-readable and exhaustive
in its coverage.
## Other links
[ROADMAP](/ROADMAP.html) documents upcoming features and other work required on
the project. If you're looking to contribute, this is a great place to start.
[dehub-remote](/cmd/dehub-remote) TODO
[git-http-server](/cmd/git-http-server) TODO

105
ROADMAP.md
View File

@ -4,12 +4,29 @@ This document describes currently planned features and events related to the
dehub project. It's intention is to help prioritize work. There are no dates dehub project. It's intention is to help prioritize work. There are no dates
set, only a sequence of milestones and the requirements to hit them. set, only a sequence of milestones and the requirements to hit them.
## Milestone: Be able to add other developers to the project ## Milestone: Publicize project
* Coalesce command Must be able to feel good about showing the project publicly, as well as be able
* Fast-forward perms on branches to accept help from people asking to help.
* Authorship in commit messages
* README with a "Getting Started" section * Restrict new branches so that they must be ancestors of main.
* Fast-forward perms on branches (so they can be deleted)
* `init` command.
* Ammending commits.
* Figure out commit range syntax, use that everywhere.
* Support short hash names
* Ability to specify a pgp key manually, even if it's not in the project.
* Ability to require _any_ signature on a commit, even if it's not in the
config.
* Create a branch which is just a public "welcome thread", which can be part of
the tutorials.
* Tutorials
* Update SPEC; it's sloppy, out-of-date, and incomplete.
* Maybe move external host?
## Milestone: IPFS support
* Big ol' question mark on this one.
## Milestone: Versions ## Milestone: Versions
@ -17,50 +34,56 @@ set, only a sequence of milestones and the requirements to hit them.
* Add dehub version to the SPEC, make binary aware of it * Add dehub version to the SPEC, make binary aware of it
* Figure out a release system? * Figure out a release system?
## Milestone: Enough polish to show off the project
* Maybe coalesce the `accessctl`, `fs`, and `sigcred` packages back into the
root "dehub" package.
* Polish all error messages
* Polish commands
- New flag system, some kind of interactivity support (e.g. user doesn't
specify required argument, give them a prompt on the CLI to input it
rather than an error). This is partially done, in that a new flag system
has been started. Needs further work.
- Review flags, probably make some of them into positional arguments,
document everything better.
- Possibly save state locally in order to speed things along, such as
"account id" which probably isn't going to change often for a user.
* HTTP docs
- Some kind of addition to the docker server to auto-serve md files.
- Reorganize all the md files under a `docs` directory, except the README.
- Make sure linking between documents works.
* More/better tests?
## Milestone: IPFS support
* Big ol' question mark on this one.
## Milestone: Minimal plugin support ## Milestone: Minimal plugin support
* SPEC and implement. Things which should be pluggable, initially: * SPEC and implement. Things which should be pluggable, initially:
- Conditions * Conditions
- Signifiers * Signifiers
* Filters
* Commits???
## Milestone: Minimal notifications support ## Milestone: Minimal notifications support
* Some way to store notification settings locally, and run a command which shows * Some way to store notification settings locally, and run a command which shows
a sequence of events since the last time you ran it. a sequence of events since the last time you ran it.
- The command should keep a history of all of its outputs, and allow the * The command should keep a history of all of its outputs, and allow the
user to see that history (in case they run the command, then clear the user to see that history (in case they run the command, then clear the
output by accident). output by accident).
- The user should be able to specifically get notifications on threads * The user should be able to specifically get notifications on threads
they're a part of, threads by branch name pattern, files by path they're a part of, threads by branch name pattern, files by path pattern,
pattern, and keywords in commit messages. and keywords in commit messages.
# Misc Polish
These tasks aren't necessarily scheduled for any particular milestone, but they
are things that could use doing anyway.
* Maybe coalesce the `accessctl`, `fs`, and `sigcred` packages back into the
root "dehub" package.
* Polish all error messages. A good error message has the following qualities:
* If wrapping an error which was returned from a sub-call:
* Uses `fmt.Errorf` with the `%w` format directive at the end.
* Phrased as if the sentence starts with the word "while", e.g. "opening
file: %w".
* Only includes information the caller of that function/method couldn't
already know.
* Polish commands
* New flag system, some kind of interactivity support (e.g. user doesn't
specify required argument, give them a prompt on the CLI to input it
rather than an error). This is partially done, in that a new flag system
has been started. Needs further work.
* Review flags:
* probably make some of them into positional arguments
* add flag shortcuts
* document everything better.
* POSIX compatible-ish flags?
* Possibly save state locally in order to speed things along, such as
"account id" which probably isn't going to change often for a user.
* More/better tests
* Commits need much better test coverage.

68
SPEC.md
View File

@ -1,9 +1,15 @@
# .dehub # SPEC
This document attempts to describe, at minimum, every aspect of the dehub
protocol which would be required to know in order to create an alternate
implementation of dehub. It is currently sloppily written and incomplete.
## .dehub
The `.dehub` directory contains all meta information related to The `.dehub` directory contains all meta information related to
decentralized repository management and access control. decentralized repository management and access control.
## config.yml ### config.yml
The `.dehub/config.yml` file takes the following structure: The `.dehub/config.yml` file takes the following structure:
@ -60,7 +66,7 @@ access_controls:
- action: deny - action: deny
``` ```
# Change Hash ## Change Hash
When a change commit (see Commits section) is being signed by a signifier there When a change commit (see Commits section) is being signed by a signifier there
is an expected data format for the data to be signed. The format is a SHA-256 is an expected data format for the data to be signed. The format is a SHA-256
@ -86,12 +92,12 @@ hash of the following pieces of data concatenated together:
The raw output from the SHA-256 is then prepended with a `0` byte (for forward The raw output from the SHA-256 is then prepended with a `0` byte (for forward
compatibility). The result is the raw change hash. compatibility). The result is the raw change hash.
# Comment Message Hash ## Comment Message Hash
When a comment commit (see Commits section) is being signed by the signifier of When a comment commit (see Commits section) is being signed by the signifier of
the author there is an expected data format for the data to be signed, very the author there is an expected data format for the data to be signed, very
similar to how change hashes are signed. The format is a SHA-256 hash of the similar to how change hashes are signed. The format is a SHA-256 hash of the
following pieces of data communicated together: following pieces of data concatenated together:
* A uvarint indicating the number of bytes in the comment message. * A uvarint indicating the number of bytes in the comment message.
* The message. * The message.
@ -99,7 +105,7 @@ following pieces of data communicated together:
The raw output from the SHA-256 is then prepended with a `0` byte (for forward The raw output from the SHA-256 is then prepended with a `0` byte (for forward
compatibility). The result is the raw comment hash. compatibility). The result is the raw comment hash.
# Credentials ## Credentials
All file changes need to have some kind of credential to be accepted into the All file changes need to have some kind of credential to be accepted into the
`main` branch (see Main Branch section). Each credential is encoded as a yaml `main` branch (see Main Branch section). Each credential is encoded as a yaml
@ -108,7 +114,7 @@ object with a `type` field.
All credentials contain enough information to correspond them to a specific All credentials contain enough information to correspond them to a specific
signifier in the `config.yml`, so as to be able to verify them. signifier in the `config.yml`, so as to be able to verify them.
## PGP Signature Credential ### PGP Signature Credential
Currently there is only a single credential type, the `pgp_signature`, which Currently there is only a single credential type, the `pgp_signature`, which
signs a raw change hash (which is communicated out-of-band of the object): signs a raw change hash (which is communicated out-of-band of the object):
@ -120,7 +126,7 @@ pub_key_id: XXX
body: "base-64 signature body" body: "base-64 signature body"
``` ```
# Commits ## Commits
All commit messages in dehub repositories are expected to follow the following All commit messages in dehub repositories are expected to follow the following
template (newlines included, yaml comments start with `#` and are only for template (newlines included, yaml comments start with `#` and are only for
@ -138,7 +144,7 @@ fieldA: valueA
fieldB: valueB fieldB: valueB
``` ```
## Change Commits ### Change Commits
Commits of type `change` correspond to the standard git commit; they encompass a Commits of type `change` correspond to the standard git commit; they encompass a
set of file changes as well as a message describing the changes which occurred. set of file changes as well as a message describing the changes which occurred.
@ -180,7 +186,7 @@ credentials:
body: "base-64 signature body" body: "base-64 signature body"
``` ```
## Credential Commits ### Credential Commits
Commits of type `credential` contain one or more credentials for some hash Commits of type `credential` contain one or more credentials for some hash
(presumably a change hash, but in the future there may be other types). The (presumably a change hash, but in the future there may be other types). The
@ -202,7 +208,7 @@ credentials:
body: "base-64 signature body" body: "base-64 signature body"
``` ```
## Comment Commits ### Comment Commits
Commits of type `comment` contain a message for others to read. The commit Commits of type `comment` contain a message for others to read. The commit
message head is not spec'd, but should be a human-readable description of "who message head is not spec'd, but should be a human-readable description of "who
@ -220,7 +226,7 @@ type: comment
# forwards compatibility. See the Comment Message Hash section. # forwards compatibility. See the Comment Message Hash section.
message_hash: XXX message_hash: XXX
message: > message: >
Heay all, how's it going? Hey all, how's it going?
Just wanted to pop by and say howdy. Just wanted to pop by and say howdy.
@ -233,39 +239,7 @@ credentials:
body: "base-64 signature body" body: "base-64 signature body"
``` ```
# Branches ## TODO
dehub branches correspond 1-to-1 with branches in the underlying git repo. All * Access controls
commits in a dehub branch should contain an encoded message as specified in the * Update credential commit section
Commits section of this document, and possibly file changes as appropriate.
## Main Branch
The "primary" branch of a dehub repo is the `main` branch. All new commits
being appended to the HEAD of the `main` branch are subject to the following
requirements:
* Must be `change` commits.
* Must conform to all requirements defined by the `access_controls` section of
the `config.yml`, as found in the current HEAD. If the commit is the initial
commit of the branch then it instead uses the `config.yml` found in itself.
* Must be a "fast-forward" commit (this may be amended later, but at present it
simplifies implementation).
## Thread Branches
Branches which are not the `main` branch are referred to as "threads", and have
much less stringent requirements than the `main` branch:
* They can contain commits of any type, as long as the commits come from those
with an account defined in the `config.yml`.
* `change` commits are not subject `access_controls` requirements.
# TODO
* access control patterns related to who may push to MR branches, and what types
of commits they can push.