Commit Graph

572 Commits

Author SHA1 Message Date
Alex Auvolat
ddb2b29bfd
Rename datacenters into zones (doc not yet updated) 2021-05-28 14:07:36 +02:00
Alex Auvolat
b490ebc7f6
Many improvements on ring/replication and its configuration:
- Explicit "replication_mode" configuration parameters that takes
  either "none", "2" or "3" as values, instead of letting user configure
  replication factor themselves. These are presets whose corresponding
  replication/quorum values can be found in replication/mode.rs

- Explicit support for single-node and two-node deployments
  (number of nodes must be at least "replication_mode", with "none"
  we can have only one node)

- Ring is now stored much more compactly with 256*8 + n*32 bytes,
  instead of 256*32 bytes

- Support for gateway-only nodes that do not store data
  (these nodes still need a metadata_directory to store the list
  of bucket and keys since those are stored on all nodes; it also
  technically needs a data_directory to start but it will stay
  empty unless we have bugs)
2021-05-28 14:07:36 +02:00
Quentin Dufour
c8aa1eb481
Add preliminary support for Duck 2021-05-15 10:24:20 +02:00
Quentin Dufour
5fdabf3e75
Add basic support for the "Versioning" command 2021-05-14 22:33:26 +02:00
Alex Auvolat
6ccffc3162
Improved XML serialization
- Use quick_xml and serde for all XML response returned by the S3 API.
- Include tests for all structs used to generate XML
- Remove old manual XML escaping function which was unsafe
2021-05-06 22:37:15 +02:00
Trinity Pointard
e4b9e4e24d
rename types to CamelCase 2021-05-03 22:15:09 +02:00
Trinity Pointard
6644df6b96
fix clippy warnings on garage 2021-05-03 22:11:42 +02:00
Trinity Pointard
f8ae8fc4be
fix clippy warnings on web 2021-05-03 22:11:42 +02:00
Trinity Pointard
84856e84e5
fix clippy warnings on api 2021-05-03 22:11:41 +02:00
Trinity Pointard
4a1e079e8f
fix clippy warnings on model 2021-05-03 22:11:41 +02:00
Trinity Pointard
f5a0cf0414
fix clippy warnings on table 2021-05-03 22:11:41 +02:00
Trinity Pointard
f05bb111c2
fix clippy warnings on util and rpc 2021-05-03 22:11:41 +02:00
Trinity Pointard
88925ebe22
add clippy to the CI 2021-05-03 22:11:37 +02:00
Quentin Dufour
631c36b3ff S3 API: support ListBuckets 2021-05-03 21:55:30 +02:00
Alex Auvolat
ee2a3d363b
Remove STREAMING-AWS4-HMAC-SHA256-PAYLOAD (see #64) 2021-05-03 17:30:40 +02:00
Alex Auvolat
575726358c
Tune Sled configuration
- Make sled cache size and flush interval configurable
- Set less agressive default values:
  - cache size 128MB instead of 1GB
  - Flush interval 2 seconds instead of .5 seconds
2021-05-03 17:27:43 +02:00
Alex Auvolat
339c611789
Add links to git/matrix/drone on top of documentation 2021-05-03 10:45:45 +02:00
Quentin Dufour
ef4d6e782a
Add minio & rclone to our functional tests
It is now possible to configure which clients
you do not want to test with the env variable SKIP_XXX=1,
XXX being the client name. eg. SKIP_S3CMD=1 ./script/test-smoke.sh
2021-05-02 14:59:58 +02:00
Alex Auvolat
a981244f11
Replace talk links with permalinks 2021-04-30 15:51:32 +02:00
Alex
b1f60579a1 Merge pull request 'created doc/talks subfolder and added my talk and the previous one done at the wide team' (#67) from talks into main
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/67
2021-04-30 15:47:11 +02:00
LUXEY Adrien
be3b1d8f91 created doc/talks subfolder and added my talk and the previous one done at the wide team 2021-04-30 15:37:43 +02:00
Alex Auvolat
dcfc32cf85
Many S3 compatibility improvements:
- return XML errors
- implement AuthorizationHeaderMalformed error to redirect clients to
  correct location (used by minio client)
- implement GetBucketLocation
- fix DeleteObjects XML parsing and response
2021-04-28 01:05:40 +02:00
Alex
368eb35484 Merge pull request 'Correctly encode ampersand' (#61) from bug/ampersand into main
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/61
2021-04-27 23:15:01 +02:00
Alex Auvolat
642186c530
Fix #59 (& issue) 2021-04-27 23:10:43 +02:00
Quentin Dufour
e01f74e763
Introduce test case that demonstrates #59 (the & problem) 2021-04-27 23:09:26 +02:00
Alex Auvolat
119217f9f6
change a few comments 2021-04-27 16:53:47 +02:00
Trinity Pointard
2812a027ea
change some more comments and revert changes on TableSchema 2021-04-27 16:49:07 +02:00
Trinity Pointard
74373aebcf
make most requested changes 2021-04-27 16:47:08 +02:00
Alex Auvolat
16300bbd89
remove useless comment 2021-04-27 16:44:01 +02:00
Alex Auvolat
f2e1fca658
re-disable drone caches 2021-04-27 16:37:11 +02:00
Alex Auvolat
640e3921d8
Use pre-prepared Docker image in CI to speed things up 2021-04-27 16:37:11 +02:00
Alex Auvolat
7f1a50dbd9
Update drone CI signature 2021-04-27 16:37:11 +02:00
Trinity Pointard
1273802994
use volume for cargo home
so it gets preserved between steps, and registry download is done only once
2021-04-27 16:37:11 +02:00
Alex Auvolat
4af39ab47a
fix drone CI signature 2021-04-27 16:37:11 +02:00
Alex Auvolat
1c97b82c45
Skip Drone CI caching 2021-04-27 16:37:11 +02:00
Trinity Pointard
5c4b2cec3c
document garage crate 2021-04-27 16:37:10 +02:00
Trinity Pointard
1e3df189d0
document api crate 2021-04-27 16:37:10 +02:00
Trinity Pointard
a2e1617d84
document web crate 2021-04-27 16:37:10 +02:00
Trinity Pointard
67585a4ffa
attempt at documenting model crate 2021-04-27 16:37:10 +02:00
Trinity Pointard
b437610812
attempt at documenting table crate 2021-04-27 16:37:10 +02:00
Trinity Pointard
f871689571
run cargo fmt on util and make missing doc warning 2021-04-27 16:37:10 +02:00
Trinity Pointard
8e0524ae15
document rpc crate 2021-04-27 16:37:10 +02:00
Trinity Pointard
f9bd2d8fb7
document util crate 2021-04-27 16:37:10 +02:00
Alex Auvolat
bf36f1f16a
Update documentation 2021-04-27 16:37:10 +02:00
Alex Auvolat
f7c09daa46
Try to fix Drone 2021-04-27 16:37:09 +02:00
Alex Auvolat
6b2b400292
small simplify 2021-04-27 16:37:09 +02:00
Alex Auvolat
8c33d565d6
Merge discovery loop with consul 2021-04-27 16:37:09 +02:00
Alex Auvolat
948e44a3f6
cargo fmt 2021-04-27 16:37:09 +02:00
Alex Auvolat
3e2e38c830
Print stats 2021-04-27 16:37:09 +02:00
Alex Auvolat
2e53e31cdd
Cargo fmt 2021-04-27 16:37:09 +02:00