Jonathan Davies
f6eaf3661c
garage/Cargo.toml: Updated timeage from 0.3 to 0.4.
2023-01-26 11:13:11 +00:00
Jonathan Davies
d3b2a68988
{garage,util}/Cargo.toml: Updated toml from 0.5 to 0.6.
2023-01-26 11:13:11 +00:00
Jonathan Davies
b4a1a6a32f
util/time.rs: Updated deprecated associated function to timestamp_opt().
2023-01-26 11:13:11 +00:00
Jonathan Davies
bcac889f9a
Cargo.toml: Updated clap from 3.1.18 to 4.1.
2023-01-26 11:13:11 +00:00
Jonathan Davies
9e08a05e69
k2v-client/Cargo.toml: Loosen dependencies.
2023-01-26 11:13:11 +00:00
Jonathan Davies
36944f1839
Cargo.toml: Updated base64 from 0.13 to 0.21.
2023-01-26 11:13:07 +00:00
Jonathan Davies
93c3f8fc8c
api/Cargo.toml: Updated url from 2.1 to 2.3.
2023-01-23 19:16:58 +00:00
Jonathan Davies
1c435fce09
api/Cargo.toml: Updated httpdate from 0.3 to 1.0.
2023-01-23 19:16:54 +00:00
Jonathan Davies
dead123892
api/Cargo.toml: Updated pin-project to 1.0.12.
2023-01-23 18:39:35 +00:00
Jonathan Davies
5c3075fe01
Cargo.toml: Updated zstd from 0.9 to 0.12.
2023-01-23 18:08:14 +00:00
Jonathan Davies
4962b88f8b
tests/s3/website.rs: Added website hosting authorization check tests.
2023-01-13 09:39:02 +00:00
Jonathan Davies
100b01e859
Implemented website hosting authorization endpoint.
...
Fixes : #468
2023-01-13 09:38:58 +00:00
Alex Auvolat
dac254a6e7
Merge branch 'main' into k2v-watch-range-2
2023-01-11 17:09:37 +01:00
Alex
94d723f27c
Merge pull request 'Implement rpc_secret_file
' ( #466 ) from felix.scheinost/garage:feature/implement-rpc-secret-file into main
...
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/466
2023-01-11 16:04:35 +00:00
Alex Auvolat
638c5a3ce0
PollRange: add extra RPC delay after quorum is achieved,
...
to give a chance to the 3rd node to respond
2023-01-11 16:12:07 +01:00
Alex Auvolat
5b5ca63cf6
Poll cleanup
2023-01-11 15:17:27 +01:00
Alex Auvolat
cbfae673e8
PollRange & PollItem: min timeout = 1 sec
2023-01-11 15:03:08 +01:00
Alex Auvolat
bba13f40fc
Correctly return bad requests when seeh marker is invalid
2023-01-11 12:27:19 +01:00
Alex Auvolat
ba384e61c0
PollRange: return immediately if no seen marker is provided
2023-01-11 12:03:17 +01:00
Alex Auvolat
09a3dad0f2
Lock once for insert_many
2023-01-11 11:35:36 +01:00
Alex Auvolat
32aab06929
k2v-client libary poll_range and CLI poll-range
2023-01-11 11:14:29 +01:00
Alex Auvolat
de1111076b
PollRange integration test
2023-01-11 10:04:41 +01:00
Alex Auvolat
b83517d521
Implement PollRange API endpoint
2023-01-10 15:22:25 +01:00
Alex Auvolat
43fd6c1526
PollRange RPC
2023-01-10 12:54:24 +01:00
Alex Auvolat
789540ca37
Type definition for range seen marker
2023-01-10 11:59:57 +01:00
Jonathan Davies
4cfb469d2b
block/metrics.rs: Added compression_level metric.
2023-01-10 10:40:03 +00:00
Jonathan Davies
df1d9a9873
system.rs: Integrated SystemMetrics into System implementation.
2023-01-10 10:39:50 +00:00
Jonathan Davies
aac348fe93
Added system_metrics.rs file.
2023-01-10 10:38:50 +00:00
Alex Auvolat
9f5419f465
Make K2V item timestamps globally increasing on each node
2023-01-10 11:03:52 +01:00
Alex Auvolat
a48e2e0cb2
K2V: Subscription to ranges of items
2023-01-10 10:30:59 +01:00
Felix Scheinost
d6ea0cbefa
Add tests for rpc_secret_file
2023-01-07 14:19:36 +01:00
Felix Scheinost
7b62fe3f0b
Error on both rpc_secret
and rpc_secret_file
2023-01-07 13:49:03 +01:00
Felix Scheinost
f2106c2733
Implement rpc_secret_file
2023-01-04 18:35:10 +01:00
Alex
02e8eb167e
Merge pull request 'PutObject: better cleanup when request is interrupted in the middle' ( #462 ) from interrupted-cleanup into main
...
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/462
2023-01-04 14:43:45 +00:00
Alex Auvolat
29dbcb8278
bg var operation on all nodes at once
2023-01-04 13:25:57 +01:00
Alex Auvolat
f3f27293df
Uniform framework for bg variable management
2023-01-04 13:07:13 +01:00
Alex Auvolat
13c5549886
Remove token_bucket.rs
2023-01-04 11:47:56 +01:00
Alex Auvolat
936b6cb563
When saving block, delete .tmp file if we could not complete
2023-01-03 17:34:26 +01:00
Alex Auvolat
0650a43cf1
PutObject: better cleanup on Drop (incl. when request is interrupted in the middle)
2023-01-03 17:05:17 +01:00
Alex Auvolat
1fc220886a
Fix Consul & Kubernetes discovery with new way of doing background things
2023-01-03 16:55:59 +01:00
Alex Auvolat
1d5bdc17a4
use impossible enum type
2023-01-03 16:04:06 +01:00
Alex Auvolat
c106304b9c
more idiomatic and shorter
2023-01-03 16:00:19 +01:00
Alex Auvolat
33f25d26c7
fix doc and add tests for migrate.rs
2023-01-03 15:53:13 +01:00
Alex Auvolat
d6d571d512
cargo fmt
2023-01-03 15:30:21 +01:00
Alex Auvolat
a54b67740d
move debug_serialize to garage_util::encode
2023-01-03 15:29:29 +01:00
Alex Auvolat
8d5505514f
Make it explicit when using nonversioned encoding
2023-01-03 15:27:36 +01:00
Alex Auvolat
426d8784da
cleanup
2023-01-03 15:08:37 +01:00
Alex Auvolat
cdb2a591e9
Refactor how things are migrated
2023-01-03 14:44:47 +01:00
Alex Auvolat
939a6d67e8
Merge branch 'main' into internals-rework
2023-01-02 15:07:44 +01:00
Alex Auvolat
6775569525
Bump everything to v0.8.1
2023-01-02 14:15:33 +01:00
Alex Auvolat
6b857a9b8c
cargo fmt
2023-01-02 13:50:42 +01:00
Alex
7f7d53cfa9
Merge pull request 'improvements to CLI and new debug features' ( #448 ) from cli-improvements into main
...
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/448
2023-01-02 12:42:24 +00:00
Alex
1af4a5ed56
Merge pull request 'Fix router keyword handling ( fix #442 )' ( #446 ) from router-keywords-fix into main
...
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/446
2022-12-15 08:40:26 +00:00
Alex Auvolat
1fcd0b371b
online repair workers: retry on error
2022-12-14 16:31:31 +01:00
Alex Auvolat
13c8662126
factorize
2022-12-14 16:16:55 +01:00
Alex Auvolat
e6f14ab5cf
better error message handling
2022-12-14 16:11:19 +01:00
Alex Auvolat
510b620108
Get rid of background::spawn
2022-12-14 16:08:05 +01:00
Alex Auvolat
dfc131850a
Simplified and more aggressive worker exit logic
2022-12-14 15:25:29 +01:00
Alex Auvolat
d4af27f920
Add missing notify
2022-12-14 13:54:21 +01:00
Alex Auvolat
a19bfef508
Improve error message on rpc connection failure
2022-12-14 12:57:33 +01:00
Alex Auvolat
d56c472712
Refactor background runner and get rid of job worker
2022-12-14 12:51:42 +01:00
Alex Auvolat
2183518edc
Spawn all background workers in a separate step
2022-12-14 12:28:07 +01:00
Alex Auvolat
83c8467e23
Proper queueing for delayed inserts, now backed to disk
2022-12-14 11:58:06 +01:00
Alex Auvolat
f8e528c15d
Small refactor of tables internals
2022-12-14 10:48:49 +01:00
Alex Auvolat
d1279e04f3
Fix error messages
2022-12-13 16:18:01 +01:00
Alex Auvolat
041b60ed1d
Add block.rc_size, table.size and table.merkle_tree_size metrics
2022-12-13 15:54:03 +01:00
Alex Auvolat
f8d5409894
cli: more info displayed on error in garage stats
2022-12-13 15:46:04 +01:00
Alex Auvolat
d6040e32a6
cli: prettier table in garage stats
2022-12-13 15:43:22 +01:00
Alex Auvolat
d7f90cabb0
Implement block retry-now
and block purge
2022-12-13 15:02:42 +01:00
Alex Auvolat
687660b27f
Implement block list-errors
and block info
2022-12-13 14:23:45 +01:00
Alex Auvolat
9d82196945
cli: new worker info command
2022-12-13 12:24:30 +01:00
Alex Auvolat
a51e8d94c6
cli: rename resync-n-workers into resync-worker-count
2022-12-13 11:44:11 +01:00
Alex Auvolat
de9d6cddf7
Prettier worker list table; remove useless CLI log messages
2022-12-12 17:17:05 +01:00
Alex Auvolat
0e61e3b6fb
Fix bucket creation tests to take permissions into account
2022-12-12 15:47:55 +01:00
Alex Auvolat
a0abf41762
Fix router keyword handling ( fix #442 )
2022-12-12 12:05:37 +01:00
Alex Auvolat
2ac75018a1
Properly enforce allow_create_bucket
2022-12-12 12:03:54 +01:00
Alex Auvolat
533afcf4e1
simplify
2022-12-11 18:17:08 +01:00
Alex Auvolat
5ea5fd2130
Always return 200 OK on /v0/health, reinstate admin api doc as draft and complete it
2022-12-11 18:11:28 +01:00
Alex Auvolat
d7868c48a4
Separate /health (simple text answer) and /v0/health (full json answer, authenticated)
2022-12-05 15:38:32 +01:00
Alex Auvolat
280d1be7b1
Refactor health check and add ability to return it in json
2022-12-05 15:28:57 +01:00
Alex Auvolat
2065f011ca
Implement /health admin API endpoint to check node health
2022-12-05 14:59:15 +01:00
Alex Auvolat
26121bb619
Fix logs appearing twice
2022-11-29 22:23:27 +01:00
Alex Auvolat
54e800ef8d
Tentative fix for issue #414
2022-11-21 17:13:41 +01:00
Alex Auvolat
c1fb65194c
Add sled default in garage_model also
2022-11-21 14:25:54 +01:00
Alex Auvolat
67941000ee
put sled as default feature in garage_db
2022-11-21 14:08:21 +01:00
Quentin Dufour
dc50fa3b34
Fix typo in admin API on BucketInfo
2022-11-11 16:56:56 +01:00
Alex Auvolat
e03d9062f7
Show a nice message and a backtrace when Garage panics
2022-11-04 16:39:02 +01:00
Alex Auvolat
8d3bbf5703
Clearer error messsages
2022-11-04 16:07:33 +01:00
Alex Auvolat
5b18fd8201
Add garage bucket cleanup-incomplete-uploads command
2022-11-04 11:55:59 +01:00
Alex Auvolat
57b5c2c754
Change reqwest rustls features
2022-10-18 22:11:27 +02:00
Alex Auvolat
8bc5caf7aa
Fix issue with 'http(s)://' prefix
2022-10-18 21:17:11 +02:00
Alex Auvolat
2da8786f54
move things around
2022-10-18 19:13:52 +02:00
Alex Auvolat
5d8d393054
Load TLS certificates only once
2022-10-18 19:11:16 +02:00
Alex Auvolat
002b9fc50c
Add TLS support for Consul discovery + refactoring
2022-10-18 18:38:20 +02:00
Alex
5670599372
Merge pull request 'Use status code 204 No Content for empty responses' ( #403 ) from tobikris/garage:http-no-content into main
...
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/403
2022-10-18 14:20:44 +00:00
Tobias Krischer
7865003323
Use status code 204 No Content for empty responses
2022-10-17 10:55:26 +02:00
Alex Auvolat
c050a59fd0
Fix conditional testing in garage_db
2022-10-14 18:27:18 +02:00
Alex Auvolat
fcaee3bea0
definitively expunge openssl from dependencies everywhere
2022-10-14 18:10:36 +02:00
Alex Auvolat
8d04ae7014
cargo2nix unstable (patched), rust 1.63.0, nixpkgs 22.05 (32-bit builds are broken)
2022-10-14 14:30:48 +02:00
Alex Auvolat
ad917ffd3f
Fix instant substractions that might have panicked
2022-09-29 15:53:54 +02:00
Alex Auvolat
1f97ce37e6
Shutdown properly on SIGTERM/SIGHUP and on Windows signals
2022-09-28 10:41:59 +02:00
Alex Auvolat
1778e4b318
Fix span name for api server requests
2022-09-26 16:21:30 +02:00
Alex Auvolat
782630fc27
Initialize metrics exporter earlier ( fix #389 )
2022-09-20 17:50:22 +02:00
Alex Auvolat
ded444f6c9
Ability to have custom timeouts in request strategy (not used)
2022-09-20 16:01:41 +02:00
Alex Auvolat
357b72f4ff
Merge branch 'main' into configurable-timeouts
2022-09-20 15:19:58 +02:00
Alex Auvolat
1f7b050b7d
Change a warn! into a debug!
2022-09-20 11:49:48 +02:00
Alex Auvolat
56592e1853
RPC performance changes
...
- configurable ping timeout
- single, much higher, configurable RPC timeout
- no more concurrency semaphore
2022-09-19 20:31:00 +02:00
Alex Auvolat
5d4b6f2173
Faster GetObject workflow for getting entire objects
2022-09-19 12:19:59 +02:00
Alex Auvolat
76f42a1a2b
Properly return HTTP 204 when deleting non-existent object ( fix #227 )
2022-09-14 17:07:55 +02:00
Alex Auvolat
e46dc2a8ef
Allow for hostnames in bootstrap_peers and rpc_public_addr ( fix #353 )
2022-09-14 16:09:38 +02:00
Alex Auvolat
ab722cb40f
Add checks on replication_factor of layouts we use ( fix #363 , fix #364 )
2022-09-13 16:22:23 +02:00
Alex Auvolat
38be811b1c
Fix clippy lint that says we should implement Eq
2022-09-13 16:08:00 +02:00
Alex Auvolat
44733474bb
Remove/change println! in server code ( fix #358 )
2022-09-13 16:01:55 +02:00
Alex Auvolat
07febd3ecd
Ensure data dir is created immediately when Garage starts ( fix #349 )
2022-09-13 15:57:27 +02:00
Alex
11bdc971e2
Merge pull request 'use netapp streaming body' ( #343 ) from netapp-stream-body into main
...
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/343
2022-09-13 15:26:08 +02:00
Alex Auvolat
ff30891999
Use streaming block API for get with Range requests
2022-09-13 15:13:07 +02:00
Alex Auvolat
28a4af73ca
Use netapp 0.5 published from crates.io
2022-09-13 13:11:44 +02:00
Alex Auvolat
b823151a0b
improvements in block manager
2022-09-12 16:57:38 +02:00
Alex Auvolat
f91fab8582
Simplify+improve async hasher by using bounded channel
2022-09-12 16:23:43 +02:00
Alex Auvolat
7f54706b95
Merge branch 'lx-perf-improvements' into netapp-stream-body
2022-09-08 15:50:56 +02:00
Alex Auvolat
d9d199a6c9
Merge branch 'main' into lx-perf-improvements
2022-09-08 15:49:17 +02:00
Alex Auvolat
ceb1f0229a
Move version back into util
2022-09-07 18:36:46 +02:00
Alex Auvolat
f310fce34b
Inject GIT_VERSION even later
2022-09-07 18:30:15 +02:00
Alex Auvolat
06df301de5
Fix merge
2022-09-07 18:16:01 +02:00
Alex Auvolat
8adc654713
Merge branch 'main' into improve-deps
2022-09-07 18:13:27 +02:00
Alex Auvolat
107853334b
Fix build error
2022-09-07 18:10:19 +02:00
Alex Auvolat
1449204439
Add warnings when features are not included in build
2022-09-07 18:02:13 +02:00
Alex Auvolat
2e00809af5
Error messages when system-libs XOR bundled-libs != 1
2022-09-07 17:57:12 +02:00
Alex Auvolat
2559f63e9b
Make all HTTP services optionnal
2022-09-07 17:54:16 +02:00
Alex Auvolat
28d86e7602
Report build features in garage --help
2022-09-07 17:05:21 +02:00
Alex Auvolat
db61f41030
Move GIT_VERSION injection later in build chain to reduce build times
2022-09-07 11:59:56 +02:00
Alex Auvolat
907054775d
Faster copy, better get error message
2022-09-06 22:25:23 +02:00
Alex Auvolat
6b958979bd
Merge branch 'lx-perf-improvements' into netapp-stream-body
2022-09-06 22:13:01 +02:00
Alex Auvolat
c2cc08852b
Reenable node ordering
2022-09-06 19:31:42 +02:00
Alex Auvolat
6f02c36a89
cargo fmt
2022-09-06 17:59:41 +02:00
Alex Auvolat
0f5689c169
Include code from v0.5.1 directly to remove dependencies
2022-09-06 17:52:50 +02:00
Alex Auvolat
1e92e9f782
Disable k2v tests when feature is disabled
2022-09-06 17:29:46 +02:00
Alex Auvolat
431dee050f
Remove opentelemetry-otlp dep in api/
2022-09-06 17:25:44 +02:00
Alex Auvolat
b886c75450
Make all DB engines optional build features
2022-09-06 17:09:43 +02:00
Alex Auvolat
48ffaaadfc
Bump versions to 0.8.0 (compatibility is broken already)
2022-09-06 16:47:56 +02:00
Alex Auvolat
454d8474ef
Fix clippy
2022-09-06 15:43:50 +02:00
Jakub Jirutka
ea36b9ff90
Allow building without Prometheus exporter (/metrics endpoint)
...
prometheus and opentelemetry-prometheus add 7 extra dependencies in
total and increases the size of the garage binary by ~7 % (with
fat LTO).
2022-09-06 01:15:09 +02:00
Jakub Jirutka
e7af006c1c
Make OTLP exporter optional via feature "telemetry-otlp"
...
opentelemetry-otlp add 48 (!) extra dependencies and increases the
size of the garage binary by ~11 % (with fat LTO).
2022-09-06 01:14:47 +02:00
Alex Auvolat
729a910e14
Remove Heed default features
2022-09-05 16:40:13 +02:00
Alex Auvolat
07e6bcde85
Merge branch 'main' into lx-perf-improvements
2022-09-05 12:40:17 +02:00
Jakub Jirutka
7511ba5530
Allow linking against system-provided libsqlite
...
Unfortunately, rusqlite uses the opposite logic for enabling/disabling
bundled libraries to others (libsodium-sys, zstd-sys). Cargo features
are very limited and doesn't allow to enable feature A in a dependency
iff feature B is disabled.
Note, lmdb-rkv-sys doesn't need any special treatment because it
automatically links against system liblmdb if found via pkgconf.
Linux distros should build garage with
`--no-default-features --features system-libs` to disable bundled-libs
and enable system-libs.
2022-09-03 19:15:57 +02:00
Jakub Jirutka
a6e40b75ea
Add feature "system-libs" to enable linking against system libraries
...
If this feature is enabled, libsodium-sys and zstd-sys will link
dynamically against system-provided libraries instead of building
and linking statically the bundled (possibly outdated and vulnerable)
copies of them. This feature is intended mainly for linux package
maintainers.
2022-09-03 18:44:34 +02:00
Alex Auvolat
e1751c8a9c
fix clippy
2022-09-02 17:24:26 +02:00
Alex Auvolat
5d4b937a00
Ability to have up to 4 concurrently working resync workers
2022-09-02 17:18:13 +02:00
Alex Auvolat
5e8baa433d
Make BlockManagerLocked fully private again
2022-09-02 16:52:22 +02:00