Commit Graph

4688 Commits

Author SHA1 Message Date
rawtaz 7ddfd6cabe
github: Add question to feature issue template 2020-08-24 00:22:44 +02:00
Alexander Neumann b1b3f1ecb6
Merge pull request #2674 from MichaelEischer/prune-strict-checks
prune: Stricter error checks
2020-08-23 10:30:41 +02:00
rawtaz fa135f72bf
Merge pull request #2439 from htrendev/ignore-sync-on-macos
Ignore not supported error on sync() when using local backend
2020-08-22 01:38:49 +02:00
Hristo Trendev 51c22f4223 local backend: ignore not supported error on sync()
Closes #2395
2020-08-22 01:27:07 +02:00
rawtaz 1a5b66f33b
Merge pull request #2879 from 0xMH/master
Added more explantation for --one-file-system option
2020-08-21 19:21:01 +02:00
NoNE da6a34e044 doc: Add more explanation for --one-file-system option 2020-08-21 19:19:13 +02:00
Bruce Dillahunty fe69b83074
Update key password prompt (#2847) 2020-08-19 21:42:08 +02:00
Michael Eischer 08d24ff99e prune: Include ID of all missing blobs in error message 2020-08-16 11:36:14 +02:00
Michael Eischer d8b80e9862 Add changelog 2020-08-16 11:36:14 +02:00
Michael Eischer 1c84aceb39 prune: Test for abort on damaged repositories 2020-08-16 11:36:12 +02:00
Michael Eischer 575ed9a47e Test that rebuild-index errors when old index cannot be removed 2020-08-16 11:34:01 +02:00
Michael Eischer 8f811642c3 Add support for integration tests to wrap the backend 2020-08-16 11:34:01 +02:00
Michael Eischer f4b9544ab2 prune: Add test that repack aborts on wrong blob 2020-08-16 11:34:01 +02:00
Michael Eischer 367449dede prune: Reduce memory allocations while repacking
The slicing operator `slice[low:high]` default to 0 for the lower bound and
len(slice) for the upper bound when either or both are not specified.
Fix the code to use `cap(slice)` to check for the slice capacity.
2020-08-16 11:34:01 +02:00
Michael Eischer 7042bafea5 prune: Abort repacking when a pack contains a wrong blob
If a blob in a pack file can be decrypted successfully but contains data
that results in a different hash than stated in the header pack, then
abort repacking. As both the pack header and the blob are
cryptographically verified this either means than a malicious entity
tampered with the backup or indicates hardware problems on the client.
prune should fail with an error in both cases.
2020-08-16 11:34:01 +02:00
Michael Eischer 744a15247d prune/rebuild-index: Fail if an old index cannot be removed
The old behavior was problematic in the context of rebuild-index as it
could leave old, possibly invalid index files behind without returning a
fatal error.

Prune calls rebuildIndex before removing any data from the repository.
For this use case failing to delete an old index MUST be treated as a
fatal error. Otherwise the index could still contain an old index file
that refers to blobs/packs that were later on deleted by prune. Later
backup runs will assume that the affected blobs already exist in the
repository which results in a backup which misses data.
2020-08-16 11:34:01 +02:00
Michael Eischer 3ba19869be prune: Abort if any used blobs are missing
The previous check only approximately verified whether all required
blobs were found. However, after forgetting a few snapshots the
repository contains lots of unused blobs whose number can be sufficient
to make up for missing packs.

When coupled with a malfunctioning backend that temporarily returns broken
data this could cause restic to regard the corresponding packs as
invalid and thereby delete data that's still in use. This change lets
restic play it safe and refuse to delete anything if data is missing.
2020-08-16 11:34:01 +02:00
aawsome 0fed6a8dfc
Use "pack file" instead of "data file" (#2885)
- changed variable names, especially changed DataFile into PackFile
- changed in some comments
- always use "pack file" in docu
2020-08-16 11:16:38 +02:00
rawtaz 643bbbe156
doc: Correct wording for repository (#2884)
Replaces "backend" with "repository" to match restic output.
2020-08-11 22:30:17 +02:00
MichaelEischer eca0f0ad24
Merge pull request #2863 from aawsome/index-no-duplicates
Don't save exact duplicates in merged index
2020-08-08 18:24:14 +02:00
MichaelEischer 08dee8a52b
Merge pull request #2865 from greatroar/unused
Dead code removal
2020-08-08 16:15:28 +02:00
Alexander Weiss b112533812 Don't save exact duplicates when merging indexes 2020-08-05 06:32:02 +02:00
Alexander Weiss 5e63294355 Add benchmark MasterIndexAlloc 2020-08-05 06:32:02 +02:00
MichaelEischer 84b6f1ec53
Merge pull request #2874 from MichaelEischer/fix-verbose
Adjust description of --verbose=n parameter
2020-08-04 23:21:05 +02:00
MichaelEischer 06fb4ea3f0
Merge pull request #2805 from jwilk-forks/verbose
Fix examples of --verbose with argument
2020-08-04 23:11:35 +02:00
Michael Eischer e38d415173 Adjust description of --verbose=n parameter 2020-08-04 23:07:53 +02:00
greatroar d81a396944 Dead code removal
Found by running golangci-lint on the entire code base.
2020-08-04 08:38:57 +02:00
rawtaz 0b21ec44b7
Merge pull request #2869 from josephrocca/patch-1
doc: --verbose 2 to --verbose=2
2020-08-04 01:06:58 +02:00
josephrocca bd36731119
`--verbose 2` to `--verbose=2`
`--verbose 2` seems to be incorrect here (gives an error/warning that "the `2` directory does not exist, skipping")
2020-08-04 08:39:43 +10:00
MichaelEischer 38a2f9c07b
Merge pull request #2864 from greatroar/dump-error
Fix error handling in dump (err != err)
2020-08-03 20:09:42 +02:00
rawtaz 5af2815627
Merge pull request #2821 from renard/mount-nolock
cmd/mount: honur --no-lock flag
2020-08-03 20:09:26 +02:00
MichaelEischer b55de2260d
Merge pull request #2868 from MichaelEischer/fix-getusedblobs
prune: Stop progress bar after searching used blobs
2020-08-03 19:58:59 +02:00
Sébastien Gross 9be4fe3e84 cmd/mount: honour --no-lock flag
Do not lock the repository if --no-lock global flag is set. This allows
to mount repositories which are archived on a read only system.

Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
2020-08-03 19:45:39 +02:00
Michael Eischer 05116e4787 prune: Cleanup progress bar handling while repacking 2020-08-03 19:32:46 +02:00
Michael Eischer 04f79b9642 prune: Stop progress bar after searching used blobs 2020-08-03 19:31:49 +02:00
greatroar 8b358935a0 Fix error handling in dump (err != err) 2020-08-03 09:43:44 +02:00
MichaelEischer 66d089e239
Merge pull request #2841 from aawsome/optimize-getUsedBlobs
Extract get used blobs in prune as separate function
2020-08-01 22:40:33 +02:00
MichaelEischer 49d3efe547
Merge pull request #2840 from aawsome/delete-parallel
Make delete parallel
2020-08-01 22:26:15 +02:00
Alexander Weiss 9762bec091 Use optimized getUsedBlobs in prune 2020-08-01 21:07:31 +02:00
Alexander Weiss 0eb8553c87 add changelog for #2840 2020-08-01 20:43:18 +02:00
Alexander Weiss d3692f5b81 Delete files in parallel in rebuild-index 2020-08-01 20:43:18 +02:00
Alexander Weiss 1c0b61204b Delete files in parallel in forget 2020-08-01 20:43:18 +02:00
Alexander Weiss 2ee654763b Delete files in parallel 2020-08-01 20:39:24 +02:00
Alexander Neumann b7b479b668
Merge pull request #2599 from MichaelEischer/tweak-mem-usage
Reduce memory usage when searching for used blobs
2020-08-01 13:49:02 +02:00
Alexander Neumann 4cf9656f12
Merge pull request #2861 from MichaelEischer/fix-rclone-crash
rclone: Don't panic after unexpected subprocess exit
2020-08-01 13:08:50 +02:00
Alexander Neumann 2580eef2aa
Merge pull request #2318 from classmarkets/2175-named-keys
Allow specifying user and host when adding keys
2020-08-01 13:06:31 +02:00
Michael Eischer 2d7ab9115f Add changelog entry 2020-08-01 12:29:16 +02:00
Michael Eischer a178e5628e Remove duplicate TreeLoader interface 2020-08-01 12:29:16 +02:00
Michael Eischer af66a62c04 FindUsedBlobs: Test that seen blobs are skipped
This also copies the TreeLoader interface from internal/walker to allow
stubbing the repository in the call to `FindUsedBlobs`.
2020-08-01 12:29:16 +02:00
Michael Eischer 9ea1a78bd4 FindUsedBlobs: Check for seen blobs before loading trees
The only effective change in behavior is that that toplevel nodes can
also be skipped.
2020-08-01 12:29:16 +02:00