clitools
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Releases Wiki Activity
4,578 Commits 8 Branches 44 Tags 200 MiB
Commit Graph

4578 Commits

Author SHA1 Message Date
MichaelEischer b84f5177cb
Merge pull request #2790 from greatroar/fix-quadratic-read 
Fix quadratic file reading in restic mount
 2020-07-12 18:42:14 +02:00
greatroar 4cf1c8e8da Changelog entry for subquadratic reading in mount 2020-07-12 18:27:16 +02:00
greatroar 58719e1f47 Replace mount's per-file cache by a global LRU cache 2020-07-12 18:27:16 +02:00
greatroar d42c169458 Fix quadratic file reading in restic mount 2020-07-12 18:27:16 +02:00
rawtaz 8598bb042b
Merge pull request #2769 from darkdragon-001/patch-1 
Update documentation for environment variables
 2020-07-12 00:40:21 +02:00
darkdragon-001 c6b74962df Update 040_backup.rst 
Changes proposed in #2763:

- Adding `RESTIC_CACHE_DIR` environment variables (introduced in #2425 for Unix and #2607 for Mac, Win).
- Adding used system-wide environment variables with links to the corresponding section.
 2020-07-12 00:31:56 +02:00
MichaelEischer 2c72924ffb
Merge pull request #2813 from greatroar/encrypt-benchmark 
Fix repository_test.BenchmarkSaveAndEncrypt
 2020-07-05 17:52:10 +02:00
greatroar 02bec13ef2 Fix repository_test.BenchmarkSaveAndEncrypt 
The benchmark was actually testing the speed of index lookups.

name              old time/op    new time/op        delta
SaveAndEncrypt-8     101ns ± 2%    31505824ns ± 1%    +31311591.31%  (p=0.000 n=10+10)

name              old speed      new speed          delta
SaveAndEncrypt-8  41.7TB/s ± 2%       0.0TB/s ± 1%         -100.00%  (p=0.000 n=10+10)

name              old alloc/op   new alloc/op       delta
SaveAndEncrypt-8     1.00B ± 0%  20989508.40B ± 0%  +2098950740.00%  (p=0.000 n=10+10)

name              old allocs/op  new allocs/op      delta
SaveAndEncrypt-8      0.00             123.00 ± 0%            +Inf%  (p=0.000 n=10+9)

(The actual speed is ca. 131MiB/s.)
 2020-07-05 17:41:42 +02:00
MichaelEischer 64976b1a4d
Merge pull request #2815 from scelfo/patch-1 
--hostname flag is deprecated
 2020-06-29 22:04:56 +02:00
Tony Scelfo 6a607d6ded
--hostname flag is deprecated 
Update documentation to follow the deprecation warning "Flag --hostname has been deprecated, use --host" that is given on the latest build(s).
 2020-06-29 12:53:57 -06:00
rawtaz 6fedf1a7f4
Merge pull request #2809 from lessfoobar/patch-2 
Update 080_examples.rst
 2020-06-26 00:12:05 +02:00
@lessfoobar df946fd9f8
Update 080_examples.rst 2020-06-26 00:01:30 +02:00
rawtaz 1bc80c3c8d
Merge pull request #2802 from andreaso/cachedir-https-links 
Use https:// for linking CACHEDIR.TAG specs
 2020-06-21 20:01:42 +02:00
Andreas Olsson 0fcef2ec23
Use https:// for linking CACHEDIR.TAG specs 
That site might not have supported https:// when those links were
originally added. It does now.

Also dropping the _spec.html_ ending of the url, there being a `<link
rel="canonical" ...>` tag suggesting that that no longer being the
preferred address.
 2020-06-21 19:36:26 +02:00
MichaelEischer 212607dc8a
Merge pull request #2760 from greatroar/backend-benchmark 
Fix backend benchmarks + a micro-optimization
 2020-06-17 23:17:05 +02:00
greatroar 190d8e2f51 Flatten backend.LimitedReadCloser structure 
This inlines the io.LimitedReader into the LimitedReadCloser body to
achieve fewer allocations. Results on linux/amd64:

name                                      old time/op    new time/op    delta
Backend/BenchmarkLoadPartialFile-8           412µs ± 4%     413µs ± 4%    ~     (p=0.634 n=17+17)
Backend/BenchmarkLoadPartialFileOffset-8     455µs ±13%     441µs ±10%    ~     (p=0.072 n=20+18)

name                                      old speed      new speed      delta
Backend/BenchmarkLoadPartialFile-8        10.2GB/s ± 3%  10.2GB/s ± 4%    ~     (p=0.817 n=16+17)
Backend/BenchmarkLoadPartialFileOffset-8  9.25GB/s ±12%  9.54GB/s ± 9%    ~     (p=0.072 n=20+18)

name                                      old alloc/op   new alloc/op   delta
Backend/BenchmarkLoadPartialFile-8            888B ± 0%      872B ± 0%  -1.80%  (p=0.000 n=15+15)
Backend/BenchmarkLoadPartialFileOffset-8      888B ± 0%      872B ± 0%  -1.80%  (p=0.000 n=15+15)

name                                      old allocs/op  new allocs/op  delta
Backend/BenchmarkLoadPartialFile-8            18.0 ± 0%      17.0 ± 0%  -5.56%  (p=0.000 n=15+15)
Backend/BenchmarkLoadPartialFileOffset-8      18.0 ± 0%      17.0 ± 0%  -5.56%  (p=0.000 n=15+15)
 2020-06-17 13:11:45 +02:00
greatroar f4cd2a7120 Make backend benchmarks fairer by removing checks 
Checking whether the right data is returned takes up half the time in
some benchmarks. Results for local backend benchmarks on linux/amd64:

name                                      old time/op    new time/op    delta
Backend/BenchmarkLoadFile-8                 4.89ms ± 0%    2.72ms ± 1%   -44.26%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFile-8           936µs ± 6%     439µs ±15%   -53.07%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFileOffset-8     940µs ± 1%     456µs ±10%   -51.50%  (p=0.008 n=5+5)
Backend/BenchmarkSave-8                     23.9ms ±14%    24.8ms ±41%      ~     (p=0.690 n=5+5)

name                                      old speed      new speed      delta
Backend/BenchmarkLoadFile-8               3.43GB/s ± 0%  6.16GB/s ± 1%   +79.40%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFile-8        4.48GB/s ± 6%  9.63GB/s ±14%  +114.78%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFileOffset-8  4.46GB/s ± 1%  9.22GB/s ±10%  +106.74%  (p=0.008 n=5+5)
Backend/BenchmarkSave-8                    706MB/s ±13%   698MB/s ±31%      ~     (p=0.690 n=5+5)
 2020-06-17 13:11:45 +02:00
MichaelEischer aba270df7e
Merge pull request #2786 from aawsome/memory-list-blobs 
Loop over index files for 'list blobs'
 2020-06-15 22:27:29 +02:00
Alexander Weiss b5543cff5d Loop over index files for 'list blobs' 
=> reduces memory consumption a lot!
 2020-06-14 17:25:34 +02:00
Alexander Neumann 285b5236c2
Merge pull request #2684 from MichaelEischer/cleanup-output 
Cleanup command output code
 2020-06-14 11:25:40 +02:00
Michael Eischer bb1e258bb7 cat/stats: Get stdout from globalOptions 2020-06-14 11:08:11 +02:00
Michael Eischer 182655bc88 Replace fmt.Printf/Println/Fprintf with wrapper functions 
cmd/restic/globals.go already provides Printf, Println and Warnf wrapper
which get their output streams from the globalOptions object. This
allows for stream replacements when testing.
 2020-06-14 11:08:11 +02:00
MichaelEischer 74bc7141c1
Merge pull request #2781 from aawsome/reduce-index-memory 
Reduce index memory
 2020-06-14 10:58:28 +02:00
Alexander Weiss 1361341c58 don't save duplicate packIDs when using internal/repository/Index.Store 2020-06-14 07:56:24 +02:00
Alexander Weiss ce4a2f4ca6 save packIDs and duplicates separately 
A side remark to the definition of Index.blob:

Another possibility would have been to use:
blob       map[restic.BlobHandle]*indexEntry

This would have led to the following sizes:
key: 32 + 1 = 33 bytes
value: 8 bytes
indexEntry:  8 + 4 + 4 = 16 bytes
each packID: 32 bytes

To save N index entries, we would therefore have needed:
N * OF * (33 + 8) bytes + N * 16 + N * 32 bytes / BP = N * 82 bytes

More precicely, using a pointer instead of a direct entry is the better memory choice if:
OF * 8 bytes + entrysize < OF * entrysize <=> entrysize > 8 bytes * OF/(OF-1)
Under the assumption of OF=1.5, this means using pointers would have been the better choice
if sizeof(indexEntry) > 24 bytes.
 2020-06-14 07:56:21 +02:00
Alexander Weiss cf979e2b81 make offset and length uint32 2020-06-14 07:50:19 +02:00
Michael Eischer d92e2c5769 simplify index code 2020-06-14 07:50:19 +02:00
Alexander Weiss 7419844885 add changelog, benchmark, memory calculation 2020-06-14 07:50:15 +02:00
rawtaz 1d66bb9e62
Merge pull request #2784 from MichaelEischer/minio-sha256-changelog 
Add changelog for #2709
 2020-06-13 18:33:54 +02:00
Michael Eischer 0b2c31b05b Add changelog for #2709 2020-06-13 18:13:36 +02:00
MichaelEischer dd7b4f54f5
Merge pull request #2709 from greatroar/minio-sha256 
Use Minio's optimized SHA-256
 2020-06-12 23:32:58 +02:00
MichaelEischer 6896c6449b
Merge pull request #2779 from greatroar/archiver-comment 
Fix up comment on archiver.BlobSaver.Save
 2020-06-12 23:04:21 +02:00
MichaelEischer 735a8074d5
Merge pull request #2773 from aawsome/index-uploads+knownblobs 
Fix non-intuitive repo behavior
 2020-06-12 22:41:04 +02:00
Alexander Weiss 70347e95d5 disable index uploads for prune command 
+ modifications of changelog
 2020-06-12 09:24:38 +02:00
greatroar 0fa3091c78 Fix up comment on archiver.BlobSaver.Save 2020-06-11 13:40:30 +02:00
Alexander Weiss 91906911b0 Fix non-intuitive repository behavior 
- The SaveBlob method now checks for duplicates.
- Moves handling of pending blobs to MasterIndex.
  -> also cleans up pending index entries when they are saved in the index
  -> when using SaveBlob no need to care about index any longer
- Always check for full index and save it when storing packs.
  -> removes the need of an index uploader
  -> also removes the verbose "uploaded intermediate index" messages
- The Flush method now also saves the index
- Fix race condition when checking and saving full/non-finalized indexes
 2020-06-11 13:05:23 +02:00
MichaelEischer fae7f78057
Merge pull request #2755 from greatroar/solaris-workaround 
Remove Solaris build workaround in integration tests
 2020-06-10 23:30:51 +02:00
MichaelEischer ac9ec4b990
Merge pull request #2778 from MichaelEischer/fix-unlock-hint 
Revive hint to the unlock command if a repository is locked
 2020-06-10 21:26:38 +02:00
Michael Eischer 087c770161 Revive hint to the unlock command if a repository is locked 
errors.Fatalf wraps a error and just keeps an error message as a string.
This prevents the `restic.IsAlreadyLocked(err)` check from working as
the error is no longer an ErrAlreadyLocked.

Just add an additional remark to the error using `errors.WithMessage`.
 2020-06-10 20:58:56 +02:00
MichaelEischer 6856d1e422
Merge pull request #2749 from aawsome/fix-fullindex 
Change condition for full index
 2020-06-10 20:40:19 +02:00
Alexander Weiss 8c1261ff02 changed condition for full index 2020-06-07 22:00:49 +02:00
rawtaz 26704be17f
Merge pull request #2776 from nunoperalta/patch-1 
Fix #2774 - make unable to umount message clearer
 2020-06-07 17:19:50 +02:00
nunoperalta 2c3360db98
Fix #2774 - make unable to umount message clearer 2020-06-07 15:57:39 +01:00
MichaelEischer cba6ad8d8e
Merge pull request #2711 from greatroar/update-x-net 
Update golang.org/x/net for fewer allocations in http2
 2020-05-25 20:43:29 +02:00
greatroar 2a3312ac35 Remove Solaris build workaround in integration tests 
This was introduced in #1821 because of issues with Go 1.9, but that
compiler version is no longer supported.
 2020-05-25 10:33:36 +02:00
greatroar c35c4e0cbf Update golang.org/x/net for fewer allocations in http2 
name                                          old allocs/op  new allocs/op  delta
BackendREST/BenchmarkLoadFile-8                  2.15k ± 0%     2.14k ± 0%  -0.27%  (p=0.002 n=20+20)
BackendREST/BenchmarkLoadPartialFile-8             587 ± 0%       582 ± 0%  -0.88%  (p=0.000 n=20+20)
BackendREST/BenchmarkLoadPartialFileOffset-8       584 ± 0%       579 ± 0%  -0.90%  (p=0.000 n=19+16)

The throughput isn't significantly higher or lower.
 2020-05-24 16:39:17 +02:00
MichaelEischer 84475aa3a8
Merge pull request #2730 from greatroar/mount-build-tags 
Simplify build tags for restic mount
 2020-05-23 20:33:43 +02:00
MichaelEischer f12f9ae240
Merge pull request #2741 from tbm/typos 
Fix typos
 2020-05-21 23:40:12 +02:00
Martin Michlmayr 5cc1760fdf
Fix typos 2020-05-16 14:05:26 +08:00
MichaelEischer 32ac5486e9
Merge pull request #2732 from restic/rawtaz-update-contributing 
Reword parts of Providing Patches in CONTRIBUTING.md
 2020-05-12 21:37:08 +02:00