1
0
mirror of https://github.com/restic/restic.git synced 2024-07-02 08:40:55 +02:00
restic/internal
Michael Eischer 6f53ecc1ae adapt workers based on whether an operation is CPU or IO-bound
Use runtime.GOMAXPROCS(0) as worker count for CPU-bound tasks,
repo.Connections() for IO-bound task and a combination if a task can be
both. Streaming packs is treated as IO-bound as adding more worker
cannot provide a speedup.

Typical IO-bound tasks are download / uploading / deleting files.
Decoding / Encoding / Verifying are usually CPU-bound. Several tasks are
a combination of both, e.g. for combined download and decode functions.
In the latter case add both limits together. As the backends have their
own concurrency limits restic still won't download more than
repo.Connections() files in parallel, but the additional workers can
decode already downloaded data in parallel.
2022-07-03 12:19:26 +02:00
..
archiver archiver: Reduce tree saver concurrency 2022-07-02 22:42:34 +02:00
backend Merge pull request #3470 from MichaelEischer/sanitize-debug-log 2022-07-02 19:00:54 +02:00
bloblru bloblru: Fix comment for New function 2022-03-28 22:25:25 +02:00
cache cache: Don't Lstat before creating the tag file 2022-05-10 18:52:39 +02:00
checker adapt workers based on whether an operation is CPU or IO-bound 2022-07-03 12:19:26 +02:00
crypto crypto: Remove unused error 2020-09-05 10:07:16 +02:00
debug redact swift auth token in debug output 2022-07-02 18:47:35 +02:00
dump Refactor internal/dump + concurrent load/write 2021-11-01 23:01:55 +01:00
errors all: Move away from pkg/errors, easy cases 2022-06-14 08:36:38 +02:00
filter Add tests for validating exclude patterns 2022-05-11 22:41:00 +02:00
fs Add simple test for fs.TempFile on windows 2022-04-09 23:37:58 +02:00
fuse copy/find/ls/recover/stats: Memorize snapshot listing before index 2022-04-09 12:26:30 +02:00
hashing hashing: Fix up comments 2022-05-11 21:36:10 +02:00
limiter cmd/restic, limiter: Move config knowledge to internal packages 2022-06-22 18:29:58 +02:00
migrations migrate: Cleanup option to request repository check 2022-06-04 23:45:00 +02:00
mock upgrade_repo_v2: Use atomic replace for supported backends 2022-05-09 22:31:30 +02:00
options Don't crash if SecretString is uninitialized 2022-07-02 19:44:28 +02:00
pack repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
repository adapt workers based on whether an operation is CPU or IO-bound 2022-07-03 12:19:26 +02:00
restic adapt workers based on whether an operation is CPU or IO-bound 2022-07-03 12:19:26 +02:00
restorer adapt workers based on whether an operation is CPU or IO-bound 2022-07-03 12:19:26 +02:00
selfupdate Refactor file handing for self-update. 2022-04-09 21:40:33 +02:00
test all: Move away from pkg/errors, easy cases 2022-06-14 08:36:38 +02:00
textfile Add more error handling 2021-01-30 20:19:47 +01:00
ui Print number of bytes added to the repo 2022-07-02 18:55:12 +02:00
walker adapt workers based on whether an operation is CPU or IO-bound 2022-07-03 12:19:26 +02:00