Commit graph

4 commits

Author SHA1 Message Date
Benjamin Drung
043ab3bbf0
tarfilter: Compile prefix pattern only once
According to Debian bug #978742, mmtarfilter has a slow performance with
many path exclusions. The execution can be speed up if the regular
expression is only compiled once instead of every time in the hot loop.

Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
2021-02-22 13:15:16 +01:00
Josh Triplett
5a7dbc10c7
Optimize mmtarfilter to handle many path exclusions
mmtarfilter uses fnmatch to handle path exclusions and inclusions.
Python's fnmatch handles shell patterns by translating them to regular
expressions, with a 256-entry LRU cache. With more than 256 path
exclusions or inclusions, this LRU cache no longer works, and every
invocation of fnmatch on every file in every package will re-translate
and re-compile a regular expression, resulting in much worse
performance.

Translate all the shell patterns to regular expressions once. For an
mmdebstrap invocation with around 500 path filters, this speeds up
mmdebstrap by more than a minute.
2021-01-06 14:58:10 +01:00
c51c69ca13
tarfilter/taridshift: use argparse 2020-11-27 00:32:14 +01:00
465c056434
no longer needs to install twice when --depkgopt=path-exclude is given by filtering the tarball with new tarfilter utility 2020-08-25 13:02:33 +02:00