Commit Graph

3 Commits (5a7dbc10c74167c8f0105f5803b81317cf676f42)

Author SHA1 Message Date
Josh Triplett 5a7dbc10c7
Optimize mmtarfilter to handle many path exclusions
mmtarfilter uses fnmatch to handle path exclusions and inclusions.
Python's fnmatch handles shell patterns by translating them to regular
expressions, with a 256-entry LRU cache. With more than 256 path
exclusions or inclusions, this LRU cache no longer works, and every
invocation of fnmatch on every file in every package will re-translate
and re-compile a regular expression, resulting in much worse
performance.

Translate all the shell patterns to regular expressions once. For an
mmdebstrap invocation with around 500 path filters, this speeds up
mmdebstrap by more than a minute.
3 years ago
Johannes 'josch' Schauer c51c69ca13
tarfilter/taridshift: use argparse 4 years ago
Johannes 'josch' Schauer 465c056434
no longer needs to install twice when --depkgopt=path-exclude is given by filtering the tarball with new tarfilter utility 4 years ago