6 commits

Author	SHA1	Message	Date
Johannes Schauer Marin Rodrigues	bd5d3c3dab	tarfilter: remove leftover debugging statement	2021-05-07 09:20:36 +02:00
Johannes Schauer Marin Rodrigues	bbf12c221d	tarfilter: fixup last commit by formatting with black	2021-02-22 13:45:55 +01:00
Benjamin Drung	043ab3bbf0	tarfilter: Compile prefix pattern only once ... According to Debian bug #978742, mmtarfilter has a slow performance with many path exclusions. The execution can be speed up if the regular expression is only compiled once instead of every time in the hot loop. Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>	2021-02-22 13:15:16 +01:00
Josh Triplett	5a7dbc10c7	Optimize mmtarfilter to handle many path exclusions ... mmtarfilter uses fnmatch to handle path exclusions and inclusions. Python's fnmatch handles shell patterns by translating them to regular expressions, with a 256-entry LRU cache. With more than 256 path exclusions or inclusions, this LRU cache no longer works, and every invocation of fnmatch on every file in every package will re-translate and re-compile a regular expression, resulting in much worse performance. Translate all the shell patterns to regular expressions once. For an mmdebstrap invocation with around 500 path filters, this speeds up mmdebstrap by more than a minute.	2021-01-06 14:58:10 +01:00
Johannes 'josch' Schauer	c51c69ca13	tarfilter/taridshift: use argparse	2020-11-27 00:32:14 +01:00
Johannes 'josch' Schauer	465c056434	no longer needs to install twice when --depkgopt=path-exclude is given by filtering the tarball with new tarfilter utility	2020-08-25 13:02:33 +02:00