0.5.0 test case failures on Fedora Rawhide #178

Open
opened 9 months ago by gms · 7 comments
gms commented 9 months ago

AFAICS, all the old test case failures are now gone, but there a few new cases that fail on Fedora Rawhide:

src/img2pdf_test.py:395: AssertionError
=========================== short test summary info ============================
FAILED src/img2pdf_test.py::test_jpg_2000_rgba8[internal] - subprocess.Called...
FAILED src/img2pdf_test.py::test_jpg_2000_rgba8[pikepdf] - subprocess.CalledP...
FAILED src/img2pdf_test.py::test_jpg_2000_rgba16[internal] - subprocess.Calle...
FAILED src/img2pdf_test.py::test_jpg_2000_rgba16[pikepdf] - subprocess.Called...
FAILED src/img2pdf_test.py::test_miff_cmyk8[internal] - AssertionError: asser...
FAILED src/img2pdf_test.py::test_miff_cmyk8[pikepdf] - AssertionError: assert...
FAILED src/img2pdf_test.py::test_miff_cmyk16[internal] - assert 6.35075 > 50
FAILED src/img2pdf_test.py::test_miff_cmyk16[pikepdf] - assert 6.35075 > 50
================== 8 failed, 335 passed, 8 skipped in 58.13s ===================

https://kojipkgs.fedoraproject.org//work/tasks/4018/108274018/build.log

The last two are related to this assertion:

            assert psnr != b"0"
            assert psnr != b"0 (0)"
            assert psnr_re.fullmatch(psnr) is not None, psnr
            psnr = psnr_re.fullmatch(psnr).group(1)
            psnr = float(psnr)
            assert psnr != 0  # or otherwise we would use the exact variant
>           assert psnr > 50
E           assert 6.35075 > 50
src/img2pdf_test.py:395: AssertionError

The two before:

            assert psnr != b"0"
>           assert psnr != b"0 (0)"
E           AssertionError: assert b'0 (0)' != b'0 (0)'
src/img2pdf_test.py:390: AssertionError

And the other all generate the same stderror output, similar to this instance:

>           raise CalledProcessError(retcode, cmd)
E           subprocess.CalledProcessError: Command '['compare', '-metric', 'AE', '-alpha', 'off', '/tmp/pytest-of-mockbuild/pytest-0/jpg_2000_rgba80/in.jp2', '/tmp/pytest-of-mockbuild/pytest-0/jpg_2000_rgba81/gs-1.png', 'null:']' returned non-zero exit status 1.
/usr/lib64/python3.12/subprocess.py:413: CalledProcessError
----------------------------- Captured stderr call -----------------------------
2190
AFAICS, all the old test case failures are now gone, but there a few new cases that fail on Fedora Rawhide: ``` src/img2pdf_test.py:395: AssertionError =========================== short test summary info ============================ FAILED src/img2pdf_test.py::test_jpg_2000_rgba8[internal] - subprocess.Called... FAILED src/img2pdf_test.py::test_jpg_2000_rgba8[pikepdf] - subprocess.CalledP... FAILED src/img2pdf_test.py::test_jpg_2000_rgba16[internal] - subprocess.Calle... FAILED src/img2pdf_test.py::test_jpg_2000_rgba16[pikepdf] - subprocess.Called... FAILED src/img2pdf_test.py::test_miff_cmyk8[internal] - AssertionError: asser... FAILED src/img2pdf_test.py::test_miff_cmyk8[pikepdf] - AssertionError: assert... FAILED src/img2pdf_test.py::test_miff_cmyk16[internal] - assert 6.35075 > 50 FAILED src/img2pdf_test.py::test_miff_cmyk16[pikepdf] - assert 6.35075 > 50 ================== 8 failed, 335 passed, 8 skipped in 58.13s =================== ``` https://kojipkgs.fedoraproject.org//work/tasks/4018/108274018/build.log The last two are related to this assertion: ``` assert psnr != b"0" assert psnr != b"0 (0)" assert psnr_re.fullmatch(psnr) is not None, psnr psnr = psnr_re.fullmatch(psnr).group(1) psnr = float(psnr) assert psnr != 0 # or otherwise we would use the exact variant > assert psnr > 50 E assert 6.35075 > 50 src/img2pdf_test.py:395: AssertionError ``` The two before: ``` assert psnr != b"0" > assert psnr != b"0 (0)" E AssertionError: assert b'0 (0)' != b'0 (0)' src/img2pdf_test.py:390: AssertionError ``` And the other all generate the same stderror output, similar to this instance: ``` > raise CalledProcessError(retcode, cmd) E subprocess.CalledProcessError: Command '['compare', '-metric', 'AE', '-alpha', 'off', '/tmp/pytest-of-mockbuild/pytest-0/jpg_2000_rgba80/in.jp2', '/tmp/pytest-of-mockbuild/pytest-0/jpg_2000_rgba81/gs-1.png', 'null:']' returned non-zero exit status 1. /usr/lib64/python3.12/subprocess.py:413: CalledProcessError ----------------------------- Captured stderr call ----------------------------- 2190 ```
josch commented 9 months ago
Owner

I'm fighting with those as well as I'm now uploading the package to Debian. I'm currently using this patch:

https://salsa.debian.org/debian/img2pdf/-/blob/master/debian/patches/fix_tests.patch

But that didn't fix all yet. The remaining issues are regressions in imagemagick.

The img2pdf testcases really frustrate me as imagemagick behaviour changes pop up faster than I can bisect and report them in my free time...

I'm fighting with those as well as I'm now uploading the package to Debian. I'm currently using this patch: https://salsa.debian.org/debian/img2pdf/-/blob/master/debian/patches/fix_tests.patch But that didn't fix all yet. The remaining issues are regressions in imagemagick. The img2pdf testcases really frustrate me as imagemagick behaviour changes pop up faster than I can bisect and report them in my free time...
josch commented 9 months ago
Owner
Filed here: https://github.com/ImageMagick/ImageMagick6/issues/285
josch commented 9 months ago
Owner

AFAICS, all the old test case failures are now gone

@gms could you close those issues that do not apply to Fedora anymore? Thanks!

> AFAICS, all the old test case failures are now gone @gms could you close those issues that do not apply to Fedora anymore? Thanks!
gms commented 9 months ago
Poster

AFAICS, all the old test case failures are now gone

@gms could you close those issues that do not apply to Fedora anymore? Thanks!

Sure, I've checked my notes and the old ones that affected Fedora were already closed (i.e. #152 and #161).


Yes, imagemagick is a moving target.

> > AFAICS, all the old test case failures are now gone > > @gms could you close those issues that do not apply to Fedora anymore? Thanks! Sure, I've checked my notes and the old ones that affected Fedora were already closed (i.e. #152 and #161). --- Yes, imagemagick is a moving target.

On NixOS, applying 4a7dbda0f4/debian/patches/fix_tests.patch only leaves the two test_miff_cmyk16 test cases failing: https://github.com/NixOS/nixpkgs/pull/264119

On NixOS, applying https://salsa.debian.org/debian/img2pdf/-/blob/4a7dbda0f473f7c5ffcaaf68ea4ad3f435e0920d/debian/patches/fix_tests.patch only leaves the two `test_miff_cmyk16` test cases failing: https://github.com/NixOS/nixpkgs/pull/264119
josch commented 9 months ago
Owner

@dotlambda thank you for testing! What version of imagemagick is that?

@dotlambda thank you for testing! What version of imagemagick is that?

For the record, test_miff_cmyk16 fails for me only when ImageMagick is built with --enable-hdri. So whatever the fix is, it will need to account for both variants.

For the record, `test_miff_cmyk16` fails for me only when ImageMagick is built with `--enable-hdri`. So whatever the fix is, it will need to account for both variants.
Sign in to join this conversation.
No Milestone
No project
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: josch/img2pdf#178
Loading…
There is no content yet.