Skip to content

Very bad ImageMagick resizing performance with PHP 7.1 and newer in Docker #1100

Closed
Closed
Very bad ImageMagick resizing performance with PHP 7.1 and newer in Docker#1100

Description

@jhogervorst
jhogervorst
opened on Dec 7, 2020

After updating (from non-Docker PHP 5) to PHP 7.4 in Docker, I was experiencing performance problems with resizing images using ImageMagick. (Details: https://serverfault.com/q/1045274/31887.) After some trial-and-error, it seems that some ImageMagick performance issue is present starting in PHP 7.1 when using the Docker image.

Test script & environment

To investigate the issue, I’ve made the following simple script, with a ~2MB JPEG as input, which resizes the image and prints the time taken for each step:

<?php

$image_name = "imgbench.jpeg";
$image_size = 4000;
$image_quality = 50;

echo "PHP version: " . phpversion() . PHP_EOL;

$start = $step = microtime(true);

function bench($msg, $total = false) {
	global $start, $step;
	$prev = $total ? $start : $step;
	$now = microtime(true);
	echo number_format($now - $prev, 2) . " seconds - " . $msg . PHP_EOL;
	$step = $now;
}

$image = new Imagick(); bench("init");
$image->readImage($image_name); bench("read");
$image->stripImage(); bench("strip");
$image->resizeImage($image_size, $image_size, Imagick::FILTER_LANCZOS, 1, true); bench("resize");
$image->setImageCompressionQuality($image_quality); bench("compress");
$image->setInterlaceScheme(Imagick::INTERLACE_JPEG); bench("interlace");
$image_blob = $image->getImageBlob(); bench("blob");
bench("TOTAL", true);

I created a new AWS EC2 t3.medium instance with the Amazon Linux AMI and installed Docker using sudo yum install docker. Then, I built the following Dockerfile (using docker build -t imgbench .; with [version] substituted by some valid version):

FROM php:[version]
COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/
RUN install-php-extensions imagick
COPY ./ ./

Finally, I ran the test script from above using docker run -it imgbench php imgbench.php.

All the files used for this test, can be found here: imgbench.zip.

Results

I obtained the following output/results:

PHP 7.0: ~2s PHP 7.1: ~19s PHP 7.2: ~19s PHP 7.3: ~19s PHP 7.4: ~19s
PHP version: 7.0.33 PHP version: 7.1.33 PHP version: 7.2.34 PHP version: 7.3.25 PHP version: 7.4.13
0.00 seconds - init 0.00 seconds - init 0.00 seconds - init 0.00 seconds - init 0.00 seconds - init
0.25 seconds - read 0.21 seconds - read 0.15 seconds - read 0.16 seconds - read 0.15 seconds - read
0.00 seconds - strip 0.00 seconds - strip 0.00 seconds - strip 0.00 seconds - strip 0.00 seconds - strip
0.94 seconds - resize 18.10 seconds - resize 18.07 seconds - resize 17.99 seconds - resize 18.37 seconds - resize
0.00 seconds - compress 0.00 seconds - compress 0.00 seconds - compress 0.00 seconds - compress 0.00 seconds - compress
0.00 seconds - interlace 0.00 seconds - interlace 0.00 seconds - interlace 0.00 seconds - interlace 0.00 seconds - interlace
0.65 seconds - blob 0.56 seconds - blob 0.56 seconds - blob 0.55 seconds - blob 0.59 seconds - blob
1.84 seconds - TOTAL 18.87 seconds - TOTAL 18.78 seconds - TOTAL 18.71 seconds - TOTAL 19.12 seconds - TOTAL

It is clear that starting from PHP 7.1, something is wrong with the performance of the \Imagick::resizeImage() operation, since it’s approximately 10 times slower than in PHP 7.0.

Other factors

I also ran the test script on the same instance without Docker (using PHP from sudo yum -y install php70-cli php70-common php70-pecl-imagick) for PHP 7.0, 7.1, and 7.2. All versions showed similar (good) performance, indicating that the problem is not related to the PHP version itself, but rather something in the Docker image.

Also, I tried a Dockerfile without using install-php-extensions from mlocati/php-extension-installer, by installing ImageMagick manually in the Dockerfile:

RUN apt-get update \
	&& apt-get install -y libmagickwand-dev \
	&& pecl install imagick-3.4.4 \
	&& docker-php-ext-enable imagick

This yielded similar performance as reported above (i.e., good performance on PHP 7.0, and bad performance on newer versions).

Activity

  1. jhogervorst
    changed the title [-]Very bad ImageMagick resizing performance using PHP 7.1 or newer in Docker[/-] [+]Very bad ImageMagick resizing performance using PHP 7.1 and newer in Docker[/+] on Dec 7, 2020
  2. jhogervorst
    changed the title [-]Very bad ImageMagick resizing performance using PHP 7.1 and newer in Docker[/-] [+]Very bad ImageMagick resizing performance with PHP 7.1 and newer in Docker[/+] on Dec 7, 2020

  3. wglambert commented on Dec 7, 2020

    @wglambert
    wglambert
    on Dec 7, 2020

    #493 is similar, or the same

    Do you get the same experience outside of Docker? Or with using Debian/Ubuntu packages over the compiled php that the image uses?

  4. krzysztof-kabala commented on Dec 8, 2020

    @krzysztof-kabala
    krzysztof-kabala
    on Dec 8, 2020

    Have you tried to use:

    RUN install-php-extensions opcache imagick

    ?

  5. jhogervorst commented on Dec 8, 2020

    @jhogervorst
    jhogervorst
    on Dec 8, 2020
    Author

    #493 is similar, or the same

    @wglambert If I understand correctly, it’s not exactly the same: #493 is about a performance problem with PHP 7.0 in Docker, while this problem does not occur with PHP 7.0, but only with later versions.

    Also, that other issue seems to be caused by some Docker image/volume characteristics. It would be weird if that was the case here, since there’s such a huge difference between PHP 7.0 and later versions with this issue (while they share the same Docker host). Unless something radical changed between those PHP versions, which magnifies the Docker issue…

    Do you get the same experience outside of Docker?

    No — on the same EC2 instance, various PHP versions perform great outside of Docker:

    I also ran the test script on the same instance without Docker (using PHP from sudo yum -y install php70-cli php70-common php70-pecl-imagick) for PHP 7.0, 7.1, and 7.2. All versions showed similar (good) performance, indicating that the problem is not related to the PHP version itself, but rather something in the Docker image.

    Or with using Debian/Ubuntu packages over the compiled php that the image uses?

    I tried several PHP versions in Ubuntu/Debian using the following Dockerfile (using ubuntu:16.04, ubuntu:18.04, ubuntu:20.04, and debian:10 as [version]):

    FROM [version]
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y php php-imagick
COPY ./ ./

    This yielded the following results:

    Ubuntu 16.04 / PHP 7.0: ~2s Ubuntu 18.04 / PHP 7.2: ~2s Debian 10 / PHP 7.3: ~19s Ubuntu 20.04 / PHP 7.4: ~16s
    PHP version: 7.0.33-0ubuntu0.16.04.16 PHP version: 7.2.24-0ubuntu0.18.04.7 PHP version: 7.3.19-1~deb10u1 PHP version: 7.4.3
    0.00 seconds - init 0.00 seconds - init 0.00 seconds - init 0.00 seconds - init
    0.17 seconds - read 0.21 seconds - read 0.17 seconds - read 0.15 seconds - read
    0.00 seconds - strip 0.00 seconds - strip 0.00 seconds - strip 0.00 seconds - strip
    0.99 seconds - resize 1.11 seconds - resize 18.20 seconds - resize 15.51 seconds - resize
    0.00 seconds - compress 0.00 seconds - compress 0.00 seconds - compress 0.00 seconds - compress
    0.00 seconds - interlace 0.00 seconds - interlace 0.00 seconds - interlace 0.00 seconds - interlace
    0.79 seconds - blob 0.64 seconds - blob 0.57 seconds - blob 0.31 seconds - blob
    1.96 seconds - TOTAL 1.95 seconds - TOTAL 18.95 seconds - TOTAL 15.98 seconds - TOTAL

    Interestingly, the problem is not present here in PHP 7.0 and 7.2, but it starts appearing from PHP 7.3 (and in PHP 7.4). So now I have no clue where the cause of the problem is, since we’re seeing the same problem but with different versions here…

  6. jhogervorst commented on Dec 8, 2020

    @jhogervorst
    jhogervorst
    on Dec 8, 2020
    Author

    Have you tried to use:

    RUN install-php-extensions opcache imagick

    ?

    @krzysztof-kabala I did not use opcache in my initial testing.

    I just tried using opcache with my original test setup (so using the php:7.* base versions), for all PHP versions from 7.0 to 7.4. I ran the test script a few times, to give opcache the opportunity to ‘warm up’.

    The performance did not change notably. The big difference between PHP 7.0 and later versions was still present.

  7. jhogervorst commented on Dec 8, 2020

    @jhogervorst
    jhogervorst
    on Dec 8, 2020
    Author

    I did some more research, and created the following table (data from php -i):

    Source PHP version Imagick version ImageMagick version Test performance
    yum php70-* 7.0.33 3.4.4 6.9.10-68 ✅ ~2s
    yum php71-* 7.1.33 3.4.4 6.9.10-68 ✅ ~2s
    yum php72-* 7.2.34 3.4.4 6.9.10-68 ✅ ~2s
    Docker php:7.0 7.0.33 3.4.4 6.9.7-4 ✅ ~2s
    Docker php:7.1 7.1.33 3.4.4 6.9.10-23 ❌ ~19s
    Docker php:7.2 7.2.34 3.4.4 6.9.10-23 ❌ ~19s
    Docker php:7.3 7.3.25 3.4.4 6.9.10-23 ❌ ~19s
    Docker php:7.4 7.4.13 3.4.4 6.9.10-23 ❌ ~19s
    Docker php:7.0-alpine 7.0.33 3.4.4 7.0.7-11 ✅ ~3s
    Docker php:7.1-alpine 7.1.33 3.4.4 7.0.8-68 ✅ ~3s
    Docker php:7.2-alpine 7.2.34 3.4.4 7.0.10-25 ✅ ~3s
    Docker php:7.3-alpine 7.3.25 3.4.4 7.0.10-25 ✅ ~3s
    Docker php:7.4-alpine 7.4.13 3.4.4 7.0.10-25 ✅ ~3s
    Docker ubuntu:16.04 7.0.33 3.4.0RC6 6.8.9-9 ✅ ~2s
    Docker ubuntu:18.04 7.2.24 3.4.3RC2 6.9.7-4 ✅ ~2s
    Docker ubuntu:20.04 7.4.3 3.4.4 6.9.10-23 ❌ ~16s
    Docker debian:10 7.3.19 3.4.3 6.9.10-23 ❌ ~19s

    The PHP version does not seem to affect the issue, since versions 7.1 and 7.2 do or do not have the performance problem in different scenarios. Similarly, the Imagick version does not seem to affect it.

    However, all the scenarios with bad performance are using the same version of ImageMagick: 6.9.10-23. Maybe that points to something?

  8. jhogervorst commented on Dec 8, 2020

    @jhogervorst
    jhogervorst
    on Dec 8, 2020
    Author

    Workaround

    Alright, I found a workaround: the php:alpine packages do not have the performance problem!

    They do seem slightly slower than the other performant environment, but that’s ‘just’ a ~50% increase of processing time, rather than 10 times slower (900% increase).

    I’ve updated the table above to include the specific versions.

  9. krzysztof-kabala commented on Dec 8, 2020

    @krzysztof-kabala
    krzysztof-kabala
    on Dec 8, 2020

    "base" php image:

    FROM php:7.4
COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/
RUN install-php-extensions imagick
COPY ./ ./

    log:

    ➜ php index.php
PHP version: 7.4.13 ImageMagic version: ImageMagick 6.9.10-23 Q16 x86_64 20190101 https://imagemagick.org
0.00 seconds - init
0.24 seconds - read
0.00 seconds - strip
17.58 seconds - resize
0.00 seconds - compress
0.00 seconds - interlace
0.66 seconds - blob
18.48 seconds - TOTAL

    "custom" php image (with libjpeg62-turbo installed + custom ImageMagic setup):

    FROM amsdard/php:7.4

RUN curl -fsSL 'http://www.imagemagick.org/download/ImageMagick.tar.gz' -o ImageMagick.tar.gz && \
    tar xvzf ImageMagick.tar.gz && \
    cd ImageMagick-* && ./configure && make && make install && /sbin/ldconfig /usr/local/lib && \
    cd .. && rm -rf ImageMagick.tar.gz ImageMagick-* && \
    POLICY_XML_LOCATION="$(find /usr/local/etc/ -name 'policy.xml')" && \
    cp /files/imagemagick-policy.xml $POLICY_XML_LOCATION && \
    pecl install imagick && \
    docker-php-ext-enable imagick

COPY ./ ./

    log:

    ➜ php index.php
PHP version: 7.4.12 ImageMagic version: ImageMagick 7.0.10-46 Q16 x86_64 2020-12-08 https://imagemagick.org
0.00 seconds - init
0.27 seconds - read
0.00 seconds - strip
0.58 seconds - resize
0.00 seconds - compress
0.00 seconds - interlace
0.84 seconds - blob
1.69 seconds - TOTAL

    seems like ImageMagic issue, not a php or docker one :)

  10. tianon commented on Dec 9, 2020

    @tianon
    tianon
    on Dec 9, 2020
    Member

    Can confirm -- downloaded the example imgbench.jpeg and used the following Dockerfile:

    FROM debian:buster

RUN apt-get update && apt-get install -y --no-install-recommends imagemagick && rm -rf /var/lib/apt/lists/*

COPY imgbench.jpeg /in.jpg

CMD ["bash", "-c", "time convert /in.jpg -filter Lanczos -resize 4000x4000 /out.jpg"]

    The output is exactly in line with the 19s reported:

    $ docker run --rm 2d4ff9ec15a0

real	0m15.688s
user	0m19.005s
sys	0m12.027s

    However, if I replace imagemagick with graphicsmagick-imagemagick-compat, the result is much faster:

    $ docker run --rm ba076d35c429

real	0m0.391s
user	0m1.823s
sys	0m0.142s

    So this is firmly a case for a bug at https://bugs.debian.org/src:imagemagick (although possibly an upstream bug), and a potential workaround would be to try to use libgraphicsmagick1-dev, libgraphicsmagick++1-dev, and/or graphicsmagick-libmagick-dev-compat instead.

  11. tianon
    closed this as completedon Dec 9, 2020

  12. jhogervorst commented on Dec 9, 2020

    @jhogervorst
    jhogervorst
    on Dec 9, 2020
    Author

    @tianon Thank for your efforts! I will see if I can report this to https://bugs.debian.org/src:imagemagick.

    For people finding this issue: I tried @tianon’s suggestion of using alternative packages. When using libgraphicsmagick1-dev and libgraphicsmagick++1-dev (with PHP in Docker), the performance issue remained. With graphicsmagick-libmagick-dev-compat, I was not able to install the PHP Imagick extension (it couldn’t find ImageMagick).

    So the easy workaround seems to be using php:alpine, or installing ImageMagick from source as suggested by @krzysztof-kabala.

  13. beinbm commented on Sep 3, 2022

    @beinbm
    beinbm
    on Sep 3, 2022

    If someone reaches this thead:
    Performance issue is resolved in newer Imagemagick package of debian 11 (bullseye)

    Docker: php:7.3-apache-buster
    Debian 11 (bullseye)
    Version: ImageMagick 6.9.11-60 Q16 x86_64 2021-01-25 https://imagemagick.org/
    PHP version: 7.3.33

    0.00 seconds - init
    0.17 seconds - read
    0.00 seconds - strip
    1.16 seconds - resize
    0.00 seconds - compress
    0.00 seconds - interlace
    0.30 seconds - blob
    1.63 seconds - TOTAL

    Docker: php:7.3-apache-buster
    Debian 10 (buster)
    Version: ImageMagick 6.9.10-23 Q16 x86_64 20190101 https://imagemagick.org/
    PHP version: 7.3.33

    0.00 seconds - init
    0.15 seconds - read
    0.00 seconds - strip
    26.97 seconds - resize
    0.00 seconds - compress
    0.00 seconds - interlace
    0.57 seconds - blob
    27.69 seconds - TOTAL

    For us the easiest fix of the performance problem was to upgrade debian :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions