Skip to main content


10 GB of #ImageMagic temp files filled up my /tmp. Anyone know why this lately happens? Didn't you mention something like this @Steffen K9 🐰 ?

!Friendica Support

Friendica Support reshared this.

@utzer Yep, that was me. I had that issue from time to time. Then I explored the depths ;-) of imagemagick configuration. Then it was working pretty well. But recently the occasional trouble (huge amounts of memory are allocated*, a lot of temp files written to disk) has returned.

*) around 10-15 GB RAM!
@utzer If you ask me ImageMagick is a piece of shit. Unfortunately, there is no alternative when you want animated gif support.
@Steffen K9 🐰 hmm so there is no configuration to make sure the /tmp is cleaned up? I might just purge all there that is older than 2 hours once a day or so.
@Steffen K9 🐰 I didn't have this problem before I think, but who know. The server reboots in less than 5 seconds, to the reboot was my choice just now.
I also had the same problem and after a bit of reading I have my theory:
https://www.imagemagick.org/script/architecture.php#tera-pixel
(1) ImageMagick allocates memory for the images while processing them since it doesn't know in advance how much memory it needs
(2) eventually, because the image is big or because of server load, it cannot allocate more memory
(3) at this point it will start to use the pixel cache on disk
(4) the default path specified in policy.xml is to create it in /tmp
(5) if /tmp is a ram based tmpfs like on my server it is full (or almost full) when the ram is exhausted
(6) php-imagick coredumps while allocating memory according to gdb
(7) some temp files in /tmp are not cleaned making the problem even worse

I did set the temporary path to a location on disk and did not have problems so far, but maybe I was just lucky.

imagemagick is configured in /etc/ImageMagick-7/policy.xml

<policymap>

  <policy domain="resource" name="temporary-path" value="/var/tmp"/>
@Steffen K9 🐰 tried as a replacement with the compatible multi exec version?
@utzer replaced imagemagick with graphicsmagick, replaced php-imagick with php-... (cannot remember the name) -> processing of animated gifs for Friendica doesn't work.

W/o working animated gif support it is useless because I can simply use libgd (php-gd) instead. Which doesn't have any issues with memory usage but also no animated gif support.

I think that the issue should be resolved somehow, because... it's annoying somehow to repeatedly cleanup /tmp:

704K    /tmp/magick-9401YTtiL3JwXBHL
704K    /tmp/magick-9401YUTi3giny7PD
704K    /tmp/magick-9401YVmWyesJTSuv
704K    /tmp/magick-9401_YWfVHIGDFOI
704K    /tmp/magick-9401YXaOUV2EPRAq
704K    /tmp/magick-9401YXCFXiZ3ALPd
704K    /tmp/magick-9401YXg5_k0DhtNX
704K    /tmp/magick-9401YZewnIkfRJNq
704K    /tmp/magick-9401YzmES3jxDgNI
704K    /tmp/magick-9401z3kgphI0KSJ9

It's all the same file, many times...

My /tmp is a 2 GB tmpfs mount (red line):

You can solve this either by purging ImageMagick from your setup (aka nuke it from orbit) or by configuring a different temp path for ImageMagick (with approximately 25G free space).

Btw. for me this mostly happens when someone is uploading and sharing a large animated gif locally. I recently did that myself with a 5MB gif and the server was swapping around 15GB of data.
You can also reduce the memory ImageMagic can use in /etc/ImageMagick-7/policy.xml:
<policy domain="resource" name="memory" value="2GiB"/>

I think the 2G default limit is for every frame of the animated gif
@Ermanno@Steffen K9 🐰 Ah, thanks for the hint, but for me it is already set to 256 Mib:
<policy domain="resource" name="memory" value="256MiB"/>

Comparing that to the 704K from above this is still within the limit. My impression is that it is over and over repeated in tryting to convert something and maybe the return code is not considered when it failed. Or maybe it is, and this is why it is always retried forever. Maybe this should only retried 3 times or so and then the job should fail and be removed. I don't know...
Maybe the 704K from above are just nulls....
the php-imagick extension runs ImageMagick as a library inside the same address space as php, if ImageMagick segfaults or otherwise abnormally ends the entire php thread dumps core and is terminated. There is no return code since the call never returns and even the code that should check the return cose no longer runs.
To check the return code ImageMagick commands should be run externally and not using a php extension.
@Ermanno Ah, ok, maybe @Steffen K9 🐰 can say more about this and change the invocation of imagick to a plain external call instead of via php-imagick?
Unlikely since doing it creates a lot of different problems. The solution is to use a disk based tmep path for imagemagick
Yep. That's possible. The 'convert' command (which is part of "ImageTragick") could be called externally for converting/resizing large images. This would need a coding in Friendica where images larger than X pixels are handed over to 'convert' instead of using php-imagick.
@Steffen K9 🐰 Ok, I changed the path from /tmp to /var/tmp (on disk, not tmpfs) and changed the setting (and others from your example below) in imagicks policy definition.

The result seems to be this:

95G /var/tmp/

¯_(ツ)_/¯
@utzer@Steffen K9 🐰 In thet last 10 mins there are again lots of cruft files:

405M total

I have no idea how to solve this issue for good...
@Steffen K9 🐰 FTR, after removing the files the size is now back to normal:

# find /var/tmp/ -iname 'magick-*' -type f -exec /bin/rm {} \+
# du -sch /var/tmp/
595M    /var/tmp/


So, it's not like that much disk space makes it better and imagick happily consumes disk space as long as there is free space... :)

Can we get rid off of imagick please? I cannot remove it from system because other stuff may rely on it, so maybe a config option for GIF suppot/imagick would help here?
Just uninstall php-imagick. As long you have installed php-gd (libgd) nothing will happen. Nothing, except no animated GIFs anymore.

If you ask me GIF should die (again). It died once together with the marquee HTML element back in the 90s. Then some asshole invented memes and brought that shit back to life. ;-)
@Roland Häder@Steffen K9 🐰@Ermanno I agree. Reason for moving /tmp to tmpfs was reducing the disk IO.
Disk IO may not be an issue if you run 1 or 2 small VMs on your host server, but when you do run a dozen VMs with lots of different apps and websites, disk IO is actually a real issue.
@Steffen K9 🐰 would you be so kind to give some feedback in a few days?
@Steffen K9 🐰 I was also wondering why we even process gif, of course there are other formats that are processed and need to be processed, but why do we do anything with gif?
And what changed in Friendica, I think the current stable does not have such problems, or do you guys see the same there?
@utzer@Steffen K9 🐰 I had the same issue with the stable release as well, maybe even with the release before that...
@Ingo Jürgensmann thanks for replying. Then maybe my public server on stable (smaller than my personal by every number I can think of (except number of accounts) just didn't have this problem yet.

Also about the clean up script, I didn't find the time yet. Aber aufgeschoben ist nicht aufgehoben.
@utzer Well, maybe it's not that much of an issue of ImageMagick, but also of PHP version? For example between Debian and Ubuntu?
@Ingo Jürgensmann php versions should be the same, it is the same OS and one of the newer versions of php.
@Ingo Jürgensmann it is two different VMs, so it could be different configuration.