Heads up Mastodon operators: libvips 8.16.0 doesn't work with ruby-vips 2.2.2, which is currently used by Mastodon.
*** watch out if your system automatically updates libvips to 8.16.0 ***
This was one of the root causes of our media processing outage last night. Essentially when we were trying to process media, the conversion between different file sizes and metadata stripping was erroring out do to an incompatibility between ruby-vips 2.2.2 and libvips 8.16.0, resulting in the error:
```
Error while optimizing <name>: VipsForeignLoad: "/tmp/<name>.<ext>“ is not a known file format
```
This occurred for jpg, webp, and several other file formats, despite `vips -l foreign` showing everything setup correctly.
ruby-vips 2.2.3 is known to fix the issue, and will be used in mastodon 4.3.4 when it is released. Until then either don't update libvips or switch MASTODON_USE_LIBVIPS to false until mastodon 4.3.4 (unless you're comfortable modifying the Gemfile to use the newer ruby-vips version)
What we still have absolutely
... show moreHeads up Mastodon operators: libvips 8.16.0 doesn't work with ruby-vips 2.2.2, which is currently used by Mastodon.
*** watch out if your system automatically updates libvips to 8.16.0 ***
This was one of the root causes of our media processing outage last night. Essentially when we were trying to process media, the conversion between different file sizes and metadata stripping was erroring out do to an incompatibility between ruby-vips 2.2.2 and libvips 8.16.0, resulting in the error:
```
Error while optimizing <name>: VipsForeignLoad: "/tmp/<name>.<ext>“ is not a known file format
```
This occurred for jpg, webp, and several other file formats, despite `vips -l foreign` showing everything setup correctly.
ruby-vips 2.2.3 is known to fix the issue, and will be used in mastodon 4.3.4 when it is released. Until then either don't update libvips or switch MASTODON_USE_LIBVIPS to false until mastodon 4.3.4 (unless you're comfortable modifying the Gemfile to use the newer ruby-vips version)
What we still have absolutely no explanation for though is that libvips 8.16.0 was installed on hachyderm systems on January 20th 2025, but we only started seeing issues yesterday after running regular system updates wherein libvips was not touched.
Ruby and gems were reinstalled yesterday due to an update of libicu, but libicu was also updated on Jan 20th 2025 so we did the same reinstall steps on that date too.
We didn't receive any error reports from people using hachyderm of not being able to upload media or media failing to load for them.
#mastodon #hachyderm @hachyinfra
