IMagick created huge temp files (1.7TB)

Post any defects you find in the released or beta versions of the ImageMagick software here. Include the ImageMagick version, OS, and any command-line required to reproduce the problem. Got a patch for a bug? Post it here.
Locked
AnavPT
Posts: 33
Joined: 2020-04-15T04:01:15-07:00
Authentication code: 1152

IMagick created huge temp files (1.7TB)

Post by AnavPT »

Hello guys,

I'm into start / running a stock illustrations website with Symbistock what uses ImageMagick.

I uploaded around 40 GB EPS and JPG combos. If everything goes (the developers saying), Symbiostock only does a preview generation on upload from the JPG of the EPS, JPG combo. The different file sizes which available on my site are generated when a user purchases.

It's strange tho because it eats up all the server resources (a lot) when doing this "only preview generation".

Now what brought me here is that I got a huge temp directory.

These files should be deleted somehow.

My hosting, MDDhosting uses ImageMagick 6.7.8-9.

On this page https://imagemagick.org/download/releases/ I couldn't even found that version when searching with ctrl-f.

So my question(s):

Why are those files aren't deleted automatically? I guess its a bug. What is more probable, the problem is with my hosting, or is it with my software?
How should get those files deleted? I though of a cronjob that does it: https://stackoverflow.com/questions/118 ... -tmp-files

Help is much appreciated I'm not that tech-savvy and the developer has limited support.

P.S. If somebody could solve it, I'm ready to pay for it. F.e. Write me a cron-job that deletes the files

Thank you!

AnavPT
Posts: 33
Joined: 2020-04-15T04:01:15-07:00
Authentication code: 1152

Re: IMagick created huge temp files (1.7TB)

Post by AnavPT »

When I was searching for a solution, I stumbled upon threads that was about on limiting ImageMagick's resource usage while do not letting to stop the operation. This is another problem. ... As I said above it eats up all the resources and my site becomes unresponsive when the triggering cronjob runs the command which uses IM.

So if somebody could solve me this too, again I'm ready to pay for it.

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: IMagick created huge temp files (1.7TB)

Post by magick »

Read https://imagemagick.org/script/security-policy.php and implement the recommended practices. With a proper security policy, no temporary files should remain after a workflow completes.

AnavPT
Posts: 33
Joined: 2020-04-15T04:01:15-07:00
Authentication code: 1152

Re: IMagick created huge temp files (1.7TB)

Post by AnavPT »

Ok, thank you very much! So that's it. That's where tweak needed.

As I said I'm not tech-savvy. Could you (or anybody else) write these rules for my scenario?

Also, should I implement it, or my hostings needs to make these changes?

Ty very much!

AnavPT
Posts: 33
Joined: 2020-04-15T04:01:15-07:00
Authentication code: 1152

Re: IMagick created huge temp files (1.7TB)

Post by AnavPT »

*Also my JPGs are 70-100 MP in dimensions and I'm on a shared hosting.

By reducing let's say to 4 MP, could I achieve anything?

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: IMagick created huge temp files (1.7TB)

Post by magick »

70-100MP in dimensions? What does that mean? What is the maximum width and height of your JPEG image?

We can't write security rules for you other than recommending you start with the security policy we provide @ https://imagemagick.org/script/architecture.php and build on that.

AnavPT
Posts: 33
Joined: 2020-04-15T04:01:15-07:00
Authentication code: 1152

Re: IMagick created huge temp files (1.7TB)

Post by AnavPT »

E.g. 8500 x 9500 pixels JPEG

EPS: 4-4.25 MegaPixel constant with different aspect ratios

Could you suggest me what what field of expertise freelancer should I ask to make it for me?

BTW when I was on the biggest hosting plan (has plenty of reasources) it ate up that too.

Thanks!

User avatar
magick
Site Admin
Posts: 11254
Joined: 2003-05-31T11:32:55-07:00

Re: IMagick created huge temp files (1.7TB)

Post by magick »

For paid help, post to https://imagemagick.org/discourse-serve ... m.php?f=24. A 8500x9500 image requires about 2GB of memory or disk to read and write. Some image types can be problematics and thus the need of a security policy. Take a Postscript image, for example. It might contain 100 pages and ImageMagick will attempt to render all the pages. If you only need the first page, you can specify that with 'image.ps[0]' for example.

Given the above, a single JPG image should not produce a 1.7TB temp file. You need to trace the workflow, chances are you or someone tries to upload a multi-frame image that ended up exceeded the capacity of your host.

AnavPT
Posts: 33
Joined: 2020-04-15T04:01:15-07:00
Authentication code: 1152

Re: IMagick created huge temp files (1.7TB)

Post by AnavPT »

Not 1 image but 40 GB of it caused it.

AnavPT
Posts: 33
Joined: 2020-04-15T04:01:15-07:00
Authentication code: 1152

Re: IMagick created huge temp files (1.7TB)

Post by AnavPT »


Locked