Memory Leak with s3 Fuse s3fs Over https

We have a hosting set up on AWS (amazon web services) and we use fuse and s3fs to mount an s3 bucket to the filesystem to store files uploaded by users.

We noticed however that the s3fs process was consuming quite a lot of memory.  This is apparently due to a documented bug, that as of recent has not been resolved. The problem was so bad that one day we checked and the s3fs process was sitting on over 2 GB of memory even when the site was inactive.

The simple fix was to switch the way fuse was connecting to s3 from https to http. Voila! The problem was fixed. The publicly facing files are still accessible over https, although we we are not storing anything very sensitive anyway. This does mean that data is being transmitted from one amazon server to another unencrypted, so take this into consideration when deciding. Now s3fs uses only about 100KB of memory, even while actively moving data to our s3 bucket.

This page has some more technical details on the bug: