Recent Posts
- Tunegrove Blog Launched!
- Note to Self: Avoid Bulk SMS
- Memory Leak with s3 Fuse s3fs Over https
- Back to School
- My new HP TouchPad – Better, Faster, Stronger, Overclocked
- New Site Design 2011
- Disable Verizon V-Cast popup / autostart on Android HTC Incredible
- Mock Draft 2010 iPhone, iPad and iPod touch now Available!
- What is SEO? An introduction to Search Engine Optimization and search result rank
- Detect File Types For Files With No Extension
- Seperate the directory and file portions of a string path in PHP
- Migrating wordpress to a new server
- A customized web-based wysiwyg that can clean a paste from MS Word
- Google Talk Crashes When I Select Show current music track Status, using Winamp
- How to Fix Problmes with CSS Float Elements in IE, FireFox, Safari, etc
Memory Leak with s3 Fuse s3fs Over https
We have a hosting set up on AWS (amazon web services) and we use fuse and s3fs to mount an s3 bucket to the filesystem to store files uploaded by users.
We noticed however that the s3fs process was consuming quite a lot of memory. This is apparently due to a documented bug, that as of recent has not been resolved. The problem was so bad that one day we checked and the s3fs process was sitting on over 2 GB of memory even when the site was inactive.
The simple fix was to switch the way fuse was connecting to s3 from https to http. Voila! The problem was fixed. The publicly facing files are still accessible over https, although we we are not storing anything very sensitive anyway. This does mean that data is being transmitted from one amazon server to another unencrypted, so take this into consideration when deciding. Now s3fs uses only about 100KB of memory, even while actively moving data to our s3 bucket.
This page has some more technical details on the bug:
http://code.google.com/p/s3fs/issues/detail?id=191