Since Tuesday’s outage on Tempo, we’ve been working hard to not only prevent that from happening again, but to further improve our system’s response time and optimize our resource utilization. This kind of spring cleaning and infrastructure maintenance was long overdue and took a backseat while we were building out the most recent update.
I’m happy to report, and hopefully you’ve noticed, that we’ve managed to not only stabilize our runaway memory consumption, but to dramatically improve response time over all (go Unicorn!). Now that things have stabilized and in fact improved, we’ll be working to further cut down on memory consumption in our application code. It’s very easy to have memory bloat in Ruby code, because Ruby doesn’t really release memory back to the system, so we’re profiling our code to cut back where ever we can.
As an aside, there’s this great Engine Yard post about memory bloat in Rails apps with pointers for pinpointing problem spots and cutting back. However, I take issue the notion that “it’s not a memory leak, it’s bloat.” That’s a lot of garbage, not returning unused memory to the system is a leak, and to pretend otherwise is a laughable bit of delusion. I agree that it’s important to recognize this as a known flaw of the platform, but to start thinking of it as a feature is probably not a good idea.