[SLDEV][VWR] Viewer Memory Manager

[Tateru Nino]

Palmer Truelson (Zen Linden) wrote:
> Sheet Spotter wrote:
>>
>> The Viewer Roadmap includes a “Viewer Memory Manager” project with a 
>> “Fix Viewer Memory Leaks” deliverable.
>>
>> http://wiki.secondlife.com/wiki/Viewer_Roadmap
>>
>> http://wiki.secondlife.com/wiki/Viewer_Memory_Manager
>>
>> Has there been progress on the Viewer Memory Manager?
>>
>> Is there an opportunity to contribute to this initiative?
>>
> Figured I could give a quick update.
>
> We tried tcmalloc for a bit in windows, but we have not come up with a 
> solution for a bug tcmalloc has when built in release mode in VC8 
> where it fails its own unit tests. I've left a report of it on the 
> google perf tools' homepage, and I'm hoping Craig or one of the other 
> google folks takes a look. By all means, take a look and see if you 
> can get tcmalloc to run in release mode in VC8. You would have at 
> least my undying gratitude. :)
>
> However, I did port (possibly a little hackily... I need to clean it 
> up) their profiler to windows. The client runs at about 5 fps, and 
> there are some image loading problems, but we are investigating a leak 
> we found that appears to be coming from some GL drivers, though it's a 
> little hairy to pin down and we're waiting to hear back from ATI.
>
> I'm unfortunately not focused on this project right now as the crash 
> reporter is keeping me busy, but others are, and we'd love to hear if 
> you come up with anything useful.
>
> Also, we haven't tried heap layers, nedmalloc, or any of the other 
> memory managers du jour yet, so if you get some promising results, 
> especially backed up by data, we'd love to hear it.

We used dlmalloc for our linux-based multi-million-user grid system a 
few years ago. Well worth the effort for us. It was a big win. One of 
the key things that made it a winner for us was that it's performance 
was no _worse_ than most other allocators, and that it could return 
unused heap back to the system pool -- a feature which no other 
allocator at the time seemed to do, and which turned out to be a 
critical win for controlling memory growth on our servers.

The conditions of that release are fairly specific - there must be 
contiguous unused memory at the end of the heap for a release to 
function (a heap shrink via sbrk) but when we put it into practice, by 
golly, it worked extremely well! It certainly allowed us to compete 
favorably with operating systems that couldn't shrink the heap (almost 
all of them), and with operating systems that _could_ but applications 
that _couldn't_.

We ended up relinking _everything_ with it, and enjoying the reduced 
memory pressure overall and better peak availability of memory to processes.

-- 
Tateru Nino
http://www.massively.com/