[opensource-dev] Mesh viewers and tcmalloc issues

Henri Beauchamp sldev at hotmail.com
Sun Oct 2 09:45:01 PDT 2011


On Sun, 02 Oct 2011 10:20:58 -0400, Mike Chase wrote:

> One more note.  I have 16gb of memory on this system so the large heap 
> really isnt a problem per-se.

It is, because you will not be able to get more than 3Gb of virtual
memory per process, and when this virtual space gets fragmented (which
*does* happen during "long" sessions with tcmalloc and its default
release rate), your viewer will crash trying to allocate the next
unfragmented space that won't fit its currently allocated but
fragmented pool.

> I still see occasional exits (with no crashdump so I can't
> easily say what exactly is happening).

That's exactly what happens when your process tries to use over
3G of memory... Run the viewer under gdb, wait for it to crash
and do a backtrace to get the stack trace and see how it crashes
during a tcmalloc call for allocating memory...

> RSS on my system is 10000 so I'm not coming even close to that.

You'll still be stuck with the 3Gb limit per process, thanks to
Intel's insane addressing modes and poor intsruction set design...
If only IBM had chosen Motorola's 680x0 line of CPUs back in
the days when they opted for the 8086, the PCs would have had
32bits OSes with flat memory models from the very start, without
weird stuff such as paging, PAE and whatnot !...
Today, we are still paying the price of the poor design of the
x86 CPUs...

> Henri, I'm not saying your wrong (though setting a param outside
> the specified operational range for the library doesn't feel like
> a *fix* to me).

The tcmalloc algorithm for releasing memory is crippled, period.
And as I wrote, this is a *workaround* (I never wrote is was a
*fix* !). The true fix would be to fix tcmalloc itself, or better,
to do without tcmalloc !

> I do believe more testing is needed.

Feel free to test harder, but I made my mind on it, and the
Cool VL Viewer is already benefitting from my findings. You
are of course free to ignore them (I also wrote in my first
message: "For what it is worth...", so it's up to you, really).

Henri.


More information about the opensource-dev mailing list