[sldev] OpenJPEG decoder update

[David Fries]

On Tue, Mar 06, 2007 at 07:27:23PM +0000, Tofu Linden wrote:
> Tofu Linden wrote:
> > Tofu Linden wrote:
> > A progress report: these all look like good changes.  However!  When
> > I enable the parameters.cp_limit_decoding=LIMIT_TO_MAIN_HEADER hint,
> > I get a large number of these:
> > 
> > [ERROR] 00001fb2: expected a marker instead of 8abf
> > ERROR -> j2k_to_image: failed to decode image!

Something sounds really wrong here.  The first number 1fb2 is the
length into the j2k image and 8114 is well beyond the image header.
It really shouldn't be getting that far if it is limited to the main
header.  I included a patch to dump each bad image to disk.  I assume
base (the object that has the image) doesn't get modified while in
getMetadecode() or added to.  You might need to modify the directory
the file is being written to.

That same error is printed in two locations in j2k.c.
If you could recompile OpenJPEG modifying the 'expected a marker
instead' in BOTH places, by adding the function or something and
figure out which one is getting called that might help.

> > ... and memory fills up very quickly, OOMing the process.
> > Without that hint, I've been able to spend a lot of time in
> > crowded areas without any mysterious memory bloat.  I'm using
> > the official OpenJPEG 1.1.1.
> > 
> > I'm still looking into it.  If I don't get it diagnosed soon
> > I'll at least check in the other fixes.
> 
> Hey, I can leave LIMIT_TO_MAIN_HEADER enabled as long as the error
> condition returns the pre-patch '1' instead of 'FALSE' - then everything
> actually seems pretty groovy, comparatively good decode speed and
> no Mystery Bloat.  I'll have a look into why, tomorrow.
> 
> - Tofu

I'm running FL-1.13.3.58185 right now.  I don't get the 'expected a
marker instead of' error and I don't get the message saying that
getMetadecode() failed.  Just for kicks and grins I went and set
getMetadecode() to return false at the top of the routine with only
base.updateRawDiscardLevel(); being called first.  No obvious memory
leak, also no textures.

-- 
David Fries <david at fries.net>
http://fries.net/~david/ (PGP encryption key available)
-------------- next part --------------
Index: llimagej2coj.cpp
===================================================================
RCS file: /home/david/src/.cvs/programming/SecondLife/linden/indra/llimagej2coj/llimagej2coj.cpp,v
retrieving revision 1.9.2.1
diff -u -p -r1.9.2.1 llimagej2coj.cpp
--- llimagej2coj.cpp	20 Feb 2007 05:24:12 -0000	1.9.2.1
+++ llimagej2coj.cpp	7 Mar 2007 03:51:30 -0000
@@ -33,6 +33,9 @@
 #include "lltimer.h"
 #include "llmemory.h"
 
+#include <fstream>
+#include <sstream>
+
 LLImageJ2CImpl* fallbackCreateLLImageJ2CImpl()
 {
 	return new LLImageJ2COJ();
@@ -361,6 +364,23 @@ BOOL LLImageJ2COJ::getMetadata(LLImageJ2
 	image = opj_decode(dinfo, cio);
 	if(!image)
 	{
+		static int index;
+		std::stringstream ss;
+		ss << "/tmp/textures/" << index++ << ".j2k";
+		std::string filename=ss.str();
+		std::ofstream file(filename.c_str());
+		if(!file)
+		{
+			fprintf(stderr, "Error opening file %s\n",
+				filename.c_str());
+		}
+		else
+		{
+			file.write((char*)base.getData(), base.getDataSize());
+		}
+	}
+	if(!image)
+	{
 		fprintf(stderr, "ERROR -> j2k_to_image: failed to decode image!\n");
 		opj_destroy_decompress(dinfo);
 		opj_cio_close(cio);