[sldev] OpenJPEG meta decode, Second Life patches
David Fries
david at fries.net
Sun Feb 11 14:26:06 PST 2007
OpenJPEG feature, partial image decode
OPJ_limit_tags_for_decode.diff
Contains the modifications to OpenJPEG to limit the decoded tags to
the ones specified. There is a set of enumerations specified in the
openjpeg.h header file and an additional variable in the
opj_dparameters structure to hold a bitmask of those values. If it is
zero, all of them are decoded, if it is another value then decoding
will be stopped when any of those tags are encountered. There is a
macro OPJ_LIMIT_FOR_SIZE that contains the two required to get just
the image size. Only the j2k decoding uses this extra flag and only
j2k_decode checks the parameter, so it isn't terribly fine grained.
But it works great to get just the size.
Bug fixes and the Second Life side of the above patch.
llimagej2coj_bug_fixes.patch
These are against the src-FL-1.13.3.57876 source code release, but it
should work with many others. The first look client with a thread for
texture decoding is required to get the full performance improvements
with this. I have a dual core CPU so someone else will have to
comment how well it works with a single CPU system, I know I could set
all the threads to one CPU, but I'll let someone else do that.
New bug with both patches. VWR-123
llimagej2coj_fixes.diff
LLImageJ2COJ::decodeImpl
Destroy OpenJPEG decompression and cio structures before checking the
image so that they don't have to be called in two different places.
Bug:
return FALSE if the image decode fails not 1
Bug: VWR-66 segfault on decoding images of clothing
A rework of the patch included in VWR-66 which fixes the problem with
texture scaling. decodeImpl would provide a reduce value which would
cause the resulting decode to be 2^reduce smaller in width and height,
with the final image taking up the top left of the buffer with the
rest black. This resulted in what should have been reduced size
textures taking up much more texture memory than they should and the
image only taking up a small part of that texture. The VWR-66 bug
report was on the crash, which the patch just avoided until it decoded
the entire image, but VWR-94 fixes.
Bug: VWR-94 Buffer overflow in decoding image.
See VWR-94 for the details, but the patch is included in here. VWR-68
and VWR-94 are not related, they are independent bugs in the same
code.
LLImageJ2COJ::getMetadata
Performance improvement:
parameters.cp_limit_tags=OPJ_LIMIT_FOR_SIZE;
This is only available when OpenJPEG has been patched with the above
patch. This makes use of the partial decode feature of that patch.
It looks to me that Second Life will call the getMetadecode when it
gets the first part of an image. When getMetadecode was doing a full
decode of the image, opj_decode would return NULL because the image
wasn't there to decode. Some code would then see that the image was 0
by 0 and I'm guessing would flag the image as corrupt and restart the
texture download. That last part I'm guessing on, but now that it is
only decoding enough to get the image size information it can stop
decoding much sooner and the part of the image it needs is there so it
doesn't mark it as bad. Win win.
Bug: VWR-113 bad getMetadata return code
Even when the image failed to decode (see last issue) the return value
was 1 instead of 0 (FALSE). Fixed.
Now that I've sent in the patches, who is going to be the first to
join me in Second Life with them?
Space CoLab 100,200,24
-- SpacedOut Frye
--
David Fries <david at fries.net>
http://fries.net/~david/ (PGP encryption key available)
-------------- next part --------------
Index: libopenjpeg/openjpeg.h
===================================================================
--- libopenjpeg/openjpeg.h (revision 338)
+++ libopenjpeg/openjpeg.h (working copy)
@@ -311,6 +311,41 @@
} opj_cparameters_t;
+/** Stop after tags. */
+typedef enum LIMIT_TAGS {
+ OPJ_TAG_SOC = 0x000001, /**< start of codestream */
+ OPJ_TAG_SOT = 0x000002, /**< start of tile-part*/
+ OPJ_TAG_SOD = 0x000004, /**< start of data */
+ OPJ_TAG_EOC = 0x000008, /**< end of codestream */
+ OPJ_TAG_SIZ = 0x000010, /**< image and tile size */
+ OPJ_TAG_COD = 0x000020, /**< coding style default */
+ OPJ_TAG_COC = 0x000040, /**< coding style component */
+ OPJ_TAG_RGN = 0x000080, /**< region-of-interest */
+ OPJ_TAG_QCD = 0x000100, /**< quantization default */
+ OPJ_TAG_QCC = 0x000200, /**< quantization component */
+ OPJ_TAG_POC = 0x000400, /**< progression order change */
+ OPJ_TAG_TLM = 0x000800, /**< tile-part lengths */
+ OPJ_TAG_PLM = 0x001000, /**< packet length, main header */
+ OPJ_TAG_PLT = 0x002000, /**< packet length, tile-part header */
+ OPJ_TAG_PPM = 0x004000, /**< packet packet headers, main header */
+ OPJ_TAG_PPT = 0x008000, /**< packet packet headers, tile-part header */
+ OPJ_TAG_SOP = 0x010000, /**< SOP marker value */
+ OPJ_TAG_EPH = 0x020000, /**< EPH marker value */
+ OPJ_TAG_CRG = 0x040000, /**< component registration */
+ OPJ_TAG_COM = 0x080000, /**< comment */
+#ifdef USE_JPWL
+/* UniPG>> */
+ OPJ_TAG_EPC = 0x100000, /**< EPC marker value (Part11) */
+ OPJ_TAG_EPB = 0x200000, /**< EPB marker value (Part11) */
+ OPJ_TAG_ESD = 0x400000, /**< ESD marker value (Part11) */
+ OPJ_TAG_RED = 0x800000, /**< RED marker value (Part11) */
+#endif /* USE_JPWL */
+/* <<UniPG */
+} OPJ_LIMIT_TAGS;
+
+/** The needed tags to decode just the size of the image. */
+#define OPJ_LIMIT_FOR_SIZE (OPJ_LIMIT_TAGS)(OPJ_TAG_SOC | OPJ_TAG_SIZ )
+
/**
Decompression parameters
*/
@@ -330,6 +365,15 @@
if == 0 or not used, all the quality layers are decoded
*/
int cp_layer;
+ /**
+ Limits the tags that are decoded.
+ This is a bitwise OR of the tags to be decoded. If a tag is
+ encountered that isn't in the list decoding ceases and the function
+ returns.
+ if != 0 only the given tags are decoded.
+ if == 0 all tags are decoded.
+ */
+ OPJ_LIMIT_TAGS cp_limit_tags;
/**@name command line encoder parameters (not used inside the library) */
/*@{*/
Index: libopenjpeg/j2k.c
===================================================================
--- libopenjpeg/j2k.c (revision 338)
+++ libopenjpeg/j2k.c (working copy)
@@ -1342,37 +1342,39 @@
int states;
/** action linked to the marker */
void (*handler) (opj_j2k_t *j2k);
+ /** The enum value used to limit decode to specific tags. */
+ OPJ_LIMIT_TAGS limit_tag;
} opj_dec_mstabent_t;
opj_dec_mstabent_t j2k_dec_mstab[] = {
- {J2K_MS_SOC, J2K_STATE_MHSOC, j2k_read_soc},
- {J2K_MS_SOT, J2K_STATE_MH | J2K_STATE_TPHSOT, j2k_read_sot},
- {J2K_MS_SOD, J2K_STATE_TPH, j2k_read_sod},
- {J2K_MS_EOC, J2K_STATE_TPHSOT, j2k_read_eoc},
- {J2K_MS_SIZ, J2K_STATE_MHSIZ, j2k_read_siz},
- {J2K_MS_COD, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_cod},
- {J2K_MS_COC, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_coc},
- {J2K_MS_RGN, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_rgn},
- {J2K_MS_QCD, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_qcd},
- {J2K_MS_QCC, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_qcc},
- {J2K_MS_POC, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_poc},
- {J2K_MS_TLM, J2K_STATE_MH, j2k_read_tlm},
- {J2K_MS_PLM, J2K_STATE_MH, j2k_read_plm},
- {J2K_MS_PLT, J2K_STATE_TPH, j2k_read_plt},
- {J2K_MS_PPM, J2K_STATE_MH, j2k_read_ppm},
- {J2K_MS_PPT, J2K_STATE_TPH, j2k_read_ppt},
- {J2K_MS_SOP, 0, 0},
- {J2K_MS_CRG, J2K_STATE_MH, j2k_read_crg},
- {J2K_MS_COM, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_com},
+ {J2K_MS_SOC, J2K_STATE_MHSOC, j2k_read_soc, OPJ_TAG_SOC},
+ {J2K_MS_SOT, J2K_STATE_MH | J2K_STATE_TPHSOT, j2k_read_sot, OPJ_TAG_SOT},
+ {J2K_MS_SOD, J2K_STATE_TPH, j2k_read_sod, OPJ_TAG_SOD},
+ {J2K_MS_EOC, J2K_STATE_TPHSOT, j2k_read_eoc, OPJ_TAG_EOC},
+ {J2K_MS_SIZ, J2K_STATE_MHSIZ, j2k_read_siz, OPJ_TAG_SIZ},
+ {J2K_MS_COD, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_cod, OPJ_TAG_COD},
+ {J2K_MS_COC, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_coc, OPJ_TAG_COC},
+ {J2K_MS_RGN, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_rgn, OPJ_TAG_RGN},
+ {J2K_MS_QCD, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_qcd, OPJ_TAG_QCD},
+ {J2K_MS_QCC, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_qcc, OPJ_TAG_QCC},
+ {J2K_MS_POC, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_poc, OPJ_TAG_POC},
+ {J2K_MS_TLM, J2K_STATE_MH, j2k_read_tlm, OPJ_TAG_TLM},
+ {J2K_MS_PLM, J2K_STATE_MH, j2k_read_plm, OPJ_TAG_PLM},
+ {J2K_MS_PLT, J2K_STATE_TPH, j2k_read_plt, OPJ_TAG_PLT},
+ {J2K_MS_PPM, J2K_STATE_MH, j2k_read_ppm, OPJ_TAG_PPM},
+ {J2K_MS_PPT, J2K_STATE_TPH, j2k_read_ppt, OPJ_TAG_PPT},
+ {J2K_MS_SOP, 0, 0, OPJ_TAG_SOP},
+ {J2K_MS_CRG, J2K_STATE_MH, j2k_read_crg, OPJ_TAG_CRG},
+ {J2K_MS_COM, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_com, OPJ_TAG_COM},
/* UniPG>> */
#ifdef USE_JPWL
- {J2K_MS_EPC, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_epc},
- {J2K_MS_EPB, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_epb},
- {J2K_MS_ESD, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_esd},
- {J2K_MS_RED, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_red},
+ {J2K_MS_EPC, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_epc, OPJ_TAG_EPC},
+ {J2K_MS_EPB, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_epb, OPJ_TAG_EPB},
+ {J2K_MS_ESD, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_esd, OPJ_TAG_ESD},
+ {J2K_MS_RED, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_red, OPJ_TAG_RED},
#endif /* USE_JPWL */
/* <<UniPG */
- {0, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_unk}
+ {0, J2K_STATE_MH | J2K_STATE_TPH, j2k_read_unk, -1}
};
static void j2k_read_unk(opj_j2k_t *j2k) {
@@ -1522,6 +1524,7 @@
opj_cp_t *cp = (opj_cp_t*)opj_malloc(sizeof(opj_cp_t));
cp->reduce = parameters->cp_reduce;
cp->layer = parameters->cp_layer;
+ cp->limit_tags = parameters->cp_limit_tags;
/* UniPG>> */
#ifdef USE_JPWL
cp->correct = parameters->jpwl_correct;
@@ -1599,6 +1602,12 @@
opj_event_msg(cinfo, EVT_ERROR, "%.8x: unexpected marker %x\n", cio_tell(cio) - 2, id);
return 0;
}
+ /* If a partial decode is requested, stop if the current tag
+ * isn't in the list.
+ */
+ if (j2k->cp->limit_tags && !(j2k->cp->limit_tags & e->limit_tag)) {
+ return image;
+ }
if (e->handler) {
(*e->handler)(j2k);
}
Index: libopenjpeg/j2k.h
===================================================================
--- libopenjpeg/j2k.h (revision 338)
+++ libopenjpeg/j2k.h (working copy)
@@ -197,6 +197,8 @@
int reduce;
/** if != 0, then only the first "layer" layers are decoded; if == 0 or not used, all the quality layers are decoded */
int layer;
+ /** if != 0, then only decode specific tags, abort on any other; if == 0 decode all tags */
+ OPJ_LIMIT_TAGS limit_tags;
/** 0 = no index || 1 = index */
int index_on;
/** XTOsiz */
-------------- next part --------------
Only in llimagej2coj: .#llimagej2coj.cpp.1.7
Only in llimagej2coj: .llimagej2coj.cpp.swp
Only in llimagej2coj: .llimagej2coj.h.swp
Only in llimagej2coj: CVS
diff -upr llimagej2coj_FL-1.13.3.57876.orig/llimagej2coj.cpp llimagej2coj/llimagej2coj.cpp
--- llimagej2coj_FL-1.13.3.57876.orig/llimagej2coj.cpp 2007-02-09 23:49:21.000000000 -0600
+++ llimagej2coj/llimagej2coj.cpp 2007-02-10 20:23:08.000000000 -0600
@@ -122,46 +122,58 @@ BOOL LLImageJ2COJ::decodeImpl(LLImageJ2C
/* decode the stream and fill the image structure */
image = opj_decode(dinfo, cio);
- if(!image)
- {
- fprintf(stderr, "ERROR -> j2k_to_image: failed to decode image!\n");
- opj_destroy_decompress(dinfo);
- opj_cio_close(cio);
- return 1;
- }
/* close the byte stream */
opj_cio_close(cio);
-
/* free remaining structures */
- if(dinfo) {
+ if(dinfo)
+ {
opj_destroy_decompress(dinfo);
}
+ // The image decode failed if the return was NULL or the component
+ // count was zero. The latter is just a sanity check before we
+ // dereference the array.
+ if(!image || !image->numcomps)
+ {
+ fprintf(stderr, "ERROR -> j2k_to_image: failed to decode image!\n");
+ return FALSE;
+ }
+
// Copy image data into our raw image format (instead of the separate channel format
- S32 width = 0;
- S32 height = 0;
S32 img_components = image->numcomps;
S32 channels = img_components - first_channel;
if( channels > max_channel_count )
- {
channels = max_channel_count;
- }
- width = image->x1 - image->x0;
- height = image->y1 - image->y0;
+
+ // Component buffers are allocated in an image width by height buffer.
+ // The image placed in that buffer is ceil(width/2^factor) by
+ // ceil(height/2^factor) and if the factor isn't zero it will be at the
+ // top left of the buffer with black filled in the rest of the pixels.
+ // It is integer math so the formual is written in ceildivpo2.
+ // (Assuming all the components have the same width, height and
+ // factor.)
+ S32 comp_width = image->comps[0].w;
+ S32 f=image->comps[0].factor;
+ S32 width = ceildivpow2(image->x1 - image->x0, f);
+ S32 height = ceildivpow2(image->y1 - image->y0, f);
raw_image.resize(width, height, channels);
U8 *rawp = raw_image.getData();
- for (S32 comp = first_channel; comp < first_channel + channels; comp++)
+ // first_channel is what channel to start copying from
+ // dest is what channel to copy to. first_channel comes from the
+ // argument, dest always starts writing at channel zero.
+ for (S32 comp = first_channel, dest=0; comp < first_channel + channels;
+ comp++, dest++)
{
- S32 offset = comp;
+ S32 offset = dest;
for (S32 y = (height - 1); y >= 0; y--)
{
for (S32 x = 0; x < width; x++)
{
- rawp[offset] = image->comps[comp].data[y*width + x];
+ rawp[offset] = image->comps[comp].data[y*comp_width + x];
offset += channels;
}
}
@@ -323,6 +335,9 @@ BOOL LLImageJ2COJ::getMetadata(LLImageJ2
/* set decoding parameters to default values */
opj_set_default_decoder_parameters(¶meters);
+ // Only decode what's required to get the size data.
+ parameters.cp_limit_tags=OPJ_LIMIT_FOR_SIZE;
+
//parameters.cp_reduce = mRawDiscardLevel;
/* decode the code-stream */
@@ -349,7 +364,7 @@ BOOL LLImageJ2COJ::getMetadata(LLImageJ2
fprintf(stderr, "ERROR -> j2k_to_image: failed to decode image!\n");
opj_destroy_decompress(dinfo);
opj_cio_close(cio);
- return 1;
+ return FALSE;
}
/* close the byte stream */
@@ -371,5 +386,6 @@ BOOL LLImageJ2COJ::getMetadata(LLImageJ2
base.setSize(width, height, img_components);
/* free image data structure */
- opj_image_destroy(image); return TRUE;
+ opj_image_destroy(image);
+ return TRUE;
}
diff -upr llimagej2coj_FL-1.13.3.57876.orig/llimagej2coj.h llimagej2coj/llimagej2coj.h
--- llimagej2coj_FL-1.13.3.57876.orig/llimagej2coj.h 2007-02-09 23:49:21.000000000 -0600
+++ llimagej2coj/llimagej2coj.h 2007-02-10 20:15:33.000000000 -0600
@@ -40,6 +40,11 @@ protected:
/*virtual*/ BOOL getMetadata(LLImageJ2C &base);
/*virtual*/ BOOL decodeImpl(LLImageJ2C &base, LLImageRaw &raw_image, F32 decode_time, S32 first_channel, S32 max_channel_count);
/*virtual*/ BOOL encodeImpl(LLImageJ2C &base, const LLImageRaw &raw_image, const char* comment_text, F32 encode_time=0.0);
+ int ceildivpow2(int a, int b)
+ {
+ // Divide a by b to the power of 2 and round upwards.
+ return (a + (1 << b) - 1) >> b;
+ }
// Temporary variables for in-progress decodes...
LLImageRaw *mRawImagep;
More information about the SLDev
mailing list