[sldev] SL Database purge

[Felix Duesenburg]

Argent Stonecutter wrote:
> On 2008-04-29, at 22:42, Felix Duesenburg wrote:
>> Argent Stonecutter wrote:
>>> On 2008-04-29, at 13:33, Phoenix wrote:
>>>> We only intentionally remove unreferenced assets. This means that it's
>>>> in no agent/task/region/etc inventory, and not even referenced via
>>>> uuid in a script state.
>>>
> 
>>> Even if it's in a box inside a box inside a box inside a box, N deep?
> 
>> That shouldn't make a difference to whether an asset key can be found or
>> not. The box is really just a kind of reference, either in a relational
>> database or in a directory structure.
> 
> Indeed, but there are any number of ways that can be implemented that
> would make the reference to the contents of the box require checking an
> additional data structure, and even unpacking a serialized data structure.
> 

How do I explain this... Try to understand that what you see on the
surface, e.g. a box seemingly hiding away something, is not the way how
things are physically stored. The information about the asset itself and
the information about its location are two completely different things,
so different that they are /physically/ kept in separate places.

The asset alone consists of a key and an attached file of various types
and is most likely stored in a flat structure, e.g. a database table
with the keys as index and the contents as blobs ('binary large objects'
in MySQL), or simply as files in a folder with the key as filename
(which is probably more efficient).

The information about its location on the other hand requires a tree
structure of some sophistication, since here we don't have just folders
similar to the ones on your hard drive or SL inventory, but objects can
also reside in-world or in the inventory of other objects. Nevertheless
this is no hindrance to traversing the tree to any depth of nesting. I
have previously implemented such structures for relational databases,
but it's even cooler to do with links in UNIX-like file systems.

It is really quite easy to search both parts for asset keys and make a
comparison to see if either an asset itself or the info about its
location has gone missing. This is done in two steps:

- One, for all keys in the asset list, traverse the structure tree and
see if you can find it. If there is at least one match, it has at least
one instance living in someone's inventory or in-world, and everything
is ok. If there is no match, the connection between the asset and its
owner has somehow been lost, and this is the case being talked about above.

- Two, traverse the structure tree and look up every key if you can find
it in the asset list. If yes, ok, if not, we have a reference but no
asset (inventory loss).

Other than for backup or archiving, it makes no sense to serialize all
this stuff as it would make searching really hard indeed, or a
performance killer at least.

I am of course not representing how LL has implemented all of this,
because I have no knowledge about their internals. This is simply the
most likely way how such a technology can be conceived, and I'm sure LL
has opted for the solution that is the most efficient and suitable for
the purpose.

HTH,
Felix