[sldev] [IDEA] A new data structure for SL database servers (3)

[Yuri Pankratov]

> CAIDA has had Walrus out for years

Indeed. But my post was not about Walrus or any other CAIDA production at 
all...it was about the "Metrized Small World" data structure.

> I seriously doubt Secondlife data can be coerced into an directed graph 
> structure at the moment. Perhaps parts.

Why do you think it is imposible? Is there some fundamental obstacle that 
prevents SL data from being stored in XML format??? I'd like to hear your 
argumentation.

> Nor does a traversed graph structure lend itself to high performance.

I disargee with you on that part. The data strcuture I am talking about can 
be as effective as distributed hash tables. And again, you provide no 
argumentation.


----- Original Message ----- 
From: Ann Otoole
To: Yuri Pankratov ; sldev at lists.secondlife.com
Sent: Thursday, October 02, 2008 6:59 PM
Subject: Re: [sldev] [IDEA] A new data structure for SL database servers


CAIDA has had Walrus out for years:

http://www.caida.org/tools/visualization/walrus/

Visualization:

http://www.caida.org/tools/visualization/walrus/gallery1/

I seriously doubt Secondlife data can be coerced into an directed graph 
structure at the moment. Perhaps parts.

Nor does a traversed graph structure lend itself to high performance.

However a true object model does lend itself well to a graph model and code 
can be diagnosed for defects rapidly when properly documented in a metadata 
repository and displayed using tools that can model relationships in a 3D 
representation. In fact I think it would do wonders for the future of 
Secondlife if the developers had the time and inclination to come up with a 
way to render code relationships inside of a Secondlife simulator so they 
could "fly through" and spot broken relationships and acyclic (recursive) 
references that lead to software failures.

Such a tool would certainly be "eating one's own dog food" and would likely 
lead to significant improvements over time in code reliability.

The problem would be the 15,000 primitive limit in the simulators.

Anyway I encourage the Lab's developers to have a look at the rich toolset 
that CAIDA makes available. Many many gems in that repository.






----- Original Message ----
From: Yuri Pankratov <ypankrat at meralabs.com>
To: sldev at lists.secondlife.com
Sent: Wednesday, October 1, 2008 5:23:23 PM
Subject: [sldev] [IDEA] A new data structure for SL database servers


Hello everyone,

I am new to this mailing list so let me introduce myself first. My name is 
Yuri Pankratov, I am a post-grad student and a research engineer from Russia 
. I have been a SL resident for quite some time now and while I find SL to 
be a wonderful idea with great implementation, I have to say that it is 
facing some scalability problems: with so many concurrent users the database 
servers are extremely stressed. This causes many unwanted things to happen, 
like increased lag, inaccessible inventory or inventory loss. I have 
suffered all these things myself.

There is good news, though. It happened so I became a research engineer in 
an R&D company. One of the company's projects is devoted to the creation of 
an innovative distributed data structure which would allow to create 
virtually infinite data repositories. An even better news is that the data 
structure provides a O(log n) data search algorithm, which means that you 
can quickly find required data even in a very large data repository. The 
data structure in its current implementation is intended to store data in 
XML format. That gave me a thought: "Hey, most SL data is stored in XML 
form, so maybe our data structure would be just the thing for storing SL 
data!". I would be really glad if  Linden developers would take interest in 
my idea. I also welcome everyone to express their opinion on the topic.

The name of my company is MeraLabs and the name of the data structure is 
"Metrized Small World". Now, some useful links:

My company's website: http://meralabs.com

This is an example of how the data structure looks:
http://meralabs.com/media/meralabs/images/skeleton_100k_v2.png

An article describing the data structue in detail:
http://meralabs.com/media/meralabs/publications/files/METRIZED_SMALL_WORLD_PROPERTIES_BASED_DATA_STRUCTURE.pdf

The company's press-release which metions the data structure:
http://www.24-7pressrelease.com/press-release/meralabs-company-presents-an-innovative-data-storage-technology-63155.php

Also you may want to see another our press-release which describes our 
concept of Cognitive Internet (I think it goes well with the philosophy of 
SL creators):
http://www.24-7pressrelease.com/press-release/meralabs-envisions-that-the-internet-of-the-future-will-be-cognitive-66192.php

Best regards,
Yuri Pankratov.