[sldev] P2P/Squid Web Textures: Enabling Greater Quality Images - draft 2

[Dzonatas]

First, let me reply to a few questions in summary here:

* Image Quality
On a questions about image quality comparisons, this is highly 
noticeable with sculpties that have the slightest artifact of 
difference. Higher quality images are not just about visible textures 
that go on the face a prim.

* Static Transfer Rates, Bandwdith, and Lag
On top of that is the cost, I believe people understand the cost of per 
GB that remote storage and colocation facilities charge. Besides that, 
there is also the immense bandwidth increase with larger images, so we 
also want to reduce the cost of server lag. For these reasons, the sim 
network can only be seen as a fallback rather than a primary resource to 
acquire the images.

* Squid &. P2P torrents
I did forget to jot down about Squid, but I did remember to put the 
"web" in between "p2p WEB textures." To make it clearer, I change the 
title of this to "P2P/Squid Web Textures."
Squid is not a drop-in solution from what has been found/tested so far. 
It works well in some situations and horrible in others. I'm thinking a 
shallow Squid that is P2P enabled to connect to others like Squids will 
be the best solution. The Squid would be able to determine if a P2P 
connection or a http redirect (to another Squid) is the best optimal 
route. It would be a custom solution, so there is no way to immediately 
use Squid as-is. It can help avoid sim and viewer changes, however.  
Main advantage here is the ability/potential to transfer smaller images 
faster without the bottleneck of torrent initialization. Main 
disadvantage: facilities like Amazon S3 only support P2P and can't do a 
custom squid.

* P2P Viewer-Side & Security
Obvious, the P2P option on the viewer side has to be optional. To enable 
P2P in the viewer is probably the easier solution, but not everyone will 
be able to use it. For those that can't use P2P in the viewer, they 
would have to override all request to the Squid texture server address. 
As for IP addresses, we could enable Tor, but I doubt it is really 
needed just to enable higher quality images.

* Aggregation/Combined Textures
Yep. This has been thought about in many ways. The sim server could 
currently patch tiles of images together and send them as one, but from 
what I gather this kind of option just won't happen on the sim level. It 
doesn't appear to be about process time, as it appears more about lag 
and that the sim network shouldn't be dealing directly with textures 
like they are now. The feature to combine textures together is a gain 
for optimal performance, especially over P2P. It is of lower-priority, 
however.

* P2P Scrapes/DHT
If there was a way to filter out non-SL users from scrapes and the DHT, 
this would help optimize torrents that are smaller in size. It's an 
idea, but I'd mark it down as lower-priority.

+++

There has already been some work done (or has been available) on the sim 
side. For the proof-of-concept stage, we don't need to make any sim side 
changes. In the future, special accounts can help the sim optimize the 
connection for daemons or provide an optional level of security.

A quick update to the steps without sim changes:

In order to improve the quality of textures without an increase in cost 
on the sims, we need to use P2P Web textures.

Steps required*:

1. A mod in the viewer that enables lossless compression for image uploads.
2. Optional use of the extra prim parameter 
(https://jira.secondlife.com/browse/SVC-350), to help redirect the 
viewer to Squid addresses or torrent trackers rather than the defaults.
3. A mod in the viewer to support P2P torrents -- made optional -- and 
default to Squid addresses.
4. An external site to host the default torrents, and a fallback to 
collect data from the sim for images that are not stored on the host 
Squid or found over DHT (if enabled).
5. A mod in a viewer to try to download all textures via the P2P/Squid 
method *first* from #4

*NOTE: These are just basic steps involved subject to change.

The other steps posted in the previous version help insure there won't 
be an extra cost hit on the sim network, so these were taken out to 
narrow the above list.

What else needs to change before we fill-in the details?

-- 
Power to Change the Void
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.secondlife.com/pipermail/sldev/attachments/20070707/548737d2/attachment.htm