[opensource-dev] New HTTP Library & Project Viewer

[Carlo Wood]

On Sun, 29 Jul 2012 10:07:18 +0200
Henri Beauchamp <sldev at free.fr> wrote:

> On Sun, 29 Jul 2012 05:23:15 +0200, Carlo Wood wrote:
> 
> > On Fri, 27 Jul 2012 16:59:18 -0400
> > holydoughnuts <holydoughnuts at gmail.com> wrote:
> > 
> > The implementation that I wrote for Singularity (not yet in the
> > the official release) is more on the side of "blazing fast" :p.
> > The bottleneck is entirely server-side, but I've seen download
> > speeds of 1 MB/s non-stop till all textures were there.
> 
> With just a few tweakings to the texture fetcher (and in particular
> a setting that permits up to 32 concurrent HTTP fetches), I get much
> more than 1MB/s in the Cool VL Viewer... In fact, I'm more in the
> 10MB/s+ range while rezzing in heavily textured areas and the rezzing
> time is pretty low... This has been so for many months (well over
> a year actually) already.

I agree that that would be an advantage for the user; but it's not
clear to me if it is allowed (or will be in the near future) to
have 32 concurrent fetches.

The rationale that it SHOULD be allowed is that the user will
download those textures anyway; so, you might as well allow
them to get it quickly, in a short time: this has no influence
on the average number of open file descriptors on the server.

I understand that LL is afraid that once viewers can keep sockets
open (for reuse) that the servers run out of filedescriptors. Of course,
this is all to true. Therefore I propose to set a limit on the
maximum number of UNUSED filedescriptors. However, there should be
no limit or a much higher limit, on the number of connections that
are actually in use. The question is then: when is a filedescriptor
"unused"? Well, I'd say that for the sake of re-use, it can be
considered unused if there hasn't been a new request for it for
-say- a second.  It's probably best to define multiple stages,
that is:
* Allow a maximum of 32 connections.
* Allow up to 32 connections that are unused for 1 second.
* Allow up to 8 connections that are unused for 10 seconds.
* Allow up to 2 connections that are unused for 1 minute.
* Allow 1 connection to be unused for 20 minutes (or whenever
one leaves a region and it can be determined that this server
won't be needed or used anymore).

Of course this means that upon teleport to a new region with
many unknown textures, the viewer will make 32 new SSH connections,
but that is MUCH less than one new connection PER TEXTURE, as we
have now. Then all those connections are reused until all textures
have been downloaded (a few seconds).

Other strategies are possible, like NOT immediately creating a
new connection as long as the maximum of 32 connections haven't
been used up yet, but waiting with that until there's a queue
of requests building up.

Bottom line is, I'm afraid that Linden Lab is going to take the
(too) simplistic route as they often have done in the past, and
do something very silly like:
* Allow a maximum of 2 connections, and allow to keep them
  open indefinitely.

I hope you see the difference :p
I argue that something like that makes no sense and would hurt
the user because they'd have to wait unnecessary long before
everything downloads.

-- 
Carlo Wood <carlo at alinoe.com>