[sldev] Script/Parcel/Memory Limits

[Aleric Inglewood]

On Wed, Dec 16, 2009 at 2:21 AM, Kent Quirk <q at lindenlab.com> wrote:

> "We're planning to make script memory usage along with our proposed
> script limits visible to all Residents for an extended period before
> enforcing any limits.

This is doomed to fail, and well for the following reason:

The whole implementation is wrong: you have implemented FIXED
limits per parcel and avatar, while they should have been
dynamic. No matter what user feedback will be received, you're
not going to change this implementation; the user input will
only be used to decide on these fixed limits (if they can
be set lower I imagine, cause after the Ex-street disaster
I don't believe that user input will ever be used to say
"hey, maybe we should set the limits higher and add some
more memory to the servers").

I understand where they "limit per avatar" and "limit per parcel"
comes from:

In case of abuse, in case of actual being out of memory, SOMEONE
has to be responsible and THAT person has to be punished. That
means you need to be able to see who is using too much memory
which means per avatar, and per parcel (since then the parcel
owner is responsible).

However, in the case of homesteads we have four homesteads
on one server... So, rather than punishing parcel owners you
should punish the region owner...

Now here is the problem:

By not having dynamic limits, the limits need to be
*MUCH* lower than they could have been!

Assume the following variables:

Max server memory: S
Region memory used: R1, R2, ...
Parcel memory used: P11, P12, .. P21, P22, ..
Avatar memory used: A1, A2, ...

If all these are FIXED limits, then no doubt you will have
for homesteads:

S = R1 + R2 + R3 + R4

R1 = P11 + P12 + P13 + ... + A_pool

where A_pool is the avatar pool, the amount of memory
reserved for avatars.

If you do not want the NORMAL user to suffer (theKellz's said
on IRC that only the *extreme* cases would be affected and
normal users wouldn't feel a thing), then EACH of those
hard limits must be set pretty high (at the high end of
the normal curve of normal usage).

For the sake of simplicity, lets assume that such a high
limit is four times higher than the average usage (which
imho is on the low side), then that means that this system
will enforce an average usage of only 25% of the maximum
avaiable server memory, which is FOUR times lower than
what we have now. Since a lot of servers ALREADY run into
the max S currently, I predict that no matter what you
try with this system, LOTS AND LOTS of regions will MAJORLY
suffer.

Example 1:

Someone owns four homesteads, all running on the same
server. He doesn't really need that much prims for his
dancing, he needs memory and cpu time. He also liked
a few parks and sea around his dance island, so he
got himself four homesteads, put almost nothing in
three of them and used the memory of the server entirely
to run the heavily visited disco on the fourth.
This is legit usage of resources: he owns all regions
on the whole server, so he owns the server.
Once you put the limits into effect, he will suddenly
only get 1/4 for the disco region of what he had before
MINUS the parcel pool, for the scripts of the visitors
that visit. In effect, almost all visitors will suddenly
run into script problems.

Example 2:

Someone owns a full region. He does the same as above:
he devided the region into several parcels: one parcel
is HUGE, but empty. Other small parcels were created
because that way he could set different music urls:
one for each dance floor!
Once you put the limits into effect, the empty parcels
suddenly reserve a HUGE ammount of resources "just in
case" the "owner" will need it. No ammount of user
feedback can change the limits such that those parcels
can still donate their resources to the dancefloor
parcels... and the whole region breaks down.

Example 3:

As many land tycoons do... full sims are divided into
many parcels, and each parcel is rented out to individuals.
Those individuals are usually close friends (or they
wouldn't have rented on that sim (or even voted in there).
So, they often hang out in one spot. Sometimes in the
house of one of them, more often on the common beach.
And then they have parties. Before, it didn't matter
where they went: they used maybe 30% of the server
resources at all times and things worked just fine.
Then Linden Lab comes with their limits... suddenly
S = R1 + R2 + R3 + R4
and
R1 = P11 + P12 + P13 + ... + A_pool
and NO amount of user feedback is going to change
that, because if you'd change that '=' to '<'
then abuse would still be possible and swapping
could still happen, it simply isn't possible to
allow '<'.
However, that means that the resources of each parcel
is suddenly a fraction... with their 8 parcels and
residents, they suddenly can only have parties
with only 1/8 of the previous resources... and
30% being way larger... that was the end of their
parties.

How to stop this major disaster?
==========================

In order to stop this from becoming major disaster
number X, the whole limiting system HAS to be changed:

There should be NO limits on memory usage whatsoever
UNTIL the server resources are almost used up.

If S == 1.0, and R1 == 0.8, then that is ok as long
as R2 == 0.0, R3 == 0.1 and R4 == 0.05.

Ok, if THEN R2 suddenly wants to use 0.2, then
R1 is the one that has to be limited, BUT NOT BEFORE.

In other words: the limits must be dynamic and ONLY
be set once there is need to set them.

I didn't give an example where memory from the avatar
pool is needed for parcels or the other way around,
but that is exactly the same.

What you should do is make a LLMemoryResource class
and derive everything that uses memory from that class.
This class would enforce memory usage limits.

All those instances together, their ACTUAL memory
usage, added up should be less than the memory on
the WHOLE server (if you REALLY want not support
Example 1, so be it; in which case it should add
up to the whole region).

Lets say the limit of the server is S, then the limit
for EVERY object is set to S.

When a new LLMemoryResource is created, or an exiting
one wants to increase it's memory usage, you check
that the total sum is still less than S first. Once
it becomes larger than S (to sum of everything) THEN
you run an algorithm that sets actual limits.
Whatever that algorithm is, the estate manager must
be able to configure it! Ie, he must be able to say:
that parcel gets 90% of the resources; or the
avatar pool gets 50% of the resources (or 10%).

This would then result in some scripts not being
able to run anymore. They should print "Out of Memory"
in the script debug console and stop running (and
then free their memory usage). If one tries to set
such a script to running again, it would check if
that is possible due to it's limit and either start
or give an error message and refuse to run.