[sldev] [LSL] [POSTFIX PROBLEMS] Object Email Receive from external domain problems

Chance Unknown chance at kalacia.com
Mon Jun 11 13:55:30 PDT 2007
Previous message: [sldev] [LSL] [POSTFIX PROBLEMS] Object Email Receive from external domain problems
Next message: [sldev] [LSL] [POSTFIX PROBLEMS] Object Email Receive from external domain problems
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
the part of the system thats the turd is not the postfix mail server. its
the locater service that identify the location of the sim a prim or object
is in. think of how functional your friends online presence information has
been in the past, and then apply the same general idea to location of
prims.... the underlying infrastructure used for position location should
have an enema.

otoh -- llHTTPRequest(). those reach out and provide their presence to the
website they are contacting; there is no layer that has to be used to locate
in world presence information to direct the messages because basically, SL
is polling.

use the robust solution. shun the thing that gives you the grief. and then
you too can dance on the swiss alps with julie andrews singing the -- i'd
like to give the world a coke --- song.

On 6/11/07, Erik Hill <zortiger at gmail.com> wrote:
>
> Couldnt you do some load balancing on  the XML-RPC, multiple servers, even
> round robin DNS could do better then a single server.
>
> SL really needs a realtime incoming communication method.
>
> My problem with useing HttpRequest is that its already being used in my
> system .. just below 1 call per second on large scale deployments. Around 36
> objects (max) making HttRequests to get data into the system per sim.
>
> That dosent leave much time for any other httprequests, let alone others
> that arnt from my system and that are in that sim and under the same user.
>
> The consept im useing is more on the lines of a thin client.  Considering
> LSL cant really becalled robust or fast. It would take me about 40 scripts
> and several months to recreate what can be done in  a hour in most other
> languages, let alone data storage.. I need to store several mb's of
> information, and have it acessable fast - something thats not vary easy to
> do with LSL.
>
> So im porting every thing possable out of the server, and haveing only the
> reactions in LSL.
>
> Im hoping to also deploy something like this in upwards of 50 sims. The
> larger the scale the more important it becomes to get rid of unnessary data
> calls, it becomes necessary load on my server, and the outbound connection
> of SL....
>
> Stable inbound connection would decrease the amount of load id have to put
> on my server, and decrease the httprequest calls that result in nothing more
> then saying 'Do i have a message?'
> When it should just simpily be able to drop a message to whatever object
> needs to get a message, saves bandwidth and resources, no extra sql querys
> to see if the message was gotten or if it was sent or if there is a message
> waiting.
>
>
>
>
>
>
>
>
>
> On 6/11/07, Kelly Washington <kelly at lindenlab.com> wrote:
> >
> >  We are currently (this minute, really) investigating emails from
> > outside to LSL being broken.
> >
> > Some more comments about communication between LSL and outside SL:
> >
> >  - All XML-RPC requests go through a central server.  That 'server' is
> > really an apache host running a C++ cgi.  This cgi module looks up object
> > presence info (what host it is on) based on the channel ID and routes the
> > message accordingly.  This server is consistently overloaded although I
> > believe we upgraded it, again, recently.  As a central point of failure it
> > will never scale as well as needed, and should eventually be deprecated.
> >
> >  - All email from outside goes through a single server that maps to
> > lsl.secondlife.com.  The emails are passed to a perl script that checks
> > some edge cases and looks up presence info (what host the object is on) in
> > backbone (the python service that hosts agent presence as well).  However,
> > objects only have presence in backbone if they have called llGetNextEmail at
> > least once; if there is no presence info for the object or the object is not
> > where we think it is then it is put in the database and retrieved the next
> > time the object calls llGetNextEmail.  However, like xml-rpc email from
> > outside SL to LSL goes through a central server that will never scale as
> > well as needed and will eventually need to be deprecated or changed.  By
> > changed I mean for example requiring region information in addition to
> > object ID in the email address used.  (Email from lsl->lsl cheats, is never
> > really an email and avoids the problems mentioned above except the presence
> > part.  If presence is broken then lsl->lsl email will fail unless the
> > objects are on the same sim)
> >
> > *NOTE: While I used the word 'deprecated' above we currently have no
> > solid plans to do so, I am not saying they are already deprecated and we
> > will work with and communicate with the community a LOT before we move ahead
> > with any plans related to that.  Additionally we expect more communication
> > alternatives to be in place before then.  We are still supporting these
> > methods and as I mentioned in the first line we are currently investigating
> > the current email issues.
> >
> >  - llHTTPRequest is the bomb.  Really.  It is lightweight and
> > distributed.  The entire action happens on the host the region with the
> > object is on.  There is no central server, no required presence
> > registration.  Lots and lots of requests can be done at the same time with
> > essentially no effect on the rest of the simulator process.   Yes there have
> > been bugs before but most of those are now resolved.  It is much better to
> > have an object ping a server every second with an llHTTPRequest than it is
> > use either XML-RPC or email->lsl.
> >
> >  - Future communication methods: Zero Linden has talked about LSL
> > communication methods at his office hours and has some pretty interesting
> > stuff on his wiki page about it:
> > http://wiki.secondlife.com/wiki/User:Zero_Linden/Office_Hours/Discussion#Incoming_HTTP
> >
> > No one, to my knowledge, is currently working on this.
> >
> >  - Kelly
> >
> > Chance Unknown wrote:
> >
> > email between objects is more robust and breaks with less frequency than
> > the xmlrpc interface. but, when your sim is constipated on the email, trying
> > to find someone to come fix it is a nightmare. and without support channels
> > or access to lindens. you can feel basically screwed. HTTPRequest() is the
> > way to go right now.
> >
> > On 6/11/07, Erik Hill <zortiger at gmail.com > wrote:
> > >
> > > This dosent sound vary good
> > >
> > >  XMLRPC is already vary screwed up.
> > >
> > > Ill be testing this within the next few days my self.
> > >
> > > I am currently working on an application that has tight integration
> > > between a website and SL, that normally means some sort of communication.
> > > While i plan to make use of email for a good minority of it, as its
> > > only simi time sensitve, XMLPRC is a must for time sensitive data, such as
> > > results when they interact with a web page, I need both up and working
> > > correctly in order for my plans to completely work.
> > >
> > > I know from my current venders that rely on XMLRPC that its failor
> > > rate is up in the 20-30% depending on the time of day.
> > > From what i can figure out single threaded applications tell horror
> > > stories about SL's XMLRPC functions, such as getting stuck infinity reading
> > > an open port that never gets data.
> > > >From what iv seen with the XMLRPC ill have to build a threaded
> > > application that opens up multiple attempts at the same time to get data in
> > > to SL, then shuts down after a thread receives data back from the object.
> > > Its the only way I can see to get data into SL at a realtime speed, and
> > > realtime is necessary in some cases.
> > >
> > > But yea, im next to getting into the email stuff for my project, so
> > > testing its delay is short list of things to do.
> > >
> > > On 6/11/07, Kamilion < kamilion at gmail.com> wrote:
> > > >
> > > > Holy crap. This is far worse than I expected.
> > > > I didn't want to use XMLRPC because I figured that a single point of
> > > > failure and lag of a couple minutes wasn't worth it.
> > > >
> > > > >From what I knew, postfix ran on every simulator; so it was more
> > > > distributed and hopefully more robust.
> > > >
> > > > Now it looks like something is SERIOUSLY wrong in email land.
> > > >
> > > > Last I knew, Michael, James, and Charity Linden were messing with
> > > > postfix back in april's blog post.
> > > >
> > > > Now -- What can we do about this? I'm glad I said something,
> > > > otherwise
> > > > this might have been a quiet annoyance.
> > > >
> > > > I know postfix can be blazingly fast, I used to use it on my own
> > > > domain before moving over to Google Apps for your Domain, So there's
> > > > obviously some kind of misconfiguration somewhere.
> > > >
> > > > If you feel like sharing; I'd like a copy of that PHP and LSL you
> > > > use;
> > > > because this is EXACTLY the sort of setup I'm trying to achieve (PHP
> > > > Mail to LSL).
> > > >
> > > > So far, I've had to bash at PHP to even get it to talk to Google's
> > > > AfyD server.
> > > > Google uses SSL/TLS *ONLY* for SMTP auth, so php's mail() function
> > > > doesn't work so well.
> > > >
> > > > After doing exhaustive research, I came up with the following
> > > > options:
> > > > http://www.bytewize.com/linux/BetterSMTPhelp.php
> > > > BetterSMTP, a phpbb2.x mod to use the SquirrelMail SMTP classes.
> > > > Looked to be broken and forum posts about it were awash with PLZ
> > > > HELP.
> > > > No good.
> > > > http://www.vulgarisoip.com/?p=17
> > > > PHPGmailer, a Cut & Paste job on PHPMailer to kitbash it into using
> > > > TLSSTART. Also no good, PHPMailer isn't maintained anymore and this
> > > > kitbash comes up with SSL errors on connection close.
> > > >
> > > > The winner:
> > > > Still poking along, I ran across this bug report for Wordpress:
> > > > http://trac.wordpress.org/ticket/4326    #4326 (WP can't use Google
> > > > Apps For Your Domain email accounts.)
> > > > Which linked me here:
> > > > http://www.shiftthis.net/wordpress-swift-smtp-plugin/
> > > > Which linked me here:
> > > > http://www.swiftmailer.org/
> > > > "Swift is a free feature-rich PHP Mailer -- Swift is a fully OOP
> > > > library for sending e-mails from PHP websites and applications. It
> > > > does not rely on PHP's native mail() function which is known for
> > > > using
> > > > high server resources when sending multiple emails. Instead, Swift
> > > > communicates directly with an SMTP server or a MTA binary to send
> > > > mail
> > > > quickly and efficiently."
> > > >
> > > > Score!
> > > > One of the features it lists: "SSL & TLS Support - for Gmail
> > > > servers"
> > > >
> > > > So, here's how I hacked together a nice little test script:
> > > >
> > > > <?php
> > > > require_once "swiftmailer/Swift.php";
> > > > require_once "swiftmailer/Swift/Connection/SMTP.php";
> > > >
> > > > echo "Starting...\n<br>\n<br>\n";
> > > >
> > > > // Create a SMTP connection handler (Google Apps for your Domain)
> > > > $smtp = new Swift_Connection_SMTP(" smtp.googlemail.com",
> > > > Swift_Connection_SMTP::PORT_SECURE, Swift_Connection_SMTP::ENC_TLS);
> > > >
> > > > $smtp->setUsername("phpsmtp at sllabs.com");
> > > > $smtp->setpassword("smtpmail");
> > > >
> > > > // Create the Swift object interface
> > > > $swift =& new Swift($smtp);
> > > >
> > > > $to = "513d65af-98e3-f2ef-db09-9a112eca5d28 at lsl.secondlife.com ";
> > > > $subject = "Testing my subject";
> > > > $message = "A very long\n and drawn out\n message.\n\nRar!";
> > > >
> > > > //Create the message
> > > > $swiftmessage =& new Swift_Message($subject, $message);
> > > >
> > > > //Now check if Swift actually sends it
> > > > if ($swift->send($swiftmessage, $to, " phpsmtp at sllabs.com"))
> > > > { echo "Sent"; } else { echo "Failed"; }
> > > >
> > > > ?>
> > > >
> > > >
> > > > Here's a sample result:
> > > >
> > > > Delivered-To: kamilion at gmail.com
> > > > Received: by 10.100.165.16 with SMTP id n16cs242953ane;
> > > >         Sun, 10 Jun 2007 20:14:00 -0700 (PDT)
> > > > Received: by 10.64.204.19 with SMTP id b19mr8395228qbg.1181531640038
> > > > ;
> > > >         Sun, 10 Jun 2007 20:14:00 -0700 (PDT)
> > > > Return-Path: <phpsmtp at sllabs.com >
> > > > Received: from qb-out-0506.google.com (qb-out-0506.google.com [72.14.204.225
> > > > ])
> > > >         by mx.google.com with ESMTP id
> > > > q14si5481907qbq.2007.06.10.20.13.59;
> > > >         Sun, 10 Jun 2007 20:14:00 -0700 (PDT)
> > > > Received-SPF: neutral (google.com : 72.14.204.225 is neither
> > > > permitted
> > > > nor denied by best guess record for domain of phpsmtp at sllabs.com)
> > > > Received: by qb-out-0506.google.com with SMTP id q12so1717954qba
> > > >         for <Kamilion at gmail.com>; Sun, 10 Jun 2007 20:13:59 -0700
> > > > (PDT)
> > > > Received: by 10.115.110.6 with SMTP id n6mr5060449wam.1181531638487;
> > > >         Sun, 10 Jun 2007 20:13:58 -0700 (PDT)
> > > > Return-Path: <phpsmtp at sllabs.com>
> > > > Received: from ?216.218.196.73? ( [ 216.218.196.73 ])
> > > >         by mx.google.com with ESMTP id
> > > > n38sm6862970wag.2007.06.10.20.13.57
> > > >         (version=SSLv3 cipher=OTHER);
> > > >         Sun, 10 Jun 2007 20:13:58 -0700 (PDT)
> > > > Return-Path: < phpsmtp at sllabs.com>
> > > > To: Kamilion at gmail.com
> > > > From: phpsmtp at sllabs.com
> > > > Reply-To: phpsmtp at sllabs.com
> > > > Subject: My subject
> > > > Date: Sun, 10 Jun 2007 20:15:09 -0700
> > > > X-LibVersion: 3.2.6
> > > > MIME-Version: 1.0
> > > > Content-Type: text/plain; charset=iso-8859-1; format=flowed
> > > > Content-Transfer-Encoding: 8bit
> > > > Message-ID: <20070611031509.22850.1036293226.swift at vendor.sllabs.com>
> > > >
> > > > My body
> > > >
> > > >
> > > >
> > > > So, anyway.
> > > > External Domain Email's obviously broked somehow.
> > > > Included some data that could be useful for others.
> > > >
> > > > Harold, I've got land in multiple mainland and private sims. If you
> > > > feel like it, I can set up some additional prims to take multiple
> > > > metrics, and see if there's a correlation.
> > > >
> > > > sllabs.com is run from a Gentoo VPS hosted by atarack.com in
> > > > Hurricane
> > > > Electric's Fremont-II datacenter, roughly 2ms from the main LL
> > > > datacenter in SF.
> > > >
> > > > If you should decide to decline sharing your code, heck, I'll bash
> > > > up
> > > > a similar PHP&LSL script and share it in this thread, just because I
> > > > wanna get to the bottom of this; and hopefully share as much data as
> > > > I
> > > > can with the lindens so they can fix it properly.
> > > >
> > > > -- Graham Cantin (Kamilion Schnook)
> > > >
> > > > (sorry for the longwinded mails, but this stuff's archived, so it
> > > > may
> > > > help out others in the future who are browsing the SLDev archives or
> > > >
> > > > the Google Coop search :)
> > > >
> > > >
> > > >
> > > > On 6/10/07, Harold Brown <labrat.hb at gmail.com> wrote:
> > > > > Welcome to the hell of trying to use external -> internal E-Mail
> > > > with SL.
> > > > >
> > > > > For a nice history of the reliability of external -> internal
> > > > E-Mail from
> > > > > May 2 -> current date,  you can view my test log at this URL:
> > > > > http://www.rpgstats.com/SL/maillog.txt
> > > > >
> > > > > If you would like to review the history from February8 -> May2 it
> > > > is
> > > > > available here:
> > > > > http://www.rpgstats.com/SL/maillog1.txt
> > > > >
> > > > > As of the time I'm replying there seems to be about 4 hour delay
> > > > in e-mail
> > > > > processing for SL...  There have been delays up to around 24
> > > > hours.
> > > > >
> > > > > For those interested in how the test works.
> > > > >
> > > > > I have an LSL script running on my sim that once every 30 minutes
> > > > requests a
> > > > > PHP script on my webserver to send an e-mail.  The PHP script
> > > > generates a
> > > > > random string for the subject and logs the time that it sends to
> > > > the
> > > > > maillog.txt file.  The LSL script checks every 5 seconds for a new
> > > > E-Mail,
> > > > > and upon receiving one it tells the PHP script on the server that
> > > > it
> > > > > recieved an E-Mail and what the subject was.  The PHP script then
> > > > logs this
> > > > > information as well.
> > > > >
> > > > > On 6/10/07, Kamilion <kamilion at gmail.com> wrote:
> > > > > >
> > > > > > I'm having some problems with objects and receiving email from
> > > > outside
> > > > > > of the secondlife.com domain -- They seem to be failing without
> > > > an
> > > > > > error or bounce message, and the object never appears to receive
> > > > the
> > > > > > email. I was hoping someone could look into this or point
> > > > someone who
> > > > > > can at it. I submitted a ticket in the new support system,
> > > > TicketID
> > > > > > 4051-4201708.
> > > > > >
> > > > > > Synopsis:
> > > > > > Object Sending Email (Working)
> > > > > > Object Receiving Email from another Object (Working)
> > > > > > Object Receiving Email from Google Apps for your Domain hosted
> > > > system:
> > > > > > ( sllabs.com) (NOT WORKING)
> > > > > > Object Receiving Email from Google Mail ( gmail.com) (NOT
> > > > WORKING)
> > > > _______________________________________________
> > > > Click here to unsubscribe or manage your list subscription:
> > > > /index.html
> > > >
> > >
> > >
> > > _______________________________________________
> > > Click here to unsubscribe or manage your list subscription:
> > > /index.html
> > >
> > >
> > ------------------------------
> >
> > _______________________________________________
> > Click here to unsubscribe or manage your list subscription:
> >
> >
> > /index.html
> >
> >
> >
>
> _______________________________________________
> Click here to unsubscribe or manage your list subscription:
> /index.html
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.secondlife.com/pipermail/sldev/attachments/20070611/3bbaeb92/attachment-0001.htm
Previous message: [sldev] [LSL] [POSTFIX PROBLEMS] Object Email Receive from external domain problems
Next message: [sldev] [LSL] [POSTFIX PROBLEMS] Object Email Receive from external domain problems
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the SLDev mailing list