Duplicate message troubleshooting

Santiago Crespo screspo at s21sec.com
Thu Aug 16 20:25:42 CEST 2007

Hi Marco,

I think that the best way to solve this is opening a bug in the bugzilla.openmoko.org system, select "Site Infraestructure" and then "Lists":


Nobody has reported this yet:



El mar, 24-07-2007 a las 13:14 -0700, Marco Barreno escribió:
> Messages to the list are being duplicated again.  Some of the messages
> are from Gmail, but not all (e.g. some are from kostyrka.org and
> cnlohr.com).  I can try to help debug the problem from the Gmail end
> if an admin on the list machine can send me verbose SMTP logs; please
> see below for details.  Harald or Sean, can one of you send detailed
> logs?  Anyone else?
> Some Gmail message IDs with duplicated messages today:
>   e4b4377c0707240911s5d79faa4s48dea3b00a4418c4 at mail.gmail.com
>   eef811f30707240955i4348c71ai16f337968c714e67 at mail.gmail.com
>   562ec500707241157r4205bba7p919a99eb489552c6 at mail.gmail.com
> Thanks,
> Marco
> ----- Forwarded message from Marco Barreno <barreno at CS.Berkeley.EDU> -----
> From: Marco Barreno <barreno at CS.Berkeley.EDU>
> Date: Fri, 13 Jul 2007 14:36:57 -0700
> To: "Wolfgang S. Rupprecht" <wolfgang.rupprecht+gnus200707 at gmail.com>
> Cc: community at lists.openmoko.org
> Subject: Re: gmail.com problems and this list
> Hi everyone,
> Short version:
> It seems the duplicated messages are not just a Gmail problem: mails
> are also being duplicated from starband.net and possibly others as
> well, so it seems the problem is likely something on the openmoko side
> that only happens to cause duplicates with some sending domains and
> not others.  I'ts also not consistent; some messages from Gmail and
> starband.net are duplicated while others aren't.
> I've checked with the Gmail team, and the duplicated messages from
> Gmail are happening because Gmail doesn't receive the SMTP OK from the
> openmoko list server, so it retransmits because it thinks the messages
> don't get delivered.  To debug further, an admin on the list machine
> needs to enable verbose SMTP logging if it's not already enabled and
> provide information about if/when the OK is sent.  Here are some
> example message IDs for which the logs would be useful:
>   2f3aa2770707070010k7407439axfaa269de33395ea2 at mail.gmail.com
>   c914135d0707121458s509d1ee1v3a8fe77101bbcaa2 at mail.gmail.com
>   b829084f0707121328q5e4d0f28me015de4502cffb69 at mail.gmail.com
> If verbose logging isn't already enabled, however, enabling it and
> then sending logs for any future duplicated messages would work too.
> Detailed version:
> On Mon, Jul 09, 2007 at 11:14:03PM -0700, thus spake Wolfgang S. Rupprecht:
> > At least one of the opemoko machines at opens an smtp
> > connection back to the sending domain to verify the sender address of
> > any incoming message.  The smtp-back machine has messed up DNS.  The
> > claimed rDNS for that IP is "openmoko.org" but the forward DNS check
> > for that "openmoko.org" doesn't list as a valid
> > address.  If the sending machine is checking for spammers claiming a
> > bunk DNS name will reject's SMTP verify.  The opemmoko
> > machine will then interpret that failed smtp verify attempt as the
> > verified address not existing and will decline the initial incoming
> > transfer.
> Even ignoring the messed-up DNS for the moment, doing an SMTP callback
> is not a good way to check existence of an email account.  For one
> thing, whenever a spammer spoofs a From address in an innocent domain,
> that domain can get bombarded with callbacks.  For another thing,
> spammers sometimes use RCPT commands themselves to check existence of
> email addresses, so ISPs are likely to consider many such requests to
> be abusive behavior.  In the case of a large ISP (such as Gmail), if a
> lot of email is sent to a destination that does callbacks, the
> frequency of callbacks can trigger abuse prevention measures.  See
> http://en.wikipedia.org/wiki/Callback_verification for more drawbacks.
> (In particular, note: "If a server receives a lot of spam, it will do
> a lot of callbacks and if those addresses are invalid, the server will
> look very similar to a spammer who is doing a dictionary attack to
> harvest addresses. This in turn might get the server blacklisted
> elsewhere.")
> For these reasons, Gmail strongly recommends avoiding callbacks and
> instead checking SPF records (Sender Policy Framework, provided via
> DNS) to verify that the mail is indeed coming from Google (i.e. the
> SMTP connection is coming from an IP authorized to send email on
> behalf of Google).
> However, in this case it does not seem that the problem was caused by
> the callbacks.  The openmoko server accepted the mail (which it
> wouldn't do if the callback failed to verify the sender) and the DATA
> command was sent, but Gmail never received the next SMTP OK
> acknowledgment before timing out.  So one of four things was
> happening:
> 1) openmoko never sent the OK -- problem on openmoko end
> 2) openmoko waited too long before sending the OK and SMTP timed out
>     -- seems to indicate problem on the openmoko end
> 3) the OK was lost in transit -- network difficulties, neither side
>     at fault
> 4) the OK was received by Gmail -- problem at the Gmail end
> Since at least one other domain is showing the same behavior, it seems
> 1 or 2 is most likely, but we can't say for sure without more
> information.
> One interesting fact is that the openmoko server started sending mail
> back to Gmail recipients (after list expansion) before Gmail received
> the SMTP OK.  Perhaps this means the list server waits until after
> list expansion to send the OK?  That could cause it to time out pretty
> easily.
> It would help diagnosis if someone on the list machine could send the
> verbose SMTP logs (if they were recorded).  The logs posted by Harald
> Welte don't give enough information: what we really need to see is
> whether (and if so, when) the SMTP OK was sent for any of the
> failed/duplicated messages.  Here are three example message IDs:
>   2f3aa2770707070010k7407439axfaa269de33395ea2 at mail.gmail.com
>   c914135d0707121458s509d1ee1v3a8fe77101bbcaa2 at mail.gmail.com
>   b829084f0707121328q5e4d0f28me015de4502cffb69 at mail.gmail.com
> If the logging needs to have more verbosity enabled, however, the logs
> for any duplicated message after such enabling would work just as
> well.
> By the way, the Received header chains posted by David Ford don't
> necessarily show that the problem is within Gmail; there could be any
> number of legitimate reasons why the resending isn't originating at
> the last hop out of the Google network.  Yes, Gmail is sending the
> mail multiple times, but that's because it thinks the openmoko server
> did not receive the message on earlier attempts.
> I hope someone with admin access to the list machine can help debug
> this problem so we can get it fixed, wherever the bug is located.
> Thanks,
> Marco
> _______________________________________________
> OpenMoko community mailing list
> community at lists.openmoko.org
> http://lists.openmoko.org/mailman/listinfo/community
> ----- End forwarded message -----
> _______________________________________________
> OpenMoko community mailing list
> community at lists.openmoko.org
> http://lists.openmoko.org/mailman/listinfo/community
Santiago Crespo
SOC - S21Sec
600 409 281 (ext. 262)

La seguridad digital del futuro, Hoy.

La información contenida en este mail, así como los archivos adjuntos,
es CONFIDENCIAL. Grupo S21sec Gestión, S.A. garantiza la adopción de las
medidas necesarias para asegurar el tratamiento confidencial de los
datos de carácter personal. En el caso de que el destinatario del correo
no sea usted, le rogamos envíe una notificación al remitente y lo
destruya de forma inmediata.

La lectura y/o manipulación de esta información en la situación señalada
anteriormente será considerada ilegal, permitiendo a la empresa
remitente realizar acciones legales de diferente envergadura.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Esta parte del mensaje =?ISO-8859-1?Q?est=E1?= firmada
Url : http://lists.openmoko.org/pipermail/community/attachments/20070816/f95786ee/attachment.pgp 

More information about the community mailing list