QtMoko v48 neofreerunner

[Neil Jerram]

Prompted by David Matthews' reports, I've improved my understanding of
SMTP authentication mechanisms.  Little of this is specific to QtMoko or
even smartphones, but I think it's worth covering anyway in the context
of this thread.

When you want to send email via an SMTP server that isn't in your own
immediate network, the client needs to authenticate itself to the
server.  Otherwise that server would be an "open relay", which is a big
no-no.  So the client (e.g. on your GTA0x) has to send something
involving your ID and password to the server - and obviously that raises
the questions of (1) whether the server is really the right server, and
(2) whether something else can sniff the communication and work out your
ID and password.

The available SMTP auth mechanisms in QtMoko are "LOGIN" and "PLAIN",
and both of those are basically plaintext - i.e. anyone sniffing the
communication can see your ID and password.  Therefore you really need
to be using encryption - either SSL or TLS - as well, for sending email
from a QtMoko phone.  (There is a non-plaintext SMTP auth mechanism,
"CRAM-MD5", but it appears that QtMoko does't support that.)

TLS and SSL are largely the same.  In this context (SMTP), the
difference is that SSL encrypts the TCP connection before _any_ SMTP
protocol is exchanged (and requires a distinct port number from
unencrypted SMTP), whereas a "TLS" connection starts off unencrypted and
becomes encrypted once both the client and the server have agreed that
(and so can operate within the normal SMTP port 25).  Crucially, in the
TLS case, both the client and the server can ensure that they don't
exchange any authentication information until the connection has become
encrypted.

There's still question (1), whether the server you're connecting to is
really the server that you trust to know your ID/password and to see
your outgoing email.  SSL and TLS both address this by the server having
a certificate that uniquely identifies itself.  That certificate is
passed to the client, as part of the process of the connection becoming
encrypted, and the client must decide if it trusts that certificate.

In QtMoko (and I think in general) this is done by the client having a
pre-installed set of certificates that it trusts.  Then, when the client
sees the SMTP server's certificate, it must either exactly match one of
the pre-installed ones, or have been signed by one of the pre-installed
ones.  Some certificate-handling applications, like Firefox, also allow
the user to inspect an untrusted certificate and make it trusted, but
QtMoko doesn't do that.

Therefore...

- If you're using a public SMTP server, that server's certificate should
  be signed by a recognised "certificate authority" (CA), and that CA's
  certificate should be pre-installed in your QtMoko.  "apt-get install
  ca-certificates" does this.

- If you're using your own SMTP server, you can:

  (a) create your own self-signed CA certificate

  (b) create a certificate for your SMTP server, signed by your CA
  certificate

  (c) configure your SMTP server to support TLS, using the newly created
  certificate

  (d) install your own CA certificate in QtMoko by copying it to
  /usr/local/share/ca-certificates/<something>.crt, running
  update-ca-certificates, and restarting QtMoko.

  In other words, you're telling your QtMoko to trust your own CA.
  There are good instructions for (a), (b) and (c) at
  http://www.postfix.org/TLS_README.html.

There are probably other variants on the "own SMTP server" procedure
described here, but this is what I've just successfully tested for my
own SMTP setup.

Now, with all that as background...

"dmatthews.org" <mail at dmatthews.org> writes:

> Here's some more smtp debugging - I'm now using the mail.inbox.lv server which supports unencrypted and TLS access
>
> Trying port 587 with TLS:-
>
> Sep 24 20:01:37 neo Qtopia: 250-STARTTLS^M
> Sep 24 20:01:37 neo Qtopia: 250-AUTH LOGIN PLAIN^M
> Sep 24 20:01:37 neo Qtopia: 250-ENHANCEDSTATUSCODES^M
> Sep 24 20:01:37 neo Qtopia: 250-8BITMIME^M
> Sep 24 20:01:37 neo Qtopia: 250 DSN" 
> Sep 24 20:01:37 neo Qtopia: SMTP :  StartTLS: sent: STARTTLS 
> Sep 24 20:01:37 neo Qtopia: SMTP :  response: "220 2.0.0 Ready to start TLS" 
> Sep 24 20:01:38 neo Qtopia: Encrypted connect warnings: "'The root certificate of the certificate chain is self-signed, and untrusted'" 
> Sep 24 20:01:38 neo Qtopia: SMTP :  Closed connection 
> Sep 24 20:01:38 neo Qtopia: socketError: 13 : "The root certificate of the certificate chain is self-signed, and untrusted" 

I think this means that the mail.inbox.lv server provided a self-signed
certificate (or a chain of certificates rooted in a self-signed
certificate), and that QtMoko doesn't have that certificate installed.

If I run "swaks -s mail.inbox.lv -t nospam at gmail.com -tls -a LOGIN"
from a laptop, with Wireshark running at the same time, I see that
mail.inbox.lv provides a chain of 3 certificates, of which the root
certificate looks similar to /etc/ssl/certs/AddTrust_External_Root.pem
on my QtMoko.  Therefore I'd expect "apt-get install ca-certificates" to
fix any TLS certificate issue with mail.inbox.lv.

To double-check that I configured a mail.inbox.lv account on my QtMoko
and tried sending an email using it.  The log confirms that the TLS
connection was successful; my email was rejected after that because I
didn't give any ID/password:

Oct  7 11:54:29 neo Qtopia: SMTP :  newConnection 
Oct  7 11:54:29 neo Qtopia: SMTP :  Open SMTP connection 
Oct  7 11:54:29 neo Qtopia: Messaging :  Opening connection -  "mail.inbox.lv" : 25  TLS 
Oct  7 11:54:30 neo Qtopia: Messaging :  SMTP : connection established 
Oct  7 11:54:32 neo Qtopia: SMTP :  response: "220 mail.inbox.lv ESMTP relay for users of Inbox.Mail" 
Oct  7 11:54:32 neo Qtopia: SMTP :  Init: sent: EHLO qtopia-messageserver 
Oct  7 11:54:33 neo Qtopia: SMTP :  response: "250-mail.inbox.lv^M
Oct  7 11:54:33 neo Qtopia: 250-PIPELINING^M
Oct  7 11:54:33 neo Qtopia: 250-SIZE 59900000^M
Oct  7 11:54:33 neo Qtopia: 250-VRFY^M
Oct  7 11:54:33 neo Qtopia: 250-ETRN^M
Oct  7 11:54:33 neo Qtopia: 250-STARTTLS^M
Oct  7 11:54:33 neo Qtopia: 250-AUTH LOGIN PLAIN^M
Oct  7 11:54:33 neo Qtopia: 250-ENHANCEDSTATUSCODES^M
Oct  7 11:54:33 neo Qtopia: 250-8BITMIME^M
Oct  7 11:54:33 neo Qtopia: 250 DSN" 
Oct  7 11:54:33 neo Qtopia: SMTP :  StartTLS: sent: STARTTLS 
Oct  7 11:54:33 neo Qtopia: SMTP :  response: "220 2.0.0 Ready to start TLS" 
Oct  7 11:54:35 neo Qtopia: Messaging :  SMTP : Secure connection established 
Oct  7 11:54:35 neo Qtopia: SMTP :  Connected 
Oct  7 11:54:36 neo Qtopia: SMTP :  response: "250-mail.inbox.lv^M
Oct  7 11:54:36 neo Qtopia: 250-PIPELINING^M
Oct  7 11:54:36 neo Qtopia: 250-SIZE 59900000^M
Oct  7 11:54:36 neo Qtopia: 250-VRFY^M
Oct  7 11:54:36 neo Qtopia: 250-ETRN^M
Oct  7 11:54:36 neo Qtopia: 250-AUTH LOGIN PLAIN^M
Oct  7 11:54:36 neo Qtopia: 250-ENHANCEDSTATUSCODES^M
Oct  7 11:54:36 neo Qtopia: 250-8BITMIME^M
Oct  7 11:54:36 neo Qtopia: 250 DSN" 
Oct  7 11:54:36 neo Qtopia: SMTP :  From: sent: MAIL FROM:  "<neil at inbox.lv>" 
Oct  7 11:54:36 neo Qtopia: SMTP :  response: "250 2.1.0 Ok" 
Oct  7 11:54:36 neo Qtopia: SMTP :  Recv: sent: RCPT TO: < "nospam at googlemail.com" > 
Oct  7 11:54:37 neo Qtopia: SMTP :  response: "554 5.7.1 <unknown[31.100.94.40]>: Client host rejected: In order to send mail, please, login first" 
Oct  7 11:54:37 neo Qtopia: SMTP :  Closed connection 
Oct  7 11:54:37 neo Qtopia: Messaging :  transferFailure: "Send Failure" - "inbox - Error sending Email: Unexpected response from server :
Oct  7 11:54:37 neo Qtopia: 554 5.7.1 <unknown[31.100.94.40]>: Client host rejected: In order to send mail, please, login first^M
Oct  7 11:54:37 neo Qtopia: " 

> I get the same message whether I set authentication to login or plain
> (I'm uncertain which would be correct), but also to none and imap
> (which I'd expect to be wrong.

That's expected.  Authentication happens _after_ TLS setup, so if TLS is
failing, the authentication method will make no difference.

> This makes me think I was too hasty to
> blame my own mail server's self-signed certificate.

You should be able to make that case work by installing your mail
server's certificate in QtMoko, as detailed above.

> Changing to port 25 which is also supported by the server gives the same error
>
> Sep 24 20:26:18 neo Qtopia: SMTP :  Init: sent: EHLO qtopia-messageserver 
> Sep 24 20:26:18 neo Qtopia: SMTP :  response: "250-mail.inbox.lv^M
> Sep 24 20:26:18 neo Qtopia: 250-PIPELINING^M
> Sep 24 20:26:18 neo Qtopia: 250-SIZE 59900000^M
> Sep 24 20:26:18 neo Qtopia: 250-VRFY^M
> Sep 24 20:26:18 neo Qtopia: 250-ETRN^M
> Sep 24 20:26:18 neo Qtopia: 250-STARTTLS^M
> Sep 24 20:26:18 neo Qtopia: 250-AUTH LOGIN PLAIN^M
> Sep 24 20:26:18 neo Qtopia: 250-ENHANCEDSTATUSCODES^M
> Sep 24 20:26:18 neo Qtopia: 250-8BITMIME^M
> Sep 24 20:26:18 neo Qtopia: 250 DSN" 
> Sep 24 20:26:18 neo Qtopia: SMTP :  Auth: sent: AUTH LOGIN  
> Sep 24 20:26:18 neo Qtopia: SMTP :  response: "334 VXNlcm5hbWU6" 
> Sep 24 20:26:18 neo Qtopia: SMTP :  AuthUser: sent: "ZG1hdHRoZXdzQGluYm94Lmx2" 
> Sep 24 20:26:18 neo Qtopia: SMTP :  response: "334 UGFzc3dvcmQ6" 
> Sep 24 20:26:18 neo Qtopia: SMTP :  AuthPass: sent: <Base64 password hidden> 
> Sep 24 20:26:22 neo Qtopia: SMTP :  response: "535 5.7.8 Error: authentication failed: authentication failure" 
> Sep 24 20:26:22 neo Qtopia: SMTP :  Closed connection 

Most likely your ID and password are invalid, but I guess mail.inbox.lv
might also say "authentication failure" for other reasons, e.g.

- it's configured not to accept any authentication on an insecure
  channel, i.e. without TLS being set up first

- could there be specific client IP addresses associated with your
  account, and the problem is that your phone doesn't have one of those
  addresses?

> With no encryption on port 25:-

Actually there was no encryption in the previous example either.  The
only difference in the following is "PLAIN" authentication instead of
"LOGIN".

> Sep 24 20:28:50 neo Qtopia: SMTP :  Init: sent: EHLO qtopia-messageserver 
> Sep 24 20:28:50 neo Qtopia: SMTP :  response: "250-mail.inbox.lv^M
> Sep 24 20:28:50 neo Qtopia: 250-PIPELINING^M
> Sep 24 20:28:50 neo Qtopia: 250-SIZE 59900000^M
> Sep 24 20:28:50 neo Qtopia: 250-VRFY^M
> Sep 24 20:28:50 neo Qtopia: 250-ETRN^M
> Sep 24 20:28:50 neo Qtopia: 250-STARTTLS^M
> Sep 24 20:28:50 neo Qtopia: 250-AUTH LOGIN PLAIN^M
> Sep 24 20:28:50 neo Qtopia: 250-ENHANCEDSTATUSCODES^M
> Sep 24 20:28:50 neo Qtopia: 250-8BITMIME^M
> Sep 24 20:28:50 neo Qtopia: 250 DSN" 
> Sep 24 20:28:50 neo Qtopia: SMTP :  Auth: sent: AUTH PLAIN  <Base64 username/password hidden> 
> Sep 24 20:28:54 neo Qtopia: SMTP :  response: "535 5.7.8 Error: authentication failed: authentication failure" 
> Sep 24 20:28:54 neo Qtopia: SMTP :  Closed connection 

Same interpretation as for the previous case.

> Similar result if I try login authentication except the difference
> does get logged. I'm *definitely* configuring the gui with the correct
> username and password and that shows up correctly to the
> qtopiamail.conf. Why do the logs report STARTTLS when I've not
> configured that?

"250-STARTTLS" is the server saying that it _supports_ TLS.  It doesn't
mean that it is actually being used.

If you're really sure about your ID and password,

- can you send email via mail.inbox.lv from other clients?

- perhaps you need to contact their support to ask why your auth is
  being rejected.

Regards,
        Neil