Runbox privacy rating

Summary

Runbox scores 49 out of 100 (grade D) on the email providers criteria. It meets 7 of 18 criteria: no trackers or telemetry, no ads or data sales, transparency report, open protocols, custom domains, IMAP support and POP3 support. It partly meets open source, end-to-end encryption and SMTP submission. It does not meet independent audit, tells users about requests, security headers, encrypted mailbox storage, sign up without personal data, mail transport security, Sender Rewriting Scheme and ARC sealing. It is based in Norway: Nine Eyes member; EEA member (GDPR). Automated tests: Mozilla HTTP Observatory grade D-.

Score 49 out of 100. How scoring works

Criteria

• ◐Partial

  Open source ×3Weight 3 of 3

  Is all the source code needed to run the product public?

  The Runbox 7 webmail app is open source under GPL-3.0. The server components are not.

  github.com

• ✔Yes

  No trackers or telemetry ×3Weight 3 of 3

  Are the website and apps free of third-party trackers, with any analytics anonymous and any telemetry off by default?

  The privacy policy states that no third-party tracking, statistics or web beacons are used.

  runbox.com

• ✔Yes

  No ads or data sales ×2Weight 2 of 3

  Is the product funded without advertising, ad targeting or selling user data?

  Funded by paid plans. No ads, and user data is not sold.

  runbox.com

• ✖No

  Independent audit ×2Weight 2 of 3

  Has an independent security or privacy audit been published within the last three years?

  No independent audit is published.

• ✔Yes

  Transparency report ×2Weight 2 of 3

  Does the provider regularly publish how many government and legal requests it receives and how it responds?

  Publishes yearly counts of disclosure requests received, complied with and rejected.

  runbox.com

• ✖No

  Tells users about requests ×1Weight 1 of 3

  Does the provider promise to tell users about requests for their data, unless a court forbids it?

  No published policy on notifying users about data requests.

• ◐Partial

  End-to-end encryption ×3Weight 3 of 3

  Can mail be end-to-end encrypted so that the provider cannot read message contents?

  PGP and S/MIME work in desktop clients or in the webmail with browser extensions such as Mailvelope. Not built in.

  help.runbox.com

• ✖No

  Encrypted mailbox storage ×3Weight 3 of 3

  Is stored mail encrypted with a key the provider does not hold?

  Encryption of stored mail is not documented.

• ✔Yes

  Open protocols ×2Weight 2 of 3

  Can any standard app connect over IMAP, POP3, SMTP, CalDAV or CardDAV, without extra software?

  IMAP, POP3 and SMTP work with any client.

  help.runbox.com

• ✔Yes

  Custom domains ×1Weight 1 of 3

  Can mail be sent and received with your own domain?

  Every plan includes at least one custom domain.

  runbox.com

• ✖No

  Sign up without personal data ×2Weight 2 of 3

  Can an account be created without a phone number or another email address?

  Registration asks for a name, country and an alternative email address.

  runbox.com

• ✖No

  Sender Rewriting Scheme ×1Weight 1 of 3

  Is the envelope sender rewritten with SRS when mail is forwarded, so forwarded mail keeps passing SPF?

  No published documentation on SRS for forwarded mail.

• ✖No

  ARC sealing ×1Weight 1 of 3

  Does the provider validate and add ARC (RFC 8617) signatures, so authentication results survive forwarding?

  No published documentation on ARC signing or validation.

Automated tests

• …Not tested yet

  TLS configuration ×2Weight 2 of 3

  Does the website pass the Qualys SSL Labs test with a grade of A or better?

  Could not test: No endpoint could be graded

• ✖No

  Security headers ×1Weight 1 of 3

  Does the website pass the Mozilla HTTP Observatory test with a grade of A or better?

  Grade D- (25/100+)

  developer.mozilla.org

• …Not tested yet

  Modern web standards ×1Weight 1 of 3

  Does the website score 90% or higher on the Internet.nl website test?

  Not tested yet.

• …Not tested yet

  Email security standards ×2Weight 2 of 3

  Does the mail domain score 90% or higher on the Internet.nl email test?

  Not tested yet.

• ✔Yes

  IMAP support ×2Weight 2 of 3

  Does the IMAP server accept connections over implicit TLS on port 993 and advertise IMAP4rev1 or IMAP4rev2 with IDLE push?

  mail.runbox.com:993 (implicit TLS). IMAP4rev1 advertised with IDLE.

• ✔Yes

  POP3 support ×1Weight 1 of 3

  Does the POP3 server accept connections over implicit TLS on port 995 and answer CAPA with UIDL?

  mail.runbox.com:995 (implicit TLS). CAPA: TOP, USER, UIDL, SASL.

• ◐Partial

  SMTP submission ×2Weight 2 of 3

  Does mail submission work over implicit TLS on port 465 with SMTPUTF8, 8BITMIME, PIPELINING and AUTH?

  mail.runbox.com:465 (implicit TLS). Missing: SMTPUTF8.

• ✖No

  Mail transport security ×3Weight 3 of 3

  Does the mail domain enforce encrypted, authenticated delivery with MTA-STS, DANE, DNSSEC, TLS-RPT and an enforced DMARC policy?

  Passes: SPF, DMARC quarantine, TLS-RPT, DNSSEC. Missing: MTA-STS testing, DANE none.

Email standards

• ✔Yes: Mail serversMXmx.runbox.com
• ✔Yes: RFC 7208 sender policySPFpublished
• ✔Yes: RFC 7489. Quarantine or reject counts as enforcedDMARCp=quarantine
• ✖No: RFC 8461 strict transport securityMTA-STStesting
• ✔Yes: RFC 8460 TLS failure reportsTLS-RPTpublished
• ✔Yes: RFC 4033 signed DNSDNSSECsigned and validated
• ✖No: RFC 7672 TLSA records on MX hostsDANEnone
• ✖No: Brand logo record (not scored)BIMInone
• ✖No: RFC 6186 and RFC 8314 service records (not scored)SRVnone

Capabilities are what each server advertises before login. How these tests work