An Update on Spam Filtering at DreamHost

Two and a half years ago, we started a series of improvements to our email service to make it more reliable. Our email service hadn’t received as much attention as it should have over the years, and it was starting to show. Customers, friends, family, and even DreamHost team members regularly saw emails delayed or rejected because of blocklists, email client connection timeouts and errors, and various issues with webmail. Once we realized the impact we could have, we set off to fix it, and the results have been dramatic.

We’ve made a lot of progress but still have a ways to go!

Our current project is to migrate our spam filtering service to a new provider for incoming messages. Unfortunately, we’re discovering that it tends to be overly aggressive and sends some messages to your spam folder that clearly don’t belong there. Spam blocking is a bit of an art form that requires careful attention to get the balance just right, and we’re working on that balance to get acceptable results. Our new provider is working closely with us to solve the problem and prioritize development efforts that will improve incoming mail filtering at DreamHost.

One method we’re using to resolve the problem is to train the spam filtering algorithm in bulk. Some customers have written to our tech support team for help and have been asked to provide examples of messages that have been mis-filtered. Thank you to everyone who has done so; these messages have been extremely helpful in training the filtering algorithm quickly.

As we monitor the reports of incorrectly filtered messages, we’ll also be safelisting known good senders at the DreamHost level. While training the filtering algorithm can take some time, these changes will have immediate effects. We are adding well-known domains to our global safelist when they have a valid DMARC configuration. This is a manual process and is similar to the allow and blocklist settings you have in the control panel and mailboxes interface. We’re also working to improve the interface in our customer control panel to make this even easier going forward.

We have seen progress in the functionality of our spam filters over the last two weeks. We’ve seen fewer legitimate emails getting caught by the spam filter, and customer complaints are trending downward. Our spam filtering partner has further updates planned to improve its capabilities, and we are continuing our manual safelisting process. We hope to meet your expectations (and our own) very soon.

Enhancements So Far

One of the first improvements we made was to completely refresh the hardware we use for email service. Our previous generation of email hardware had reached the end of its practical life, and it struggled to keep up with the growth in email users and the amount of storage they consumed. The new hardware is faster, easier to maintain, and highly scalable. This has brought improvements to our email service speed and stability along with fewer errors for customers in their email clients.

Next, we took on the challenge of rejected customer emails due to our email servers appearing on blocklists. The unfortunate fact is that spammers are relentless and will use every method at their disposal — legal or not — to send their junk messages. That includes breaking into our customers’ email accounts and using them to send spam. The activity on these compromised accounts leads to our mail servers being placed on blocklists and, unfortunately, legitimate emails are often blocked as a result.

We implemented a third-party service to filter outgoing emails and reject sending fraudulent messages or spam. We immediately saw our email servers’ reputation improve, and their appearance on email blocklists is now a thing of the past.

Earlier this year, we changed our webmail system to use the popular open-source software, Roundcube. While this may appear to be a simple software change, there was a lot of work to migrate existing data from the old webmail system to Roundcube. We also took the opportunity to modernize the infrastructure and deployment process behind the scenes making it easier to operate going forward. When all was said and done, we had gained a more performant webmail system with a beautiful and responsive design.

Future Plans

We have several more projects we want to complete to further stabilize and improve the reliability of our email service. The next project will be a change invisible to email users. We will be updating the architecture of our email back end so that it is more manageable and fault-tolerant. This isn’t an exciting project from a “cool new features” perspective, but it will result in less downtime and errors for the service.

Looking further ahead, we plan to improve the deliverability of messages sent from our hosting machines and customer email forwarded outside of our service. These services are also affected by compromised accounts, blocklists, and spammers. Our aim is to ensure those legitimate messages are delivered to their final recipients.

We have lots of ideas to further improve the email experience once we get past these crucial initial projects. To name a few, we want to improve the email user experience in our control, provide better options for spam handling instead of simply moving it to the spam folder, and provide better integration of features in webmail.

The list may be long, but we are excited to build it!

We appreciate your patience while we work to continually improve our email offering at DreamHost. Please don’t ever hesitate to contact us to share your thoughts about email — we truly value your feedback.

The post An Update on Spam Filtering at DreamHost appeared first on Website Guides, Tips & Knowledge.

source https://www.dreamhost.com/blog/update-spam-filtering/