Docker Compose Mail Server with Roundcube Webmail Service 2026

Why Run a Mail Server with Roundcube Using Docker Compose?

You manage your own domain. Your emails contain sensitive data. Yet every time you send or receive a message through a hosted service, your data flows through someone else’s servers—where it can be indexed, analyzed, or even leaked. What if you could own the entire flow, from inbox to archive, without relying on cloud giants or third-party apps?

Running a mail server with Roundcube via Docker Compose lets you do exactly that: control your email stack end to end. It’s like building your own private post office—no middlemen, no data harvesting, full transparency. With Docker Compose, you deploy it reliably, keep services isolated, and roll back instantly if something goes wrong. And Roundcube gives you a modern, usable web interface that feels familiar, not clunky.

This guide shows you how to set up a self-hosted mail server with Roundcube using Docker Compose—no cloud dependency, clear ownership, and real privacy. The setup is reproducible, auditable, and scalable. You’re not just sending emails. You’re running your own infrastructure.

Key takeaways

• Docker Compose simplifies managing a mail server with Roundcube by containerizing each service and enabling consistent, reproducible deployments.
• Self-hosting your mail server with Roundcube gives you full control over data privacy, message retention, and access—no third-party scraping or data mining.
• Roundcube provides a feature-rich, responsive webmail interface that integrates seamlessly with standard mail protocols (IMAP/SMTP) and supports modern workflows like calendar sync and contact management.

What You Need Before Starting: Prerequisites

You'll need a dedicated domain with full DNS control, a Linux server (Ubuntu 22.04 or Debian 12 recommended) with at least 2 GB RAM and 5 GB disk space, Docker Engine and Docker Compose installed and running, and a reverse proxy like Nginx or Traefik to handle TLS termination and HTTP/2. A reliable internet connection and a static IP (or dynamic DNS) are also essential for consistent access. Without these, your mail server won’t be reachable or secure.

Domain and DNS Access

• Register a domain (e.g., yourdomain.com) via a provider like Namecheap or Cloudflare. You’ll need full access to its DNS zone.
• Set up an MX record pointing to your server’s hostname (e.g., mail.yourdomain.com) to route incoming mail.
• Create SPF, DKIM, and DMARC DNS records to prevent spoofing and improve deliverability — these are critical for inbox placement. See the SPF spec and DMARC spec for technical guidance.
• Use tools like MXToolbox to validate your records before deployment.

Server and Software Setup

• Use a bare-metal or cloud server (DigitalOcean, Linode, AWS EC2, etc.) with a clean Ubuntu 22.04 or Debian 12 install.
• Ensure the server has a static IP address or set up a dynamic DNS service (e.g., duckdns.org) if your IP changes.
• Install Docker Engine and Docker Compose following official instructions from the Docker docs. Test with docker --version and docker-compose --version.
• Set up a reverse proxy like Nginx or Traefik to manage HTTPS, load balancing, and HTTP/2 — essential for modern webmail access.
• Generate and configure TLS certificates (Let’s Encrypt via Certbot is standard). Without HTTPS, webmail clients will block access.

Once these are in place, you can proceed with defining your docker-compose.yml and deploying your mail server stack. For a pre-built, fully encrypted, self-hostable alternative with built-in webmail, calendar, drive, and AI — all with domain setup handled in minutes — consider Unifiedesk. It includes Roundcube-like webmail, email encryption at rest, and end-to-end protection. No need to manage DNS records manually if you want a secure default setup.

Docker Compose Mail Server with Roundcube Webmail Service: The Full Setup

You can run a private, self-hosted mail server with Roundcube webmail using Docker Compose by setting up Postfix for sending, Dovecot for storing, Roundcube for web access, and Nginx as a reverse proxy with Let's Encrypt TLS. All services run on a private network, use environment files for secrets, and are managed via a single docker-compose.yml. Start with a project directory, define your services, secure them with SSL, and launch everything with one command.

1. Start by creating a dedicated project directory: mkdir mail-server && cd mail-server. This keeps your mail server files isolated and organized. Avoid mixing it with other projects to prevent configuration drift.
2. Create a docker-compose.yml file. Define four core services: postfix (SMTP), dovecot (IMAP), roundcube (webmail), and nginx (reverse proxy). Use the ABNF grammar in RFC 5234 as a reference for consistent configuration syntax across services.
3. Set up SSL/TLS using Let's Encrypt via your reverse proxy. With Nginx, use nginx-proxy/acme-companion or a Traefik integration. This auto-renews certificates and ensures secure connections to your webmail interface — a standard practice for publicly accessible services.
4. Configure user accounts and aliases outside the Docker stack itself. Use a centralized auth method (like LDAP or a database) if you need scalability. For simplicity, store passwords in a .env file with POSTFIX_PASSWORD and DOVECOT_PASSWORD variables, never in plain text in your YAML.
5. Bind all services to a private Docker network. Add networks: private to each service and define networks: at the bottom with driver: bridge. This isolates your mail stack from the internet and reduces attack surface — a core principle of secure system design.
6. Use .env files for all secrets. Include variables like DOMAIN=mail.example.com, [email protected], and LETSENCRYPT_EMAIL. Never hardcode them in your compose file. This is how systems like DNS privacy standards recommend handling credentials.
7. Run docker-compose up -d to start all services in the background. Then verify they’re running with docker-compose ps. If any container fails, check logs with docker-compose logs <service> and fix configuration issues.

Security and Maintenance

You must regularly update system dependencies and monitor logs. Use tools like MxToolbox to check SPF, DKIM, and DMARC alignment and avoid delivery issues. For a ready-to-use, secure alternative that handles TLS, encryption, and auto-updates, consider Unifiedesk’s self-hosted option.

How Your Dockerized Mail Stack Works: Component Roles

You’re running a self-hosted mail server with Docker Compose, and it’s made up of five core parts: Postfix handles mail delivery and routing, Dovecot manages inbox access via IMAP or POP3, Roundcube gives you a modern web interface, Nginx serves that interface securely over HTTPS, and Let’s Encrypt provides free TLS certificates. Together, they form a complete, privacy-respecting communication stack where you control your data from inbox to server.

Core Mail Infrastructure: Postfix and Dovecot

Postfix is your mail transfer agent (MTA), meaning it receives incoming emails from other servers and decides where to deliver them—either to a local user, forward them, or relay them externally. It speaks SMTP, the standard protocol for sending and receiving mail, and is known for being fast, secure, and battle-tested over decades. It routes mail based on your domain’s MX records, which you configure at your DNS provider.

Dovecot is the mail delivery agent (MDA) and IMAP/POP3 server. It stores messages in a reliable format—your choice between mbox or maildir—and provides the underlying storage layer for your users. With system accounts, it handles authentication and keeps your inbox accessible across devices. It’s one of the few MUAs trusted by major email providers for its security and standards compliance.

Web Access and Security: Roundcube, Nginx, and Let’s Encrypt

Roundcube is the webmail interface you actually click and type into. It connects to Dovecot over IMAP to read and send messages, and to Postfix via SMTP to send outgoing mail. It’s lightweight, feature-rich, and built for usability—so you can read and compose emails without a desktop client.

But raw HTTP is dangerous. That’s where Nginx comes in: it acts as a reverse proxy, standing between the internet and your web services. It routes traffic to Roundcube and handles HTTPS encryption, ensuring all data in transit is protected. This is where TLS matters most—you’re not just using encryption, you’re enforcing it.

And who signs the SSL certificate? Let’s Encrypt. It’s a free, automated certificate authority backed by the Internet Security Research Group (ISRG)—the same team behind the ACME protocol that powers modern web security. Tools like Let’s Encrypt’s ACME protocol make HTTPS deployment effortless, especially when paired with Traefik or certbot in your Docker setup.

If you want to run a full, private workspace with mail, calendar, docs, and video meetings—without relying on cloud providers—our self-hosted version does this stack for you with built-in encryption, JMAP, and a simple setup. You still control your data, your domain, and your inbox.

Critical DNS Records for Email Delivery and Security

You must set four DNS records to secure your email domain: MX to route incoming mail to your server, SPF to authorize sending servers, DKIM to cryptographically sign outbound messages, and DMARC to define how receivers handle unauthenticated mail. Without all four, your emails risk being marked as spam or rejected outright.

Configure Your DNS Records Correctly

Let’s go through each record in plain terms. Use your domain registrar’s DNS management tool — whether Cloudflare, Namecheap, or AWS Route 53 — to add these entries.

Key DNS Records for Your Mail Server

These records work together. For instance, DMARC relies on SPF and DKIM to enforce rules. You can check your setup using DMARC Inspector or MXToolbox.

If you’re self-hosting, these records are your foundation. Misconfigured DNS is a common reason emails don’t deliver. You can verify them using public tools like RFC 7073 or RFC 5321.

Want a hosted solution with built-in DNS record setup? Unifiedesk generates and applies MX, SPF, DKIM, and DMARC records for your custom domain in minutes — no technical setup needed. Set up your custom domain with Unifiedesk and focus on using email, not managing DNS.

Avoid Common Self-Hosting Pitfalls: Lessons from 2025

You don’t need to be a security expert to run a mail server, but you do need to avoid the five most common mistakes that lead to breaches, data loss, or downtime. Always use a reverse proxy with TLS for SMTP and IMAP; never expose them directly. Run containers as non-root users. Back up mail data nightly to encrypted remote storage. Monitor logs and block bad actors with fail2ban. And keep your containers updated—scan for known CVEs using tools like Trivy. These aren’t optional; they’re baseline hygiene.

Security by Design

• Never expose SMTP or IMAP directly to the internet. Use a reverse proxy (like Nginx or Traefik) with HTTPS to encrypt traffic and shield your server from direct attack—this is a core principle of modern email security, as outlined in RFC 8314.
• Run services inside Docker containers with non-root users. If a container is compromised, root escalation is one of the fastest ways attackers gain full control—avoid it from the start.
• Back up mail data every night to encrypted remote storage. Use tools like Borg or Duplicity to compress, encrypt, and sync your mail database—your data is only safe if it’s backed up and stored securely.
• Monitor mail logs daily. Look for repeated login failures, unusual send rates, or failed connections. These are early signs of brute-force attacks or compromised accounts.

Maintenance & Monitoring

• Deploy fail2ban to automatically block IPs that make too many failed login attempts. This simple tool stops most automated attacks before they succeed.
• Update containers regularly and scan for known vulnerabilities using Trivy or similar. A single unpatched flaw in a library can expose your entire mail server—this isn’t paranoia, it’s practice.
• Test your backups monthly. You can’t trust a backup you’ve never restored. Use a test environment to simulate a full recovery.
• Keep your system clock in sync with NTP—misaligned timestamps break mail authentication and make logs harder to analyze.

Self-hosting your email is empowering, but not risk-free. If you're weighing the effort against the control, consider a managed alternative with strong privacy guarantees—like Unifiedesk’s self-hosted option. It ships with encryption at rest, per-account keys, and full control over data. And if you’d rather not manage the stack at all, Unifiedesk’s hosted email offers zero-compromise privacy with automatic compliance, all from your own domain.

Security in the Stack: How Encryption Applies to Your Setup

Transport Layer Security (TLS) protects all traffic between your browser and your mail server—both for webmail access and when sending or receiving emails over SMTP/IMAP. At rest, your mail and files remain unencrypted by default unless you use disk-level encryption like LUKS. If you run Unifiedesk in self-hosted mode, every message and file is encrypted at rest using AES-256-GCM under per-account keys, giving you strong, granular control over data privacy. Modern web sessions use HTTPS with HSTS to prevent downgrade attacks, and you can enforce client-side authentication for added protection.

TLS and In-Transit Protection

When you access Roundcube via HTTPS, TLS encrypts every byte that crosses the network—your login, your inbox view, and your outgoing messages. This is standard practice in secure web services and aligned with modern security best practices, such as those outlined in RFC 8446 for TLS 1.3. Without TLS, your credentials and emails would be exposed to snoopers on shared networks. Even if your server is in your basement, unencrypted traffic is vulnerable to packet capture, which is why TLS is non-negotiable.

At-Rest Encryption: What’s Actually Protected?

By default, mail systems store messages in plaintext on disk. If someone gains physical access to your server’s storage, they can read your mail unless you use full-disk encryption. LUKS (Linux Unified Key Setup) is the industry-standard way to encrypt entire block devices—so even if your hard drive is stolen, the data remains protected. But if you’re using a hosted mail setup and rely on your provider’s infrastructure, you’re trusting their disk encryption practices.

If you run Unifiedesk self-hosted, the encryption model is different: everything—emails, calendar entries, documents in Drive—is encrypted at rest using AES-256-GCM, with keys derived per user account. No single key ever decrypts all data across users, so even if someone compromised your database, they couldn’t read messages without the corresponding account key. This level of encryption is not automatically available in generic Docker setups; it requires explicit design.

Webmail sessions use HTTPS with HSTS (HTTP Strict Transport Security) to enforce encrypted connections and prevent attacks like SSL stripping. You can further strengthen authentication by setting up client-side certificates or OAuth2 via your identity provider. While Roundcube itself doesn’t include built-in MFA, you can layer it at the reverse proxy (e.g., with Nginx + authelia or Traefik + OIDC).

For a full-featured, privacy-first email and workspace suite with end-to-end encryption built-in—whether you're self-hosting or using a private domain—you can set up Unifiedesk with full control over encryption, data residency, and access. Self-hosted Unifiedesk gives you the tools to implement these security patterns directly. Explore how encrypted storage, secure webmail, and private collaboration come together in a single, open-source stack.

Why You Should Consider Unifiedesk Instead of DIY

You don’t need to wrestle with DNS records, TLS certificates, or email deliverability troubleshooting to run a secure, self-hosted email system. Unifiedesk automates DKIM, SPF, and DMARC setup with one click—no manual editing, no guesswork—and keeps your messages private with end-to-end encryption on the hosted platform, or AES-256-GCM at rest in self-hosted deployments. You get a full workspace suite with modern tools, not just email.

Automated Security, Real Privacy

Setting up SPF, DKIM, and DMARC for your domain is painful—requiring precise DNS records that break if misconfigured. Unifiedesk handles all of it automatically when you add a domain. No need to copy-paste TXT records or double-check syntax. If you're self-hosting, your data is encrypted at rest using per-account keys, making it unreadable without the user’s credentials. For hosted users, end-to-end encryption ensures mail and files are protected from anyone—not just the provider, but even internal access. It’s a design principle, not a selling point.

While Docker Compose lets you deploy Roundcube, it doesn’t include other core tools like calendar, video meetings, or document collaboration. Unifiedesk offers all of that natively, with a modern, consistent web UI across mail, calendar, Drive, Docs, and contacts. You’re not building an email server—you’re building a complete workspace.

Modern Features, No Legacy Hassles

You don’t get stuck with outdated protocols or fragile scripts. Unifiedesk supports JMAP, the modern standard designed for performance and reliability, alongside IMAP. File attachments go up to 25 MB, with expiring share links and secure access controls in Drive. Features like undo-send, snooze, Sieve filters, and a powerful AI assistant (which works with any OpenAI-compatible endpoint) are included by default. Need to video conference? Meet has screen sharing and recording, no extra tools required.

Whether you're hosting in the cloud or running on your own server, Unifiedesk lets you add custom domains, manage shared mailboxes, assign admin roles, and scale easily. It’s built on open standards and open source, with full transparency. While Docker Compose gives control, it also means you’re responsible for updates, backups, security patches, and configuration drift. Unifiedesk handles all of that automatically, so you don’t have to.

For a complete privacy-first workspace, check out self-hosting or get started with a custom domain—no Docker, no CLI, no learning curve.

After Deployment: Testing and Maintenance

After spinning up your Docker Compose mail server with Roundcube, verify SMTP delivery, check logs, test inbox functionality, automate TLS renewal, and monitor resource usage to keep everything running smoothly. Let’s walk through the essential steps.

Test Delivery and Inbound Mail

1. Use swaks to test SMTP delivery: swaks --to [email protected] --from [email protected] --server mail.yourdomain.com. This confirms your server accepts outgoing mail and handles authentication correctly. A successful response means SMTP routing is operational.
2. Check logs in real time with docker-compose logs -f postfix or, if using systemd, journalctl -u [email protected] -f. Logs help track delivery failures, connection issues, or rejection reasons like SPF/DKIM misconfigurations.
3. Log in to Roundcube via your web interface and send a message from your test account to another address. Then, verify it arrives. This validates both inbound delivery and the mail client’s functionality.

Automate Security and Monitor Performance

4. Set up a cron job to renew TLS certificates automatically. If using Certbot with a standalone or webroot plugin, schedule certbot renew --quiet --no-self-upgrade daily. Certificates expire every 90 days, and renewal failures can break mail delivery — Let’s Encrypt emphasizes timely renewals.
5. For Traefik users, enable automatic ACME via acme.httpChallenge in the configuration — it handles renewal without manual cron jobs. This is one reason Traefik is favored in modern self-hosted setups.
6. Monitor system health with htop for CPU and memory, df -h for disk space, or a monitoring dashboard like Grafana with Docker metrics scraped via Prometheus. High disk use on mail stores can lead to delivery failures if unchecked.

Even with solid deployment, maintenance is ongoing. A single misconfigured DNS record or expired certificate can block mail flow instantly. Regular testing and monitoring prevent downtime. If you're managing a team, consider a solution like Unifiedesk — it simplifies email, calendar, contacts, and drive with built-in security and self-hosting support.

For more on managing private email infrastructure, see our security page, which explains encryption at rest, TLS in transit, and how custom domains are secured by default.

Can You Use This Stack for Production or Daily Use?

Yes — many individuals and small teams run this exact stack in production, using Docker Compose with Roundcube for a fully private, self-hosted email solution.

But it’s not set-and-forget. Production use demands regular backups, health monitoring, TLS certificate renewal, and ongoing security updates — all handled automatically in managed platforms.

Consider Unifiedesk if you want to own your data without the setup grind.

• No Docker, no Docker Compose, no Roundcube configuration.
• Deploy private email on your domain in minutes with full control.
• Includes calendar, video meetings, Drive, Docs, and an AI assistant — all encrypted and sovereign.

Ready to put this into practice? Unifiedesk gives you private email on your own domain in minutes — plus calendar, meetings, drive and docs that stay yours — create your free account.

Frequently asked questions

Can I use Docker Compose to run a mail server at home?

Yes—but ensure your ISP allows inbound SMTP connections, and use a reverse proxy with TLS to avoid connection issues or blacklists.

Does Roundcube support IMAP and SMTP securely?

Yes, Roundcube uses IMAP and SMTP with TLS encryption when configured through a reverse proxy; always enable SSL/TLS in the config.

How do I set up DKIM for my Docker mail server?

Generate a private key with `openssl genrsa -out dkim.private 2048`, publish the public key in DNS as a TXT record under a selector.

Is self-hosting email worth it in 2026?

Yes, if you value privacy, data control, and long-term sovereignty—but be prepared for ongoing maintenance.

Can Roundcube sync with mobile devices?

Yes, via IMAP. Use a standard email client (like K-9 Mail or Mailplane) with your domain’s IMAP and SMTP settings.

What is the difference between IMAP and JMAP?

JMAP is a modern, asynchronous protocol that supports real-time sync and bulk operations, unlike traditional IMAP; Unifiedesk supports both.

Do I need a static IP to self-host email?

Not strictly—but dynamic IPs can cause deliverability issues. Use a dynamic DNS service (like DuckDNS) and monitor your IP reputation.

Can I migrate from Gmail to a self-hosted server?

Yes, using tools like `imapsync` to copy data from Gmail to your Dovecot server; ensure SPF/DKIM/DMARC are properly configured.

How does Unifiedesk compare to DIY Docker mail setups?

Unifiedesk skips complex setup, includes built-in security (DKIM, SPF, DMARC), supports self-hosting with full data control, and adds Drive, Calendar, and Meet.

Is Docker Compose mail server easy to back up?

Yes—back up the mail data directory and database (if used), and store encrypted backups offsite; consider tools like Borg or Rclone.