Running databases in the cloud can be convenient, but it can also get expensive fast. For the Prosopo team, MongoDB Atlas was initially a fast, reliable way to run a cloud database, but as our data grew, so did the bills. Over the last year, we realised that we were spending thousands of dollars per month on infrastructure that we could run more efficiently ourselves. We explored various options but ultimately decided to migrate our MongoDB deployment to Hetzner, a cost-effective cloud provider.
Here's how we managed to cut our costs by 90% without sacrificing performance or reliability.
When we first started building, MongoDB Atlas was an easy choice. Everything was set up for us, and we could focus on building our application without worrying about database management. The free tier was sufficient for our initial needs, and as we grew, scaling up was as simple as a few clicks. However, scaling came with a steep price tag. We went from paying $0 per month for a small database to over $3,000 per month for a few hundred GBs of data. The cost breakdown before we migrated looked roughly like this:
| Service | Monthly Cost | |
|---|---|---|
| Atlas M40 Instance - AWS | $1000 | |
| Atlas Continuous Cloud Backup Storage | $700 | |
| Atlas AWS Data Transfer (Same Region) | $10 | |
| Atlas AWS Data Transfer (Different Region) | $1 | |
| Atlas AWS Data Transfer (Internet) | $1,000 | ❗ |
| Total + VAT | $3000+ |
The more keen eyed among you will have noticed the huge cost associated with data transfer over the internet - its as much as the servers! We're building Prosopo to be resilient to outages, such as the recent massive AWS outage, so we use many different cloud providers. This means that a lot of our database traffic goes over the internet, which is very expensive on MongoDB Atlas, due to it running on AWS (other options are available, but we chose AWS).
When your entire stack is on AWS, data transfer costs are minimal, as shown by the "Same Region" line item above. However, this architecture creates centralisation and single points of failure, which we want to avoid.
Oh, and even though we were paying this much, we didn't have access to any support! That's a separate paid plan.
Hetzner offered a compelling alternative:
- High-performance dedicated servers at a fraction of AWS pricing
- Predictable, flat-rate pricing without surprise bills
- Strong European data center presence, which aligns with our privacy requirements
- No Data Transfer fees between our application servers and database
We decided that the only way to regain control over our costs was to move to a self-hosted solution. MongoDB Atlas was running in replica set mode but we opted to instead set up a much beefier machine with a huge amount of RAM (256GB) and fast SSDs to handle our workload. In future, if we want to move back to a distributed setup, we can always add more nodes.
The server in question costs $160 per month, which is a huge saving over our previous costs.
Our Atlas instance held about 500GB of data but the product doesn't rely on this data in real-time - its used for generating detection rules using ML modelling, and the rules are applied to our CAPTCHA providers in a follow-up step. The reason for this architecture stems from our blockchain beginnings.
We were able to take a mongodump of the database, restore it to the new Hetzner server, and then use scripts to sync any changes that happened during the migration window.
Setting up our Hetzner server wasn't just a matter of spinning up MongoDB. We run Proxmox on the host machine, created an Ubuntu VM, and deployed MongoDB in Docker. Networking added another layer of complexity: the VM sits on a private subnet (10.0.0.x) with the host handling DHCP, NAT, and port forwarding. We also use Traefik as a reverse proxy to handle SSL and route traffic from the outside world to MongoDB, which saves us from wrestling with MongoDB's own SSL certificates. While this setup gives us flexibility and security, it does require a bit more technical know-how compared to fully managed solutions.
When you migrate from MongoAtlas to a self-hosted solution, you're taking on more responsibility for managing your database. You need to make sure it is secure, backed up, monitored, and can be recreated in case of failure or the need for extra servers arises. We used the following tools:
The monitoring in MongoDB Atlas was very useful. It highlighted times when we were opening up unlimited connections due to connection leaks in our application code. However, MongoDB Atlas wouldn't let us kill the connections using admin shell commands! This meant we reached our connection limit and we had no way to restart the server. We had to kill our services to free up connections. With our own server, we can now monitor and manage connections directly via OpenObserve and, crucially, the MongoDB shell.
Backups in MongoDB Atlas were also convenient but expensive. We now use mongodump, a cron, and a storage box from Hetzner to take daily backups of our database. The backups are compressed and encrypted before being sent to the remote storage box.
We've opted for a DIY approach to our servers and databases, which has significantly reduced our costs whilst giving us more control over our infrastructure. However, this approach isn't for everyone. We're still a small team, and managing the limited number of servers we run is feasible.
The performance we're seeing from our new standalone MongoDB server is very much improved over the Atlas setup. This is simply because the specs are so much higher and the indexes can mostly be held in memory. We are aware that MongoDB Atlas has something called vector search but we weren't using it, so it wasn't a factor in our decision.
For reference, the specs look like this:
| Specification | Atlas M40 Instance | Hetzner Dedicated Server |
|---|---|---|
| CPU | 8 vCPU | 8 cores Intel Xeon W-2145 |
| RAM | 16 GB | 256 GB |
| Storage | 380 GB SSD | 4 x 1 TB NVMe SSD RAID1 |
Its more than likely that as we grow, we'll start to see value in hosted solutions again, but for now we're quite pleased with the new setup. Hopefully this experience helps to inform others starting out on their cloud journey - for a small amount of pain you can save a lot of money!