BCaching move to new server(s)
As I mentioned in the last post, I wanted to retire our old somewhat expensive Windows VPS and migrate to one or more cheaper servers, hopefully without sacrificing any quality of service.
After a couple of weeks of research, experimentation, and testing, I converted the geocache detail data away from individual flat files. Geocache detail includes short and long descriptions, hint, logs, and travelbug inventory. When I originally went to individual files I thought it was a terrible idea, but it DID get the large volume of “less often” used data out of MongoDB which saved other headaches, and it turned out to work reasonably well over the next year or so. That data has now been moved into CouchDB, with up to 25K geocache records per database file, for a total of 168 files at the moment. Each of these database files are between 50 and 150 MB and are now easier to manage than the 1.5 million individual files. Since CouchDB uses an append-only mechanism for adding/modifying data, these files grow with updates leaving unused holes in the file, but there is a nightly job that reclaims the space by compacting files after an appropriate threshold. Compacting a database file involves writing a new copy of the file without the unused data, but this has worked well since the database files are reasonably sized.
The main new server is a Linux VPS (Debian 7) with 50 GB disk, 1.5 GB RAM, and 2 CPUs. That is running CouchDB, MongoDB, MySql, Nginx, and Node, and phpbb forums.
There is also a new temporary “small” Windows VPS (30 GB disk, 768 MB, 2 CPUs) that is running the email processor, bcaching GPX file processor and the main web site.
My plan is to next focus on rewriting the email and GPX processors in python and/or node and move them to the linux server as well. Once that is done, the Asp.Net web site can be moved to a much cheaper shared hosting. The last step would be to rewrite the web site to run under node.
As part of the rewrite, I’m planning to open source all the new code which will be hosted on GitHub. I wonder if anyone might be interested in contributing.
Keep on caching!
I'd Rather Be Caching 
Leave a comment