Join me for a story about my journey migrating my personal API server to a new host. Never has there been such thriller as this adventure. There are learnings, distraction, stories, and of course a bitter sweet goodbye.

What do I host with my API server?

It’s a Node.JS express server.

Code is a joy of mine. In my spare time I tend spin up side projects and build experiments. Many of these ideas depend on storing & responding with data, thus I wrote a lightweight NodeJS express server to act as my generic multi-tenant api to server all apps I may rapidly prototype.

Some notable projects include:

• DancingQuest – Chicagoland & Wisconsin Social Dance Calendar
• WhatsOpenMcHenryCounty – Venues open during covid-19
• Google Home and Alexa Skills like the Alexa Dancing Lesson Skill
• Google Chrome Extension Database

Why Migrate?

• Better Specs!
• Upgrade OS!
• Avoid fixing broken things!

1) I use DigitalOcean as my host. The server cost $5/mo. I created the box 4yrs ago. For new boxes DigitalOcean now offers more RAM & Hard Drive space at the $5 tier.

2 ) Ubuntu upgrade form 16.04 to 18.04. I could just upgrade the system, but value in starting fresh right?

3) Ok ok, dpkg is also very broken on jstassen-01, something with the python 3.5 package being in a very bad inconsistent state. I would really like install docker and start running mysql or mongo db. I started to fix dpkg to get this working, but that went down a rabbit hole.

Given these 3 nudges, it just made sense to swap to a new box, install node and call it a day.

I got distracted at LiteSpeed!

I used DigitialOcean’s pre-configured Node boxes to get me started instead of going from a bare Ubuntu box this time. They have some nice ssh security & software firewalls prebuilt. Wait, but what are these other options?

Ooo what’s this? OpenLiteSpeed NodeJS? Never heard of it, let’s try it out!

OpenLiteSpeed is a Reverse Proxy akin in the same vain as Apache & Nginx. Hm should I be using a Reverse Proxy with my node server? Ok I’m swayed, let’s try it, can’t hurt.

After much confusion and configuration (C&C) I had things running pretty well. It required some modifications to the node app. The benefit of running the Reverse Proxy, the box can now listen on port 445 (https) and based off the domain name route to separate apps in the future. Do I need this? Not really, but don’t mind the option.

Then the code changes started

OpenLiteSpeed integrates & uses Let’s Encrypt out of the box. Previously I had the Node app handling serving up the certs. It’s rather nice to have the Node app be responsible for less. This brings the dev and production app closer together in parody.

The Database files are document based dbs stored to disk in flat files. It was nice to better organize where these were located on my file system. The migration to a docker based mysql or mongo db is a separate project. (whew avoided one distraction)

A new home

Next was updating the url to a more generic one. I previously used https://jstassen-01.jstassen.com. I could simply point that url at the new box. But that’s kinda ugly jstasssen-01 points to a server named jstassen-03 right? Hm. What about create a url like https://api.jstassen.com then it won’t matter what box it points to in the future.

Fun fact, API calls from an app won’t follow 301 redirects. So, redirecting jstassen-01.jstassen.com -> api.jstassen.com won’t work, especially with POST requests. Well Bummer.

No worries I can update all my apps to use to a new url. No big deal! … oh right that’s 11 different projects. Hm.

Half were very easy, however I wrote my first google chrome extension and Alexa skill back 4 years ago. I last updated & published them about 1 year ago. A lot of security changes have been updated for how these apps are now built. They have refined their permissions, coding patterns, and apis.Previously grandfathered in as legacy for both, but to redeploy, I needed to upgrade them. Sure, that can’t take long.

Next I noticed OAuth apps were failing. Cookies were failing to be set entirely. Kinda critical piece to remembering which user is authenticated! Interestingly Express.js by default won’t trust credentials forward to it through a reverse proxy (like LiteSpeed). Just needed to allow 1 layer of proxied requests. app.set('trust proxy', 1). Well that one liner took an evening to figure out, haha.

You mean I have to rebuild?

To use use the newest packages, I needed refactor both the google chrome extension and Alexa skills. 1 week later, whew it was complete! On the upside, all my dependencies are fresh and up to date. I now have a modern promise based ajax library and promise based db read / writes. Fancy.

I swear all I wanted to do was copy the code over to a new server and start it up. I didn’t bargain for these code improvements and refactoring.

Performance Testing

Is it faster? I ran 1,000 requests 10 concurrent against a heavy GET endpoint. The new box is on par and just marginally faster ( maybe 1%?) but it’s insignificant difference. Reasure none the less.

jstassen-02 (RIP) was a failed experiment running Plesk server. It was heavy, a RAM hog and just not optimized. Not to mention Plesk limits your vhosts. Api calls sometimes took twice as long compared to jstassen-01.

Backing up

It’s time to say farewell and delete jstassen-01. I’m not scared at all, why would I be? And yet I hesitate.

I found this youtube video with a reasonable way to create an archive of the Ubuntu box I can hold on to just in case. Can I restore from it? Hard to say. But at least the data will be there in the off chance I missed something.

# Backup
sudo tar -cvpzf jstassen-01.tar.gz --exclude=/backup/jstassen-01.tar.gz --one-file-system

# Restore
sudo tar -xvpzf /path/to/jstassen-01.tar.gz -C /directory/to/restore/to --numeric-owner

A small 3.24GB archive. I could make the archive smaller by clearing npm/yarn cache folders and clear temp files. But it’s close enough, not too bad.

Maybe next I’ll experiment with creating a new Droplet for a day (a droplet for a day probably cost something like $0.17) and try a restore. Would be interesting to understand this style of backup.

This probably was a bit overkill since I’ve also started to create 1 off private repos on GitHub and use them as a backups as well. So I a committed version my whole home directory too.

Saying “Goodbye” is never easy

Now to remove jstassen-01…

I’ll say a little homely when I click destroy…