Coding

MIgrating API servers

Leave a comment

Join me for a story about my journey migrating my personal API server to a new host. Never has there been such thriller as this adventure. There are learnings, distraction, stories, and of course a bitter sweet goodbye.

What do I host with my API server?

It’s a Node.JS express server.

Code is a joy of mine. In my spare time I tend spin up side projects and build experiments. Many of these ideas depend on storing & responding with data, thus I wrote a lightweight NodeJS express server to act as my generic multi-tenant api to server all apps I may rapidly prototype.

Some notable projects include:

Why Migrate?

  • Better Specs!
  • Upgrade OS!
  • Avoid fixing broken things!

1) I use DigitalOcean as my host. The server cost $5/mo. I created the box 4yrs ago. For new boxes DigitalOcean now offers more RAM & Hard Drive space at the $5 tier.

What happened to jstassen-02? RIP

2 ) Ubuntu upgrade form 16.04 to 18.04. I could just upgrade the system, but value in starting fresh right?

3) Ok ok, dpkg is also very broken on jstassen-01, something with the python 3.5 package being in a very bad inconsistent state. I would really like install docker and start running mysql or mongo db. I started to fix dpkg to get this working, but that went down a rabbit hole.

Given these 3 nudges, it just made sense to swap to a new box, install node and call it a day.

I got distracted at LiteSpeed!

I used DigitialOcean’s pre-configured Node boxes to get me started instead of going from a bare Ubuntu box this time. They have some nice ssh security & software firewalls prebuilt. Wait, but what are these other options?

Ooo what’s this? OpenLiteSpeed NodeJS? Never heard of it, let’s try it out!

OpenLiteSpeed is a Reverse Proxy akin in the same vain as Apache & Nginx. Hm should I be using a Reverse Proxy with my node server? Ok I’m swayed, let’s try it, can’t hurt.

After much confusion and configuration (C&C) I had things running pretty well. It required some modifications to the node app. The benefit of running the Reverse Proxy, the box can now listen on port 445 (https) and based off the domain name route to separate apps in the future. Do I need this? Not really, but don’t mind the option.

OpenLiteSpeed Listener configuration & Virtual host mapping page. This will be handy.

Then the code changes started

OpenLiteSpeed integrates & uses Let’s Encrypt out of the box. Previously I had the Node app handling serving up the certs. It’s rather nice to have the Node app be responsible for less. This brings the dev and production app closer together in parody.

The Database files are document based dbs stored to disk in flat files. It was nice to better organize where these were located on my file system. The migration to a docker based mysql or mongo db is a separate project. (whew avoided one distraction)

A new home

Next was updating the url to a more generic one. I previously used https://jstassen-01.jstassen.com. I could simply point that url at the new box. But that’s kinda ugly jstasssen-01 points to a server named jstassen-03 right? Hm. What about create a url like https://api.jstassen.com then it won’t matter what box it points to in the future.

Fun fact, API calls from an app won’t follow 301 redirects. So, redirecting jstassen-01.jstassen.com -> api.jstassen.com won’t work, especially with POST requests. Well Bummer.

No worries I can update all my apps to use to a new url. No big deal! … oh right that’s 11 different projects. Hm.

Tracking my progress. Emojis always help.

Half were very easy, however I wrote my first google chrome extension and Alexa skill back 4 years ago. I last updated & published them about 1 year ago. A lot of security changes have been updated for how these apps are now built. They have refined their permissions, coding patterns, and apis.Previously grandfathered in as legacy for both, but to redeploy, I needed to upgrade them. Sure, that can’t take long.

Next I noticed OAuth apps were failing. Cookies were failing to be set entirely. Kinda critical piece to remembering which user is authenticated! Interestingly Express.js by default won’t trust credentials forward to it through a reverse proxy (like LiteSpeed). Just needed to allow 1 layer of proxied requests. app.set('trust proxy', 1). Well that one liner took an evening to figure out, haha.

You mean I have to rebuild?

To use use the newest packages, I needed refactor both the google chrome extension and Alexa skills. 1 week later, whew it was complete! On the upside, all my dependencies are fresh and up to date. I now have a modern promise based ajax library and promise based db read / writes. Fancy.

I swear all I wanted to do was copy the code over to a new server and start it up. I didn’t bargain for these code improvements and refactoring.

Performance Testing

Is it faster? I ran 1,000 requests 10 concurrent against a heavy GET endpoint. The new box is on par and just marginally faster ( maybe 1%?) but it’s insignificant difference. Reasure none the less.

RIP jstassen-02, you were taken from us much too soon.

jstassen-02 (RIP) was a failed experiment running Plesk server. It was heavy, a RAM hog and just not optimized. Not to mention Plesk limits your vhosts. Api calls sometimes took twice as long compared to jstassen-01.

Backing up

It’s time to say farewell and delete jstassen-01. I’m not scared at all, why would I be? And yet I hesitate.

I found this youtube video with a reasonable way to create an archive of the Ubuntu box I can hold on to just in case. Can I restore from it? Hard to say. But at least the data will be there in the off chance I missed something.

# Backup
sudo tar -cvpzf jstassen-01.tar.gz --exclude=/backup/jstassen-01.tar.gz --one-file-system

# Restore
sudo tar -xvpzf /path/to/jstassen-01.tar.gz -C /directory/to/restore/to --numeric-owner

A small 3.24GB archive. I could make the archive smaller by clearing npm/yarn cache folders and clear temp files. But it’s close enough, not too bad.

Maybe next I’ll experiment with creating a new Droplet for a day (a droplet for a day probably cost something like $0.17) and try a restore. Would be interesting to understand this style of backup.

This probably was a bit overkill since I’ve also started to create 1 off private repos on GitHub and use them as a backups as well. So I a committed version my whole home directory too.

Saying “Goodbye” is never easy

Now to remove jstassen-01…

I’ll say a little homely when I click destroy…

End of line
Tips

40+ Questions to ask when searching for an apartment

Leave a comment

Apartment hunting can be fun, exciting, tricky, confusing, or filled with uncertainty.

Here are 40+ things I’ve thought of to think about when looking at a new apartment.

When comparing apartments, it might be good to build a spreadsheet and rank each apartment against each other. Or think about which items are a must have or like to have. We each rank things differently. For instance sunlight is very important to me, but for you maybe ceiling height is.

Feel free to use these as a starting point for you priorities in where you live!

Utilities
Water
Gas
Garbage
Recycling
Sewage
AC Units
Central Heating
Internet
Buy it yourself or provided?
Wired
Bike
Storage
Ok in Apartment?
Parking
Reserved
Indoor / Garage
Guest parking
Overnight guest parking
Items
Microwave
Dishwasher
Washer / Dryer
Smoke Detectors
Carbon monxided Detector
Aesthetics
Ceiling Height?
Amount of sun light
Street noise when windows open?
Wind / Breeze?
Curtains?
Fridge Noise
Coat closet
Neightbor dogs
Shower Height
Room for bed
Room for desk
Hot Water source?
Holes in wall?
Paint?
Cell service
Neighborhood
Bike paths / routes
Nearest Stores
Nearest Restaurants
Nearest Bike shop
Bus stops / Routes
Metra Station
Contract
Terms?
Breaking contract?
Sub-leasing
Downpayment
Credit Unions
Landlord terms, can they enter any time
Coding

Code Reviews on Closed Pull Requests

1 Comment

Good Code Review (CR) on Pull Request (PR) are at the heart creating high quality code. Code reviews on closed pull requests can feel awkward.

I believe we should welcome feedback on all PRs — even closed ones.

A coworker of mine came back from a short vacation and read over my PRs I had opened and merged while they were out. They spotted a bug in one of my PRs and they left a comment pointing it out. They however felt deeply guilty commenting on a closed PR and apologized to me. This is odd to me.

Merged PRs often get treated as code where the “Ship that has sailed”. Treating merged code as “untouchable” forgoes the ownership and responsibility we developers have for the code we create.

I suggest a cultural shift: To strive to produce high quality code, we ought to welcome and be open to Code Review feedback, before, during, and after the life of a Pull Request.

How we allocate time to address the feedback will range from team to team but could be new backlogged stories, re-opening a new PR, or just answering questions.

How do you feel about code reviews on closed pull requests?
Is there a better way to handle code feedback after it’s merged?

Coding

Using Immutable.js .update() instead of .get() then .set()

Leave a comment

Often you need to deeply update an complex Map or List. Using Immutable.js .update() is a powerful way to this kind of update.

Given this state

const = myState = Immutable.Map({
'123': Immutable.Map({
id: 123,
b: false,
}),
'456': Immutable.Map({
id: 456,
b: true,
}),
});

Don’t use .get() then .set()

const myUpdatedItem = myState.get('123').set('b', true);
const myNewState = myState.set('123', myUpdatedItem);

Using .get() then .set() seems simple and clean, but it’s slow, it fully replaced the the current Map. While the results are the same, ImmutableJS cannot optimize against it and cannot make some faster assumptions.

Using Immutable.js .update()

const myNewState = myState.update('123', (myMap) => {
return myMap.set('b', true);
});

It is much better to using Immutable.js .update(). Immutable can keep more references and make assumptions that allow it to optimize the update. For more, check the .update() docs.

You can also use .updateIn() for deeply nested data. Read my post on Immutable get() vs .getIn() to see how it works.

Photography

Shooting Raw instead of JPG

Leave a comment

A while back I switch from shooting Raw instead of JPG. It was a bit of a leap of faith. I didn’t know how I was going to be able to use my photos when they weren’t JPGs.

Why?

Check out this photo. If I was shooting JPG what you see on the left is all I would have. No way of saving anything. That photo is a gonner, throw it out.

But since I was shooting RAW, all the data is there. It just needs to be brought out. Heck, look at the sky right at the bottom on the line of the before/after. There’s a cloud!

Left image is over exposed with blown out sky, right image has recovered sky and can even see a cloud.

It’s photos like this that have sold me on shooting RAW.

JPG Pros / Cons

Pros:

  • Ready to be posted or shared
  • No need to do further processing in another application

Cons

  • Already processed in camera
  • Considered Final
  • (8bit)

Raw Pros / Cons

Pros:

  • More data
  • More color data (12 or 14bit)
  • Higher detail in shadows / highlights
  • You determine how the data interpreted
  • Can recover from over / under exposed images
  • Color temperature easily corrected
  • Non-destructive editing
  • Better sharpening / noise reduction

Cons:

  • Larger file size
  • Need software on the computer like Lightroom
  • Cannot use images on social sites
  • Takes longer to write a photo the your camera chip
  • Photographers with Time
  • “Flat” / Low contrast – Not developed yet
Coding

Exclude redirecting subdomains in htaccess RewriteRule

Leave a comment

Redirecting websites to be secure with https can be a little tricky.  I had to exclude redirecting subdomains in htaccess RewriteRule.  This mostly because a number of my subdomain’s code lives within a subfolder of my primary domain.

How to Exclude redirecting subdomains in htaccess RewriteRule

To exclude redirecting subdomains in htaccess RewriteRule use the following RewriteCond. This tell Apache to only use the redirect if the request is to the top level domain, not a subdomain.

RewriteCond %{HTTP_HOST} ^[^.]+\.[^.]+$

For instance, my .htaccess for jstassen.com looks like:

RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteCond %{HTTP_HOST} ^[^.]+\.[^.]+$
RewriteRule ^(.*)$ https://jstassen.com/$1 [R=301,L]

This regex only fully matches the base domain. You can see this on Debuggex.

Exclude redirecting subdomains in htaccess RewriteRule

Note: websites with www are considered a subdomain. For instance www.jstassen.com is a subdomain of jstassen.com just like blog.jstasseen.com. This rule will therefore exclude the www subdomain. I have my www subdomain auto redirect to remove the www which I recommend.

Tips, Travel

50+ Ways to Travel Cheap Around the World

Leave a comment

Thinking back on how I travel I realize that I’ve develop a set of principles for ways to travel cheap and keep costs low.

These are only general rules of thumb and all are subject to being broken.

50+ Ways to Travel Cheap
1 Week in California $45/day (Excluding flight)

Recent trip costs

Two weeks in Europe France -> Switzerland -> Germany $2500
(~$105 / day without flight)

One week in Alaska $800
(~ $40 / day without flight)

One week in Europe Poland -> Slovakia $1,800
(~$150 / day without flight)

One week in NYC $1,700
(~$97 / day without flight)

Three weeks in Japan $3,000
(~$60 / day without flight)

Ways to Travel Cheap

Pick an affordable destination

  1. Look up food cost
  2. Look up lodging cost
  3. Look up transit cost

Airfare

  1. Be flexible with departure dates, use an app that shows nearby date prices.
  2. Google Flights
  3. Hopper App
  4. Hipmunk App
  5. Chepoair
  6. Book Directly with airline
  7. Max out your carry on (Check as few bags as possible)
  8. Don’t buy things on the airplane or at airport
  9. Don’t upgrade your seat, your only on the plane for a few hours
  10. Some airlines charge to pick your seat, don’t pick a seat.
  11. If a company is paying for flights, extend the days.

Attractions

  1. Book online, sometimes there is a deal / coupons
  2. Pay as a group
  3. Share audio tours – makes it more social
  4. Consider a multi-attraction pass
  5. Stop by a hostel (even if you’re not staying), sometimes there are deals on attractions & tours
  6. Don’t buy the all access pass if you don’t have time to see everything
  7. If you are a student, pay student pricing
  8. Check if you get discounts with AAA, Your Credit Card company, AARP, etc.

Food

  1. Minimize eating out
  2. Street food / Food carts are generally affordable and good
  3. Bakeries
  4. Grocery stores
  5. Cook
  6. Snacks
  7. Refill a water bottle instead of buying beverages.
  8. Eat 2 blocks away from tourist destination.
  9. Pack and bring lunch to tourist destination,
  10. Pick cheep food destination.
  11. Share entrees
  12. Keep and eat any leftovers instead of tossing
  13. Check local customs on tipping. You might not need to tip, even though it may feel odd.

Lodging

  1. Couchsurfing
  2. Stay with friends, or friends of friends. (ask facebook)
  3. Hostel (book directly if possible or by phone, avoid sites that add a fee)
  4. AirBnB
  5. Camping
  6. Call other hotels and talk them down by comparing prices
  7. Use your AAA membership discount if you have it

Transit

  1. Rail deals (tourist pass)
  2. Split cost between travelers (car, group train ticket)
  3. Compare bus vs Train
  4. Spend time reading and understanding a city’s light rail & bus passes to find the best fit for your visit.
  5. Don’t drive the toll ways
  6. Park a little ways away
  7. Consider alternatives
  8. Walk / Bike

Currency

  1. Don’t get the currency before you leave (ok maybe a little)
  2. Don’t get currency at a cash exchange (terrible exchange rate)
  3. Withdraw cash at ATMs with a debit card
  4. Use a Credit Card with no foreign transaction fee.
  5. When given the option always charge the transaction in the local currency, it will convert with fewer fees.
  6. Prefer credit card to cash if possible

Phone

  1. Check what is free with your current provider. For instance texting both photos & videos is free internationally with AT&T, but calling and cell data costs.
  2. Disable all cellular data on your phone to avoid fees. Only enable on a per-app basis as needed.
  3. Download offline maps & language translations to reduce data use.
  4. Don’t check your voicemail.
  5. Bring an unlocked cell phone, and buy a local SIM, takes research.
  6. Call home using WiFi with VoIP. Like Google Hangouts, Skype, Whats App, FB Messenger, etc.

Other thoughts

  1. Don’t buy souvenirs, find souvenir (rocks, snacks, etc)
  2. Research Global Blue
  3. Check behind you and don’t loosing things – replacing items costs
  4. Check if you need to buy travel insurance

Did I miss something? Do I have something wrong? What ways to travel cheap do you know? Leave a comment, I’m always updating this list with more ideas / corrections!

Photography

7 Ways to Get More Likes on Instagram

Leave a comment

Starting out on Instagram is tough. I’m still building my following. I’ve figured out a couple of patterns that are best to follow when posting to get more engagement on posts. Here are 7 ways to get more likes on Instagram that I’ve found.

Post in the early morning, or the evening.

People tend are busy during the day, they check their feeds either when they are getting ready for the day or after they are starting to wind down. Of the two, evening tends lead to a slightly better output.

Here I posted around 5pm.

Post at the best time for you audience.

Know your audience. If you’re posting a shot from a past trip to another country, try to match your post with when they are awake. Thinking about your audiences you might be able to get two audiences at the same time.

For instance, this shot is taken in Switzerland, I live in Chicago. I posted at 7am (morning) Chicago time 3pm (evening) Switzerland time. I hit both my local audience and my Europe audience.

Hashtag your post

I’ve seen many great photographers fail to put any hashtags on their posts. Hashtagging is a central feature of Instagram. It’s a key way for others find your post.

I follow the four Ws + D rule:
Where? (stmaryglacier, colorado) What? (mountains) How? (lensbaby, tiltshift) Doing? (hiking).

Use only hashtags that have posts

Being creative is good, but if you make up a new hashtag or hashtags that has few posts – No one will find your photo with that hashtag.

On this shot every hashtag here has at least 1,000 posts.

Hashtag in different languages

Hashtagging in other languages broadens the size of your audience. You don’t need to use obscure hashtags. I recommend using Wikipedia instead of Google Translate.

In this shot I looked up the article ‘Dragonfly’. On the left of Wikipedia you’ll see a list of the article in different languages.

Add a geo location

Every photo was shot somewhere. You can only set one location, but there might be duplicates of the location on Instagram. Before posting try to searching the location to see which one gets tagged with similar photos.

For instance here you can see I set the location for this post to Steamboat Resort.

Skiing in a wonderland. . . . #steamboat #winter #winterwonderland #snow #ski #skiing #colorado

A post shared by Sir.Nathan Stassen (@thebox193) on

Like back

When someone likes your photo, take a moment to scroll through their feed and like a photo or two, maybe leave a comment. But wait a few minutes before doing this. What you want is to pull them back to view your feed. They sometimes recognize your username / avatar and will view your feed to like more shots. You’ve invested in them, they will now invest time back in you.

Everyone that liked this shot, I liked at lest one of their photos back.

Coding

Comparing Immutable.Set() and Immutable.OrderedSet()

Leave a comment

When comparing Immutable.Set() and Immutable.OrderedSet() you’ll need to convert the OderedSet down to just a plain Set.

Comparing Immutable.Set() and Immutable.OrderedSet()

To compare equality with a Set and OrderedSet use this:

mySet.equals( myOrderedSet.toSet() )

Immutable .equals() compares the left and right to decide if they are both Ordered. If one is ordered and the other is not, they are not equal. isOrdered(a) !== isOrdered(b).

My Feelings

I find this a bit odd, especially when the left is a plain Set. I feel regardless if the right is Ordered or not, it’s the contents that we’re interested in comparing. However this is the Immutable.js definition. You will need to downgraded the OrderedSet for comparison via .toSet() which does have a slight cost to it. Immutable performance is worth keeping an eye on.

Coding

Changing SourceTree’s Default Remote

Leave a comment

I’m going to talk about changing SourceTree’s default remote.

When you’re pushing a new branch, SourceTree will automatically guess which remote you wish to push to by default.

SourceTree defaults to origin

By default SourceTree will always pick origin to push a new branch to. Depending on your workflow, you may want to set another remote as your default, let’s say my-fork for instance.

SourceTree defaults pushing new branches to origin

Renaming a remote

Interestingly all we have to do is rename our remotes. Since SourceTree always picks origin we just need to rename to anything else. Perhaps to the name of owner of the repo like SproutSocial or FaceBook. It’s safe to rename the remotes, it’s just a nickname only used locally.

(I’ve never been a fan of “origin“, it’s terribly confusing and ambiguous for those learning git).

Steps to rename the remote

via git

git remote rename origin SproutSocial

In SourceTree

  • Open the repo settings (gear in top right)
  • Remotes tab
  • Click on the remote
  • Edit
  • Change the Remote Name

SourceTree Remotes Setting
Editing remote names

Changing SourceTree’s Default Remote

Once we’ve renamed origin, SourceTree will always pick the remote that comes first alphabetically. So, do the ‘ol classic prefixed by numbers. Here I’ll name them 01-my-fork and 02-origin.

Boom! SourceTree picks 01-my-fork by default!

Changing SourceTree's Default Remote
Setting your preferred remote as default

Simple workflow optimizations like this, make me happy.

A place for collecting my miscellaneous thoughts.