OHSHIT REPORT: Forum icons displayed as empty boxes on some browsers

shawn posted some pics said
15

When something goes terribly wrong here at Mediocre Laboratories we draft up internal memos called "OHSHIT REPORTS". We use OHSHIT REPORTS to help share details across the company about what went wrong, coordinate work across teams for how we're going to fix it, and dive deep into root causes.

I'm not going to do this all the time, but @dashcloud wanted an OHSHIT REPORT for a recent issue with our forum icons. It's going to be pretty technical, but let's give it a shot.

Background

There are a few different techniques you can use to serve a bunch of small images or icons on your site. The brute force way is to just serve a single image file for every icon you have. This approach works just fine for small sites but when you really start digging into performance optimizations you'll want to do something different.

Let's say you have 10 icons on your site and they're 1KB a piece. You need the browser to download 10KB total of images. The issue is that with each individual download there's a bit of overhead. If you can combine all 10 icons into one 10KB image, called a CSS Sprite, you only need the browser to download a single file instead of 10. Reducing the number of images being downloaded makes web sites load faster. Here's a good example of a few CSS Sprites you've probably encountered before:

Sprites aren't a new concept. Programmers used similar sprite techniques back in the 70s to optimize the performance of video games.

We were using the CSS Sprite technique for our forum icons. But CSS Sprites aren't a panacea. Two main issues made us change our mind about using CSS Sprites for our forum icons. First, the development workflow was too slow. We were recently working on a new forum feature and the addition of a new icon was taking as long, if not longer, than the feature itself. Second, it would be nice to have a solution where we could easily change the color and size of the icons without a ton of work.

If your icons are simple enough (don't contain multiple colors, complex effects, or gradients) there's another choice called icon fonts. I find icon fonts to be a better choice than CSS Sprites for our forum icons because the icons are small, simple vectors. By using an icon font we have all the advantages of CSS Sprites with a faster development workflow, the ability to easily tweak colors and sizes, and they look great on high DPI devices like the iPhone 6 Plus.

So, no @cranky1950, we weren't just fishing for web design dweeb validation.

Ok, we're switching from CSS Sprites to icon fonts. What happened?

Here's a bit more background. We don't host our static content (fonts, images, CSS, JavaScript) from the same servers that host the actual website. Instead we use a CDN (Content Distribution Network) that distributes these files to servers all over the world. When your browser needs to download a font or image or CSS stylesheet for our site it's connecting with a server that's nearby your location to optimize for performance.

We use Amazon CloudFront for our CDN:

When we were ready to make the switch from CSS Sprites to icon fonts we deployed the changes to our CDN.

In order for the CDN to work properly it needs to reside at a different domain name than meh.com so that the DNS lookup can resolve to an optimized server (based on location). The problem is certain browsers have a security feature called Cross-Origin Resource Sharing (CORS) that blocks loading of font resources from a different domain unless you set a special HTTP header called Access-Control-Allow-Origin.

We made an open source project called Electricity that helps us serve static content by following perfomance best practices. Electricity includes support for CDNs and the Access-Control-Allow-Origin HTTP header, but we had a bug that prevented the header from being set properly on font files.

So instead of our new font icon, some users were seeing empty boxes because the font file was being blocked by their browser.

It took us about an hour and 21 minutes to develop and deploy a fix but we eventually got those icons working.

tl;dr

We changed from CSS Sprites to icon fonts for our forum icons but had a bug setting the Access-Control-Allow-Origin HTTP header on font files from our CDN which resulted in CORS issues on some browsers.

P.S.

We have all sorts of tech jobs open that range from junior-level to senior-level. Looking for front-end developers and back-end developers and testers and system administrators and support engineers and business intelligence engineers and ux designers: https://meh.com/jobs