John’s Tech Blog

Excerpts from my blog, but just focused on tech and code.


Github Actions Status Checks with Heroku Pipelines

March 9, 2024

Lately I’ve started moving code projects away from CircleCI and just to use Github Actions natively within our repos. There’s absolutely nothing wrong with CircleCI! But, it’s still one more service to configure and pay for when we’re already paying for Github Actions along with our organizational account. So in an effort to tidy up a bit, we’ve switched over our CI pipelines to use Actions instead.

One of the issues I found while transitioning to Github Actions was with our Heroku Pipeline. With CircleCI, Heroku automatically had build status information on each commit. This way we could see exactly which builds passed or failed before we promoted the app to production.

I searched for solutions to this and couldn’t find anything that did what we needed, so I created my own little process.

If you’re using Heroku Pipelines and Github Actions, here’s how to get those little green checks back during your build phases.

February 26, 2024 at 7:54 AM

One of the aspects of personal websites that I love is the ability to have a little bit of creative expression and “features” that are just for fun. I’ve noticed a few personal blogs that have a ’now playing’ widget of sorts to showcase what someone is watching or listening to.

This would be simple to do with a Spotify embed widget, but I really don’t want any third-party scripts and widgets on my site. So I created a little now playing web component. This was fun. The code and instructions are published here.

🎵

Hotwire

December 23, 2020

Hotwire, a new framework from the folks at Basecamp and Hey.com was released this week. It’s short for HTML-over-the-wire (HOT-wire), which is very clever and also quite a novel concept these days.

Hotwire is an alternative approach to building modern web applications without using much JavaScript by sending HTML instead of JSON over the wire. This makes for fast first-load pages, keeps template rendering on the server, and allows for a simpler, more productive development experience in any programming language, without sacrificing any of the speed or responsiveness associated with a traditional single-page application.

Complete with a lovely and efficient screencast of DHH walking through the entire setup in a standard Rails app:

This looks very nice. Excited to give it a spin in the new year. It’s very similar to my own existing patterns, but with a bit more polish.

Stimulus 2.0 + Tailwind 2.0

December 10, 2020

Speaking of new stuff: I spent a few days this week upgrading my new project to use Stimulus 2.0 and Tailwind CSS 2.0, both out within the past few weeks.

I’m loving this new set up. I’ve used Stimulus a ton over the past few years, so that’s not particularly new to me. Although the new data values and classes APIs are pretty handy in 2.0.

But starting a project fresh in 2020 with all of the new good stuff, mostly Tailwind related, has been such a breath of fresh air. My app bundle is super small and focused on exactly what I need. I have a handful of generic Stimulus controllers that are used throughout the app to provide functionality. Nothing I’m doing there is limited to one particular context in the app: everything is reusable throughout.

There’s really never been a better time to develop for the web.

Rails 6.1 Released

December 10, 2020

Rails 6.1 has been officially released. Time keeps marching forward for this framework.

There are a couple of really nice features in here I’m looking forward to working with. I’ve been resisting using the new multi-db features for a while now and it might be perfect timing for me on Air Mail. Not that there’s anything wrong with the features, but mostly because I didn’t want to introduce extra complexity until it was absolutely necessary.

We’re currently storing a ton of user-related data (subscriptions, invoicing, preferences, usage, etc) and a totally separate set of editorial data (photos, articles, revision history, etc). I’m thinking it might be really nice to separate the two distinct functions out since they don’t need to do anything together. Then I can scale each piece of the pie independently. We’ll see.

Anyways, congrats to the Rails team. This is a great release.

How to GraphQL with Ruby, Rails, Active Record, and no N+1

November 12, 2020

Nice writeup about GraphQL in Rails by Dmitry Tsepelev at Evil Martians.

GraphQL can do wonders in a backend-only Rails application, giving your clients (whether a frontend framework or other API consumers) a single endpoint for fetching data in any shapes and sizes they might need.

As the list of the associations to load is always determined at the runtime, it is very hard to be smart about querying the database.

GraphQL seems like one of those things I’m going to need to get into eventually.

Tailwind

October 1, 2020

I'm working on a few new project ideas for the first time in a few years. When you work on the same thing for a long time you can become comfortable with just what you have. Well, I do at least. Comfort is good and it allows me to move fast. But every time I start a new project I try and play with something a bit new to get outside my comfort zone.

One of the tools I've been playing with has lately been Tailwind CSS. This one is quite a bit different than anything I've used for CSS before.

Traditionally, I don't like CSS frameworks. I find them overbearing, overly-opinionated, and bloated with tons of components and utilities that I never end up using. I've worked on countless projects with Bootstrap, Foundation and the like. They are great frameworks and they certainly have their benefits. But it's just too much for me.

A few years ago, when I started working on Air Mail, I just started from a basic CSS reset and added my own components as I went along. The front-end of the public site is very simple but the back-end content manager is much more complicated. There's a ton of CSS. All of it is custom, and all of it was tailor-made for the specific use cases I had. It works great, but it all seems a bit heavy. I've been trying to figure out a better way forward.

Tailwind takes a completely different approach. I've heard it described as an "API for your design system." That's clever. Instead of building custom CSS (or using a normal framework) Tailwind allows you to use utility classes to change the look of the markup. It's almost like old-school inline stylesbut with more more power and control. Controlling variants and breakpoints from just class names is pretty handy.

The Tailwind docs give a nice example of why the utility-first approach can be productive.

I like that it provides a basic design system in a box. There are sensible defaults, all that can be customized, but then the framework just provides a ton of utilities to get you what you need. It's refreshingly simple.

One of the things I always have disliked about other frameworks is that they force you to use their outdated, clunky, and one-size-fits-all JavaScript components. I don't want to be forced into including some old version of jQuery (or any version at all nowadays) just to use your basic CSS framework. Tailwind doesn't care what JavaScript you even use. It's just a simple CSS utility framework. The interactions are up to you.

Last but not least: to fix the bloating problem Tailwind is designed to be used with PurgeCSS to remove any code on build that your site doesn't need. This is really nice.

It's not all roses, there are going to be some hiccups. That’s a heck of a lot of classes floating around in my markup and it’s going to take me some time to get used to that. This approach is new to me, but I'm excited to give it a spin.

How to Spring Clean Your Heroku Slug Size

April 20, 2020

This week I was deploying a routine update to Heroku, but the app wouldn’t deploy. Here is the message that was displayed in the build log:

-----> Discovering process types
       Procfile declares types     -> release, sidekiq, web
       Default types for buildpack -> console, rake
-----> Compressing...
 !     Compiled slug size: 502.5M is too large (max is 500M).
 !     See: http://devcenter.heroku.com/articles/slug-size
 !     Push failed

Slug size is too large was the culprit. 502 MB! The app is decently sized but that seems way off to me. According to rails stats the app is about 58,243 lines of code. Not small, but certainly not huge. And these are mostly source code plain text files too so file size should not be an issue at this scale.

Investigating the Sizing

I used a basic du -sh * command to check the basic file sizes of the folders in the app’s repository to see what’s going on. Here’s the truncated output:

12M    app
352K    config
1.3M    db
376K    lib
332K    public
260M    node_modules
 78M    tmp
226M    vendor

So my app is relatively small, only about 17 MB. The rest of the file size comes from the various dependencies. By some standards, I don’t really even think this app has very many dependencies. There are a few dozen Ruby gems in the Gemfile and there’s nothing out of the ordinary here. Rails, Postgres, Nokogiri, etc. All things you would expect. (Although side note, the google-api-client gem is 61 MB on its own. Yikes, by far the largest gem in there. That’s something to work on for another day.) All of the gems are in vendor/ so the file size there checks out as reasonable.

The tmp directory is pretty big here too, but that’s ok. It is mostly filled with bootsnap caching files to make the app load quicker too. This would be an easy 78MB to clear out if it was the culprit, since I don’t really care how long the app takes to boot, but let’s reserve clearing it for later.

Even that massive node_modules directory isn’t really problem, since Heroku slugs are compressed before this slug size is calculated. Those dependencies compressed down are only about 80MB.

So there’s something up here. Even with my most conservative calculation the slug size should be max of around 300MB after compression. Which is still too big in my opinion for an app of this size, but still nowhere near the limit of 500 MB.

There is a nice Heroku plugin that allows you to download slugs from your builds. It’s a very handy way to download the exact slug from Heroku’s filestore on S3. I downloaded the slug in question to give it a look.

Using the same du command within the extracted slug files, there was one major diference:

961M    public

Whoa. 961MB in the downloaded and unzipped slug, but only 332K in my project’s repo itself. It turns out that inside the public/assets directory there were hundreds of asset build files dating back years. The app itself is a few years old and it looks like Heroku had stored the compiled assets in the slug for each release since the app was created. Yikes.

This is a known feature on Heroku and it is called the build cache. I was mistakenly under the impression that each app deploy would cause a new slug to be built, including the asset compilation. The build cache feature makes sense: why waste time re-downloading files and other dependencies on each build when they can be cached? But caching the output of asset creation until the end of time? That seems excessive. So let’s clean it up.

The Fix

It turns out there’s an easy fix when build cache is your problem. There’s another Heroku plugin for managing the repo that contains your app’s code. Within this plugin is the purge-cache utility, which clears the build cache.

Here’s how it works:

heroku plugins:install heroku-repo
heroku repo:purge_cache -a appname

I redeployed the app and that shaved off a few hundred MB from the slug size. We’re back in business here. But I still couldn’t get past that massive node_modules/ directory. That was 260MB of modules only used for compiling assets. Once the assets were compiled, those modules are no longer used. (This particular app does not use any node_modules at run time, they are all precompiled with webpack.)

There is a community-developed buildpack for Heroku called post-build-clean that gives us the ability to remove files from the slug after the build command finishes running. This is slightly different than the standard .slugignore file which won’t even allow you to use those files during the build phase. I installed the buildpack, added node_modules to the .slug-post-clean file and tried to deploy again.

This time around, it looks much more reasonable to me:

-----> POST BUILD CLEAN app detected
Removing directory node_modules/ from slug
-----> Discovering process types
       Procfile declares types     -> release, sidekiq, web
       Default types for buildpack -> console, rake
-----> Compressing...
       Done: 87.4M
-----> Launching...
 !     Release command declared: this new release will not be available until the command succeeds.
       Released v1019

For a medium-sized app 87 MB seems like a good size to me. The asset files continue to build up after the compilations step, but it’s going to be a while until I need to clear the cache again.

MonoLisa Font

March 21, 2020

MonoLisa is a new coding-focused font by Marcus Sterz:

As software developers, we always strive for better tools but rarely consider font as such. Yet we spend most of our days looking at screens reading and writing code. Using a wrong font can negatively impact our productivity and lead to bugs. MonoLisa was designed by professionals to improve developers’ productivity and reduce fatigue.

I’m still partial to Operator from Hoefler & Co, but MonoLisa looks very well done.