Matt Steele

My house alerts me when my cat climbs into the ceiling

2026-04-10T00:00:00Z

We have an orange cat who loves to climb into our basement's drop ceiling and wander around. It's especially adorable when she hangs out in an alcove and surveys her domain.

This does not happen on any predictable schedule, as per the whims of a cat. But I didn't want to miss out on any chances to see her perched up there.

So, I wired some cheap IoT devices together to notify us when she hops into the ceiling.

The workflow

The goal of the project was to turn on a light in the living room any time a cat is detected in the drop ceiling alcove.

To do this, I used a few Internet of Things devices I had on-hand:

A cheap Wi-Fi "security" camera (Wyze)
An even cheaper Wi-Fi RGB light (Merkury)

Out of the box these devices don't talk to each other, and are no longer supported by their manufacturers. And I wouldn't want to rely on various cloud vendors for a mission-critical workload like this, so this was a perfect job for Home Assistant.

I've used Home Assistant in the past for similarly dubious projects, so I had some familiarity with its capabilities. It's got fairly extensive automation capabilities, where a change in one device's state can trigger a workflow to modify other devices, so this seemed very much in its wheelhouse.

The devices

Integrating the light wasn't too much of a challenge, though it took some sleuthing to discover that the device used Tuya firmware, which had a straightforward integration. I was hoping to be able to modify it to not require connections to Tuya's cloud servers via a tool like tuya-cloudcutter, but this wasn't available as a predefined, easily hackable device, so it became a project for a later date.

Adding camera feeds was done by integrating the open-source Frigate NVR. As per Frigate's docs I set it up for object detection, for both people and cats:

ffmpeg:
  hwaccel_args: preset-vaapi

objects:
  track:
    - person
    - cat

detectors:
  ov_0:
    type: openvino
    device: GPU

cameras:
  wyze_ceiling:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://id:password@192.168.200.4/live
          roles:
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720

Frigate has a Home Assistant plugin that works quite well, connecting to the same MQTT broker I was already running for other IoT devices.

Hooking this into Home Assistant gave me an few data streams I can key off, like any other sensor.

Integration

Using the automation tools built into Home Assistant, I can toggle the light pretty easily:

Or in code:

alias: Cat Presence
description: ""
triggers:
  - type: occupied
    device_id: b86f8493d0516cb5f9afe35ad61effcf
    entity_id: 5a7f5d90cdaaf72fff39387fe4e377ab
    domain: binary_sensor
    trigger: device
conditions: []
actions:
  - action: light.turn_on
    metadata: {}
    target:
      entity_id: light.merkury_bw901_bulb
    data:
      brightness_pct: 100
mode: single

Overall the project went really well, and I'm still delighted when the orange light turns on while I'm hanging out in the living room. There are a few things I'd like to improve with the workflow, but it's primarily around my hosting and object detection (I'd like to move Frigate to a Raspberry Pi with an AI hat, so I can add more cameras and detect cats elsewhere)

As dumb as a project like this is, it still lends credence that integration of disparate devices can bring value greater than the sum of their parts. No one is going to offer a commercial "ecosystem" that would give me these capabilities, especially not with cheap consumer hardware that's been abandoned by the manufacturers.

The Year Of The Linux Desktop (for fitness games)

2026-01-25T00:00:00Z

A tale told in memes

I'm joining the bandwagon and declaring that 2026 will be the Year of the Linux Desktop. Specifically for the subset of fitness gaming apps, it's no longer necessary (or preferred) to be tethered to Windows.

Two long-term trends have led to this moment.

Windows has been on a "slowly, then all at once" descent in quality that really started ramping up last year.

Working with our client sysadmins on a daily basis, I get to see first-hand the OOB patches and hotfixes we've been scrambling to apply (including this weekend when Patch Tuesday broke core RDP functionality) and pray don't make the experience worse. Combined with a "Continuous Innovation" and "Controlled Feature Rollout" strategy that launches features when you least expect it, and it's made everything feel ephemeral and ready to break at any moment. When was the last time you got excited for an update, rather than dread it?

I don't think it's any surprise this corresponds with Microsoft's headfirst embrace of generative AI. Both internally (Satya claims 30% of Microsoft's code is AI-written) and externally (the "everything is Copilot" meme, the rebranding of Edge as an agentic browser), it's inescapable.

At the same time, desktop Linux has been steadily improving and growing more reliable. Some of this has to be spillover from investment in downstream OSes like ChromeOS, Android, and Steam OS, all based to some degree or other on Linux. Core peripherals like Bluetooth work across all my devices I've tried it on, on machines both designed to run Linux and on PCs with random hardware I didn't even check compatibility with before installing.

My Distro

One meme I've struggled with is the tendancy for Linux desktops to start off strong, but degrade in reliability over time.

Historically this feels like a problem I'm also culpable for. A common experience is for userland changes applied to make a Linux desktop functional would inevitably break some unrelated part of the system. Install a new Python 3 runtime for a project, and somehow you've broken your virtual desktop setup, because it relied on 2.7.

The critical drive lately has been the rise of atomic desktops. These distros are "immutable", in that core filesystems are read-only and cannot be modified, even as root. The OS (and supporting packages) are updated all-at-once, and can be rolled back to a previous release if anything goes wrong.

Almost all userland work is accomplished with high amounts of isolation. GUI apps are Flatpak, and CLIs use Homebrew, so they don't interfere with system packages. Want to setup a development environment? On Bluefin, you're pushed to use devcontainers, so everything happens inside an OCI (here Podman) container.

It's pretty incredible how far you can take this approach. Bazzite (a gaming-focused distro) works so well that it's very common to as a drop-in replacement on Steam Deck (replacing an Arch-based OS). I'm running it on my gaming desktop and spent more time fighting Windows UEFI bugs than I did getting a fully-working setup.

Bazzite ships with Steam and its Proton compatibility layer, but my games need to run outside Steam. How does that behave? Turns out, great!

Zwift

My favorite cycling-themed MMORPG doesn't have a native Linux port and probably never will. It also has quite a few Windows system dependencies (its launcher relies on an Edge-based WebView2), and of course it needs access to a GPU and peripherals like Bluetooth/ANT+. I've been trying to get Zwift running for years with tools like Wine/Lutris, but every attempt at getting it running failed, and I'd inevitably corrupt my environment installing incompatible, conflicting versions of Visual C++ Redistributables.

This has been almost completely fixed with Kim Eik's netbrain/zwift repo - an all-in-one Docker container that setups an isolated Wine environment, installs Zwift, and applies a known set of working patches. The result is a setup that's easier to run than Windows.

Not everything works yet. In particular, Bluetooth (needed to interact with the trainer, heart rate monitor, etc) isn't available, though there's ongoing development that I'm hopeful for.

In the meantime, I'm setting things up using alternate means. My trainer supports Wi-Fi connections, so no Bluetooth is needed for power, cadence, and resistance. I even set up the trainer on an isolated VLAN (as it's essentially an expensive IoT device I don't have full control over), and Podman was able to discover it after setting networking=host.

Other peripherals (controllers, HR monitors) still require Bluetooth, so I've just got an old Android device running Zwift Companion sitting on my trainer desk. Honestly this is the most "non-native" part of my entire setup, but I'm still using supported tools (and was having to use the Companion even on Windows after an update broke BLE connections).

Zwift runs at a clean 60fps, and I'm able to stream to Owncast via OBS just like before.

Beat Saber

This one I thought would be more challenging. As a VR rhythm game, I'd expect significant hurdles to a clean, performant experience.

My setup has always been a little unorthodox: on Windows I've been running a modded Beat Saber, installed using BSManager to setup extra songs/mods, and control which version I run. The game then streams to a Quest 2 headset, and I pray the latency gods are favorable. I spent the better part of last year getting this setup working well on Windows, and settled on tweaking Virtual Desktop to stream the game, limiting the resolution and frame rate along the way. Other tools like Oculus Link and Steam's own wireless streaming just weren't cutting it.

Turns out, Linux VR gaming is pretty solid these days. The LVRA community has been compiling tools and best practices for years, and many things are turnkey now. Need to stream to a Quest headset? Pick between ALVR and WiVRn. And again, both are installed as Flatpaks, so if you get it wrong, it's super easy to change course!

I went with WiVRn, as it bundles its own, Steam-independent OpenXR runtime, which has been supported by Beat Saber for years. And with a few small changes to the command-line arguments in Steam, I had vanilla Beat Saber running on my headset almost immediately. Interestingly, performance seemed to be even better on Linux than Windows; I was streaming at 120fps (on Windows I limited to 72) and my audio latency was down by nearly a third (~60ms compared to 90 on Windows). This isn't uncommon, quite a few benchmarks a game running on Linux can outperform its Windows counterpart.

Finally, I wanted to get modded Beat Saber running. Luckily BSManager provides a Linux flatpak that installed without issue, and I could download and mod versions just like on Windows. Getting it running on the headset proved to be a much bigger challenge. Out of the box, the WiVRn app list on the device wasn't detecting the modded Beat Saber, so I couldn't start it.

With some help from the BSManager Discord, I found the set of environment variables that needed set (within BSManager) to get it to launch. But the workflow was pretty horrendous: start WiVRn on the PC, then launch the client on the headset. Then take the headset off and launch Beat Saber on the PC. Then, put the headset back on and start playing.

It turns out there were a few bugs in both BSManager and WiVRn that prevented the app from being discovered, and launching properly. With some trial and error (and reading a bit more C++ than I was ready for), I was able to get it running. The details are in this GitHub Issue - and fixes have already landed in the WiVRn codebase.

This feels like a telling anecdote where the Linux gaming ecosystem is at: the fundamental components to play even complicated games are solid, and have been steadily improving. And when things don't work, there's a community to help address it. Hopefully more of this knowledge will get pulled out of private Discords and into public knowledge bases (or simply fixed at the root).

I can tinker with Linux when I need to cool down

I feel like I've just scratched the surface of what a Linux gaming PC can do. My library is pretty small, but I expect to grow it, especially with the Steam Frame inbound, offering another VR headset running Linux directly.

Of course there are other neat features you can try with a Linux machine with a decent GPU. Bluefin's AI strategy is one of local operation and control. So if you want to experiment with LLMs, you can install Ramalama, pull down a model from Huggingface you're comfortable with, and use your own GPU and resources. It feels like a breath of fresh air to have control of what's running on your machine, compared to the Windows experience of "what fresh hell with this restart bring today".

Hacking custom GIFs onto an LED Mask with mitmproxy

2025-12-01T00:00:00Z

This Halloween I went as Rorschach, the Watchmen 'hero'. I don't have much of a connection to the character; it was mostly an excuse to play with an LCD face mask I bought a few weeks earlier.

Since I was on a time crunch to implement the face mask in time for a Halloween party I was attending, I decided to go for the simplest approach: use the official Android app to upload my custom GIF onto the mask.

Fortunately, through my day job I've been introduced to a number of tools that help provide the shims necessary to make this happen, including:

mitmproxy to capture and inspect Bluetooth packets
apktool to decompile the "official" Android app to hunt for interesting strings
LightBlue to test sending Bluetooth commands to a device
Wireshark to evaluate sending Bluetooth packets

The actual code I had to write ended up being an extremely small mitmproxy add-on:

def response(flow):
    if flow.request.url.endswith('.gif'):
        # Brute force. Replace all .gif images the app downloads with my GIF
        img = open('rorschach-mask.gif', 'rb').read()
        flow.response.content = img

It was a fun exercise to get an animated GIF onto a device that clearly didn't want to be hacked like this!

Keep Your Running Cadence With a Garmin Music Playlist

2025-09-19T00:00:00Z

I've been running more this year, and one of the aspects I've been working on is running at a consistently higher cadence. Most experts recommend running at a pace of 170-180 strides per minute, but it can be rough maintaining that over time! And while some fitness devices have metronomes that can help you keep pace, I've found that listening to music with a matching tempo.

Christopher McDougall recommends the Rock Lobster approach - at 180 beats per minute, it's a suggested cadence for many runners. But my wife hates that song, my cadence is a little lower, and I certainly can't listen to it for an entire running session!

There are plenty of online playlists that target a specific running cadence, but I don't want to subscribe to a service just to keep a tempo! I've got plenty of MP3s I've collected over the years, why can't I use those?

Ahead of a half-marathon I had signed up for, I wrote a script that does just that. Give it a folder of music files, and it'll find all of them within a tempo range, and create a playlist for you to drop onto a compatible device. I use a Garmin watch, but it should work for other devices as well.

The script to generate cadence playlists is available here.

Measuring Tempo

I tried a few approaches to grab BPM data for my files. My first hope was to pull the data from an authoritative source such as MusicBrainz, but almost none of the songs in my library had a tagged tempo value.

I also tried bpm-tools, which is available on most Linux distributions. This pulled back a cadence metric, but when spot-checking a few files, I noticed the data was pretty inaccurate; about 50% of the files deviated significantly from the calculated tempo. This was made worse by the lack of a confidence metric, so I couldn't apply a low-pass filter to remove obvious errors.

I ended up using the deeprhythm library, which uses a convolutional neural network to detect tempo. It performed really well, both to quickly detect thousands of songs, and also run accurately. After spot-checking a few songs, I decided it was accurate enough for my usage, and wrote up a script to process my library.

{"tempo": 91.0, "confidence": 0.65030837059021, "file": "/mnt/data/music/beets-library/Carly Rae Jepsen/E•MO•TION/12 - When I Needed You.mp3"}
{"tempo": 111.0, "confidence": 0.7165337204933167, "file": "/mnt/data/music/beets-library/Carly Rae Jepsen/E•MO•TION/06 - Boy Problems.mp3"}
{"tempo": 145.0, "confidence": 0.6137411594390869, "file": "/mnt/data/music/beets-library/Carly Rae Jepsen/E•MO•TION/15 - Favourite Colour.mp3"}
{"tempo": 112.0, "confidence": 0.9051607251167297, "file": "/mnt/data/music/beets-library/Carly Rae Jepsen/E•MO•TION/13 - Black Heart.mp3"}
{"tempo": 118.0, "confidence": 0.7597530484199524, "file": "/mnt/data/music/beets-library/Carly Rae Jepsen/E•MO•TION/01 - Run Away With Me.mp3"}

Garmin watches can load a playlist in M3U format, so the script builds a simple playlist, and consolidates everything into a folder you can drag & drop onto a device.

0:/MUSIC/cadence/03 - Star Quality.m4a
0:/MUSIC/cadence/04 - Lucifer's Jigsaw.mp3
0:/MUSIC/cadence/03 - Brothaz.m4a
0:/MUSIC/cadence/02 - Such Great Heights.m4a
0:/MUSIC/cadence/11 - Paper Lanterns.mp3
0:/MUSIC/cadence/DSP - Le Weeknd de Nemo.mp3
0:/MUSIC/cadence/28 - The Room Where It Happens.m4a

uv is neat

I've struggled running Python apps in the past, especially if they pulled in dependencies. As chaotic as the JavaScript ecosystem is, using a library is (usually) as easy as an npm install. With Python, I've never been able to make heads nor tails of pip, pipx, requirements.txt, poetry, virtualenv, and the other tools in the ecosystem.

A coworker pointed me toward Astral's uv, which claims to be the One Tool needed to manage the chaos. And at least for this case, it works great!

I especially like its support for inline dependencies (PEP 723), which lets me specify all my project's dependencies with the script itself:

#!/usr/bin/env python3

# /// script
# dependencies = [
#   "click"
#   "deeprhythm"
# ]
# ///

import click
from deeprhythm import DeepRhythmPredictor
import os
# ...

With this, I can simply uv run <script>.py, and it'll create a virtual environment, download required dependencies (or use a local cache), and execute the script.

matt@ORTHO ~/mus> uv run bpm.py --dir /mnt/c/Users/Matt/Music/
⠹ Preparing packages... (38/53)
⠦ Preparing packages... (43/53)
nvidia-cuda-nvrtc-cu12   ------------------------------ 70.92 MiB/83.96 MiB
triton                   ------------------------------ 71.99 MiB/148.33 MiB
nvidia-cufft-cu12        ------------------------------ 71.36 MiB/184.17 MiB
nvidia-cusolver-cu12     ------------------------------ 71.01 MiB/255.11 MiB
nvidia-cusparselt-cu12   ------------------------------ 71.12 MiB/273.89 MiB
nvidia-cusparse-cu12     ------------------------------ 70.68 MiB/274.86 MiB
nvidia-nccl-cu12         ------------------------------ 70.77 MiB/307.43 MiB
nvidia-cublas-cu12       ------------------------------ 70.95 MiB/566.81 MiB
nvidia-cudnn-cu12        ------------------------------ 70.77 MiB/674.02 MiB
torch                    ------------------------------ 72.24 MiB/846.92 MiB

It does feel a little ridiculous pulling down PyTorch and a gigabyte of dependencies for a hundred-line shell script, but at least it's fast and easy!

uv also lets you execute scripts from a URL, so you can run this fom a command-line and begin processing:

uv run https://gist.githubusercontent.com/mattdsteele/082fd77c3e65faa1332a36962c11da78/raw/bbe07ad1fe8737153a49362cbe70f91d51a75fb8/cadence-playlist.py

Maybe tempo isn't the only factor I should consider

I had a fun time with the playlist, and was able to keep a decent cadence going. You can even see where a song change with a significant delta occurred.

With nothing but tempo to drive the playlist, I got a real eclectic mix in my ears. Some of the artists I experienced on the run:

Rancid (ok!)
Audioslave (nice)
The Lonely Island (uh..)
Hamilton soundtrack (still slaps!)
Jack Johnson (..I can't believe it captured BPM for an acoustic guitar)
Death Cab for Cutie (maybe not the right context to feel emo)
Brother Ali (..but this came at the right time)
Norah Jones (oh boy)
Jamiroquai (this made me virtually insane)

So yeah, maybe I should have looked through the songs in the playlist first. Or only pull in certain genres by querying beets (a very cool tool, but probably a whole other post of its own).

It might not have been the ideal soundtrack for a PR (my actual finishing time was pretty middling), but it at least wasn't as bad as this playlist.

Thank Java for Better JavaScript Dates

2025-04-15T00:00:00Z

I gave a lightning talk at NebraskaJS about the complex, decades-long relationship between JavaScript and Java's Date APIs:

The first half was a bit of a troll; I used a Java REPL (anonymized to look like Node) to demonstrate some of the more famously unexpected behavior in the Date class:

var date = new Date("March 20 2025");
date.getMonth(); // returns 2
date.getDay(); // returns 3
date.getYear(); // returns 125??

This code is syntactically correct Java and JavaScript, and behaves identically in both languages. One of the gifts of being told to "make it like Java" when adding dates to a ten-day-old language.

Java, of course, got markedly improved Date/Time APIs in version 8. Spearheaded by the lead developer of the de facto third-party date/time library, the java.time APIs were a breath of fresh air, and the new standard tool for all time-related matters.

JavaScript has been following the same path, with Maggie Pint (one of the maintainers of Moment.js) working to standardize the Temporal API. It's been a long journey, with it reaching Stage 3 of the TC39 process way back in 2021, but hasn't made its way to a browser engine just yet.

But it's finally making its way into browsers! Safari, Firefox, and Deno have it in preview release, with Chromium browsers in active development.

So it's a great time to refamiliarize yourself with the API, start checking out the polyfill, and start building fun stuff, like my gravel racing time estimator.

This blog's comments are powered by Webmentions

2025-01-19T00:00:00Z

Webmentions are a cool, standards-compliant mechanism to let your corner of the Web interact with other sites. What better way to add a comments system to my new Eleventy-powered blog?

At its core, Webmentions are just a means for one site to notify another that it's been mentioned. That said, webmentions aren't the most intuitive concept to grasp, and implementing them requires additional consideration.

Collecting mentions

Webmentions are sent via simple HTTP message with a source and target parameter. These are discovered based on a <link> tag on your page. You can host this endpoint yourself, but I'm using the excellent Webmention.io to capture all webmentions for my domain:

<link rel="webmention" webc:keep href="https://webmention.io/steele.blue/webmention" />

Webmention.io provides an API that returns all the mentions for a page (or domain). I hit this at build-time, and create a new Eleventy collection with all the entries:

const url = `https://webmention.io/api/mentions.jf2?token=${apiKey}&per-page=1000`;

export default async () => {
  const data = await EleventyFetch(url, { duration: '1d', type: 'json' });
  const grouped = groupBy(data.children, 'wm-target');
  const baseData = {};
  for (const key of Object.keys(grouped)) {
    baseData[content(key)] = grouped[key];
  }
  return baseData;
};

Displaying Webmentions

On each page, I include a Liquid component that finds any Webmentions for the URL, and renders each category (likes/reposts/comments):

This felt like the perfect place for a WebC-based component, but I ran into issues as I needed to pass in complex data as props, which isn't supported. Luckily WebC lets you mix in other templating languages, again showcasing its flexibility and pragmatism.

Since this only gets generated at build-time, it doesn't capture new mentions until I rebuild the site, which is fine for me. If you want more dynamism; you could setup a cron job to rebuild their site daily to pull in new mentions.

Webmention.io also supports outgoing webhooks, so you could directly trigger a rebuild when a new mention comes in.

And some people just pull in webmentions by fetching them client-side, and rendering interactions via JavaScript. Seia looks like a pretty straightforward Web Component providing drop-in support with Webmention.io.

Bridging from the Fediverse

Surprisingly, not 100% of online communication occurs via sites sending webmentions to each other. We'd also like to capture comments occurring on Mastodon (or other channels), even if they don't natively support webmentions. In IndieWeb parlance, this is known as a backfeed.

I'm using Ryan Barrett's Bridgy service for this. You can authorize Bridgy to poll your Mastodon account (or social networks) to discover likes/reposts/replies to your toots, and send them back to your site as Webmentions.

In addition, the Bridgy Cinematic Universe supports a number of other features, including posting to Mastodon when you publish new blog entries. And Bridgy Fed lets you create Mastodon/Bluesky/Indieweb accounts from any other permutation; a great way to implement POSSE.

Sending Outgoing Webmentions

As I'm writing posts, I want to send Webmentions to other sites which I reference. This requires a different set of tools, but again the Indieweb has provided a solid foundation to build atop.

I'm using Remy Sharp's webmention CLI tool, which parses my existing Atom feed, finds all links which support mentions, and delivers a webmention to each.

As noted in his introductory post:

The ability to send Webmentions needs to be a part of an automated workflow - the same way as posting a new WordPress blog post automatically sent pingbacks.

I use a GitHub Action to trigger this, via a single line addition to my existing build:

- run: npx webmention _site/atom.xml --limit 1 --send

You can create a more robust interactivity by enhancing your page's markup with microformats, enabling comments and more.

TMTOWTDI

One exciting facet to integrating Webmentions into your workflow is that there's no one best way to do it. You can use existing tools and services to collect and deliver mentions, or customize them to your own liking.

This does make for a more complicated, "choose your own adventure" aspect to the process, but I find this to be a feature, not a bug. Mixing prebuilt tools with my own code has given me a better understanding of what works well and what I could improve on in the future. I can optimize for certain aspects (like zero-runtime JavaScript) when they tradeoff with others (real-time updates), while using the same protocol as someone who would make a different choice.

I found these sites valuable:

Keith Grant's Adding webmentions to a static site
Bob Monsour's Adding webmentions to my site
Robb Knight's Adding Webmentions to your Site
Paul Kinlan's Using Web Mentions in a static site
Max Böck's Using Webmentions in Eleventy
Sia Karamalegos An In-Depth Tutorial of Webmentions + Eleventy

Send me a Mention

So how do you actually comment on this post? You've got a few options:

Reply to me on the Fediverse, and your post will show up here.

Or even better, write a blog post of your own and send me an outgoing webmention. Websites: the cool and underground social network that you shouldn't sleep on in 2025.

From Gatsby to Eleventy

2024-11-25T00:00:00Z

I try not to rebuild my site too frequently, as it's one of the classic blunders, alongside getting involved in a land war in Asia. But the Six-Year Itch is real, and it was time for a change.

The (Not So Great) Gatsby

My site has been powered by Gatsby since Dustin Schau (former coworker and Gatsby emeritus) ported it over as an early Christmas gift in 2018. I'm not a React guy, but it was easy to work with, and I could see the appear of a framework built around collecting arbitrary sources, and unifying them into a single data structure queryable by GraphQL.

Unfortunately, Gatsby isn't in the healthiest state these days. It wasn't able to achieve the liftoff that's expected from venture-funded frameworks. As it's fallen out of favor in the past few years, the OSS side of the framework has continued to languish since its acquisition by Netlify. Lately, I've spent more time fighting with dependencies than I do blogging, and it's felt more like a mill around my neck than a productive authoring tool.

Turning it up to Eleventy

I wanted to switch to something that's built for the long haul, and Eleventy fits the charge.

Created by Nebraskan (and former coworker) Zach Leatherman, it's a static site generator with a focus on simplicity, sustainability, and not getting in your way. It's heavily inspired by Jekyll, which was how my site was built ten years ago, so it feels like I've come full circle.

In addition to its familiarity as a tool, I appreciate what the Eleventy project prioritizes. Rather than a laundry list of features mean to check boxes, Zach advertises how stable its API is and how easy upgrades are, with major versions focused on removing dependencies and modernizing the codebase. I trust Zach when he says he doesn't tie Eleventy to a bundler, because he wants it to outlive the current bundlers.

Zach has also been at the vanguard of working through sustainability models for Eleventy, as it's been maintained as an OSS passion project, to one funded by VC-backed companies, to a crowdfunded model. It's currently found a home at Font Awesome, which feels like it's finally aligned from its values, and has a history of stewarding open source projects such as Shoelace/Web Awesome.

With the recent 3.0 release of Eleventy (now powered by ES Modules), it felt like a great time to start fresh.

From WebC to Shining WebC

I didn't want to lose out on the component model when moving off Gatsby. I've struggled with the template-first model that Jekyll, Hugo, and other SSGs use when it's not driven by a framework. And while I really like Web Components, it didn't feel like the right tool for a primarily static site, without building my own server-side rendering framework along the way.

WebC is a new authoring format, loosely coupled with Eleventy, and hit the sweet spot. It's got a similar feel to Single File Component models popularized by Vue and Astro, but it outputs markup akin to Web Components. Using WebC, I was able to rebuild most of the Gatsby components, and not feel like I was spaghettifying my codebase.

I'm using "Web Components" loosely here, as the markup is highly configurable. Most of the components simply get converted into the HTML they encapsulate, and are never generated client-side or extend HTMLElement. But if you want to render components as "real" Custom Elements with Declarative Shadow DOM, you can do so!

I'm hoping WebC sees a life beyond Eleventy projects, because it feel like a utilitarian tool designed for real problems. As it stands, this is probably the part of the codebase that's most tied to Eleventy, but if I have to rebuild the dozen-ish .webc files at some point, it shouldn't be too difficult to port to a 'vanilla' Custom Element, or whatever comes next.

Four Hundo Blaze It

It's not all conceptual benefits, one immediate advantage I see are improved build times (production builds are 2-3x faster, and the dev server starts immediately, rather than after ~30 seconds).

I've done little to optimize the site. Yet without much effort, I'm awful close to the coveted Lighthouse Four Hundos:

This is in stark contract to my Gatsby blog; I had to do quite a bit of work to make it perform like I wanted, including applying a plugin to delete all the client-side JavaScript used to hydrate components that should have been static anyway.

The site is fast by default, and the URIs remain Cool. A few features are missing from the Gatsby era (namely, custom OpenGraph images), but I feel like I've got a solid foundation to iterate on.

It feels good, like I'm working with the grain of the Web, rather than against it. The codebase is at https://github.com/mattdsteele/steele.blue. Here's to the next six years (and beyond).

Side projects should be fun

2024-04-28T00:00:00Z

Connell McCarthy's post about "overengineering everything at his wedding" was a fun read: https://connellmccarthy.com/article/wedding/

I can relate to “this wedding would be a great opportunity to build everything bespoke”, as I:

live-streamed our wedding on an alpha build of Owncast
built a custom photo booth from a Raspberry Pi
built our website and RSVP system with a Firebase and Airtable backend (probably should have blogged about that too!)
crafted, painted, and stained the welcome signage

By most objective measures, this was a colossal waste of time.

Some advice I received after getting engaged was to accept that you won't be able to maintain focus and attention on every aspect of the wedding. Trying to "make your mark" on decorations, seating order, invitation typography, etc. was a recipe for spreading yourself too thin, and doing none of them well. A more prudent approach is to drill into a few things you really care about, and make those great.

Most aspects of the wedding-industrial-complex have been fully commodified and are an easy thing to purchase and check off your to-do list. Even if you don't want to use a wedding-branded website builder like The Knot, you could also knock it out in an afternoon with Squarespace or one of a thousand WordPress templates.

So why build your own, less efficiently? One potential reason might be that the scope of your event dropped by an order of magnitude, due to having scheduled your wedding during a global pandemic. With essentially no in-person attendees, we were free to focus on the purely digital aspects.

Another reason: we both like to program, and it seemed like this would be a fun way to spawn a bunch of side projects.

Some of my favorite memories of the early part of the pandemic were pairing with my fiance on building a simple carousel with silly photos of us. Or testing the live stream software by using it to host a bad movie night with friends. Or learning just enough Python to make a custom NeoPixel ring for the bespoke photo booth.

I was reminded of these fun projects after seeing this post from Foone:

the world needs more recreational programming. like, was this the most optimal or elegant way to code this?

no, but it was the most fun to write.

There's plenty of reasons to work on a side project: maybe you want to learn something new to make yourself more marketable. Or perhaps it's a ritual you do each year to see how the ecosystem has changed. But my favorite kinds of projects are the ones that are just for the fun of it.

Mutation Testing with StrykerJS

2024-04-21T00:00:00Z

Jessica Codr and I gave a talk at NebraskaJS about StrykerJS:

Stryker is a mutation testing tool - it makes a change to your code, then runs your tests. If everything passes, then your tests aren't catching bugs you could be introducing over time. Think of it as code coverage, with specific areas to focus on.

In some ways it's similar to fuzzing: by making large number of pseudo-random changes to a codebase, you can start to get a feel for which parts are solid, and which could use more attention.

As a means to gauge the quality of your tests, I'm not sure why so many folks terminate their assessment using code coverage when mutation testing is a more robust metric. There's some additional overhead (most frameworks don't incorporate mutation testing out of the box) and the test runs are slower, but you can let your CI job run it async (or setup a nightly job) and you'll get far more robust metrics.

I've had some experience with testing Java code with PIT, and Ruby has had mutant for a while, but Stryker was a new tool to me, even though it's been around for nearly a decade. I'm hoping it becomes a useful tool in the belt!

My favorite water bottles

2024-03-24T00:00:00Z

Inspired by "my favorite mugs" posts from Cassey and Benji, here's a few of my favorite drinkware. While I do have a set of mugs I love, I probably get more use out of my water bottles. They pull double-duty as a convenient vessel for daily drinking, and for any activities on a bike.

I've raced Gravel Worlds a half dozen times, and get a new bottle every year. It's both functional (at 24oz, it's a decent size to throw on a bike) and stylish, and a mainstay of my bottle collection.

Liter-sized bottles are a mainstay of my gravel races. The extra few ounces have kept me from having to carry a hydration pack, and let me make full use of the two bottle cages on my bike. I especially like these ones from Elite with a protective cap, which helps keep the lid free from gravel dust. I've eaten enough dirt through the years!

Runner-up: the Zefal Magnum 1L bottles have traveled the most miles on my bike.

This one's substantially smaller and doesn't actually get on the bike frequently, but it makes me smile every time I break it out. I don't buy many products from meme influencers, but I make an exception for an especially good Simpsons reference. It makes me want to conquer the Murderhorn, even when I fill it with apple cores and Chinese newspapers.

Postscript - Bottle Longevity

One endless conversation that I've never heard a satisfactory answer is bottle longevity. Advice is all over the place, from "replace your bottles every season" to "use them until mold is visible and unremovable".

I tend to be pragmatic; I'll give them a good cleaning with soap and a bottle brush, and they'll be good for years. If I start to notice a funk/aftertaste, they'll get replaced. This is probably more psychological than scientific; most of the advocacy I've seen around microplastics, etc. tend to be from the same groups that are focused on trends and style, not sustainability.

Web Push is almost usable with iOS 17

2023-09-21T00:00:00Z

Apple got a lot of positive sentiment earlier this year when they launched betas of iOS 16.4 and announced support for Web Push, along with other features. At WWDC they declared "As long as you've coded to the standards and use feature detection, so you don't unwittingly exclude Safari, your users will already get the benefit of Web Push".

Having attempted to implement Web Push in a recent project I can attest that the reality was pretty different. In addition to a number of buggy and inconsistent runtime errors, there were several user-hostile limitations that effectively excluded it from being used by anyone but the most dedicated users.

As iOS 17 rolls out, we're seeing forward progress. Several of the restrictions have been removed, and the behavior appears to be more consistent, with APIs that can be feature-detected as one would expect.

No longer requires Feature Flags/Experimental Features

The big issue I encountered when trying to support Web Push was that the APIs were disabled by default in iOS 16. It was an "available" option, but tucked behind the Settings -> Safari -> Advanced -> Experimental Features setting, alongside dozens of other toggles. Functionally this killed Web Push's adoption, as you couldn't reasonably expect a general audience to modify their feature flags to support this.

The good news is that as of iOS 17, this is no longer required, and the relevant APIs have been enabled by default. There is still an option for "Notifications" in the (now renamed) Feature Flags list, but it doesn't seem to be required to enable support.

Add To Homescreen still required, still not promoted

Another user-facing limitation: only webapps which have been added to (and launched from) the Home Screen have all the APIs needed to enable Web Push. This is clearly an intentional decision by Apple to prevent abuse of a powerful feature, but it also has a similarly limiting effect of the usability of the feature.

As Maximiliano Firtman notes:

Users will never know that adding that web app to the home screen will give it more power. Apple doesn't support app banners, an installation API, or a menu term change (such as "Install"), maintaining a classic Apple decision to hide PWAs from the user while still technically available.

This requirement still exists. Given my anecdotal observation that I've never seen a non-developer iOS users add a website to their home screen, it'll take pretty dedicated in-app tutorials to guide users.

You can now (mostly) feature-detect support

One big hassle with Web Push on iOS 16 was that APIs would be available, but weren't effectively implemented when the limitations above were in place. So if you were trying to check for the existence of particular APIs before working with them, you'd only see errors at runtime for an app that wasn't added to the home screen, for example.

Luckily, this appears to have been cleared up. These combinations of checks should be sufficient:

// Service Workers supported
if (!'serviceWorker' in navigator) {
  return;
}

// Web Push supported
const registration = await navigator.serviceWorker.ready;
if (!'pushManager' in registration) {
  return;
}

A site with an installed Service Worker which is installed on the home screen will consistently pass these checks. Sites which are running in "standard" Safari won't pass the second check, and you can bail.

If you're looking to give users some guidance that the feature is available, you'll need to add some additional checks in your code; as navigator.serviceWorker will exist, but not registration.pushManager. But, since other browsers support Service Workers and not Web Push, I wasn't able to find a clean way to feature detect this, and resorted to browser sniffing, so I could show the help context.

It's workable, for some definition of workable

Ultimately I ended up using this for my gravel bike tracking website, in order to send updates to folks watching at home as I made it to each stop, which I had bounded with a geofence. I also setup a push notification to go to my phone when it "successfully" ordered a pizza. And while the push notification made it to my phone, the pizza did not ultimately get ordered.

So maybe there's the lesson: with enough dedication and grit, you can get push notifications to show up on iOS devices, and it's a little easier with iOS 17. And depending on what you're trying to integrate with, it may not even be the most brittle link in the chain anymore.

I wired up my bike's GPS to order me pizza during a gravel race 🍕

2023-09-10T00:00:00Z

As harvest season begins here in the Midwest, I once again celebrate by grinding Nebraska gravel at the Gravel Worlds Long Voyage bike race. As in previous years (2021) (2022), I spent more time writing code for a marginally-useful project than I did training. But hey, I finished!

Fueled by Pizza

My goal this year involved optimizing my food choices during the race: pizza from Casey's General Store. These convenience stores are S-Tier options when out in the middle of nowhere. In addition to snacks and drinks, most Casey's have a kitchen, and have pretty decent grab 'n go slices of pizza.

My problem: last year, there were so many faster riders ahead of me, that all the pizza was taken by the time I made it to the stops. This is an outrage!

This year, I knew I had to do better. But with time winding down on improving my fitness, I had to resort to the latent superpower of software hackery. My thought: why not have a fresh pizza ordered ahead of time, scheduled precisely for when I arrived?

More precisely, I could write a script that ordered a pizza for me, GPS-triggered by my bike leaving a geofence about an hour from the stop. Building this on top of the serverless GPS tracker I made last year should fit into the architecture pretty well.

Casey's Pizza API When

The overall design: I setup a geofence within the AWS Location service, which was monitoring my GPS tracker. When my tracker exited the geofence, it would trigger a Lambda function that calculates an ETA for my next stop, and orders the pizza.

The problem: Casey's doesn't have a public API for online ordering. So I had to resort to alternate approaches. More specifically, I fell back to screen scraping the website, everyone's favorite hack. I've had to screen-scrape for other projects (such as updating privacy on Strava activities), but since Casey's website was a complex React app that rendered everything on the client, I had to use a more powerful scraper; this time powered by Playwright.

Getting Playwright to run in a Lambda was An Experience. Full writeup here.

To keep track of the status, I also setup a push notification to be delivered to my phone (and watch) on a success or failure. Configuring Web Push to work on iOS devices is probably worthy of its own post.

Pizza False Positive

I had the triggering geofence configured around mile 180 of the race, with the pizza setup to be delivered at mile 200.

As I left the geofence, I got a push notification on my watch saying that the pizza had been successfully ordered.

But when I made it to the stop, there was nothing at the counter, and they had no record of an order. And sure enough, I checked my account, and no order had been placed. False positive.

There were a few pre-made slices available, so I picked those up. They left a bitter taste in my mouth, not just because they weren't especially fresh. Through the rest of the 300-mile race, all I could think about was what might have went wrong with my function.

After finishing the race, I made it to a computer and quickly checked the logs to see what had gone wrong. But to my chagrin, there was nothing in the logs indicating what caused the failure; it was just a silent success. I had nothing to go on to try and debug.

A few days later, I enhanced the Lambda to capture a video of the browser in action and upload it to an S3 bucket for analysis. Running a test of the updated behavior, it finally worked. I picked up a fresh Hawaiian pizza, and we enjoyed the pie from the comfort of our home.

I'm still not entirely sure why it worked. My going theory is that the Lambda had terminated processing as soon as the final form.submit() went through in the embedded Playwright browser. It's quite possible that the online ordering website saw that the browser never received a Success response, and didn't fully process the order. My guess is that the additional time spent processing and uploading the video gave the browser sufficient time to clean up properly, with serendipity in timing.

Pizza Lessons

While I was bummed the pizza ordering functionality didn't run, I think it's in a good spot to try again in upcoming races. I also learned a lot while building this out:

Consistently screen-scraping a React client-side app with a browser running in the cloud: possible, but boy howdy is it finicky. If I were to redo project, I may opt for reverse engineering one of their native apps, focusing on triggering their APIs directly
Having a good workflow to simulate geospatial behavior is necessary. At first, I setup a geofence around my house for testing, but having to leave for a walk at 11pm gets pretty old, pretty fast
That said, don't skimp on real-world testing. Prior to the race, I was mostly testing the function by running it on my local workstation, not in a Lambda. The successes of the local functions gave me false confidence that everything was working as expected
If you're worried about running out of Casey's pizza during a gravel race, another option is to just let the groups ahead of you get more than 30 minutes ahead, so they have plenty of time to make more slices

The code is available on GitHub: https://github.com/mattdsteele/spot-tracker-tracker/tree/main/pizza-function

Running a Playwright script on AWS Lambda

2023-09-09T00:00:00Z

I had a hell of a time getting a Playwright script to successfully run in an AWS Lambda. There's a few guides on various ways to handle this, but none of them worked out of the box. For posterity, here's what worked for me.

Use a Docker Container

The two primary ways I've seen Playwright and Puppeteer implemented are:

Using the default Node runtime, with the Chromium browser installed as a separate layer. This is the approach taken by the chromium-lambda npm package.
Using a Lambda Docker runtime, building off a Microsoft-provided base layer with Chromium pre-installed.

I was never able to make the Node runtime work, but did have success with the Docker runtime, so I'd recommend using that architecture.

Most of this started from Lari Haataja's excellent blog post on the various options and configuration needed to get started.

Write a standard Playwright script

You can follow the basic guidelines in the Playwright docs to launch your script.

I found it was helpful to move the Playwright code to a shared file, and have an entry point import it to run locally, and a second file with the Lambda specifics:

/* shared.ts contains Playwright logic */
import { LaunchOptions } from 'playwright';

export async function main(launchOptions: LaunchOptions): Promise<Result> {
  // launches the function
}

/* start-local.ts runs in "headful" mode */
import { main } from './shared';
const result = await main({ headless: false });

/* lambda.ts runs in "headless" mode */
import { APIGatewayProxyResult, EventBridgeEvent, Handler } from 'aws-lambda';
import { main } from './shared';

let args = [
  '--autoplay-policy=user-gesture-required',
  '--disable-background-networking',
  // More flags configured, see below
];

export const handler: Handler = async (
  event: EventBridgeEvent<'Location Geofence Event', GeofenceType>,
  context
): Promise<APIGatewayProxyResult> => {
  const results = await main({
    args,
    headless: true,
  });
};

You'll notice a number of args I passed into the Lambda - these are Chromium flags I found I had to enable to have Chromium launch successfully on the Lambda. You can see a full list of them in my repo, which I found from Vikash Loomba's GitHub.

Dockerfile

Microsoft provides Docker base images with browsers and dependencies already installed; so this will be the starting point for our containers.

Mostly this is following the guide from Lari, but I did have to pin down the specific version of the base layer so it corresponded to the Playwright version I was using (1.37.0), so my base layer looked like:

FROM mcr.microsoft.com/playwright:v1.37.0-focal as build-image

The container definition also bundles AWS's Runtime Interface Emulator, which supports starting up the container locally, using Lambda's APIs. I ended up not using this very often since I had a local, non-containerized entry point I could quickly run, but it was still a nice feature to have.

You can see the complete Dockerfile here.

There's also an entrypoint bash script which is mostly boilerplate, and can be seen here.

Build and deploy the container

To deploy to AWS, you'll need to build the container, and deploy it to a container registry in your AWS tenant. Most of this is boilerplate with the AWS CLI, and is documented on Lari's post.

I found that I was running the build/deploy frequently enough that I added it to a shell script, which you can see here.

On my (weak) machine it took about 5 minutes to build and push the container. Once available, you can create a new Lambda function, and define it as a Container Image as documented on AWS's page, and deploy using the latest tag of your container.

I had to make a few other changes to the container's configuration:

Increase memory - The default memory allocated wasn't high enough to launch and run Chromium, so I upped the memory at runtime to 3GB. You may be able to optimize this better to save some money.
Increase timeout - My script took about 2 minutes to finish running, so I had to increase the timeout so Lambda wouldn't kill it after a few seconds.
Update HOME environment variable When Chromium launches, it tries to put files in a subdirectory of $HOME, which isn't writable by default in a Lambda runtime. To get around this, you can set HOME=/tmp in the environment variables configuration. Thanks to this GitHub issue for the workaround.

Once it's all configured, you can treat this like any other function; and define triggering events using the AWS console, or whatever method you prefer. You can also run test events through to validate the page is invoking successfully.

Why go through all this? Would jsdom/WebDriver/Puppeteer be easier?

Having to start up an entire web browser to screen-scrape a site without an API is a giant pain, and I wouldn't recommend it as the primary approach to automating an action. But some websites don't have an API, and depend heavily on JavaScript to render their site (this was a React SPA), so running a full browser was the only choice I could rely on.

There's a plethora of other automation tools out there, but I wanted to try out Playwright, and found their API surprisingly usable for humans. I'd recommend this JS Party episode for a deeper introduction to Playwright.

Bonus: Record and review video playbacks

I found it difficult to debug what was actually happening when my function would run. I could add additional console.log statements, but I found it was just easier to save off a recording of the automation run and upload it to an S3 bucket for later review.

This was fairly straightforward with Playwright's video recording API:

import { readFile } from 'fs/promises';
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';

const S3_BUCKET = 'your-bucket-id';

async function uploadVideo(pathToVideo: string) {
  const Key = basename(pathToVideo);
  const Body = await readFile(pathToVideo);
  const client = new S3Client();
  const upload = new PutObjectCommand({
    Bucket: S3_BUCKET,
    Body,
    Key
  });
  const result = await client.send(upload);
}

And with that, for each run you have a full playback video to inspect!

Overly Complicate Starting Your Vehicle, Using Home Assistant, an ESP8266, and a Spare Car Fob

2023-07-02T00:00:00Z

I recently bought a new vehicle (the now-discontinued Chevy Bolt), it's great! Coming from a 14 year old car, everything about it feels space-age.

This is the first vehicle I've owned that has a remote start button on the keyfob, which I'm fascinated by. It also integrated with a mobile app, which enabled locking & remote start from a phone, but requires a monthly subscription.

I wanted to see if I could build something similar with "standard" IoT and home automation tools. It would only work when the car is at home, but it would accomplish my goal of letting me remote start my car on a cold day without having to find my keys.

I had a few goals:

Don't do anything warranty-voiding. That means no messing with the car's wiring or internal computer
Everything runs on-prem - because the "S" in IoT stands for Security
Integrate into an existing Home Assistant setup, so I can tie into rest of the home automation ecosystem I've been building up

A few others have gone down this route, by wiring up a spare fob to a microcontroller (Arduino, Raspberry Pi, etc), and "pressing" the unlock/lock/start buttons electronically. Seems feasible!

https://hackaday.com/2021/01/06/automating-your-car-with-a-spare-fob-and-an-esp8266/
https://www.hackster.io/user03583/ok-google-start-my-car-7088dd
https://gitlab.com/milagrofrost/esp8266-car-key-fob-iot/
https://github.com/Radacon/ESP_Remote_Start

Hardware

Spare Car Fob - I picked up a cheap aftermarket fob on AliExpress, though it took a bit of time confirming compatibility. I ordered this one for the 2023 Bolt: https://www.aliexpress.us/item/3256804168747183.html

The Bolt lets you program additional fobs at home, without needing a dealer or locksmith. The metal "blade" isn't cut, but I'm never using it for "actual" car operations, so I'm not too worried.

ESP8266 - Any microcontroller that can output 3.3V should work, and I'd been wanting to experiment with ESP devices for a while. I used this: https://www.amazon.com/dp/B07HF44GBT

I wired everything up using a breakout board, since the circuits went straight from GPIO pins to the fob buttons.

Some other projects I saw added diodes in the circuitry to prevent flyback voltage to the buttons, but I didn't find it was necessary. Let's see if future me regrets this!

Wiring Up the Fob

This took a bit of experimenting. Using a multimeter, I found the buttons were letting 3.3V run through, which went down to 0V when it was pressed down.

The buttons are surface mounted on the fob's PCB. Removing them would be pretty easy with the right tools, such as a heat gun. I did not have said tools, so I just cranked up my soldering iron and jabbed at it until the buttons disintegrated. Hopefully you're better at this.

Once I exposed the bare pads on the fob, there was a bit of trial and error to find which lead was connected to the fob, and which to ground (I used a breadboard and just tapped the pads while seeing if the car responded). From there, I soldered a wire between the connected pad, and onto a GPIO pin on the ESP8266, via this pinout.

Software

Home Assistant - I already had an instance running on a Raspberry Pi, so this was easy.

ESPHome - This is a really neat project that exposes an ESP8266 (or ESP32) as a device in Home Assistant. You upload their firmware on your ESP, and it'll securely integrate with your HA environment. You can then configure its operation via simple YAML, so you don't have to maintain any code or event loop.

Here's the "Unlock" section of my configuration:

output:
  - platform: gpio
    pin:
      number: GPIO5
      inverted: true
    id: UnlockButton

button:
  - platform: output
    name: "Unlock"
    output: UnlockButton
    duration: 400ms

Note the inverted: true - I found that fob buttons are "normally open", so a button press brings it back down to 0V.

Another thing I really liked about ESPHome: they have a web-based flashing tool to upload the custom firmware onto your ESP8266. Once flashed, everything else is done wirelessly via the onboard Wi-Fi chip. This is way more convenient than the Arduino or MicroPython workflows I'd experimented with; which required driver installation and a terminal emulator.

Once you've setup the appropriate buttons in the ESPHome configuration section (I found 400ms worked for lock/unlock, and 3000ms was needed for remote start), install to the ESP via an OTA update, and Home Assistant should recognize the new devices.

I then created a Dashboard in Home Assistant so I had easy access to the buttons.

A few things I learned

While testing the fob buttons, I discovered the “start vehicle” and “trigger alarm” buttons were reversed from how they were labeled. As did all my neighbors!
Soldering on a surface-mount board is hard! I ended up bricking my first fob because I got a little solder on some adjoining transistors, which short-circuited the entire device and rendered it useless. I suppose this gets better with practice, but when you treat hardware projects with the same rigor as a Geocities website, it's not something you improve at quickly.
The ESP8266 is a really cool device; and it's fascinating to have something so easily programmable be at a price point where I can just treat the microcontroller as a dedicated component to this project. With a Raspberry Pi or Arduino I'd be sure to salvage after I was done with the project, but at less than 5 bucks, I don't feel bad soldering directly onto it.
There's a few home automation/IoT aspects I haven't gotten to yet; designing and 3D printing a custom enclosure would be a nice way to finish the project. For now, it's just sitting on a shelf.

What is it good for, absolutely nothing

This setup has a few limitations. In particular, it only works when I'm on-network in my house, and the car is in physical proximity to the fob, but I consider that a feature rather than a bug. Adding some physical safeguards helps me sleep a bit better.

I still don't really know what I'm going to do with the project. Since it's tied into home automation, I could easily setup a few interesting workflows:

Automatically lock the vehicle at 10pm
If the temperature drops below freezing and it's a workday, start the car 10 minutes before my commute

I'm fully ready to admit these might be solutions in search of a problem, but I still had a fun time working on this. And maybe, the real vehicle automation is the friends we made along the way.

Generating Custom OpenGraph Cards with Gatsby and the 11ty Screenshot Service

2023-03-13T00:00:00Z

I made the sharable OpenGraph images on this site better by generating custom cards contextual to each post:

The core concept is to create a sidecar page for each post that looks like an OpenGraph card, then take a snapshot of the page, and set it as the og:image meta tag on the original post.

There are a few Gatsby plugins that attempt to help, but they rely on Puppeteer to generate screenshots at build-time, which can slow down a build pretty significantly, and adds a heavy dependency to the toolchain.

The approach I took leverages Gatsby's dynamic page generation, but leaves the image rendering to external services, only generating as-needed at runtime.

You can view the generated card for this post to get a feel for what's getting generated, and see a full implementation diff here.

React Card Component

Like other Gatsby pages, this is a React component you'll set props via Gatsby. I placed it in components/social-card.js. Since OpenGraph images are generally a fixed size, you can target your design for those specific dimensions.

import React from 'react';

// It's a Gatsby component, so pull in whatever assets are useful for the page
import avatar from '../../content/images/avatar-transparent.png';

// I'm using CSS Modules for styles, but you do you
import { card, cardTitle, cardExcerpt, cardDate, cardMetadata, cardAuthor, cardAvatar, cardTransparency } from './social-card.module.css';

const SocialCard = ({ pageContext: { title, excerpt, date } }) => {
  return (
    <main className={card}>
      <h1 className={cardTitle}>{title}</h1>
      <p className={cardExcerpt}>{excerpt}</p>
      <img className={cardAvatar} src={avatar} alt="Avatar" />
      <div className={[cardMetadata, cardTransparency].join(' ')}>
        <p className={cardDate}>{date}</p>
        <p className={cardAuthor}>steele.blue</p>
      </div>
    </main>
  );
};

export default SocialCard;

Gatsby Card Pages

In gatsby-node.js, enhance the onCreateNode method (or wherever you're currently calling createPage) to generate a new page, which will be for your component's social card.

First, query for all blog posts, pulling the props you'll want:

  // Query for all blog posts
const socialCardQuery = await graphql(`
{
  allMarkdownRemark {
    nodes {
      excerpt
      fields {
        slug
        date(formatString: "MMM DD")
      }
      frontmatter {
        title
      }
    }
  }
}
`).then((res) => res.data);

Then create a Social Card page for each post, passing in the relevant props:

const pageContexts = socialCardQuery.allMarkdownRemark.nodes.map((node) => {
  return {
    slug: node.fields.slug,
    pageContext: {
      title: node.frontmatter.title,
      excerpt: node.excerpt,
      date: node.fields.date,
    },
  };
});

const socialCard = path.resolve('src/components/social-card.js');
pageContexts.forEach((page) => {
createPage({
    component: socialCard,
    path: `${page.slug}/social-card`,
    context: {
    ...page.pageContext,
    },
});
});

Once these are getting generated, you can just add /social-card to the end of any permalink to see what's getting generated, and iterate until you're happy.

OpenGraph Screenshots via 11ty API

Ultimately OpenGraph expects an image, and we currently have a website. Fortunately for us, the Eleventy project supports a service that will generate screenshots of pages via URLs, with presets for OpenGraph dimensions.

In your blog posts, you'll want to define the og:image meta tag based on these properties. Here I'm using React Helmet, but you could also do it with the Gatsby Head API.

// Helper methods to configure the Eleventy Screenshot Service
const socialCardUrl = (url) => `${url}/social-card/`;
const screenshotUrl = (uri) => {
  const encoded = encodeURIComponent(socialCardUrl(uri));
  
  // https://github.com/11ty/api-screenshot/#manual-cache-busting
  const cacheBust = new Date().getTime();
  return `https://v1.screenshot.11ty.dev/${encoded}/opengraph/_${cacheBust}`;
};

// Then in your blog post's component, set the value:
<Helmet
  meta={[
    {
    property: 'og:image',
    content: screenshotUrl(url),
    },
    // etc
  ]}
/>

And with that, you're off to the races! As Zach put it, perhaps folks will start clicking on my posts if I work really hard on the OpenGraph images.

Keeping your Fediverse followers when moving Owncast servers

2023-03-12T00:00:00Z

So you've got an Owncast server running, and it's part of the Fediverse, so your Mastodon friends can be notified when you start a stream. Neat!

If you want to switch hosting providers, or install from scratch, you'll want to make sure your followers continue to receive notifications. This could be easy, or a bit of a challenge. Either way, here's what to do!

The easy way - Copy over the database

The simple solution is to just copy over the data/ folder from your old server, onto the new one. This includes the previous database, and assets like logos, etc.

This will work if you're moving to the same (or a newer) version of Owncast.

If that doesn't work

In my case, I couldn't migrate over because I was moving to an older Owncast version, for complicated reasons.

So, I had to manually copy over just the Fediverse/ActivityPub data from the old instance.

To finish this, you'll need the sqlite3 client on both instances. On Debian, this is sudo apt install sqlite3.

Extract ActivityPub tables and configuration values

# On the old server
cd data
sqlite3 owncast.db ".dump ap_followers ap_outbox ap_accepted_activities datastore" > owncast-activitypub.sql

Copy the owncast-activitypub.sql file off your old server using whatever means you like, scp, etc.

Tweak import script

I had to make these changes to the sql file manually:

Remove table schema creation

By default the .dump operation includes CREATE TABLE lines. Remove them from the script so you're just left with INSERT statements.

Accomodate data differences

Since I was moving onto an older version of Owncast, the database had a different schema, so you'll have to tweak the INSERT statements to reflect the version you're moving to. i.e., we're performing a "backwards migration".

You can see which database version you're running with:

# On the new instance
sqlite3 owncast.db "select value from config"

Check whether any migrations have happened to the ap_* tables; you can view the Forward migrations file for details. In particular, I was moving down to version 3, and so I had to remove the request_object column from the ap_followers line:

-- Convert this:
INSERT INTO ap_followers VALUES('https://carhenge.club/users/mattdsteele','https://carhenge.club/users/mattdsteele/inbox','Matt Steele','mattdsteele@carhenge.club','https://s3-us-east-2.amazonaws.com/carhengeclub/accounts/avatars/000/022/282/original/238f5dcb7409495b.jpeg','https://carhenge.club/79be21a3-3c15-4174-9cc2-116440ccda04','2022-11-22 04:39:12','2022-11-22 04:39:12.733741659+00:00',NULL,X'somelongvalue');

-- Into this:
INSERT INTO ap_followers VALUES('https://carhenge.club/users/mattdsteele','https://carhenge.club/users/mattdsteele/inbox','Matt Steele','mattdsteele@carhenge.club','https://s3-us-east-2.amazonaws.com/carhengeclub/accounts/avatars/000/022/282/original/238f5dcb7409495b.jpeg','https://carhenge.club/79be21a3-3c15-4174-9cc2-116440ccda04','2022-11-22 04:39:12','2022-11-22 04:39:12.733741659+00:00',NULL);

Depending on the version you're migrating to/from, you may have to make additional changes, or none at all!

Extract config table values

Remove all the INSERT INTO datastore lines except for the following:

federation_enabled
federation_go_live_message
federation_username

These need to be the same on the new host, in order for clients to validate identity.

Import data into new instance

After making these changes, copy the owncast-activitypub.sql file onto the new host, place it in the data folder, and run:

# On the new server
cd data
sqlite3 owncast.db ".read owncast-activitypub.sql"

With that, you should be able to test creating a new stream, or compose a new post in the Admin header, and you're off to the races!

Running Owncast with Hardware Acceleration on a Raspberry Pi 4

2023-03-05T00:00:00Z

Owncast is a self-hosted platform for streaming video, which I've been using for sharing all kinds of events, from Zwift racing to my wedding. It's been really pleasant to use as a Twitch alternative.

Historically, I've run my instance (https://stream.steele.blue) on a virtual server in The Cloud (specifically, a $6/month DigitalOcean instance), but wanted to explore moving it on-prem, using my own hardware. I had a few goals in mind:

Performance - Owncast uses ffmpeg under the hood to encode videos. If you have access to 'native' hardware you can enable hardware acceleration to improve performance by offloading rendering to a GPU. This isn't just a theoretical gain; on the VPS I can only encode a single medium-quality stream before I peg the CPU.
Utility - I had a Raspberry Pi lying in storage after finishing up an earlier project, which feels like an incredible waste of resources. Imagine an entire Linux computer, just sitting in a drawer!
Cost - Six bucks a month for a VPS isn't exorbitant, but it adds up. Within a year, I could buy a brand new Raspberry Pi (even at their inflated current prices)!

The Owncast docs includes a nice reference on configuring a Raspberry Pi with hardware encoding.

I used a Raspberry Pi 4 Model B with 2GB of RAM, and a 32GB MicroSD card I had leftover from previous projects.

It's connected to my home network via Wi-Fi; which is also where the source RTMP stream is at.

tl;dr

Here's what currently works, as of March 2023:

Software

You'll need to run an old version of Raspbian OS. This is a 32-bit OS, even though the Pi 4 has a 64-bit CPU. More on that later.

You'll also need a copy of ffmpeg which supports the OpenMAX (omx) codec. The version available from the package manager will be sufficient: sudo apt install ffmpeg.

The remainder of the setup from the Owncast quickstart is sufficient. For example, I enabled HTTPS by running an instance of Caddy on the Pi.

Configuration

Set up Owncast like the quickstart tells you. In the Admin console, under "Advanced Settings", change the Video Codec to "OpenMax (omx) for Raspberry Pi".

My video configuration has two stream outputs defined:

1200kbps, Low hardware usage, 24fps (Low quality)
4100kbps, Medium hardware usage, 60fps (High quality)

Feel free to experiment with other configurations, but this worked for me.

I still serve the videos from S3 storage, which I had hosted in AWS, but any of the S3 options should suffice. S3 is cheap and easily scalable, so there wasn't a need to pull that on-prem

What went wrong

As described above, I had to use an older, 32-bit version of Raspberry Pi OS, as well as an older version of Owncast, and won't be able to upgrade either of them as new releases come out. That sucks! Here's the issues I ran into when trying to use current versions. Again, this is as current as of March 2023; I'm hoping to try again in a few months.

Newer Raspberry Pi OSs don't have turnkey hardware encoding

Support for OMX on newer Raspberry Pi OS versions has gotten worse. Previously it wasn't supported on 64-bit Pi OS, but now it's gone from bullseye distros even on 32bit: https://github.com/raspberrypi/firmware/issues/1366#issuecomment-1034726587

So I took a look at enabling the new option for hardware encoding (Video4Linux). After hours of investigation, I wasn't able to find a version of ffmpeg that would let me successfully encode/decode videos with V4L hardware acceleration. The full gory details are in Issue #1379.

I'm still pretty happy with this setup

Even with the issues I ran into getting this all running, I like where it ended up. I got a better-performing instance running on cheaper hardware, at the cost of a few hours of experimentation. If I ever need to reconfigure things, it should be a 20-minute setup.

Ideally I'd have this running in a Docker container, but I didn't want to run out of spoons and was ready to declare victory and just use the software.

Over the last few months I've been reevaluating my use of cloud services, with a particular focus on virtual servers. They occupy a "muddy middle" in terms of management responsibilities. Sure, Linode is going to spin up the instance, but I'm still responsible for installing my app, applying OS patches, isolating it within my tenant, etc. All the while, I'm paying as much to rent the donkey as it would cost to buy it.

I wouldn't want to run a big distributed project on hardware I have at home, but then again, I wouldn't want to run any home project that requires a distributed setup.

But with the advent of simple Docker management tools like Portainer, I've been able to move a number of apps previously hosted on DigitalOcean or Linode out of the cloud, and onto machines running in my living room. They've got enough resources that I can scale vertically for quite a while, or pick up another Raspberry Pi or two if I want more "single app appliances". And with tools like Watchtower to help keep my containers up to date, and easier at-home network segmentation with Unifi VLANs, it's never been easier to get a simple, secure home server up and running.

Vite is Neat

2023-01-31T00:00:00Z

I spoke at NebraskaJS about Vite:

Aside from a scratchy voice initially (and having to present remotely due to a Covid recovery), it went well! I was excited to share my excitement for ES Module-centric toolchains, and the speed they afford day-to-day development.

As an example of a Vite-driven website, I showcased the GPS bike tracking website, which used Vite on its frontend. The UI isn't especially complicated, but it demonstrates the flexibility tools like Vite afford. I didn't need a full framework, and most of the functionality was Vanilla JS, along with a few geospatial tools and a mapping library.

Simply going toolchainless and using skypack might have been the easiest approach, but I still wanted some type-safety for the code I was authoring, and Vite offered an easy way to integrate TypeScript.

I do wonder if Vite and its tools are just steps along the path toward full buildless development. It's exciting to see tools embracing new primitives, while also meeting developers where they're at, and enabling productivity along the way.

Serverless Bike Tracking with a SPOT Tracker, AWS Location + Friends

2022-10-10T00:00:00Z

I recently participated in the Gravel Worlds Long Voyage bike race. Last year I tried, and failed, to finish the 300 mile course, and built a pacing calculator along the way.

This year, I felt better prepared, rode smarter, and actually finished! My intent was to simply complete the race within the time limit, with a stretch goal of finishing before sundown. I ended up making it with about an hour of sunlight to go, and I couldn't be happier.

I credit some of my success this year to spending more time riding through the year, both on longer gravel base rides, and structured intervals on the trainer. But I still had enough time to build another dubiously useful website!

That's what I want to share today; a site that captures data from a GPS tracker, and makes it available for folks to track my progress. There are a number of public versions of this (they call it dot watching), but I wanted to add some fancier features, such as making geofences at expected stops, and capturing enter/exit times.

I built out the site using AWS serverless architecture (Location Services, Lambda, and others). The code is available at https://github.com/mattdsteele/spot-tracker-tracker, and you can see the page for my ride here.

Capturing GPS Pings

I used a SPOT Messenger device; which all racers were required to carry with them. SPOT devices are neat; they were designed with fairly easy-to-use APIs, along with a set of prebuilt maps you can share with friends.

To get geofencing and other features, I had to ingress each GPS ping into the AWS Location data store. This required the use of a Lambda function, scheduled to run at the same frequency that GPS pings were emitted (5 minutes).

This (as well as all the other Lambdas in the project) were built as Go functions.

Calls to SPOT's history API returned all observations over the past 24 hours, so I needed to find a way to only send in calls that hadn't been indexed before. I ended up keeping track of the latest timestamp via an object in DynamoDB, and only returning newer data. There might be more efficient ways to do this.

Once the pings were stored as an asset in AWS Location, they remain for 30 days.

Geofencing

AWS Location also supports geofences. For each GPS ping that arrives, it can be evaluated against a set of predefined geofences. These are emitted via EventBridge, and you can react to them with Lambda functions as well.

I stored entrance/exit events for each geofence in another Dynamo table.

Routes

Cycling routes are traditionally saved as .fit files, among other formats. These files aren't usable in mapping libraries directly; they have to be converted to GeoJSON or another similar technology.

To do this, I used Tormod Erevik Lea's fit Go library, which made easy work processing the .fit files.

I wanted to make this easy to use, so I made an S3 bucket so I could upload .fit files as I set courses up. I then built a Lambda that found the latest file in the bucket, and returned the coordinates in an easy to process JSON format for the UI.

lambda.Start(func(ctx context.Context) (events.LambdaFunctionURLResponse, error) {
    config, _ := config.LoadDefaultConfig(ctx)
    files := spot.GetFilesInBucket(config)
    latestFile := files[0]
    fileContents := spot.DownloadFile(*latestFile.Key, config)
    fitFile := spot.ParseFitData(bytes.NewReader(fileContents))
    course := toCourse(fitFile)
    j, err := json.Marshal(course)
})

REST Services

AWS Location has a set of REST services and JavaScript APIs, but I found them to either be very heavyweight (using AmplifyJS), or required setting up a multitude of additional security and IAM authorization policies, which I wasn't confident I could implement properly. All I wanted to do was provide some read-only data to a website!

I ended up exposing the Lambda functions as REST endpoints. This is a new-to-me feature, which makes it dead-simple to provide read-only access to Geolocation data, or any other AWS resource. The URLs aren't pretty, and by default are on another origin, such as https://6f7w2jqblnebkk75folo4zv7j40qvxfp.lambda-url.us-east-2.on.aws/. I'd love to host these under the same domain as my UI to avoid cross-origin requests, but it appears to be quite complicated, so I'll live with CORS.

Website

The app itself is pretty barebones; it loads a full-screen MapLibreGL map, and adds the appropriate layers for routes, geofences, and GPS pings. There's a few geospatial functions to calculate how far on the course you've gone using the nearestPointOnLine function, but there's nothing too fancy there.

I built the app using Vite as my build tool and scaffolding (using their vanilla-ts template). I've been very impressed by the modern, ESM-first tools, which don't bundle anything in development, leading to a fast feedback loop. I still was able to use TypeScript and other modern affordances, and could pull in modules like date-fns as needed, without feeling overwhelmed by tooling. It's not quite toolchainless, but it's close!

The UI it built via GitHub Actions on each push, and deployed to Netlify. I also have branch deploys configured, which allows me to save off the service calls as JSON files for posterity.

Cost

Since none of the resources I'm using are persistent, the app is stupid cheap to run. My current AWS bill is about 77 cents per month. Location Services cost 15 cents, S3 storage is another 7 pennies. I'm no longer on the AWS free tier, but at these prices I might as well be.

I came into this project believing that serverless architectures are great for small hobby projects like this, and I remain more convinced this is the case. Since everything scales to zero, I can leave everything running and ready for next year.

Next Steps

There's plenty of room for improvement here. I'd love to make it easy for someone else to deploy, via a Terraform or other infrastructure-as-code options. Tools like https://github.com/GoogleCloudPlatform/terraformer may make this easy.

I'd also like to find ways to send push notifications to interested parties when I enter/exit geofences. This feels like it should be possible using Web Push, even if it's fallen out of favor as an API over time.

Overall I'm pretty happy with this project. I'm planning on trying the Long Voyage course again next year, so I'll have plenty of opportunities to procrastinate training by hacking on this further!

ACEing Ocarina of Time

2022-07-11T00:00:00Z

The Summer Games Done Quick event recently ended, and I'm still thinking about an Ocarina of Time run:

It's best if you go in without too much knowledge of the run, but if you grew up playing and reading rumors on Geocities about what was possible, you'll love the process.

The team did a great job explaining the run, which relies on Arbitrary Code Execution and other tricks.

The summary video is great as well:

This particular run achieve the level of art, but here's a few other runs I enjoyed:

Advent of Code as Soulcraft

2021-12-26T00:00:00Z

As we close out another Advent of Code season, this year the Meta Discourse is less around any of the puzzles, and more around whether you should feel pressured to participate. Especially if you see your GitHub activity feed filling up with others starting advent-of-code repos every December, it can feel like you're missing out if you don't want to, or aren't able to participate.

In general I'm sympathetic to these arguments. I'm not into the "rise and grind" and hustle side of software development. And while I've resonated with the idea that passion for programing was an essential part of having a career in software, I'm less beholden to that over time. It's totally fine to treat coding as any other job, and not have it encompass your off-hours time too!

I'd like to offer a limited defense of Advent of Code and other programming puzzles (Project Euler, Code Retreat, and the like). These particular challenges recharge me and get me excited for my day job, and I think that's a more sustainable way to approach the puzzles than "you should always be learning new things, or you'll fall behind".

For me, Advent of Code really scratches an itch to do test-driven-development in as ideal an environment as one could cook up:

The problems are well-defined and don't contain business-logic ambiguity
Sample inputs and outputs make for simple test definitions
The two-part nature of each puzzles encourages clean code and the red/green/refactor programming cycles

When I get into a TDD loop, it feels fun, almost therapeutic. It's rare that I get these opportunities during the work week, so getting the chance for fresh puzzles is a really great feeling.

If this isn't the case for you, that's fine! And Advent of Code doesn’t have to be performative, you can make your repo private (or just delete the code when you're done!). And the puzzles are around all year, you can use them as a pallets cleanser throughout the year, they’re still around!

Gravel Worlds - Long Voyage

2021-08-23T00:00:00Z

I participated in the inaugural Gravel Worlds Long Voyage bike race last weekend. It was my fifth year racing a Gravel Worlds event, and I'm not planning on stopping any time soon. Corey, Craig, and the other organizers do a great job to make everyone welcome and accepted.

I applied for the 300-mile event partially for personal redemption. In 2018 I raced Trans Iowa, but only made it about halfway through the 330+ mile course, primarily due to poor equipment and clothing choices. After T.I. ended, I wanted to give it another go, and trusted the Pirate Cycling League to keep the spirit alive.

I pulled up to the starting line having trained with teammates Rafal and Alex, both of whom are stronger riders and helped pull me through summer Bacon Rides. I also got to chat with multiple other folks who came out for the send-off, including Patrick Davlin, Robbie Benton, Jason Faas, Michelle Cleasby, Curtis Wilson, and Dustin Slivka. Seeing these folks after a year of of isolation and Zwift riding was really heartwarming.

Prior to the race starting I spent most of the previous few days worrying about the weather. The race included 15 miles of dirt/minimum maintenace roads, which can easily become unrideable when wet. The forecast called for thunderstorms the evening of the race, which would have made a long day even longer. As the race started, the weather was beautiful (though humid), so I was excited we had avoided the storms. Turns out, not the case!

As the race began I was feeling good. The race didn't include any intermediate cutoffs, but to finish you had to maintain a 10mph pace, including any stops/breaks. I was riding around 15mph, so I had some time banked. The gravel north of Lincoln is quite sandy, but was quite rideable on my Cannonball 42mm tires.

Around 25 miles in, the skies opened up and it began pouring. Luckily there was no ride-delaying lightning or thunderstorms, those all appeared to be south of us. I just got completely drenched, and ate a bunch of limestone that collected on my water bottles. As the sun began to set, the storms continued to be visible on the horizon through Omaha, and it was quite the experience to see the spider lightning from afar. I hit one section of MMR which was a little damp but completely rideable, and thought I was in the clear. How wrong I was!

At mile 54 we refuled in Weeping Water, and continued on to the east and south of Lincoln. Remember how I was so excited that the majority of the rain missed us to the south? Apparently I wasn't thinking straight, and was unpleasantly surprised when I started to pick up more mud on the next MMR, though I was able to shed it without too much hassle.

Night riding was pretty pleasant, the weather was great with a nearly full moon helping illuminate the corn and soybeans surrounding me, and cicadas helped to drown out the sound of crunching limestone beneath me.

At mile 90 we hit another mile of MMR which was partially unrideable, though I felt pretty confident just mashing through on my singlespeed as I had fewer parts to fail. I came across one rider with a completely blown deraillieur after attempting to shift while covered in mud, which unfortunately ended her event.

Finally around mile 130 I hit a dirt road that was completely unrideable, and so muddy that it was impossible to even push the bike through; mud would just get caught on the tires and jam up in the fork, seizing up the wheels. The only solution here is to hike-a-bike. I have some experience from years of racing cyclocross, but the optimal strategy of shouldering in the triangle wasn't feasible with frame bags and multiple bottle cages. So I threw the bike over my shoulder, and just tried to slog on. Those two miles of road took nearly an hour to traverse, and sapped a ton of energy I was hoping to preserve for later in the race.

We hit one more dirt road that evening, but we were able to push the bike through a grassy ditch. As dawn broke, I was praying to the gravel gods that sunlight would help dry out the remaining 7 miles of dirt on the course.

While riding overnight I also encountered a new terrain I hadn't seen in years of gravel riding - slatted bridges with gaps wider than my tires; which could easily swallow a wheel. I was probably more cautious than I needed to be when traversing them, but didn't want to take any chances suffering a mechanical because I was in Full Send mode.

As dawn broke I continued to ride with a couple folks from Pennsylvania and Michigan. As the headwinds began to come into effect, we formed a paceline and began to take pulls pretty effectively, considering we hadn't ridden together before. A few of them pulled the plug in Crete, and I rode out to Milford with another singlespeeder from Kansas. We encountered a few dirt roads, but the sunlight and wind dried them out and they were as fast as pavement.

Coming to Milford at mile 215, the math wasn't looking great for making the 11pm deadline, my pace just wasn't high enough to make the cutoff, especially since the final 50 miles would be going through the hilly Bohemian Alps. I wanted to push through more, and rode solo out to Seward, only stopping once to chat with a farmer whose driveway the course accidentally cut through. I think he was more shocked that any cyclists were out in the area, than he was upset we were technically trespassing.

I made it to Seward at mile 234 and decided to pull the plug. I probably had enough gas in the tank to make the next town at mile 267, but exhaustion after 25 hours of riding was beginning to set in, and I was fairly certain I was in the lantern rouge position and wouldn't have anyone checking on me in case something went wrong.

Things that worked well

Equipment - I was pretty happy with all the stuff I carried. For night riding, I modded a Lumintop EDC18 flashlight with an elliptical lens, and carried multiple extra 18650 batteries to swap out through the night, this gave me plenty of light and the confidence to keep a reasonable speed. Also bringing a clean change of socks was a huge morale booster.
Singlespeed gearing - I rode an easier gearing (42x22) than I generally ride gravel (42x19), with the assumption that I'd need the extra teeth as fatigue and hills crept up throughout the day. This was a good choice; I'd have been walking far more hills near the end had I chosen a "faster" cog.
Route planning - I spent the days prior to the race checking out the Nebraska DOT's surface type maps to try and gauge where the MMR was on the course. Knowing how much of a break I'd have before the next potential hike-a-bike was a good boost of self-confidence and cleared a potential point of anxiety for me. Additionally, building a route estimator helped keep me in tune with how I was progressing based on my moving speeds.

Things I'd work on improving

Bike fit - If anything was going to give on this ride it would have been my back and palms, and I think a more precise fitting and adjusting saddle position, etc. would help alleviate these issues.
Prepping for hike-a-bike - If there's a good chance I'll be shouldering a bike for multiple miles, I'm going to strip off the bottle cages on the bike frame, and wear a Camelback instead. It'll help with proper form, and the hydration pack should provide some padding for the steel frame digging into my shoulder (which is still sore!)
Applying more sunscreen - I got burned after not applying more sunscreen on the second day of riding, and sunburns traditionally sap a ton of energy from me.
Less stopped time - At each town I stopped at convenience stores anywhere from 15 to 50 minutes; this obviously eats into my elapsed pace, and is probably the lowest-hanging fruit.
General fitness - Near the end of my ride I was in dead last place of those still riding. For races with a cutoff, that's not good enough. Some structured training, sweet-spot intervals, and the like are the standard solutions here.

Overall I was pretty happy with my race; I set new personal records for the longest and farthest I've ridden in one go, it was my first completely overnight ride, and I made some new friends along the way. But I'd love to finish one of these endurance races, so I have a 2022 goal already in mind.

Making a Raspberry Pi Photo Booth with Pibooth, NeoPixels, and Giant Buttons

2021-08-17T00:00:00Z

I put a homemade photo booth together for a wedding reception we postponed for a year, on account of, gestures everywhere:

It was a fun project that let me play with new hardware and software. Here's a few photos of the project, and some of the things I learned.

The guts of the project were driven by a Raspberry Pi 4 running pibooth. It's a great project that works well out of the box, and has a plugin system if you want to anything.

The Pi, wiring, and breadboard and wiring went into a small box, along with a few buttons. The big button started captures, and the small one printed the last photo.

I tried various means of connecting up the buttons; speaker wire worked surprisingly well! But in the end a few small alligator clips and jumper wires were sufficient.

We setup a couple backdrops and had a modeling light to provide standard illumination. I would have loved to use a greenscreen along with an OBS virtual camera, but the Pi wasn't powerful enough to perform chroma keying in real-time.

Photos were taken using a webcam. Originally I was hoping to use a higher-quality mirrorless camera, but gphoto2 has issues with Sony cameras.

Printing was done using a small Canon printer, which we had on a stand below the image. I had never used a dye-sublimation printer before, but I was really impressed with it! The guests were really excited to see each color get applied sequentially, and took home a postcard-sized souvenier from the night.

We also configured each photo to get uploaded to a shared Google Photos album and had a QR code to let folks view their photos after the event; this was also supported by a few Pibooth plugins.

Hardware

I wouldn't recommend this as a shopping list; this sort of project works best reusing any pieces you have around your house. If you were to source all these parts exclusively for this project, it would be substantially more expensive than using an off-the-shelf photo booth (or renting one).

That said, here's the parts I used:

Computer: Rasperry Pi 4
Camera: Mevo Start
Camera attachment: Flex Arm with tripod adapter
Display: HP 27" monitor
Display attachment: VESA monitor mount
Printer: Canon Selphy CP510
Modeling light and tripod - Interfit Honey Badger
Buttons: Adafruit arcade buttons
Ring light: WS2812B Neopixel clone with 48 LEDs

Software

Most of the software was taken care of by pibooth, with a few additional plugins and customizations.

You can see the configuration and other customizations here: https://github.com/mattdsteele/pibooth-config

The primary mod I built was a custom plugin which provided support for the ring light, using Adafruit's neopixel Python library. It setup an "attract mode" with a rainbow cycle, as well as a countdown timer and a virtual "flash" while capturing images.

I learned a bit about Python multithreading along the way too!

@pibooth.hookimpl
def state_preview_enter(app):
    app.pixels.fill((0,0,0))
    app.pixels.show()

    proc = threading.Thread(target=countdown, args=[3, app.pixels])
    proc.daemon = True
    proc.start()
    app.neopixels_proc = proc;

Along with our wedding livestreamed with Owncast, I had a great time geeking out with my wife. Way more fun than stressing out about caterers and dress fittings.

3 Things I Learned Trying out the JavaScript Temporal API

2021-08-11T00:00:00Z

I've been preparing for a 303-mile endurance gravel bike race. Not by training or improving fitness, of course, but by building a PWA to estimate and track checkpoint ETAs.

By adjusting the sliders for my day and evening paces, the app estimates when I'll be able to take breaks, reach hazards, etc.

It's been a while since I built something on the front-end that did lots of date manipulation, but I thought this would be a nice time to try out the Temporal API, which is now in Stage 3 and has a nice polyfill available.

Here's a few things I've learned:

It's pretty similar to Java Time API

The existing JavaScript Date API sucks. And like many things, Java is the cause of, and solution to it.

The APIs were originally ported wholesale from the Java 1.1 era, with all the bugs and warts therein. Months in the year are zero-indexed, but days of the month are one-indexed? You better believe it.

Java's date/time APIs sucked badly enough, and longly enough, that most developers switched to userland libraries, in particular Joda Time. Eventually the lead maintainer of Joda Time was recruited to create a new standard date and time library.

Similarly, a thousand Node libraries/flowers have bloomed, all trying to provide a saner operations on dates (including, incidentally, a port of Joda Time). And Maggie Johnson-Pint, the core maintainer of Moment.js, has been championing the new Temporal API standard.

Having worked with both of these new "standards", I'm pretty impressed at the conceptual convergence! Most of the base classes are available in both libraries: from ZonedDateTime to Duration, to Instant. Combined with the polyfill's nice TypeScript definitions, it's really straightforward to just start working with the library, and autocomplete your way to development.

It's another example of why Java folks should feel at home with TypeScript.

Seriously, this is really nice to use:

const dayDelta = startTime.until(nightTime);
const totalHours = dayDelta.total({unit: "hours"})
const nightDelta = nightTime.until(morningTime);
const totalNightHours = nightDelta.total({unit: "hours"});

// how far can you get by sundown?
const dayDistance = totalHours * pace.day1;
if (distance < dayDistance) {
  const hoursDelta = (distance / pace.day1).toPrecision(5);
  const eta = startTime.add(`PT${hoursDelta}H`);
  return eta;
}

It doesn't support parsing human-readable dates?

One feature I was expecting, and which the Java Time API supports, is parsing and formatting "human readable dates", via a DateTimeFormatter. This doesn't appear to be in scope for Temporal API! From what I can tell, if you've got anything that isn't a flavor ISO8601, you'll need to pull in another date parsing library.

I'd love to be wrong about this, so if you know this API better than me, save me from having to pull in date-fns in the future.

It does format human-readable dates, using the Intl API

On the formatting side, it took a bit of digging but I was able to convert a ZonedDateTime to a human-readable time (think displaying just the HH:MM), using the Intl.DateTimeFormat API.

Unlike the Java formatter, which does take a literal HH:mm string, this API requires you to pass in an object of the fields you want to display:

const f = new Intl.DateTimeFormat("en-us", {
  hour: "2-digit",
  minute: "2-digit",
});
const etahhmm = f.format(eta.toInstant().epochMilliseconds);

This is a little unorthodox compared to the Java Time APIs, or Moment/date-fns/etc, but once you try it out it's pretty straightforward.

And it's not nearly as painful to use as the Golang formatter.

Overall the Temporal API has been really pleasant to use. I can't wait to use it as it gets built into browsers and Node, and not have to pull in another third-party Date library again.

Vaccinated

2021-03-11T00:00:00Z

I got vaccinated today, and my primary side effect was feeling conflicted about it.

By my standards, I should still be waiting for months to get immunized. I’m healthy, in my 30s, and have been working remote for virtually the whole last year. But I work for a railroad, which are classified by the state as essential employees in the transportation sector.

I was really torn about this. There are still so many high-risk folks who won’t be getting a chance for at least another month or two. In the state I live, immunocompromised folks are lumped in with the general public, and get no accommodations. And the math is ultimately zero-sum: every dose taken by someone who can manage their risk is a dose that can’t be given to someone more vulnerable.

But ultimately I couldn’t identify a clear distinction between my hesitation, and the more general vaccine hesitancy that’s resulting in large numbers of essential workers. And every recent story of an outbreak in a nursing home, where 40% of the unvaccinated nurses contracted Covid while 0% of those vaccinated did, solidified the decision.

I really hope we’ll be able to get out shit together and steepen the curve, and that we flip from being supply constrained to needing to seek out the remaining folks. But I’m ready to start turning the corner from the last hellish year.

Green Screen Zwifting with retroreflective fabric

2021-01-11T00:00:00Z

As group bike rides are doubly out this pandemic-laden winter, I've been doing more bike racing on Zwift, and playing with livestreaming races.

Zwift streams in-world aren't especially exciting. It's way more fun when you can watch a video of the person riding and suffering along. And if you're going to be suffering, you want to increase the production value.

I've been having a great time adding chroma key (green screen) to my video. Traditionally this has been a pain, requiring significant investments in lighting and screen positioning to do well. But with some LEDs and retroreflective fabric, you can make a well-functioning green screen on the cheap.

Most green screens use, well, green fabric. But this approach uses a different strategy: putting a ring of green LEDs around a camera and pointing it at retroreflective fabric, so the green light shines directly back at the source.

For the most part, I followed this guide on Brainy Bits.

I was able to build this all out for about $40, which is a steal compared to other green screens.

Materials Needed

I presume you have a webcam already.

Retroreflective fabric: I picked up four yards from an eBay seller in Canada.
Some way to hang your fabric: I used some magnets, but you could sew it around a PVC frame, or just tape it to a wall
LED Ring: 16 LED NeoPixel clone from AliExpress works well
Something to drive the LEDs: I used an Arduino I had lying around, but anything that can drive a NeoPixel works, such as this IR remote

Building it

If you're using an Arduino to drive the LEDs, you should be able to hook up 5V, ground, and a data pin to the LED rings. A simple app using Adafruit's library:

#include <Adafruit_NeoPixel.h>

#define PIN        6 // On Trinket or Gemma, suggest changing this to 1
#define NUMPIXELS 16 // Popular NeoPixel ring size

Adafruit_NeoPixel pixels(NUMPIXELS, PIN, NEO_GRB + NEO_KHZ800);

void setup() {
  pixels.begin(); // INITIALIZE NeoPixel strip object (REQUIRED)
}

void loop() {
  pixels.clear(); // Set all pixel colors to 'off'
  for(int i=0; i<NUMPIXELS; i++) { // For each pixel...
    pixels.setPixelColor(i, pixels.Color(0, 30, 0));
    pixels.show();   // Send the updated pixel colors to the hardware.
  }
}

Place the ring around your webcam; I used some Velcro stickers so I can easily remove it.

Hang up the fabric; it should be far enough away from your bike that it won't get in the way while you're riding, but close enough that you don't need to buy a ton of it. I ended up cutting the fabric to make two 3'x6' banners, and hanging on the ceiling with neodymium magnets.

Chroma Key in OBS

Power the LEDs to a pure green. You can probaby run it at a low power; if you go too high you'll get some green on you, which isn't ideal. I have mine at 25% of max.

This can be set in OBS as a Filter on your webcam Source. You can play with the properties, but I was able to get by with the default values.

Then setup your stream layout. A decent placement is in the bottom left of the screen, Zwift doesn't put anything useful in there.

Happy Zwifting! Your FTP won't be any higher, but at least you'll look a little more stylish riding in Watopia.

Getting the PPPoE Credentials from your C4000XG without calling CenturyLink

2020-08-24T00:00:00Z

If you've switched to CenturyLink recently, you might want to hook up your own router directly to the ONT, and bypass the router they provide/rent to you.

This is pretty easy to do! See this Reddit post for more details.

One thing you'll need to get from your ISP is the PPPoE userid and password the router uses to connect up to CenturyLink. Normally this requires a phone call, but if they provided you a Greenwave C4000XG, you can grab it from the router's admin console.

Simply log in to the C4000XG's web UI, and go to Advanced Setup -> WAN Settings, with your DevTools open. Look for the XHR request /cgi/cgi_get?Object=Device.PPP.Interface, and root around the JSON to find the credentials:

Or, just paste this into your console:

fetch('/cgi/cgi_get?Object=Device.PPP.Interface', {
  headers: new Headers({
    'X-Requested-With': 'XMLHttpRequest',
  }),
})
  .then((d) => d.json())
  .then((res) => {
    const data = res.Objects.find((x) => x.ObjName === 'Device.PPP.Interface.1')
      .Param;
    const user = data.find((x) => x.ParamName === 'Username');
    const pass = data.find((x) => x.ParamName === 'Password');
    console.log(`User: ${user.ParamValue}`);
    console.log(`Pass: ${pass.ParamValue}`);
  });

Not sure if this is service journalism or just a reminder to myself when I forget in a month!

Setting up a Livestream with Owncast

2020-08-17T00:00:00Z

So you've decided you want to livestream your event and run the server yourself instead of relying on Twitch? Neat! Here's the recipe I used for my wedding.

This is a single-camera setup, streaming through OBS to your server and a Zoom room.

Photo credit: Justin Duster

Hardware

To run the livestream, you'll need:

Digital camera - We used a Canon DSLR, connected to a computer via USB. You can also use the HDMI out along with a capture card if your device supports a "clean" video out.

It needs to be discoverable as a video input in OBS; not all digital cameras are! Our first attempt was with a 2015-era Canon camcorder, but it wasn't readable as an input source.

Audio input - If you have access to a mixer at the event, you can patch into the output and get high-quality from multiple sources. In our case, we had audio from a few microphones as well as another laptop running the soundtrack.

Server running Owncast - I used a DigitalOcean VPS for this. Setting up Owncast is documented in its Quickstart. I had it running in a Docker container.

Computer broadcasting via OBS - Hook up the audio/video sources into OBS and broadcast a stream to your Owncast server.

Depending on what else you have going on concurrently (such as simulcasting to Zoom), you may need a relatively powerful workstation for this. Our first attempt at broadcasting was on my i5 laptop, and it began dropping frames almost immediately! Switching to a gaming laptop resolved the issue.

Streaming pipeline

Owncast supports serving video files directly from the server, but we ran into constraints both on bandwidth and CPU utilization. A better approach is to configure Owncast to upload files to an S3-compatible storage and serve them via HTTP.

DigitalOcean's Spaces, an S3-compatible offering, fit the bill. You can follow Owncast's S3 guidelines for the storage provider of your choice.

User Interface

You can simply point folks to the Owncast's web UI. It's got chat integrated and is pretty nice!

I just wanted to emed the video onto our website, so we were responsible for embedding the HLS stream on our website.

You'll need a library to read the HLS playlist (from the Owncast server at /hls/stream.m3u8). We used videojs for a nicer "live" UI, but you can also use a native <video> element alongside the hls.js library.

Simulcasting to Zoom

Basically I followed this guide, which configures OBS's output as a "virtual camera" to feed into a Zoom room.

You also have the option to have Zoom be the "driver" of the stream to an RTMP server, but that puts an ugly Zoom watermark on the stream. If you have the bandwidth, it's better to upload via OBS and also have Zoom running.

Running the stream

We put together a Google doc with the playbook:

30 minutes before the ceremony: Launch OBS and configure input sources. Test audio levels, etc.

10 minutes before: Begin streaming a "Starting soon" scene to Owncast and Zoom

5 minutes before the ceremony: Begin recording the stream in OBS (to save a local copy)

Start of ceremony - Switch OBS from "Starting soon" scene to "real" video input

If you're simulcasting to Zoom, I also recommend deputizing someone as co-host, so they can admit viewers, mute noisy folks, etc.

Stuff I learned along the way

Test your stream beforehand - I did a few test streams in the weeks leading up to the event (I streamed some movies with friends), and the tests paid off like crazy. You don't want to be figuring on your event date what your server's optimal encoding settings are, or whether you need to host your video files via S3.

Communicate with collaborators - Ten minutes prior to my wedding starting, I was SSHed into the server, recompiling the Owncast container to switch off CDN file serving. Why? Because I miscommunicated with my videographer where a "test URL" would be setup, which resulted in an improper OBS configuration. Oops.

For short events, scaling your server vertically is cheap and easy - I started with a cheap $5/month server on DigitalOcean, but increased its CPU and RAM substantially (to their $160 server) to support transcoding 3 quality levels concurrently.

Since DO bills by the hour, it was still cheap! I only ran the server at this level for a day during testing and the ceremony, so it didn't actually cost more than a couple bucks to run. Way cheaper than spending a ton of time tuning server settings to reach optimum levels.

Thanks again to Justin Duster for helping run the event, so I didn't have to stress out too much on my wedding day!

Indieweb Livestreaming your Wedding with Owncast

2020-08-12T00:00:00Z

I got married! It looked a little different than we originally planned, due to it being, um, 2020. Venue restrictions due to the global pandemic reduced the number of guests we could invite by 90%. And we had no desire to put our friends and family in harm's way.

But hey, it's 2020! With new tools like Owncast, you can run a livestream for all the guests who can't make it to your event in person. And you can do so without giving up control of your content, or acceding to the whims of companies who might not have your best interest at heart.

I'll writeup the technical recipe we used in a later post, but thought the desire to host a livestreaming server was worth writing about separately.

Photo credit: Ben Turner

Twitch meets Indieweb

One of the lessons I've seen on the Web the past few years has been the cost to the increasing consolidation and siloing of your own content. Writing on Medium instead of hosting your own blog, for example, means you're subject to their nagware, interstials to drive readers toward their native app, and other user-hostile actions. Your content is there for the purpose of increasing Medium's engagement and MAU numbers.

Streaming sites are just as subject to this nefarios behavior. Twitch videos are routinely subject to DMCA takedowns, YouTube prevents you from streaming unless you have a requisite number of subscribers, and Facebook takes down DJ sets so routinely that DJs are forced to mix faster to avoid detection algorithms

And content disappears. When Twitch's predecessor justin.tv pivoted to gaming, its millions of videos were all taken offline. Mixer just shut down last month. More sites to add to the internet's cemetary.

This was a real concern for our wedding: we were going to have copyrighted music before the ceremony, and didn't want to just hope the takedown bots wouldn't find us.

But through cheap cloud VPS hosting and open-source tooling, we now have the tools to take back control of our streaming world.

Streaming with Owncast

The Owncast project couldn't have arrived at a better time. Billed as "Twitch in a box", it lets you start up a server to publish an RTMP video stream, transcode it to various qualities, and host an HLS-compatible livestream that will automatically choose the proper bitrate for the user's bandwidth. It also provides a website to view the stream alongside a chat feed. Plus it's written in Go, so it was easy to start contributing new features and bugfixes.

Combined with tools like OBS to publish a stream, you can easily start broadcasting with a $5 VPS from the cloud provider of your choice.

And since you own the server, you can customize it to your liking. We just wanted the video and weren't interested in chat, so we embedded the stream directly on the wedding website. We also were able to use the Owncast status API to detect when the stream was live, and disabled the embed while the stream wasn't running.

Publish Once, Syndicate Everywhere

Some of our wedding guests asked for a Zoom link, so we set up a simulcast to broadcast to a Zoom room, alongside our wedding website. This was a convenient way to meet folks where they already were.

In total we had 47 folks streaming via the Owncast server, and another 50 or so on the Zoom call. It's a nice way to garner the benefits of the POSSE philosophy, and be pragmatic about garnering the reach social networks afford without relinquishing control of your content.

Maybe mashups in 2020 are just Jamstack sites

2020-07-06T00:00:00Z

Millhouse voice Remember mashups? They're back, in Jamstack form.

Secret Strava

2020-06-15T00:00:00Z

I built a thing that I'm not sure anyone else will find useful, but it was fun and I learned a few things.

Strava needs better privacy controls

As a social network for athletes, Strava's pretty good. It might be the only good one around these days! But their privacy controls are lacking. You can set each activity to be visible to everyone, only your followers, or just yourself, but it's a manual tag, which can be especially cumbersome if your activities are auto-uploaded from your GPS device.

Ideally you'd be able to set some heuristics on your activities, such that "commutes" stay private, but races and long weekend rides are more visible. This isn't possible out of the box, but you can cobble something together with their API.

So that's what Secret Strava does: every time you upload an activity, it checks against a set of rules on whether it should be made public, such as the distance ridden, if it's tagged as a commute, etc. This is done using a combination of a webhook subscription, their REST API, and some screen-scraping glue.

Building the app

One of the top Strava client libraries is @dblock's strava-ruby-client, so I built an Ruby app so I could use it.

The app itself runs as a few Sinatra endpoints, running on GCP as a Cloud Run app, with OAuth tokens stored in a Fauna database.

A few things I enjoyed about this stack

Cloud Run for serverless containers - I don't like maintaining servers for side projects, but Ruby isn't natively supported with Lambda, Cloud Functions, or any of the other major serverless providers.

So I was happy to hear about Knative, and Google's Cloud Run implementation, which provides similar managed, autoscaling, scales-to-zero-instances to any Docker container.

It was pretty neat to take my running app from my workstation, and push to GCP just by creating a Dockerfile, without having to make any other app changes.

Database as a service - This has gotten a lot easier since I last had to persist anything. Fauna's model lets you create a store and query it just by defining a GraphQL schema. Even compared to other managed databases this is pretty straightforward.

CI/CD with GitHub Actions - These are still great. I'm not sure I'll go back to anything else for OSS projects.

Stuff I had trouble with

GraphQL queries on the server - Maybe this is better in other server-side languages, but this part was rough. Even using GitHub's graphql-client, I missed out on most of the things I get from Apollo+TypeScript, such as development-time syntax verification. Compared to other ORM approaches it felt very messy.

Ruby - It's been almost 10 years since I wrote any Ruby, so I'm definitely a little rusty, but everything felt sloppy compared to the TypeScript/Java/Go I'm used to coding in.

Without any basic editor assistance like method autocomplete, auto-imports, or syntax errors, I had to rely much more on unit tests (which luckily are still great in Ruby!) and manual scripts.

But Ruby's reliance on building DSLs, and monkey-patching existing classes consistently kept me guessing.

Strava's API, and social APIs generally - As I started to build this out, Strava made some pretty significant changes to their API, with the intention of converting more users into subscribers.

Even prior to these recent change, they've been reducing the power of their APIs, such as removing the ability to change an activity's privacy, so I had to resort to screen-scraping.

I have very little faith that Strava has third-party developers in mind when building their platform, but this appears to be the bargain we've struck for acess to any data from within the walled gardens of a social network.

Tom Scott has a good overview of the tradeoffs we've decided as an industry to make (and RIP to the legendary Yahoo Pipes):

Heroku was ahead of its time

I really liked building and deploying to GCP using Cloud Run. The abstraction for general-purpose HTTP servers feels right, and the auto-scaling means the price is right.

But at the end of the day, I'm not sure it's any more advanced than Heroku's original PaaS product from like, 10 years ago. It's crazy how advanced Heroku was compared to the other toolchains at the time.

Has the last decade of Cloud Native tooling really just been a process of standardizing the offering across other languages, and making the PaaS tooling work with Docker/Kubernetes?

I'd love to know more about what other PaaS offerings are like. From my perspective, Cloud Foundry/Heroku/OpenShift/Knative are all pretty interchangeable. If there's more nuance here, let me know!

and Renderless Web Components

2020-05-04T00:00:00Z

I keep buying dumb Bluetooth devices and hooking them up to websites, but there's just enough boilerplate to make integrating them a hassle. So, I made a little Custom Element to help automate some of the boilerplate.

You'd use <bt-device> like any other HTML element:

<!-- Connects to a Thermos Smart Lid water bottle -->
<bt-device
  service="40fc0000-8a8d-4a32-a455-c1148e24a9f1"
  characteristic="40fc0001-8a8d-4a32-a455-c1148e24a9f1"
  notifications="true"
></bt-device>

And then grab the element and connect up to receive notifications:

const device = document.querySelector('bt-device');
await device.connect();
device.addEventListener('data', evt => {
  console.log(evt.detail.data);
});

And that's basically it! Building this makes it easy to add more advanced, cross-cutting features like automatic reconnect, exponential backoff, and the like.

Renderless Web Components

This is a Web Component, but doesn't use Shadow DOM, Stencil, or any library. In fact, it doesn't render anything to the DOM at all. Rather, it's just distributed as a Custom Element to make for easy integration with just a script tag and some HTML; just like the good old days.

I'm not sure if this pattern has a formal name, but I've been calling them "renderless web components". I've seen a handful of these in the wild, such as Polymer's iron-ajax for making Ajax calls, or app-pouchdb-query to handle database queries.

I'm no React expert, but there are a few interesting renderless components out there, such as powerplug. Components that delegate to a render prop (such as Downshift) seem to be operate with a similar philosophy.

For web-facing libraries I quite like the idea of building them with HTML, rather than JavaScript, as the starting point. For no other reason, It's pretty neat getting Bluetooth functionality in a browser without leaving the "HTML" tab in JSBin! And it fits in nicely with declarative, component-based architecture; I've integrated this component into Angular and LitElement apps super easily.

I'm not sure all web utilities can be built this way, but it's something to consider when building your next library!

Going Dark (Mode)

2019-10-14T00:00:00Z

Inspired by Jeremy Keith's post on implementing Dark Mode on the web, I set it up on my blog. It's really easy!

The diff nearly fits on one screen; most of which is converting colors over to CSS custom properties. Once that's configured, Dark Mode is easy as pi:

@media (prefers-color-scheme: dark) {
  :root {
    --primary-color: #efefef;
    --secondary-color: #1a1a21;
    --link-color: #1f8dba;
  }

I'm developing on Windows these days, which has supported Dark Mode for a while. So testing it out was pretty easy:

Building Fast, Tiny GitHub Actions with Go and Docker

2019-09-23T00:00:00Z

GitHub Actions are sweet! It's still in beta but it's a great way to automate tasks after things happen like code pushes, comments on Issues, pull requests, etc.

As an author of a GitHub Action, I'm really enamored by the architecture: each "build" is run on a virtual machine in Azure, with each action running inside a Docker container that GitHub executes. As Kyle Daigle mentions on a Changelog podcast::

As a small developer my entire side business could be a Docker container. Not running it, not supporting the payment for it, just a Docker container.

GitHub does this by building and executing your Docker container on the fly. So if you want your Action to finish quickly, you'll want to spend some time optimizing.

These are lessons I've learned building an action to execute a Particle function. I'm not an expert, but they worked for me!

Optimization 1: Use a container deployed to Docker Hub

You can reference an action by pointing to a GitHub repository containing its source:

# Compiled just-in-time
uses: mattdsteele/particle-action@master

But, the first step for each of your user's workflow will be to checkout the action's source and compile the container. This takes time!

So, if you build the container and publish it to a Docker registry like Docker Hub:

# Precompiled, users just pull the container
- uses: docker://mattdsteele/particle-github-action:latest

For my particular action, this improves builds by over 60 seconds!

Of course, you'll need to build and upload your container each time you push a change to your action. Luckily, there's a GitHub Action for that :)

Optimization 2: Trim your Docker images

When referencing a Docker registry, GitHub has to download the image each time. And if you're not careful, Docker containers can be massive, which can slow down how fast your action executes. So the smaller you can make your container, the better.

Don't rely on a big parent image

My first version of the action relied on curl, which even on a large-ish base like Debian wasn't available by default. So here's where I started:

$ docker images
REPOSITORY                SIZE
particle-bash             90.8MB

Switching from Debian to a smaller Linux base like alpine is a great way to start optimizing:

REPOSITORY                SIZE
particle-alpine           12.6MB

Better!

Build a self-contained executable in Go

Going further, we can use Go to build a single, self-contained executable that doesn't rely on any operating system tools! So, I rewrote the action, switched the base to FROM golang:latest, and got this:

REPOSITORY                SIZE
particle-golang           810MB

Yikes! This is due to Docker images being additive; so the container includes the entire Go build toolchain.

Use a Docker multi-stage build

Let's fix this by setting up a Docker multi-stage build, where we build the Go binary in one stage, and copy it into the empty "scratch" image:

# build stage
FROM golang:alpine AS build-env
RUN apk --no-cache add build-base git gcc ca-certificates
COPY go.mod particle.go /src/
RUN cd /src && go build -o main

# final stage
FROM scratch

COPY --from=build-env /src/main /
COPY --from=build-env /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
ENTRYPOINT ["/main"]

(I also had to copy over the SSL certificates to the scratch image, since I was making HTTPS calls.)

REPOSITORY                SIZE
particle-scratch          7.27MB

Nice!

Optimize the Go binary

We can optimize the image size further by optimizing the Go binary with a couple flags:

RUN cd /src && CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags="-w -s" -o main

Another improvement:

REPOSITORY                SIZE
particle-scratch-optim    5.33MB

In total, these optimizations took my GitHub action from a 6+ minute run down to 30 seconds. Pretty sweet when you're waiting for the action to complete to get candy.

Why I Built Blumhouse to Purge My Twitter History

2019-03-20T00:00:00Z

Or: why would I delete gold like this?

I've been on Twitter for eleven years and in that time amassed a trail of thousands of tweets. I couldn't tell you what 99% of them are about, because Twitter is meant to be ephemeral. Unless you're Milkshake Duck.

Every week or two someone's old Twitter history gets weaponized against them. Sometimes it's Ken Bone or Josh Hader's history cropping up after achieving minor Internet fame.

But more often it's the latest form of oppo research; wherein trolls like Mike Cernovich dig up old tweets, and use them in bad faith to smear their ideological opponents. Trevor Noah, Sarah Jeong, Patton Oswalt - all fell victim to this in the last year.

So yeah, I want Twitter to behave more like Snapchat or Instagram Stories.

Twitter will probably never provide this out of the box. So I built Blumhouse, an app that purges (get it?) old tweets from your timeline. Before the purge, it also archives them in a private database and generates a static website for your old takes to live out their lives in privacy.

Turnkey Userland

There are a number of prebuilt tools to this problem. TweetDelete is a one-button approach that requires zero programming. There are also self-hosted tools like timeliner or ephemeral that provide archive or delete capabilities.

You can also request an archive of your data from Twitter, if you like messing with giant zips of XML.

But in true NIH fashion, none of them did exactly what I wanted so it became my latest side project. This also gave me an opportunity to play with Google's Cloud Platform, which recently announced Serverless support for Go, among other goodies.

But if you're cool with setting up Twitter API keys and a bit of cloud provisioning, feel free to run this yourself. Details are in the README.

This really should be built-in

The system isn't perfect; my tweets are still in the Internet Archive, the Library of Congress, and probably a bunch of other systems. But at least folks would have to put in a little work to root around in the past.

Ideally Twitter would provide this feature. Maybe something like "Hide all tweets older than X days from your timeline". Ideally old tweets would be purged from Twitter entirely.

There's no intrinsic need for this lengthy paper trail. Even if your data is the product, Twitter's whole appeal to advertisers is centered around the right now, not about my ephemera in 2011.

Other systems come closer - Mastodon lets you specify per-toot privacy levels. If these private settings were editable, you could write a simple script to mark old posts as private.

Toolchainless

2019-02-26T00:00:00Z

I have a complicated relationship with build tools. I like TypeScript, Gulp, and the like. I think that compilers are the new frameworks. Heck, even this static site is built with Gatsby; an overcomplicated toolchain to generate HTML files if there ever was one.

But build tools are a Roach Motel in your stack: once you add them in, you're unlikely to ever abandon 'em. And it's never been easier to start a project with a ludicrously complicated toolchain. You're just one npx create-react-app away from a running project, powered by 1023 dependencies in node_modules. Easy, but not simple.

That's the bargain we've made to improve our ergonomics. React and Vue and Angular are so incredible that we use them in spite of their toolchains, not because of them. But I'm excited about new methodologies that are breaking that link, and letting us develop without tools for the first time since the good ol' jQuery days.

You might not need build tools

Some tools have gone away because browsers got better. With features like CSS Custom Properties, you might not need Sass anymore. And you can decompose your app into ES Modules and load as needed using import/export statements; no Webpack or Rollup required.

Some build tools went away because your editor got smarter. I love TypeScript, but Visual Studio Code's inference works pretty damn well for JavaScript as well; providing autocomplete and lots of other goodies.

Some build tools can be removed because CDNs are making a comeback. unpkg delivers up all of NPM via a simple URL, so you don't have to pull in Webpack just to load a dependency.

And new libraries are taking advantage of the toolchainless approach. Take htm - an in-browser project that uses ES6 tagged template literals to generate Hyperscript in the browser. You can drop it into a Preact app and remove the JSX transpilation step:

render({ page }) {
  return html`
    <div class="app">
      <${Header} name="ToDo's (${page})" />
      <button onClick=${() => this.addTodo()}>Add Todo</button>
      <${Footer}>footer content here<//>
    </div>
  `;

In fact, you can get a full Preact + htm project up and running with a single import statement:

import {
  html,
  Component,
  render,
} from 'https://unpkg.com/htm/preact/standalone.mjs';

Drop that into a script tag in your index.html file and you've got a working Preact app that can run anywhere.

Pizza Compass

I recently rebuilt an old project, Pizza Compass, a PWA that points you to the closest pizza. The app I cloned claims to be the most important app ever made, which I can't deny.

The first version of Pizza Compass was built in 2013 using jQuery, before toolchains took over. When I decided to rebuild it using modern UI components, I went with a toolchainless approach.

Built with the Preact + htm stack above, the codebase is modern and clean. And it felt great. I can still build with components:

const PizzaCompass = ({ loc, heading, currentLoc }) => {
  const b = bearing(currentLoc, loc);
  const headingDelta = 180 - (heading - b);
  const distance = Math.floor(distanceFrom(currentLoc, loc) * 10) / 10;
  return html`
    <div class="app">
      <header><h1>${loc.name}</h1></header>
      <${Pizza} rotation=${headingDelta} />
      <footer><h1>${distance} km</h1></footer>
    </div>
  `;
};

But I'm not beholden to any toolchain. Builds are instantaneous because there's nothing to build. The project doesn't even contain a package.json file. Hell, I can run the whole thing using Python's SimpleHTTPServer.

The whole thing is 150-ish lines of Preact code, and when I'm done I can push to GitHub and have Netlify deploy the folder directly. It takes less than a second.

How far does this scale?

I don't know. Eventually you'll probably reach a point where you want to add in tooling for asset optimization, or smarter bundling, or supporting IE11.

But for a weekend project like this, it worked great. And if it ever gets big enough, I can check in the tooling Roach Motel then.

I like this approach because it aligns with the Principle of Least Power. Not every project needs to have 1023 dependencies in their node_modules. And the more you default to using an opaque toolchain for everything, the more likely you'll get bit with opaque errors.

Toolchains are a chainsaw. And these days, you don't have to use a chainsaw for everything.

Web Components Aren't Weird Anymore

2018-08-08T00:00:00Z

I gave a talk at Barcamp Omaha on Web Components:

In it, I tried to first answer the question: why aren't Web Components as popular as React? They both offer reusable components; building blocks which let you assemble sites easily and without rework. But React (and Vue, Angular, etc) took off, and Web Components never did. Why aren't we Using The Platform (tm)?

There are a number of factors, but my take: for nearly their entire existence, Web Components were super weird to use.¹

A few years ago, to get a Web Component on the screen, you had to:

Load up four giant (and slow) polyfills, because only Chrome implemented it natively
Use them via the Polymer framework, which looked different than your current codebase and didn't interop with it easily
Pull them in via bespoke (HTML imports) and outdated (Bower) methods
Bundle them using Vulcanizer and other tools that don't work with Webpack

This sucks! And even if you were sold on the promise of Web Components, a person can only take so much weirdness before they give up and move onto a reliable toolchain.

But over the last year, Web Components have slowly lost their weirdness. Because the controversial parts of the spec were jettisoned:

They've been implemented in all mobile and most desktop browsers
You can use standard tools like NPM and Webpack to build and publish your components
They interop seamlessly with most frameworks
Your framework probably exports them (Angular, Vue, and Dojo do natively; and React can with a wrapper)
There's lots of "Web Component Native" frameworks beyond Polymer, including my favorite Stencil

We're finally at the point were it's easier to add a Custom Element into any app than it is a React component. Now that's weird.

1. Essentially, this entire talk is a rehash of the "Weirdness Budget" concept the Polymer folks talk about here.

Rube Goldberg Machines

2018-04-16T00:00:00Z

Lately, 80% of my YouTube usage has been watching Rube Goldberg machines. Here's a few of my favorite:

The Blue Marble - This entire channel is so well done.

The Cake Server - I love the mechanics as metaphor here. The food theme is infused throughout, and never ceases to delight.

Wintergatan Marble Machine - Not a Rube Goldberg machine per se, but I think it embodies the essence of the concept. As Dave Rupert notes, "It’s raw, creaky, clumsy, and beautiful."

Welcome to RSS Club

2018-04-15T00:00:00Z

RSS is the most exciting technology of 2018.

I don't it's that much hyperbole. A decade after its heydey, RSS embodies some of the best parts of the Web.

Dave Rupert coined the idea, and is collecting sites with RSS-exclusive content here. You should join. It's fun.

I presume you've already got an RSS setup you like. I recommend NewsBlur. I'm happy to pay for it.

Why RSS Can Save The World

From Wired's It's Time For an RSS Revival:

"There are multiple approaches to connecting to news. Social felt pretty interesting at first, but when you mix social and algorithmic, you can easily get into these noise bubbles, or areas where you don't necessarily feel 100 percent in control of the algorithm," says Edwin Khodabakchian, cofounder and CEO of popular RSS reader Feedly. "A tool like Feedly gives you a more transparent and controllable way to connect to the information you need."

I really like this sentiment. Most RSS feeds operate on a "pull" model, compared to social media's algorithmic "push" approach. And push approaches are designed to maximize engagement by constantly recommending new, "interesting content".

This is dangerous. From Maciej Cegłowski:

One problem is that any system trying to maximize engagement will try to push users towards the fringes. You can prove this to yourself by opening YouTube in an incognito browser (so that you start with a blank slate), and clicking recommended links on any video with political content. When I tried this experiment last night, within five clicks I went from a news item about demonstrators clashing in Berkeley to a conspiracy site claiming Trump was planning WWIII with North Korea, and another exposing FEMA’s plans for genocide.

RSS has technical defects, but social media is defective by design.

Lately, folks have been recommending "hacks" to make Twitter friendlier and less damaging. Most of these, like "demetricating" or disabling retweets, are built into RSS. So why not just use the better tool?

Join The Club

As Dave mentioned, there are only a few rules to RSS Club:

1st rule of RSS Club is “Don’t Talk About RSS Club”.
2nd rule of RSS Club is “Don’t Share on Social Media”.
3rd rule of RSS Club is “Provide Value”.

I use Jekyll, with essentially the same setup as Dave. Put rss_only: true in a post's front matter, and filter it from your main page.

JavaScript Gardening with the Particle Photon

2018-03-18T00:00:00Z

The last few years I've been trying my hand at starting a vegetable garden from seed. Previous years have left a lot to be desired; normally I end forgetting to water my anemic-looking seedlings, feeling sad, and harvesting a single ear of corn at the end of the season. 🌽

This year, I want to have some data when things go wrong. And I wanted to see what's new in the new Geocities. So, time to build some Arduino-based sensor systems in JavaScript!

The Sensors

I wanted to keep track of the soil moisture as the seedlings were growing indoors. This can be done with a Soil Moisture Sensor, which just sends a reading to an analog GPIO pin.

I also picked up a few DHT11 thermometer/humidity sensors. These are a little tricker to use directly, but the Adafruit DHT Library make it easy to collect readings.

Raspberry Pi and Photon Farming

The microcontroller world has advanced quite a bit since I last checked! In 2014 I used the Johnny-Five library to get sensor data into a Node app, but it required running Node on a "full" computer, then connecting to a USB-tethered Arduino. Tethering, ugh.

Nowadays, you can run Node directly on a Raspberry Pi, and connect to the Pi's GPIO pins using a J5 plug-in. Sinc the Pi 3 has integrated Wi-Fi, the only cord you need is power.

Unfortunately, the Pi doesn't have analog GPIO inputs, which are needed for the soil moisture sensor. A few minicomputers such as the Next Thing C.H.I.P. can do the job.

But in the end I was smitten with the Particle Photon, a Wi-Fi-enabled Arduino. It's super neat and crazy cheap. You connect it up to your wireless network, and emit events to the Particle cloud. They have a JavaScript SDK, which lets you receive these events in a Node app.

I can't tell you how satisfying it is to remotely flash new code to an Arduino while in another room, and have your app get new data 30 seconds later. You can install Arduino libraries, and it's low-powered enough that a 10k mAH USB battery runs the whole contraption for 3 days or so. Plus, it fits in a breadboard.

At least I'm sprouting a dataset

To handle the sensor data, I pulled in RxJS, which I've done before and find to be a super-solid approach. This let me take the Particle-provided EventEmitter, convert it to an reactive stream, and create a new stream out of the last 5 readings, averaged:

import { getEventStream } from './photon-stream';

const event$ = getEventStream();

const stream = (eventName: string) => {
  return event$
    .filter(d => d.name === eventName)
    .map(d => {
      return { value: parseFloat(d.data) };
    })
    .bufferCount(5)
    .map(vals => {
      return vals.reduce((prev, curr) => prev + curr.value, 0) / vals.length;
    })
    .map(value => {
      return {
        value,
        time: new Date(),
      };
    });
};

export const soil$ = stream('soilMoisture');

This went straight into an InfluxDB instance, and mapped using Grafana. The whole thing is running in a few Docker containers on a DigitalOcean VPS, which was pretty nice to setup.

Seems like a lot of work

It was a fun side project, but since I can now check on the sensor levels at all hours, I'm probably not reducing the number of mental cycles I'm thinking about the seedlings. And in the time it took me to build and test the hardware, I probably could have tilled my yard or done something more directly beneficial to my garden.

But I'm thinking it can pay off in the long-run. Once the seedlings get moved outside to the garden, I can transplant it outside, and control a solenoid valve to automate watering throughout the growing season.

So yeah, JavaScript hardware is very possible, still cool, and has only gotten easier in the last 5 years. Go and build something cool with it!

You can view the code here, and see the charts here (user/user).

Inspiration comes from John Hobbs's homemade chicken incubator.

The Neverending Side Project

2018-02-06T00:00:00Z

I love side projects. I keep track of what I'd like to work on in a GitHub repo.

Most side projects don't need much oversight. That's kind of the idea: you spend a few hours building something fun, blog about it, and move onto something else.

But I think there's space for another kind of project: one that spans years. The formula's simple:

Build a site/app/whatever
Let it sit on a shelf for a year
Dust off the cobwebs and make some enhancement, switch frameworks, whatever
Repeat

The Codebase of Theseus

One project I've been building is a statistics app for Super Bowl Squares, a mostly-random gambling endeavor. It's a dumb game, worthy of a dumb side project. You can view the latest version here.

I built my first version of the app back in 2011 as a fresh-faced front-end dev, with nothing but spaghetti jQuery.

Over the years, I've migrated and rebuilt the app numerous times:

Original jQuery version (2011)
Added a build toolchain with Webpack and npm (2014)
Rewrote using AngularJS 1.x (2015)
Migrated to Angular 2 beta (2016)
Rebuilt with TypeScript and the Angular CLI (2018)

Inheriting Your Legacy

A long-lived side project gives you the chance to confront your old habits and see how far you've progressed. I started with jQuery because that's all I knew. I can still see parts of me in the old codebase, but I also see how my coding style has evolved.

A long-lived side project also gives you breathing room to ask how much stock to put into trends. My original jQuery app still loads faster, has 60% less code, and (to my mind) is more understandable than my latest version built atop Angular 5. Have I actually made things better? Have we as an industry?

I didn't get to all the updates I wanted to make to the app this year; I wanted to add some Redux-style state management with ngrx. But it's okay, I know what I'll be working on next year.

Light Up Your Bike with Web Bluetooth and StencilJS

2017-12-17T00:00:00Z

Every year the Omaha cycling community puts on the Bike De'Lights ride: an opportunity to illuminates your bike and tour the city's Christmas lights. And every year I try to find a new way to drive a five meter RGB light strip.

This year I've been most excited about Web Bluetooth, and Web Components. Each technology has the opportunity to open previously closed ecosystems, and make them broadly accessible, and I wanted to use them to build a lighting system that anyone could control with their phone; no apps required.

Hardware

The controller and batteries fit in the bike's saddlebag, and the light strip had sticky tape that stuck to the frame. It's a pretty simple setup.

Reverse Engineering the Bluetooth Controller

The Bluetooth LED controller was intended to be controlled via a native Android app, but it's garbage. But since it's a Bluetooth Low Energy device, it's hackable!

To reverse engineer the controller, I ended up using the strategy described in Uri's post: recording interacting with the app and playing it back on my laptop using Wireshark.

I found one Bluetooth service/characteristic you could send two types of commands via Uint8Arrays: an arbitrary RGB color, or a preset color scheme (mostly fades).

The code to control the device is available here.

Building with Web Components and StencilJS

Web Bluetooth only works on Chrome devices, so building the controller app using Web Components was an easy decision and used no polyfills.

I've been pretty enamored with StencilJS lately, and built the app with a handful of web components. Even the code to control the Bluetooth device is a <bluetooth-strip> web component!

Stencil made it really simple to build a fast site, and fast. Since it's all TypeScript-based, I could create a mock component that implemented the same interface as the <bluetooth-strip> component, and test the app without having to continuously connect to a real device.

Broadcasting the URL via the Physical Web

Once I built the site, I deployed to a static host, but my goal was to let other riders control my lights using their phone. And it's a huge bummer to have them open Chrome and type in a giant URL.

This is where the Physical Web comes in - a device can broadcast a URL, and nearby devices can be notified and interact with it.

I installed Node on a Next Thing CHIP, and used the eddystone-beacon module to broadcast my StencilJS URL.

Fun fact: the USB battery was way larger than the computer it was powering!

Code is available here.

Results

Overall the project went really well! Unlike previous lighting systems, no soldering was required, and I didn't run into any technical glitches during the ride. These two are likely related.

One issue I noticed was that folks don't understand how the Physical Web stuff works. I told a few cyclists about my setup, but had to work with them to pull out their phones, and show where the link appeared in their Notifications dropdown.

My primary goal with the project was to let others control the lights, and I succeeded with that. My secondary goal was to have a random cyclist connect and control my lights. That didn't happen, and I think the issues there are intrinsic to the design of the Physical Web.

You can play with the site here.

See previous lighting projects here and here.

Every Java Developer Should Learn TypeScript

2017-11-01T00:00:00Z

I gave a talk to the Omaha Java Users Group about TypeScript:

I think there's a lot of things TypeScript has to offer Java developers, namely:

Static Typing - This one's obvious; if you're used to a compiler providing thousands of tiny continuously-running unit tests, you're going to miss that on the front-end.

But what's even cooler is TypeScript's features of incremental and inferred typing. You can achieve full type safety in this code:

class MathFns {
  static for(val) {
    return {
      square() {
        return val * val;
      },
    };
  }
}

const fns = MathFns.for(5);
console.log(fns.square());

By adding a single type token:

class MathFns {
  static for(val: number) {
    return {
      square() {
        return val * val;
      },
    };
  }
}

const fns = MathFns.for(5);
console.log(fns.square());

Inferred typing means you add drastically fewer tokens than you would to make Java code typesafe; conservatively you'd have to add least six type tokens in an equivalent Java program. Incremental typing means you can convert existing JavaScript code into TypeScript without it being all-or-nothing.

I also appreciate TypeScript's structural type system, which also provides strong type-safety without having to declare that a function implements a particular interface, or extends from a parent class, etc. This article goes deep into the differences.

It's also easy to integrate into a Java workflow, with Maven and Gradle plugins if you wanna use them, as well as tooling for Eclipse, IntelliJ, and other IDEs.

I'm still getting over the cognitive dissonance that a tool from Microsoft, created by the C# language designer, might be the best way to ease Java developers into the front-end. But now that I'm used to static typing, I get angry when I have to go back to vanilla JavaScript, and I have TypeScript to thank for that.

Regardless of what Rich Hickey says.

Marginal Revolutions

2017-03-26T00:00:00Z

Weekend illnesses are a great opportunity to work through an Instapaper queue. I'll blame the head cold, but I see connective tissue across several articles.

First, Scott Alexander makes a compelling case that Trump supporters aren't immune to logic and truth. One presumes they are, because when's the last time you debated a Trumpist and changed their minds?

But that's because these debates usually entail a handful of drive-by "CHECKMATE ATHEISTS" encounters. Rather, one should focus on a longer-term, convince-one-person-over-five-years approach:

Am I saying that if you met with a conservative friend for an hour in a quiet cafe to talk over your disagreements, they’d come away convinced? No. I’ve changed my mind on various things during my life, and it was never a single moment that did it. It was more of a series of different things, each taking me a fraction of the way. As the old saying goes, “First they ignore you, then they laugh at you, then they fight you, then they fight you half-heartedly, then they’re neutral, then they then they grudgingly say you might have a point even though you’re annoying, then they say on balance you’re mostly right although you ignore some of the most important facets of the issue, then you win.”

Instead, we throw up our hands and proclaim we're in a post-truth era, because the longer-term approach is hard:

Improving the quality of debate, shifting people’s mindsets from transmission to collaborative truth-seeking, is a painful process. It has to be done one person at a time, it only works on people who are already almost ready for it, and you will pick up far fewer warm bodies per hour of work than with any of the other methods.

Next, in The Heroism of Incremental Care Atul Gawande argues that our health care valorizes specialties with obvious, immediate results (think: surgery, orthopedics) over longer, more incremental approaches (internal medicine, immunology). And yet, the longer-term approach saves more lives:

In the United Kingdom, where family physicians are paid to practice in deprived areas, a ten-per-cent increase in the primary-care supply was shown to improve people’s health so much that you could add ten years to everyone’s life and still not match the benefit.

The internists' secret weapon: access to way more data:

“It’s the relationship,” they’d say. I began to understand only after I noticed that the doctors, the nurses, and the front-desk staff knew by name almost every patient who came through the door. Often, they had known the patient for years and would know him for years to come. In a single, isolated moment of care for, say, a man who came in with abdominal pain, Asaf looked like nothing special. But once I took in the fact that patient and doctor really knew each other—that the man had visited three months earlier, for back pain, and six months before that, for a flu—I started to realize the significance of their familiarity.

Finally, my hometown of Omaha is looking to solve its public transit problems by building a new $150M streetcar system. It's a massive, big-bang solution to public transit woes, which is odd, given that Nebraska spends less on public transit than any other state.

Similarly, the city has let its road infrastructure crumble so much over the years that it's converting some roads back to gravel, and declaring them "unmaintained".

In each of these fields, the more successful approach is one that emphasizes slow, incremental changes. No one doctor appointment resolves a chronic condition. But the aggregate of routine visits captures more data, and solves more problems, than the alternative.

You see it in software, too. Most orgs celebrate the dramatic rewrite of an aging codebase. But to accomplish it they'll move resources away from longer-term projects, like making testing and continuous integration more consistent across the enterprise.

Making those long-term investments is hard. And it's an uphill battle. When short-term solutions work they're more dramatic, are easier to measure, and are more fun. So of course folks prioritize them.

But it's not just that the longer-term approach is more effective. It's that the alternative is often actively harmful.

If you don't think you can rationally convince a Trumpist they're wrong, then what's your alternative? Sophistry, emotional appeals, or shutting down discourse a la the antifa folks? But these approaches can be wielded by both sides; your side will only win by coincidence of having more effective storytellers. And Trump's camp sure seems to have more expertise in this area.

So try embracing the slow burn. This post might not convince you of its efficacy, but I'll leave it to you to find the one weird trick that does.

Web Bluetooth Is Your New Squeeze

2017-03-13T00:00:00Z

One of the side-effects to living that Internet-Of-Things lifestyle is that you end up with a lot of pseudo-smart devices, mostly collecting dust. I wanted to see if Web Bluetooth could breathe new life into them.

Turns out, hacking Bluetooth Low Energy toys is way more fun than actually using them. I spoke at NebraskaJS about what I learned, and why I'm excited about hooking Bluetooth up to the Web.

Getting Started

Watch the talk, then check out these links:

Devices

BBQ Thermometer - Code

I picked this one up from Meh. No specs available, but you can reverse engineer the temperature if you know some Big Endian notation.

For part of the talk I advanced my slides by dropping the thermometer into glasses of hot & cold water. I called it TDD - Thermometer-Driven-Development. Uncle Bob was right, TDD truly is the pinnacle of professional software development.

Elfy Smart Light - Code

The Elfy is a color-changing night light I picked up from a rando site in China. It looks the part. You can hit it, and it'll alternate between a set of colors.

For one demo, I hooked up an <input type="color"> to the Elfy. I also wired it up to change color every time the BBQ Thermometer registered a new temperature.

Jessica Codr and I also built a timer for Toastmasters competitions. This app is a great example of using Web Bluetooth for progressive enhancement. For browsers that don't support Web Bluetooth, the app shows the color with a <div> on the screen. If there's a Elfy nearby, you also get it on the device.

No specs on the Bluetooth services, so I had to reverse engineer it using Wireshark, as described in the video.

Bicycle Speed/Cadence Sensor - Code

I used a Wahoo Bluetooth/ANT+ sensor. Luckily this used well-known Bluetooth GATT profiles, so hooking it up was a piece of cake; just required a little math.

I took some cues from Max Goodman's Bicycle.js talk. But his Progressive Web App was far too useful, so I just built a Flappy Bird clone. You know how we do.

Sphero BB-8 - Code

The hottest Christmas toy of 2015 meets the technical stylings of 1985. Opera's devrel team originally wrote the code to control the toy. I removed the code that made it spin off my desk, and instead just had it change colors. It's a testament to the remix culture of the web.

Building Custom Elements That Work With AngularJS 1.x and Angular

2016-09-08T00:00:00Z

So Web Components! They've recently gotten a lot of love, and a lot of hate.

Let's talk about Custom Elements in particular. This is the technology that lets you drop <relative-time> on a page, and it renders into something usable and pretty. No framework required.

The Web Components proponents propose composing Web Components into existing frameworks, particularly as leaf nodes. So you'd write your Angular/React/whatever app that contained your business logic, and its templates would be made up of Custom Elements, standard HTML, and other Angular/React/whatever components.

I haven't seen this interop story in action (or documented) in many places. In particular, how do you build a Custom Element that'll work in both AngularJS (1.x) and Angular 2+ apps?

Note: I'll be using on the new "V1" version of the Custom Elements spec. Here's a good intro article, and here's a polyfill.

Building The Custom Element (View on GitHub)

Let's make a <countdown-timer> Custom Element.

You give it the number of seconds you want the timer to run, and it'll spit out an event (with a message) when the countdown ends.

We'll use a "one-way data flow" architecture - the element will accept its inputs via properties, and spit out DOM Events for its outputs. This is the architecture Angular uses, (and the recommended approach for modern Angular 1 apps.

Here's the element in all its dumb glory:

class CountdownTimer extends HTMLElement {
  connectedCallback() {
    const template = `
			<button class="countdown-start">Start the countdown</button>
			<span class="seconds-left"></span>
			`;
    this.innerHTML = template;

    // Useful references
    this.button = this.querySelector('.countdown-start');
    this.secondsDisplay = this.querySelector('.seconds-left');

    // Initialize
    this.button.addEventListener('click', () => this.handleClick());
  }

  handleClick() {
    this.updateTimer();
    this.button.disabled = true;
    this.button.innerHTML = 'YOU DID IT';
    this.updateTimer();
    const counter = window.setInterval(() => {
      this.seconds--;
      this.updateTimer();
      if (this.seconds === 0) {
        window.clearInterval(counter);
        console.info('BOOM');
      }
    }, 1000);
  }

  updateTimer() {
    this.secondsDisplay.innerHTML = this.seconds;
  }
}

window.customElements.define('countdown-timer', CountdownTimer);

Not much to it - initialize stuff in the connectedCallback hook, and then add your functionality.

Angular (View Demo)

Consuming this in Angular is pretty straightforward: you use the [prop]="value" syntax to bind to a property, and the (event)="handler()" syntax to bind to events.

A component that uses it might have a template that looks like:

<p>
  <label
    >How long to count down? <input [(ngModel)]="secondsLeft" type="number" />
  </label>
</p>
<countdown-timer
  [seconds]="secondsLeft"
  (countdown-ended)="handleCountdownEnded($event.detail)"
>
</countdown-timer>

One thing to watch out for, if you get an error like this:

Can't bind to 'seconds' since it isn't a known property of 'countdown-timer'.
1. If 'countdown-timer' is an Angular component and it has 'seconds' input, then verify that it is part of this module.
2. If 'countdown-timer' is a Web Component then add "CUSTOM_ELEMENTS_SCHEMA" to the '@NgModule.schema' of this component to suppress this message.

You'll need to add CUSTOM_ELEMENTS_SCHEMA to your @NgModule declaration, via:

import { NgModule, CUSTOM_ELEMENTS_SCHEMA } from '@angular/core';

@NgModule({
  // module boilerplate
	schemas: [ CUSTOM_ELEMENTS_SCHEMA ]
})

AngularJS (1.x) (View Demo)

Out of the box, our Custom Element won't play nicely with AngularJS. There are two problems:

Its templating system binds to an element's attributes, while our component updates the element's properties
AngularJS isn't aware of the events your Custom Element fires, and doesn't hook into the normal & callback bindings

So! We can fix this in a couple different ways:

Option 1: Change the Custom Element

On the input side, we can add some code to our Custom Element that looks for attribute changes. There are lifecycle hooks in the spec for this:

static get observedAttributes() {
  return ['seconds'];
}

attributeChangedCallback(name, oldVal, newVal) {
  if (name === 'seconds') {
    this.seconds = newVal;
  }
}

And then bind to the attribute in our template (note the use of ng-attr to prevent the element from seeing the raw ``):

<countdown-timer ng-attr-seconds="{{$ctrl.secondsLeft}}"></countdown-timer>

On the output side, we can bind to the event manually in our controller, and wrap it in $scope.$apply() to make sure a digest runs:

$element.on('countdownEnded', e => {
  // If we don't do a digest, this doesn't get picked up immediately
  $scope.$apply(() => {
    this.message = e.detail.message;
  });
});

But this has limitations:

Binding to attributes means you're limited to passing String expressions to your Custom Element
You lose encapsulation by binding to the Custom Element's events in your Angular controller. If you wanted to write idiomatic AngularJS, you'd have to create a wrapper component that provided an on-countdown-ended attribute, and run a digest manually. But now you're writing wrapper components for every Custom Element you import, and we were trying to get away from that!
Plus, you're modifying your supposedly framework-agnostic Custom Element to satisfy a particular framework

So yeah, that sucks. Hope there's a better way!

Option 2: Use a Glue Directive

Should've read ahead in my post, I guess.

Rob Dodson wrote a set of directives to help Custom Elements interop with Angular 1. It's labelled for use with the Polymer project, but it'll work for any Custom Element, including ours.

Since we're using a one-way data flow, we can add the ce-one-way directive to our AngularJS template:

<countdown-timer
  ce-one-way
  seconds="$ctrl.secondsLeft"
  on-countdown-ended="$ctrl.countdownEnded()"
></countdown-timer>

We did have to make one tweak to our Custom Element to make this work: we renamed the Event to countdown-ended, to match the naming pattern library expects. Could be worse, I suppose.

Framework Migration, and Framework Independence

At work, we maintain an enterprise component library, like many large orgs do. It's currently built as a set of Angular 1 directives. As we begin to investigate Angular, we want to make the migration process smooth as butter.

We could rebuild the component library as a set of Angular components, and AngularJS interoperability could come through the ngUpgrade module, probably.

But this begs the question: what happens when we ditch Angular? Our components would again be dependent on a single framework, and they'd have to be rewritten once again.

Building your company's component library on Custom Elements solves multiple problems.

Short-term, it helps you migrate from AngularJS 1.x to Angular, since both applications can use the same component library.

Long-term, it helps isolate your company's component library from the Sturm und Drang of front-end frameworks. So long as a framework interacts with the DOM (and they all do), they can use your components.

So you can support that weird Ember team, the stodgy server-rendered JSP folks, and even the framework-less static page hipsters.

So yeah, Custom Elements are awesome and you should give them a looskie. As Dion Almaer noted: How would the component landscape look if we weren’t all rebuilding our own houses?

How to Run a Board Game Library At Your Conference After-Party

2016-08-28T00:00:00Z

We just wrapped up another sold-out NEJS CONF. It was pretty great, and we learned a lot our second year around.

One of the changes we made this year was to offer board games at the conference after-party, rather than just setting attendees loose with dinner and drinks.

I thought it worked pretty well, but I haven't seen many other tech conferences run with this concept. Here's how we set it up, what we learned along the way, and how you can incorporate games into your conference.

tl;dr:

Collect games from your friends' libraries
Take some simple precautions to make sure nothing gets lost
If you've got the budget, raffle off a couple games at the end of the night

Why board games?

YASSSSS! Collaborative game play at #NEJSConf afterparty. pic.twitter.com/OG9MdMNdKD
— Andrea Goulet (@andreagoulet) August 26, 2016

Because they're catalysts for a memorable after-party:

Games facilitate new connections among attendees - We encouraged attendees to interact with out speakers and conference organizers, and games work marvelously for this. A first-time conference attendee might worry they'll run out of things to talk to an experienced speaker about, but they'll probably be more comfortable playing a game of Codenames with them, and it'll likely be more memorable.

It's an accessible alternative to a boozefest - Lots of conferences market their evening events as an opportunity to blow off steam. This often translates to: we have an open bar, so get shitfaced on our dime. These can be fun, but they exclude entire sections of attendees if that's the only entertainment you offer. Amanda Harlin reminded us of this at our own conference last year, and we took it to heart.

It's cheap and easy to organize - Looking over the other budget line items for the after-party, this was dirt-cheap to setup. Compared to renting the venue, and organizing food, bar, and decorations, providing board games is a rounding error on both your money and time.

Collecting the game library

preeeeety sure @nejsconf is a board game conference pic.twitter.com/Vuq4NefEo7
— Matt Steele (@mattdsteele) August 26, 2016

If you're a board gamer, you might have a personal library you can use to see the board game table. If so, great! But if you're like me and mooch off your buddies' collections, you'll probably need to ask others to borrow their games for the evening.

Talk to your friends and ask if they've got a set of board games they'd be willing to lend out. Given the rising popularity of board gaming, you probably have a few friends with the hobby.

If any of them are planning to attend the conference, even better! We offered to discount donors' admission as a way of saying thanks.

Good kinds of games

You want to get games that meet a few criteria:

Short - preferably under an hour to setup and play.
Easy to learn - Folks don't want to pour through manuals, they want to be unboxing and playing in a few minutes.
Easy to play - Hardcore strategy games like Power Grid are tons of fun, but probably overkill for most attendees at this point in the day.

We also grabbed a few train-based games, to match our conference's theme (the venue was an old railroad station).

Here's some games that fit this criterion (and got played a lot):

Dominion - A fun deck-builder that'll scratch the itch of the ex-Magic: The Gathering players that sneak into your conference.
Wits & Wagers - A trivia game where you can win even if you don't know any answers? Sold.
Pandemic - Cooperative games are particularly great at generating tables of cheers and high-fives.
Concept - A great word association game. Another nice feature: folks can join at any time and don't have to wait for a new table to start up.

Here's the complete list of games we collected for NEJS CONF, if you're curious.

Keep them safe

Since you'll likely be consolidating a collection worth hundreds or thousands of dollars, folks will want to know that it's in good hands. And bad things can happen, unfortunately. A few things you want to have a plan for:

Make an inventory - We used a Google Spreadsheet to keep track of people's games. I made the spreadsheet a few days before and shared it with the game donors, and then re-inventoried all the games during the conference.
Have a plan for duplicate games - There were a few games the library had multiple copies of. Make a note of these, and mark them appropriately. Ask the owners if there are any expansions, sleeved cards, or other identifiable markings in the box.
Return games at end of the night - Donors want their games back sooner rather than later, and you don't want to keep a boatload of games in your car, anyhow.

Games Ambassadors

Geekway To The West

You'll want to recruit a Game Ambassador or two - these are volunteers who facilitate play among non-gamers. We recruited the folks who donated their personal library to be Ambassadors during the after-party. You can also ask for volunteers at your friendly local gaming store.

This is the secret sauce to running a successful game table. Most attendees won't be gamers, and don't want to read through pages of rules when they could be socializing. Having a Game Ambassador around to help run a table removes this risk, and helps introduce more games to more people.

We probably doubled the number of tables playing board games as a result of our Ambassadors floating around, checking out games, and teaching them.

Giveaways

We worked with Spielbound, our friendly local board game cafe, to purchase a dozen board games, and we raffled them off at the end of the night. This was the most expensive part of the event, and ran about $300.

You'll want to announce the raffle rules ahead of time, and have answers to common questions. For example: Who's eligible for the raffle? Do you get to choose your game or is it first-come-first-serve? Do you have to be present for the raffle to win?

NEJS CONF board game raffle info: pic.twitter.com/8CwXSPkEdd
— NEJS CONF (@nejsconf) August 26, 2016

Be aware of gaps in the drawing. For example, we told folks if they attended the after-party, you were registered. We reused a sign in sheet to get drink tickets from the bar, but quickly realized that not everyone was drinking, so our list was incomplete. So, 10 minutes before the raffle, we had to scramble and hand out raffle tickets from table to table. It wasn't pretty.

For the raffle drawing/winners yourself, you can announce them via the PA, or in Twitter/Slack.

(One option: We kept the games in shrinkwrap during the after-party, but you could also use them as part of your games library. This is similar to the Play and Win feature popularized by other conferences like Geekway. We ended up not doing this, as we had enough library games that it wasn't worth the extra hassle to set it up.

View this post on Instagram

Best conference ever? Breaking out the boardgames at #nejsconf with new pals! @NEJS #fridaynightnerdnight

A post shared by Dania Lieberthal (@drlieber) on Aug 26, 2016 at 3:52pm PDT

Then, run the event

At this point, running the library is pretty straightforward. A few other bits to consider:

Shelving - Putting games on tables is lame and you'll run into a mess fast. Get some actual shelves, and put your games on 'em. I picked up these at Lowe's for $30.
Check in/out - If someone wanted a game, we just took their conference namebadge and put it in a box. When they returned the game they got it back. Simple.

Other than that, the event pretty much runs itself. You might have to wait a while for the last table to finish up their games before you can pack up, but otherwise it's easy as pie.

Happy gaming!

@mattdsteele @nejsconf I feel like I've made a mistake by not planning to be there
— Nick (@programmerman) August 25, 2016

The Languages Which Almost Became CSS (Recap)

2016-07-06T00:00:00Z

NebraskaJS recently held a lightning talk night, entitled "Present a Blog Post". Cool gimmmick, bro. And you didn't even need to write the post you were presenting on!

Fortunately, I read a fascinating article by Zack Bloom on The Languages Which Almost Became CSS: a rip-roaring tour through the process by which we got to the styling language we all know and love (for various definitions of 'love').

I also included a couple lessons I think we can take from the whole experience. And if you listen closely, there's a Game of Thrones allusion thrown in near the end.

Thanks to Zack for researching and posting such a cool article!

Upgrading to Angular 2 using ngUpgrade

2016-04-27T00:00:00Z

I spoke at ng-nebraska on how the ngUpgrade module can help bridge the gap between Angulars 1 and 2.

Demo App (upgraded to ng2)

Code on GitHub

Slides

Reacting to Heart Rate and Bike Sensors With RxJS

2016-01-03T00:00:00Z

Recently I hooked up a Raspberry Pi to my bike and made an LED light strip synchronize with my heart rate and pedal speed. I got some great feedback on the project, including "overly complex", "Rube Goldberg wannabe", and "you need another hobby".

One challenge with this project was how to effectively manage sensor data from multiple sources (a heart rate monitor and a bicycle speed/cadence sensor) and have it control a single output (the LED strip). Additionally, I needed to manipulate the data before I could send it to the LED light strip. I also wanted to be able to test my system with "fake" inputs, so I wouldn't have to strap on my heart rate monitor every time I wanted to tweak the app.

It turns out that Reactive Programming techniques are perfect for a use case like this.

Reactive Programming

If you're new to the concept of reactive programming, check out these guides:

RxMarbles - you could just play around on this site and learn most of what you need
Introduction to Reactive Programming course on egghead.io

Creating the streams

Sensor input was grabbed from the ant-plus library. It emits a Node-style Event periodically (every 250ms for the heart rate monitor, and on every pedal stroke for the cadence sensor).

RxJS makes it easy to convert these to a stream:

let Rx = require('rx');

//`sensor` emits 'cadenceData' events periodically
let rawCadence = Rx.Observable.fromEvent(sensor, 'cadenceData');

This produces an Observable stream of cadence events. Each event in the stream gives me a full data object but I only need one element (the currently calculated cadence). I also filter out some noisy events (initially you get some events without the calculated cadence set):

let calculatedCadence = rawCadence
  .map(stream => stream.CalculatedCadence)
  .filter(cadence => !isNaN(cadence));

module.exports = calculatedCadence;

Rendering the stream

The LED lights were represented by a class that stored the light's current state:

class BikeLights {
  constructor() {
    this.intensity = 0; //controlled by heart rate data
    this.color = { r: 0, g: 0, b: 0 }; //controlled by cadence data
  }

  setIntensity(intensity) { //led light strip code }
  setRgb(percent) { //led light strip code }
}

And in the application:

calculatedCadence
  .map(data => remap(data, 25, 100, 0, 1))
  .subscribe(data => lights.setRgb(data));

Here I'm using an Arduino-style remap function to convert my cadence range (25rpm during easy pedaling, 100rpm during fast sprints) to a 0-1 range, which the lights class expects. Then we pass that data to the LED lights.

Faking out the stream

Creating a fake stream for testing was a piece of cake. In a separate file:

//mock-cadence-stream.js

// Start emitting 30rpm, then switch to 90rpm after 5 seconds
let mockedCadenceStream = Rx.Observable.interval(1000)
  .timeInterval()
  .map(e => (e.value >= 5 ? 30 : 90));

module.exports = mockedCadenceStream;

In my production code, I simply commented out the stream I wanted to use:

let cadenceStream = require('./mock-cadence-stream');
//Uncomment for real ANT+ data
//let cadenceStream = require('./cadence-stream');

Simple, but it gets the job done.

A stream of hearts

Getting data from the heart rate monitor was similar but required a few additional RxJS tricks. I used these Reactive functions:

distinctUntilChanged to only emit when my HR changed
debounce as 250ms was too rapid for my use
flatMap and flatMapLatest to convert the event data (beats per minute) into a stream of "heartbeat" events

You can see the full implementation on GitHub.

I guess this was a weekend hack

This was my first experience using RxJS and Reactive Programming, and it shows. I didn't write any automated tests to verify the behavior, but I think it would have been simple to do. You can even mock out (and speed up) the passage of time using a Virtual Time Scheduler.

I also coded up some pretty terrible error handling code to switch the lights to 'idle' mode after 5 seconds without sensor data. After thinking about it for a bit, I'm pretty sure I could have used a standard debounce operator rather than the recursive monstrosity I created.

But even as a non-expert in reactive paradigms I really liked using RxJS! If you need to manage asynchronous events at a higher level than callbacks or setTimeout allow, give it a shot.

Driving an LED Light Strip with Heart Rate and Bike Sensors

2015-12-12T00:00:00Z

I recently partook in Bike De'Lights, Omaha's two-wheeled tour of the Christmas lights on display in Midtown. Cyclists are encouraged to decorate their bikes with battery-powered lights, jingle bells, and get into the festive spirit.

I treat this as an opportunity to build an overly-complex hardware project. Last year I wired up an Arduino to generate remote-control signals to drive an LED light strip. It was fun, but there were some opportunities to make it better (or just further complect the system), which is what I did!

My goal for the project: hook up an LED light strip to my bike frame, and have it respond to my heart rate and pedaling cadence, using the ANT+ sensors on my bike. Specifically, I wanted the lights to change color from green to red as I pedaled faster, and have the lights blink in sync with my heart rate.

I used a Raspberry Pi and an app written in JavaScript to do this. The code is on GitHub. It was rickety, but it worked! Here's how I put it together.

Hardware

The core of the project:

Raspberry Pi Zero and a 4GB micro SD card
5M Waterproof RGB LED Light Strip
USB portable charger and micro-USB cable

To power the LED light strip:

TIP120 Transistors (3x)
8 AA battery holder and 9V snap connector
A handful of jumper wires and a prototyping board

To capture sensor input from the bike:

ANT+ USB receiver
Speed/Cadence Bike Sensor
Heart Rate Monitor
Micro-USB to USB adapter (the Pi Zero has micro-USB inputs)

I had most of these items on hand from previous projects, but had to buy the LED Light Strip and transistors. In total this project uses less than $20 of consumable parts.

Everything fit in my bike's saddle bag; this worked really well due to the super-tiny Pi Zero. I placed the Pi and the circuit board in a makeshift enclosure I made from a to-go container, and held it down using electrical tape.

Schematic

The circuit to power the LEDs is pretty straightforward:

The Pi sends PWM signals to the red/green/blue input wires, and the 12V of batteries goes to the LED light strip.

Also note: the dimensions are off in the diagram - the circuitboard was actually bigger than the Raspberry Pi Zero!

Software

The Raspberry Pi was running Raspbian and a program I wrote in Node.js.

I used Thomas Sarlandie's pi-blaster library to send PWM signals to the LED strip. The library creates a /dev/pi-blaster endpoint on the filesystem, which you write to and it send the signals to the GPIO pins. There's even a Node.js wrapper which made adding it to my app super simple.

To capture ANT+ data, I used Alessandro Vergani's ant-plus library, which emits Node-style events when new sensor data comes in. This worked well, but only the heart rate monitor was supported out of the box. So the majority of the coding I did for the project was implementing the ANT+ speed/cadence sensor into the library. The pull request shows the fun involved there.

The actual code to integrate sensor input and have it drive the LEDs was another fun learning experience. Since the heart rate and cadence sensors produced a stream of events, I pulled in RxJS to treat them as Observables, and used reactive programming techniques to map the inputs into a stream that could "render" an LED strip. That's probably another blog post in itself.

Once I had a working protytpe, I configured a systemd service to start the Node application when the Pi booted up. This was especially useful as I had to reboot the Pi multiple times during the ride.

Running the Lights

Overall, I'd say the results were a solid B/B+. The lights were super bright, and lots of folks complemented them as I rode. It was especially cool to see the lights start flashing like crazy as we climbed a big hill and my heart rate spiked. But some parts could have worked better.

The cadence sensor connection was buggy as hell. About 50% of the time it wouldn't connect to the sensor on startup. I programmed the lights to fade to blue after 5 seconds any pedaling events, for times when I was coasting downhill or stopped. But when the sensor didn't connect, the lights stayed blue until I could stop, reboot the Pi, and try it again. I'm not sure why this happened. My guess is it had to do with interference from other sensors, since the problem was especially pronounced at stops and locations with lots of other cyclists.

I would also have fastened everything down better. Omaha streets are often potholed and bumpy, and there were several times when the batteries fell out of the enclosure, shutting all the lights off. I also just used jumper wires to connect the circuit board to the LED strip, and those fell out a number of times. A little more soldering and electrical tape would have fixed these issues. A shakedown ride or two would have also helped; most of my testing was done on an indoor bike trainer.

Also, I would have have given myself more time to build it out. I was working on the cadence sensor code until 4am the previous night, which probably contributed to the overall bugginess of the code.

But with those caveats in mind, I was happy with the result! In particular I liked that I could use a Raspberry PI and write JavaScript code to power hardware, using tools I was already familiar with. This solidifies my belief that hardware is the new Geocities, and that it's possible to build pretty slick (if a little rickety) physical projects using JavaScript.

References

All the code I used is on GitHub. If you use this as the basis for your project, shoot me a note, I'd love to hear how it goes!

Other pages I found useful:

Bringing ArnoldC to JavaScript

2015-05-12T00:00:00Z

At NebraskaJS I introduced my dumbest project yet: a compiler which converts ArnoldC programs into JavaScript, with full support for Source Maps.

(Start at 17:46 for the ArnoldC action)

ArnoldC

ArnoldC is a game changer in the world of software development. It's a procedural language where every keyword is a catchphrase from an Arnold Schwarzenegger movie.

Here's helloworld.arnoldc:

IT'S SHOWTIME
  TALK TO THE HAND "hello world"
YOU HAVE BEEN TERMINATED

Or, to write a = (b + 5) * 2:

GET TO THE CHOPPER a
  HERE IS MY INVITATION b
  GET UP 5
  YOU'RE FIRED 2
ENOUGH TALK

(Note the unique order of operations in ArnoldC: mathematical operations are applied as a left-to-right stack, with no operator hierarchy.)

ArnoldC has been around for a few years, but it had one fatal flaw: the reference implementation compiles JVM bytecode. And while Write Once Run Anywhere might have been portable enough for 1998, my programs need to run on the Assembly Language of the Web.

ArnoldC-to-JavaScript

To be honest, I didn't write most of the compiler. I built on the excellent work done by Thomas Crevoisier, who did most of the heavy lifting with his arnoldc-to-javascript project. I simply added Source Maps and did some refactoring. But I learned lots along the way!

Demo

Here's an AngularJS controller written in ArnoldC, that simply prints out FizzBuzz up to 100. Hit F12 and check it out in its source-mapped glory. Note that there are a few esoteric things ArnoldC can't do, such as understand what this.value means. But thanks to a liberal use of eval(), anything is possible in arnoldc.js! Just don't tell Douglas Crockford.

Source code is available here.

Build your own compiler

I never took a compilers course in school, but friends who graduated from better CS programs than me used descriptive phrases like "brain-melting" or "nightmare-ish". Luckily, you don't need to know how to write a compiler to write a compiler in JavaScript! If you understand a few key concepts, you can take advantage of libraries and tooling that make writing your own compiler a breeze.

I registered for a Stanford compilers course, but only watched up through the "Parsing" section. That's probably all you need to get started.

Broadly, a compiler works in two phases: lexing and parsing. The lexer takes your source code and converts it into tokens, such as START_IF_STATEMENT, or ADDITION_OPERATOR.

You then run your code through a parser, which is where the tokens get assembled into an Abstract Syntax Tree. From here, you have a structured program and can write a translation from the input program into JavaScript.

Jison

Writing lexers and parsers is difficult, so I've been told. But you don't have to do it! Instead, you can define your language (keywords and structure) using regular expressions, and use Jison to generate the lexer and parser automatically.

Jison was written by Mozilla's Zach Carter, and forms the spine of the ArnoldC compiler. Jison is a port of the C program Bison, which performs a similar role using a less fun language. But many of its docs might be useful to peruse.

Here's part of ArnoldC's lexer, written in Jison:

%lex
%%

\s+                             /* skip whitespaces */
"IT'S SHOWTIME"                 return 'BEGIN_MAIN'
"YOU HAVE BEEN TERMINATED"      return 'END_MAIN'
\-?[0-9]+                       return 'NUMBER'
"TALK TO THE HAND"              return 'PRINT'
"@I LIED"                       return 'FALSE'
<<EOF>>                         return 'EOF'

/lex

Keywords go on the left, and the name of the token is on the right. The keywords are regular expressions, so they can be as complicated as you need.

The parser then takes these keywords and creates a language out of them. Here's a higher-level ArnoldC concept of a "statement":

statement
  : PRINT integer
      { $$ = new yy.PrintExpression(@1.first_line, @1.first_column, $2); }
  | PRINT string
      { $$ = new yy.PrintExpression(@1.first_line, @1.first_column, $2); }
  | DECLARE_INT variable SET_INITIAL_VALUE integer
      { $$ = new yy.IntDeclarationExpression(@1.first_line, @1.first_column, $2, $4); }
//etc

integer, string, etc each have their own definitions, as well. There's a bunch of Jison-specific code here, but the gist is that a statement can look like any of these expressions. And for each expression, you can write a JavaScript function that knows how to handle that code. You can use Jison-specific variables like $2, which simply represents the actual value of integer in the source code (since it's the second token in the PRINT integer expression.

You can also see the use of first_line and first_column, which is the basis for the source maps the compiler generates.

So from there, you can assemble a bunch of statement expressions in a row, and create a statements meta-expression:

statements
  : statements statement
    { $$ = $1.concat($2); }
  |
    { $$ = []; }
  ;

This is a recursive definition that, at its root, creates an empty array. And for each statement in a row it finds, it just adds it to the array. This recursive pattern tends to show up a lot Jison definitions.

So now, you can define an entire program like this:

program
  : methods BEGIN_MAIN statements END_MAIN methods EOF
      { return $1.concat($5).concat(new yy.MainExpression($3, @2.first_line, @2.first_column, @4.first_line, @4.first_column)); }
    ;

So you've got a BEGIN_MAIN keyword, a set of statements, an END_MAIN keyword, and on either side a set of methods expressions (which are defined similarly to statements).

Parsing with Source Maps

The actual parsing and conversion to source maps is just standard JavaScript. You get your inputs (in this case, a set of tokens from an ArnoldC program), and return the JavaScript code that represents that section of code.

You have a few choices on what to return from your parser functions. The original ArnoldC compiler returned JavaScript Strings that simply got concatenated into the final .js file. Or, you can use Mozilla's excellent source-map library to return both the generated .js, as well as a Source Map you can define along the way.

I followed Mozilla's guide essentially word-for-word, so I'll just link to that excellent article; you'll want to read it and re-read it. It uses Jison, and also shows how to integrate Source Maps into a compiler. This is where the bulk of the actual compiler work is done, and will be unique to each language.

ArnoldC's parser functions each returns a SourceNode object, which can contain other SourceNode objects. This is done for each expression in the abstract syntax tree you've built out.

Once you've parsed the entire AST, you can use the library's toStringWithSourceMap function - it returns an object with map and code properties, which can then be saved off to the file system.

Here's an example of the final code for transpiling ArnoldC's print expression TALK TO THE HAND:

PrintExpression.prototype.compile = function(indent, fileName) {
  return this._sn(indent, fileName, 'console.log( ')
    .add(this.value.compile(indent, fileName))
    .add(' );\n');
};

All of this might sound complicated (and at first it was), but I found that there's a pattern to it all, and once you learn that pattern, everything kind of falls into place.

Other Resources

These videos from 2013 Front-Trends were super helpful for me to wrap my head around these concepts: Zachary Carter's talk on Jison, and Nick Fitzgerald's overview of Source Maps.

The original ArnoldC port has lots of great examples, including solutions to Project Euler problems.

Next up? Chicken.js

Source Maps in 2015

2015-05-11T00:00:00Z

I gave a talk at NebraskaJS about my journey with source maps:

Video

Demos

The source is available on GitHub.

Slides

We've complected things with tools

You probably don't deploy your JavaScript or CSS to production in the same format as you write it. Or rather, you probably shouldn't. Maybe you're concatenating files, or minifying the assets, or even compiling from another language like Sass or CoffeeScript. Tools like Rails' Asset Pipeline do this automatically.

But! Debugging concatenated, minified code is a huge pain. Prior to Source Maps, the best tool available was Chrome's Pretty Print. But it's only a partial solution, as it doesn't restore variable names, show code transformations, etc. It's a slog working through the generated code and try to figure out what's going on.

The solution? More tools!

I'd heard of Source Maps a few years ago. At the time, it was being touted as the solution to the tooling problem we created. Way back in 2012, Ryan Seddon published an article outlining all the cool stuff you could do with the technology. But it didn't seem to catch on; I hardly ever noticed a site with source maps enabled.

So what gives? From my perspective, the problem wasn't with individual tools, it was with the toolchain. If any step in your app's asset pipeline didn't properly support source maps, everything fell apart. So while you may have source maps configured for Sass, if you then ran it against Autoprefixer, you're at the mercy of Autoprefixer's support.

And the more complicated your asset pipeline, the more likely one link was going to fail. Supporting Source Maps right is hard. Check out the release notes for Grunt's uglify plugin. Nearly every release in 2013 and 2014 was fixing or enhancing something with source maps.

An end to the teething years

But I think we're at a turning point. For my Node-based toolchain, source maps are finally working across the board.

Concatenate? Added less than a year ago.
Uglify? Took a complete rewrite to support.
Sass? Just got consistently working maps in node-sass last week.

Some of this credit has got to go to Nick Fitzgerald and the excellent work Mozilla has done on the source-map library, which provides the bulk of the code folks need to enable source maps without having to dive deep into the spec. It works great, has a simple and flexible API, and over 200 npm modules depend on it.

It works for The Terminator

The source-map library works so well, I took it and implemented a new language that compiles to JavaScript, with full source map support. Granted, the language is ArnoldC and quite possibly the dumbest project ever conceived, but it works, damnit! And I never once had to open the Source Maps specification to learn its mysterious secrets.

(ArnoldC should probably be detailed in its own post, this has gotten way too long.)

So if you've been waiting to investigate Source Maps, now might be a great time to start. You have nothing to lose but your one-character named minified functions.

We're hosting a JavaScript conference!

2015-02-15T00:00:00Z

On August 7, we'll be throwing the first JavaScript conference ever held in Nebraska.

I'm pretty sure this is the first technical conference held at a world-class zoo, and that's just the start. We're planning on a killer lineup of local and nationally-known speakers, as well as showcasing what makes Nebraska such a great state to live and write code.

I'm stoked to be on the organizing committee, alongside folks far more talented than I. If we've assembled the Justice League of Nebraska JavaScripters, I'd be Aquaman.

We're still getting everything ready, but you can sign up for the conference's mailing list right now.

Hope to see you there!

Building a Handcrafted IR Blaster for Arduino-Powered Bike Lights

2014-12-14T00:00:00Z

I recently participated in the Bike De'Lites, a casual bike tour around Omaha's Christmas-decorated neighborhoods.

I like building silly electronics for rides like this, and was inspired by an LED light strip Zach Leatherman made recently. Zach attached 5 meters of LED lights to his bike frame, but I wanted to take it a step further. Static lights are cool, but I wanted to make them dance.

I wired the lights and attached them to my bike using the method Zach outlined, so it's worth checking out his setup.

LED light strips are great: they're cheap (about $10), super bright, and can be battery-powered. (There are "smart" strips that have individually addressible lights, but they're quite pricy.)

I picked up the E-Goal 3528 RGB LED strip off Amazon. It included a remote control to adjust the lights.

(The colors are a misnomer - the "purple" option simply turns the red and blue LEDs on at the same time.)

The remote control was nice, but I didn't want to be holding it and pushing buttons while riding. So I reverse-engineered the remote and made an automated one using an infrared LED and an Arduino.

Seeing Infrared

Infrared remotes are finicky. They send pulses of IR light at specific frequencies (usually 38Khz, but not always) to a demodulator that understands only those codes. Each manufacturer has their own style, and many of the manufacturers attempt to make their signals more "robust" by sending the same command two or three times in succession. Here's lots more on IR theory.

It doesn't take much to build a homemade IR blaster - first you need to decode the signal from the original remote, then replay it using an infrared LED (available at your local Radio Shack, if it hasn't shut down yet).

Debugging the remote

I shouldn't have expected much from a $10 gadget, so it's no surprise that the light strip and remote did not come with any instructions or data sheet.

After learning about infrared, I borrowed a USB Infrared Toy, which provided a convenient way to both send and recieve IR signals. Surprisingly, it had the best support on Windows, which I wasn't setup for.

I used LIRC, which looked like the best-supported Linux package to read and send IR commands. LIRC's remote database already had over 2000 devices available but the LED strips were not on there. Luckily the irrecord command lets you create your own device configurations, using a CLI-based wizard.

Aside: if you want to get the most features out of the IR Toy on Linux, don't use the LIRC version available out of the default repository - get this package instead. Otherwise it's stuck in the inferior "IRman" format.

Here's the configuration it generated:

begin remote

  name  remote.conf
  bits           16
  flags SPACE_ENC|CONST_LENGTH
  eps            30
  aeps          100

  header       8899  4526
  one           538  1688
  zero          538   558
  ptrail        554
  repeat       8901  2261
  pre_data_bits   16
  pre_data       0xF7
  gap          108279
  toggle_bit_mask 0x0

  begin codes
    on                       0xC03F
    off                      0x40BF
    dimmer                   0x807F
    brighter                 0x00FF
    flash                    0xD02F
    ...etc
  end codes
end remote

The header section defined the protocol for the code, and then it lists out the 16-bit hex values for each of the buttons. Pretty slick stuff!

IRduino

Next was configuring the Arduino to send the IR codes. As a JavaScript guy, I had hoped to use Johnny-Five but had to discount it. First, this had to fit inside my saddle bag, and J5 requires the Arduino be tethered to a computer. Second, the Firmata protocol doesn't support pulse-based units. So I went back to Arduino's native C library.

The circuit was pretty straightforward, just a single infrared LED and a resistor:

I looked into sending the pulses using "pure" Arduino code, but found the IRRemote library which abstracted lots of it for me.

IRRemote even included configurations for several popular code formats (Sony, NEC, RC5) but these didn't work for me. IRRemote's backup is the sendRaw command, which lets you send a raw bit stream of data.

Another caveat: I had hoped to use an Arduino Leonardo clone, but the IRremote library has a bug with that model, so I switched to the Uno, which worked fine.

Monday Night RAW

Unfortunately the configuration file LIRC generated didn't work with this (the hex values were too abstract), so I had to re-run the irrecord command using the -f flag to create raw codes. They looked like this:

begin raw_codes

  name on
    8874    4565     511     597     533     554
    533     575     511     575     554     554
    554     554     511     597     533     554
    554    1685     511    1685     533    1685
    575    1706     533     554     554    1706
    511    1685     533    1706     554    1685
    554    1685     511     575     554     533
    575     533     533     575     511     554
    554     533     533     575     554     554
    554    1685     533    1685     554    1685
    575    1685     533    1685     533    1685
    554

  name off
    //etc

Ugly, but it generated properly.

Then it was just a matter of converting these values to a C array and sending it IRRemote:

IRsend irsend;

unsigned int on[67] = { 8874,4565,511,597,533,554,533,575,511,575,554,554,554,554,511,597,533,554,554,1685,511,1685,533,1685,575,1706,533,554,554,1706,511,1685,533,1706,554,1685,554,1685,511,575,554,533,575,533,533,575,511,554,554,533,533,575,554,554,554,1685,533,1685,554,1685,575,1685,533,1685,533,1685,554};

irsend.sendRaw(on, 67, 38); //command, array length, khz

After a bit of testing, it all worked! I whipped up a small program to go through a program of Christmas-y (red & green) light series, and sequenced through them after a random delay (from one to twenty seconds).

Scope Creep

Like most projects, if I had more time I'd have made it more complicated. I was hoping to get the data from my bike's cadence sensor over the ANT+ protocol. That way, each time I pedaled the lights could change colors.

Alternatively, I could attach a tilt sensor to the bike and have the lights turn red while climbing uphill, and green while going downhill.

I also was a little disappointed with the IR signal - a handful of times the sensor wasn't aligned with the LED and would miss the signal. My friend Jonathan Rice suggested ditching the IR reciever and sending raw commands directly through the connectors.

All in all I'm pretty happy with this setup. It's way nicer than the battery-powered Christmas lights you'd pick up in a store. If you've got a night ride coming up, give it a shot!

Resources

All the code and configuration is available in this Gist
Bookmarks tagged #lirc on Delicious
More photos/videos of the lights on Flickr

Responsive Images using and srcset/sizes

2014-07-01T00:00:00Z

I spoke at Omaha Coffee & Code about the work the Responsive Images Community Group has been up to lately:

Big heavy images are responsible for most of the bloat on a responsive website, but it doesn't have to be that way. Here's the latest HTML elements you can use to keep your images small, and your site lean & fast.

Slides

Screencast

Thanks to Eric Portis's amazing post on srcset/sizes, which really helped me understand how these new elements worked, and from which I borrowed liberally.

Photo credit: Rebecca Stavick

Responsive Web Design Workshop at Interface

2014-05-13T00:00:00Z

It's been nearly four years since Ethan Marcotte published a little piece A List Apart on Responsive Web Design. It's the post that launched a thousand redesigns. Like many, I was first introduced to RWD after seeing the Boston Globe's redesign, and I began to implement it at work (which now sports a solid responsive homepage) and on this blog.

But not everyone is sold on the concept. Lots of folks are building desktop-only sites, or siloing a fraction of their content off into an m-dot subdomain. Others have tried, but struggle with building a high-performing responsive site, or think their users "don't want to do this on mobile".

That's why I'm so excited to be working with the crazy-talented Jerod Santo and Shonna Dorsey to lead a workshop on Responsive Web Design at the Interface Web School. It'll be a 4 day course this July that spans the gamut of building for the responsive Web, starting with the 3 key elements and moving onto tooling, performance, webapps, IE support, and lots more. And it'll run in the evenings, so you don't even have to convince your boss to take time off work for it.

You can register on Interface's website. It'll be fun!

[

(PS: This post is using Picturefill, which is one approach to responsive images I'll talk about during the course.)

Hardware is the new Geocities

2014-01-16T00:00:00Z

I gave a talk at NebraskaJS on hardware hacking, Johnny-Five, and the importance of amateur goofery.

It was the most risky talk I've ever given. I demoed an Arduino-powered Power Glove, modified my slides on the fly with a WebSocket-enabled potentiometer, and flew a quadcopter drone over the audience using a Wiimote.

Video

I also gave a 20-minute version of this at Barcamp Omaha, which you can watch if you enjoy seeing quadcopters crash and burn.

vine: h3zEv1Qzl0l

Video and more courtesy of Zach Leatherman at NebraskaJS.

Go(lang) for Broke

2013-12-05T00:00:00Z

A few weeks ago I took part in the third Hack Omaha, a civic hackathon put on by Matt Wynn, Open Nebraska, and others.

We built a website to visualize Omaha Public School's enrollment data. It was a blast to build, and not just because it helped answer burning questions of teammate, parent, and all-around cool guy Alex Gates. Alex wanted to know why his daughter's kindergarden class had 39 children in it when classes began.

Here's what I learned.

Going Nuts

This was all Jerod Santo's fault.

At the first Hack Omaha, Jerod amazed me by building his project in a days-old JavaScript framework. It was a high-wire act, but he convinced me that I should follow suit.

So when Hack Omaha rolled around this year, Jerod and I decided to team up and try a language neither of us knew. His suggestion: Go - Google's new(ish) language. Juan Vazquez joined in, and a team based on ignorance was born.

Go's docs describe the language:

Go compiles quickly to machine code yet has the convenience of garbage collection and the power of run-time reflection. It's a fast, statically typed, compiled language that feels like a dynamically typed, interpreted language.

Going to School

I spent some time learning the language, and the Go tour was my main source. Perhaps it's the Ken Thompson-inspired origins, but it was nice working with a language that didn't condescend to you and treat you as a programming neophyte. I often feel like this about other guide, such as _why's Poigniant Guide to Ruby.

A Vodka Martini, Chilled

At this point, we had an idea that an app would be Go serving up an API on the backend, and a web frontend running AngularJS. The Changelog also had a really helpful post outlining Go's web application ecosystem.

Go has an impressive standard library and can serve HTTP easily out of the box, but we decided to apply a framework to provide a little structure. We went with Martini; a Sinatra-esque library that is less than a month old, in a proud Hack Omaha tradition.

Our services ended up with this structure.

m.Get("/schools", func(res http.ResponseWriter) string {
  return render(res, allSchools)
})

This should look similar to Sinatra and its bretheren. Martini gives you the objects you need (like a ResponseWriter) via parameterized Dependency Injection, which is super convenient.

JSON encoding, HTTP headers, enabling CORS, all worked out of the box.

I've heard it's also testable, but nary a unit test was written during the hackathon. I should have known better, based on my lessons from last time. That said, Go has testing built into the language, so it'll be there when we're ready.

Databases, GORM Style

Go has database operations built into the standard library; just bring your own database-specific adapter. We went with MySQL, which has a great Go adapter. And writing queries is lame, so we tried out an ORM called Gorm. This was a little more mature, as it was a full 6 weeks old.

Using Gorm is pretty simple: define a struct with types named like your database columns:

type School struct {
Id int64
Name string `sql:"size:255"`
CountyId int64
DistrictId int64
Lat float64
Lon float64
}

And populate them:

var school = School{}
schoolId := params["id"]
db.Where("id = ?", schoolId).First(&school)

ORMs are appealing, especially when your database tables are denormalized by design to allow simple querying. But the additional complexity Gorm brought might not have been worth it. ORMs are known as the Vietnam of Computer Science for a reason.

Throughout Friday and all of Saturday, we were unable to connect to the database using Gorm's classes. It wasn't until nearly midnight on Saturday that we were able to figure out the correct incantations to get MySQL and Gorm to pull in data.

Clearer documentation might have helped, but in the end it turned out to be mostly user error.

Error handling in Go is idiomatically performed by returning two values from a function: the result you were expecting and an err parameter. If the err value is null, things went great. Otherwise, you have an error that needs handling. As we were connnecting to databases in Gorm, we weren't retrieving that second parameter; so we never knew we had a badly formed database URI, even though the library was begging to tell us.

Lesson: understand the idioms of a language you're futzing around with.

Go Fly a Kite

It wasn't all sunshine. Beyond database and library issues, here's a few problems I encountered with Go:

I like functional collections, and using slices (Go's version of an array) in a functional manner proved impossible. Go appears to have functions for filter, map, and others, but we never got them working. It appears that Go programmers are content to use for loops for everything, which is what we ended up doing.
Go provides strict checks at compile time. This includes failing the build when you have declared a variable or import that you haven't used. Normally I love these checks, but this proved more annoying than useful in the iterative environment of a hackathon.
As a user of third-party libraries, Go's package management is great (simply type go get github.com/package and it's available). But I never figured how to split apart our single-file Go program into a decent structure. We settled on placing our structs in one file, and everything else in another. They all shared the main package, as well.

So you won, right?

Nope. This project did.

So we should blame Go?

Nah. The Boards team had a better product, and changing our backend wouldn't have made the difference.

If anything, I'm ready to double down on Go for the next hackathon.

You get productivity out of the box with a great standard library, plus the performance of a natively-compiled app, plus a syntax that feels both classical and modern.

So forget the LAMP or MEAN stacks. Try building your next application on GAP: Go, AngularJS, and Postgres.

View the final code on GitHub

vine:hUKDzMbhOBx

Clickbait HTTP Status Codes

2013-11-20T00:00:00Z

502 Unbelievable Bad Gateways (with photos)
— Brian Essbe (@SortaBad) November 19, 2013

Nice. Here's more clickbait HTTP Status Codes:

These 302 Found Puppies Will Restore Your Faith in Humanity

403 Insane Quirks You Won't Believe Aren't Forbidden

Here's 200 GIFs of Lil John that will make you say "OK!"

304 Unbelievable Twerking Photos We Swear Have Not Been Modified

Here Are The 404 Most Essential Hide And Seek Tips To Ensure You Are Not Found

Why Did Tina Fey Say These 406 Quotes Are Not Acceptable?

410 Nostalgic Things 90s Kids Will Recognize As Gone

Check Out These 418 Weird Tricks To Make You A Teapot

A fresh coat of paint

2013-10-20T00:00:00Z

After years of using my website as a glorified landing page to other content, I've taken the time to build something proper. The link dump is gone, and I've moved my blog from WordPress.com to my domain. The site is also available on GitHub.

Google Reader expatriates: update your RSS feeds: http://www.matthew-steele.com/feed/atom.xml

Why redesign?

A few reasons:

Link dumps are lame. The meat of this site is on the blog, so it makes sense to show that up front. No one likes to wait while they wait.
I began using WordPress.com 6 years ago, when I was working mostly on the back-end and didn't want to think about website layouts or CSS. Times have changed.
It's theraputic. I might be the only person who finds joy in moving periodically in the physical world. It's a great time to trash old artifacts and make way for the new.

It's said that gearheads never have their own car in working condition. Car enthusiasts always have some project they're working on, and use their personal ride as a experiment. I wanted a place to experiment on my own.

Goodbye to all that

I went with a "wait until it's needed" approach for most content. The site is JavaScript free (save Google Analytics) and doesn't include any CSS libraries. I love jQuery, Modernizr and Bootstrap, but sometimes the only tools you need are your bare hands.

This has resulted in a substantial shrinking of page weight. A text-only post on this platform is fully 86% lighter on page load than its WordPress cousin.

I've also held off on including comments. I like the idea of hashing out conversations elsewhere (like Twitter or Google+) and then writing a new post with the results. If that doesn't work out, Disqus is a simple solution.

Breaking it down

I've ditched WordPress, PHP, and databases. The site is now 100% statically generated and hosted on Amazon S3. Building it has been exciting.

Jekyll generates the static site, and I'm using Grunt to automate many of the other tasks needed to build the site. I first heard of this combo from Zach Leatherman's redesign, and it seemed ingenious. Even though the tools are written in different languages, they play really nicely thanks to the grunt-shell plugin.

Grunt automates everything. Compiling LESS files, running a local server, deployments to Amazon S3; all a grunt command away.

Posts are written in Markdown, which feels right and lets me blog in Vim.

While designing, I went mobile-first and responsive. With such a simple static page, I was able to just pull up localhost on the iOS simulator and my Android tablet and call it good. It also simplified my CSS by a pretty substantial amount.

Since I now have control over the layout, I also took the time to add an I Live in Omaha banner to the page. There's an <img> version of the banner, but I decided to rewrite it using pure HTML and CSS3. This made it responsive and Retina-sharp. On small screens, it's a footer at the bottom of the site. As you gain more real estate, it jumps to the top of the page, and eventually gets fixed to the top. Since it's progressively enhanced, this also avoids many of the issues with position:fixed on mobile devices.

Talk amongst yourselves

Let me know what you think. I'm on Twitter.

Exploring the Device APIs

2013-08-06T00:00:00Z

I gave a lightning talk at NebraskaJS on some of the new features available through the Device API. More importantly, it marks the glorious return of the Wayne's World light sensor.

Slides

Screencast

Diving into the Device API

2013-06-19T00:00:00Z

I recently read Tim Wright's article on A List Apart detailing the Device API; a collection of W3C standards that let you obtain access to a number of hardware sensors.

I found this fascinating, and had to try them out. Here's what I've learned.

Note: for most of these demos, you'll want to try them out in specific browsers (either Android Firefox or iOS Safari); as of today, browser support is limited. The most up-to-date information on browser implementation is on this page.

All demo code is available on GitHub.

Battery Status

View Demo (works in Android Firefox)

The Battery Status API is straightforward - it adds a new object window.navigator.battery that you can inspect and discover how much juice is left in the device for you to suck out (this is exposed by battery.dischargingTime and is measured in seconds).

battery.charging returns a boolean. You can inspect the charged level using battery.level, but the most interesting parts are the events, which let you capture when a device starts charging, or reaches a threshold over 60%, etc.

The demo captures the chargingchange event and changes some background colors. but you could do lots of things with this data. For example, it might be prudent to turn off 3D CSS transformations at a certain battery threshold, as they're quite power-hungry. By treating power as a feature test, you can take progressive enhancement to a new level.

Availability

Firefox currently supports this API on desktop and mobile Android. WebKit appears to have implemented it briefly last year, but it's currently disabled and work to re-enable it appears to have stalled. Similarly, Chromium has a patch built but it doesn't seem to have ever landed.

Ambient Light Sensor

View Demo (works in Android Firefox)

Most phones have an ambient light sensor - it's mostly used to dim the screen in low-light environments. The Ambient Light Sensor API works with any sensors that can read light levels, including an embedded camera. There's lots of environment-specific modifications to a site you could perform with this.

Availability

Firefox on Android has it, and it's implemented on the desktop in OS X. There's a patch for Windows 7 that appears to have stalled. I don't see any evidence that other browsers are planning to implement it.

Device Orientation

A native app that recently captured my attention was the well-advertised Pizza Compass for iOS. It does exactly what you'd expect. I wanted to try a version in pure HTML5. Right now it only points you to a pizza place in my city, but it meets my use case, so why abstract?

View Demo (works in Mobile Safari/iOS Chrome)

There's a number of device location/orientation sensors in modern cell phones, and it's not always clear which HTML5 feature to use:

Accelerometer/Device Tilt: DeviceOrientation
Latitude/Longitude: Geolocation
Compass: DeviceOrientation (using the alpha property)

There's been plenty of apps that use Latitude/Longitude; many use it to assist in a "Store Locator" feature. Google's also done a few interesting experiments with DeviceOrientation, but I haven't seen many instances of its use in the wild.

Here Be Dragons

The basic functionality works on current iOS browsers. However, I ran into a number of quirks on other platforms that also support the DeviceOrientation API; which means you can't build interoperable apps with it.

Firefox increases the alpha property as the device rotates clockwise. All other tested devices decrease the property when rotating clockwise.
The alpha property's initial value is all over the place. Some browsers (iOS Mobile Safari) set it at 0 based on the initial orientation of the device. Other browsers set 0 to a particular orientation, but it's inconsistent (Android Browser is 0 at west, Firefox is 0 at north, Chrome is ambiguous).
The event provides an absolute property, assuming the alpha value is calibrated to formal Euler Angles (Firefox and Chrome for Android implement this). There doesn't seem to be consistency even with this.

Availability

Most mobile browsers support these APIs, with the caveats stated above.

Conclusion

These features are really slick, but they're hampered by the same issues that hinder the web platform generally: platform fragmentation and buggy implementations. But take the long view, and soon enough you'll be able to build real-world sites that capture environment input in lots of cool ways.

Unit Testing JavaScript when you're Afraid of JavaScript

2013-03-16T00:00:00Z

I spoke at Nebraska Code Camp on how to unit test JavaScript when you're deathly afraid of the language.

We've gone from writing toy programs in JavaScript to full-blown applications, but our testing skills haven't caught up. Weirdly, lots of us are coming to JavaScript from other languages that value and encourage testing. What happened?
Turns out, unit testing JavaScript is different than other languages, and doing it well means you have to rethink how you write JS code.

Slides

Screencast

What Zelda Taught Me About Front End Engineering

2012-12-06T00:00:00Z

I spoke at the HaymarketDev meetup in Lincoln on treating JavaScript like a real language.

I was coming down from an NES Zelda bender, and I think it shows.

Slides

Screencast

Video:

Unit Testing in JavaScript with Jasmine

2012-10-21T00:00:00Z

I spoke at the Omaha Java User Group this month about what testing JavaScript is like for a JS neophyte:

Town Tester - How well does your city unit test?

2012-09-18T00:00:00Z

In my talk Zen and the Art of TDD, I included a slide that showed only 43% of all Github repositories from Omaha developers included unit tests:

Today I'm releasing the code I used to obtain this data: a script called Town Tester.

View town-tester on Github

It's a Ruby script that queries the Github API to do the following tasks:

Find users with their Location set to the value you provide
Clones all "popular" repos (not a dotfiles project, at least 1 watcher, not a fork)
Checks for the existence of tests (based on a filename that includes "test" or "spec")
Creates a .csv with data on each repo, and prints statistics broken down by language

I'm excited for the possibilities of this project:

Want to showcase how awesome your local community is? Steal developers away from San Francisco by proving your city can test better than their piddling 56% rate.
Put an end to the Great Semicolon Debates. Whichever developer has more JavaScript repositories with tests wins all syntax arguments by default!
Gamification is all the rage. Why not track your city's testing ratio over time, and give XP to developers that increase the ratio the most? Each month, the winner could get free drinks at your local Beer && Code.
Just in time for the Global Day of Coderetreat, you could lure Corey Haines to your hometown by being the best tested city in the world. I'm pretty sure he doesn't really want to go to Sydney or Honolulu anyway.

I know I'm just scratching the surface of this venture.

Also, please don't game the system by adding a "test.txt" file to the root of each of your repositories.

Zen and the Art of TDD

2012-09-09T00:00:00Z

Over the weekend at Barcamp Omaha I gave a talk on why I practice test driven development. Though TDD is over a decade old, it still isn't a practice we do with much regularity. So this is my Top 5 list of reasons why I think TDD is an important skill to develop.

While prepping this talk, I discovered that only 43% of GitHub repositories from Omaha developers have any tests at all. How many were built with TDD is certainly an even smaller fraction.

Slides with Audio

Video + Slides

Slides

JavaScript demo app | Jasmine test suite

The code for the app and the slides are also available on Github.

Lessons Learned from the first Hack Omaha

2012-04-19T00:00:00Z

This weekend I took part in Hack Omaha - the city's first hackathon with a focus on making apps from public data. We built an app that gameified health inspection data. It was awesome. Nate's already written up an hour by hour recap of our team's experience, but I thought I'd share specifics of what I learned.

Pitch an idea, even if it's not a winner. Nick Wertzburger started the pitch session off with a joke app (I think) that he tweeted earlier. It was one of only a handful of pitches, and his team ran with it and melded it into an awesome heatmap page; my favorite project of the weekend.
Sinatra rocks. My day job consists of writing Web Services, with a capital W and S. The process is often heavyweight, cumbersome, and requires numerous approval and manual configuration steps. It was beyond refreshing to just write a get /matchup method, paste in some JSON, and have a working service. Prototyping service design before you've even gotten a dataset gives you lots of flexibility to change your design on the fly.

Heroku is great, except when it isn't. We ran the app's service layer on a shared (read: free) Heroku instance and a shared Postgres database. This was my first experience hosting apps on Heroku, so I relied on Steve's expertise. Pushing changes was simple as pie, but we ran into numerous issues getting Rake migrations to function correctly. We ended up creating databases on my machine, and using Heroku's backup/restore feature to load up production. It's not pretty but it got the job done.
Designers are worth their weight in gold. With all due respect to Nate's work, we could have used someone to help with the usability, icon design and overall polish of our app. Most projects were in the same boat. But they were in extremely limited supply here.
ORM flexibility is helpful. Since you have no idea what tech stack you'll be working with, you don't want to require teammates to have a particular database already installed. For example, Steve didn't have Postgres installed on his MacBook, but we just configured a SQLite instance on his box, set up his ActiveRecord configuration to connect to it and he was off and running.

Don't let your VCS hold you back. We decided to use GitHub to host the source, but only half the team had any git experience. Rather than try to learn a crash-course on git, they used a shared Dropbox folder as the repository location.
Colocation isn't necessary. We spent most of Saturday working from our individual houses, and we stood up a Google+ Hangout to help. The video chat and screensharing worked really smoothly. We probably had an advantage over the folks who worked at the hackathon venue, as wi-fi was spotty the whole weekend.
We could have been more ambitious with our tech stack. We were familiar with almost every piece of what we built. Then I look at Omaha Bounty Hunter, which was developed against a five-day old JavaScript framework, and I feel a little sad that we didn't try something farther out there. At the very least, we could have tried a document database like Mongo, given that nothing we were doing was relational.
Keep projects small and focused. We were essentially finished with our app by 6pm on Saturday. After that, we spent the rest of the time play testing, tweaking the design, and adding features like analytics, win/loss counting, etc. But having a small, achievable project meant we weren't scrambling to get basic functionality working at the last minute.

I don't know Ruby very well. I kept running into syntax issues, like trying to return early out of a block (which isn't allowed). I also spent a ton of time learning the methods on Enumerable, and figuring out how attributes in ActiveRecord models function. You only have to look at the number of Hash.new and Array.new in the codebase to see that we're still noobs at this.
Untested code becomes legacy code, fast. We cranked out the services with nary a unit test in sight. If it didn't cause a syntax error, we shipped it. By the end, all code was making into Sinatra routes, which meant we had to reload our web app each time we wanted to change a `group_by`, or see if our ActiveRecord query was right. This slowed us down noticeably, even by day 3 of the project.
Not all prepwork is fruitful. I tried to learn two new technologies before the hackathon: Neo4J and PostGIS. I thought each might be helpful, and I spent weeks trying to learn just enough to fake knowing it for a weekend. But I ended up using neither. Rather, a 2-hour session with Twitter Bootstrap proved far more useful than anything else I knew.

Much thanks to Steve, Nate and Mike for putting up with me this weekend. And many, many thanks to Matt for setting up the event. I'm ready to do it again.

I have a cameo in The Clean Coder

2011-11-04T00:00:00Z

On my way back from India, I finally had a chance to read The Clean Coder, Uncle Bob's latest book. It was something I'd been personally wanting to do for a while now; having been introduced to the tenants of software craftsmanship, professionalism, and test-driven-design as part of reading his previous book, Clean Code.

It also was something I'd wanted to do because last year, my company brought in Robert C. Martin for a two-day course on TDD for me and a dozen other employees. And it was amazing. My colleague Shawn compared it to "getting private guitar lessons from Jimi Hendrix".

To my delight, this came up as an anecdote in Chapter 6 of his book, "Practice":

Since then many programmers have adopted a martial arts metaphor for their practice sessions. The name Coding Dojo seems to have stuck. Sometimes a group of programmers will meet and practice together just like martial artists do. At other times, programmers will practice solo, again as martial artists do.
About a year ago I was teaching a group of developers in Omaha. At lunch they invited me to join their Coding Dojo. I watched as twenty developers opened their laptops and, keystroke by keystroke, followed along with the leader who was doing The Bowling Game Kata.

Reading this on the plane, I couldn't help but smile. Uncle Bob has had a tremendous affect on my development style and coding habits, so I'm glad to hear that I had an influence on him, however small that might be.

Postscript: At the start of the session, we each introduced ourselves and explained what we wanted to get out of the course. Jessica's conversation went like this:

Jessica: My name's Jessica Codr [pronounced "coder"]
Uncle Bob: [pause] That's the best last name I've ever heard. I need to put your name in my next book!

I think this counts.

Test-driven source code formatting

2011-05-02T00:00:00Z

This morning, I had to fight a production error in an application I support. It turns out that using SELECT * statements with Spring JDBC can cause problems when you add new columns to the table. It has something to do with Oracle caching; I'm not sure.

So, SELECT * statements are evil. But how do you ensure your codebase doesn't contain any? Being a good test-driven developer, I wanted to have a failing test before making wide swaths of changes to my source code. The solution I settled on was to write a new static analysis rule, using Checkstyle, to warn me when it identified a SELECT * statement. Here's what it looks like (you can put this in a Checkstyle v5 xml file):

<module name="RegexpSinglelineJava">
  <property name="format" value="SELECT.*[\. ]\*"/>
  <property name="ignoreComments" value="true"/>
  <property name="message" value="Do not use SELECT * statements"/>
</module>

It's a regular expression that looks for a SELECT, followed by anything, and then either a dot or a space followed by the asterisk. This way, we can capture:

SELECT * from table
SELECT t.* from table t

But not:

SELECT count(*) from table

Adding this check to our build immediately caught 13 instances where we performed this nefarious deed. Using a continuous integration build like Hudson, it was easy to identify and track how we were progressing in removing these from the build:

Of course, there a number of issues with this approach:

Queries defined outside of a .java file aren't scanned
The regex misses queries defined over multiple lines

But it's a nice quick solution to an immediate problem, and we can iterate to solve it. I hadn't thought about using static analysis tools as a form of test-driven development, but it seems like a natural extension of the red/green/refactor cycle.

Scramble Squares puzzle game solver

2010-12-26T00:00:00Z

My folks got a golf tile puzzle game called "Scramble Squares". I was tired of trying to solve it manually, so I wrote a little Ruby app to find all the valid solutions. The code's on GitHub:

https://github.com/mattdsteele/scramblesquares-solver

After uploading it, I realized that quite a few others have already published solutions. But mine's the only one with unit tests :)

Omaha Public Library card symbology

2010-01-28T00:00:00Z

In case you're like me, use Cardstar, and want to put your Omaha Public Library card on your phone, here's the custom barcode you should setup:

Symbology: Codabar

Start code: A

Stop code: C

Inverted: no

Hope Google's indexing this. Knowledge is power!

Now this is a story all about how my bike got flipped, turned upside down

2009-10-18T00:00:00Z

As previously noted, my bike was stolen a couple weeks ago. Fortunately, some stories have happy endings, and it is now residing back at my house. What follows is the harrowing tale of how I got it back (note: it's not all that harrowing).

Union Pacific has two sets of bike racks around its HQ building:

Rack 1 is positioned next to the front door of the building, and is underneath an overhang that helps protect bikes from the elements. Rack 2, newly added this year, is out next to 13th Street and affords none of the amenities of the original set. Needless to say, Rack 1 is as popular as a Homecoming queen, while Rack 2 gets rejected like a guy who just upgraded his D&D Avenger character up to level 6.

On this particular day, I was late biking into work, and all the spots in Rack 1 were taken, including several people who had double-used the rack spaces. Rather than trying to force my way in, I decided to park my bike on Rack 2, which happened to be empty when I locked it up.

Actually, "locked" is probably an inappropriate phrase. The previous week, I lost my old lock of several years on a ride home. So I went out to the Bike Rack and looked over their offerings, deciding on this particular model. Kryptonite rates its security as a "1", which they state is appropriate "if you live in the 'burbs and have a Rottweiler next to your bike". A couple pieces of twine tied together probably would have offered only slightly more protection than what I was using.

If there was ever a valid time for a lock to be guilty of EPIC FAIL, this was it. On the first day of its operation, it was broken and my bike was taken sometime while I was working. The thief didn't leave the broken lock, so I don't know whether they cut the cable, or found some way to crack the combination, though I'm guessing they used a pair of bolt cutters to snap it.

When I realized my bike was gone, I told a few people in the building about the theft, and got a ride home from a coworker. That evening, I called the police's theft report line and gave them a my phone number and a brief description. And of course I tweeted about it.

The next day, I got a call back from the police, and gave them more details about the theft. When the officer asked me if I had the bike's serial number and I replied "no", she responded with a disappointed-sounding "Oh."

I also got some great advice from my coworkers Scott and Brady, the latter having had his own bike stolen earlier this year. Brady told me what he did to recover his bike, including handing out flyers to local businesses and pedestrians, checking in pawn shops in the area, and blogging about it. He also mentioned the best way of recovering a bike was if I had the serial number available, which made me doubly skeptical that I would ever see it again.

Then for a long time, nothing happened.

On a Sunday afternoon, about two weeks after the theft, I got a call from an officer working in the Pawn unit, saying he got a description of a bike similar to mine in an area pawn shop. Following up, I gave him some additional details about the bike (going off memory and the one photo I had available), and a few days later he sent me some photos of a bike that was 100%, definitely mine. I sent him a reply and asked him to contact me when I could pick it up.

Then for a long time, nothing happened.

A week and several voicemails later, I finally got a response from the police, who sent a letter to both me and the pawn shop, declaring that I was the rightful owner and could pick up the bike. So yesterday, I ventured downtown to the Mid City pawn shop, and paid the $35 fee needed to get my bike out of hock.

I've learned a lot of things through this whole ordeal, most of which are outlined in Brady's own post about recovering his bike; so I won't repeat them here.

But the most important part is to record your bike's serial number. If you have a bike, do it now. I was a lucky bastard to have mine recovered without it, but everything I've experienced tells me that it would have made the process much easier. Pawn shops are required to file a bike's serial number with the police before they can sell it. Without the serial number, I had to trust that the pawn shop and the police operated on the same wavelength, and would be able to identify the merchandise by my vague description: "It's light blue. Or is it teal? Man, I wish I had more than just one picture of this thing."

So I'm back in business! I picked up a new Kryptonite-built U-Lock from the Trek store and plan on riding into work tomorrow. No mere larceny can hold me down:

I only ask that Omaha's thief population let me keep my bike for at least one week before attempting another swindling.

The King of Kong - A Fistful of Inaccuracies

2008-02-18T00:00:00Z

One of the more interesting films I watched last year was the King of Kong, a documentary about the quest to break (or retain) the Donkey Kong high score world record. As Zach Leatherman points out, there seemed to be some glaring (if trivial) inaccuracies in the film, such as the exclusion of a third contender for the high score title. Nothing to get too worked up over though; you have to edit the film to make it more enjoyable. That's show business.

It looks like we've been had. Jason Scott, a filmmaker working on his own arcade documentary, has some not-so-kind words to say:

[This is] a documentary that rips entire groups of good-hearted people as shadowy, conniving scumbags with razor-thin morality hurts the scene being portrayed and hurts the people themselves. All this effort, just to turn reality into a faked up drama worthy of a dime store pulp.

Some of the core themes of the documentary apparently are conjured up by omitting inconvenient facts that would singlehandedly disprove the theme. For example:

Billy denies Steve the satisfaction of playing one-on-one on Donkey Kong. They'd played Donkey Kong one-on-one a year before the documentary was filmed at a previous championship.

Read the whole thing. For more, take a look at Twin Galaxies' verbose comments on the film, or Billy Mitchell's interview with The A/V Club.

Inelegant code affects your reputation

2008-01-21T00:00:00Z

You've probably heard about (or signed up with) Mint, the startup which promises to liberate us all from the Quickens of the world and revolutionize personal finance. It's gotten quite a few high-profile positive reviews.

Mint has an incredibly hard sell. Before handing over all of your financial information to a startup run by five folks, they have to convince you to really, truly, deeply trust them. You're handing over the user name, password, and any personal security questions for your banks and credit cards. Mint has to overcome barriers not seen since Paypal's startup. The site prominently displays page after page detailing their focus on security.

I was eventually convinced and signed up. Mint's account entry page is well designed and ajax-y; set up a new account, and it fires off a few XHRs, keeping you notified of login/signup activity. Because I'm a curious fellow, I inspected the contents of one:

<ajax-response><response type="" id="">	<timestamp>0</timestamp>
  <json>[{
    "id":xxxx,
    "name":"My Personal Bank",
    "uri":"https://www.mypersonalbank.com",
    "fiId":12345,
    "status":123,
    "terminal":false,
    "balance":9999.99,
    "refreshed":"01/21/2008 14:06:15",
    "isFirst":false,
    "html":"<div class='cardfi ' id='pollElem-12341234'>
            .... html continues ...
    </div>"}]
  </json>
</response>
</ajax-response>

I've cleaned up the response and stripped out any of my personal data; for security, you know?

In any case, notice the data structure Mint's using to encapsulate its information. The entire content is wrapped in XML, with an <ajax-response> element as its envelope. There's a <timestamp> elemented appended too, but the bulk of the data is encoded in the <json> element.

In <json>, basic bank information is encoded in key-value pairs, such as the bank's name, URI, etc. Then there's the "html" key. Raw HTML lives here, containing mostly data stored in a <table> block. This is the real meat and potatoes of the XHR; it shows the actual status of the connection.

So backtracking a bit, notice how that data is sent across the server. The actual markup used to update the page is sent, wrapped in a JSON envelope, which is itself wrapped in an XML envelope. Mint <i>must</i> be taking security seriously, as they require you to decode <i>three</i> completely different markup wrappers in the scope of a single ajax call.

In all seriousness, this isn't groundbreaking stuff. Code like this goes up in enterprise applications all the time, and worse kludges won't make it to the <a href="http://thedailywtf.com/">Daily WTF</> any time soon.

So why worry about it?

As Web apps become more complicated and handle ever increasing amounts of sensitive data, companies can no longer merely claim they are designed with a focus on security. Fair or not, the code we write says just as much (maybe more!) about how much thought has gone into an application's design. If your company is already fighting against a torrent of criticism that perceives your product as rapidly designed and possibly insecure, something as simple as redundant nested data structures can have a profound effect on what people think about you and your software.

Mint probably won't lose users explicitly because of the data format of their XHRs. However, it provides one more piece of ammunition critics can use to paint it as untrustworthy, and may make prospective users just a little more wary of handing over their personal data. In a hyper-competitive environment like startup software development, why risk it?

Unintended consequences of the Internet

2007-11-27T00:00:00Z

So Scott Adams has decided to cut down on his blogging, because he doesn't like losing money being misunderstood:

Every blog post reduced my income, even if 90% of the readers loved it. And a startling number of readers couldn’t tell when I was serious or kidding, so most of the negative reactions were based on misperceptions.

Let me be the first to stake out a controversial position: this is terrible. Not only is Scott one of the most clear and concise writers I've read, he's popularized fantastic webcomics and allowed me at least one reprieve from work, every day.

He doesn't need to retire. I have a solution to his problem:

؟

Introducing the irony mark. You use it, get this, to indicate when you're being ironic. Examples include:

Agile programming is nothing but writing code and complaining؟
Hillary Clinton should stop running for President and make me a sandwich؟

I expect Scott to start posting regularly once again, and my royalty check to arrive shortly thereafter.؟

Screen scraping Google Spreadsheets exported as HTML

2007-11-03T00:00:00Z

Let's say you had a wealth of information in a Google spreadsheet that you wanted to access using Javascript. And let's say you also forgot you could export the doc as a .csv file (or in my case, wanted to try your hand at javascript screen scraping). What's a super-simple way you could accomplish this?

Glad you asked, because I've got just the solution for you. It requires using Prototype, though you could use it with just about any other library which lets you use CSS selectors.

First, create an array with which to store your data:

var items = [];

Then, you'll have want to create a function which takes the raw HTML input and parses it into a set of objects:

function parseData(googleDoc) {
  var divItem = document.createElement('div');
  divItem.innerHTML = googleDoc;
  $(divItem).hide();
  document.body.appendChild(divItem);
  var tblMain = $('tblMain');

  $(tblMain)
    .getElementsBySelector('td.rAll')
    .each(function(n, i) {
      if (i == 0) {
        return; //if you have a header row, you don't want to include it
      }
      var item = n.parentNode;
      var kids = $(item).getElementsBySelector('td.g');
      items[i - 1] = parseRow(kids);
    });
  document.body.removeChild(divItem);
}

function parseRow(kids) {
  var obj = {};
  obj.title = kids[0].innerHTML;
  obj.column1 = kids[1].innerHTML;
  obj.column2 = kids[2].innerHTML;
  //...etc
  return obj;
}

Then, just pull up the document with an Ajax call:

function loadData() {
  var req = new Ajax.Request('url/to/googleDoc.html', {
    method: 'get',
    onComplete: function(req) {
      googleDoc = req.responseText;
      parseData(googleDoc);
    },
  });
}

Simple, huh?

UI designs in Leopard that I really hope grow on me

2007-10-29T00:00:00Z

The Menu bar: Transparently awful

Whoever gave the go-ahead on adding transparencies to everything menu-related needs to get run over by a train. It's distracting and focuses your attention away from the text of the menu. This is a double whammy, considering you've already been distracted from an application's window the menu. To make this less transparent is just downright cruel. It's like they took the worst feature from Vista's UI, gave it some rounded corners, and vomited it straight to your desktop.

Context menus are no better:

Thanks for splitting my menu in half, jackass.

The New Dock : Let's get busy

I don't care so much about the "Shelf" vs "Dock" sound and fury that's been circling the web lately, but the screen reflections around the application's icons are super distracting:

If that horizontal line slicing the dock background in half isn't enough to grind your gears, then you are less likely than I to fly off the handle over UI changes. Really, how do you sleep at night?

The reflections under the icons are even worse, since they actively prevent you from seeing whether your application is in use. Was the little black triangle too usable, so they had to scale back my satisfaction?

At least Leopard gives me the tools with which to express my frustration:

The Thatcher Illusion

2007-09-23T00:00:00Z

Want to see something cool?

Aside from upside down, pretty normal, right? Compare it to these:

What the heck happened here, and why does the left photo look like something you'd find on the pages of Fark? Consider this: the two pairs of photos are exactly the same, except rotated 180 degrees.

This is the best example of the Thatcher Illusion, an optical illusion I stumbled upon this weekend. People don't tend to recognize faces holistically; rather, we focus on local features like the eyes or mouth. As a result, if you invert the photo but leave the eyes rotated "normally", it looks fine. But do the inverse and you've midwived a monstrosity. You can see more examples (including a polarity-reversed photo of Tony Blair) here.

This reminds me of my favorite facial optical illusion: