Those Prophetic Compiler Advocates

The other day I was reading the Internet, and I stumbled across a few posts advocating that one should write a compiler as a side project, because it will teach a person so much computer science. Despite having 43 side projects, I decided that the posts advocating that people write compilers as a side project are really, really convincing, and started fiddling with one right away. (If it ever gets into a state where I’d want other people to see it, I’ll put it up on GitHub and put a link here.)

In particular, Steve Yegge advocates writing compilers as a way of learning some of the really important things in computer science.

After I read that post, I started thinking about all the weird things I’ve worked on that are really a lot like compilers. For example, I worked on a project that allowed users to make tagged email templates, so that they could write things like “Dear {contact:firstname}” and it would come out as “Dear Frederick” or whatever when they sent the email. To get this working, I needed to compute the smallest set of queries that would let us populate all the little variable fields in the template. To do this, we split a string into tokens, build a tree out of tokens (where contact table is the root, intermediate nodes are tables we’re joining to, and leaves are columns we’re fetching data from), and emit a different strings, composed by arranging the tree’s nodes. The new string is a query we can send off to the database. It’s a weird-proprietary-email-language to weird-proprietary-query-language transpiler, but at end of the day it still tokenizes, parses, builds a tree, and emits code for consumption by some other program. For some value of “compiler” it’s a compiler.

I was working on a similar project the other day. Basically, some queries should be done through a REST API, and some queries can be done by calling SQL server directly. The API is easier to call, because it’s part of an existing SDK and someone else already wrote the complicated code, but calling SQL is a lot faster, so we want to call SQL when the query is simple enough that we’re confident of getting the SQL right. So we need to, guess what, convert the filter expressions and column sets and whatnot from the query object into some kind of tree, walk the tree, decide whether we can emit SQL, and if so emit SQL. Again, this is a lot like a compiler.

I noticed both of these projects’ similarities to compiler building after reading Steve Yegge’s post. It’s weird how slowly computer science changes, in some ways. I mean, compilers have been on computer science curricula pretty much forever (at least, forever in Internet years), and it’s 2016.

But there you have it. Being able to build a simple compiler that takes in strings and emits instructions is still super important. Turns out those prophetic compiler advocates were onto something.

So, if you really want to do something with computer science that will drive your skills forward, consider building a compiler. I hear the “dragon book” (so named for its cover art) is the one to read, and it’s definitely on my reading list for the future. When I get through it, I’ll make sure to post a review.

Till next week, happy learning,


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s