Fixing YAML for Fun

One of the nicest parts of working as a consultant developer is that I get to help solve other people's problems, and that I get paid decently well to do so. This means that I spend my days solving mostly interesting problems, and that once I leave work, I can leave those problems behind me. Previously, while working on a doctoral thesis and then as the CTO of a startup, it wasn't quite so easy to leave work problems at work.

With this background, it does feel a little odd to "just" do a full day's work, and then have "free time" (another odd concept). So I've ended up filling some of that time with more coding, but on my own terms: working on problems that I get to pick entirely myself. Doing so I've found that code can on occasion feel like an artistic medium, especially when it's sufficiently disconnected from commercial concerns.

In other words, I write and publish open source software for fun. I let myself fall down rabbit holes and do things right, because that's sometimes more important than doing things efficiently.

All of which is a long-winded and only slightly relevant intro to this: I've just released the 1.0 version of my JavaScript YAML library, which is available as yaml on npm. As far as I know, it is the most complete and correct parser and stringifier for all versions of YAML, and it is the only library that makes it possible to work with YAML comments.

In case YAML is not a familiar term, it brands itself as a "human friendly data serialization standard", and as such it's one of the most common configuration file formats that are currently in use. I started using it for a project last fall, but then got stuck when I discovered that none of the existing tools (in any language!) supported YAML comments, which I wanted to also write. As the project in question was one I was doing outside of work, my solution wasn't to work around the problem this presented, but to fundamentally solve it.

I've done my best to make sure that the documentation for yaml is as thorough as possible, so it's perhaps best if I leave the intricacies of its API outside this blog post. Besides those, there are two particular tales to tell from its development so far that helped make me realise that this thing I'd built might not just be a toy that I was playing with:

During the early development of yaml, I went and gathered all of the test cases I could find for YAML. As I got more and more of the language features implemented, I could un-comment more and more of the tests, until finally there were only a couple left. But these final tests baffled me; I'd lifted them directly from the spec, and my parser was failing with them. I batted my head at them for days – literally! – until I finally realised that it wasn't my code that was broken, it was the spec itself. Or at least its included code examples. So I submitted issues and pull requests for the spec and its accompanying test suite, and most of them have now been resolved.

The other major event that's happened so far is that the library gained its first public user a little over a month ago, when Prettier 1.14 was released with YAML support. During the development leading to that as well as subsequently, some rather obscure bugs have been found and squished.

As with any other software, nothing's ever quite finished and the story never ends. Even now I'm thinking of rewriting parts of the parser, and there are improvements to make, but this feels like a good spot to call the library ready for production use and brand it as 1.0. Also, with this done I can finally continue from where I was last fall, and finally fix JavaScript localization. But more on that later.

Eemeli Aro

Liity keskusteluun