12 Days of Retool: A Parser for an Abstract Syntax Tree | Retool Blog

To help wind down an exciting year at Retool, your friends on the Developer Experience team (Christina, Amit, and Kevin) will be sharing twelve holiday hacks to explore some parts of JavaScript and the Retool platform you might not have seen before. Every working day from today (December 7th) until December 22nd, check the Retool blog for a new hack to enjoy! A complete list of the posts so far can be found at the bottom of this page.

In today's post, we'll discover what an abstract syntax tree is, what you might use it for, and how you can see it in action by including an external JavaScript library in a Retool application.

A Parser for an Abstract Syntax Tree

One the first day of Retool holiday hacks, I'd like to share with you - a parser for an abstract syntax tree (AST).

It's possible that you've never had to directly deal with an AST before, but they are a critical component of the build tools and compilers we use every day. An AST creates an object representation of the syntactical content of a block of text, usually a source code file. An AST can be used to determine the order in which operations are performed in a program, or check it for syntax errors. You might even use an AST to automatically generate source code from dynamic input of some kind.

To play around with an AST and see what data it can give us, we will build an AST parser and visualizer for JavaScript in Retool. You can see it in action here - you can also download the application JSON definition, and import it into your own Retool instance to play around with it yourself.

To generate an AST object for a piece of JavaScript code, copy your code into the text box on the left. Click the "Generate AST" button to create a JSON visualization of your code in the container on the right! If you'd like a quick snippet of code to test with, here's the code from the screenshot above - it's part of the popular JavaScript utility library lodash.

1/**
2 * A faster alternative to `Function#apply`, this function invokes `func`
3 * with the `this` binding of `thisArg` and the arguments of `args`.
4 *
5 * @private
6 * @param {Function} func The function to invoke.
7 * @param {*} thisArg The `this` binding of `func`.
8 * @param {Array} args The arguments to invoke `func` with.
9 * @returns {*} Returns the result of `func`.
10 */
11function apply(func, thisArg, args) {
12  switch (args.length) {
13    case 0: return func.call(thisArg);
14    case 1: return func.call(thisArg, args[0]);
15    case 2: return func.call(thisArg, args[0], args[1]);
16    case 3: return func.call(thisArg, args[0], args[1], args[2]);
17  }
18  return func.apply(thisArg, args);
19}
20
21module.exports = apply;
22

JavaScript

How it works

To generate the AST object, I used acorn, a popular open source library built to generate ASTs for JavaScript code. Acorn is available on a public CDN here. Note that this hack uses version 8.7.1. I tried using the latest version as of this writing (8.8.1), but I encountered errors when using this build (both in and outside of Retool).

In order to use acorn (or any an external JavaScript library from a CDN) in Retool, I needed to add it to the set of external scripts loaded by my Retool app. After doing so, the "acorn" object was available globally anywhere I could write JavaScript in Retool. In the event handler for the "Generate AST" button, I added a short script to access the contents of the input text field, generate an AST object from that code, and store the resulting object in a temporary state variable.

1// Using acorn to create an AST from a chunk of JS code
2const ast = acorn.parse(codeTextArea.value);
3
4// "astObject" is the name of my temporary state variable
5astObject.setValue(ast);

JavaScript

With the AST generated and stored in a temporary state variable, I now just need to display it in a usable form. Luckily, Retool has a built-in component for visualizing JSON data. In the properties for this component, I use the double curly brace data binding format to bind the value for the JSON Explorer to the value of my temporary state variable.

Binding the JSON Explorer component value to a state variable containing an AST object

Wrapping up our present

I hope you enjoyed hacking around with abstract syntax trees in Retool! After adding in an external JavaScript library from a CDN, and dragging a few components together, we were able to build a UI to visualize the AST for a piece of JavaScript code. And if you're really into trees... you should check out this awesome piece from Bob Nisco on how Retool recently built a complex tree UI for the Retool editor.

Today's hack is just the beginning of 12 Days of Retool. Check back tomorrow for another holiday demo featuring "Two Tuple Hacks".

All 12 Days of Retool posts

Reader