Jsonnet is better than YAML for generating JSON

YAML is a well-known language for converting to JSON files. But as we have been generating Docker Compose for internal integration tests, we’ve found Jsonnet to be better. While Pomerium supports both options for getting JSON files, there’s a big discrepancy in efficiency between the two languages.

Keep in mind that this piece is strictly to discuss the use of each language when generating JSON. (And we use the term “generate” loosely here — we understand that YAML converts.)

Let’s dive right into comparing the pros and cons of YAML vs Jsonnet.

YAML

YAML (YAML Ain’t Markup Language) is a data serialization language. It is widely used for writing configuration files, data exchange between web services, and other similar purposes. It’s very popular because it’s designed to be human-readable and relatively easy to use.

Pros

One of the main advantages of YAML is its approachability. The format is designed to be easy for humans to read and write, and it does not use as many brackets and other symbols as other formats like JSON or XML. This makes it easy to understand and edit configuration files, and it also reduces the risk of errors caused by misplacing or forgetting a bracket or other symbol.

Another advantage of YAML is its support for multiple data types. YAML can handle strings, numbers, booleans, objects, arrays, and even null values, making it a versatile format for many different types of data. Additionally, YAML supports comments, which can be used to add explanations and annotations to configuration files and other data.

Cons

However, there are also some downsides to using YAML (especially if we’re trying to generate JSON).

While there are formal specifications for how YAML should be used, it still contains a certain degree of complexity. This results in nine ways to make a string, which can lead to developer confusion and incompatibility issues between different implementations of YAML.

Another disadvantage of YAML is its lack of support for certain types of data, such as dates and times. While YAML can handle numbers and strings, it does not have built-in support for more complex data types, which can make it more difficult to work with certain types of data.

Finally, YAML is unwieldy when trying to write loops or higher order programming. There’s a reason why people use programs to generate YAML (such as Helm) in certain cases; if you want to call the same function repeatedly with minor modifications, you would need to manually write it out or have a program do so.

Jsonnet

Jsonnet is a data serialization format and a programming language. As a superset of JSON, Jsonnet can parse any valid JSON file and add extra functionality to it. Jsonnet is specifically designed to easily work with JSON data, especially in large and complex projects, by providing features such as variables, conditionals, and loops. While we love Jsonnet for generating JSON (and highly recommend it, hence this piece), let’s be objective about its pros and cons.

Pros

Quite possibly the large advantage of Jsonnet is the ability to handle large and complex data sets. Being a programming language with functions and programmatic concepts, Jsonnet allows developers to use variables, conditionals, and loops to manipulate JSON data, making it easier to work with large and complex projects. Developers will find this to be a timesaver as they don’t need to manually update and maintain large JSON files. Additionally, Jsonnet also provides advanced programmatic features such as inheritance, which allows developers to reuse code and make changes more easily.

Another advantage of Jsonnet is its ability to manage configuration data. Jsonnet allows developers to define and manage configuration data in a single file, making it easy to manage and update. This can be especially useful for large and complex projects, where managing configuration data can be a challenging task.

A major security concern (and good reason why programming languages are discouraged) is whether the programming language used for configurations can become an attack vector; Jsonnet alleviates that problem. While Jsonnet is a programming language, it is purposefully limited in what it can do and is not Turing complete. While many programming languages are a dangerous vector of attack, Jsonnet’s limited capabilities actually make it extremely safe while giving developers the best of both worlds.

Finally, Jsonnet already looks a lot like JSON! This helps because the developer has a strong idea (not just abstract) of what the final result will look like.

Cons

To be fair, there are reasons why Jsonnet isn’t already ubiquitous.

One of the main disadvantages is that it has a steeper learning curve compared to YAML. As a programming language, Jsonnet requires developers to learn its syntax and semantics before they can use it effectively. Additionally, Jsonnet also requires additional setup and configuration, which can be time-consuming and may not be suitable for small projects.

Another disadvantage of JSONNET is that it may not be as widely supported as YAML. YAML is a well-established and widely used data serialization format, and many tools and libraries support it. However, Jsonnet is relatively new, making tooling limited. This means that developers may have to put in additional effort to find libraries and tools that support Jsonnet or develop additional workflows, such as using Jsonnet to generate the JSON to be loaded instead of automating this process.

Finally, the documentation is…odd. One might even call it intimidating. Because it’s not widely used (yet, we’re hoping to change that), Jsonnet won’t have a rich archived history of helpful StackOverflow posts.

Examples

The proof is always in the pudding, so here’s a few examples of where we recommend Jsonnet over YAML.

1. Jsonnet makes it easy to embed files directly into a config, such as TLS certificates:

environment: {
  CERTIFICATE: std.base64(importstr '../files/databroker.pem'),
  CERTIFICATE_KEY: std.base64(importstr '../files/databroker-key.pem'),
},

In YAML content can be embedded using block literals.

environment: {
  CERTIFICATE: std.base64(importstr '../files/databroker.pem'),
  CERTIFICATE_KEY: std.base64(importstr '../files/databroker-key.pem'),
},

But it’s not possible to import a file from the file system. In this case, there’s no mechanism to perform the necessary base64 encoding on the content. This makes it harder to maintain.

2. Jsonnet functions make it easy to generate complex JSON objects. For example generating a docker-compose service with network aliases:

local ComposeService(name, definition, additionalAliases=[]) =
  {
    [name]: definition {
      networks: {
        main: {
          aliases: [name] + additionalAliases,
        },
      },
    },
  };

YAML can include object blocks using anchors:

definition: &definition
  pomerium:
    networks:
      main:
        aliases: [pomerium]

pomerium: 
  << *definition

But anchors don’t allow customization of the data in the same way a function does, limiting their usefulness.

3. Jsonnet array comprehensions make it easy to generate arrays of JSON objects compared to YAML. Here’s some docker-compose service dependencies:

depends_on: {
  [name + '-ready']: {
    condition: 'service_completed_successfully',
  }
  for name in [
    'fortio',
    'mock-idp',
    'postgres',
    'trusted-httpdetails',
    'trusted-1-httpdetails',
    'trusted-2-httpdetails',
    'trusted-3-httpdetails',
    'untrusted-httpdetails',
    'verify',
    'websocket-echo',
    'wrongly-named-httpdetails',
  ]
},

YAML has no mechanism for generating lists. They must be written manually which can be very verbose.

Conclusions

Like all tools, it’s a matter of knowing which tool to use for the job at hand. YAML still has a place! But if you ever find yourself needing to generate JSON, we recommend considering using Jsonnet’s stronger capabilities if writing out the YAML seems daunting or even unwieldy.