Generate diagrams programmatically

January 11, 2023

by Alexander Wang

D2 is a modern text-to-diagram language. It is built with extensibility in mind.

...D2 has a built-in API for manipulating the AST at a high-level (more intelligent than just CRUD on AST nodes). This lets anyone build up and edit a diagram programmatically. The goal isn't to enable a specific use case, but rather unforeseen ones for the infinite workflows out there.
- https://d2lang.com/tour/future#developer-tooling

This blog post will demonstrate a concrete example of that, by using D2's language API to build a diagram that visualizes the schema after each line of SQL statement.

We'll turn this set of statements...


CREATE TABLE movie;
ALTER TABLE movie ADD COLUMN id integer;
ALTER TABLE movie ADD COLUMN star integer;
ALTER TABLE movie ADD COLUMN budget integer;
ALTER TABLE movie ADD COLUMN profit integer;
ALTER TABLE movie ADD COLUMN producer integer;
ALTER TABLE movie ADD COLUMN dialogue integer;

CREATE TABLE actor;
ALTER TABLE actor ADD COLUMN id integer;
ALTER TABLE actor ADD COLUMN name text;
ALTER TABLE actor ADD COLUMN native_lang integer;

CREATE TABLE producer;
ALTER TABLE producer ADD COLUMN id integer;
ALTER TABLE producer ADD COLUMN name text;
ALTER TABLE producer ADD COLUMN native_lang integer;

CREATE TABLE language;
ALTER TABLE language ADD COLUMN id integer;
ALTER TABLE language ADD COLUMN name text;

ALTER TABLE movie ADD CONSTRAINT fk_movie_actor FOREIGN KEY (star) REFERENCES actor (id)
ALTER TABLE movie ADD CONSTRAINT fk_movie_producer FOREIGN KEY (producer) REFERENCES producer (id)
ALTER TABLE movie ADD CONSTRAINT fk_movie_language FOREIGN KEY (dialogue) REFERENCES language (id)
ALTER TABLE producer ADD CONSTRAINT fk_producer_language FOREIGN KEY (native_lang) REFERENCES language (id)
ALTER TABLE actor ADD CONSTRAINT fk_actor_language FOREIGN KEY (native_lang) REFERENCES language (id)

... into these SVGs.

Notice how each step corresponds with a SQL query from above, creating a visualization of the schema being built up.

Step 1: Setup

The following code will call D2 libraries to create a new diagram with a single shape and render the SVG file. I will hide the imports and skip the error checking to avoid boilerplate. You can find runnable code for this demo in the GitHub repository.


func main() {
	ctx := context.Background()
	// Start with a new, empty graph
	_, graph, _ := d2lib.Compile(ctx, "", nil)

	// Create a shape, "meow"
	graph, _, _ = d2oracle.Create(graph, "meow")

	// Turn the graph into a script (which would just be "meow")
	script := d2format.Format(graph.AST)

	// Initialize a ruler to measure font glyphs
	ruler, _ := textmeasure.NewRuler()

	// Compile the script into a diagram
	diagram, _, _ := d2lib.Compile(context.Background(), script, &d2lib.CompileOptions{
		Layout:  d2dagrelayout.DefaultLayout,
		Ruler:   ruler,
	})

	// Render to SVG
	out, _ := d2svg.Render(diagram, &d2svg.RenderOpts{
		Pad: d2svg.DEFAULT_PADDING,
	})

	// Write to disk
	_ = ioutil.WriteFile(filepath.Join("out.svg"), out, 0600)
}

All of this code is from D2's docs on how to use it as a library.

This gives us the following SVG:

Step 2: Parse SQL statements

The following code goes through SQL statements and gives us structured data, like what command and what table that command is called on. Bunch of assumptions, e.g. statements must be one line. Not important/point of demo.


type Query struct {
	Command string
	Table   string
	Column  string
	Type    string

	ForeignTable  string
	ForeignColumn string
}

func parseSQL(plan string) (out []Query) {
	lines := strings.Split(plan, "\n")

	for _, line := range lines {
		if len(strings.TrimSpace(line)) == 0 {
			continue
		}
		out = append(out, parseSQLCommand(strings.Trim(line, ";")))
	}
	return out
}

func parseSQLCommand(command string) Query {
	q := Query{}

	words := strings.Split(command, " ")
	if strings.HasPrefix(command, "CREATE") {
		q.Command = "create_table"
		q.Table = words[2]
	} else if strings.Contains(command, "ADD COLUMN") {
		q.Command = "add_column"
		q.Table = words[2]
		q.Column = words[5]
		q.Type = words[6]
	} else if strings.Contains(command, "ADD CONSTRAINT") {
		q.Command = "add_foreign_key"
		q.Table = words[2]
		q.Column = strings.Trim(strings.Trim(words[8], "("), ")")
		q.ForeignTable = words[10]
		q.ForeignColumn = strings.Trim(strings.Trim(words[11], "("), ")")
	}

	return q
}

Step 3: Call d2oracle

d2oracle is the API package within D2. It can be used to create and delete objects and connections, set attributes, and move objects to different containers. For example, given this D2 script:


netflix: {
  movie
}
hulu

let's watch -> netflix.movie

You can call d2oracle.Move(graph, "netflix.movie", "hulu.movie") to move the movie object from one container to another, and the resulting script will be:


netflix
hulu: {
  movie
}

let's watch -> hulu.movie

Going back to our demo, each SQL statement can then be translated into a d2oracle call.


func (q Query) transformGraph(g *d2graph.Graph) *d2graph.Graph {
	switch q.Command {
	case "create_table":
		// Create an object with the ID set to the table name
		newG, newKey, _ := d2oracle.Create(g, q.Table)
		// Set the shape of the newly created object to be D2 shape type "sql_table"
		shape := "sql_table"
		newG, _ = d2oracle.Set(g, fmt.Sprintf("%s.shape", newKey), nil, &shape)
		return newG
	case "add_column":
		newG, _ := d2oracle.Set(g, fmt.Sprintf("%s.%s", q.Table, q.Column), nil, &q.Type)
		return newG
	case "add_foreign_key":
		newG, _, _ := d2oracle.Create(g, fmt.Sprintf("%s.%s -> %s.%s", q.Table, q.Column, q.ForeignTable, q.ForeignColumn))
		return newG
	}
	return nil
}

Step 4: Putting it all together

Read in the .sql file.
Turn each raw string statement into a structured command (with function from step 2).
Pass each command into step 3's function to programmatically edit the diagram.
Render an SVG file after each line.


ctx := context.Background()
_, graph, _ := d2lib.Compile(ctx, "", nil)
ruler, _ := textmeasure.NewRuler()

// 1----
f, _ := ioutil.ReadFile(filepath.Join("plan.sql"))

// 2----
queries := parseSQL(string(f))

for i, q := range queries {
  // 3----
  graph = q.transformGraph(graph)

  script := d2format.Format(graph.AST)

  diagram, _, _ := d2lib.Compile(context.Background(), script, &d2lib.CompileOptions{
    Layout: d2dagrelayout.DefaultLayout,
    Ruler:  ruler,
  })

  // 4----
  out, _ := d2svg.Render(diagram, &d2svg.RenderOpts{
    Pad: d2svg.DEFAULT_PADDING,
  })

  _ = ioutil.WriteFile(filepath.Join("svgs", fmt.Sprintf("step%d.svg", i)), out, 0600)
}

Diagrams backed by text

If you had used a graphics library to do this, the resulting SVG is ossified. But, with D2, for any diagram it produces, you can get the text backing it. Inserting this line at the end of our program...


_ = ioutil.WriteFile("out.d2", []byte(d2format.Format(graph.AST)), 0600)

...gets you this D2 script:


movie: {
  id: integer
  star: integer
  budget: integer
  profit: integer
  producer: integer
  dialogue: integer
}
movie.shape: sql_table
actor: {
  id: integer
  name: text
  native_lang: integer
}
actor.shape: sql_table
producer: {
  id: integer
  name: text
  native_lang: integer
}
producer.shape: sql_table
language: {
  id: integer
  name: text
}
language.shape: sql_table
movie.star -> actor.id
movie.producer -> producer.id
movie.dialogue -> language.id
producer.native_lang -> language.id
actor.native_lang -> language.id

Want to customize this diagram for an audience? Give to your product manager or designer to tweek and reuse elsewhere? Style to your liking, or enrich with interactivity? There's no need to have to modify the source code for an offshoot. Having the backing text to a diagrams means you can store, version-control, and further modify in an easy, accessible format. Maybe your colleague takes the diagram and gives a few notes:

Link to Playground

Being extensible and composable have been key to D2's popularity in the 2 months it's been open-source. It's enabled us to move fast on integrating technologies like MathJax for Latex, or RoughJS for the hand-drawn look above. It's also enabled the community to easily build on top of D2. And importantly, it gives users optionality, like the ability to swap out layout engines, or the choice of themes (or to make their own!).