A Livebook Smart-Cell to Render Diagrams
I wrote my first Livebook smart-cell which renders diagrams from a textual description.
I recently discovered Kroki, an opensource tool which generates images from the text notation of popular diagramming languages. I then thought, Livebook makes such an excellent application companion for runnable documentation. What if we could easily embed architectural (or any kind) of diagrams without having to go through some complicated pipeline to convert the diagram definition into an image first?
Yes, we can!
An old Chinese proverb goes “Up-to-date runnable documentation with diagrams is worth a thousand Confluence pages”. Elixir’s standard documentation generator, ex_doc makes it trivial to include Livebook notebooks in your documentation and run them. It even supports Mermaid out of the box, you just need to wrap the source in mermaid backticks.
```mermaid
graph TD;
A-->B;
A-->C;
B-->D;
C-->D;
```
What if your diagrams are not part-fish? No problem, there’s kino_kroki.
You quickly can give it a go by following the link below:
Note: Please mind that you need to be running Livebook > 0.6
To try it from an existing notebook, throw kino_kroki to the mix with:
Mix.install([:kino_kroki])
Then you’ll notice a new type of smart-cell titled diagram.
An editor will appear where you can paste and edit the diagram source and when you evaluate the cell, the diagram will be rendered.
Prior Art
Smart-cells are a powerful way to extend the functionality of Livebook. There’s a fantastic guide bundled with Livebook, to help you write your own smart-cell, which you can read here.
I took inspiration from github_graphql_smartcell which is somewhat similar to Kino.Kroki in the sense that they both include a source editor.
How it Works
As mentioned in the intro, this smart-cell is powered by Kroki. It’s API takes the diagram source as a URL encoded and compressed string and responds with an image.
The heart of the smart-cell is this tiny module:
defmodule Kino.Kroki do
@moduledoc """
A simple encoder for the online diagram renderer https://kroki.io/
"""
@doc """
Returns a `Kino.Markdown` image to render the diagram.
### Examples
iex> Kino.Kroki.new(Kino.Kroki.Sample.get(:graphviz), :graphviz)
%Kino.Markdown{
content: "![svg](https://kroki.io/graphviz/svg/eJx9kM"
}
"""
@spec new(graph :: String.t(), type :: Kino.Kroki.Samples.type()) :: Kino.Markdown.t()
def new(graph, type) do
graph
|> :zlib.compress()
|> Base.url_encode64()
|> then(&"https://kroki.io/#{type}/svg/#{&1}")
|> then(&"")
|> Kino.Markdown.new()
end
end
Compressing and encoding in Elixir and OTP as you can see, is particularly simple, compared to some other platforms (see examples). With the time left I contributed the Elixir example.
Making it Smart
So far, there’s nothing particularly dynamic about Kino.Kroki, it’s a
regular Elixir module which can return a Markdown image.
To better understand the concept of a smart-cell, consider it a code template parameterized through UI interactions. At any point you can convert a smart-cell into a code cell for further modifications.
We’ll dissect each part of the module below:
defmodule Kino.KrokiSmartcell do
use Kino.JS
use Kino.JS.Live
use Kino.SmartCell, name: "Diagram"
# omitted
end
We’re defining a module for the smart-cell, which adds Kino.JS making
the module asset-aware allowing us to inject JavaScript for our cell
through the asset "main.js" macro as well as CSS through asset "main.css".
We also use Kino.JS.Live since we want the cell to be dynamic,
listening to the update_type event then setting a diagram sample
according to the selected type.
- With the
init/2callback, that receives the argument and the “server” context, we initialise the type, setting it to “graphviz” and we also fetch and assign a sample textual definition of a GraphViz diagram. More details about the module returning samples will be available further below.
@default_type "graphviz"
@impl true
def init(_attrs, ctx) do
ctx =
ctx
|> assign(type: @default_type)
|> assign(diagram: Kino.Kroki.Samples.get(@default_type))
{:ok, ctx}
end
With the handle_connect/1, which is invoked whenever a new client connects,
we set up the initial state of the new client.
@impl true
def handle_connect(ctx) do
{:ok, %{type: ctx.assigns.type, diagram: ctx.assigns.diagram}, ctx}
end
- The
handle_event/3callback is responsible for handling any messages sent by the client. We fetch a sample diagram source for the selected type, update the context and broadcast an event which will be handled by JavaScript side of this cell and update the editor element.
@impl true
def handle_event("update_type", type, ctx) do
ctx = assign(ctx, type: type)
diagram = Kino.Kroki.Samples.get(type)
ctx = update(ctx, :diagram, fn _ -> diagram end)
broadcast_event(ctx, "update_type", %{type: ctx.assigns.type, diagram: diagram})
{:noreply, ctx}
end
With the to_attrs/1 callback we compute the arguments for to_source/1. We set the type
and diagram source.
@impl true
def to_attrs(ctx) do
%{"type" => ctx.assigns.type, "diagram" => ctx.assigns.diagram}
end
We implement to_source/1 which returns the Elixir code for our cell.
This is what will replace our smart-cell if we convert it to a “dumb” cell.
We return the AST of the Elixir source which by calling Kino.Kroki.new/2 renders a markdown image
generated by the Kroki server.
Kino.SmartCell.quoted_to_string/1 converts the AST to a readable, formatted code string.
@impl true
def to_source(attrs) do
quote do
Kino.Kroki.new(unquote(attrs["diagram"]), unquote(attrs["type"]))
end
|> Kino.SmartCell.quoted_to_string()
end
The JavaScript source of Kino.Kroki is:
asset "main.js" do
"""
export function init(ctx, payload) {
ctx.importCSS("main.css");
ctx.importCSS("https://fonts.googleapis.com/css2?family=Inter:wght@400;500&display=swap");
root.innerHTML = `
<div class="app">
<form>
<div class="container">
<div class="row header">
<div class="inline-field">
<label class="inline-input-label label">Diagram type</label>
<select class="input" name="type"/>
<option value="blockdiag">BlockDiag</option>
<!-- rest of the options omitted -->
</select>
</div>
<div class="logo">
<span>Powered by: </span>
<a href="https://kroki.io">
<img alt="kroki" src="https://kroki.io/assets/logo.svg"/>
</a>
</div>
</div>
<div class="row">
<div class="field grow">
<label class="input-label">Diagram Source</label>
<textarea
id="diagram-source"
name="diagram"
class="input textarea code"
placeholder=""
rows="25">#{Kino.Kroki.Samples.get(@default_type)}</textarea>
</div>
</div>
</container>
</form>
</div>
`;
const typeEl = ctx.root.querySelector(`[name="type"]`);
const diagramEl = ctx.root.querySelector(`#diagram-source`);
typeEl.addEventListener("change", (event) => {
ctx.pushEvent("update_type", event.target.value);
});
ctx.handleEvent("update_type", (event) => {
typeEl.value = event.type;
diagramEl.value = event.diagram;
});
ctx.handleSync(() => {
// Synchronously invokes change listeners
document.activeElement &&
document.activeElement.dispatchEvent(new Event("change"));
});
}
"""
end
We use the asset/2 macro to define the necessary JS code inline, but
there’s also the option to set a path to load assets with:
use Kino.JS, assets_path: "lib/assets/kino_kroki"
We need to export an init function for our JavaScript module. The
first parameter ctx is the client-side context which provides a
variety of useful functions such as importCSS which we use to load CSS
from a URL.
We initialise the root element of the smart-cell with root.innerHTML
and set up event handlers. With ctx.pushEvent we push a client-side
change to the cell server and with ctx.handleEvent we apply changes
from the server.
typeEl.addEventListener("change", (event) => {
ctx.pushEvent("update_type", event.target.value);
});
ctx.handleEvent("update_type", (event) => {
typeEl.value = event.type;
diagramEl.value = event.diagram;
});
ctx.handleSync(() => {
// Synchronously invokes change listeners
document.activeElement &&
document.activeElement.dispatchEvent(new Event("change"));
});
The complete source of Kino.Kroki can be found here.
Finishing Touches
We need to make sure Livebook knows about this new smart-cell to list it as one of the available options.
In the mix.exs of Kino.Kroki there is:
def application do
[
mod: {Kino.Kroki.Application, []}
]
end
and in the application module:
defmodule Kino.Kroki.Application do
@moduledoc false
use Application
@impl true
def start(_type, _args) do
# 👇 This tells Kino we implemented a smart-cell
Kino.SmartCell.register(Kino.KrokiSmartcell)
Supervisor.start_link([], strategy: :one_for_one, name: KinoDB.Supervisor)
end
end
Diagram Samples
Kino.Kroki is the last module of the smart-cell. It reads and parses a text file
containing samples for the diagram types Kroki can generate images from.
Let’s take it apart to see if there are any useful patterns in there (hint: there are).
The text file containing the samples has the following format:
<--- sample:blockdiag --->
blockdiag {
Kroki -> generates -> "Block diagrams";
Kroki -> is -> "very easy!";
Kroki [color = "greenyellow"];
"Block diagrams" [color = "pink"];
"very easy!" [color = "orange"];
}
<--- sample:wavedrom --->
{ signal: [
{ name: "clk", wave: "p.....|..." },
{ name: "Data", wave: "x.345x|=.x", data: ["head", "body", "tail", "data"] },
{ name: "Request", wave: "0.1..0|1.0" },
{},
{ name: "Acknowledge", wave: "1.....|01." }
]}
We start by assigning the location of the file to a module attribute:
defmodule Kino.Kroki.Samples do
@samples_file :code.priv_dir(:kino_kroki)
|> Path.join("samples.txt")
Then with:
@external_resource @samples_file
We instruct the compiler to recompile Kiko.Kroki.Samples each time there’s a change to the samples file.
We then parse the file into a map where the keys are diagram types and the values are samples.
@samples_separator ~r/<--- sample:(?<type>.+) --->\n/m
@samples @samples_file
|> File.read!()
|> String.split(@samples_separator,
include_captures: true,
trim: true
)
|> Enum.chunk_every(2)
|> Enum.map(fn
[separator, diagram] ->
{String.to_atom(Regex.named_captures(@samples_separator, separator)["type"]),
String.trim(diagram)}
end)
|> Map.new()
We also define a type type, which is a union of all the supported diagram types.
@type type :: unquote(Enum.reduce(Map.keys(@samples), &{:|, [], [&1, &2]}))
This is rendered nicely in the docs and helps the user of this library provide a valid argument when used programmatically.
Outro
There’s a lot of ground to cover with the recent extensibility improvements the amazing developers, community and sponsors a bringing to Livebook. I recommend checking out the docs for:
If you are seeking inspiration for your next notebook or smart-cell, check out notes.club which is an Elixir app to help you discover notebooks contributed by the community.
- Coming soom to a notebook near you “an
open_api_spexsmart-cell”, stay tuned!
Spotted a Mistake?
Please contact me on twitter, or in the comments, or submit a PR for corrections.