terrastruct / tala Goto Github PK

Hi,
I previously worked on a pet project similar to d2 - but not nearly as good! :) - based on graphviz, for producing technology architecture pictures. I found that a lot of diagrams would look much closer to one that a person would draw by introducing a 'grow-clusters?' parameter.

It would take a picture from this:

to this:

Graphviz doesn't have that setting and didn't have the man-power to introduce it so I had to hack the svg and run an optimisation over it. It was ugly!

Can work wonders though for some diagrams. Any thoughts for tala?

This setting might not work as tala; In graphviz (dot) you can force clusters into a row/ rank in the dot algorithm for introducing a bunch of between cluster invisible edges and from my experiments with tala, it works quite differently to dot.

E.g. make a file, run it with TALA, it should look like this: asdf

TALA has a pathfinding algorithm. We can replace the naive straight lines of grids with better routing

Getting this error

WARNING: THIS COPY OF TALA IS UNLICENSED AND IS FOR EVALUATION PURPOSES ONLY
missing api token: could not find $TSTRUCT_TOKEN or api_token in auth file ~/.config/tstruct/auth.json

Would be nice to get a warning before, now I have to reset my d2.

use latex

Note, this may be well outside the scope of what tala tries to achieve since it describes geometric relationships, but I post here it here in case it is useful.

The code below results in some unnecessarily long connectors.

D: D:{
A
I
P}

M :M:{
A
I
P
}

S: L: {
A
I
P
H
}


# Surface Intermediate Mixing
S.A <-> M.A
S.I <-> M.I
S.P <-> M.P

# THC downwelling
S.H -> D.A

# THC advection
D.A -> D.I
D.I -> D.P
M.P -> M.I
M.I -> M.A

# THC Upwelling
M.A <- D.A
M.I <- D.I
M.P <- D.P
M.A -> S.H

Maybe some people want to model the effect of routers or something? I don't know. I'd suggest making coalescing a non-default config option. This behavior doesn't correctly represent semantics at the higher layers of the software stack.

Trying to evaluate tala on Windows. d2 version 0.5.0, tala version 0.3.8. Get this error. Elk works fine.

C:>d2 -l tala -s test.d2.txt

WARNING: THIS COPY OF TALA IS UNLICENSED AND IS FOR EVALUATION PURPOSES ONLY

missing api token: could not find $TSTRUCT_TOKEN or api_token in auth file ~.config\tstruct\auth.json

Run d2plugin-tala.exe --help for more info.

WARNING: THIS COPY OF TALA IS UNLICENSED AND IS FOR EVALUATION PURPOSES ONLY

missing api token: could not find $TSTRUCT_TOKEN or api_token in auth file ~.config\tstruct\auth.json

Run d2plugin-tala.exe --help for more info.

err: failed to compile: failed to unmarshal json: json: cannot unmarshal object into Go struct field Attributes.attributes.constraint of type []string

I can fake a token and then it says

failed to verify $TSTRUCT_TOKEN: illegal base64 data at input byte 4

but the final error is the same.

windows has convention to use USERPROFILE as well as some other paths. should detect runtime and choose appropriately, instead of always assuming HOME is the env var for home directory.

related: https://github.com/mitchellh/go-homedir/blob/main/homedir.go

See terrastruct/d2#121

Just copy the example command from the frontend @alixander

Maybe you should open-source TALA, and try to earn money on the IDE -- which looks great!

We're deciding between D2 and the Mermaid ecosystem, and I like the output of yours (much nicer graphs etc). But I'm also hesitant to build all our docs on a semi-closed ecosystem where the primary engine isn't free, and Mermaid is natively supported by Github and many others.

Any business based on open-source is always going to be in this pinch, give too much away, and you don't earn anything, give too little away, and you won't have any adoption.

I'm not saying it's an easy choice, but from my (very limited) vantage point, D2 looks great but has much lower traction than open-source Mermaid. Maybe selling the canonical IDE for D2 will still be a great business, even if you properly open-source TALA?

(I know D2-lang is open-source, but it's success is still coupled with your firm + TALA not being open-source is still a downside)

Not writing this to be annoying, just want to provide some feedback.

Hi,
On large diagrams, Tala introduces quite a lot of empty space.
Would it be possible to introduce a setting to 'compress' the space?

Hi,

We've just purchased TALA license and trying to integrate it into our CI. We provided TSTRUCT_TOKEN environment variable with one of the tokens we received. However, d2plugin-tala --help fails with:

[2024-02-12T22:03:41.916Z] WARNING: THIS COPY OF TALA IS UNLICENSED AND IS FOR EVALUATION PURPOSES ONLY

[2024-02-12T22:03:41.916Z] 

[2024-02-12T22:03:41.916Z] failed to verify $TSTRUCT_TOKEN: illegal base64 data at input byte 5

And our diagrams are generated with the watermark.

On top of the err: Timed out: EdgeRouting error specified in #20, experimenting with different seeds to generate a fairly large diagram would also sometimes produce either of the following errors:

1. failed to compile: exit status 2
   stderr:
   ...
   panic: runtime error: invalid memory address or nil pointer dereference
   [signal 0xc0000005 code=0x0 addr=0x70 pc=0x11116ba]
   

2. failed to compile: exit status 1
   stderr:
   ...
   err: Reached a bad state: Dimensions w:13458, h:20121 reached after stage GapNormalization
   

The larger the diagram, the higher the chances of these errors occurring (and the lower the chances of successfully generating a diagram).

The diagram does not have to contain $>= 100$ nodes for these errors to start cropping up. I encountered the above issues on a moderately-sized diagram of around 50-60 nodes/shapes (with a maximum of 2 hierarchy levels), 90-100 connections, and a decent amount of text (around 13000 characters including shape labels) sprinkled across the entire diagram.

TALA should be able to handle individual direction on edges.

a -> b: { direction: left }

We can ensure this.

It is useful to group shapes into more than one container. For example, one container can represent where something is physically deployed, while a cross-cutting container could show how it is logically deployed.

Hi! I just thought it would be nice to have the option of specifying the TALA seed in the source file. This would make diagrams created with TALA more reproducible. Now, if somebody makes a diagram and finds a seed that yields a particularly nice result, the diagram cannot be readily reproduced even in the same version of TALA (unless the author notes the seed in a comment).

Tala panics when I try to generate a svg from this content:

# generated at 2023-04-14T09:36:16.817Z

____header: |||md
Index
||| { near: top-center }
____body: |||md
[HLD](hld/index.svg) | [LLD](lld/index.svg)
|||

Example

% cat perdu.d2
# generated at 2023-04-14T09:36:16.817Z

____header: |||md
Index
||| { near: top-center }
____body: |||md
[HLD](hld/index.svg) | [LLD](lld/index.svg)
|||
% d2 --layout=tala perdu.d2
err: failed to compile: exit status 2
err: stderr:
err: panic: runtime error: invalid memory address or nil pointer dereference
err: [signal SIGSEGV: segmentation violation code=0x2 addr=0x40 pc=0x10592a6f0]
err:
err: goroutine 67 [running]:
err: github.com/terrastruct/src/backend/tala/transpilers/d2transpiler.PopulateNodes.func3(0x14000e09450, 0x18?)
err:    github.com/terrastruct/src/backend/tala/transpilers/d2transpiler/d2transpiler.go:87 +0xd0
err: github.com/terrastruct/src/backend/tala/transpilers/d2transpiler.PopulateNodes.func1(0x14000e09450, 0x0?, 0x14000c94f30)
err:    github.com/terrastruct/src/backend/tala/transpilers/d2transpiler/d2transpiler.go:37 +0x38
err: github.com/terrastruct/src/backend/tala/transpilers/d2transpiler.PopulateNodes.func1(0x14000e08000, 0x57199f883bccd?, 0x14000c94f30)
err:    github.com/terrastruct/src/backend/tala/transpilers/d2transpiler/d2transpiler.go:40 +0x7c
err: github.com/terrastruct/src/backend/tala/transpilers/d2transpiler.PopulateNodes({0x14000d2ba98?, 0x104c53df4?}, 0x140008fd970, 0x14000caafc0)
err:    github.com/terrastruct/src/backend/tala/transpilers/d2transpiler/d2transpiler.go:82 +0x204
err: github.com/terrastruct/src/backend/tala/transpilers/d2transpiler.RaceSeeds({0x106a76418, 0x14000a3c780}, 0x106a76418?, 0x14000caafc0, {0x14000580760, 0x3, 0x140001e6e40?}, 0x0, 0x0)
err:    github.com/terrastruct/src/backend/tala/transpilers/d2transpiler/d2transpiler.go:155 +0xa8
err: github.com/terrastruct/src/backend/tala/transpilers/d2transpiler.Layout({0x106a76418, 0x14000a3c780}, 0x0?, 0x0?)
err:    github.com/terrastruct/src/backend/tala/transpilers/d2transpiler/d2transpiler.go:317 +0x244
err: main.talaPlugin.Layout({0x14000c84540?, 0x14000c33d40?}, {0x106a76418?, 0x14000a3c780?}, 0x0?)
err:    github.com/terrastruct/src/backend/tala/cmd/d2plugin-tala/plugin_tala.go:161 +0x100
err: oss.terrastruct.com/d2/d2plugin.layout({0x106a76418, 0x14000a3c780}, {0x106a79660, 0x1400093ed60}, 0x140003c8d90)
err:    oss.terrastruct.com/[email protected]/d2plugin/serve.go:106 +0x94
err: oss.terrastruct.com/d2/d2plugin.Serve.func1({0x106a76418, 0x14000a3c780}, 0x140003c8d90)
err:    oss.terrastruct.com/[email protected]/d2plugin/serve.go:56 +0x2b8
err: oss.terrastruct.com/util-go/xmain.(*State).Main.func1()
err:    oss.terrastruct.com/[email protected]/xmain/xmain.go:105 +0x68
err: created by oss.terrastruct.com/util-go/xmain.(*State).Main
err:    oss.terrastruct.com/[email protected]/xmain/xmain.go:103 +0x110
% 

Other info

• Same problem when using stdio instead of a file
• No problem when using elk instead of tala
• I'm using up-to-date versions, as far as brew upgrade is concerned

% d2 --version
0.4.0
% d2plugin-tala --version
v0.3.5

We love Terrastruct. We mainly use the GUI version. We would like an option to allow displaying HTML or Markdown tables in a friendly, customizable way.

Things like:

• Header being bold
• Cell shadowing each 2nd row.
• Cell outline

Markdown tables need to be more robust, and the SQL Tables option is not meant for general data.
Thanks!

in this diagram, the long label on a connection placed there has nowhere good to go. it should be ordered with shorter connections, making more space for the long connection

client1

client2

server

redis

client1 -> server: updateBoard
server -> redis: broadcast request
server -> client1: undoData
redis -> server: request
server -> client2: request

Is it possible to get the surface ocean container on top of the deep ocean container? The following code results in a good layout, but since the top and bottom are reversed, it could be more physical. Is there a way to get in the 'correct' order?

Atmosphere -> Surface Ocean.Nitrogen Fixation: N2
Surface Ocean.Nitrogen Fixation -> Surface Ocean.Photosynthesis: NH4
Surface Ocean.Photosynthesis -> Deep Ocean.Nitrification: Org. N
Deep Ocean.Nitrification -> Deep Ocean.Denitrification/Anammox: NH3
Deep Ocean.Denitrification/Anammox -> Atmosphere: N2
Deep Ocean.Nitrification -> Surface Ocean.Photosynthesis: NH3

I encountered a scenario where I preferred the elk layout to the tala layout because the tala layout wasn't symmetrical. Given that a tala design is considered to have a preference for symmetry, I thought it was worth submitting as an issue. See below.

elk

tala

d2

Records: Records of Activities {
    Training -> _.Person Role: is a
    Trainee -> _.Person Role: is a
    Remediator -> _.Person Role: is a
    Quality Reviewer -> _.Person Role: is a
}

Person -> Records.Training: can perform
Person -> Records.Trainee: can be a
Person -> Records.Remediator: can be a
Person -> Records.Quality Reviewer: can review

Person -> Person Role: performs many\nroles

so that items can wrap

Not sure if we're missing some docs here? https://terrastruct.com/tala is broken right now @alixander

TALA should be able to respect a max-width/height

          Here a variation on this problem. There is probably no way to decide algorythmically whether the photosynthesis box should be on top or on the side of the nitrogen fixation box (except that this solution would use less space).  When I was thinking of a layout grammar (for another project), I though of having syntax along the following lines:

Ocean.Surface.Nitrogen Fixation -> Ocean.Surface.Photosynthesis: NH4: right

i.e. each connection would have an optional direction keyword. Is that a feature that could be considered in the future?

direction: up

# Atmosphere -> Ocean.Surface.Nitrogen Fixation: N2
Ocean.Surface.Nitrogen Fixation -> Ocean.Surface.Photosynthesis: NH4

Ocean.Surface.Photosynthesis -> Ocean.Deep.Nitrification: Org. N
Ocean.Deep.Nitrification -> Ocean.Deep.Denitrification/Anammox: NH3
# Ocean.Deep.Denitrification/Anammox -> Atmosphere: N2
Ocean.Deep.Nitrification -> Ocean.Surface.Photosynthesis: NH3

Originally posted by @uliw in #32 (comment)

d2 in most diagrams I have tried favors scaling vertically, especially when using containers. However, in the common scenario of using a diagram for a Powerpoint, it would be ideal to instead expand wide while filling most of the space of a 16x9 aspect ratio. TALA seeds are the best bet to get it to expand wide, but it can do some funky stuff when connecting containers - 1 container up top, 4 containers at bottom, etc.

A simple way to consistently create something like the below where the containers & connections fill the space well would be a value addition to TALA

On routing a somewhat complex process flow diagram with order 100 nodes (but less than 200) the edge routing times out with:

err: Timed out: EdgeRouting

Unfortunately I am not able to share the failing example and wasn't able to quickly generate a minimal reproducible example.

Both ELK and dagre fare fine with this case, but do generate quite wide diagrams.

The clusters functionality in TALA works great, but i have found that it breaks if the parent has columns, if this is intended it should be mentioned somewhere because comming across this after it working great on nodes without grid-column does seem like a bug.

Here is a playground link with a small reproduction of this issue.

graphics here are useful

https://sales.jetbrains.com/hc/en-gb/articles/207240845-What-is-a-perpetual-fallback-license

when I start local server with

d2 -w simple.d2 --layout tala   

I get

failed to compile: plugin has routing feature but does not implement RoutingPlugin

Without tala I see diagram correctly:

d2 -w simple.d2

Content of simple.d2:

foo -> bar

The code below produces a rather tangled layout (i.e., connectors cross, are lengthy and boxes are scattered). Can this be improved? And if this cannot be done automatically, would layout hints be an option to find a better solution?

S: Surface: {
A
I
P
}

I :Intermediate:{
A
I
P
}

D: Deep :{
A
I
P}

S.A <-> I.A
S.I <-> I.I
S.P <-> I.P

I.A <- D.A
I.I <- D.I
I.P <- D.P