plafer / tissue.jl Goto Github PK
View Code? Open in Web Editor NEWFramework for building computational graphs that process any real time data source.
Home Page: https://plafer.github.io/Tissue.jl/stable/
License: MIT License
Framework for building computational graphs that process any real time data source.
Home Page: https://plafer.github.io/Tissue.jl/stable/
License: MIT License
We want an error message system as the one in Rust. For every possible error by the user, we should output an error message which explains what went wrong concisely, as well as an error code such as E1032
, which users can use in a call to Tissue.describe_error(:E1032)
in the REPL, and get a verbose description of the errors, common mistakes related to it, examples on how to fix it, etc.
This will be especially useful when writing macros, to help the user use the correct syntax.
It would be great to have a GUI that lets you define your graph topology, as well as all your calculator types. So you would add nodes, annotate them with their type (e.g. FaceDetectionCalculator
), add edges between the nodes, and the GUI would create a bunch of files with
@graph
definitionTissue.process(c::TheCalculatortype, ...)
function definition, ready to be filled in by the userIt could simultaneously be used as a graph visualization tool: launch it with a file that has a @graph
definition in it, and it will visualize the graph for you, on which you can make edits.
It would be nice to be able to define a subgraph to be used as just another node in graphs, similar to how they do it in mediapipe. However, in mediapipe, a subgraph and a graph have the same interface. We'll need to make them different because we don't expose input and output streams coming in/out of graphs.
The idea would be that you can define a subgraph with a similar language to @graph
, and be able to use it just as you'd use any other calculator. close(MySubgraph)
calls close(calculator)
on all the calculators in the subgraph. Similarly with open(calculator)
(#10) when we have it.
Defining a subgraph would look like:
using Tissue
struct DinkyCalculator end
Tissue.process(c::DinkyCalculator, input_stream) = 42
struct MergerCalculator end
Tissue.process(c::MergerCalculator, in1, in2) = in1 + in2
@subgraph MySubgraph begin
@inputstreams subgraph_input1, subgraph_input2
@calculator c1 = DinkyCalculator()
@calculator c2 = DinkyCalculator()
@calculator c3 = MergerCalculator()
@bindstreams c1 (input_stream = subgraph_input1)
@bindstreams c2 (input_stream = subgraph_input2)
@bindstreams c3 (in1 = c1) (in2 = c2)
# Sets the output stream of MySubgraph as the output stream of c3
@outputstream c3
end
Using that subgraph in a graph would look like:
using Tissue
struct SourceCalculator end
Tissue.process(c::SourceCalculator) = 42
struct PassthroughCalculator end
Tissue.process(c::PassthroughCalculator, input) = input
@graph MyGraph begin
@calculator source = SourceCalculator()
@calculator p1 = PassthroughCalculator()
@calculator p2 = PassthroughCalculator()
@calculator subgraph = MySubgraph()
@bindstreams p1 (input = source)
@bindstreams p2 (input = source)
@bindstreams subgraph (subgraph_input1 = p1) (subgraph_input2 = p2)
end
After graphs are parameterized (#4), we can reuse that same idea with subgraphs to allow parameters to be passed down to internal calculators.
The current flow limiter is currently an open loop controller. Based on how fast calculators a running, we pull new packet from the source calculator. We should "close the loop" by actually monitoring the channels in the graph to make sure they're not overflowing. This could technically happen, for example if there are sudden changes in how fast a certain calculator runs (e.g. the network drops and the calculator is waiting for the connection to come back up).
A graph can be in 3 different states: READY, RUNNING, or STOPPING.
start()
is called, the graph moves in RUNNING statestop()
is called, or the source returns nothing
, the graph moves in STOPPING state, until wait_until_done()
is called and returns, after which it goes back into READY stateNote that wait_until_done()
also does cleanup, and must be called before starting the graph again.
We can also expose a get_state(graph)
which returns the current state, if ever someone needs it.
Preferably we implement #10 before this one.
Most likely not all applications will want to run as fast as possible. The user should be able to specify a desired_fps
parameter somewhere in @graph
.
We need to share the insight that if packets are treated as immutable, and no global state is used, then a graph's outputs given a set of inputs will always be the same (assuming of course no random number is used insight a calculator).
For example, this example might give different results on different runs because global state is used.
using Tissue;
x = 0
struct Source
count
Source() = new(0)
end
function Tissue.process(c::Source)
c.count += 1
return c.count > 50 ? nothing : 1
end
struct Adder end
function Tissue.process(c::Adder, num)
global x += 1
return num
end
struct Multer end
function Tissue.process(c::Multer, num)
global x *= 2
return num
end
struct Barrier end
function Tissue.process(c::Barrier, num1, num2)
global x
x
end
@graph NonReproducibleGraph begin
@calculator source = Source()
@calculator adder = Adder()
@calculator multer = Multer()
@calculator barrier = Barrier()
@bindstreams adder (num = source)
@bindstreams multer (num = source)
@bindstreams barrier (num1 = adder) (num2 = multer)
end
The example for using mutable is very similar: Source
sends a mutable struct
object with an Atomic{Int}
field, and adder
and multer
modify that field similar to how they modify global x
in this example.
Currently, if a graph is stop
ed and wait with wait_until_done
, it cannot be started again, because close()
will have been called on all calculators. We should then
open(calculator)
function which we call in start(graph)
for all calculators.This would allow users to turn off the graph in certain situations (e.g. during off-hours) without needing to completely shut down their application.
Version: 0.2.0
Julia version: 1.6.2
To reproduce, failing test:
using Tissue
using Test
using Base.Threads: @spawn
flag_touched = false
struct Source end
Tissue.process(s::Source) = 42
struct Dropper end
Tissue.process(c::Dropper, in_num) = nothing
struct Touch end
function Tissue.process(c::Touch, in_num)
global flag_touched
flag_touched = true
end
@graph G begin
@calculator source = Source()
@calculator dropper = Dropper()
@calculator toucher = Touch()
@bindstreams dropper (in_num = source)
@bindstreams toucher (in_num = dropper)
end
g = G()
start(g)
sleep(0.5)
@test flag_touched == true
This is because we use that name internally. We should switch to gensym
'ed names instead, or var names.
This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml
to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix
on this issue.
I'll open a PR within a few hours, please be patient!
There is currently no way to dynamically change the parameters of calculators in a graph (e.g. pass as command line argument). The user should be able to name parameters in @graph
, and use them in the body.
A potential syntax would be:
@graph MyGraph(param1, param2) begin
@calculator c = SomeCalculator(@param param1, @param param2)
...
end
We might also want optional and/or keyword arguments too. And multiple constructors.
It would be great to have a tracer that traces (record + timestamp) all the packets that traveled through the graph. We would then have a visualizer where you see the actual packets going through the graph.
This would be great to get a feel for how your graph runs: which calculators are taking longer that others? Is some calculator being slow only on some types of inputs? It would probably be very useful in debugging a graph, say if it unsuspectedly hangs, or goes slower than expected.
There could also be a "timeline view", say, at the bottom, that shows you the more traditional way that traces are visualized (similar to how MediaPipe's tracer is visualizing it).
Main concepts: documents things like the flow limiter, immutability (#16), and whatever else needs a deep dive.
Design decisions: Document why we made certain decisions (e.g. only one output stream per calculator, graphs don't have input/output streams, etc.)
Some code might need to be run in the main thread (e.g. all UI operations in QT).
The idea is that if it takes 10ms to generate a packet, then we should start generating the packet 10ms in advance so that the data is ready for when the graph is ready to process it.
To implement this:
evaluate_packet_period(graph)
, we should subtract the source calculator's exec_time
from the new source period before setting it.The initial flow estimate is very much an underestimate, and the graph takes a few seconds to get to its limiting fps, depending on how we tune the alpha
parameter. The main reason for that underestimate is probably that the initial estimate takes into account compilation time.
We should rethink how to do this so that the graph gets to its limiting fps as fast and robustly (i.e. don't overflow the channels in any case) as possible. New design should also fix #3.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.