chrisranderson / beholder Goto Github PK
View Code? Open in Web Editor NEWA TensorBoard plugin for visualizing arbitrary tensors in a video as your network trains.
A TensorBoard plugin for visualizing arbitrary tensors in a video as your network trains.
At this line:
Line 116 in 4b5f245
In this plot, the x axis is the number of times it has been called, and the y axis is the time.
Wizards of the Coast might not be happy with the name "beholder". From their System Reference Document (pdf):
The following items are designated Product Identity, as defined in Section 1(e) of the Open Game License Version 1.0a, and are subject to the conditions set forth in Section 7 of the OGL, and are not Open Content: Dungeons & Dragons, D&D, Player’s Handbook .... beholder, gauth, carrion crawler, tanar’ri, baatezu, displacer beast, githyanki, githzerai, mind flayer, illithid, umber hulk, yuan-ti.
Suggestions welcome!
From #33 (comment).
Vertical stacking will allow people to use huge networks, and will also allow layer information. Layer info can include:
Beholder
constructor.Maybe make a small histogram of the entire section to the right of the section.
Users set what displays as black and what displays as white.
I should be able to use https://www.tensorflow.org/versions/r0.12/api_docs/python/image/resizing#resize_images and https://www.tensorflow.org/versions/r0.12/api_docs/python/image/encoding_and_decoding to handle all Pillow stuff. I'm wondering what the performance will be like in comparison.
It could be useful to use hue and value instead of just value. What would hue represent? I could constrain hue to a certain range to keep the visualization from getting too noisy.
Could be a FPS slider. This would be nice so that people don't have to worry about the visualizations slowing down their network. They can always leave the update function in their code.
Using images was easy to implement, but is inflexible and slow.
There is some error occured when importing the module:
Traceback (most recent call last):
File "train.py", line 28, in <module>
from beholder.beholder import Beholder
File "/usr/local/lib/python2.7/dist-packages/beholder/beholder.py", line 10, in <module>
from beholder import im_util
File "/usr/local/lib/python2.7/dist-packages/beholder/im_util.py", line 79, in <module>
colormaps = np.load('{}/colormaps.npy'.format(resources_path()))
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 370, in load
fid = open(file, "rb")
IOError: [Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/beholder/resources/colormaps.npy'
Python: 2.7
TF version: 1.2.0
OS: Ubuntu 16.04
Bazel: 0.5.3
Pip: 9.0.1
See the explanation here: https://matplotlib.org/users/colormaps.html. They're also "relatively friendly to common forms of colorblindness" (see https://bids.github.io/colormap/).
Here's an example using the magma colormap (chosen by an informal survey of coworkers):
There's a bit of a slowdown in generating the image, but it's not terrible. For this small network, it was about .03 seconds. With all data shown, it was .75 seconds. For a big VGG network showing all data, it was a bit over 2 seconds (this is all on my laptop using the battery). I haven't timed slowdown across the entire pipeline, since images now have 2 additional channels.
Here's the code, where *_data
is an array of shape [256, 3]:
def apply_colormap(image, colormap='magma'):
string_to_array = {
'magma': _magma_data,
'inferno': _inferno_data,
'plasma': _plasma_data,
'viridis': _viridis_data,
}
colormap_data = string_to_array[colormap]
return (colormap_data[image]*255).astype(np.uint8)
@jart Do you think this should be the default? Thoughts in general?
Checking the return values, etc. can be added in later.
I'll summarize the changes I think need to be made to TensorBoard here.
tensorboard/BUILD
:
a. add dependency for the plugin. This seems problematic - what if I want to change the name of the plugin? Could you add tensorboard/plugins/third-party
and then I put all of my stuff in there, and the following steps are automatically done? This could be a big request, but that way people can install new plugins just by cloning the repo, no TensorBoard update required. Maybe the third-party repos have some configuration file that TensorBoard reads for all the external changes that need to be made.
tensorboard/components/tf_globals/globals.ts
:
a. add tab. In line with my thoughts in 1, I'm picturing instead of
it is
with all of the third party plugins in a dropdown or something.
tensorboard/components/tf_tensorboard/BUILD
a. add dependency for a new dashboard.
tensorboard/components/tf_tensorboard/tf-tensorboard.html
:
a. import new dashboard
b. add template to content div
c. add _modeIs<plugin>
function
tensorboard/main.py
a. add plugin to the list of plugins.
What do you think? Does this look right? I think these are all of the changes I've made to TensorBoard. I finally have the full pipeline working though! This is by no means final, but these frames are changing (requested at 10 FPS, just changing the source on an img tag using setInterval
, which I think I'll change so that it automatically adjusts depending on whether it's receiving the same frame over and over or not):
To: @jart @dandelionmane @wchargin
I think @jart and @dandelionmane both mentioned potentially merging this repo into TensorBoard. Here are some of my questions.
To: @dandelionmane, @jart, @caisq
I named it Beholder (for now) because... it's a viz project, I'm a nerd, it's short, and I'm too lazy to think of anything else right now. Anyway, here it goes. It's pretty close to how @dandelionmane described it in tensorflow/tensorboard#130.
People should push tensors to the front end with two function calls: a constructor, Beholder
, and an update
function. Here's the flow I'm imagining.
Beholder
, with configuration options, including:
logdir
: where the logs go.window_size
: how many frames to use for calculating variance.tensors
: a list of tensors to visualize. Default behavior is "grab everything I can find".scaling
: either "layer"
or "network"
. Determines how to scale the parameters for display. Scales using the min/max of the layer or the entire network.beholder.update()
in the train loop. If visualizing variance, a size-limited queue will be used to hold the t
most recent tensors, one for each time update
is called.
logdir/beholder/mode
.logdir/beholder/current_frame
or something. Only one file will exist there at the time, so there's no need to worry about disk space. The current worry is more about memory since I'm thinking of storing millions of parameters for several timesteps.
cv2.imwrite
, but I suppose I could make a tensor from the numpy array (sounds like it could be expensive, but I haven't tested anything) and then use tf.summary.image
and a FileWriter
to save the image.t
steps? It will be faster to do it this way.Currently all values are being truncated to the number of pixels (ish) that are being display. Add an option that allows people to see every single parameter.
Or just do #10.
is_active
in beholder_plugin
.From #33 (comment). @dandelionmane, can you clarify what you meant by this?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.