Hi,
I follow command in https://github.com/awslabs/mxboard

1. pip install mxboard
2. pip install tensorflow tensorboard
3. tensorboard --help

thanks!

RuntimeError: module compiled against API version 0xc but this version of numpy is 0xa
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xa
Traceback (most recent call last):
File "/home/jacky4323/.local/bin/tensorboard", line 7, in
from tensorboard.main import run_main
File "/home/jacky4323/.local/lib/python2.7/site-packages/tensorboard/main.py", line 40, in
from tensorboard import default
File "/home/jacky4323/.local/lib/python2.7/site-packages/tensorboard/default.py", line 34, in
import tensorflow as tf
File "/home/jacky4323/.local/lib/python2.7/site-packages/tensorflow/init.py", line 22, in
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/home/jacky4323/.local/lib/python2.7/site-packages/tensorflow/python/init.py", line 52, in
from tensorflow.core.framework.graph_pb2 import *
File "/home/jacky4323/.local/lib/python2.7/site-packages/tensorflow/core/framework/graph_pb2.py", line 15, in
from tensorflow.core.framework import node_def_pb2 as tensorflow_dot_core_dot_framework_dot_node__def__pb2
File "/home/jacky4323/.local/lib/python2.7/site-packages/tensorflow/core/framework/node_def_pb2.py", line 15, in
from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
File "/home/jacky4323/.local/lib/python2.7/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 15, in
from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
File "/home/jacky4323/.local/lib/python2.7/site-packages/tensorflow/core/framework/tensor_pb2.py", line 15, in
from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
File "/home/jacky4323/.local/lib/python2.7/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 22, in
serialized_pb=_b('\n/tensorflow/core/framework/resource_handle.proto\x12\ntensorflow"r\n\x13ResourceHandleProto\x12\x0e\n\x06\x64\x65vice\x18\x01 \x01(\t\x12\x11\n\tcontainer\x18\x02 \x01(\t\x12\x0c\n\x04name\x18\x03 \x01(\t\x12\x11\n\thash_code\x18\x04 \x01(\x04\x12\x17\n\x0fmaybe_type_name\x18\x05 \x01(\tBn\n\x18org.tensorflow.frameworkB\x0eResourceHandleP\x01Z=github.com/tensorflow/tensorflow/tensorflow/go/core/framework\xf8\x01\x01\x62\x06proto3')
TypeError: init() got an unexpected keyword argument 'syntax'

TypeError: Conflict register for file "mxboard/proto/resource_handle.proto": tensorboard.ResourceHandleProto is already defined in file "tensorboard/compat/proto/resource_handle.proto". Please fix the conflict by adding package name on the proto file, or use different name for the duplication.
...
File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard_init_.py", line 21, in
from .writer import SummaryWriter
File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard\writer.py", line 28, in
from .proto import event_pb2
File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard\proto\event_pb2.py", line 16, in
from mxboard.proto import summary_pb2 as mxboard_dot_proto_dot_summary__pb2
File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard\proto\summary_pb2.py", line 16, in
from mxboard.proto import tensor_pb2 as mxboard_dot_proto_dot_tensor__pb2
File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard\proto\tensor_pb2.py", line 16, in
from mxboard.proto import resource_handle_pb2 as mxboard_dot_proto_dot_resource__handle__pb2
File "C:\Users\selcu\anaconda3\lib\site-packages\mxboard\proto\resource_handle_pb2.py", line 94, in
_sym_db.RegisterMessage(ResourceHandleProto)
File "C:\Users\selcu\anaconda3\lib\site-packages\google\protobuf\symbol_database.py", line 84, in RegisterMessage
self.RegisterMessageDescriptor(desc)
File "C:\Users\selcu\anaconda3\lib\site-packages\google\protobuf\symbol_database.py", line 95, in RegisterMessageDescriptor
self.pool._AddDescriptor(message_descriptor)
File "C:\Users\selcu\anaconda3\lib\site-packages\google\protobuf\descriptor_pool.py", line 238, in _AddDescriptor
self._CheckConflictRegister(desc, desc.full_name, desc.file.name)
File "C:\Users\selcu\anaconda3\lib\site-packages\google\protobuf\descriptor_pool.py", line 191, in _CheckConflictRegister
raise TypeError(error_msg)
TypeError: Conflict register for file "mxboard/proto/resource_handle.proto": tensorboard.ResourceHandleProto is already defined in file "tensorboard/compat/proto/resource_handle.proto". Please fix the conflict by adding package name on the proto file, or use different name for the duplication.

Process finished with exit code 1

What is the version using tensorboard? The demo of embedding is invalid.

In [1]: import mxboard

In [2]: sw = mxboard.SummaryWriter('/tmp/log')

In [3]: sw.add_text('test','test', 0)

In [4]: sw.add_text('test','test', 1)

In [5]: sw.close()

In [6]: sw.flush()

creates the following event file

        7q9     c,A-<A  |F,A*"
 :tesBtestJ

textЪ%6 A*
:tesBtesty:=% 

But tensorboard doesn't show any output

Hi,

MXNet nightly builds have been moved from PyPI to S3 (pypi/support#243). If this project requires the nightly builds for CI validation, please make sure that your CI pipeline now takes the nightly builds from https://dist.mxnet.io/python. Furthermore, as MXNet 1.7 and 2.0 are currently in progress simultaneously, we are releasing nightly builds for both 1.x and 2.x. Take mxnet-cu102 for example:

If you need the nightly builds for 1.x:

pip install --pre "mxnet-cu102<2" -f https://dist.mxnet.io/python

If you need the nightly builds for 2.x:

pip install --pre "mxnet-cu102>=2" -f https://dist.mxnet.io/python

Feel free to close this issue if the necessary changes have been made, or If the project doesn't rely on MXNet nightly builds. Thanks.

how to record multiple scalar summaries in one plot

I notice that if using mxboard, the logging package would not work properly. For example, it would write the log or print on screen.
Is there a way to avoid this issue?

After pip install mxboard, when I import mxboard, it errors:

`import mxboard as mb
TypeError Traceback (most recent call last)
in ()
----> 1 import mxboard as mb

/home/users/nn/code/py/mxnet/python/mxboard/init.py in ()
19
20 from future import absolute_import
---> 21 from .writer import SummaryWriter
22
23 version = '0.1.0'

/home/users/nn/code/py/mxnet/python/mxboard/writer.py in ()
26 import os
27 import logging
---> 28 from .proto import event_pb2
29 from .proto import summary_pb2
30 from .event_file_writer import EventFileWriter

/home/users/nn/code/py/mxnet/python/mxboard/proto/event_pb2.py in ()
14
15
---> 16 from mxboard.proto import summary_pb2 as mxboard_dot_proto_dot_summary__pb2
17
18

/home/users/nn/code/py/mxnet/python/mxboard/proto/summary_pb2.py in ()
14
15
---> 16 from mxboard.proto import tensor_pb2 as mxboard_dot_proto_dot_tensor__pb2
17
18

/home/users/nn/code/py/mxnet/python/mxboard/proto/tensor_pb2.py in ()
14
15
---> 16 from mxboard.proto import resource_handle_pb2 as mxboard_dot_proto_dot_resource__handle__pb2
17 from mxboard.proto import tensor_shape_pb2 as mxboard_dot_proto_dot_tensor__shape__pb2
18 from mxboard.proto import types_pb2 as mxboard_dot_proto_dot_types__pb2

/home/users/nn/code/py/mxnet/python/mxboard/proto/resource_handle_pb2.py in ()
39 message_type=None, enum_type=None, containing_type=None,
40 is_extension=False, extension_scope=None,
---> 41 options=None, file=DESCRIPTOR),
42 _descriptor.FieldDescriptor(
43 name='container', full_name='tensorboard.ResourceHandleProto.container', index=1,

TypeError: new() got an unexpected keyword argument 'file'`

I understand that visualizing gluon networks might be a bit trickier because of its dynamic nature. But I think that it would be nice to be able to recursively get the blocks that are part of Sequential network and display them. In my case I am using non-hybridizable blocks so can't get the -symbol.json

add_embedding() is checking for images is not None but when adding the embedding config at the end of the functions it expects images.shape which fails if images=None.

net=nn.HybridSequential()
with net.name_scope():
net.add(nn.Conv2D(64,kernel_size=3,strides=1,padding=1),
nn.BatchNorm(),
nn.Activation('relu'))
......
grads = [i.grad() for i in net.collect_params().values()]
assert len(grads) == len(param_names)
for i, name in enumerate(param_names):
sw.add_histogram(tag=name, values=grads[i], global_step=epoch, bins=1000)

File "F:\Anaconda3\envs\gluon\lib\site-packages\mxnet\gluon\parameter.py", line 522, in grad
"because grad_req='null'"%(self.name))

RuntimeError: Cannot get gradient array for Parameter 'hybridsequential2_batchnorm0_running_mean' because grad_req='null'

Is it possible to configure the logger of mxboard? If one uses frequent statements as such

with SummarWriter(logdir=.) as sw:
  sw.add_scalar()

the log of the application becomes quite cluttered with outputs like:

[INFO:root] Successfully opened events file: events.out.tfevents.1522834055.186590d665ad.ant.amazon.com
[INFO:root] Wrote 1 event to disk
[INFO:root] Wrote 1 event to disk
[INFO:root] Wrote 6 events to disk

Hi,

When I run python setup.py install,it show some errors

but I already install protobuf3 by using pip install protobuf

Hi,

I just found a ting bug in how the current mxboard generating the sprite image for embedding projector. This bug may result in wrong embedding-thumbnail association. A simple example borrowed from here is shown as below.

import numpy as np
import mxnet as mx
from mxnet import gluon
from mxboard import SummaryWriter


batch_size = 65


def transformer(data, label):
    data = data.reshape((-1,)).astype(np.float32)/255
    return data, label

train_data = gluon.data.DataLoader(
    gluon.data.vision.MNIST('./data', train=True, transform=transformer),
    batch_size=batch_size, shuffle=False, last_batch='discard')

initialized = False
embedding = None
labels = None
images = None

for i, (data, label) in enumerate(train_data):
    if i >= 1:
        # only fetch the first batche of images, i.e. 65 images -> 8*9 sprite image
        break
    if initialized:
        embedding = mx.nd.concat(*(embedding, data), dim=0)
        labels = mx.nd.concat(*(labels, label), dim=0)
        images = mx.nd.concat(*(images, data.reshape(batch_size, 1, 28, 28)), dim=0)
    else:
        embedding = data
        labels = label
        images = data.reshape(batch_size, 1, 28, 28)
        initialized = True

with SummaryWriter(logdir='./logs') as sw:
    sw.add_embedding(tag='mnist', embedding=embedding, labels=labels, images=images)

The above code takes 65 images of handwritten digits from the MNIST dataset and log them as embedding vectors with labels and original images. The generated sprite image is of grid 8 * 9, which is not square:

But, the current TensorBoard requires a square sprite image for the embedding projector working correctly (see here). So, the above non-square sprite image will make the thumbnails misaligned with their embeddings:

(e.g. embedding at the center of label 2 is displayed with thumbnail '9')

This issue can be easily solved by changing just one line to constrain the produced sprite image to be precisely squared:

I tested it locally and would like to make a pull request. ;)

I have set the i.grad != "null" which I found in here but now I receive error conv0_weight' was not initialized on context cpu(0). It was only initialized on [gpu(0), gpu(1)], but if I run it on just one gpu I receive the assertion error len(grads) == len(param_names) this occurs on all nets with a BatchNorm ones I have written and one in gluon's model zoo tried Inception, Densenet121 error:

grads = [i.grad() for i in net.collect_params().values() if i.grad_req != 'null']
50 assert len(grads) == len(param_names)
51

in (.0)
47 sw.add_graph(net)
48
---> 49 grads = [i.grad() for i in net.collect_params().values() if i.grad_req != 'null']
50 assert len(grads) == len(param_names)
51

/usr/local/lib/python3.6/dist-packages/mxnet/gluon/parameter.py in grad(self, ctx)
570 "Cannot get gradient array for Parameter '%s' "
571 "because grad_req='null'"%(self.name))
--> 572 return self._check_and_get(self._grad, ctx)
573
574 def list_grad(self):

/usr/local/lib/python3.6/dist-packages/mxnet/gluon/parameter.py in _check_and_get(self, arr_list, ctx)
225 "Parameter '%s' was not initialized on context %s. "
226 "It was only initialized on %s."%(
--> 227 self.name, str(ctx), str(self._ctx_list)))
228 if self._deferred_init:
229 raise DeferredInitializationError(

RuntimeError: Parameter 'densenet0_conv0_weight' was not initialized on context cpu(0). It was only initialized on [gpu(0), gpu(1)].

The following small blocks, how to set up can be wrapped in a large block, the network structure added with self.name_scope ()

Is there a way to make SummaryWriter append to old event file, instead of creating new one each time it's called?

Or is there any other method to prepare log files, so that if I interrupt and then continue training later, get a desired Tensorboard output (and not just graph with duplicated values)?.

1. Clarify how to install Other required packages.
2. Add project description on PyPI.

Hello

It would be helpful for serverless ML architecture to publish mxboard logs into AWS cloudwatch.

import mxnet as mx
from mxboard import SummaryWriter


with SummaryWriter(loggroup='/aws/training', logstream=f'{training-job-name}/mxboard') as sw:
    for i in range(10):
        # create a normal distribution with fixed mean and decreasing std
        data = mx.nd.normal(loc=0, scale=10.0/(i+1), shape=(10, 3, 8, 8))
        sw.add_histogram(tag='norml_dist', values=data, bins=200, global_step=i)

Parameter 'resnetv20_conv0_weight' was not initialized on context cpu(0). It was only initialized on [gpu(0), gpu(1)].

Trying to get the parameter in the proper context. If I run on one gpu I receive an assertion error on the len(grads)== len(param_names) I have also received this error when running my own nets with BatchNorm.

Hi all,

given a pretrained model, is it possible to use mxboard and the pretrianed model to visualize network graph?

Thanks

Thanks for this project!

It may be nice to include support to write logs directly to S3.
Tensorboard supports reading data directly from S3 tensorboard --logdir=s3://bucket/path. Tensorflow seems to support writing directly to S3.

cuda 9.0
unbuntu 16.043
mxnet-cu90
After installing the above software, there were no problems, but when I installed mxboard and tensorboard, there were problems.

I followed this gluon's document classification tutorial and tried to visualise my training with mxboard. So far I can visualise my model architecture, training loss, and accuracy.

However, my problem is when I want to record histograms of my model. Python throws this error:

Runtime Error: "Cannot get gradient array for Parameter [my layer name] because grad_req='null'"

then I checked my model initialise block and found these lines:

model.embedding_layer.weight.set_data(vocab.embedding.idx_to_vec)
model.embedding_layer.collect_params().setattr('grad_req', 'null')

and mxnet didn't run the gradient for this layer because I want to use my pre-trained embedding.

So, my question is how do I get gradients for the other layers except for the embedding layer?

Hi,

I was using mxboard to visualize a symbolblock. It seems that instead of exposing inputs, the final graph exposed the weights and hid the inputs. The following codes generate the attached plots.

import mxnet as mx
import mxnet.symbol as S
from mxnet import gluon
from mxboard import SummaryWriter

def _signed_embed(vocab_size, num_embed, weight_initializer=mx.init.Uniform(0.1)):
    inputs = S.var('inputs')
    label  = S.var('label')
    embed_weight = S.var('embed_weight', shape=(vocab_size, num_embed),
                           init=weight_initializer)
    embed_outputs = S.broadcast_mul(
        embed_weight.take(inputs), # batch * ... * num_embed
        label.expand_dims(-1), # batch * ... * 1
    )
    return gluon.SymbolBlock(embed_outputs, [inputs, label])
with SummaryWriter(logdir='./logs') as sw:
    sw.add_graph(_signed_embed(100,12,)(S.var('inputs'), S.var('label')))

The output file (ignore the binaries)

^Lembed_weight^R^Dnull*T
^Eparam^RK^RI{ __init__ :  [\ uniform\ , {\ scale\ : 0.1}] ,  __shape__ :  (100, 12) }
^T
^Ltake8/inputs^R^Dnull
/
^Ktake8/take8^R^Dtake^Z^Lembed_weight^Z^Ltake8/inputs
^Z
^Rexpand_dims8/label^R^Dnull
W
^Yexpand_dims8/expand_dims8^R^Kexpand_dims^Z^Rexpand_dims8/label*^Y
^Eparam^R^P^R^N{ axis :  -1 }
V
^]broadcast_mul8/broadcast_mul8^R^Mbroadcast_mul^Z^Ktake8/take8^Z^Yexpand_dims8/expand_dims8"^B^Hd<95>vº.

Please let me know if this is indeed a bug or there are some other issues with my codes. My installed versions are mxboard==0.1.0 with mxnet-cu90==1.3.0b20180601.

Hello, i install mxboard and run the demo py, but raise errors:

No handlers could be found for logger "mxboard.event_file_writer"

my conda list is as follows:
mxboard 0.1.0
mxnet 1.2.0
tensorboard 1.12.0
tensorflow 1.12.0

how can i solve this, thank u very much!

The documentation states that sw.add_embedding() can take a list of elements, convertible to string as an argument to the labels parameter.
However, the code prevents this both in add_embedding() and in _make_metadata_tsv() by throwing an error if labels is a List and not NDArray or NPArray.

It will be helpful to display operator's input/output tensors with their shape information.

Need to document the recommended way of using SummaryWriter:
with SummaryWriter vs. sw = SummaryWriter(...).

Add an FAQ section in README.

1. How to initialize a SummaryWriter instance: with SummaryWriter(...) or sw = SummaryWriter(..)?
2. What is the meaning of verbose?
3. Should I call flush() on a SummaryWriter object?
4. How to improve the logging performance?
5. Can I use the same tag for adding different types, e.g. image and histogram? No. One of them is going to be overwritten by the other.