gaphex / bert_experimental Goto Github PK

Hello, I appreciate if you can help in the following issue,
I am using the same logic (same code) to recommend movies, everything seems to be working except when am wanting the result, i got this runtime error " Graph is finalized and cannot be modified issue "
I think that the generator code part in buid_input_fn isn't working fine
I hope if you have an explanation and a solution , i ll be really thankful

Hi i try to use BertLayer encoder and run my model in google colab.
I use tensorflox 1.x

First i clone your github code

!git clone https://github.com/gaphex/bert_experimental

sys.path.insert(0, 'bert_experimental')

from bert_experimental.finetuning.text_preprocessing import build_preprocessor
from bert_experimental.finetuning.bert_layer import BertLayer
from bert_experimental.finetuning.modeling import BertConfig, BertModel, build_bert_module

When i use the encoder in my model

inp1 = tf.keras.Input(shape=(1,), dtype=tf.string)

encoder = BertLayer(bert_path="./bert-module/", seq_len=48,n_tune_layers=3, do_preprocessing=True,
                             verbose=False, pooling="mean", trainable=True, tune_embeddings=False)

h1 = tf.keras.layers.Dense(50, activation='relu')(encoder(inp1))

I have an error :

NameError Traceback (most recent call last)

in ()
9
10
---> 11 h1 = tf.keras.layers.Dense(50, activation='relu')(encoder(inp1))
12 h1 = tf.keras.layers.Dropout(0.1)(h1)
13

1 frames

/tensorflow-1.15.2/python3.6/tensorflow_core/python/autograph/impl/api.py in wrapper(*args, **kwargs)
235 except Exception as e: # pylint:disable=broad-except
236 if hasattr(e, 'ag_error_metadata'):
--> 237 raise e.ag_error_metadata.to_exception(e)
238 else:
239 raise

NameError: in converted code:
relative to bert_experimental/bert_experimental/finetuning:

bert_layer.py:124 call
    pooled = masked_reduce_mean(seq_output, input_mask)
bert_layer.py:314 masked_reduce_mean
    return tf.reduce_sum(mul_mask(x, m), axis=1) / (

NameError: name 'mul_mask' is not defined

this code run without error in july, now there is a scope pb with mul_mask

Thanks for your help

Luc

Hi all,

first of all, thank you for a great resource (especially when I get it to work). I am trying to use this approach for fine tunning (NER) in biomedical NLP with BioBERT. Could you maybe point me in the right direction as to how to solve the issues and successfully load BioBERT?

Traceback (most recent call last):
File "/Users/juricaseva/Documents/projects/know-nlp/src/models/ner/Train_BiLSTM.py", line 115, in
model.train_model(corpus_features, splits, evaluation)
File "/Users/juricaseva/Documents/projects/know-nlp/src/models/ner/Train_BiLSTM.py", line 76, in train_model
model = bilstm_model.get_model()
File "/Users/juricaseva/Documents/projects/know-nlp/src/util/neural_models.py", line 150, in get_model
))((encoder(word_input)))
File "/anaconda3/envs/know-nlp-playground/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 824, in call
self._maybe_build(inputs)
File "/anaconda3/envs/know-nlp-playground/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 2146, in _maybe_build
self.build(input_shapes)
File "/Users/juricaseva/Documents/projects/know-nlp/src/util/bert_experimental/finetuning/bert_layer.py", line 34, in build
trainable=self.trainable, name=f"{self.name}_module")
File "/anaconda3/envs/know-nlp-playground/lib/python3.6/site-packages/tensorflow_hub/module.py", line 154, in init
self._spec = as_module_spec(spec)
File "/anaconda3/envs/know-nlp-playground/lib/python3.6/site-packages/tensorflow_hub/module.py", line 36, in as_module_spec
return load_module_spec(spec)
File "/anaconda3/envs/know-nlp-playground/lib/python3.6/site-packages/tensorflow_hub/module.py", line 64, in load_module_spec
return registry.loader(path)
File "/anaconda3/envs/know-nlp-playground/lib/python3.6/site-packages/tensorflow_hub/registry.py", line 45, in call
self._name, args, kwargs))
RuntimeError: Missing implementation that supports: loader(*('/Users/juricaseva/Documents/projects/know-nlp/pretrained/research/biobert_v1.1_pubmed',), **{})

Hi,
Running the colab lets me encounter an issue in step 2: module 'tensorflow' has no attribute 'InteractiveSession'

Any ideas to solve that one to be able to move on?

Please help solve this

Hi,
I tried to adjust the code in order to work with TF2. However I'm new in TF and I got stuck and I could not find the alternative for tf.contrib.layers.layer_norm in TF2, some people said that tf.keras.layers.LayerNormalization will be equivalent but input parameters are different.

Thanks for your hardworking,
I'm looking forward to seeing the code is compatible on TF2

Hi, first of all, thanks a lot for the effort in the Medium articles on BERT, i found them super helpful for my research

I'm trying to run this notebook, but it does not seem to be working anymore on colab, due to changes on the standardized TF and Python versions (It defaults to 2.1 and 3.7 nowadays). Would you mind giving me the original requirements which were used so i can reproduce the experiment?