Hi, I'm trying to load the newly released official Chinese models from Google. These m

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

reopening - I guess it is about the <a href="https://github.com/google-research/ALBERT

<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/us

Load Google's official Chinese ALBERT model about bert-for-tf2 HOT 4 CLOSED

kpe commented on August 14, 2024

Load Google's official Chinese ALBERT model

from bert-for-tf2.

Comments (4)

kpe commented on August 14, 2024

@hankcs - are you referring to the BERT weights (i.e. https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip or https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip)?

If you have problems loading the weights, could you please post the exact source code you are using here, or check with the README.md, i.e.:

model_name = "chinese_L-12_H-768_A-12"
model_dir = bert.fetch_google_bert_model(model_name, ".models")
model_ckpt = os.path.join(model_dir, "bert_model.ckpt")

bert_params = bert.params_from_pretrained_ckpt(model_dir)
l_bert = bert.BertModelLayer.from_params(bert_params, name="bert")

# use in Keras Model here, and call model.build()

bert.load_bert_weights(l_bert, model_ckpt)

from bert-for-tf2.

kpe commented on August 14, 2024

reopening - I guess it is about the https://github.com/google-research/ALBERT weights, i.e. https://storage.googleapis.com/albert_models/albert_base_zh.tar.gz

from bert-for-tf2.

kpe commented on August 14, 2024

@hankcs - loading of the official Google ALBERT weights should now be possible -

model_name = "albert_base_v2"
model_dir    = bert.fetch_google_albert_model(model_name, ".models")
model_ckpt   = os.path.join(albert_dir, "model.ckpt-best")

model_params = bert.albert_params(model_dir)
l_bert = bert.BertModelLayer.from_params(model_params, name="albert")

# use in Keras Model here, and call model.build()

bert.load_albert_weights(l_bert, model_ckpt)      # should be called after model.build()

from bert-for-tf2.