Git Product home page Git Product logo

Comments (6)

adarob avatar adarob commented on June 25, 2024

Can you share what command you used to launch the fine-tuning? This looks like it may be because the SentencePiece model is not the correct one.

from text-to-text-transfer-transformer.

desperadoola avatar desperadoola commented on June 25, 2024

The sentencepiece model I use is trained by myself, and it's ok when I do pre-training and other fine-tune task. The above error seems to be raised when I set model_dir to an existed one. When I update the model_dir, this error doesn't happen again.

t5_mesh_transformer  \
  --tpu="${TPU_NAME}" \
  --gcp_project="${PROJECT}" \
  --tpu_zone="${ZONE}" \
  --model_dir="${MODEL_DIR}/${EXP_NAME}" \
  --t5_tfds_data_dir="${DATA_DIR}" \
  --gin_file="dataset.gin" \
  --gin_param="utils.tpu_mesh_shape.model_parallelism = 1" \
  --gin_param="utils.tpu_mesh_shape.tpu_topology = '2x2'" \
  --use_model_api \
  --mode="finetune" \
  --train_steps="50000" \
  --mixture_or_task="${TASK_NAME}" \
  --pretrained_model_dir="${MODEL_DIR}/baseline-spm-fix-22000-unsupervised_mix_zh" \
  --checkpoint_mode="specific" \
  --checkpoint_steps="999600" \
  --gin_file="learning_rate_schedules/constant_0_001.gin" \
  --gin_param="MtfModel.save_checkpoints_steps = 5000" \
  --gin_param="MtfModel.batch_size = 128" \
  --gin_param="MtfModel.model_parallelism=1"

from text-to-text-transfer-transformer.

desperadoola avatar desperadoola commented on June 25, 2024
File "/usr/local/lib/python3.6/site-packages/mesh_tensorflow/transformer/dataset.py", line 513, in pack_dataset
    shapes = dataset.output_shapes
AttributeError: 'ShuffleDataset' object has no attribute 'output_shapes'
  In call to configurable 'pack_dataset' (<function pack_dataset at 0x7f317ac32ae8>)
  In call to configurable 'pack_or_pad' (<function pack_or_pad at 0x7f317ad39d90>)
  In call to configurable 'mesh_train_dataset_fn' (<function mesh_train_dataset_fn at 0x7f317ac442f0>)

from text-to-text-transfer-transformer.

craffel avatar craffel commented on June 25, 2024

ShuffleDataset has no output_shapes member in TF 2. I am guessing you have installed TF 2. This library is designed for TF 1.15.
https://github.com/google-research/text-to-text-transfer-transformer/blob/master/setup.py#L58
tensorflow/datasets#1278

from text-to-text-transfer-transformer.

yanneyanne avatar yanneyanne commented on June 25, 2024

Hi,

I am having the same issue as in #23 (comment), however, I am running tensorflow 1.15.2.

from text-to-text-transfer-transformer.

craffel avatar craffel commented on June 25, 2024

I believe you need to use 1.15, not 1.15.2.

from text-to-text-transfer-transformer.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.