Pretrain from scratch about text-to-text-transfer-transformer HOT 1 CLOSED

When one calls model.finetune as is done in the colab, the operative gin config file which resides in the pretrained model directly is automatically loaded.
https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/models/mtf_model.py#L308
Parsing this config file provides values for things like make_layer_stack. When using model.train, there is no pre-trained model directory so there is no operative config to look at. One option would be to decide on a model you want to use and load the operative config for that file, e.g. for the Base model

    import gin
    with gin.unlock_config():
      gin.parse_config_file("gs://t5-data/pretrained_models/base/operative_config.gin")

Note that you still may need to provide or modify other gin parameters.

from text-to-text-transfer-transformer.