need help to build a dataset from local numpy data about datasets HOT 4 CLOSED

Hi,

Can you confirm that this message is related to your dataset preparation not working? Seems like it could be just a warning that shouldn't prevent the script from executing (see #2761)

You can also try using:

tfds.core.utils.gcs_utils._is_gcs_disabled = True
os.environ['NO_GCE_CHECK'] = 'true'

To avoid the error message

Let us know if it works.

from datasets.

The error message you encountered suggests that TensorFlow is trying to authenticate with Google Cloud services to retrieve authentication tokens, but it's unable to do so because it's running in an environment where it can't access the necessary credentials.

Since you're working with a local dataset and don't need to interact with Google Cloud storage, you can disable the Google authentication by setting the environment variable GOOGLE_APPLICATION_CREDENTIALS to an empty string before running your script.

Here's how you can modify your script to disable Google authentication:

import os

# Disable Google Cloud authentication
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = ''

# Now import the required modules
import glob
import numpy as np
import tensorflow as tf
import tensorflow_datasets as tfds
import tensorflow_hub as hub

# Your existing code follows...

Adding this code at the beginning of your script will prevent TensorFlow from attempting to authenticate with Google Cloud services, and it should resolve the error you encountered.

Additionally, make sure that the file paths you're providing in your script ('/home/universal_sentence_encoder') are correct and accessible within your Docker container environment.

Once you've made these modifications, try running your script again:

python language_table_use_dataset_builder.py

This should allow your script to build the dataset without encountering authentication errors. If you encounter any further issues, please let me know, and I'll be happy to assist you further.

from datasets.

The error message you encountered suggests that TensorFlow is trying to authenticate with Google Cloud services to retrieve authentication tokens, but it's unable to do so because it's running in an environment where it can't access the necessary credentials.

Since you're working with a local dataset and don't need to interact with Google Cloud storage, you can disable the Google authentication by setting the environment variable GOOGLE_APPLICATION_CREDENTIALS to an empty string before running your script.

Here's how you can modify your script to disable Google authentication:

import os

# Disable Google Cloud authentication
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = ''

# Now import the required modules
import glob
import numpy as np
import tensorflow as tf
import tensorflow_datasets as tfds
import tensorflow_hub as hub

# Your existing code follows...

Adding this code at the beginning of your script will prevent TensorFlow from attempting to authenticate with Google Cloud services, and it should resolve the error you encountered.

Additionally, make sure that the file paths you're providing in your script ('/home/universal_sentence_encoder') are correct and accessible within your Docker container environment.

Once you've made these modifications, try running your script again:

python language_table_use_dataset_builder.py

This should allow your script to build the dataset without encountering authentication errors. If you encounter any further issues, please let me know, and I'll be happy to assist you further.

Thanks for your suggestion, but it seems that the issue is still remaining

from datasets.

Hi,

Can you confirm that this message is related to your dataset preparation not working? Seems like it could be just a warning that shouldn't prevent the script from executing (see #2761)

You can also try using:

tfds.core.utils.gcs_utils._is_gcs_disabled = True
os.environ['NO_GCE_CHECK'] = 'true'

To avoid the error message

Let us know if it works.

it works, thanks

from datasets.