googlecloudplatform / tensorflow-recommendation-wals Goto Github PK

In the file wals.py line 147 suggest to change to:

input_tensor = tf.SparseTensor(indices=np.array([data.row, data.col]).T, # it has to be transposed !!!!
We have tested and evaluated the result with the suggestion. It looks right.

Hi I am just curious why python 2 version is required to run this project?

reviewing https://github.com/tensorflow/models/tree/08bb9eb5ad79e6bceffc71aeea6af809cc78694b/official/recommendation for how to get started

WARNING: The gcloud ml-engine commands have been renamed and will soon be removed. Please use gcloud ai-platform instead.
Traceback (most recent call last):
File "/home/contacto/miniconda2/envs/tfrec/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/home/contacto/miniconda2/envs/tfrec/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/contacto/tensorflow-recommendation-wals/wals_ml_engine/trainer/task.py", line 22, in
import model
File "trainer/model.py", line 25, in
import wals
File "trainer/wals.py", line 21, in
from tensorflow.contrib.factorization.python.ops import factorization_ops
ImportError: No module named contrib.factorization.python.ops
Tue Nov 26 01:21:40 UTC 2019

Hi,

I went through the tutorial by Google on recommendation systems with tensorflow given in the link below:
https://cloud.google.com/solutions/machine-learning/recommendation-system-tensorflow-apply-to-analytics-data

I went through this tutorial until I reached the part in the link above: Apply to Data from Google Analytics (Part 3)
The training part was successful. However, the tuning part was not. I used the command as it was on the tutorial to run the tuning, which is

./mltrain.sh tune gs://your_bucket data/ga_pageviews.csv --data-type web_views

obviously replacing your_bucket data with the name of my bucket, which is rec_sys_bucket_part2.

I get a few errors in the following order in the log:

2019-07-23T19:50:33.494398117Z master-replica-0 Trial Id : 1 usage: task.py [-h] --train-file TRAIN_FILE --job-dir JOB_DIR

2019-07-23T19:50:33.494867086Z master-replica-0 Trial Id : 1 [--latent_factors LATENT_FACTORS] [--num_iters NUM_ITERS]

2019-07-23T19:50:33.495102882Z master-replica-0 Trial Id : 1 [--regularization REGULARIZATION] [--unobs_weight UNOBS_WEIGHT]

2019-07-23T19:50:33.495305061Z master-replica-0 Trial Id : 1 [--wt_type WT_TYPE] [--feature_wt_factor FEATURE_WT_FACTOR]

2019-07-23T19:50:33.495655059Z master-replica-0 Trial Id : 1 [--feature_wt_exp FEATURE_WT_EXP] [--gcs-bucket GCS_BUCKET]

2019-07-23T19:50:33.495904922Z master-replica-0 Trial Id : 1 [--output-dir OUTPUT_DIR] [--verbose-logging] [--hypertune]

2019-07-23T19:50:33.496123075Z master-replica-0 Trial Id : 1 [--data-type DATA_TYPE] [--delimiter DELIMITER] [--headers]

2019-07-23T19:50:33.496273040Z master-replica-0 Trial Id : 1 [--use-optimized]

2019-07-23T19:50:33.496448040Z master-replica-0 Trial Id : 1 task.py: error: argument --train-file: expected one argument

2019-07-23T19:50:34.097594976Z master-replica-0 Trial Id : 1 Command '['python', '-m', u'trainer.task', u'--hypertune', u'--gcs-bucket', u'gs://rec_sys_bucket_part2/ga_pageviews.csv', u'--train-file', u'--data-type', u'--verbose-logging', u'web_views', u'--unobs_weight', u'0.10133471358860466', u'--latent_factors', u'43', u'--regularization', u'4.9950598320961008', u'--feature_wt_exp', u'6.3398703380823136', '--job-dir', 'gs://rec_sys_bucket_part2/ga_pageviews.csv/jobs/wals_ml_tune_20190723_194628/1']' returned non-zero exit status 2

I'm not sure which part of task.py is responsible for this. I added some print lines for check, but i don't see the print results when I run the job.

Any clues on how I can debug this?
Thanks,
A

Is this parameter is same ' feature weight exponent' mentioned in https://cloud.google.com/solutions/machine-learning/recommendation-system-tensorflow-overview

If not could you please help me explaining what is it?

There are only two parameteres for the mltrain script if you want to run it locally: 'local' and the path to the data. However, in your mltrain script you don't allow for less than three arguments passed:
if [[ $# < 3 ]]; then usage exit 1 fi
I have to add a bogus argument to get this to work.
Ex: mltrain.sh local data.csv does not work
Ex: mltrain.sh local garbage data.csv does work

File "/root/.local/lib/python2.7/site-packages/trainer/model.py", line 203, in _page_views_train_and_test
ix = pds_items.searchsorted(item)[0]
IndexError: invalid index to scalar variable.

When trying to train the model as documented here: https://cloud.google.com/solutions/machine-learning/recommendation-system-tensorflow-create-model

I get a number of errors. See StackOverflow post here: https://stackoverflow.com/questions/62752842/could-not-load-dynamic-library-libcuda-when-training-tensorflow-recommendation-m

Not sure how to proceed.

found this example is legacy but there are no current similar examples using ai-platform

While trying to run the command
./mltrain.sh train ${BUCKET} data/ratings.dat --delimiter ::
The following error appears.
and I tried deactivating conda and installing setuptools. but it still shows the same error:

Fri Feb 5 10:11:32 UTC 2021
ERROR: (gcloud.ai-platform.jobs.submit.training) Packaging of user Python code failed with message:
Traceback (most recent call last):
File "/home/ramiz_f/tensorflow-recommendation-wals/wals_ml_engine/setup.py", line 15, in
from setuptools import find_packages
ModuleNotFoundError: No module named 'setuptools'
Try manually building your Python code by running:
$ python setup.py sdist
and providing the output via the --packages flag (for example, --packages dist/package.tar.gz,dist/package2.whl)
Fri Feb 5 10:11:32 UTC 2021

WARNING: The gcloud ml-engine commands have been renamed and will soon be removed. Please use gcloud ai-platform instead.
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/contacto/tensorflow-recommendation-wals/wals_ml_engine/trainer/task.py", line 22, in
import model
File "trainer/model.py", line 20, in
import pandas as pd
ImportError: No module named pandas

Are you sure about your calculation of the RMSE ?
I may be wrong but you may have forgotten in your calculation all the null values in the 'actual' variable(a scipy coo_matrix) because scipy coo_matrix is a sparse matrix (that is to say zero values should not be in the matrix).

So we may add the prediction value to the error calculation because in that case, a prediction is made but the real value is 0.

Am I clear ? Did I miss something ? Am I wrong ?