m3dev / redshells Goto Github PK

View Code? Open in Web Editor NEW

43.0 43.0 14.0 628 KB

Machine learning tasks which are used with data pipeline library "luigi" and its wrapper "gokart".

License: MIT License

Python 100.00%

redshells's People

Contributors

Stargazers

Watchers

Forkers

kgkgzrtk ryusuketa tkazusa embeddedsamurai r2en teppei-kanayama yamasakih mski-iksm hi-king vaaaaanquish nogawanogawa hirosassa tsujimotohironori yokomotod

redshells's Issues

tf.Session is deprecated.

warning now comes out.

W1022 09:22:52.705346 4633392576 deprecation_wrapper.py:119] From python3.6/site-packages/redshells-0.1.7-py3.6.egg/redshells/model/early_stopping.py:19: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

follow tensorflow 2.0

to follow Python 3.8+

classification_report on fit data is almost worthless

classification_report on fit data is almost worthless.
Maybe, this log will missleading.
line: https://github.com/m3dev/redshells/blob/master/redshells/train/utils/utils.py#L16

For example.

[2020/03/03 23:22:07][redshells.train.utils.utils][INFO](utils.py:21)
               precision    recall  f1-score   support
           0       1.00      1.00      1.00      3060
           1       1.00      1.00      1.00       252
    accuracy                           1.00      3312
   macro avg       1.00      1.00      1.00      3312
weighted avg       1.00      1.00      1.00      3312

Pipeline between Train and Optimize task are not connected

Hi,

Thanks for presenting examples of important tasks in machine learning, OptimizeTask and TrainTask.
I want to know how I can all task by running only TrainTask in below code.

https://github.com/yamasakih/redshells/blob/feature/add-thunderbolt-example/examples/thunderbolt_example.py

This code is based on the following site.
https://github.com/vaaaaanquish/gokart_redshells_thunderbolt_example/blob/master/Example.ipynb

When I ran TrainTask, gokart raised below error.

[2019-11-12 22:08:40,934][WARNING](worker.py:658) Will not run sample.TrainXGBoostModel(n_estimators=50, test_size=0.2) or any dependencies due to error in complete() method:
Traceback (most recent call last):
  File "/Users/yamasakih/anaconda3/envs/gokart/lib/python3.7/site-packages/luigi/worker.py", line 409, in check_complete
    is_complete = task.complete()
  File "/Users/yamasakih/anaconda3/envs/gokart/lib/python3.7/site-packages/gokart/task.py", line 69, in complete
    is_completed = all([t.exists() for t in luigi.task.flatten(self.output())])
  File "thunderbolt_example.py", line 104, in output
    return self.make_target('TrainXGBoostModel.pkl')
  File "/Users/yamasakih/anaconda3/envs/gokart/lib/python3.7/site-packages/gokart/task.py", line 102, in make_target
    unique_id = self.make_unique_id() if use_unique_id else None
  File "/Users/yamasakih/anaconda3/envs/gokart/lib/python3.7/site-packages/gokart/task.py", line 182, in make_unique_id
    self.task_unique_id = self.task_unique_id or self._make_hash_id()
  File "/Users/yamasakih/anaconda3/envs/gokart/lib/python3.7/site-packages/gokart/task.py", line 191, in _make_hash_id
    dependencies = [_to_str_params(task) for task in luigi.task.flatten(self.requires())]
  File "thunderbolt_example.py", line 93, in requires
    param = self.clone(OptimizeXGBoostModel).load()
  File "/Users/yamasakih/anaconda3/envs/gokart/lib/python3.7/site-packages/gokart/task.py", line 146, in load
    return _load(self._get_input_targets(target))
  File "/Users/yamasakih/anaconda3/envs/gokart/lib/python3.7/site-packages/gokart/task.py", line 144, in _load
    return targets.load()
  File "/Users/yamasakih/anaconda3/envs/gokart/lib/python3.7/site-packages/gokart/target.py", line 58, in load
    with self._target.open('r') as f:
  File "/Users/yamasakih/anaconda3/envs/gokart/lib/python3.7/site-packages/luigi/local_target.py", line 165, in open
    fileobj = FileWrapper(io.BufferedReader(io.FileIO(self.path, mode)))
FileNotFoundError: [Errno 2] No such file or directory: './resources/model/OptimizeXGBoostModel_c3153844698935b3eb1eb42d91e218fb.pkl'

So, I have to run OptimizeTask, and next run TrainTask. It takes a little time :(

I'd like you to tell me if better coding works. Thank you in advance!

Use default user index

When flag with_user_embedding is False, using default user index would be better rather than using index zero.

Return of TravisCI

Because change GitHub rate structure, we need to go back to TravisCI from GitHub Actions.

＿人人人人人人人人人人人＿
＞　Deadline is 11/13.　＜
￣Y^Y^Y^Y^Y^Y^Y^Y^YY^￣

Indexing scipy.csr_matrix with numpy.ndarray not supported in scipy==1.3.1

matrix[user_indices, item_indices] in graph_convolutional_matrix_completion.py L.404 returns error in scipy==1.3.1, since indexing with numpy.ndarray is not possible.

Possible Solution

matrix[len(user_indices), len(item_indices)]
or
specific scipy version as scipy==1.2.1 at redshells/setup.py

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.