Comments (6)
Hi Lorenzo,
The scaler object is being stored as a class attribute (see here) so after you call scale() on the training data the scaler object is saved. Then when the validation and testing dataloaders are created they will be created with the training scaler already as an attribute. This means that calling scale() on the validation and test does use the same scaler object as the training.
The own scaler option is there in case you aren't using the prepare_model function and wish to set everything up yourself and you might want to pass a scaler object you have previously saved.
from ukaeagroupproject.
Ah, I see. I'm gonna test a small thing on that soon.
On another note, I'm getting the following error:
File "/lustre/home/pr5739/qualikiz/UKAEAGroupProject/src/scripts/train_LZ.py", line 49, in main
comet_logger, train_data, val_data, test_data = prepare_model(
File "/lustre/home/pr5739/qualikiz/UKAEAGroupProject/src/scripts/utils.py", line 91, in prepare_model
train_data.scale(categorical_keys=categorical_keys)
TypeError: scale() got an unexpected keyword argument 'categorical_keys'
Does that look right?
from ukaeagroupproject.
Ah, yes sorry, the function was changed recently to allow the option to pass in variables that aren't to be scaled so the corresponding scale functions in the datasets needed to be updated. I had the changes in my branch but they weren't in main, I have now put the changes in main, check if it works now.
from ukaeagroupproject.
ah gotcha, but shouldn't you pass ignore=categorical_keys in the scaler then?
from ukaeagroupproject.
Technically yes, I think as it's not doing anything in the QLKNN and AutoEncoder datasets atm but neither of these problems get passed any categorical values. But yes, that should be fixed for completeness.
from ukaeagroupproject.
ok no worries, not critical atm!
from ukaeagroupproject.
Related Issues (20)
- clarification HOT 1
- scaling... again HOT 6
- Found the damn error in CL HOT 5
- consistency
- To Do: save scaler! Or unscale all losses before saving
- Memory Issues in train_models fn HOT 3
- Refactoring needed? HOT 3
- Test loss confusion
- add classifier loss in outputs HOT 2
- hyperparameter tuning on random baseline
- Set random seed for reproducibility
- Fix naming: validation data named as test in output_dict, test named as holdout. HOT 1
- Particle Flow Variables for Second Regressor HOT 2
- Dataset problems HOT 4
- scaler
- code testing
- mutliple fluxes data prep
- distance penalty
- maybe it's the data on my end? HOT 16
- Output dictionary for two outputs HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ukaeagroupproject.