How to pull the number of iterations completed for each learner? about flaml HOT 4 CLOSED

Hi flippercy,

Thank you for your question.

If you want to get that info from an AutoML instance, a quick solution is to get it through the
_iter_per_learner attribute of the AutoML instance you created. The attribute is valid as long as the fit() function is already called.

E.g.

'''
automl_experiment = AutoML()
automl_experiment.fit(X_train, y_train)
print(automl_experiment._iter_per_learner)

'''

Please let me know whether this gives what you want.

Thanks,
Qingyun

from flaml.

Hi Qingyun:

Thank you for your response. I've noticed this attribute; however, it did not return what I expected. For example, after running your notebook example on automl with "estimator_list": ['lgbm' , 'rf' , 'xgboost', 'catboost' , 'extra_tree']:

[flaml.automl: 04-06 16:39:06] {986} INFO - iteration 62 current learner rf
[flaml.automl: 04-06 16:39:06] {1140} INFO - at 50.1s, best rf's error=0.3612, best lgbm's error=0.2907
[flaml.automl: 04-06 16:39:06] {1181} INFO - selected model: LGBMClassifier(colsample_bytree=0.8536465090001736,
learning_rate=0.03760148187622565, max_bin=255, max_leaves=62,
min_data_in_leaf=33, n_estimators=231, objective='binary',
reg_alpha=0.04663711639211432, reg_lambda=1.324798873241794,
subsample=0.9982731696185565)
[flaml.automl: 04-06 16:39:06] {939} INFO - fit succeeded

automl._iter_per_learner

{'lgbm': 1000000, 'rf': 0, 'xgboost': 0, 'catboost': 1000000, 'extra_tree': 0}

Did I do anything wrong?

Best,

from flaml.

Hi fippercy,

Sorry that the update of _iter_per_learner actually also checks the size of the data sample used and thus not exactly what you want.

You can get it through self._search_states[learner_name].total_iter

E.g. use the following code to pull the info for all the learners:

learner_iter_dict = {k:v.total_iter for (k,v) in automl._search_states.items()}

from flaml.

We can add a property iterations_per_estimator in AutoML.

from flaml.