automl / autoweka Goto Github PK

View Code? Open in Web Editor NEW

326.0 33.0 106.0 99.2 MB

Auto-WEKA

Home Page: http://www.cs.ubc.ca/labs/beta/Projects/autoweka/

Shell 9.67% Batchfile 1.30% Java 77.77% Python 6.35% HTML 4.42% Ruby 0.48%

autoweka's Introduction

You can find the manual here. Javadoc is available here.

This project will not be updated anymore. We may make exceptions for pull requests.

autoweka's People

Contributors

$fracpete avatar$

Stargazers

Watchers

Forkers

farooqzuberi otoalves sweetcard phiferd zhensongqian lucasjuliano skobets phillette adhishp lukaszkn strategist922 rdk gabrieleguidi cibic89 grseb9s ashiqrh jaydenwhyte sniperxiaojun miracleday23kevin rashidzaman84 rleplae cys4 geoblog heoa mucapaz micseb slymnsn hoangcuong2011 tzirakis liujiandu zheng0115 manuelfritz kartha01 porld hujiaogen renyi533 thuzarwin bradbann manishpatwal jiranat roymachinelearning dailyactie cjr0106 szvajk propixel-prc chenxingqiang nunofernandes-plight cephasax hangjie720 jsonbao rshigemura vdubya greensuse chongshengzhang lxf44944 xiangnanhe mgmoghadam cbganap jaynoel pawanrana victorzhang2014 dimitrisnt sanchai-snap dawncc vyngo tsiakmaki avinashtgoje stjordanis 836304831 csjunxu fhoebe kiamesdavies jerrynpc cx365 phymucs hyowong helegraf benouinirachid shyamalschandra ghacupha zhongbineden big-data-ai gaybro8777 yuanyichuangzhi 5l1v3r1 noxymon ledduy610 andreysfc solversa anhhai986 bramamoorthy justinuliu charygao shadowridgedev jombaba jeffgan99 lkampoli webclinic017 mattyas-yahya jtmedley21

autoweka's Issues

Couldn't find any attribute eval methods

Running:

$ java -cp autoweka.jar weka.classifiers.meta.AutoWEKAClassifier -t dataset.arff -timeLimit 15 -no-cv

I get the following error:

java.lang.RuntimeException: Failed to prepare the experiment
	at autoweka.smac.SMACExperimentConstructor.prepareExperiment(SMACExperimentConstructor.java:46)
	at autoweka.ExperimentConstructor.run(ExperimentConstructor.java:238)
	at autoweka.ExperimentConstructor.buildSingle(ExperimentConstructor.java:158)
	at weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:333)
	at weka.classifiers.evaluation.Evaluation.evaluateModel(Evaluation.java:1527)
	at weka.classifiers.Evaluation.evaluateModel(Evaluation.java:650)
	at weka.classifiers.AbstractClassifier.runClassifier(AbstractClassifier.java:141)
	at weka.classifiers.meta.AutoWEKAClassifier.main(AutoWEKAClassifier.java:263)
Caused by: java.lang.RuntimeException: Couldn't find any attribute eval methods
	at autoweka.ExperimentConstructor.generateAlgorithmParameterConditionalGroupForDAG(ExperimentConstructor.java:390)
	at autoweka.smac.SMACExperimentConstructor.printParamFile(SMACExperimentConstructor.java:258)
	at autoweka.smac.SMACExperimentConstructor.prepareExperiment(SMACExperimentConstructor.java:31)
	... 7 more

Please advise.

XML parser doesn't work with all versions of Java

From Hakan via the WEKA mailing list:

Hi.

It seems that it is a problem running Auto-WEKA with Weka 3.8.0 (as well
and the lastest SVN version) together with Java 1.8. The error in the error
log is "java.lang.RuntimeException: Failed to load from XML file" (see
below for the full log from the terminal).

Note that it works with with Java 1.7, at least Open JDK version 1.7.0_101.

What I can see the problem occurs for any data set, e.g.
breast-cancer.arff, contact-lenses.arff (from the Weka distribution). When
writing this I tested with just 1 minute timeLimit but it don't seems to
matter. Also, I've tested both via the Auto-WEKA tab and selecting via the
Classifier tab (classifiers.meta.AutoWEKAClassifier) with the same result.

This problem was reported to the list by valerio jus in March 3 but I can't
find any solution.

My system:

Linux Ubuntu 14.04 LTS (64-bit, 32Gb RAM, CPU Intel i7-5820K)
Weka 1.8.0 (as mentioned above, it's the same error with the latest SVN
version, 3.9.1-SNAPSHOT)
Oracle Java version 1.8.0_91
Auto-Weka version 1.0.1. (I tested with the Git version of Auto-Weka with
the same result.)

I'm starting Weka like this (shell script):
"""

!/bin/sh

export CLASSPATH=
export JAVA_HOME=/usr/lib/jvm/java-8-oracle/
export JAVA_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/server
$JAVA_HOME/bin/java -server -Xmx16g -Xms2G -Xss1G -jar
/home/hakank/weka_cvs/svn/trunk/weka/dist/weka.jar weka.gui.GUIChooser
"""

Here's the more detailed output:
"""
2016-05-07 20:53:40.357 [Thread-10] INFO AutoWEKAClassifier - Estimated
mean quality of final incumbent config 5 (internal ID: 0x7566) on test set:
29.16666666666667, based on 1 run(s) on 1 test instance(s).
May 07, 2016 8:53:40 PM com.sun.xml.internal.bind.v2.util.XmlFactory
createParserFactory
SEVERE: null
org.xml.sax.SAXNotRecognizedException:
http://javax.xml.XMLConstants/feature/secure-processing
at org.apache.xerces.parsers.AbstractSAXParser.setFeature(Unknown
Source)
at org.apache.xerces.jaxp.SAXParserImpl.setFeatures(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl.(Unknown Source)
at
org.apache.xerces.jaxp.SAXParserFactoryImpl.newSAXParserImpl(Unknown Source)
at org.apache.xerces.jaxp.SAXParserFactoryImpl.setFeature(Unknown
Source)
at
com.sun.xml.internal.bind.v2.util.XmlFactory.createParserFactory(XmlFactory.java:121)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.getXMLReader(UnmarshallerImpl.java:139)
at
javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:157)
at
javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:204)
at autoweka.XmlSerializable.fromXML(XmlSerializable.java:93)
at autoweka.XmlSerializable.fromXML(XmlSerializable.java:75)
at autoweka.Experiment.fromXML(Experiment.java:172)
at
autoweka.TrajectoryMerger.mergeExperimentFolder(TrajectoryMerger.java:38)
at
weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:308)
at weka.gui.explorer.AutoWEKAPanel$11.run(AutoWEKAPanel.java:411)

May 07, 2016 8:53:40 PM com.sun.xml.internal.bind.v2.util.XmlFactory
createParserFactory
SEVERE: null
org.xml.sax.SAXNotRecognizedException:
http://javax.xml.XMLConstants/feature/secure-processing
at org.apache.xerces.parsers.AbstractSAXParser.setFeature(Unknown
Source)
at org.apache.xerces.jaxp.SAXParserImpl.setFeatures(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl.(Unknown Source)
at
org.apache.xerces.jaxp.SAXParserFactoryImpl.newSAXParserImpl(Unknown Source)
at org.apache.xerces.jaxp.SAXParserFactoryImpl.setFeature(Unknown
Source)
at
com.sun.xml.internal.bind.v2.util.XmlFactory.createParserFactory(XmlFactory.java:121)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.getXMLReader(UnmarshallerImpl.java:139)
at
javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:157)
at
javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:204)
at autoweka.XmlSerializable.fromXML(XmlSerializable.java:93)
at autoweka.XmlSerializable.fromXML(XmlSerializable.java:75)
at autoweka.Experiment.fromXML(Experiment.java:172)
at
autoweka.TrajectoryMerger.mergeExperimentFolder(TrajectoryMerger.java:38)
at
weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:308)
at weka.gui.explorer.AutoWEKAPanel$11.run(AutoWEKAPanel.java:411)

2016-05-07 20:53:40.973 [Thread-9] ERROR XmlSerializable - Error
unmarshalling XML response
java.lang.RuntimeException: Error unmarshalling XML response
autoweka.XmlSerializable.fromXML(XmlSerializable.java:97)
autoweka.XmlSerializable.fromXML(XmlSerializable.java:75)
autoweka.Experiment.fromXML(Experiment.java:172)

autoweka.TrajectoryMerger.mergeExperimentFolder(TrajectoryMerger.java:38)

weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:308)
weka.gui.explorer.AutoWEKAPanel$11.run(AutoWEKAPanel.java:411)

    at autoweka.XmlSerializable.fromXML(XmlSerializable.java:97)
    at autoweka.XmlSerializable.fromXML(XmlSerializable.java:75)
    at autoweka.Experiment.fromXML(Experiment.java:172)
    at

autoweka.TrajectoryMerger.mergeExperimentFolder(TrajectoryMerger.java:38)
at
weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:308)
at weka.gui.explorer.AutoWEKAPanel$11.run(AutoWEKAPanel.java:411)
Caused by: java.lang.IllegalStateException:
org.xml.sax.SAXNotRecognizedException:
http://javax.xml.XMLConstants/feature/secure-processing
at
com.sun.xml.internal.bind.v2.util.XmlFactory.createParserFactory(XmlFactory.java:128)
at
com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.getXMLReader(UnmarshallerImpl.java:139)
at
javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:157)
at
javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:204)
at autoweka.XmlSerializable.fromXML(XmlSerializable.java:93)
... 5 more
Caused by: org.xml.sax.SAXNotRecognizedException:
http://javax.xml.XMLConstants/feature/secure-processing
at org.apache.xerces.parsers.AbstractSAXParser.setFeature(Unknown
Source)
at org.apache.xerces.jaxp.SAXParserImpl.setFeatures(Unknown Source)
at org.apache.xerces.jaxp.SAXParserImpl.(Unknown Source)
at
org.apache.xerces.jaxp.SAXParserFactoryImpl.newSAXParserImpl(Unknown Source)
at org.apache.xerces.jaxp.SAXParserFactoryImpl.setFeature(Unknown
Source)
at
com.sun.xml.internal.bind.v2.util.XmlFactory.createParserFactory(XmlFactory.java:121)
... 9 more
java.lang.RuntimeException: java.lang.RuntimeException: Failed to load from
XML file

autoweka.TrajectoryMerger.mergeExperimentFolder(TrajectoryMerger.java:58)

weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:308)
weka.gui.explorer.AutoWEKAPanel$11.run(AutoWEKAPanel.java:411)

at

autoweka.TrajectoryMerger.mergeExperimentFolder(TrajectoryMerger.java:58)
at
weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:308)
at weka.gui.explorer.AutoWEKAPanel$11.run(AutoWEKAPanel.java:411)
Caused by: java.lang.RuntimeException: Failed to load from XML file
at autoweka.XmlSerializable.fromXML(XmlSerializable.java:80)
at autoweka.Experiment.fromXML(Experiment.java:172)
at
autoweka.TrajectoryMerger.mergeExperimentFolder(TrajectoryMerger.java:38)
... 2 more

"""

code style

Do you have any specific code style in your project, or each of contributors writes "as is"?
It would be nice to bring the code to the same code style, otherwise it's a little bit unconvinient for other developers to read and contiribute.

Problem running Auto-WEKA: NULL

Hi,
I´m trying to run autoWEKA however I always get the same error. The output log only contain: 12:58:47: Started Auto-WEKA for myproject
12:59:08: null
I´m using a macos El capitan, with java 8 update 111. I tried with attribute as class as well.

Java executable could not be found. Please refer to "Known Issues" in the Auto-WEKA manual.

java.lang.Exception: Java executable could not be found. Please refer to "Known
Issues" in the Auto-WEKA manual.
weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassif
ier.java:346)
weka.gui.explorer.AutoWEKAPanel$13.run(AutoWEKAPanel.java:451)

    at weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClas

sifier.java:346)
at weka.gui.explorer.AutoWEKAPanel$13.run(AutoWEKAPanel.java:451)

OS:win7,weka:3.8,autoweka:2.3

Autoweka 2.5 threads wont kill on Stop - Weka explorer 3.8

Please add it in your to-do list.

Check parameter space definition

There are some errors that suggest that the parameter space is not correctly specified wrt to some conditional dependencies, e.g.

[WARN ] [PROCESS-ERR] java.lang.RuntimeException: Failed to set classifier options [-I, 8, -H, 0.442016595562837, -R, 5, -F, 4, -Q, -P, 100, -L, 1e50, -S, 1, -W, weka.classifiers.bayes.BayesNet, --, -D, -Q, weka.classifiers.bayes.net.search.local.SimulatedAnnealing] for classifier weka.classifiers.meta.LogitBoost: Weight pruning with resampling not allowed.

SMAC generates numbers that can't be parsed by Java

For example

Failed to instantiate 'weka.classifiers.meta.AttributeSelectedClassifier' with options [-S, weka.attributeSelection.GreedyStepwise, -E, weka.a ttributeSelection.CfsSubsetEval, -W, weka.classifiers.functions.GaussianProcesses, --, -L, 0.007631852164298776, -N, 0, -K, weka.classifiers.functions.supportVector.RBFKernel -C 0.001563538449922339]: For input string: "0.001563538449922339"
 java.lang.RuntimeException: Failed to instantiate 'weka.classifiers.meta.AttributeSelectedClassifier' with options [-S, weka.attributeSelection.GreedyStepwise, -E, weka.attributeSelection.Cfs SubsetEval, -W, weka.classifiers.functions.GaussianProcesses, --, -L, 0.007631852164298776, -N, 0, -K, weka.classifiers.functions.supportVector.RBFKernel -C 0.001563538449922339]: For input string: "0.001563538449922339"
    at autoweka.ClassifierRunner._run(ClassifierRunner.java:250)
    at autoweka.ClassifierRunner.access$000(ClassifierRunner.java:29)
    at autoweka.ClassifierRunner$RunnerThread.doWork(ClassifierRunner.java:84)
    at autoweka.WorkerThread.run(WorkerThread.java:26)
 Caused by: java.lang.NumberFormatException: For input string: "0.001563538449922339"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:492)
    at java.lang.Integer.parseInt(Integer.java:527)
    at weka.classifiers.functions.supportVector.CachedKernel.setOptions(CachedKernel.java:128)
    at weka.classifiers.functions.supportVector.RBFKernel.setOptions(RBFKernel.java:191)
    at weka.core.Utils.forName(Utils.java:1096)
    at weka.classifiers.functions.supportVector.Kernel.forName(Kernel.java:364)
    at weka.classifiers.functions.GaussianProcesses.setOptions(GaussianProcesses.java:761)
    at weka.core.Utils.forName(Utils.java:1096)
    at weka.classifiers.AbstractClassifier.forName(AbstractClassifier.java:89)
    at weka.classifiers.SingleClassifierEnhancer.setOptions(SingleClassifierEnhancer.java:112)
    at weka.classifiers.meta.AttributeSelectedClassifier.setOptions(AttributeSelectedClassifier.java:316)
    at weka.core.Utils.forName(Utils.java:1096)
    at weka.classifiers.AbstractClassifier.forName(AbstractClassifier.java:89)
    at autoweka.ClassifierRunner._run(ClassifierRunner.java:241)
    ... 3 more

how to get full report of the auto-weka

Currently, i can only see best result. That's good.
But, i want to get full report of the of auto-weka. I mean, results of all classifiers and parameters used in the run.
Thanks in advance.
Awal
[email protected]

Support classifiers from WEKA packages

Will require

parameter space definition
code that checks which classifiers are available before starting the SMAC run

Manual relies on mdframed.sty

The PDF manual uses mdframed.sty, which doesn't seem to be easily installable anymore. Remove this dependency.

Object does not implement or subclass weka.classifiers.Classifier: builtin.list

Hi,

As a weka novice, I ran a test of auto-weka using the weka python wrapper, getting this error. The test was run on an AWS EC2 Linux instance.

My python script is as follows:

import weka.core.packages as packages
from weka.classifiers import Classifier
from weka.core.converters import Loader
from weka.core.dataset import Instances
from weka.classifiers import Classifier
import weka.core.serialization as serialization
import weka.core.jvm as jvm

jvm.start()

loader = Loader("weka.core.converters.ArffLoader")
data = loader.load_file('./weather.arff')
data.class_is_last()
classifier = Classifier("weka.classifiers.meta.AutoWEKAClassifier",
["-seed", "123", "-timeLimit", "15", "-memLimit", "1024",
"-nBestConfigs", "1", "-metric", "errorRate", "-parallelRuns", "1"])
classifier.build_classifier(data)
outfile = './weather.model'
serialization.write_all(outfile, [classifier, Instances.template_instances(data)])

jvm.stop()
quit()

End of script

Both WEKA_JAR_PATH and CLASSPATH were explicitly exported.
export WEKA_JAR_PATH=/usr/local/share/weka/weka-3-8-1/weka.jar
export CLASSPATH=/usr/local/share/weka/weka-3-8-1/weka.jar

The Auto-Weka package was installed as follows:
java weka.core.WekaPackageManager -install-package Auto-WEKA

The traceback output was:
Traceback (most recent call last):
File "autoweka.py", line 16, in
"-nBestConfigs", "1", "-metric", "errorRate", "-parallelRuns", "1"])
File "/usr/local/lib/python2.7/site-packages/weka/classifiers.py", line 54, in init
self.enforce_type(jobject, "weka.classifiers.Classifier")
File "/usr/local/lib/python2.7/site-packages/weka/core/classes.py", line 573, in enforce_type
raise TypeError("Object does not implement or subclass " + intf_or_class + ": " + get_classname(jobject))
TypeError: Object does not implement or subclass weka.classifiers.Classifier: builtin.list

I bring this to your attention, not really knowing if I omitted something or if this is a bug.

Can you help?

Colin Goldberg

Argument quoting is broken

Many things in WEKA take other things which in turn have arguments as arguments. This requires quoting the nested argument so they are attributed to the right thing. This is broken beyond level 2 in Auto-WEKA, e.g.

[WARN ] [PROCESS-ERR]  java.lang.RuntimeException: Failed to set classifier options [-R, AVG, -S, 1, -B, weka.classifiers.functions.SimpleLogistic -W 0.9930455546716422, -B, weka.classifiers.trees.DecisionStump, -B, weka.classifiers.functions.SMO -C 0.5254457157872984 -N 2 -K "weka.classifiers.functions.supportVector.PolyKernel -E 3.999607340248572, -END, -B, weka.classifiers.trees.RandomForest -I 4 -K 1 -depth 18] for classifier weka.classifiers.meta.Vote: Quote parse error.
[WARN ] [PROCESS-ERR]   at autoweka.ClassifierRunner._run(ClassifierRunner.java:267)
[WARN ] [PROCESS-ERR]   at autoweka.ClassifierRunner.access$000(ClassifierRunner.java:26)
[WARN ] [PROCESS-ERR]   at autoweka.ClassifierRunner$RunnerThread.doWork(ClassifierRunner.java:81)
[WARN ] [PROCESS-ERR]   at autoweka.WorkerThread.run(WorkerThread.java:21)
[WARN ] [PROCESS-ERR]  Caused by: java.lang.Exception: Quote parse error.
[WARN ] [PROCESS-ERR]   at weka.core.Utils.splitOptions(Utils.java:981)
[WARN ] [PROCESS-ERR]   at weka.classifiers.MultipleClassifiersCombiner.setOptions(MultipleClassifiersCombiner.java:92)
[WARN ] [PROCESS-ERR]   at weka.classifiers.RandomizableMultipleClassifiersCombiner.setOptions(RandomizableMultipleClassifiersCombiner.java:92)
[WARN ] [PROCESS-ERR]   at weka.classifiers.meta.Vote.setOptions(Vote.java:318)
[WARN ] [PROCESS-ERR]   at autoweka.ClassifierRunner._run(ClassifierRunner.java:262)
[WARN ] [PROCESS-ERR]   ... 3 more

Model evaluation not using cross validation

When Auto-WEKA reported the performance of its best classifier on my data, it looked great compared to other methods I've tried in the past: 87% accuracy, 0.957 ROC Area. To validate this result, I took the same feature selection and classification methods and tried them outside of Auto-WEKA (CfsSubsetEval with GreedyStepwise search method, meta.Bagging with lazy.IBk), making sure to set all parameters identical to those reported by Auto-WEKA.

When doing 10-fold cross-validation on this Bagging of IBk, the performance was nowhere near that reported by Auto-WEKA (71%, 0.719 ROC Area). However, when I performed the testing on the training data ("Use training set"), WEKA reported the same improved performance (87%, 0.957 ROC area). I also checked the other reported values (confusion matrix, TP/FP rates, precision, recall, F-measure, PRC-area, etc) and they were identical.

This suggests that Auto-WEKA isn't actually performing the 10-fold cross-validation that the documentation suggests it is when evaluating classifiers, but rather testing on the training data instead.

Trying with the Iris dataset supplied with Weka confirms the issue:
Auto-WEKA reports 98% accuracy
The same settings (default simple logistic w/ no feature selection), 10-fold CV reports 94%
The same settings testing on training data reports 98%

I'm not totally certain I haven't misunderstood something, but this seems like a glaring problem if I haven't.

Allow run to be killed from WEKA GUI

At the moment, the stop button does nothing.

CUDA support or other ways to speed up the algorithms?

Is there any plans to add CUDA support or other ways to speed up the algorithms?

Allow to restrict parameter space dynamically

E.g. allow only a single meta classifier at the top and tune its base learners.

java.lang.NoClassDefFoundError: javax/xml/bind/JAXBContext

Hi there,
I am using Windows 10,Weka 3.8.2, Autoweka-2.6.zip. Java version is:

java version "9"
Java(TM) SE Runtime Environment (build 9+181)
Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)

Here is the log of the error I got:

09:53:15: Weka Explorer
09:53:15: (c) 1999-2017 The University of Waikato, Hamilton, New Zealand
09:53:15: web: http://www.cs.waikato.ac.nz/~ml/weka/
09:53:15: Started on Saturday, 21 April 2018
09:53:26: Base relation is now xalan2-weka.filters.unsupervised.attribute.Remove-R1-2 (3320 instances)
09:53:26: java.lang.NullPointerException
09:53:30: Started Auto-WEKA for xalan2-weka.filters.unsupervised.attribute.Remove-R1-2

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by weka.core.WekaPackageClassLoaderManager (file:/C:/Program%20Files/Weka-3-8/weka.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of weka.core.WekaPackageClassLoaderManager
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
java.lang.NullPointerException
weka.core.Instances.(Instances.java:179)
weka.gui.explorer.ClassifierPanel.updateCapabilitiesFilter(ClassifierPanel.java:3262)
weka.gui.explorer.ClassifierPanel.capabilitiesFilterChanged(ClassifierPanel.java:3302)
weka.gui.explorer.Explorer.notifyCapabilitiesFilterListener(Explorer.java:343)
weka.gui.explorer.PreprocessPanel$21.run(PreprocessPanel.java:775)
java.desktop/java.awt.event.InvocationEvent.dispatch(Unknown Source)
java.desktop/java.awt.EventQueue.dispatchEventImpl(Unknown Source)
java.desktop/java.awt.EventQueue.access$500(Unknown Source)
java.desktop/java.awt.EventQueue$3.run(Unknown Source)
java.desktop/java.awt.EventQueue$3.run(Unknown Source)
java.base/java.security.AccessController.doPrivileged(Native Method)
java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
java.desktop/java.awt.EventQueue.dispatchEvent(Unknown Source)
java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
java.desktop/java.awt.EventDispatchThread.run(Unknown Source)

    at weka.core.Instances.<init>(Instances.java:179)
    at weka.gui.explorer.ClassifierPanel.updateCapabilitiesFilter(ClassifierPanel.java:3262)
    at weka.gui.explorer.ClassifierPanel.capabilitiesFilterChanged(ClassifierPanel.java:3302)
    at weka.gui.explorer.Explorer.notifyCapabilitiesFilterListener(Explorer.java:343)
    at weka.gui.explorer.PreprocessPanel$21.run(PreprocessPanel.java:775)
    at java.desktop/java.awt.event.InvocationEvent.dispatch(Unknown Source)
    at java.desktop/java.awt.EventQueue.dispatchEventImpl(Unknown Source)
    at java.desktop/java.awt.EventQueue.access$500(Unknown Source)
    at java.desktop/java.awt.EventQueue$3.run(Unknown Source)
    at java.desktop/java.awt.EventQueue$3.run(Unknown Source)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
    at java.desktop/java.awt.EventQueue.dispatchEvent(Unknown Source)
    at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
    at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
    at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
    at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
    at java.desktop/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
    at java.desktop/java.awt.EventDispatchThread.run(Unknown Source)

Exception in thread "Thread-5" java.lang.NoClassDefFoundError: javax/xml/bind/JAXBContext
autoweka.XmlSerializable.toXML(XmlSerializable.java:59)
autoweka.XmlSerializable.toXML(XmlSerializable.java:43)
autoweka.ExperimentConstructor.run(ExperimentConstructor.java:245)
autoweka.ExperimentConstructor.buildSingle(ExperimentConstructor.java:158)
weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:336)
weka.gui.explorer.AutoWEKAPanel$13.run(AutoWEKAPanel.java:451)

    at autoweka.XmlSerializable.toXML(XmlSerializable.java:59)
    at autoweka.XmlSerializable.toXML(XmlSerializable.java:43)
    at autoweka.ExperimentConstructor.run(ExperimentConstructor.java:245)
    at autoweka.ExperimentConstructor.buildSingle(ExperimentConstructor.java:158)
    at weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:336)
    at weka.gui.explorer.AutoWEKAPanel$13.run(AutoWEKAPanel.java:451)

Caused by: java.lang.ClassNotFoundException: [weka.core.WekaPackageLibIsolatingClassLoader (Auto-WEKA)] Unable to find class: javax.xml.bind.JAXBContext
at weka.core.WekaPackageLibIsolatingClassLoader.findClass(WekaPackageLibIsolatingClassLoader.java:509)
at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
... 6 more

Problem evaluation classifier: Java executable could not be found

I'm using latest version of Weka and I downloaded Auto Weka 2.3 with Weka Package Manager.

Now when I select the AutoWeka Classifier and click start, I get Problem evaluation classifier: Java executable could not be found.

I'm on Windows 10.

Of course I have JRE and JDK installed and no other issues running other stuff requiring them (i.e. Eclipse).

What should I do?

Thanks

It should be quick to change k in k-fold cross validation

I feel there should be a config file to customise auto-weka without having to re-compile from source. Any workarounds?

Problem running Auto-WEKA: Quote parse error.

Hi There,

I've just installed Auto-WEKA using WEKA 3.8 and its Package Manager.
Then I've created my model using StringToWordVector and simply started Auto-WEKA.

But at the end of the Process, it fails with the error "Quote parse error."

Error with duplicate bin ranges

I get this error: A duplicate bin range was detected. Try increasing the bin range precision.

Classification problem with 62 numerical inputs.

This is the dump from the console:
[Thread-146] INFO weka.classifiers.meta.AutoWEKAClassifier - classifier: weka.classifiers.meta.RandomCommittee, arguments: [-I, 56, -S, 1, -W, weka.classifiers.trees.RandomForest, --, -I, 8, -K, 0, -depth, 0], attribute search: null, attribute search arguments: [], attribute evaluation: null, attribute evaluation arguments: []
java.lang.IllegalArgumentException: A duplicate bin range was detected. Try increasing the bin range precision.

I have another instance with Kappa as the metric and it works fine?
UPDATE: same problem. I suspect it's the outliers in the data and this suggests a fix is to increase digit precision when starting auto-weka

George

2Whomit may concern

Be aware of that In order to use Auto-WEKA through the WEKA GUI on OSX, WEKA must be run from the command line instead of the Mac application. For example,
cd /Applications/weka-3-8-1-oracle-jvm.app/Contents/Java/
java -jar weka.jar
Note that Java needs to be installed on your system for this to work. This is because Auto-WEKA needs a java executable to spawn worker processes, and the JRE bundled with the Mac application does not contain one.

Results doesn't match the corresponding Weka classifier

When I run Auto-WEKA and get a result, I try to build the same classifier in regular Weka to see if the results match, but it gives me completely different results.

I tried using the same seed, but results are still very different, going from 10% or even more difference in some metrics.

Also, I tried using the code provided to simulate the classifier in Java and after building the classifier as given, I validate the model like follows:

Evaluation evaluation = new Evaluation(instances);
evaluation.crossValidateModel(classifier, instances, 10, new Random(123));

It also give me very different results than the shown in Auto-WEKA tab. Is this correct or am I doing something wrong?

Thanks.

what Auto-Weka reports exactly?

I am looking the AutoWeka package that looks awesome. I tested it for a few datasets of a regression problem and then became uncertain how to interpret the results reported. Specifically, I am running the package in WEKA classifier GUI (the general classifier tab, not the special AutoWeka). I used 10 parallel runs in one test and let it run for 24 hours and then at the end AutoWeka gave me a report with expected mean absolute error and root mean square error etc. But to what this report refers to? I have learned AutoWeka splits the data to training and test sets and then further uses 10-fold CV in the training set to optimise the hyperparameters. I assume then the test set is used to check the performance further. So, my question is if the results reported in the WEKA GUI results buffer are 1) results based on the test set evaluation within each run?, 2) results from the individual CV folds of the training part or 3) full trained model to entire data? It seems difficult to find this information from the documentation. I have learned that AutoWeka saves more detailed information to the /tmp folder but I have not found any documentation for the files there either - one needs to guess what they could mean. E.g. there is a file called predictions.csv which I guess are the CV fold predictions on the training set - but then are there somewhere the test set predictions available as well?

I am hesitating in using this magnificant package as I a m not sure what the results on the prediction capability mean. I would appreciate some help or pointers to further reading if anyone can provide.

regards, Kari

Base learner must implement Randomizable

Running:

$ java -cp autoweka.jar weka.classifiers.meta.AutoWEKAClassifier -t iris.arff

I get the following warning:

2017-05-02 11:04:39.484 [Thread-1] WARN AutoWEKAClassifier - 2017-05-02 11:04:39.484 [main] WARN Experiment - [WARN ] [PROCESS-ERR]  2017-05-02 11:04:39.467 [Thread-0] WARN ClassifierRunner - Training classifier (weka.classifiers.meta.RandomCommittee [-I, 5, -S, 1, -W, weka.classifiers.rules.OneR, --, -B, 3]) failed: Base learner must implement Randomizable!
2017-05-02 11:04:39.485 [Thread-1] WARN AutoWEKAClassifier - 2017-05-02 11:04:39.484 [main] WARN Experiment - [WARN ] [PROCESS-ERR]  java.lang.IllegalArgumentException: Base learner must implement Randomizable!
2017-05-02 11:04:39.485 [Thread-1] WARN AutoWEKAClassifier - 2017-05-02 11:04:39.484 [main] WARN Experiment - [WARN ] [PROCESS-ERR]  	at weka.classifiers.meta.RandomCommittee.buildClassifier(RandomCommittee.java:150)
2017-05-02 11:04:39.485 [Thread-1] WARN AutoWEKAClassifier - 2017-05-02 11:04:39.485 [main] WARN Experiment - [WARN ] [PROCESS-ERR]  	at autoweka.ClassifierRunner$BuilderThread.doWork(ClassifierRunner.java:459)
2017-05-02 11:04:39.485 [Thread-1] WARN AutoWEKAClassifier - 2017-05-02 11:04:39.485 [main] WARN Experiment - [WARN ] [PROCESS-ERR]  	at autoweka.WorkerThread.run(WorkerThread.java:26)

2017-05-02 11:05:59.883 [Thread-1] WARN AutoWEKAClassifier - 2017-05-02 11:05:59.883 [main] WARN Experiment - [WARN ] [PROCESS-ERR]  2017-05-02 11:05:59.836 [Thread-0] WARN ClassifierRunner - Training classifier (weka.classifiers.meta.RandomCommittee [-I, 34, -S, 1, -W, weka.classifiers.lazy.IBk, --, -K, 5, -I]) failed: Base learner must implement Randomizable!
2017-05-02 11:05:59.883 [Thread-1] WARN AutoWEKAClassifier - 2017-05-02 11:05:59.883 [main] WARN Experiment - [WARN ] [PROCESS-ERR]  java.lang.IllegalArgumentException: Base learner must implement Randomizable!
2017-05-02 11:05:59.883 [Thread-1] WARN AutoWEKAClassifier - 2017-05-02 11:05:59.883 [main] WARN Experiment - [WARN ] [PROCESS-ERR]  	at weka.classifiers.meta.RandomCommittee.buildClassifier(RandomCommittee.java:150)
2017-05-02 11:05:59.883 [Thread-1] WARN AutoWEKAClassifier - 2017-05-02 11:05:59.883 [main] WARN Experiment - [WARN ] [PROCESS-ERR]  	at autoweka.ClassifierRunner$BuilderThread.doWork(ClassifierRunner.java:459)
2017-05-02 11:05:59.883 [Thread-1] WARN AutoWEKAClassifier - 2017-05-02 11:05:59.883 [main] WARN Experiment - [WARN ] [PROCESS-ERR]  	at autoweka.WorkerThread.run(WorkerThread.java:26)

...

I'm using macOS Sierra, version 10.12.4 (16E195)

Provide parameter to configure logging

Allow to log more detail, e.g. tried configurations.

irace and TPE experiment constructor properties have hardcoded broken paths

autoweka.irace.IRaceExperimentConstructor.properties and autoweka.tpe.TPEExperimentConstructor.properties contain paths that reference external things and are broken. Should be replaced by something that either references something in the repo/distribution or an external resource in a portable way.

Update to latest version of WEKA

Running AutoWEKA from WEKA only works if run from the AutoWEKA folder

Otherwise it can't find the SMAC executable.

Setup Auto-Weka in Eclipse Errors

Hi, anyone did manage somehow to setup/run AutoWeka in Eclipse? Tested it under IntelliJ and it works.

In Eclipse:
2017-09-07 13:41:12.928 [Thread-1] WARN Util - Could not auto-detect the location of your Auto-WEKA install - have you moved the classes away from the 'params' diectory?
2017-09-07 13:41:12.931 [Thread-1] WARN ExperimentConstructor - No property file ./autoweka.smac.SMACExperimentConstructor.properties.properties found

FilteredClassifier with autoweka disables CLI only options

As title suggests. I get this error: "Weka exception: Illegal options: -no-cv -split-percentage 66 -no-predictions"

Implement anytime behaviour

Provide feedback on current progress (i.e. error) after each iteration.
Allow to abort at any point, saving the results from the previous iteration.

Problem running Auto Weka 2.5

Hello team,

I am facing some problem in installing Auto Weka 2.5 on my machine. Please can you guys help me out and guide me with the same.

Steps followed:

Simply, downloaded the package from this website : http://www.cs.ubc.ca/labs/beta/Projects/autoweka/
After unzipping the package, double clicked on the autoweka.jar file.

Error : The Java JAR file "autoweka.jar" could not be launched.
Check the Console for possible error messages.

I am trying to install in my MAC machine, with java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11).

Any help is highly appreciated. Thanks in advance.

IllegalAccessError on "Save model" from Auto-WEKA panel

After running Auto-WEKA from the Auto-WEKA panel, right clicking the run in the result list and clicking "Save model" yields the following error:

Exception in thread "AWT-EventQueue-0" java.lang.IllegalAccessError: tried to access method weka.gui.explorer.ClassifierPanel.saveClassifier(Ljava/lang/String;Lweka/classifiers/Classifier;Lweka/core/Instances;)V from class weka.gui.explorer.AutoWEKAPanel$12

As expected, this error doesn't occur if the AutoWEKAClassifier is run from the Classify panel.

Implement AdditionalMeasureProducer interface

Return final estimated performance as additional measure.

Check experiment classpath for running classifiers

I see

[WARN ] [PROCESS-ERR]  java.lang.RuntimeException: Failed to set classifier options [-S, weka.attributeSelection.GreedyStepwise, -E, weka.attributeSelection.HoldOutSubsetEvaluator, -W, weka.classifiers.trees.RandomTree, --, -M, 16, -K, 0, -depth, 19, -N, 3] for classifier weka.classifiers.meta.AttributeSelectedClassifier: Can't find class called: weka.attributeSelection.HoldOutSubsetEvaluator

in the logs sometimes. The class is there in autoweka.jar (which is in the classpath), so not sure what's causing this. Explicitly adding weka.jar to the classpath doesn't solve this.

Best parameter configurations shouldn't use Auto-WEKA internal format

Could not find SMAC executable

I'm getting this error whenever I push the "Start" button.
My guess is that this is a problem with Windows systems when a space is in the smac.bat path. In my case the full path is "c:\Users\Davide Rossi\wekafiles\packages\Auto-WEKA\smac-v2.10.03-master-778\smac.bat" that is reported as "c:\Users\Davide%20Rossi\wekafiles\packages\Auto-WEKA\smac-v2.10.03-master-778\smac.bat" in the error message.

Problem running Auto-Weka : Failed to prepare Experiment

Hi,

I installed the Auto-Weka recently. The Weka version I have is 3.8.2. However when I try to run through WEKA GUI I get the following error message :
"Problem running Auto-Weka : Failed to prepare Experiment ".

Please help.

Thanks,
Balachandar

AutoWeka and time series forecast

Just a question.
Is AutoWeka suitable for time series forecast ? Because what I have read is it uses internal cross validation and it is not always right for time series.

*options functions should call super class options

Empty arguments list

Hello,

I have done a small python wrapper around the Auto-WEKA project using JPype, but when I try to access to an arguments array I obtain an empty array.

For this configuration:
2017-09-28 17:11:18.685 [main] INFO AutoWEKAClassifier - classifier: weka.classifiers.trees.RandomForest, arguments: [-I, 10, -K, 0, -depth, 0], attribute search: null, attribute search arguments: [], attribute evaluation: null, attribute evaluation arguments: []

I obtain something like this when I access to the arguments list (classifierArgs field):
('', '', '', '', '', '')

I have no issue when I try to access to the classifier class (classifierClass field) so I am wondering if there is any issue or if JPype plays tricks on me.

Project containing Run/Compile Errors ( very anoying )

Hi,

I downloaded this project and created a new Java Project from Existing Ant Buildfile (), choosed the build.xml in the autoweka-master folder and thought that's it I got the project.

Modyfing AutoWekaPanel:
Reader r = new java.io.BufferedReader(new java.io.FileReader("rec/iris.arff"));
then runing AutoWekaPanel and pressing start.

2017-09-07 13:41:12.928 [Thread-1] WARN Util - Could not auto-detect the location of your Auto-WEKA install - have you moved the classes away from the 'params' diectory?
2017-09-07 13:41:12.931 [Thread-1] WARN ExperimentConstructor - No property file ./autoweka.smac.SMACExperimentConstructor.properties.properties found

I am 100% sure, that I didn't move anything anywhere.
I am 100% sure, that there doesn't exist any SMACExperimentConstructor.properties.properties file in the whole project.

Please could u make your project also accessible for normal humanbeings, it's really frustrating when you need more time to get the project running/compiling than for using.

Greets, Nick

Autoweka from matlab

I'm trying to integrate autoweka with my code. I'm working from matlab environment and as it seems I get the following error when instantiating AutoWEKAClassifier.

Java exception occurred:
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
	at java.util.ArrayList.rangeCheck(Unknown Source)
	at java.util.ArrayList.get(Unknown Source)
	at autoweka.ApplicabilityTester.getApplicableClassifiers(ApplicabilityTester.java:62)
	at autoweka.ExperimentConstructor.loadClassifiers(ExperimentConstructor.java:336)
	at autoweka.ExperimentConstructor.run(ExperimentConstructor.java:210)
	at autoweka.ExperimentConstructor.buildSingle(ExperimentConstructor.java:158)
	at weka.classifiers.meta.AutoWEKAClassifier.buildClassifier(AutoWEKAClassifier.java:334)

This seem to an issue with instantiating the different weka classifiers even though they are found in the config:

weka.classifiers.bayes.BayesNet.params
weka.classifiers.bayes.NaiveBayes.params
weka.classifiers.bayes.NaiveBayesMultinomial.params
...

Any assistant will be appreciated ....

maven / gradle support

If you do not depend on some specian Ant features, it would be nice to have maven or gradle support in your project.

An option just to add a dependency from mvnrepository to a custom maven or gradle project will we fantastic!

Unable to run autoweka with weka experimenter

Hi. I am unable to run autowekaclassifier within weka experimenter. The process always interrupts within a few minutes. The only messages are "autoweka.Configuration.Collection" followed by "Interrupted" which is followed by "There was 1 error". Running with debug set to True does not provide insight. Any insight you might be able to provide would be greatly appreciated. Thank you!