uvasrg / evademl Goto Github PK

Hi,
I have an issue in clverhans_models. I install cleverhans but I find there is no conv_2d in cleverhans.utils.
Do you have some special reference?

Thank you in advance.

Hi! I'm trying to replicate the results from your EvadeML GP attacks paper on PDFs. I ran into a couple of issues running the code. I trained the two PDF classifier models, and when trying to run Step 2 (with Step 1 detection server running) - ./utils/generate_ext_genome.py [classifier_name] [benign_sample_folder] [file_number] for the two classifiers, I get two different errors:

1. PDFRate
   $ ./utils/generate_ext_genome.py pdfrate ~/research/datasets/benign 1
Traceback (most recent call last):
File "./utils/generate_ext_genome.py", line 83, in <module>
selected_files = pdf_geno.select_files()
File "./utils/generate_ext_genome.py", line 38, in select_files
classifier_results = self.classifier(file_paths)
File "./utils/generate_ext_genome.py", line 24, in classifier
return self.classifier_func(*args)
File "./utils/generate_ext_genome.py", line 18, in <lambda>
self.classifier_func = lambda *args:query_classifier(classifier_name, *args)
File "/home/susobhan/research/EvadeML/utils/../lib/detector.py", line 22, in query_classifier
results = server.query_classifier(classifier_name, file_paths, seed_sha1)
File "/usr/lib/python2.7/xmlrpclib.py", line 1243, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1602, in __request
verbose=self.__verbose
File "/usr/lib/python2.7/xmlrpclib.py", line 1283, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib/python2.7/xmlrpclib.py", line 1316, in single_request
return self.parse_response(response)
File "/usr/lib/python2.7/xmlrpclib.py", line 1493, in parse_response
return u.close()
File "/usr/lib/python2.7/xmlrpclib.py", line 800, in close
raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: "<type 'exceptions.ValueError'>:The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()">

2. Hidost -
   $ ./utils/generate_ext_genome.py hidost ~/research/datasets/benign 1
Traceback (most recent call last):
File "./utils/generate_ext_genome.py", line 83, in <module>
selected_files = pdf_geno.select_files()
File "./utils/generate_ext_genome.py", line 38, in select_files
classifier_results = self.classifier(file_paths)
File "./utils/generate_ext_genome.py", line 24, in classifier
return self.classifier_func(*args)
File "./utils/generate_ext_genome.py", line 18, in <lambda>
self.classifier_func = lambda *args:query_classifier(classifier_name, *args)
File "/home/susobhan/research/EvadeML/utils/../lib/detector.py", line 22, in query_classifier
results = server.query_classifier(classifier_name, file_paths, seed_sha1)
File "/usr/lib/python2.7/xmlrpclib.py", line 1243, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1602, in __request
verbose=self.__verbose
File "/usr/lib/python2.7/xmlrpclib.py", line 1283, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib/python2.7/xmlrpclib.py", line 1316, in single_request
return self.parse_response(response)
File "/usr/lib/python2.7/xmlrpclib.py", line 1493, in parse_response
return u.close()
File "/usr/lib/python2.7/xmlrpclib.py", line 800, in close
raise Fault(**self._stack[0])
xmlrpclib.Fault: <Fault 1: "<type 'exceptions.AttributeError'>:'dict' object has no attribute 'decision_function'>

Could you please help me with this? Apologies if I'm doing something wrong here.

Hi there,
I am trying to generate a test case mentioned on Page 9 of your academic paper.
"However, the count javascript feature is not an accurate count of the number of embedded
JavaScript code pieces in a PDF. It just extracts the number of JavaScript keywords, but these keywords are optional in script execution. The targeted PDF reader will execute the JavaScript
even without the /Javascript keyword."

Can you help me with a test file or hash that can help me replicate this.

Hi! I am using hidost as a classifier for starting detection_agent_server. But classifiers/hidost_wrapper.py prompts <Fault 1: "<type ' exceptions.AttributeError'>:'dict' object has no attribute 'decision_function'">

The model file I am using is the SL2013.pickle file generated in the repository hidost-reproduction (https://github.com/srndic/hidost-reproduction), what can I do to fix this?

It is listed in the help message that we can specify an oracle (like cuckoo in the case of your experiment in the research paper) using the "-o" argument but it doesn't seem to work? I also took a look at the gp.py script and there is no "-o" argument at all. Is this intended? If so, is there any way we can reproduce the experiment using cuckoo as an oracle?

Hi,

I am trying to replicate the experiment results. After following the instructions on the hidost-reproduction repo (downloading the data tar file and doing sudo make on root directory), I can't find the features.nppf file required for the project.conf entry in the main EvadeML root directory.

Am i required to do some work with the main hidost repo (https://github.com/srndic/hidost)? I was under the impression that the hidost-reproduction would completely reproduce the experiment and remove the need for the first repo.

Any help in understanding the situation will be helpful. Thank you!

I'm trying to reproduce your experiment and I'm having a few issues. If you have any insights into causes, I'd really appreciate any suggestions you have.

1. When setting up the hidost classifier what model did you use (specified in project.conf > model_path . It looks like some models can be generated by running the code here: https://github.com/srndic/hidost-reproduction . Is that the right model to use, or do you have suggestions on where to find the right pkl file?

2. When trying to do
   Running step 1: "./utils/generate_ext_genome.py [classifier_name] [benign_sample_folder] [file_number] "
   We encountered the following issue where the rpcxml server was not found. Are there additional setup steps to launch the detection server?

dev.log

I have already installed the modified pdfrw (pdf malware parser). Any idea why this could be happening?

Could you provide the data used in your study to support replication and validation?

Specifically, providing a list of hashes for the seed malicious PDFs and the benign PDFs should allow your evaluation to be replicated with higher accuracy.