j40903272 / malconv-keras Goto Github PK

This is the implementation of MalConv proposed in [Malware Detection by Eating a Whole EXE](https://arxiv.org/abs/1710.09435) and its adversarial sample crafting.

License: MIT License

Python 100.00%

adversarial-examples keras malware-detection python3

malconv-keras's Introduction

📖 About me 👋

😄 I'm a PhDc at NTUCSIE
⭐ I have 232 stars on GitHub
🐼 I'm a member of @MSLAB
🦊 My research area includes Generative AI, Recommendation, Beyasian learning and Adversarial learning
🐨 My programming skills include C/C++, Java, Javascript, SQL and Python

malconv-keras's People

Contributors

Stargazers

Watchers

malconv-keras's Issues

Your net structure is different from the article decribe. The order of the structure is not correct.

hi,@j40903272,there is pull request for you.Your code is good,but i want to join you to make it better.
The structure of malware is:

Your code struct is as follow:

The change struct is:

So,you can see the correct struct is the second one.And i have test both two,the second one also has a better performance.

The missing ReLU layer before GlobalMaxPool1D

Dear j40903272,

Thank you very much for sharing the code!!

According to the picture on NVIDIA website(https://devblogs.nvidia.com/malware-detection-neural-networks/), there should be one ReLU layer after the multiply and before the GlobalMaxPool1D layer. This ReLU seems not explicitly mentioned in the paper.

I can not find datasets

Hello,

I got error while training as I could not find the datasets mentioned in example.csv.

E:\dataset\malware\malware\fbd1a4b23eff620c1a36f7c9d48590d2fccda4c2_cc82281bc576f716d9a0271d206beb81ad078b53 not exist E:\dataset\malware\malware\5746b9ed6b78d8f1cdb0b8bb9016624a46430346_007439fd0f1bfcb25e540a2b5ddbc96d7ea09afd not exist E:\dataset\malware\malware\c095da034535f15a27c073dce54212a28e1af683_8e86441bc4f6a7fc492779caf280f1d769e0cd4d not exist E:\dataset\malware\malware\3138cef67d651745c2db3744a70d6bf861e2a728_7b5ec86f006dc6d35d48821ca1311561bd86157a not exist E:\dataset\malware\malware\3f3fc4ed6a51a4385de9a38b164bcc14a4334eac_20002faf28adfd94ca98cf6ced46f14334b53684 not exist E:\dataset\malware\malware\80913fa3c7d18d89c47b42e929cf9c98729aa565_2dcb3285f944bc0b6e7cb7612dad360b9458ccdc not exist E:\dataset\malware\malware\ead2a1884a52c93c654853d2891a1208ac6ee2b8_b75311ed885fb252554ab6dc228326d388043630 not exist E:\dataset\malware\malware\19c625db4021d4934916f42b97777098b0b8cf15_0192ba11326fe2298c8cb4de616f4d4140213838 not exist E:\dataset\malware\malware\2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist E:\dataset\malware\malware\488e5eea345e24440f7d0d2a32fbafda314ee6ca_df473c0493d503828157e32664e28357a4094f7a not exist E:\dataset\malware\malware\c47ed37c0e1e8be110f889f5989aea6b1bb7fda4_42c3033e34d27c951f7db7ae6aa5f45b8ef472ac not exist E:\dataset\malware\malware\0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
Can you please tell me where I can find and download them?

about the dataset

can you refer some dataset that I can play with?

error with training the dataset

hi @j40903272
I'm somewhat new to this so i apologize for my preliminary questions.
i can't seem to get the training phase to work.
when i type python3 train.py mytrainset.csv it says that the malware and benign files does not exist.
I have 400 malwares and 400 benign files which all of them are in .bytes format. i placed them in the MalConv-keras directory in a file called train. then i made a csv file called mytrainset.csv (which is in the MalConv-keras directory).
so i have a couple of questions:
1 - what am i doing wrong?? or rather what should i do??
2- where should i put my malware of or benign files??
3- is the .bytes format OK or should i use different formats like .asm or .exe or ...
I look forward to your reply, thnx

UnboundLocalError: local variable 'final_adv' referenced before assignment

Hi.Thank you for the code.
When i run'python gen_adversarial.py ../saved/example.csv'

FILE: /benign_samples/com.lekusoft.android.wallpaper.a20110701077.apk file length: 1215354 , Exceed max length ! Ignored ! original score: 0.99955493 Traceback (most recent call last): File "gen_adversarial.py", line 115, in <module> adv_samples, log = gen_adv_samples(model, fn_list, args.pad_percent, args.step_size, args.thres) File "gen_adversarial.py", line 96, in gen_adv_samples bin_adv = bytes(list(final_adv)) UnboundLocalError: local variable 'final_adv' referenced before assignment
Is it the data problem?

python train.py example.csv error windows pycharm

Hi @j40903272
When i try to run python train.py example.csv i get this error:
it has couple of errors i think

Using TensorFlow backend.
WARNING:tensorflow:From C:\Users\alika\PycharmProjects\DeepLearning\venv\lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in
a future version.
Instructions for updating:
Colocations handled automatically by placer.
Train on 18 data, test on 2 data
WARNING:tensorflow:From C:\Users\alika\PycharmProjects\DeepLearning\venv\lib\site-packages\tensorflow\python\ops\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.

Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From C:\Users\alika\PycharmProjects\DeepLearning\venv\lib\site-packages\tensorflow\python\ops\math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
Epoch 1/100
0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist
3ac6ef121770866e84cb9b3f799e373da53a8e02_9eed86e0fc9ad5cfb9642aeca6e792c56c1ecac0 not exist
0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
3138cef67d651745c2db3744a70d6bf861e2a728_7b5ec86f006dc6d35d48821ca1311561bd86157a not exist
2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist
2b5137a1658c040ebe72bba8506c8405a06e173e_48c0985d5f2bb938d526fc1f18e8c1ad7b7c62d8 not exist
0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
12113db281913797b6d58079b45089d4d057f766_ce1c8adb310886deb3ace648d4152ddeb1ed32fb not exist
2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist
7a359bcc1c7ac5f18eff7c3459dadefa9f9e4610_3b7ac6b0a7a720460526c78628c8616dad8c6a1f not exist
0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
05202b7ebc42f2a159154f99cec58fc1bcfe2c17_8cc75062dbe1ddc363fa5178312845709f669e37 not exist
2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist
c47ed37c0e1e8be110f889f5989aea6b1bb7fda4_42c3033e34d27c951f7db7ae6aa5f45b8ef472ac not exist
0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
fbd1a4b23eff620c1a36f7c9d48590d2fccda4c2_cc82281bc576f716d9a0271d206beb81ad078b53 not exist
2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist
18a62e8ee8522c26e9970373895209ee15a56841_4f0f6e9d21bbbe1842e8e8d6d911561108389662 not exist
0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
509038aad80431b8aa0c9b29bfce07fe7134fc7a_263fbe72e691a0c047f75ce75a585ba0af84ac94 not exist
2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist
ead2a1884a52c93c654853d2891a1208ac6ee2b8_b75311ed885fb252554ab6dc228326d388043630 not exist
0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
c095da034535f15a27c073dce54212a28e1af683_8e86441bc4f6a7fc492779caf280f1d769e0cd4d not exist
2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist
488e5eea345e24440f7d0d2a32fbafda314ee6ca_df473c0493d503828157e32664e28357a4094f7a not exist
0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
5746b9ed6b78d8f1cdb0b8bb9016624a46430346_007439fd0f1bfcb25e540a2b5ddbc96d7ea09afd not exist
2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist
80913fa3c7d18d89c47b42e929cf9c98729aa565_2dcb3285f944bc0b6e7cb7612dad360b9458ccdc not exist
0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
6bc4d5ba3fa9c0f1c1431de8d214cc90bcfe4b55_4f53fe0260bec1166add1d1977e461984c448473 not exist
2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist
3f3fc4ed6a51a4385de9a38b164bcc14a4334eac_20002faf28adfd94ca98cf6ced46f14334b53684 not exist
0778a070b283d5f4057aeb3b42d58b82ed20e4eb_f205bd9628ff8dd7d99771f13422a665a70bb916 not exist
19c625db4021d4934916f42b97777098b0b8cf15_0192ba11326fe2298c8cb4de616f4d4140213838 not exist
2a7a2505e43decf3583ec11bbad6ead816ef1861_2906bffe07800fe9ff93d544bff8a46e56aa64d9 not exist
3ac6ef121770866e84cb9b3f799e373da53a8e02_9eed86e0fc9ad5cfb9642aeca6e792c56c1ecac0 not exist
Traceback (most recent call last):
3138cef67d651745c2db3744a70d6bf861e2a728_7b5ec86f006dc6d35d48821ca1311561bd86157a not exist
  File "train.py", line 74, in <module>
2b5137a1658c040ebe72bba8506c8405a06e173e_48c0985d5f2bb938d526fc1f18e8c1ad7b7c62d8 not exist
12113db281913797b6d58079b45089d4d057f766_ce1c8adb310886deb3ace648d4152ddeb1ed32fb not exist
    history = train(model, args.max_len, args.batch_size, args.verbose, args.epochs, args.save_path, args.save_best)
7a359bcc1c7ac5f18eff7c3459dadefa9f9e4610_3b7ac6b0a7a720460526c78628c8616dad8c6a1f not exist
  File "train.py", line 47, in train
05202b7ebc42f2a159154f99cec58fc1bcfe2c17_8cc75062dbe1ddc363fa5178312845709f669e37 not exist
    validation_steps=len(x_test)//batch_size + 1)
c47ed37c0e1e8be110f889f5989aea6b1bb7fda4_42c3033e34d27c951f7db7ae6aa5f45b8ef472ac not exist
  File "C:\Users\alika\PycharmProjects\DeepLearning\venv\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
fbd1a4b23eff620c1a36f7c9d48590d2fccda4c2_cc82281bc576f716d9a0271d206beb81ad078b53 not exist
    return func(*args, **kwargs)
18a62e8ee8522c26e9970373895209ee15a56841_4f0f6e9d21bbbe1842e8e8d6d911561108389662 not exist
  File "C:\Users\alika\PycharmProjects\DeepLearning\venv\lib\site-packages\keras\engine\training.py", line 1418, in fit_generator
509038aad80431b8aa0c9b29bfce07fe7134fc7a_263fbe72e691a0c047f75ce75a585ba0af84ac94 not exist
ead2a1884a52c93c654853d2891a1208ac6ee2b8_b75311ed885fb252554ab6dc228326d388043630 not exist
c095da034535f15a27c073dce54212a28e1af683_8e86441bc4f6a7fc492779caf280f1d769e0cd4d not exist
488e5eea345e24440f7d0d2a32fbafda314ee6ca_df473c0493d503828157e32664e28357a4094f7a not exist
    initial_epoch=initial_epoch)
5746b9ed6b78d8f1cdb0b8bb9016624a46430346_007439fd0f1bfcb25e540a2b5ddbc96d7ea09afd not exist
  File "C:\Users\alika\PycharmProjects\DeepLearning\venv\lib\site-packages\keras\engine\training_generator.py", line 217, in fit_generator
80913fa3c7d18d89c47b42e929cf9c98729aa565_2dcb3285f944bc0b6e7cb7612dad360b9458ccdc not exist
    class_weight=class_weight)
6bc4d5ba3fa9c0f1c1431de8d214cc90bcfe4b55_4f53fe0260bec1166add1d1977e461984c448473 not exist
  File "C:\Users\alika\PycharmProjects\DeepLearning\venv\lib\site-packages\keras\engine\training.py", line 1211, in train_on_batch
3f3fc4ed6a51a4385de9a38b164bcc14a4334eac_20002faf28adfd94ca98cf6ced46f14334b53684 not exist
19c625db4021d4934916f42b97777098b0b8cf15_0192ba11326fe2298c8cb4de616f4d4140213838 not exist
    class_weight=class_weight)
  File "C:\Users\alika\PycharmProjects\DeepLearning\venv\lib\site-packages\keras\engine\training.py", line 804, in _standardize_user_data
    check_array_length_consistency(x, y, sample_weights)
  File "C:\Users\alika\PycharmProjects\DeepLearning\venv\lib\site-packages\keras\engine\training_utils.py", line 237, in check_array_length_consistency
    'and ' + str(list(set_y)[0]) + ' target samples.')
ValueError: Input arrays should have the same number of samples as target arrays. Found 0 input samples and 18 target samples.

i look forward to your reply

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.