guelfoweb / peframe Goto Github PK
View Code? Open in Web Editor NEWPEframe is a open source tool to perform static analysis on Portable Executable malware and malicious MS Office documents.
PEframe is a open source tool to perform static analysis on Portable Executable malware and malicious MS Office documents.
Similar to #17, matching filenames will result in a lot of backtracking as Python tries to match the .+
in strings without a match.
Here's an example using a large string containing no matches:
import re
test = ('ABCDEF' * 100000)
re.findall("(.+(\.([a-z]{2,3}$)|\/.+\/|\\\.+\\\))+", test)
Matching the max path length on Windows, and putting a 260 character limit in place of the .+
seems a better solution. It will extract filenames and not consume as many cycles when no matches are found.
Hi, That how did I installed:
Traceback (most recent call last):
File "peframe.py", line 38, in
from modules import magic
File "C:\Users******************\peframe-master\peframe-master\peframe\modules\magic.py", line 150, in
libmagic = ctypes.CDLL(dll)
File "C:\Python27\lib\ctypes_init_.py", line 366, in init
self._handle = _dlopen(self._name, mode)
WindowsError: [Error 126] The specified module could not be found
Help?
I receive the following error message during an installation via pip:
ERROR: Invalid script entry point: <ExportEntry peframe = peframe.peframecli:None []> for req: peframe==6.0.3 - A callable suffix is required. Cf https://packaging.python.org/specifications/entry-points/#use-for-scripts for more information.
$ bash install.sh Check for python3...... which: no python3 in (/c/Users/Fetah/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/mingw64/bin:/usr/bin:/c/Users/Fetah/bin:/c/Program Files (x86)/Common Files/Intel/Shared Libraries/redist/intel64/compiler:/c/ProgramData/Oracle/Java/javapath:/c/Program Files (x86)/Intel/iCLS Client:/c/Program Files/Intel/iCLS Client:/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/WINDOWS/System32/WindowsPowerShell/v1.0:/c/Program Files/Hewlett-Packard/SimplePass:/c/Program Files/Intel/Intel(R) Management Engine Components/DAL:/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/DAL:/c/Program Files/Intel/Intel(R) Management Engine Components/IPT:/c/Program Files (x86)/Intel/Intel(R) Management Engine Components/IPT:/cmd:/c/Program Files/nodejs:/c/Program Files (x86)/Windows Kits/8.1/Windows Performance Toolkit:/c/Users/Fetah/Downloads/Compressed/ideaIU-2017.2.5.win/bin:/c/Users/Fetah/Downloads/Compressed/apache-maven-3.5.4-bin/apache-maven-3.5.4/bin:/c/Program Files (x86)/Yarn/bin:/c/Program Files/Java/jdk1.8.0_144/bin:/c/Users/Fetah/Downloads/Compressed/gradle-4.3-bin/gradle-4.3/bin:/c/Program Files (x86)/CodeBlocks/MinGW/bin:/h/Program Files/Microsoft VS Code/bin:/c/xampp/php:/h/composer:/c/Users/Fetah/Downloads/Compressed/apache-maven-3.5.4-bin/apache-maven-3.5.4/bin:/c/Program Files (x86)/Bitvise SSH Client:/c/Program Files/dotnet:/c/Program Files (x86)/GnuWin32/bin:/f/useful tool/MavTools:/c/Users/Fetah/PycharmProjects/PEReader/dist/GetDetail:/f/useful tool/MavTools/Sigcheck:/c/Users/Fetah/PycharmProjects/PEReader:/c/Users/Fetah/PycharmProjects/PEReader/dist/ThreadTest:/c/Users/Fetah/PycharmProjects/PEReader/dist/GetDetailTh01:/c/Users/Fetah/Downloads/openssl-1.0.2j-fips-x86_64/OpenSSL/lib:/c/Users/Fetah/Downloads/openssl-1.0.2j-fips-x86_64/OpenSSL/include:/c/Users/Fetah/AppData/Local/Microsoft/WindowsApps:/c/Users/Fetah/Downloads/Compressed/ideaIU-2017.2.5.win/bin:/c/Users/Fetah/AppData/Local/Yarn/bin:/c/Users/Fetah/AppData/Roaming/Composer/vendor/bin:/c/Users/Fetah/.dotnet/tools:/c/Users/Fetah/AppData/Local/Programs/Python/Python36-32/Scripts:/c/Users/Fetah/AppData/Local/Programs/Python/Python36-32:/usr/bin/vendor_perl:/usr/bin/core_perl) install.sh: line 8: sudo: command not found install.sh: line 9: sudo: command not found Check for pip3......... Install libssl-dev..... install.sh: line 18: sudo: command not found Install swig........... install.sh: line 21: sudo: command not found Install dependencies... Requirement already satisfied: pefile in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from -r requirements.txt (line 1)) (2018.8.8) Requirement already satisfied: python-magic in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from -r requirements.txt (line 2)) (0.4.15) Requirement already satisfied: yara-python in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from -r requirements.txt (line 3)) (3.8.1) Requirement already satisfied: virustotal-api in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from -r requirements.txt (line 4)) (1.1.10) Requirement already satisfied: oletools in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from -r requirements.txt (line 5)) (0.53.1) Requirement already satisfied: M2Crypto in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from -r requirements.txt (line 6)) (0.21.1) Requirement already satisfied: future in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from pefile->-r requirements.txt (line 1)) (0.17.1) Requirement already satisfied: requests>=2.2.1 in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from virustotal-api->-r requirements.txt (line 4)) (2.21.0) Requirement already satisfied: pyparsing in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages\pyparsing-2.2.0-py3.6.egg (from oletools->-r requirements.txt (line 5)) (2.2.0) Requirement already satisfied: urllib3<1.25,>=1.21.1 in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from requests>=2.2.1->virustotal-api->-r requirements.txt (line 4)) (1.24.1) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from requests>=2.2.1->virustotal-api->-r requirements.txt (line 4)) (3.0.4) Requirement already satisfied: idna<2.9,>=2.5 in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from requests>=2.2.1->virustotal-api->-r requirements.txt (line 4)) (2.6) Requirement already satisfied: certifi>=2017.4.17 in c:\users\fetah\appdata\local\programs\python\python36-32\lib\site-packages (from requests>=2.2.1->virustotal-api->-r requirements.txt (line 4)) (2019.3.9)
Hi there ... I saw you use O(n) algorithms to search for known anti vm patterns ( and generally speaking for known patterns from userdb ) ... no need to say, if you plan to make your db bigger ( and more updated ) this is gonna be SLOW, especially with big PE files ( i.e. few executable code with a big .rsrc section ).
It would be a better approach to use the Aho-Corasick algorithm which is perfect in such circumstances ... you can find a Python implementation here, unless you are familiar with finite state automata, trie data structures, etc and you want to implement your own ( which I guarantee is quite funny to do :) ).
Hi Gianni,
it would be really awesome to see peframe also on pypi. Could please push it to pypi. :-)
cheers
jl
Hi,
Thanks for having developed this tool for Linux users.
I'm just a novice to both malware analysis and in using github. I don't know whether this is a matter that needs to be brought to notice of developers.
While using the userdb.txt for parsing using a python program that I wrote, I came across many symbols that were not natural for a .txt file encoding.
Kindly advice the encoding format so that I can use suitable python library to parse the contents of userdb.txt.
There are many symbols such as for space etc.
https://raw.githubusercontent.com/guelfoweb/peframe/master/peframe/signatures/userdb.txt
Line 10723:
[PcShare ΄¼þÀ¦°v4.0 -> Ξ¿ɷǒ靍
Thanks
Hi,
The regex pattern ".(curl|wget)." can cause an analysis to seemingly hang at this point in fileurl.py due to excessive backtracking:
fuzz_match = re.findall(value, string, re.IGNORECASE | re.MULTILINE)
I've done some testing by generating random strings of a defined length and using the same regex check on them... and found completion times of:
8K = 1s
16K = 3.9s
32K = 15.8s
64K = 63.9s
128K = 256.5s
So, when a Pony sample of mine resulted in a 570K string being produced, this basically never completed.
The string either needs to be broken up, or the regex revised. For now I've just put a condition in to skip this particular check for strings longer than 64K.
Thanks,
Chris
The stock stringsmatch.json
has a fuzzing value regex .*(curl|wget).*
. For large strings (thousands of characters) without a match this will result in a lot of backtracking as Python tries to greedily match .*
and backtracks when it fails.
Ben Federickson has a good write-up on this.
Using (curl|wget)
alone would be enough to flag a possible connection. To maintain some context of where the match was made, the match could use (.{,10})(curl|wget)(.{,10})
to provide up to 10 characters of around the match, but there will still be a significant overhead in many cases.
HI,
It just a query, Do you have any plan to give a GUI to peframe?
I would like to work for it. I can create GUI for peframe.
It looks like the license headers still show different licenses, some GPL:
$ git grep -l 'GNU General Public Licen'
peframe/modules/dump.py
peframe/modules/info.py
peframe/modules/loadfile.py
peframe/modules/pecore.py
peframe/modules/secalert.py
peframe/modules/stdoutput.py
peframe/modules/strings.py
and some MIT:
$ git grep -l 'The MIT'
peframe/modules/antivm.py
peframe/modules/apialert.py
peframe/modules/apiantidbg.py
peframe/modules/apimutex.py
peframe/modules/cert.py
peframe/modules/directories.py
peframe/modules/directory.py
peframe/modules/fileurl.py
peframe/modules/funcexport.py
peframe/modules/funcimport.py
peframe/modules/help.py
peframe/modules/meta.py
peframe/modules/peid.py
peframe/modules/resources.py
peframe/modules/sections.py
peframe/modules/stringstat.py
peframe/modules/virustotal.py
peframe/modules/xor.py
peframe/peframe.py
Since setup.py
declares MIT as the package license, I would appreciate of the license would be made consistent. From a Debian packager's point of view, it would also be beneficial (while not absolutely required) to have a proper LICENSE file in the root directory.
Given how terribly complex it has become to install M2Crypto on OSX it would be nice if peframe could still be made to work without it dropping some functionality.
Hi,
today I tried to play around with this tool with some of my samples.
But I noticed that some of my sample with show this kind of error if the sample is analyse for the first time. If I run the same command with the same sample, then the error will gone.
This is the error looks like:
$ python peframe.py ../binaries/PE32/small/de85ae919d48325189bead995e8052e7
Short information
------------------------------------------------------------
File Name de85ae919d48325189bead995e8052e7
File Size 44420 byte
Compile Time 2009-03-28 01:32:04
DLL No
Sections 3
Hash MD5 de85ae919d48325189bead995e8052e7
Hash SAH1 0f1892137e3a42997eaf21bef7540616c9d5fbc3
Imphash b48d4f95b75d3b29bfe6a5d6b20bb1d0
Packer Yes
Anti Debug Yes
Anti VM No
Directory Import
Packer matched [5]
------------------------------------------------------------
Packer Microsoft Visual C++ v6.0
Packer Microsoft Visual C++ 5.0
Packer Microsoft Visual C++
Packer Microsoft Visual C++ v6.0
Packer Installer VISE Custom
Anti Debug discovered [2]
------------------------------------------------------------
Function TerminateProcess
Function UnhandledExceptionFilter
Suspicious API discovered [28]
------------------------------------------------------------
Function CopyFileA
Function CreateProcessA
Function CreateThread
Function DeleteFileA
Function ExitThread
Function GetCommandLineA
Function GetComputerNameA
Function GetCurrentProcess
Function GetModuleFileNameA
Function GetModuleHandleA
Function GetProcAddress
Function GetStartupInfoA
Function GetTickCount
Function GetUserNameA
Function LoadLibraryA
Function RegCloseKey
Function RegCreateKeyExA
Function Sleep
Function TerminateProcess
Function UnhandledExceptionFilter
Function VirtualAlloc
Function VirtualFree
Function WSAStartup
Function WriteFile
Function closesocket
Function connect
Function send
Function socket
File name discovered [10]
------------------------------------------------------------
Executable %s\admin$\system32\dnsapi.exe
Library ADVAPI32.dll
Library KERNEL32.dll
Library MPR.dll
Library NETAPI32.dll
Library WS2_32.dll
Executable \msupd.exe
Library urlmon.dll
Library user32.dll
Library wininet.dll
Url discovered [1]
------------------------------------------------------------
Url http://fukyu.jp/updata/ACCl3.jpg
Wait please... storing info into the db
Traceback (most recent call last):
File "peframe.py", line 401, in <module>
autoanalysis(filename)
File "peframe.py", line 349, in autoanalysis
au_show_sec,au_show_furl,au_show_meta)
File "modules/db_manage.py", line 107, in dbmanage
cur.execute('''INSERT INTO show_url (id, url) VALUES (?, ?)''', (idrif, au_show_furl[1][i]))
sqlite3.InterfaceError: Error binding parameter 1 - probably unsupported type.
This happen to some of my sample only and not all of it.
Is it normal? Or do I miss something? Thanks.
please check.
Starting with 6.0.0, setup.py
is gone. This makes it more difficult to continue packaging peframe
.
Hi,
When i run peframe with below configuration:
python peframe.py --json /malware0.exe
For recreating the error use the below file: ( i am giving the hash of file)
SHA256: d6dcfa69ef0e437fbcc60a1ea4f03019e4814fa90b789e0e80d5179022e2b118
I am getting below error for some of the file. ( not for all files)
Traceback (most recent call last):
File "peframe.py", line 94, in
autoanalysis(pe, filename, json=True); exit(0)
File "peframe.py", line 50, in autoanalysis
pecore.get_fileurl(filename),
File "modules/pecore.py", line 75, in get_fileurl
show_fileurl = fileurl.get(filename)
File "modules/fileurl.py", line 175, in get
return filelist, arrayURL
UnboundLocalError: local variable 'filelist' referenced before assignment
I am using on Ubuntu system.
Sad face....I have peframe 5.0.1 installed via apt and I love it, but fail with peframe 6:
Traceback (most recent call last):
File "peframe-cli.py", line 278, in <module>
result = peframe.analyze(filename)
File "/opt/peframe/peframe.py", line 97, in analyze
"filetype": filetype(filename),
File "/opt/peframe/peframe.py", line 41, in filetype
return magic.from_file(filename)
AttributeError: module 'magic' has no attribute 'from_file'
anything I can do to troubleshoot this? Thank you.
When processing a PE file with exports that are unnamed (referenced only by ordinal value), peframe will throw the following exception because func['function'] contains None.
File "/usr/lib/python2.7/site-packages/peframe-5.0-py2.7.egg/peframe/peframe.py", line 362, in stdoutput
print func['function'][0:15].ljust(15), func['address']
TypeError: 'NoneType' object has no attribute 'getitem'
Depending on where you want to address this, one solution would be around line 361:
for func in output['pe_info'][item]:
if func['function'] is None:
print "Unnamed export".ljust(15), func['address']
else:
print func['function'][0:15].ljust(15), func['address']
Thanks.
Hi,
Be able to Extract the Thumbprint serial certificate
It's posible extract it, using pefile
Hello,
I tried running peframe and i keep getting the below error. I am running MacOSX 10.15.6 Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT 2020; root:xnu-6153.141.1~1/RELEASE_X86_64 x86_64
. I did try running it on my kali vm and i got the same err
Traceback (most recent call last):
File "/usr/local/bin/peframe", line 33, in <module>
sys.exit(load_entry_point('peframe==6.0.3', 'console_scripts', 'peframe')())
File "/usr/local/bin/peframe", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/local/Cellar/[email protected]/3.9.1_7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/usr/local/Cellar/[email protected]/3.9.1_7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 790, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/usr/local/lib/python3.9/site-packages/peframe-6.0.3-py3.9.egg/peframe/peframecli.py", line 289, in <module>
result = peframe.analyze(filename)
File "/usr/local/lib/python3.9/site-packages/peframe-6.0.3-py3.9.egg/peframe/peframe.py", line 143, in analyze
"features": features.get_result(pe, filename),
File "/usr/local/lib/python3.9/site-packages/peframe-6.0.3-py3.9.egg/peframe/modules/features.py", line 110, in get_result
"xor": get_xor(filename),
File "/usr/local/lib/python3.9/site-packages/peframe-6.0.3-py3.9.egg/peframe/modules/features.py", line 37, in get_xor
key_delta = xor_delta(search_string, l)
File "/usr/local/lib/python3.9/site-packages/peframe-6.0.3-py3.9.egg/peframe/modules/features.py", line 21, in xor_delta
return delta.tostring()[:-key_len]
AttributeError: 'array.array' object has no attribute 'tostring'
I have created a REST API for PEframe to easily integrate it with other tools or deploy a web instance.
DockerHub: https://hub.docker.com/r/eshaan7/peframe-rest-server
Hi !
just to say, i tested peframe and it is very powerfull and simple at the same time.
awesome work you did here !
I am currently trying to use Peframe in an automated way:
--> my customers send me many files / days
--> a preliminary check get me a score about the files (as do PeStudio) to see if it should be interesting to investigate.
Do you think it would be possible to have a scoring system in PEframe (even it's not very representative), so we could use a trigger (for example, if score > 50, we send the file to our CERT)
Kind regards
This project is not designed with lib functionality in mind. Output formats are exclusive to formatted text, and json formatted text (via json.dumps()). Which is severely limiting. Objects being returned needs to be an option for those of us that want to build larger applications with this code.
So there are a few bugs that prevent this from working:
get_exported_functions
dll
attribute for the exportsshow_exported_functions
(s/len_imported/len_exported)I've included a patch below:
diff --git a/modules/pecore.py b/modules/pecore.py
index b0cb853..a98758b 100644
--- a/modules/pecore.py
+++ b/modules/pecore.py
@@ -408,11 +408,10 @@ def get_exported_functions(filename):
try:
for exp in pe.DIRECTORY_ENTRY_EXPORT.symbols:
# print hex(pe.OPTIONAL_HEADER.ImageBase + exp.address), exp.name, exp.ordinal
- dll = exp.dll
- for imp in exp.expports:
- address = hex(pe.OPTIONAL_HEADER.ImageBase + exp.address)
- function = exp.name
- array.append([dll, address, function])
+ dll = "" #exp.dll
+ address = hex(pe.OPTIONAL_HEADER.ImageBase + exp.address)
+ function = exp.name
+ array.append([dll, address, function])
return array
except:
return False
diff --git a/peframe.py b/peframe.py
index 4e2a972..c09b2ce 100644
--- a/peframe.py
+++ b/peframe.py
@@ -196,7 +196,7 @@ def show_exported_functions(filename):
arrayDll = []
# print exported
if exported:
- len_imported = len(exported)
+ len_exported = len(exported)
for i in range(0, len_exported):
arrayDll.append(exported[i][0])
dllfound = set(arrayDll)
=== SECTIONS ===
NAME RVA VSZ RAW_SZ RAW_PTR nREL REL_PTR nLINE LINE_PTR FLAGS
.text 1000 20456 20600 400 0 0 0 0 60000020 R-X CODE
.rdata 22000 9c1a 9e00 20a00 0 0 0 0 40000040 R-- IDATA
.data 2c000 215e4 1fa00 2a800 0 0 0 0 c0000040 RW- IDATA
.rsrc 4e000 13f4c 14000 4a200 0 0 0 0 40000040 R-- IDATA
Like pedump
Hi, I'm looking to package peframe for Debian and in order to automate checking for new versions, etc. it would be nice if you could tag released versions using Git tags (e.g. v5.0.1
)here on GitHub.
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.