libyal / winevt-kb Goto Github PK

Windows Event Log Knowledge Base

License: Apache License 2.0

Python 98.93% Shell 0.57% Makefile 0.05% PowerShell 0.45%

winevt-kb's Introduction

winevt-kb is a project to build a Windows Event Log knowledge base.

winevtrc is a Python module part of winevt-kb to allow reuse of Windows
Event Log resources.

For more information see:

* Project documentation: https://winevt-kb.readthedocs.io/en/latest

winevt-kb's People

Stargazers

Watchers

Forkers

yang123vc benua74 benstobbs fengjixuchui

winevt-kb's Issues

Export change to Markdown

Change export to output to ~~asciidoc~~ Markdown instead of Google code wiki format

export.py: AttributeError: 'dict' object has no attribute 'iteritems'

While trying to run export.py (ae85a49) with Python 3.6.9, I'm getting this:

Traceback (most recent call last):
  File "scripts/export.py", line 567, in <module>
    if not Main():
  File "scripts/export.py", line 560, in Main
    exporter.Export(options.source, output_writer)
  File "scripts/export.py", line 211, in Export
    source_path, database_reader, output_writer)
  File "scripts/export.py", line 120, in _ExportMessageFiles
    output_writer.WriteMessageFile(message_file)
  File "scripts/export.py", line 349, in WriteMessageFile
    self._WriteMessageTable(message_table)
  File "scripts/export.py", line 306, in _WriteMessageTable
    for identifier, string in message_table.message_strings.iteritems():
AttributeError: 'dict' object has no attribute 'iteritems'

Traceback with McAfee DLLs mfeapfk.sys

Seems like the issue is on McAfee side since it specifies a directory path instead of the full path :

C:\Program Files\Common Files\McAfee\SystemCore\\
Traceback (most recent call last):
  File "extract.py", line 1089, in <module>
    if not Main():
  File "extract.py", line 1070, in Main
    extractor.ExtractEventLogMessageStrings(output_writer)
  File "extract.py", line 475, in ExtractEventLogMessageStrings
    message_file = self._OpenMessageResourceFile(message_filename)
  File "extract.py", line 409, in _OpenMessageResourceFile
    file_object = resolver.Resolver.OpenFileObject(path_spec)
  File "/usr/lib/python2.7/site-packages/dfvfs/resolver/resolver.py", line 106, in OpenFileObject
    file_object = resolver_helper.OpenFileObject(path_spec, resolver_context)
  File "/usr/lib/python2.7/site-packages/dfvfs/resolver/os_resolver_helper.py", line 47, in OpenFileObject
    file_object.open(path_spec=path_spec)
  File "/usr/lib/python2.7/site-packages/dfvfs/file_io/os_file_io.py", line 113, in open
    self._file_object = open(location, mode=mode)
IOError: [Errno 21] Is a directory: u'/mnt/nbd2p2/Program Files/Common Files/McAfee/SystemCore'

$ regfmount /mnt/nbd2p2/Windows/System32/config/SYSTEM /mnt/tmp0
$ cat "/mnt/tmp0/ControlSet001/services/eventlog/System/mfeapfk.sys/(values)/EventMessageFile"
C:\Program Files\Common Files\McAfee\SystemCore\\

The same for mfeavfk.sys, mferkdet.sys

add scripts

script to read eventlogs and message strings directly from images
- extract script
script to analyze logon/logoff (e.g. event id 4624)
- http://social.technet.microsoft.com/wiki/contents/articles/17055.event-ids-when-a-new-user-account-is-created-on-active-directory.aspx
- add Registry key access (id 4663)
script to analyze process start/stop

AttributeError: 'module' object has no attribute 'MESSAGE_FILE_TYPE_EVENT'

After installing dependencies, cloning and installing master, running export.py --db winevt-rc_new.db --string_format pep3101 dir_from_extract/ results in:
AttributeError: 'module' object has no attribute 'MESSAGE_FILE_TYPE_EVENT'

Fixed with:

unzip /usr/local/lib/python2.7/dist-packages/winevtrc-20160418-py2.7.egg EGG-INFO/scripts/export.py
sed -i 's|database.MESSAGE_FILE_TYPE_EVENT|database.definitions.MESSAGE_FILE_TYPE_EVENT|' EGG-INFO/scripts/export.py
sudo zip -u /usr/local/lib/python2.7/dist-packages/winevtrc-20160418-py2.7.egg EGG-INFO/scripts/export.py

fix issue

[INFO] Processing: C:\Program Files\VMware\VMware Tools\Guest SDK\vmStatsProvider\win64\vmStatsProviderMsgs.dll
[WARNING] Missing message file key for: C:\Program Files\VMware\VMware Tools\Guest SDK\vmStatsProvider\win64\vmStatsProviderMsgs.dll
[WARNING] Missing message file key for: C:\Program Files\VMware\VMware Tools\Guest SDK\vmStatsProvider\win64\vmStatsProviderMsgs.dll
Traceback (most recent call last):
  File "winevt-kb/extract.py", line 1083, in <module>
    if not Main():
  File "winevt-kb/extract.py", line 1064, in Main
    extractor.ExtractEventLogMessageStrings(output_writer)
  File "winevt-kb/extract.py", line 531, in ExtractEventLogMessageStrings
    event_log_provider, message_file, normalized_message_filename)
  File "winevt-kb/extract.py", line 884, in WriteMessageFile
    database_writer.WriteResources()
  File "winevt-kb/database.py", line 950, in WriteResources
    self._WriteMessageTables()
  File "winevt-kb/database.py", line 800, in _WriteMessageTables
    self._message_resource_file, message_table, language_identifier)
  File "winevt-kb/database.py", line 732, in _WriteMessageTable
    self._WriteMessageTableLanguage(message_file_key, language_identifier)
  File "winevt-kb/database.py", line 770, in _WriteMessageTableLanguage
    language_identifier, message_file_key)ValueError: Unknown format code 'd' for object of type 'unicode'

missing comma in setup.py

user: ~/winevt-kb $ sudo python setup.py install
  File "setup.py", line 139
    cmdclass={
           ^

setup.py is missing a comma after the scripts list

add dfwinreg support

~~replace directly usage of pyregf with dfwinreg~~

Incompatible with stable Plaso release (1.4 atm)

If a user wants to build its own winevt-kb.db it needs these scripts. At this moment these scripts depend on dfvfs-20160227 or higher. Plaso Stable version 1.4 comes with dfvfs-20160108. In the future, is it possible to make stable releases of packages/scripts like these linked to stable releases of Plaso? Tnx

extract/export improve event provider lookup

improve event provider support
- ~~link up message file with event log provider~~
- check EventSourceName/Name handling
- ~~add ProviderGuid~~
- handle/check for event providers with different GUIDs across different versions of Windows; e.g. XP has no provider GUID (also see: issue #7)
~~add parameter expansion support~~
~~add category file support~~
~~add volume-only image support~~
brute force scan for the available locales

Create a Python version of winevt-rc.db

Create a Python version of winevt-rc.db (e.g. pywinevt-rc.db) that includes:

~~message strings processed by _ReformatMessageString~~
~~Add metadata table to indicate Python-based string formatting.~~

extract.py: exception unsupported resource type: 0x00000000

I'm trying to build new winevt-kb.db so that i could use plaso tagging analyser with sysmon logs. My assumption is that you can't add sysmon events to tag_windows.txt without rebuilding winevt-kb.db , is that correct? psort.py reports immediatly "killed" while using custom sysmon tags.

I have mounted filesystem under win10sysmon/image -path. I noticed that i'll have to use dfvfs version 20180831 or extract.py does not run at all, and if i used newer version of dfvfs, for some reason i'll get apfs related -errors which are not relevant at all(?)

./extract.py --db win10sysmon/ win10sysmon/image/
Windows version: 10.0.17763.737.

[INFO] Current control set
[INFO] Processing: %SystemRoot%\system32\dimsjob.dll
[INFO] Processing: %SystemRoot%\System32\gpprefcl.dll
[INFO] Processing: %SystemRoot%\system32\wbem\WinMgmtR.dll
[INFO] Processing: %SystemRoot%\System32\appmgr.dll
[INFO] Processing: %SystemRoot%\system32\perfctrs.dll
[INFO] Processing: %SystemRoot%\system32\sxproxy.dll
[INFO] Processing: %SystemRoot%\System32\AxInstSv.dll
Traceback (most recent call last):
  File "./extract.py", line 291, in <module>
    if not Main():
  File "./extract.py", line 272, in Main
    extractor_object.ExtractEventLogMessageStrings(output_writer)
  File "/usr/local/lib/python2.7/dist-packages/winevtrc/extractor.py", line 431, in ExtractEventLogMessageStrings
    message_filename, definitions.MESSAGE_FILE_TYPE_EVENT)
  File "/usr/local/lib/python2.7/dist-packages/winevtrc/extractor.py", line 226, in _ExtractMessageFile
    mui_language = message_file.GetMUILanguage()
  File "/usr/local/lib/python2.7/dist-packages/winevtrc/resource_file.py", line 134, in GetMUILanguage
    if language_identifier in mui_resource.language_identifiers:
IOError: pywrc_resource_get_language_identifiers: unable to retrieve number of languages. libwrc_resource_read_value: unsupported resource type: 0x00000000. libwrc_resource_get_number_of_languages: unable to read resource value.

wevt_template script: add storage media image support

Change wevt_template script to support reading a message file directly from a storage media image

Add support to

Resolve .mui file for message table
Resolve .mun file for wevt_template

handle duplicate event log providers

Duplicate event log provides in portable message resource database currently causing issues.

SELECT * FROM event_log_providers WHERE log_source == 'Microsoft-Windows-Eventlog';
154|Microsoft-Windows-Eventlog|System|{fc65ddd8-d6ef-4962-83d5-6e5cfe9ce148}
378|Microsoft-Windows-Eventlog|Security|{fc65ddd8-d6ef-4962-83d5-6e5cfe9ce148}

winevtec.db support message strings for different versions

Update the portable (exported) message string database to support message strings for different versions

[WARNING] Found duplicate alternating message string: 0xc00007d1 in LCID: 0x00000409 and version: 5.1.2600.5512.
Previous: Unable to collect process virtual memory information. The first four bytes (DWORD) of the Data section contains the status code.

New:Unable to collect process virtual memory information. Status code
returned is data DWORD 0.

split off reusable functionality to winevtrc module

~~split off reusable functionality in scripts to winevt-rc module~~
add unit tests (+/-)
~~create test resource files with VS~~

Add message files to readthedocs documentation

Currently adding the message files will cause the readthedocs build to time out

See

if documentation build can be made more efficient
ask for more resources
other (different platform, build on github, etc.)

Also see: https://docs.readthedocs.io/en/stable/guides/build-using-too-many-resources.html