Git Product home page Git Product logo

sealighter's Introduction

Sealighter - Easy ETW Tracing for Security Research

CI

I created this project to help non-developers dive into researching Event Tracing for Windows (ETW) and Windows PreProcessor Tracing (WPP).

Features

  • Subscribe to multiple ETW and WPP Providers at once
  • Automatically parse events into JSON without needing to know format
  • Robust Event filtering including filter chaining and filter negation
  • Output to Standard out, File, or Windows Event Log (to be ingested by other tools)
  • Get event stack traces
  • Configurable Buffering many events in a time period into one with a count, to reduce the number of events generated

Screenshot of Sealighter running

Overview

Sealighter leverages the feature-rich Krabs ETW Library to enable detailed filtering and triage of ETW and WPP Providers and Events.

You can subscribe and filter multiple providers, including User mode Providers, Kernel Tracing, and WPP Tracing, and output events as JSON to either stdout, a file, or the Windows Event Log (useful for high-volume traces like FileIO). No knowledge of the events the provider may produce, or their format, is necessary, Sealighter automatically captures and parses any events it is asked.

Events can then be parsed from JSON in Python, PowerShell, or forwarded to Splunk or ELK for further searching.

Filtering can be done on various aspects of an Event, from its ID or Opcode, to matching a property value, to doing an arbitrary string search across the entire event (Useful in WPP traces or when you don't know the event structure, but have an idea of its contents). You can also chain multiple filters together, or negate the filter. You can also filter the maximum events per ID, useful to investigate a new provider without being flooded by similar events.

Why this exists

ETW is an incredibly useful system for both Red and Blue teams. Red teams may glean insight into the inner workings of Windows components, and Blue teams might get valuable insight into suspicious activity.

A common research loop would be:

  1. Identify interesting ETW Providers using logman query providers or Looking for WPP Traces in Binaries
  2. Start a Session with the interesting providers enable, and capture events whilst doing something 'interesting'
  3. Look over the results, using one or more of:
    • Eyeballing each event/grepping for words you expect to see
    • Run a script in Python or PowerShell to help filter or find interesting captured events
    • Ingesting the data into Splunk or an ELK stack for some advanced UI-driven searching

Doing this with ETW Events can be difficult, without writing code to interact with and parse events from the obtuse ETW API. If you're not a strong programmer (or don't want to deal with the API), your only other options are to use a combination of older inbuilt windows tools to write to disk as binary etl files, then dealing with those. WPP traces compounds the issues, providing almost no easy-to-find data about provider and their events.

Projects like JDU2600's Event List and ETWExplorer and give some static insight, but Providers often contain obfuscated event names like Event(1001), meaning the most interesting data only becomes visible by dynamically running a trace and observing the output.

So like SilkETW?

In a way, this plays in a similar space as FuzzySec's SilkETW. But While Silk is more production-ready for defenders, this is designed for researchers like myself, and as such contains a number of features that I couldn't get with Silk, mostly due to the different Library they used to power the tool. Please see Here for more information.

Intended Audience

Probably someone who understands the basic of ETW, and really wants to dive into discovering what data you can glean from it, without having to write code or manually figure out how to get and parse events.

Getting Started

Please read the following pages:

Installation - How to start running Sealighter, including a simple config, and how to set up Windows Event logging if required.

Configuration - How to configure Sealighter, including how to specify what Providers to Log, and where to log to.

Filtering - Deep dive into all the types of filtering Sealighter provides.

Buffering - How to use buffering to report many similar events as one

Parsing Data - How to get and parse data from Sealighter.

Scenarios - Walkthrough example scenarios of how I've used Sealighter in my research.

Limitations - Things Sealighter doesn't do well or at all.

Why it's called Sealighter

The name is a contraction of Seafood Highlighter, which is what we call fake crab meat in Oz. As it's built on Krabs ETW, I thought the name was funny.

Found problems?

Feel free to raise an issue, although as I state in the comparison docs I'm only a single person, and this is a research-ready tool, not a production-ready.

Props and further reading

sealighter's People

Contributors

pathtofile avatar tandasat avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sealighter's Issues

Default config file causes error

Using the provided default config file will raise an exception..

https://github.com/pathtofile/Sealighter/blob/main/docs/CONFIGURATION.md#kernel_traces

{
    "session_properties": {
        "session_name": "My-Process-Trace",
        "output_format": "stdout",
        "buffering_timout_seconds":  10
    },
    "user_traces": [
        {
            "trace_name": "proc_trace",
            "provider_name": "Microsoft-Windows-Kernel-Process",
            "keywords_any": 16
        },
        {
            "trace_name": "guid_trace",
            "provider_name": "{382b5e24-181e-417f-a8d6-2155f749e724}",
            "filters": {
                "any_of": {
                    "opcode_is": [1, 2]
                }
            },
            "buffers": [
                {
                    "event_id": 1,
                    "max_before_buffering": 1,
                    "fields": [
                        "ImageName"
                    ]
                }
            ]
        },
    ],
    "kernel_traces": [
        {
            "trace_name": "kernel_proc_trace",
            "provider_name": "process",
        }
    ]
}

[json.exception.parse_error.101] parse error at line 31, column 5: syntax error while parsing value - unexpected ']'; expected '[', '{', or a literal

Sealighter crashes for MOF providers

Hi,
I tried starting a trace session for the Provider "Active Directory Domain Services: Core" (GUID: 1C83B2FC-C04F-11D1-8AFC-00C04FC21914) but Sealighter crashes after printing the session info.

    "user_traces": [
        {
            "trace_name": "test_file_trace",
			"provider_name": "{1C83B2FC-C04F-11D1-8AFC-00C04FC21914}"
        }
   ]

Looks like the same happens for other MOF Providers as well.

Since it's not mentioned directly in the documentation - are MOF-Providers generally not supported?
KrabsETW does support MOF and I was able to start the session and parse the events directly via Krabs.

Also, is there any flag to activate some verbose debugging output for Sealighter?
Thanks!

Issues in recent Windows Version 10.0.19042.746

Sealighter "seems" not to work properly under Windows 10.0.19042.746, tested with Windows vanilla install today.
No Eventchannel for sealighter is beeing created.

Here is the debug data:

Output of sealighter -d

[DEBUG][Korben Dallas] Check requirements
[Korben Dallas] [*] Requirements OK
[DEBUG][Korben Dallas] Get the name of the DLL to hijack
[Korben Dallas] [*] DLL to hijack: EventAggregation.dll
[Korben Dallas] [*] Current user is SYSTEM? -> FALSE
[DEBUG][Korben Dallas] Found a potential Process candidate: PID=744 - Image='winlogon.exe' - User='NT-AUTORIT─T\SYSTEM'
[DEBUG][Korben Dallas] This token is not restricted.
[DEBUG][Korben Dallas] Found 2/2 required privileges in token.
[DEBUG][Korben Dallas] Found a valid Token candidate.
[SYSTEM] [*] Impersonating SYSTEM...
[DEBUG][SYSTEM] Create object directory '\GLOBAL??\KnownDlls'...
[SYSTEM] [*] Created Object Directory: '\GLOBAL??\KnownDlls'
[DEBUG][SYSTEM] Create symbolic link '\GLOBAL??\KnownDlls\EventAggregation.dll'...
[SYSTEM] [*] Created Symbolic link: '\GLOBAL??\KnownDlls\EventAggregation.dll'
[DEBUG][Korben Dallas] Create symbolic link '\??\GLOBALROOT -> \GLOBAL??'...
[Korben Dallas] [*] Created symbolic link: '\??\GLOBALROOT -> \GLOBAL??'
[DEBUG][Korben Dallas] Call DefineDosDevice to create '\KnownDlls\EventAggregation.dll' -> '\KernelObjects\EventAggregation.dll'
[Korben Dallas] [*] DefineDosDevice OK
[DEBUG][Korben Dallas] Impersonate SYSTEM again
[SYSTEM] [*] Impersonating SYSTEM...
[DEBUG][SYSTEM] Check whether the symbolic link was really created in '\KnownDlls\'
[SYSTEM] [+] The symbolic link was successfully created: '\KnownDlls\EventAggregation.dll' -> '\KernelObjects\EventAggregation.dll'
[DEBUG][SYSTEM] Map our DLL to section '\KernelObjects\EventAggregation.dll'
[DEBUG][SYSTEM] Loaded payload DLL, image size: 664576 bytes
[DEBUG][SYSTEM] Found file for transaction: C:\Windows\system32\mfc140.dll
[DEBUG][SYSTEM] Opened file 'C:\Windows\system32\mfc140.dll' for transaction.
[DEBUG][SYSTEM] Wrote 664576 bytes of embedded payload DLL to transacted file.
[SYSTEM] [*] Mapped payload DLL to: '\KernelObjects\EventAggregation.dll'
[DEBUG][SYSTEM] Enable privilege SeAssignPrimaryTokenPrivilege
[DEBUG][SYSTEM] Create a primary token
[DEBUG][SYSTEM] Creating protected process with command line: C:\Windows\system32\services.exe aa34e445-7b2d-4680-85be-576f6af2578 -d
[SYSTEM] [*] Started protected process PID 6312, waiting...
[*] Trace Process started, press ctrl+c to stop...
[DEBUG] (DLL) DllMain (process attach)

after pressing CTRL+C the following output was given

[Korben Dallas] Setting Stop Event
[-] The DLL was successfully loaded into the PPL Process
[DEBUG][SYSTEM] Unmap section '\KernelObjects\EventAggregation.dll'...
[DEBUG][SYSTEM] Process exit code: 0
[+] Trace completed :)

Output of sysinternals DebugView64

00000001	0.00000000	[6312] [DEBUG] (DLL) DllMain (process attach)	
00000002	0.00076680	[6312] [DEBUG] (DLL) DEBUG mode enabled	
00000003	0.00103590	[6312] [DEBUG] (DLL) GUID='aa34e445-7b2d-4680-85be-576f6af2578'	
00000004	0.00140630	[6312] [DEBUG] (DLL) Object to delete: \KnownDlls\EventAggregation.dll	
00000005	0.00167800	[6312] [DEBUG] (DLL) NtOpenSymbolicLinkObject('\KnownDlls\EventAggregation.dll', WRITE_DAC) OK	
00000006	0.00185290	[6312] [DEBUG] (DLL) SetKernelObjectSecurity OK	
00000007	0.00206650	[6312] [DEBUG] (DLL) NtClose OK	
00000008	0.00226410	[6312] [DEBUG] (DLL) NtOpenSymbolicLinkObject('\KnownDlls\EventAggregation.dll', DELETE) OK	
00000009	0.00246180	[6312] [DEBUG] (DLL) NtMakeTemporaryObject OK	
00000010	0.00265990	[6312] [DEBUG] (DLL) [*] KnownDll entry 'EventAggregation.dll' removed.	
00000011	0.00285470	[6312] [DEBUG] (DLL) Hooking Main Entry to EXE	
00000012	0.00440480	[6312] [DEBUG] (DLL) [+] StartTracing: SUCCESS	
00000013	0.00466570	[6312] [DEBUG] (DLL) In hooked entrypoint, starting ETW Trace	
00000014	0.00507830	[6312] Session Name: Sealighter-Trace	
00000015	0.00542880	[6312] Outputs: event_log	
00000016	0.00572280	[6312] [DEBUG] (DLL) DllMain (thread attach)	
00000017	0.00592730	[6312] User Provider: {F4E1897C-BB5D-5668-F1D8-040F4D8DD344}	
00000018	0.00614790	[6312]     Trace Name: Microsoft-Windows-Threat-Intelligence	
00000019	0.00630860	[6312]     Keywords: All	
00000020	0.00667620	[6312]     No event filters	
00000021	0.00710980	[6312] Starting User Trace...	
00000022	0.00712180	[6312] -----------------------------------------	
00000023	0.00748660	[6312] [DEBUG] (DLL) Start Event Watcher Thread	
00000024	9.97445393	[6312] [DEBUG] (DLL) DllMain (thread attach)	
00000025	9.97730446	[6312] [DEBUG] (DLL) Was told to stop ETW Trace	
00000026	9.97807980	[6312] [DEBUG] (DLL) Stopping Sealighter	
00000027	9.97884369	[6312] [DEBUG] (DLL) DllMain (thread detach)	
00000028	9.97974968	[6312] [DEBUG] (DLL) Finished ETW Trace	
00000029	9.98050785	[6312] [DEBUG] (DLL) DllMain (process detach)	

Additional notes

Microsoft seems to have patched some bypass flaws in the meanwhile.
But this already happend in 2018, so I should not be the reason for this issue, but I am adding this note anyway, just in case it "may" be relavant after all.

"Status: fixed"

https://bugs.chromium.org/p/project-zero/issues/detail?id=1336

ETW Provider not generating any event

Hello, I'm trying to use Sealighter to get events (and filter them) from the following provider: {5BBB6C18-AA45-49B1-A15F-085F7ED0AA90} (for NTLM authentication).

So I created this config file:

{ "session_properties": { "session_name": "seatrace", "output_format": "stdout", "output_filename": "C:\\Users\\user\\Downloads\\sealighter\\output-ntlm.json" }, "user_traces": [ { "trace_name": "ntlm_trace", "provider_name": "{5BBB6C18-AA45-49B1-A15F-085F7ED0AA90}", "dump_raw_event": true } ] }

Using other providers everything works well, but with this in particular nothing pops out (only the two events of the start/end of the session).

The activity that should trigger events is, for example:

net use \\192.168.1.4 /user:test

Using logman like this:

logman create trace t -p "{5BBB6C18-AA45-49B1-A15F-085F7ED0AA90}" 0x0FFFFFFFFFFFFFFF 5 -ow out.etl

and doing exactly the same stuff, events are generated without any problem.

Since I have no problems with other providers, I really don't know why this happens.
Any guess?

License

Hi, thank you for this cool project. Just wondering about the license for it, could that be added?

No events for Microsoft-Windows-Security-Auditing

I'm curious to see if you have any ideas why this provider Microsoft-Windows-Security-Auditing aka EventLog-Security aka Security log won't work with sealighter. I don't see any events when running this config

{
"session_properties": {
"session_name": "My-Process-Trace",
"output_format": "event_log",
"buffering_timout_seconds": 10
},
"user_traces": [
{
"trace_name": "mystuff",
"provider_name": "Microsoft-Windows-Security-Auditing"
}
],
"kernel_traces": [ ]
}

I've tried, Microsoft-Windows-Security-Auditing, EventLog-Security, Security and {54849625-5478-4994-a5ba-3e3b0328c30d} none produces events. Suppling EventLog-Security or Security in the provider name just produces a "name provider error in the configuration" message. I'm running sealighter as system and I see the events i'm after fire in the security log, but my sealighter produces zip.

This little example works:
https://github.com/microsoft/krabsetw/blob/master/examples/ManagedExamples/UserTrace005.cs

Anyhow, I know this project isn't active, but I have found your creation very interesting in some of my research. I'm digging in trying to understand how you built sealighter and I just have to say thank you for the work you've done.

timestamp is a few hours less than the actual time

Call FileTimeToLocalFileTime before calling FileTimeToSystemTime to correct the time zone of FILETIME

std::string convert_filetime_string
(
const FILETIME from
)
{
SYSTEMTIME stime;
FILETIME localFileTime;
FileTimeToLocalFileTime(std::addressof(from), std::addressof(localFileTime));
::FileTimeToSystemTime(std::addressof(localFileTime), std::addressof(stime));
std::string to = convert_systemtime_string(stime);
return to;
}

Dangerous use of printf

printf(message);

The util functions log_messageA and log_messageW pass an arbitrary message string to printf/wprintf as format template.

I noticed this while using the Microsoft-Windows-WinINet provider, which crashes Sealighter when encountering percent-encoded data in an URL.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.