Git Product home page Git Product logo

uiforetw's Introduction

UIforETW is a user interface for recording ETW (Event Tracing for
Windows) traces, which allow amazingly deep investigations of
performance problems on Windows. Its goals include:
 - making recording ETW traces easy for non-developers
 - making it easy to record additional contextual data such as user
input and CPU temperature in order to make trace analysis easier
 - making trace management easier for developers
 - working around bugs in WPT (Windows Performance Toolkit)

Tutorials on how to use ETW to investigate performance problems on
Windows can be found here:
https://tinyurl.com/etwcentral

For specific details on this project see this post which includes
some documentation and an explanation for why this project was
created:
https://randomascii.wordpress.com/2015/04/14/uiforetw-windows-performance-made-easier/

UIforETW makes it much easier to control how traces are recorded than
using batch files or Microsoft's wprui. UIforETW also works around
numerous problems with ETW tracing (fixing symbol loading issues)
and adds extra features such as categorizing chrome processes,
monitoring working sets, etc.

UIforETW also lists all the recorded traces and displays editable notes
associated with each one.

UIforETW has some features that are specific to Chrome developers - but
it is fully functional for non-Chrome developers as well.

When adding TraceLogging and EventSource providers, UIforETW supports
the same *Name naming convention as other ETW tools. See
https://blogs.msdn.microsoft.com/dcook/2015/09/08/etw-provider-names-and-guids/
to ensure that your provider name and GUID conform to the convention.
EventSource providers should automatically match.

If you want to use UIforETW then you should download the latest
etwpackage.zip from the releases page at:
https://github.com/google/UIforETW/releases

If you want to build or modify UIforETW then you should clone the repo
from https://github.com/google/UIforETW.git and then build
UIforETW\UIforETW.sln.

Pull requests are welcomed. For information on contributing see the
CONTRIBUTING file. When writing commit messages try following the
general guidelines from here:
http://chris.beams.io/posts/git-commit/
Small pull requests are preferred - it's better to do several small
pull requests, each with a unifying theme - than to do one huge pull
request.

This is not an official Google project and is not supported by Google
in any way.

uiforetw's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

uiforetw's Issues

License?

There appear to be a variety of licenses in this code. An Apache 2.0 license is provided, and some files (like the ETWEventDemo files) include a Google-tagged Apache 2.0 header boilerplate. In looking through, I also noticed some MIT-licensed files, but that is not a problem for me.

However, I'm looking at using a custom ETW provider, and I see that there is the seemingly-intended-to-be-reused etwprof.h, which bears this header comment: Copyright (c) Cygnus Software, All rights reserved. Since it's in the specific file, I'd imagine it overrides the implication that the rest of the code is Apache 2.0, and that therefore I only have the rights granted by copyright law and the GitHub terms.

Could this be clarified and either a) called out, if intentional, or b) remedied by inclusion of appropriate and uniform license boilerplate in each file? I'd really like to use this for more than just the super-useful GUI, but I want to make sure I'm respecting the terms under which it is offered.

Need trace information in a tooltip

When looking through old traces I often want to know the OS, username, hardware specs, top CPU consumers, or similar pieces of information. All of this is available by loading the trace, but it would be great to have it in a tooltip or other display in UIforETW.

This information could be extracted from UIforETW. One possibility would be to store it in a somewhat-structured section at the beginning of the .txt files for each trace file. This section (perhaps all lines starting with a '#') would never be shown to the user but would be silently retained and used to quickly get this information.

The information could be extracted from traces in bulk or, optionally, immediately after a trace is recorded (as a background process).

Any thoughts on the format and design are appreciated. It's not a particularly difficult feature, but getting it wrong could make future development more difficult.

Error? Not sure

Whenever I start/stop a trace, it's trying to rename the Amcache.hve file but it's inadvertently failing to do so. I understand from the release notes that this is by design to reduce the startup delay with XPerf. I do have the admin permissions on the system but the file is in use by the "System" process preventing the UIforETW application to rename the file.

Below is the message in the log window.

Failed to rename C:\Windows\AppCompat\Programs\Amcache.hve to C:\Windows\AppCompat\Programs\Amcache_temp.hve

Zip compression needed

Currently the only supported form of trace compression in UIforETW is what is built in to Windows 8 and above. If you are recording or viewing traces on Windows 7 you are out of luck. Transparent .zip file compression would be very helpful. The idea would be to compress each trace into a .zip file with the same name and then delete the .etl file. The .zip file will still show up in the trace list (that code is there already), and UIforETW would need to automatically extract the trace on demand.

Traces could be automatically deleted when they haven't been referenced in a while (garbage collection) or they could be deleted whenever the compress-all-traces command is executed.

Need support 3rd profiles

Is there any plan to support additional profiles (or has supported?)? just like WPRUI's Add Profiles... function! That's quit convenient and necessary! thanks very very much!

edit: extra user mode providers maybe be good enough!!!

Handle leak (not our fault) in EnergyLib64

I don't think this is UIforETW's fault, but I can't figure out how to contact the Intel Power Gadget team (yet), and need to track it somewhere.

It appears that a HANDLE is leaked inside the power gadget library:

=======================================
VERIFIER STOP 0000000000000901: pid 0x4D38: A HANDLE was leaked. 

    0000000000000808 : Value of the leaked handle. Run !htrace <handle> to get additional information about the handle if handle tracing is enabled.
    0000000D6AC92360 : Address to the allocation stack trace. Run dps <address> to view the allocation stack.
    0000000D732BEFD0 : Address of the owner dll name. Run du <address> to read the dll name.
    00007FFA2AFA0000 : Base of the owner dll. Run .reload <dll_name> = <address> to reload the owner dll. Use 'lm' to get more information about the loaded and unloaded modules.


=======================================
This verifier stop is continuable.
After debugging it use `go' to continue.

=======================================

(4d38.68d4): Break instruction exception - code 80000003 (first chance)
vrfcore!VerifierStopMessageEx+0x6d0:
00007ffa`026d2190 cc              int     3
*** WARNING: Unable to verify checksum for UIforETW_devrel.exe
0:000> !htrace 0000000000000808 
--------------------------------------
Handle = 0x0000000000000808 - OPEN
Thread ID = 0x00000000000068d4, Process ID = 0x0000000000004d38

0x00007ffa429b3a4a: ntdll!NtCreateFile+0x000000000000000a
0x00007ffa02673e6f: vfbasics!AVrfpNtCreateFile+0x00000000000000ff
0x00007ffa3fd21e18: KERNELBASE!CreateFileInternal+0x0000000000000308
0x00007ffa3fd21af6: KERNELBASE!CreateFileW+0x0000000000000066
0x00007ffa0267438e: vfbasics!AVrfpCreateFileWCommon+0x0000000000000166
0x00007ffa0267449a: vfbasics!AVrfpKernelbaseCreateFileW+0x000000000000004a
0x00007ffa0267438e: vfbasics!AVrfpCreateFileWCommon+0x0000000000000166
0x00007ffa02674438: vfbasics!AVrfpKernel32CreateFileW+0x0000000000000058
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Program Files\Intel\Power Gadget 3.0\EnergyLib64.dll - 
0x00007ffa2afa4201: EnergyLib64!IntelEnergyLibInitialize+0x0000000000000081
0x00007ff7b19c2d6b: UIforETW_devrel!CPowerStatusMonitor::CPowerStatusMonitor+0x00000000000001ab
0x00007ff7b19cd220: UIforETW_devrel!CUIforETWDlg::CUIforETWDlg+0x00000000000002a0
0x00007ff7b19cc34a: UIforETW_devrel!CUIforETWApp::InitInstance+0x00000000000000ca
0x00007ffa09d500ae: mfc120u!AfxWinMain+0x0000000000000076

--------------------------------------
Parsed 0x997 stack traces.
Dumped 0x1 stack traces.
0:000> dps 0000000D6AC92360 
0000000d`6ac92360  0000000d`6ac50470
0000000d`6ac92368  00090000`00007801
0000000d`6ac92370  00007ffa`02674438 vfbasics!AVrfpKernel32CreateFileW+0x58
0000000d`6ac92378  00007ffa`2afa4201 EnergyLib64!IntelEnergyLibInitialize+0x81
0000000d`6ac92380  00007ff7`b19c2d6b UIforETW_devrel!CPowerStatusMonitor::CPowerStatusMonitor+0x1ab [c:\users\alexander riccio\documents\github\uiforetw\uiforetw\powerstatus.cpp @ 383]
0000000d`6ac92388  00007ff7`b19cd220 UIforETW_devrel!CUIforETWDlg::CUIforETWDlg+0x2a0 [c:\users\alexander riccio\documents\github\uiforetw\uiforetw\uiforetwdlg.cpp @ 87]
0000000d`6ac92390  00007ff7`b19cc34a UIforETW_devrel!CUIforETWApp::InitInstance+0xca [c:\users\alexander riccio\documents\github\uiforetw\uiforetw\uiforetw.cpp @ 212]
0000000d`6ac92398  00007ffa`09d500ae mfc120u!AfxWinMain+0x76 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\winmain.cpp @ 37]
0000000d`6ac923a0  00007ff7`b19da57a UIforETW_devrel!__tmainCRTStartup+0x162 [f:\dd\vctools\crt\crtw32\dllstuff\crtexe.c @ 618]
0000000d`6ac923a8  00007ffa`42142d92 KERNEL32!BaseThreadInitThunk+0x22
0000000d`6ac923b0  00007ffa`42929f64 ntdll!RtlUserThreadStart+0x34
0000000d`6ac923b8  00007ffa`09d500ae mfc120u!AfxWinMain+0x76 [f:\dd\vctools\vc7libs\ship\atlmfc\src\mfc\winmain.cpp @ 37]
0000000d`6ac923c0  00007ff7`b19da57a UIforETW_devrel!__tmainCRTStartup+0x162 [f:\dd\vctools\crt\crtw32\dllstuff\crtexe.c @ 618]
0000000d`6ac923c8  00007ffa`42142d92 KERNEL32!BaseThreadInitThunk+0x22
0000000d`6ac923d0  00007ffa`42929f64 ntdll!RtlUserThreadStart+0x34
0000000d`6ac923d8  00007ffa`42142d92 KERNEL32!BaseThreadInitThunk+0x22

Breaking at the creation point shows what the handle actually is for:

Breakpoint 1 hit
KERNELBASE!CreateFileW:
00007ffa`3fd21a90 4883ec58        sub     rsp,58h
0:000> r
rax=0000000000000000 rbx=0000000000000003 rcx=00007ffa2ca59880
rdx=00000000c0000000 rsi=0000000000000000 rdi=00000000c0000000
rip=00007ffa3fd21a90 rsp=000000011345de78 rbp=00007ffa2ca59880
 r8=0000000000000000  r9=0000000000000000 r10=00007ffa0268c570
r11=000000011345e058 r12=0000000000000000 r13=0000000000000000
r14=00007ffa3fd21a90 r15=0000000000000000
iopl=0         nv up ei pl nz na pe nc
cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202
KERNELBASE!CreateFileW:
00007ffa`3fd21a90 4883ec58        sub     rsp,58h
0:000> du 00007ffa2ca59880
00007ffa`2ca59880  "\\.\EnergyDriver"

I found this while doing some unrelated refactoring, and, well... grrr.

Xperf - Error 193 and The Trace File does not exist error

Whenever I start tracing, I see a couple of errors being logged with Xperf and I'm not able to save the trace buffers either which reports an error "The Trace File does not exist error". Below is the log from UIforETW.

UIforETW Version:
1.41

OS:
Windows 7 SP1 Professional


wevtutil.exe um "C:\Program Files\UIforETW\bin\etwproviders.man"
wevtutil.exe im "C:\Program Files\UIforETW\bin\etwproviders.man" /mf:"C:\Users\TOM1.GRE\AppData\Local\Temp\ETWProviders.dll" /rf:"C:\Users\TOM1.GRE\AppData\Local\Temp\ETWProviders.dll"

Starting tracing to in-memory circular buffers...
xperf.exe -start "NT Kernel Logger" -on Latency+POWER+DISPATCHER+DISK_IO_INIT+FILE_IO+FILE_IO_INIT+VIRT_ALLOC+MEMINFO -stackwalk PROFILE+CSWITCH+READYTHREAD -buffersize 1024 -minbuffers 600 -maxbuffers 600 -buffering -start UIforETWSession -on Microsoft-Windows-Win32k:0xfdffffffefffffff+Multi-MAIN+Multi-FrameRate+Multi-Input+Multi-Worker+Microsoft-Windows-Kernel-Memory:0xE0 -buffersize 1024 -minbuffers 100 -maxbuffers 100 -buffering
Error 193 starting C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\xperf.exe, xperf.exe -start "NT Kernel Logger" -on Latency+POWER+DISPATCHER+DISK_IO_INIT+FILE_IO+FILE_IO_INIT+VIRT_ALLOC+MEMINFO -stackwalk PROFILE+CSWITCH+READYTHREAD -buffersize 1024 -minbuffers 600 -maxbuffers 600 -buffering -start UIforETWSession -on Microsoft-Windows-Win32k:0xfdffffffefffffff+Multi-MAIN+Multi-FrameRate+Multi-Input+Multi-Worker+Microsoft-Windows-Kernel-Memory:0xE0 -buffersize 1024 -minbuffers 100 -maxbuffers 100 -buffering
Error 193 starting C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\xperf.exe, xperf.exe -start "NT Kernel Logger" -on Latency+POWER+DISPATCHER+DISK_IO_INIT+FILE_IO+FILE_IO_INIT+VIRT_ALLOC+MEMINFO -stackwalk PROFILE+CSWITCH+READYTHREAD -buffersize 1024 -minbuffers 600 -maxbuffers 600 -buffering -start UIforETWSession -on Microsoft-Windows-Win32k:0xfdffffffefffffff+Multi-MAIN+Multi-FrameRate+Multi-Input+Multi-Worker+Microsoft-Windows-Kernel-Memory:0xE0 -buffersize 1024 -minbuffers 100 -maxbuffers 100 -buffering

Tracing is started.
xperf.exe -capturestate UIforETWSession Microsoft-Windows-Win32k:0xfdffffffefffffff+Multi-MAIN+Multi-FrameRate+Multi-Input+Multi-Worker+Microsoft-Windows-Kernel-Memory:0xE0

Saving trace to disk...
xperf.exe -flush "NT Kernel Logger" -f "C:\Users\tom1.GRE\AppData\Local\Temp\kernel.etl" -flush UIforETWSession -f "C:\Users\TOM1.GRE\AppData\Local\Temp\user.etl"
Trace save took 0.0 s
Merging trace...
xperf.exe -merge "C:\Users\TOM1.GRE\AppData\Local\Temp\kernel.etl" "C:\Users\TOM1.GRE\AppData\Local\Temp\user.etl" "E:\etwtraces\2016-08-22_12-25-55_tom.green.etl"
Trace merge took 0.0 s
Stripping Chrome symbols - this may take a while...
python.exe -u "C:\Program Files\UIforETW\bin\StripChromeSymbols.py" "E:\etwtraces\2016-08-22_12-25-55_tom.green.etl"
Stripping Chrome symbols took 0.0 s
Preprocessing trace to identify Chrome processes...
python.exe -u "C:\Program Files\UIforETW\bin\IdentifyChromeProcesses.py" "E:\etwtraces\2016-08-22_12-25-55_tom.green.etl"
Finished recording trace.

Stopping tracing...
xperf.exe -stop UIforETWSession -stop "NT Kernel Logger"
Tracing stopped.

Using system default settings or custom settings for fonts

I'm using a preview build of Win10
I have a TV that is hooked up to a laptop, The TV is smaller than 32" and larger than the laptop's 15", 720p. The font is not large enough for me to read easily at the distance I sit from the TV.
There are system font sizes at Control Panel\All Control Panel Items/Display It would be nice if Microsoft provided more suggestions than the items listed there, but that's another problem.

Flamegraph viewer should be detached from UIforETW

If you use the trace list context menu to view a flame graph (Scripts-> Create flame graph) then the flame graph viewer may end up starting as a child process of the Python script, which means that UIforETW ends up hung until you close the viewer. The viewer process should be explicitly disconnected.

This happened when using IE as an SVG viewer. It probably depends on whether or not IE was previously active.

Max length of MSR name?

I think we're pretty safe passing a 1024-character buffer to GetMsrName, but what's the actual maximum length for an MSR name?

I couldn't find it in the Intel developers manual - although I very well could have missed it. That manual is huge.

It wouldn't bother me if GetMsrName accepted the destination buffer's size, but right now I see no guarantee that GetMsrName won't overrun the buffer. This applies to every other application that uses the power gadget library, so it's important.

flamegrapth.exe creates an empty .flamegrapth.txt

Running flamegrapth.exe on a trace:

> c:\tools\UIforETW\bin\flame_graph.exe --trace 2016-08-25_08-17-45_ew.etl --process_name ssh-agent.exe
Converting trace file to CSV format.
Reading trace events.
Generating flame graph.
Wrote flame graph data in file 2016-08-25_08-17-45_ew.etl.flamegraph.txt

The written CSV is 300 MB in size, the 2016-08-25_08-17-45_ew.etl.flamegraph.txt is empty. The CSV file does contain ssh-agent stacks:

            CSwitch,      19092,    ssh-agent.exe (29588),      24056,    5,   -1,         129,        0,             Idle (   0),          0,    0,   -1,         Running,        Executive,  NonSwap,    129,   4,   4,          0,    0,    0
            CSwitch,      19094,             Idle (   0),          0,    0,   -1,           0,        0,   SourceTree.exe (25860),      18684,   10,   -1,         Waiting,       WrResource,  NonSwap,    155,   2,   5,   80418816,    0,    0
        ReadyThread,      19095,   SourceTree.exe (25860),      21952,   SourceTree.exe (25860),      18684,       Unwait,               1,   3,      
              Stack,      19095,      21952,   1, 0xfffff802bbd9e0fd,     ntoskrnl.exe! ?? ::FNODOBFM::`string'
              Stack,      19095,      21952,   2, 0xfffff802bbccb235,     ntoskrnl.exe!ExpReleaseResourceForThreadLite
              Stack,      19095,      21952,   3, 0xfffff802bbcca554,     ntoskrnl.exe!ExReleaseResourceAndLeavePriorityRegion
              Stack,      19095,      21952,   4, 0xfffff960bec6d942,   win32kbase.sys!0xfffff960bec6d942
              Stack,      19095,      21952,   5, 0xfffff802bbd591a3,     ntoskrnl.exe!KiSystemServiceCopyEnd

              Stack,      19092,      24056,   1, 0xfffff802bbd539af,     ntoskrnl.exe!SwapContext
              Stack,      19092,      24056,   2, 0xfffff802bbd533d6,     ntoskrnl.exe!KiSwapContext
              Stack,      19092,      24056,   3, 0xfffff802bbcdb80a,     ntoskrnl.exe!KiSwapThread
              Stack,      19092,      24056,   4, 0xfffff802bbcdb299,     ntoskrnl.exe!KiCommitThreadWait
              Stack,      19092,      24056,   5, 0xfffff802bbcdaf05,     ntoskrnl.exe!KeWaitForSingleObject
              Stack,      19092,      24056,   6, 0xfffff80119d41297,         NTFS.sys!NtfsNonCachedIo
              Stack,      19092,      24056,   7, 0xfffff80119d435cf,         NTFS.sys!NtfsCommonRead
              Stack,      19092,      24056,   8, 0xfffff80119d428f2,         NTFS.sys!NtfsFsdRead
              Stack,      19092,      24056,   9, 0xfffff80118d57895,       FLTMGR.SYS!FltpLegacyProcessingAfterPreCallbacksCompleted
              Stack,      19092,      24056,  10, 0xfffff80118d55816,       FLTMGR.SYS!FltpDispatch
              Stack,      19092,      24056,  11, 0xfffff802bbc90b2d,     ntoskrnl.exe!IoPageRead
              Stack,      19092,      24056,  12, 0xfffff802bbc8f024,     ntoskrnl.exe!MiIssueHardFaultIo
              Stack,      19092,      24056,  13, 0xfffff802bbc8e3d2,     ntoskrnl.exe!MiIssueHardFault
              Stack,      19092,      24056,  14, 0xfffff802bbca9823,     ntoskrnl.exe!MmAccessFault
              Stack,      19092,      24056,  15, 0xfffff802bbd57bbc,     ntoskrnl.exe!KiPageFault
              Stack,      19092,      24056,  16, 0xfffff802bbc79c2a,     ntoskrnl.exe!RtlpLookupFunctionEntryForStackWalks
              Stack,      19092,      24056,  17, 0xfffff802bbc77f7c,     ntoskrnl.exe!RtlpWalkFrameChain
              Stack,      19092,      24056,  18, 0xfffff802bbc77ddf,     ntoskrnl.exe!RtlWalkFrameChain
              Stack,      19092,      24056,  19, 0xfffff802bbe17781,     ntoskrnl.exe!EtwpTraceStackWalk
              Stack,      19092,      24056,  20, 0xfffff802bbe175c5,     ntoskrnl.exe!EtwpStackWalkApc
              Stack,      19092,      24056,  21, 0xfffff802bbcdd195,     ntoskrnl.exe!KiDeliverApc
              Stack,      19092,      24056,  22, 0xfffff802bbd51b33,     ntoskrnl.exe!KiApcInterrupt
              Stack,      19092,      24056,  23, 0xfffff802bbcd1399,     ntoskrnl.exe!KiExitDispatcher
              Stack,      19092,      24056,  24, 0xfffff802bbcd0df3,     ntoskrnl.exe!KeReleaseMutant
              Stack,      19092,      24056,  25, 0xfffff802bbffd826,     ntoskrnl.exe!NtReleaseMutant
              Stack,      19092,      24056,  26, 0xfffff802bbd591a3,     ntoskrnl.exe!KiSystemServiceCopyEnd

Running flamegraph.exe without process_name writes 3 MB .flamegraph.txt.

VS 2015

Should UIforETW be upgraded to VS 2015? The free VS 2015 community edition can build UIforETW and the VC++ 2015 compiler has additional C99 and C++ features that may make development easier, such as %zd, constexpr, etc.

Allow Win10 Build 10586 WPT on Win7 again

The Th2 version Build10586 of the Win10 WPT now works fine with Windows 7 and includes the required events.

In CUIforETWDlg::OnInitDialog you don't need to exclude Win7 users any longer from the Win10 Th2 version.

I see an issue

How do I contribute an idea/fix without doing a coding fix?

StripChromeSymbols.py does not function under python 3, ONLY when launched from UIforETW

This ticket is partly so that I can track the issue.

StripChromeSymbols.py, as called from CUIforETWDlg::StripChromeSymbols fails under Python 3.

The failing output is:

> xperf -i "C:\Users\Alexander Riccio\Documents\etwtraces\2015-04-23_02-28-21_AlexanderRiccio.etl" -tle -tti -a symcache -dbgid
Traceback (most recent call last):
  File "C:\Users\Alexander Riccio\Documents\GitHub\UIforETW\UIforETW\..\bin\StripChromeSymbols.py", line 177, in <module>
    main( )
  File "C:\Users\Alexander Riccio\Documents\GitHub\UIforETW\UIforETW\..\bin\StripChromeSymbols.py", line 104, in main
    for line in os.popen(command).readlines():
  File "C:\Python34\lib\os.py", line 943, in popen
    bufsize=buffering)
  File "C:\Python34\lib\subprocess.py", line 823, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "C:\Python34\lib\subprocess.py", line 1005, in _get_handles
    p2cread = _winapi.GetStdHandle(_winapi.STD_INPUT_HANDLE)
OSError: [WinError 6] The handle is invalid

I've confirmed that the script runs properly on it's own - I commented child.Run out, and pasted the command into a separate command prompt, and the script ran correctly - but when run programmatically from UIforETW, it fails.

I also tried replacing StripChromeSymbols.py with a version that did nothing other than try to call "echo \"text n crap\"" via subprocess.getoutput, and that too failed from UIforETW.

This behavior is not documented anywhere, and is certainly surprising.

My best guess is that it has something to do with HANDLE inheritance.

Better 32-bit support

"On a ponderously slow 32 bit machine I tried to get a trace, but UIforETW32 reported:

Starting tracing to in-memory circular buffers...
xperf: error: UIforETWSession: Not enough storage is available to process this command. (0x8).
Error starting tracing. Try stopping tracing and then starting it again?
Process exit code was 80070008 (2147942408)

It's a 32 bit Windows 10 install on a machine that was upgraded from Vista to 7 to 10. (I suggested that maybe reinstalling as x64 to get the benefit of the 4G of ram in the machine would be worthwhile, but he said he'd rather just buy a whole new computer if he was going to go to all that bother.)

Anyway, I guess it was probably just out of contiguous memory? Maybe the 32.exe could use a smaller buffer, if that's possible to configure."

Even on a 64-bit OS it is possible to not have enough memory for tracing. Ideally the requested memory would be lowered in that case rather than failing.

Using system default settings or custom settings for fonts

I'm using a preview build of Win10
I have a TV that is hooked up to a laptop, The TV is smaller than 32" and larger than the laptop's 15", 720p. The font is not large enough for me to read easily at the distance I sit from the TV.
There are system font sizes at Control Panel\All Control Panel Items/Display It would be nice if Microsoft provided more suggestions than the items listed there, but that's another problem.

Adding "Randomascii Exclusive (module and function)" to analysis causes WPA to crash

When using the custom WPA default settings (via "Copy startup profile"), if I add Randomascii Exclusive (module and function) (under Computation, CPU Usage (Sampled)), and then Load Symbols, WPA will crash when loading the symbols.

It also happens if I Load Symbols, and then add Randomascii Exclusive (module and function).

Sometimes, it happens without Load Symbols, though I get a stacktrace when it crashes this way.

The crash is caused by an access violation in managed code, which means Application Verifier doesn't really help much.

Demonstration: http://youtu.be/J8IfCzKq4WE

the result .etl is too big

my application is a mmo game.it's easily grow up to few GBs just record few minutes.
i want just record the virtualalloc, i don't need the information about cpu usage.
can you add some options?
thanks.

ETWforUI not tracking already-running managed process

I'm trying to run traces on an already-running mixed managed/unmanaged code production application. But I'm not getting any ETL files. I can successfully get Perfview to give me ETL files. What should I check?

Don't hard code "C:" drive.

The program refuses to start up on my machine where the Windows drive is "W:" and not "C:" because of the hard coded "C:" at

wptDir_ = L"C:\\Program Files (x86)\\Windows Kits\\8.1\\Windows Performance Toolkit\\";

The code should really use SHGetKnownFolderPath().

Unit tests needed

Unit tests are needed, especially of the path and split functions and their edge conditions.

Open to optional .NET tracing/stacks?

I have this working in a proof of concept change, needs to add the check-box and some other changes before it would be "real". Before I put in any more effort would this be something you would be open to/interested in?

internal error on second build

Using VS2015 Update 2 RC compiling for Win32 or x64 with a release configuration I get an internal error the second time I build.

I can work around this by deleting bin\UIforETW_devrel.iobj between builds.

This issue does not occur with a debug configuration.

I suspect it's a compiler bug but I don't know the project well enough to submit it as such to Microsoft.

1>------ Build started: Project: UIforETW, Configuration: Release x64 ------
1> UIforETWDlg.cpp
1> Generating code
1>c:\code\uiforetw\uiforetw\workingset.cpp : fatal error C1001: An internal error has occurred in the compiler.
1> (compiler file 'f:\dd\vctools\compiler\utc\src\p2\main.c', line 249)
1> To work around this problem, try simplifying or changing the program near the locations listed above.
1> Please choose the Technical Support command on the Visual C++
1> Help menu, or open the Technical Support help file for more information
1> link!RaiseException()+0x48
1> link!RaiseException()+0x48
1> link!CxxThrowException()+0x65
1> link!std::_Xout_of_range()+0x1f
1> link!InvokeCompilerPass()+0x252c9
1> link!InvokeCompilerPass()+0x1b56e
1> link!InvokeCompilerPass()+0x22eec
1> link!InvokeCompilerPass()+0x2332e
1> link!InvokeCompilerPass()+0x232f9
1> link!InvokeCompilerPass()+0x233cb
1> link!InvokeCompilerPass()+0x22b04
1> link!InvokeCompilerPass()+0x22d86
1> link!DllGetC2Telemetry()+0x115837
1>
1>c:\code\uiforetw\uiforetw\workingset.cpp : fatal error C1001: An internal error has occurred in the compiler.
1> (compiler file 'f:\dd\vctools\compiler\utc\src\p2\main.c', line 249)
1> To work around this problem, try simplifying or changing the program near the locations listed above.
1> Please choose the Technical Support command on the Visual C++
1> Help menu, or open the Technical Support help file for more information
1> link!RaiseException()+0x48
1> link!CxxThrowException()+0x65
1> link!std::_Xout_of_range()+0x1f
1> link!InvokeCompilerPass()+0x252c9
1> link!InvokeCompilerPass()+0x1b56e
1> link!InvokeCompilerPass()+0x22eec
1> link!InvokeCompilerPass()+0x2332e
1> link!InvokeCompilerPass()+0x232f9
1> link!InvokeCompilerPass()+0x233cb
1> link!InvokeCompilerPass()+0x22b04
1> link!InvokeCompilerPass()+0x22d86
1> link!DllGetC2Telemetry()+0x115837
1>
1>LINK : fatal error LNK1257: code generation failed
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

Tracing to disk is artificially limited

UIforETW intentionally limits the duration of a to-disk trace, to a fixed amount of time. This is useful for preventing accidental disk space exhaustion, but prevents tracing of long running, but low intensity, operations. This bit me while profiling a Disk Cleanup "scan", and rebuilding the Windows Search index.

There are a few things we can do, in order of code involved, from least to most:

  1. Divide the constant limit by the sampling frequency, and use that as a limit. (a few lines of code, here and there)
  2. Provide the user with a spinner/slider control to adjust the default limit. (an extra control in the settings dialog)
  3. Save as two, but stop early if the user runs low on disk space because of the trace. (moderate work, infrequent use, but it'd prevent the out-of-space condition that'll cause the entire system to misbehave)
  4. Heuristically determine how much is too much, given free disc space; given proportional trace file size vs. free space; given user activity; given planetary alignment; among other things. (lots of work, lots of code that'll be rarely used - but a small awesome factor)

The out of space condition I mentioned in 3 can be quite surprising to the user, as few programs properly handle it, and is a real possibility once the user can adjust the time limit.

An installer to install the MFC DLLs is needed

In order to keep the UIforETW binary small so that it can be occasionally checked in to github without bloating the repo it is built to use the DLL versions of the C run-time and MFC. However this means that it doesn't work on machines that don't have these DLLs installed. An installer that installs these DLLs could be checked in once and then used with multiple versions of UIforETW.

Use Microsoft-Windows-Kernel-Memory instead of scanning working sets

The Microsoft-Windows-Kernel-Memory records working set data more efficiently and more richly than UIforETW's scanning: "when used with Keyword 0x40 KERNEL_MEM_KEYWORD_MEMINFO_EX, Windows captures every 0.5s: Count, ProcessID, WorkingSetPageCount, CommitPageCount, VirtualSizeInPages, PrivateWorkingSetPageCount."

"Windows 8 (Build 9200) also supports KERNEL_MEM_KEYWORD_WS_SWAP (0x80). Win7 only supports KERNEL_MEM_KEYWORD_MEMINFO (keyword 0x20)..."

This was first suggested here:

16c2171#commitcomment-17227542

Previously when trying to use built-in ETW providers working-set information I found that WPA would not graph the data that was recorded but WPA graphs the Microsoft-Windows-Kernel-Memory data quite nicely. These flags should be used where supported, and if possible the existing scanning code should be turned off or removed.

%etwtracedir% in etw_cpuusage_longterm.bat should be quoted

These three lines:

@rem Make sure %etwtracedir% exists
@if exist %etwtracedir% goto TraceDirExists
@mkdir %etwtracedir%

should really be:

@rem Make sure %etwtracedir% exists
@if exist "%etwtracedir%" goto TraceDirExists
@mkdir "%etwtracedir%"

I'd submit a PR, but my local install of Git is acting up. Grr.

UIForETW doesn't detect depot_tools' python

While saving traces to disk I get:

"Preprocessing trace to identify Chrome processes...
Couldn't find python."

I have python, but like most Chrome Windows developers, it's depot_tools' which actually means I have python.bat (not python.exe itself) in my PATH.

Cheers,
Gab

Server Core 2008 Compat

We have a couple of systems where we've been running ETW tracing recently and our Server Core 2008 platform wasn't able to install UIforETW out-of-the-box and complained that oledlg.dll was required.

Screenshots?

Sometimes it can be hard to tell what's going on in a trace - and there's not always a good description available (for example, I tried to record a trace of the chrome "save file dialog" issue, and failed to explain what I recorded in the actual trace). What if we recorded a screenshot every second (or so)? ETWProvider could emit an ETW event that has the name of an image file, and perhaps we could link to it in WPA? That'd make I easier to get an overview of what's going on in the trace.

Video would be cool, but totally not worth the effort.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.