charmplusplus / projections Goto Github PK

Original issue: https://charm.cs.illinois.edu/redmine/issues/515

I received an in-person request for vector graphics output (svg preferred, but we might as well support things like pdf and eps) from Projections for ease of including them in papers and allowing things like lossless zoom. Additionally, the ability to use a white background should be made more clear, as that is more amenable to inclusion in papers.

Original author: Nils Deppe (@nilsdeppe)
Original issue: https://charm.cs.illinois.edu/redmine/issues/1524

Because of the structure of our code, we have only one entry method which then forwards to the correct member function for the data it received. I can trace all the user functions with Charm++ and they show up in Timelines. However, it would be very valuable to us if we could see them in the Time Profile as well. More generally, if bracketed user events could get promoted to being visible everywhere where entry methods are, say by being able to hide/show them like in Timelines that would be really awesome.

Original author: Yanhua Sun
Original issue: https://charm.cs.illinois.edu/redmine/issues/455

The logs are here
/fastscratch/Yanhua/logs_apoa1

Select 8 timesteps and display in usage profile, check for example PE 7, the utilization is wrong
Extreme of least idle is totally messed up.

Original author: Keshav Santhanam
Original issue: https://charm.cs.illinois.edu/redmine/issues/976

No body.

Original author: Maxx Macica
Original issue: https://charm.cs.illinois.edu/redmine/issues/523

Call Table has some problems, including:

• The main body of the panel isn't actually added to the panel...
• The "Change Colors" button does nothing
• The tool uses AWT instead of Swing for Panels, etc. It should look more like the rest of Projections
• The "Close" button gets cut off when the textArea is added to the panel
• and possibly more.

Original author: Yanhua Sun
Original issue: https://charm.cs.illinois.edu/redmine/issues/506

Currently, when we open timeline by default the time range is from 0 to maximum end time. However, for a lot of projections logs, only a small portion has real data.
Please refer to this log set,/dcsdata/home/jessie/PPL/NAMD/benchmarks/vesta/prj_root_2048_2. Actually this is true for most NAMD logs.

Instead, in timeline open dialog, we should be able to only pick the time range with valid data. (Begin computation, end computation) time range. This should be relatively easy to implement

Original issue: https://charm.cs.illinois.edu/redmine/issues/1004

Sanjay requested a view that would show the total number of messages in certain sized buckets and that would elide empty spaces between messages.

Histogram does this to some degree, but I think the intent is to make it a bit more sophisticated.

Original author: Zhengqi Yang
Original issue: https://charm.cs.illinois.edu/redmine/issues/497

In the main window, if open up "Tools" menu, and then use "Enter" button to open a menu item, the OK button in the pop-up window will be clicked(triggered) automatically if there is one. (Tested with a few menu items)

Original author: Jim Phillips (@jcphill)
Original issue: https://charm.cs.illinois.edu/redmine/issues/699

When tracing messages back, any message generated by a non-entry such as a periodic callback triggers tracing back from the previous entry on that PE. Tracing should stop at the periodic call, or potentially trace back to the entry that created the periodic callback.

Original author: Nils Deppe (@nilsdeppe)
Original issue: https://charm.cs.illinois.edu/redmine/issues/1525

It would be really useful to be able to pass a vector<double> to the traceUserBracketedEvent function to record various numbers during runtime. These could, for example, be PAPI counters. It might make sense to ASSERT the size of all vectors for a specific event number are the same, but it could be considered a feature to let them be different.

This feature would be even more useful if analysis of the numerical values stored was possible, say average, outlier, max, min, etc.

Original issue: https://charm.cs.illinois.edu/redmine/issues/995

Having 'Color by user supplied parameter' in the Timeline tool is quite useful. To make corresponding displays from other tools, it would be useful to have that option elsewhere, too.

I'll open sub-issues for the cases I'm most interested in.

Original issue: https://charm.cs.illinois.edu/redmine/issues/1780

Currently Projections shows chare array element indices as 3 ints, even though up to 6D arrays are supported.

Issue for the problem in Charm's tracing: https://charm.cs.illinois.edu/redmine/issues/1694

Original author: Lukasz Wesolowski
Original issue: https://charm.cs.illinois.edu/redmine/issues/505

Given that we normally assign a communication thread in Charm++ to a separate core, it would be useful to be able to visualize the utilization of this core to determine whether it is under- or overutilized. This could help, for example, when determining how many comm threads are optimal for a particular run.

We should be able to show comm thread utilization in the usage profile view.

Original author: Yanhua Sun
Original issue: https://charm.cs.illinois.edu/redmine/issues/435

I do not know whether others have this problem. I found that after we added SDAG_RTS entry
in Charm, the time profile , usage profile in Projections can not display correctly.

In timeline, the SDAG_RTS seems not much time. However, when using time profile, it is totally messed up. All overhead time goes away. Instead, it is all SDAG_RTS time. This is wrong.

The projections data can be found here

/dcsdata/home/jessie/PPL/git/benchmarks/leanmd/prj_root

Original issue: https://charm.cs.illinois.edu/redmine/issues/386

When I load a coarse-resolution histogram (200 bins, 100 bytes each) that covers my entire run's set of message sizes, all of the counts are graphed correctly. However, if I set some finer resolutions, some of the data is omitted from the subsequent display entirely. This could be extremely misleading!

With the attached log, a setting of 105 bins, 50 bytes wide, and starting bin size 1000 renders the complete data set. Halving the width and doubling the bin count (210 bins, 25 bytes each) truncates the data above the midpoint of the display. If I further refine the requested display (e.g. 700 bins, 16 bytes wide), nearly all of the data are apparently discarded - I'm left with just three bars out of the dozens I expected.

Original issue: https://charm.cs.illinois.edu/redmine/issues/703

For the timeline view, when a non-default color scheme (e.g. color by frequency) is chosen, doing manual color selection does not change the color of entry points on the timeline.

Original issue: https://charm.cs.illinois.edu/redmine/issues/794

Make the didactic projections logs publicly available and create a freestanding tutorial page for people to learn how to use Projections by themselves.

Original issue: https://charm.cs.illinois.edu/redmine/issues/414

Extrema Analysis shows overhead where Usage Profile/Timeline shows idle time.

This can be reproduced with:
/sharedscratch/harshitha/cosmomslb200knodelbck9/
time range 104.2 s to 105.19
load 0-31743:1024 pes

Original issue: https://charm.cs.illinois.edu/redmine/issues/997

No body.

Original issue: https://charm.cs.illinois.edu/redmine/issues/1283

I compiled leanmd with +tracemode summary and ran it with +sumDetail. On opening the leanmd.sum.sts file with projections visualization tool and selecting usage profile, End Time in the range selection is shown as 2441000.0000000001 s, which is invalid.
On selecting that invalid range (or a valid range like 0 - 2.4 ms), a blank usage profile is shown.

Original issue: https://charm.cs.illinois.edu/redmine/issues/1170

Currently, when logs numbered beyond the number specified in the sts file exist in a folder, Projections attempts to load more and gives the exception:

Exception in thread "Thread-8" java.lang.RuntimeException: The sts file only specifies 4 PEs, but you are trying somehow to load pe 5
	at projections.Tools.Timeline.Data.getData(Data.java:924)
	at projections.Tools.Timeline.TimelineRunnableFileReader.run(TimelineRunnableFileReader.java:15)

There's no reason to even attempt to load these.

Original issue: https://charm.cs.illinois.edu/redmine/issues/618

Some indication of the number of available tasks on each PE (i.e. for each PE, a graph of message queue size vs. time) would be useful to identify situations where a PE is starved or overworked.

Original author: Yanhua Sun
Original issue: https://charm.cs.illinois.edu/redmine/issues/459

In projections, if you click on a entry method which is inline or local, it can not be traced back. This is due to the nested begin_processing entries
in the log.

We want to trace it to its parent entry method.

Original author: Maxx Macica
Original issue: https://charm.cs.illinois.edu/redmine/issues/514

Right now there are many different files that are used to read and process logs. We should make it more unified and simpler.

Original issue: https://charm.cs.illinois.edu/redmine/issues/440

If the user selects a large number of processes or large time intervals on large logs, display some warning to ensure users don't unknowingly overstress their machines.

Original issue: https://charm.cs.illinois.edu/redmine/issues/991

By default, the Usage Profile view (and most Projections views) are sorted by ascending PE index. For the sake of improving the ease of analysis, it should be sortable on other properties, such as the amount of time spent in a particular EP on each PE. This can be extended to an arbitrary degree, allowing sorting on different aspects of the data, such as sorting on total utilization, etc.

Original issue: https://charm.cs.illinois.edu/redmine/issues/415

Profile and improve the performance of projections. Look into using hardware acceleration for drawing.

Original issue: https://charm.cs.illinois.edu/redmine/issues/438

In the communication vs. time graph, the external received bytes count is higher than the total received bytes count. This is clearly fallacious.

Original issue: https://charm.cs.illinois.edu/redmine/issues/996

No body.

Original issue: https://charm.cs.illinois.edu/redmine/issues/1884

When I manually insert traceBegin and traceEnd calls into my application to trace say a few iterations out of many, then open Projections on those traces, Projections will display the entire time period that the application ran by default. But only a small sliver of that is stuff that I actually care about. It would be nice to by default start the time range at the first traceBegin call and end it at the last traceEnd call.

Original issue: https://charm.cs.illinois.edu/redmine/issues/256

Opening this so that the work is visible, both to external users and for internal workload balancing.

Original issue: https://charm.cs.illinois.edu/redmine/issues/483

Overview is dropping many entry methods from its display making cores appear to be idle when they are not. These appear correctly in timeline.

For example see:
~/work/w256_4k.tracered.rhor1.t22

If you examine the interval 15.123s-16.884s (saved) for cores 2664 to 2689 (as shown in attached screen shots) the overview looks wildly different from the compact timeline.

Original issue: https://charm.cs.illinois.edu/redmine/issues/517

From the Usage Profile view, Tools -> Change Colors does not do anything. It should open a new window that allows the user to select new colors (cf. operation from the Time Profile graph view)

Original issue: https://charm.cs.illinois.edu/redmine/issues/436

The majority of people who use extrema analysis use the "least idle time" view. Change the GUI so that this is the default choice.

Original author: Nikhil Jain
Original issue: https://charm.cs.illinois.edu/redmine/issues/280

The Y axis title for processor utilization graph is not displayed correctly.

Original issue: https://charm.cs.illinois.edu/redmine/issues/1293

EPExecTime has a large number of zeros.

To reproduce this issue:

1. Compile charm with enable-tracing.
   ./build ChaNGa netlrts-linux-x86_64 smp --enable-tracing --with-production -j4

2. After downloading ChaNGa, change its makefile to add -tracemode summary to LDFLAGS.

3. Point ChaNGa to charm
   export CHARM_DIR=/dcsdata/home/nbhat4/Work/software/changa/charm

4. Compile ChaNGa
   make -j4

5. Place the input file ./snapshot1M.param in the same directory as ChaNGa. I've stored the input files (snapshot1M .param and .std) in /scratch/nitin/changa/inputFiles/ on charity).

6. Run changa on one core (with or without +sumDetail)
   ./charmrun +p1 ++ppn 1 ./ChaNGa -n 1 -p 2 -b 32 +bench +traceroot ./proj_summ_logs_32/ +sumDetail ./snapshot1M.param +pemap 1 +commap 0 > changa_output

7. Open the traceroot directory and try opening the ChaNGa.0.sum file and look at the EPExecTime row. (A lot of entry methods have their values as zero).

Original author: Osman Sarood
Original issue: https://charm.cs.illinois.edu/redmine/issues/258

The message sizes only represent the 112 bytes header if pack unpack for messages is used. Parameter marshalling works correctly.
Example:

1. charm/examples/ring: Currently the example uses pack unpack methods for custom Msg type. Trace-projections fails to log the correct message size. The example currently uses a group. However, the behavior is the same for arrays.
2. If you change the same code to use parameter marshalling instead of custom message, the message size is shown correctly.

Original issue: https://charm.cs.illinois.edu/redmine/issues/1229

When scrolling around in timeline, performance can be quite bad, leading to a great deal of lag and a poor user experience.

Original issue: https://charm.cs.illinois.edu/redmine/issues/924

Will attach screenshot momentarily.

Original issue: https://charm.cs.illinois.edu/redmine/issues/1021

Try loading the projections logs in prudence:/scratch/prateek/temper-projections-mar24/pr1 using the current git head of projections. Now try to open a timeline.

Nothing will be displayed, as there is a chain of errors.

Works fine using the old 07142013 binary distribution. Which is a tragedy in that it includes no changes made in the past 2 years.

On a related note, 2 years seems like a long time to go between releases.

Original issue: https://charm.cs.illinois.edu/redmine/issues/457

Trace to sender, as invoked by the right click pulldown menu, fails to load for OpenAtom traces for nearly all entry methods. Failure error message is "Message was sent from outside the current time range". However, that is a clear misdiagnosis. When you have loaded an entire time step and examine methods from phases late in the time step, there is no way for the sender to have come from outside the step.

See ~bohm/work/w256_4k.t4 for example traces. It has a saved interval which matches a complete timestep. You can load a few timelines (0-3) and try to backtrace any method. The only back trace invocations which succeeded for me are for messages which where sender PE=receiver PE.

Original issue: https://charm.cs.illinois.edu/redmine/issues/439

When in histogram views, choosing one interval shows more than one bar when it shouldn't.

Original author: Nikhil Jain
Original issue: https://charm.cs.illinois.edu/redmine/issues/21

Java based 3D view in projections is buggy - fix it.

Original issue: https://charm.cs.illinois.edu/redmine/issues/1230

When loading files into Timeline (and later), there's a lot of memory usage. For some logs, e.g. prudence:/scratch/nitin/charity_proj_logs/1, the memory usage is high enough that they take unreasonably long to load. This can probably be made more efficient.

Original issue: https://charm.cs.illinois.edu/redmine/issues/1057

It would be useful to be able to use projections just for user event tracing. For very finegrained applications like ROSS, tracing every event can be expensive and doesn't necessarily add much. The more interesting part is tracing individual phases with user events, which can only be done when the full trace infrastructure is turned on.

Original author: Jim Phillips (@jcphill)
Original issue: https://charm.cs.illinois.edu/redmine/issues/698

When communication thread tracing is enabled, the "trace messages forward" feature only traces off-node messages to the sending node's communication thread. Tracing forward should cascade until the message arrives at the actual destination.

Original issue: https://charm.cs.illinois.edu/redmine/issues/1271

The README is out of date in at least its build instructions, maybe other things

Original author: Yanhua Sun
Original issue: https://charm.cs.illinois.edu/redmine/issues/507

Now, we can save time range to history and to disk so that next time when we load we can directly use them. Similarly for large scale run, it is useful to save interesting processor data set.

Original author: Maxx Macica
Original issue: https://charm.cs.illinois.edu/redmine/issues/521

When converting a time to a humanReadableString, the rounding causes us to stop reading our log files early. For example, look at dcsdata/jessie/PPL/git/benchmarks/leanmd/prj_root/. The actual latest time of all the log files is 7610759, but the endTimeField in the range dialog box sets the end time to 7.61s. As a result, the end computation signal is never reached when processing the logs and we lose some of our data.

Original issue: https://charm.cs.illinois.edu/redmine/issues/518

When attempting to load a file on the command line with a path that contains a space, the parser splits on the space.

For example:
ronak`machine:~$ projections ~/Test\ Dir/test.sts
Internal Error: Path [/home/ronak/Dir] supplied for file detection is not a directory! Please report to developers!