We don't free blob values if they are stored in local arrays.  E.g. if we have 
a leaf function that accpets an array of blobs.  This leads to memory leaks.

E.g. see test 851

Swift/T coasters is functional, but the user experience could be greatly 
improved

* One-step start of the Coaster service through Turbine or Swift/T
* Better packaging - don't bundle separate Swift/K distro
* Improved configuration mechanism - file?

In Hemant's NFHEDM app and possibly one of Ketan's apps we are caching data and 
open file descriptors in user memory.  The user has no way to finalize these 
resources.  Add a shutdown hook.

The vanilla machine files, e.g. machine.vanilla-gcc.sh, rely on sniffing 
dependencies, e.g. stc, turbine, tclsh from the user's path.  Then genleaf 
makes certain assumptions about the layout of these components, e.g. the tcl 
layout is: $TCL_HOME/bin/tclsh8.6 $TCL_HOME/lib/libtcl8.6.so.

There are problems with this in several scenarios:
* The user does not have these components on their path.
* The user has the wrong version of the component on their path (e.g. a Tcl 
different to the one Turbine was compiled with)
* The user's system does not match the expected layout.  This is true for 
common distributions, including Ubuntu linux's Tcl packages.

I just ran into these issues trying to use genleaf on my system and gave up.  I 
don't think I can use it without reconfiguring Turbine to use a different Tcl 
or rewriting the script.

We've already got a mechanism to handle all this in the configure script: why 
not just use the information in turbine-build-config.sh?

There is a bug in the frontend of the compiler where variable definitions from 
imported modules are not visible in modules that imported them. I believe the 
issue is the order in which the files are compiled.  See test 061

I was thinking about how to automatically distribute Swift arrays/ADLB 
containers in such a way that we keep small containers on one server, but split 
larger containers once they grow.  I think we really want this to be 
transparent to ADLB clients (unlike the old distributed containers design).

We could use consistent hashing for this - every time the master server for a 
container reaches some threshold, increase the number of servers the container 
is split between.  Readers/writers could be forwarded to the correct server if 
they attempt to read on master.  This could potentially be a performance boost 
if clients were able to cache the number of shards of the container (with hash 
table/LRU) to avoid contacting the master.

I think there are some tricky aspects to it.  First is managing the migration.  
We don't want to block server/worker operations while it's in progress, but we 
could potentially just block reads/writes to that particular container while 
it's happening.

Second is reference counting.  I think we probably want to have the master be 
responsible for detecting when the refcount goes to zero... but we don't want 
all refcount increments/decrements going to the master (that would eliminate 
most of the perf/scalability benefits).  We need some kind of scheme.  Possibly 
when the master has many refcounts it could delegate some of them to each 
sub-container.  If it's own refcounts went to zero for a container, it would 
then have to try to gather additional refcounts from sub-containers.

It would be helpful in some situations to be able to link with a different 
command to the compile command.  Currently some build files assume that $(CC) 
is the linker command.

Basic enums in Swift.

Must be set before running swift-t, because by the time the settings script is 
sourced, turbine-config has already been sourced.  Fix this. 

Currently Aget/AMGet have a limitation that you can't have simultaneous 
requests for different work types - or more generally, that requests can't be 
matched out of order.

Should we close stdin in the Turbine startup script?  

Cf. exm-users 5/9/2014.

Currently there's not really much documentation about buffer sizes for ADLB_Get 
and ADLB_Put.  In practice it depends on the buffer provided to ADLB_Get being 
large enough to handle any task the user code calls ADLB_Put for.  If this is 
not true, a buffer overflow results.  We should probably modify the ADLB_Get 
signature to take an additional argument specifying the size of the provided 
buffer.

The right thing to do would be to only match it to tasks that are small enough. 
This is a minor pain and will require modifying the server task matching logic 
a bit.

A less nice solution would be to just say that the user code is required to 
post buffers big enough for any task and abort the program if we detect an 
issue.

Very rarely, tests 355 or 425 at O1 will fail, usually when run under valgrind. 
 The problem turns out to be an idle detection bug related to notifications.

The situation is:
- Given 3 servers, Master, A, and B
- Master is checking idleness
- A does remote subscribe to data on B
- Idle check request comes before B has a chance to reply
- A says (incorrectly) that it is idle
- At some point before or after B says it is idle

This is unlikely to happen in practice with any frequency.

system() as in Swift/K.  Interested? 

Currently the order is:

type the_new_type the_old_type;
typedef alias_for_old_type the_old_type;

In C the order is reversed:
typedef the_old_type alias_for_old_type;


Alternatives:
1. Do nothing
2. Swap them
3. Support both - infer which is meant
4. Detect C ordering, suggest user swaps it (either error or warning)

Alt 1:
Pros:
* Easy
* Backwards compatible with existing scripts
* Consistent with struct declarations: type <new type name> <definition of type>
   type the_new_type { ... } 
   type the_new_type the_old_type

Cons:
* Some ongoing confusion for developers familiar with C

Alt 2:
Pros:
* Consistent with C

Cons:
* Somewhat inconsistent with the struct declaration order
* Breaks backwards compatibility

Alt 3:
Pros:
* Whichever alternative user writes will work - no read error->edit->compile 
cycle required

Cons:
* Some minor implementation work
* More than one way to do same thing - overhead of documentation, inconsistency 
between user's code, etc
* Errors messages may be slightly less helpful, since we don't know which way 
was intended.

Alt 4:
Pros:
* Backwards compatibility/consistency (see Alt 1)
* No changes to language/docs

Cons:
* Some minor implementation work
* Requires C user to do another read warning/error -> edit -> compile cycle

The current API for interacting with local files from Tcl is crude and 
undocumented

We should fix this.

Detect circular dependencies leading to deadlock.

E.g. 
a = b;
b = c;
c = a;

or with conditionals

c = a + 2;
if (...) {
   a = b;
} else {
   a = c;
}

The basic strategy to do this would be to construct a dataflow graph and then 
check for cycles or dead-ends.  E.g. in a procedure there will be some set of 
variables A which start off with an initial value (including arguments, and 
variables which are initialised to a constant value).  If we follow all 
"depends-on" links from a variable v back through the graph, all paths should 
terminate at a variable in the set A.  If there is a cycle, or if the path 
terminates at an uninitialised variable, we have a deadlock.  There are a 
couple of complications I see:
* conditional statements - the value of the variable could come from one branch 
or another, so need to have some kind of OR operator (like the Phi operator 
from SSA) to represent this
* loops - I'm not sure what the implications of these are yet
* assigning to array members.  Sometimes we can work out exactly which array 
index is assigned, but in general we don't know, so we will have to have 
special treatment in the dataflow graph for the unknown array indices

We build nicely out of the box with no configuration if we're compiling against 
mpich. OpenMPI requires modifying MPI library names etc in turbine.  Should be 
able to build against system OpenMPI or compiled-from-source OpenMPI with the 
same minimal effort.

Add user documentation for checkpointing feature.

Currently these don't support parallel tasks.

As did OmpSs

The ADLB API lets you to provide an arbitrary array of work types, e.g. if you 
wanted work types 5,1,25,4.

This is mostly ignored by our implementation of ADLB, which assumes types are 
0,1,2,...,n.

I don't think there's that much value in supporting this remapping of types at 
the ADLB level, since there's a fixed number of types and the same thing can be 
just as well accomplished at the application level if the application really 
wants to use some alternative scheme for numbering types.  Maybe we should just 
remove the feature from the API?

The MPI_T interfaces should allow us to determine the unexpected message queue 
length.  This is useful to understand whether servers are congested.

There's an interaction between multiple worker types (issue 503) and worker 
services (issue 524).  Should the same service get started on all worker types 
uniformly, or should there be different options for different worker types?

The "default" behaviour that will occur if we don't modify anything is that the 
services will only get launched on CPU workers.

E.g. if we're in the ADLB_Store(id, ..) call, we should be able to pass a debug 
symbol in as an argument.  The compiler knows this info when it's generating 
the code, and we'd like to include this info in log/error messages.

The current syntax for foreign tcl functions is clunky and doesn't support all 
use cases well since it only supports a few fixed formats.  It also requires 
extending the syntax to add other types of foreign functions.  Something 
cleaner might be:

foreign tcl (int o) f (string x) {
  pkg = "mypkg";
  pkg_version = "0.0";
  func = "myfunc"
}

foreign tcl (int o) f (string x) {
  template = "set <<o>> <<x>>"
}

http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/projekt
e/vampirtrace

Release 0.8.0

Jonathan Ozik ran into an issue where he linked against the non-system Python 
during build but at runtime it picked up his system python.  I guess Python 
wasn't added to the rpath correctly.

We currently use a hash table to map from (id, subscript) to a list of 
notifications.  This works great for supporting lookup of a (id, subscript) 
given that we know the pair we're interested in.  However, it doesn't work for 
the following:

* Locating all subscripts for a ID (e.g. if ID is closed, and want to work out 
if any subscribes will never be fulfilled).
* Locating all subscripts that have (id, subscript) as a prefix.

A radix tree would work for this.

If user cannot write to out.tcl because of permissions, you get a Java stack. 

Our current intended model is that the user provides PROCS and PPN and we 
divide PROCS/PPN to obtain NODES.  SLURM does not currently follow this model. 

Using this syntax and finding it pretty concise but still redundant:

  file list<argv("list")>;

Wondering if this would be neat: 

main (file list, string name, int n) { ... }

This would automate the use of argv and type conversion, creating the 
appropriate local variables.

We need to define calling conventions for calling C functions with Swift/T 
data.  I'm enumerating some of the initial issues that I can think of.

+-------------------------+
| Function signatures     |
+-------------------------+
- Do we want to pass in any additional metadata arguments?  E.g. Swift/T state, 
MPI communicators, etc
- C input arguments match Swift/T input arguments
- How to support multiple outputs?
  -> Ideally single outputs can be handled as return value
- How to support varargs??

+-------------------------+
| Data types              |
+-------------------------+
Scalar/simple types
- numeric types -> native ADLB representation (int64_t, double, etc)
- strings -> null-terminated

Uninterpreted data
- blob -> struct? or two arguments?

Composite types
- Arrays (sparse) -> ???
- Structs -> ???

+-------------------------+
| Options                 |
+-------------------------+
1. Pass native ADLB representations into C function
2. Unpacked representations based (loosely?) on SWIG conventions
3. Unpacked representations not based on SWIG

xpt_init2 is hard-coded into turbine.tcl .

Ketan ran this and got a misleading MPI-related error message.  We should be 
able to handle this better. 

Try 
https://svn.ci.uchicago.edu/svn/vdl2/SwiftApps/swift-galaxy/swift/script.swift 
r7833

Try building fork executor. 

Ketan ran this and got a misleading MPI-related error message.  We should be 
able to handle this better. 

Try 
https://svn.ci.uchicago.edu/svn/vdl2/SwiftApps/swift-galaxy/swift/script.swift 
r7833

In place of issue 200.  Provide some sort of support to watch particular 
variables in user program.  More scalable/less noisy than logging everything.

http://google-opensource.blogspot.com/2015/03/farewell-to-google-code.html

On August 24, 2015 it's going read-only

The current syntax for foreign tcl functions is clunky and doesn't support all 
use cases well since it only supports a few fixed formats.  It also requires 
extending the syntax to add other types of foreign functions.  Something 
cleaner might be:

foreign tcl (int o) f (string x) {
  pkg = "mypkg";
  pkg_version = "0.0";
  func = "myfunc"
}

foreign tcl (int o) f (string x) {
  template = "set <<o>> <<x>>"
}

Use MCS machine Frisbee.  Swift/T has apparently regressed on the Mac.

Yadu, are you interested in this? 

Maybe we should have the option to configure (either globally or per-variable) 
the policy for placing data.  E.g. on the nearest server or randomly.

Thoughts, Justin?

We have:

type MyStruct {
  int x;
}

Justin suggested we should instead have:

struct MyStruct {
  int x;
}

Alternatives:
1. Remove/deprecated "type" support
2. Parallel support for both forms

Pros: 
* Familiarity for people coming from C

Cons:
* Adds a reserved keyword
* Another inconsistency with Swift/K to resolve
* (Alt 1) Breaks existing scripts
* (Alt 2) Leaves more than one way to do same thing - additional documentation 
and mental overhead for developers

We don't get rank numbers on the BG/Q - I am providing an option for ADLB to 
produce them. 

https://bitbucket.org/saws/saws/wiki/Home

Provide a method to implement third-party command line executors.

Should be able to ship a third-party executor as a Swift include + Tcl package. 
 Constraints are:

* Must be able to dynamically load as Tcl package (current design can do this)
* May have declarations in user script/header script (TODO)
* Must not require changes to Turbine Tcl/C code to add new executor
* Must not require changes to STC to add new executor

I think the third-party executor would need:
- to implement the turbine_executor interface, and register itself with Turbine 
when the package is loaded
- declare its existence in the Swift include and specify a "run" tcl proc that 
should be called to launch the task

As a test to confirm it's working, we should be able to replace any 
special-cased coaster logic in STC and Turbine with generic logic.

What needs more documentation?
Calling a CUDA function from Swift/T.

Preferably a file/function name
http://www.mcs.anl.gov/exm/local/guides/leaf.html