solomono / kiama Goto Github PK

In the rewriting module we use a home-grown technique for creating a clone of a 
term with some 
or all of the children changed.  It should be possible to use the new Scala 
2.8.0 copy methods for 
case classes to achieve this in a more standard fashion.

Move the test utility functions like expect, isFail etc into the util package 
from the test module and 
avoid duplication in multiple places.

For many purposes combinator-based parsers are not ideal since details like 
tree construction need 
to be specified manually.  Hence, we want to also support some methods for 
getting tree data into 
Kiama  code using generated parsers.  Rats! seems like a good choice since it 
has the expressibility 
of parsing expression grammars plus good facilities for modular grammars.

The current Rats! generates Java parsers which we can call from Scala, but 
difficulties are created if 
we want to build Scala trees.

Work up an example for this and see what happens.  I think the idea is that you 
need to provide the 
Product interface on your non-Product classes and then everything is OK.

We already have test that can check that a processor produces the right output 
when given a 
particular input. We also need the next step.  For examples where the input is 
processed to an 
executable form, we also need tests that run the compiled program on inputs and 
make sure that 
the outputs are correct.  This would be used in the Oberon0 and ISWIM examples.

At the moment we use an identity hash map to Unit, but could probably subclass 
the hash sets or 
something with an identity comparison.

This is also now relevant for uncached based attributes which need an identity 
set to detect 
circularity but are currently using a hashmap.

Congruences in Stratego allow you to write things like App (d, e) to pattern 
match on App's, apply d 
to the first component and e to the second component, failing if either d or e 
fail.

Add a generic congruence combinator that gets us as far as we can to having 
this notation for 
Kiama.

Replace dodgy ant-based build process with one using sbt

The current lambda2 implements some of the strategies in 

Dolstra, E., and Visser, E. Building interpreters with rewriting strategies. In 
Proceedings of the 2nd 
Workshop on Language Descriptions, Tools and Applications (2002), vol. 65 of 
Electronic Notes in 
Theoretical Computer Science, pp. 57–76.

but we would like to do them all.

http://www.program-transformation.org/Sts/TILChairmarks

We originally used failure because Stratego's fail clashed with JUnit's fail.  
But failure clashes with 
the Scala parser library's failure, which is used more in this kind of code. 
So, since we don't really 
use JUint's fail and we are moving away from direct use of  JUnit hopefully, 
switch back to fail.

Generate list of all expressions obtained by incrementing or decrementing any 
single literal in an 
original expression.

Just try to see if there are any obvious bottlenecks on the example code. Can 
we use something like 
VisualVM?

Much of this is working and is ready to go, but we are writing larger specs 
using it to test the library 
in real applications.  Some enhancements are pending the results of that work. 
We also need to 
write some documentation.

There is no need for Kiama to have a packrat parsing library when the standard 
Scala library has 
one, so remove our version and move the tests over to use the new one. Note any 
capabilities that 
are lacking and report to Scala team.

Add interface to allow the cache of specific cached attributes to be turned off 
dynamically

We have some tracing code that can be enabled in the rewriting module, but it's 
not everywhere it 
probably should be.  Do we really need it?  How much is it slowing us down?

Do de do

For example, see the following for an idea of the kind of feature we want.

Boyland, J. T. Descriptional Composition of Compiler Components. PhD thesis, 
University of Cali- 
fornia, Berkeley, Sept. 1996. Available as technical report UCB//CSD-96-916.

Magnusson, E., Ekman, T., and Hedin, G. Extending attribute grammars with 
collection attributes - 
evaluation and applications. In Proceedings of the Seventh IEEE International 
Working Conference on 
Source Code Analysis and Manipulation (2007), IEEE Press.

At the moment the classpath refers to each of the JARs in lib_managed, but this 
is fragile since the 
exact jars in there will vary depending on the transitive dependencies of the 
libraries that we use 
directly.  If possible, make the .classpath file just refer to the directory so 
we don't have to update it 
all the time.

For some projects we would like to be able to process trees that come from 
tools such as SDF. This requires us to be able to do rewrites etc on ATerms.  
At present we are looking at a typed implementation that translates a signature 
in Scala case classes and provides typed readers and writers for the ATerm 
representation.

At the moment if you want to use things like childAttr you need to make your 
tree nodes inherit 
from Attributable, which makes them Products etc.  A more flexible situation 
would be to make the 
interface needed by things like childAttr into an interface that has an 
implementation from 
Attributable if you are using Products, but you could define it yourself if you 
want.

Each example should have command(s) that show how to run the example in a built 
system.  We 
should update these to work in the new sbt world.

Write wiki pages for each example.

The Oberon0 example has only been lightly tested.  We should have a 
comprehensive test suite to 
make sure that more complex things like procedure calls, arrays, records are 
working.

Once 2.8 is out we need to port the library to it.  Apart from minor changes 
due to language 
updates, we expect work to be needed on collection-related code mostly.  Also, 
we plan at this 
point to remove the Kiama parsing library and switch over to use the Scala 2.8 
library since it will 
now also have packrat parsers.

What steps will reproduce the problem?
1. Build the project
2. Look at what's in the main jar :)

What is the expected output? What do you see instead?

The examples shouldn't be in the distributed jar; they have lots of classes 
that have similar or identical names to users' AST nodes.

What version of the product are you using? On what operating system?
1.0.0.a1 (trunk as of a couple days ago)

Along the lines of the existing page on parsing combinators.

This seems to be the only thing in Attributable that is not related to 
structural properties.  Or to put 
it another way, if you are not using structural properties you only really want 
this from that module.  
So, we should put it in Attribution somewhere so that Attributable could be 
ignored in some cases.

Stratego's dynamic rewrite rules are ones that can be modified as the rewriting 
proceeds and are conventionally used to implement context-sensitive rewriters 
such as that needed for name analysis. We tend to implement these kinds of 
things using attribution because it is more natural, but we have some work that 
needs to do things in the Stratego way.

See CRAG paper for improvements to basic algorithm that we are currently using.

At some stage we would like to be only dependent directly on Scala libraries, 
not on any java.* stuff.

The dup method in the Rewriter uses reflection to find the primary constructor 
of a term class so 
that it can create a new term with the constructor but possibly different 
children. We could cache 
these which might usefully limit the amount of reflection that is done. Do some 
measurements to 
see if this helps in typical cases.  Is there some non-performance impacting 
way to make it 
optional?

See 
http://swerl.tudelft.nl/twiki/pub/Main/TechnicalReports/TUD-SERG-2008-038.pdf 
for 
information on the kind of thing we plan to add.

Need an overview along the lines of the parsing combinator page that already 
exists.

At the moment you have to specify the name of the main class exactly, whereas 
it would be nice to 
support completion as in the test-only command which completes over test 
classes.

Add 'nowhere present' check, eg for DoubleNeg example, but others too, intadds 
is ok, do that for 
the others, also uniquevars should check for no duplicates in vars, or collect 
in list and check that 
they are sorted "var" ones.

For example, in the rewriting library. This would mean moving the declaration 
of ==> into a more 
central place from the Attribution module.

At the moment we have packages starting with "kiama", whereas it probably 
should be "org.kiama" 
or "com.googlecode.kiama" or something.

In some cases we use objects to encapsulate reusable code and import it where 
needed. In others 
we use traits and extend them to get the reuse. Both approaches usually work. 
The trait one results 
in different instances of the reused entities in each context where they trait 
is extended. We need to 
come up with a principled way of deciding which approach is appropriate for a 
given situation and 
apply it across Kiama.

Check for correctness and ease of running on "real" programs

At present, rewriting strategies are untyped much as in Stratego.  We are 
prevented from 
constructing ill-typed terms by Scala's static typing, but there are currently 
no constraints on 
combinations of strategies.  Adding type parameters along the lines of function 
types should be 
possible.

Now that the testing framework can do execution tests, we should have some in 
the ISWIM 
example.  There are some coded up in tests/SECDTests.scala, but there should be 
some file-to-file 
ones as well.  See the Oberon0 ExecutionTests for examples of how to do this.

It would be nice to have command editing etc. Try JLine?

Examples such as Oberon0 do not attach positions to AST nodes at present so all 
reports refer to 
0,0.  We should use positioned to correctly position all nodes.

The last release on Scala 2.7 with changes since 0.8.

To further reduce our dependencies, we should try to replace our plain tests 
that are written in 
JUint style with ScalaTest style ones.

General tidy up

- make sure everything has something
- check markup on code refs in scaladoc
- use more complex markup, such as @param decls, @see links?

Weak maps might help garbage collection performance. Since we must use an 
identity hash map, 
we need to find an implementation of a weak identity hash map.

This also needs to fix the ones in the circular attributes, including the sets 
computed and visited, 
which are maps at the moment.

From Lennart: Only the JCL provides a weak hash map implementation, and it 
doesn't allow 
overriding of the equality/hash key getting mechanism. So to use weak hash maps 
everywhere, 
we'd need an IdentityEquality wrapper object (similar to the ArgAttributeKey 
class). Of course, this 
would make all this even more expensive. Alternatively, a brand new 
IdentityWeakHashMap 
implementation could work.

We have an implementation of a compiler for a cut-down version of Obr that we 
use in teaching.  
We should consider adding it to the examples since it does full (albeit) simple 
semantic analysis and 
code generation for a real language.