devosoft / pepper Goto Github PK

We have the power of python, we should be asserting against the objects and not the string representations of the objects

• Make unit tests

Detail: We can actually hand individual hunks of code to the parser to see what it does with them, and then assert against the object it hands back. This would be useful for testing individual lumps, and not just testing the output of the program.

requires evaluating truthyness of arithmetic expressions

also we have to parse/evaluate arithmetic expressions

Add subprocess'd tests to run the compiler against preprocessed code to make sure we're nailing all the things we need to nail to be an actual preprocessor

• find out how to disable preprocessing in whatever compilers we're testing
• make sure the compilers we're testing will run on Travis

This would probably matter more when we get to our milestone, but it would be good if we monitored performance so we don't accidentally triple our runtime at some point.

Should be fairly simple: either subprocess 'time' a bunch of runs, or use python's own time profiling monitors; run a few hundred rounds of the preprocessor, take the average and assert it's below a certian range.

We could make it CI-only so it doesn't clog the normal make test call.

https://gcc.gnu.org/onlinedocs/cpp/Diagnostics.html

pick your favorite CI system and set up automatic testing for pushes to all branches

diff-quality only checks the diff of the commits....against master, which is bad if you're committing directly to master.

https://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html#Variadic-Macros

Will likely require a short code freeze, since it'll modify the top of a lot of files and break git change tracking

iostream uses this:

#pragma GCC system_header

there's a lot of shift/reduce warnings coming out of the parser, should fix thise

• delete the parsetab.py file from the src/pepper folder
• run the preprocessor in debug mode on your favorite file
• resolve problems manually in the parser source

followup to #25, now we need to make function-like macros that can take arguments for their expansions (since these are the actually interesting macros).

This will require changes to the lexing and parsing structure, probably.

we have to re-scan expanded macro lines to detect if they contain additional macros, and expand them accordingly.

this should be (relatively) trivial after #17--we can now just take the output line and shove it back into the preprocessor if we expanded a macro in the last expansion.

Will add more complexity and side effects to the preprocessor module, should refactor later.

We keep running into problems where the test coverage is slightly below 100% generally due to impossibilities/weirdness in the coverage.

I've started ignoring the CI status, which is terrible, so I think I'm gonna lessen the coverage restriction to 95% or somesuch.

ex. this is valid

#if defined(M)
 test;
#elif 1
std::cout << "test " << std::endl;
#endif

https://gcc.gnu.org/onlinedocs/cpp/Concatenation.html#Concatenation

https://docs.pytest.org/en/3.0.0/xdist.html

useful if we keep adding subprocessed tests--these will always be slower than true unit tests.

rply is a cleaner interface to generating our parse tree.

• look further into differences between ply & rply
  rply repo

• implement mypy static checker into make test and TravisCI testrunner
• type as much of the library as can be reasonably typed
• update docs/procedures to include typing in code review?

• will enjoy enhanced autocomplete suggestions
• enforcing expectations on programs
• safer code

• develop checklist for release
  • develop acceptance tests
    • compiling stuff from Empirical...?
    • requires #28
• update install instructions to include package
• implement versioneer responder and command line argument

use http://peterdowns.com/posts/first-time-with-pypi.html

For Pip, it looks like someone already snagged 'pepper':
https://pypi.python.org/pypi/pepper/0.2

We'll need a different package name to upload to pip.

@mercere99 @nahumj Opinions?

Currently there's a preprocess method on all the nodes that does nothing

Prereqs: Needs design of how to talk to a compiler

How much do we care?

• 'load this example output file'
• constants for the example output files
• make tests use subprocess.run instead of subprocess.popen

• parse structure is currently set up where it incorrectly labels expressions
  -i.e "#if x" is a statement and not a expression, the only expression is x, and the naming convention is incorrect
• find ways to condense safe_code_expr, code_expr, and valid_expr
• condense/fix whitespace around directives
• change abstract symbol tree to abstract syntax tree
• currently, if & if def has a stack of tuples where the value at index[0] has either the name of the macro for ifdef, and the if count number for the if directive. Needs to be refactored out

:P

to prevent smelly code from leaking in, set up a flake8 pre-push hook

• tokenize the forward slash "" and keep adding for expansions
• need to add that support for CFGs for valid_expressions to evaluate #if expressions

Currently if we define an Macro with empty parentheticals, it crashes when we try to call it
i.e.

#define M() 1

int main()
{
#if M()
std::cout <<"call to with parentheses" << std::endl;
#endif

    return 0;
}

returns pepper.symbol_table.PepperSyntaxError: Macro M expects args, but was given none.

but should be valid

why not to use SyntaxError and the diff between pepper errors

Currently CFG's fail when a #if for example is written with whitespace before the pound.

currently when we fatal error we use an exception, which will include a (useless) python stack trace. Instead, we should use a custom exception with an include trace or somesuch.

clang may be magical enough to allow us to integrate with it and pretend that we're a Real Compiler (tm)