Configure packaging for .net 4.0 and 4.5

Look at https://confluence.jetbrains.com/display/TCD8/Working+with+Feature+Branches
And pull requests checking: http://blog.jetbrains.com/teamcity/2013/02/automatically-building-pull-requests-from-github-with-teamcity/

Migrate to standard .NET logging.

• Explore and investigate possibility of different libs and wrappers.
  • Logary
  • NLog
  • etc
• Create helper functions for tmp output files and folders creation.

Collect all grammars in one specific place. "By ref" usage in projects should be implemented instead of files copying.

• SQL
• Calc
• ExtCalc
• JSON
• All tests should be unified

Target repo: YC.GrammarZOO

WTF? Why should I specify git user name and password in more then one place to reset git repo?

1. .Fsx script
2. VCS root configuration
3. "Git set user" build step
4. ...

These parameters should be placed in one config.

Migrate to .net 4.5. Configure build 4.5.

Implement Unger parsing algorithm as backend of YC.

• Parsing Techniques - A Practical Guide
• Reconciling Unger’s parser as a top-down parser for CF grammars for experimental purposes

Some tests for frontends and generators are excluded from solutions. It is necessary to include them back, and review to fix strange dependencies.

• Remove Irony frontend
• Remove FParsec generator

Fix git problem in GLLBranch build.

Seems, short manual on wiki about TeamCity configuration will be useful.

• Investigate algorithms for syntax errors detection in string expressions.
• Implement "on-line" and postponed errors detection for ARNGLR.
• Compare performance of implemented algorithms.
• Implement errors notification and navigation in R# plug-in.
• Proof correctness of implemented algorithm.

1. Книга: Parsing Techniques - A Practical Guide - Dick Grune
2. Диплом.
3. RNGLR.
4. По ошибкам:

• Syntax error handling in scannerless generalized LR parsers (2007) by Ron Valkering
• Natural and Flexible Error Recovery for Generated Modular Language Environments
  MAARTJE DE JONGE, LENNART C. L. KATS, and EELCO VISSER,
• Finding Counterexamples from Parsing Conflicts Chinawat Isradisaikul Andrew C. Myers
• Providing Rapid Feedback in Generated Modular Language Environments Adding Error Recovery to Scannerless Generalized-LR Parsing Lennart C. L., Kats Maartje de Jonge, Emma Nilsson-Nyman, Eelco Visser
• GLR Parsing for Erroneous Input Stuart Malone, Sue Felshin

5. По включению

Support string expression analysis in C++ based on R# SDK 9 and common YC infrastructure.

Implement and test.

• F# code generation must be unified with RNGLR generator as far as tree for translation may be unified (just remove redudant nodes).
• Fix this bugs:
  • #148
  • #149
  • #150
• May be synchronized with #172

Merge ARNGLR into dev.

Investigate: http://fsharp.github.io/2014/09/19/fsharp-libraries.html

Fix #111 first!

In current implementation ParserInputGraph is FA without epsilon transitions. It should be transformed into DFA. It is necessary to simplify regular set parsing with GLL.

If the same ident tokens, but different FAs, will be union, we get incorrect calculation of semantics or another language.

If doesn't matter which FA is in token, you can't store FA in token.

Replace tuples which used for reductions representations with struct.

Merge dev into master and fix stable version of R# plugin.

Implement Earley parsing algorithm as YC backend.

• Parsing Techniques - A Practical Guide

Merge cfg into dev.

Create demo for JS string Eval to staged metaprogramming translation. It may be actual in order to communication with authors of next articles:

• Information Flow Analysis for a Dynamically Typed Functional Language with Staged Metaprogramming
• Position Paper: The Science of Boxing Analysing. Eval using Staged Metaprogramming

Investigate string analysis integration with online IDE.

• It may PHP + **SQL or any other combinations of popular language and embedded language.
• Create a list of the most popular IDEs
• Investigate possibility of integration (open API, standard API for extensions )
• Choose online IDE for extension implementation.

Implement YC IL to Grammar Flow Graph (GFG) converter and GFG visualizer. It should be YC backend. GFG description available here.

Investigate

A Graphical Model for Context-Free Grammar Parsing
Keshav Pingali, Gianfranco Bilardi

Possible available here.

Create web demo for SEL analysis for SQL.

Look at

• Issue #57
• Issue #6

@unstope , @VereshchaginaE , Updated version file (with incremented version) should be pushed to git after build.

Fix Master build. Seems we have problems with VCS root configuration or Git properties. For details look at the end of log: http://teamcity.codebetter.com/viewLog.html?buildId=180464&buildTypeId=YaccConstructor_Master&tab=buildLog

Создать надстройку над http://fslab.org/FSharp.Charting/ для использования в замерах и сравнении производительности. Хочется получить обёртку, которая специализирована для рисования графиков в областе замера произволительности, которая позволяет по имеющейся функции и генератору входных данных (или набору функций и генераторов) получить график(графики), пригодные не только для визуального анализа, но и публикации в статьях.

Ожидаемая функциональность:

• Отрисовка графика "на лету" по мере получения новых измерений
• Возможность логарифмического масштабирования осей
• упрощенеи создания легенды и подпичей осей/ графиков
• Сохраниение результатов
  • csv
  • Картинка
    • Растр (png, jpg, etc)
    • Вектор (svg, pdf, eps, etc)
• Сравнение с другими измерениями
  • Загрузка из csv (использовать CsvProvider)
  • Передача нескольких функций для измерения

http://teamcity.codebetter.com/viewLog.html?tab=buildLog&buildTypeId=YaccConstructor_YcCoreGeneralBuild&buildId=186323#_state=453&focus=14045

fatal: unable to access 'https://YcGeneralUser:*******@/': Failed connect to :443; No error

Seems that it is url for google repo.

Why CI configurations are so unstable?

Publish NuGet packages with R# plugins: T-SQL in C# and Calc in C#.

Investigate reasonability and possibility of migration to ProjectScaffold. If it is possible and reasonable then migrate.

Checklist. Necessary functions of infrastructure:

• Tests automation
• Integration with TeamCity
• Build automation
• Documentation generation and publishing

• Move BlockResizeArray from RNGLR to FSharpx.Collections.
• Implement standard set of functions:
  • map
  • iter
  • fold
  • init
  • zeroCreate
  • find
  • tryFind
  • filter
  • IEnumerable interface
• Implement tests:
  • map
  • iter
  • fold
  • init
  • zeroCreate
  • find
  • tryFind
  • filter
  • IEnumerable interface
• Implement performance tests. Compare with ResizeArray and Array.
• Create documentation.

Merge RegApproximation into dev and remove RegApproximation branch.

Implement grammar modularity for YC/Yard.
Article for inspiration: Modular grammar specification, Adrian Johnstone, Elizabeth Scott, Mark van den Brand. Sci. Comput. Program. 87: 23-43 (2014)

Merge GLL into dev

FSharpx.Core library has been replaced by FSharpx.Extras.
Need to change them in YC.SDK.sln, branch dev

Use Utils.SourceText for position representation in abstract lexing.

Rework TSQL.Tests. It is very strange project...

@unstope , it is necessary to add prefix "YC." to packages' names.

Seems that
v.unprocessedGssVertices.RemoveAll(fun _ -> true) |> ignore
is
v.unprocessedGssVertices.Clear()

verticesToRecalc.RemoveAll(fun _ -> true) |> ignore
can be simplified too.

Seems that PassingReductions for all Vertex can be stored in VInfo.

Merge abstractLexer into dev and remove abstractLexer branch.

Web UI for YC. Web-demo.
Technology: WebSharper

• Algorithms visualization

• Algorithms description

• Algorithms comparison

• Predefined examples

• Interactivity

  • Grammars changing/specification
  • Input specification

• Unify and move to one single repo with name YC.Web

  • Recursive autmata visualization. Input: grammar from editor or file, output: RA
  • #174
  • #175
  • YC.BioGraph
  • YC.GraphParsingDemo
  • YC.Bio.Web

• Remove osolete repos.

• Integration with Appharbor or other hosting service.

Implement fully-featured string embedded languages support for MSVS + ReSharper.
Features:

• syntax highlighting;
• matched pairs highlighting;
• errors navigation and highlighting (integration with task #24);
  • approximation graphs visualization and navigation from graph to code and back;
• JS, C# and C++ (implementation task #29 as subtask) support as host language;
• T-SQL and HTML as embedded language

It is good time to kill one of them!

• Merge YC.FST project into QuickGraph
  • Look at corresponded issue
• Implement optimal algorithm for FST compose
  • For example look at this article
• Reorganize YC.FST
• Performance comparison
  • Manually created FSTs
  • Calc lexer
  • TSQL lexer

@VereshchaginaE Execution of YC.YaccConstructor.exe failed with next message

D:\...\sbsql\SqlMigration\SqlParser\packages\YC.Core.0.3.0.1-On-branch--local\lib\net40>YC.YaccConstructor.0.3.0.1-On-branch--local.exe  -af -ag

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Mono.Addins, Version=1.0.0.0, Culture=neutral, PublicKeyToken=0738eb9f132ed756' or one of
its dependencies. The system cannot find the file specified.
   at <StartupCode$YC-YaccConstructor>.$YaccConstructor.Program.main@()

Seems, that mono.addins ddls are missed.

Create unified tool based on FsLex for classical lexers and lexers for regular set tokenization generation.

• FsLex and Abstract lexing generator and runtime unification.
  • Abstract lexing sources: \src\AbstractLexer.*
  • FsLex\Yacc: \src\FsLex* \src\FsYacc*
  • Replace FsLex usages with result of unification
• Full unicode support. Synchronize with task #63 .
• Case sensitive|insensitive switch for case sensitive|insensitive languages processing
• Recursive FA support.
• Flag for switch classical lexer or lexer for SEL generation.

Migrate to R# SDK v9.*

Try to remove strange assembly redirect from app.config files.

<dependentAssembly>
        <assemblyIdentity name="JetBrains.Platform.ReSharper.Util" publicKeyToken="1010a0d8d6380325" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-8.1.555.*" newVersion="8.1.23.546" />
</dependentAssembly>

Today, 40 commits went into the dev branch, which queued 40 builds on codebetter CI, which have so far occupied both Jetbrains agents for about 6 hours, with about 1 hour to go.

For the future, would it be possible to change the build config so that each commit is not built separately? E.g. this would have reduced the build agent utilisation today from about 7 hours to about 10 minutes.