Suppose that when marking up a document I highlight DEF in the text ABCDEFGHI. When I go to preview the exported HTML file, only DE is highlighted.

This is going to be an absolute nightmare, and truthfully I'm only doing this for bugs that show up, and I'll only truly handle error recovery IF people actually use this application. The amount of work this will take is MASSIVE. So far, I just catch errors and terminate, but the REAL issue is determining WHICH exceptions are recoverable, HOW to recover from them, and which are not and what to do at that point. It wouldn't be worth it if no one ends up using it because exceptions really only occur when you do something very wrong. Defensive programming is great and all, but exceptions are unavoidable.

As I began to ponder, "how can I possibly describe the complex inner workings of the project", I began breaking things down until eventually it all fit within a very simple model... While remaining mostly functionally pure. It makes it easy to describe in the report as well.

There are many other details requiring planning but I can create the report while also planning it out. More updates to come!

I really can't be bothered to trawl through the unclear and partially undocumented tangle that is NetBeans' API, so I figured out an alternative way to implement syntax highlighting. This...

https://github.com/GeSHi/geshi-1.0

Note, I can't really rely on this. In fact, I don't want to rely on an internet connection at all. So what I will be doing is creating my own tables using this.

The table for going from extension to syntax highlighting is here...

https://github.com/GeSHi/geshi-1.0/blob/master/src/geshi.php#L1469

There are also a vast amount of files that contain the respective syntax highlighting...

https://github.com/GeSHi/geshi-1.0/tree/master/src/geshi

So what I will be doing is simply parsing the table out, creating an automated tool that will parse the syntax highlighting files into a more familiar format, JSON, and I can go from there. From there I can create a basic lexer, which should actually be fun and enjoyable.

Note: This isn't for the presentation layer that is exported (although technically it CAN if I wanted to, as relying on a library for presentation isn't the long-term solution) but rather for what is displayed in NetBeans. Syntax Highlighting is very important, and this may actually be fun. Can teach myself some nice tools too.

Refactoring

There is a MASSIVE refactor that is already planned, and NEEDS to be done before January 23rd. Quite a few of these pertain to making this repository "readable" to others, because right now it isn't manageable, even for me (and I wrote it). There is a lot of things still there that I inherited, a lot of things that I've experimented with a long time ago and did not remove, and a lot of things that need vast improvement. Plus, considering that any new requests will come in next semester, I need to ensure it is resilient to changes. I'll start doing these next week or week after latest.

The things that need doing are as follows...

Event Propagation

As defined in my End-Of-The-Semester, there is a much simpler model for this that can (potentially) improve the control flow and also ensure it is understandable to the outsider. Talk is cheap, but it definitely is planned out, and with RxJava, it can make it a LOT easier to reason about.

Documentation

After implementing the model, the next thing is the documentation model. I should assume the reader has no idea what RxJava is, and probably the same way for streams and functional programming in general, although I won't be describing what EVERYTHING is. Documentation for what each component does, what each event is for, what certain functional operators are doing, etc, are important, ESPECIALLY the more complex ones which aren't clear at a glance.

Future-Proofing

Since Drue Coles will most likely tell me when something is wrong with it or needs to be added, I need to make it as easy as possible to integrate new features, and soon.

Distribution

I need to actually distribute it on the plugin portal. I gotta hope that it works as is, but it seems to run fine on my machine so hopefully there are no potential issues.

After writing Haskell, the sheer amount of boilerplate code is making me almost physically ill. As well this project is getting boring and I'm not learning anything new, so gradually shifting to Scala would let me further my knowledge of functional programming, as well as let me add another language under my belt.

@dcoles-bloomu

The real transition will happen over winter break of course. Scala runs on the JVM and produces valid ByteCode and so I can swap out Java files for Scala piece by piece. Should keep it interesting.

• Implement a way to swap between files within a single project
• Implement a way to save the user's session for a project that can be later restored
• Implement a way to markup all files in a project (w/ filtering)
• Implement a way to have all markup files in the same hierarchy as the project to a .zip file

@dcoles-bloomu

Could you potentially create a checklist of all features you wish to see added? I can't guarantee they will all be done by the end of this semester, but should be by next semester.

Checklists look like this...

- [ ] Empty Checklist
- [x] Checked Checklist

Which would show...

• Empty Checklist
• Checked Checklist

I think this is for the best. It will handle the issue of syntax highlighting by allowing me to use Javascript and CSS, and it will look a lot better in the long run. As well, with the new model, it should be easy to change and extend MarkupView to have it's own tree.

One of the things I realize is that, unfortunately, there needs to be a platform-independent way to preserve global configurations. Even things like saving default highlight color has to be stored somewhere. The official plugin most likely can't rely on saving a simple properties file in the CLASSPATH since the CLASSPATH won't be deterministic. Hence, it will need to be saved somewhere on disk.

On Windows, the registry could be used. However, that will not be sufficient on Mac OS X or Linux, and so I need to think of a decent place to store them. I'm assuming NetBeans has a way to preserve data, but the data should be general enough to be exported.

@dcoles-bloomu

This is a potential question for next meeting. As well, I'm afraid I may have to say I'm going to fall a bit shorter again this week, due to having 1 2-hour Community Service, 2 Homework Assignments, and 3 exams all on a single week. Hence, I'll have to postpone session management until later, but that doesn't stop me from planning and design.

In particular, while plotting out session management quite a few things hit me as things that need to be addressed.

Session Management

I'm starting with the interfaces before implementation, and so far, there are two new core interfaces that should handle management of sessions.

SessionObject

SessionObject describes an object that needs to be preserved in some state or form. The two major ones are TemplateMarkup, the templates you spoke of, and Markup itself (previously referred to as Note).

TemplateMarkup

As there needs to be a way to not only add, modify, and remove existing templates (I.E: Create a project-specific template, joining multiple templates into one (I.E: Assignment-Specific vs Language-Specific vs Class-Specific, etc.). The issue of where to put this aside, the more prevalent question is: "How do I preserve certain templates?".

If you find that, for an assignment, there is a very specific mistake almost all students have made, you most likely will want to create a temporary template for that particular assignment. As well, you need to be able to have those temporary templates persist between user sessions, until deletion. The obvious approach would be allowing you to create your own unique category, say "Temporary" where you can store all temporary markups, and then allow you to delete them all at your leisure, pretty much having them be permanent, be delete-able.

Markup

All Markup objects, of course, for a project, must be preserved, and is the most crucial for allowing the instructor to come back later to markup a student's assignment without having to do it all over again.

SessionManager

A SessionManager is a manager for SessionObject objects. They simple handle keeping a reference to all of their SessionObject objects, and mediate all interaction between it and the outside world. The two prevalent ones are TemplateManager and MarkupManager.

TemplateManager

Keeps track of TemplateMarkup objects, and is responsible for preserving changes to created, modified, and deleted templates.

MarkupManager

Keeps track of MarkupManager objects, and is responsible for preserving any Markup objects created, modified, and deleted.

Problems

It's becoming clearer and clearer to me, just how much I miss Android's approach to Life Cycles, or to having Context objects in general. The issues come down to: "When do I save the user's session"?

If the user decides to exit out in some unnatural way, all progress is lost unless the session gets saved prematurely. Luckily, the NetBeans TopComponent has a method called componentClosed() which notify me when the user is going to close he component, giving me time to save the current session. With the help of Guava's EventBus I can send information to all subscribers telling them that its time to shut down.

This problem can be solved (with reliance on the NetBeans plugin API). SessionManager can be extended to force all managers to enforce serialization and deserialization on close.

After exporting the marked up documents, could there be some quick indication of success. Perhaps a message box -- I wouldn't mind if I had to click an OK button for each export. Or it could just be a message that appears and fades out in a second or two.

• HTML Syntax Highlighting for MarkupView
• HTML conversion of single file (with syntax highlighting)
• Make markups configurable to allow custom underline colors
• Make markups not underline whites space before and after newline (like in editor)
• Improve the note popup's look-and-feel

• Display templates in MarkupView

• Add category support for templates (Tree-Like structure)

• Implement selection handler for template and updating based on templates

• Allow user to specify their own '.json' file to import

• Allow user to modify the template (I.E, remove, add, or even create their own from scratch)

• Allow user to export the template (in the event they wish to export to another machine)

• Implement right-click context menu for TreeViewNode, each option with an ActionListener

Project-Markup and Session Data

Project-Structure and Serialization

Imagine a file-tree hierachy beginning at the root Project_Name, which contain many sub-directories and even files. For simplicity, I will denote (and document)
the process of how I envision a project to look like.

Legend:
    [N]: A directory containing N children (either other directories, dubbed sub-directory, or valid source files)
    (N): A file containing N markups

Project_Name
    |-> Directory [3]
    |       |-> SubDirectory0 [1]
    |       |       |-> File1 (1)
    |       |-> SubDirectory1 [0]
    |       |-> File2 (2)
    |-> File3 (0)

Note that some files may contain 0 markups, or there may be some sub-directories without valid files or even be empty. This must be taken into account when printing all to HTML.
Imagine that only sub-directories containing .java files are considered valid. Now lets show how the manager should encapsulate all valid marked-up files.

Legend:
    "String_Key" -> [Note0..Note(N-1)]: A mapping of a key to it's appropriate markups (given the prefix Note).

Manager: [
    "Project_Name/Directory/SubDirectory/File1" -> [Note0],
    "Project_Name/Directory/File2" -> [Note0, Note1]
]

Note that it will append a '/' after each directory (or sub-directory) it finds. Now the above should be serialized to the following .json file (although this will be better formatted for readability).

{
    "project_name": "Project_Name",
    "managers": [
        {
            "key":"Project_Name/Directory/SubDirectory0/File1",
            "notes": [
                {
                    "message":"Note0's Message",
                    "id":"Note0",
                    "color":"#0000FF",
                    "bounds": [
                        {"start":"250", "end":"275"},
                        {"start":"290", "end":"310"},
                        {"start":"320", "end":"321"}
                    ]
                }
            ]
        },
        {
            "key":"Project_Name/Directory/File2",
            "notes": [
                {
                    "message":"Note0's Message",
                    "id":"Note0",
                    "color":"#FF0000",
                    "bounds": [
                        {"start":"550", "end":"585"},
                        {"start":"590", "end":"610"},
                        {"start":"620", "end":"621"}
                    ]
                },
                {
                    "message":"Note1's Message",
                    "id":"Note1",
                    "color":"#FF0000",
                    "bounds": [
                        {"start":"775", "end":"800"},
                        {"start":"810", "end":"850"},
                        {"start":"860", "end":"861"}
                    ]
                }
            ]
        }
    ]
}

Note that separating it this way allows Notes to have the same id, since they are held by different managers.

There are 2 occassions that need to be taken into account. The structure of the project COULD change, and it must be
flexible enough to handle any such changes, intentional or not.

What happens when we remove a file that was previously marked up? The manager will still be created from the session data, however since we locate the mapped
markups of each file by searching by it's key, it will merely go unused. This does produce a memory leak in the sense that the manager will be wasting resources,
but it's not too much of a deal.

Note that this applies to adding new files (same as if it had not been marked up) and should be handled appropriately. One thing that cannot be handled is changes in
the actual file itself. That's something of an invariant: The file must not be changed after it's original markup.

Project to HTML

This should be relatively simple compared to composing the structure of the project itself. With the above example, it should produce the appropriate .html files in the
appropriate directories.

Project_Name
    |-> Directory
    |       |-> SubDirectory0
    |       |       |-> File1.html
    |       |-> SubDirectory1
    |       |-> File2.html
    |-> File3.html
    |-> Essentials
            |-> highlight.pack.js
            |-> styles.css

Note that it should produce empty directories even if they do not contain any .html files. This is a design decision (and possibly a bad one), as I'm hoping to emulate the
look-and-feel of a project as much as possible. Also note that the Essentials folder will contain the library used to handle syntax highlighting so that it will not
create a dependency on always being online. All of this should be archived into a .zip file. Again, its meant to be as convenient and user-friendly as possible.

The process of marking up all files in a project is similar to marking up a single file, in that it will invoke the same handler to generate the HTML for each file, and if
there exists a manager for that given file it will inject it into the correct locations, otherwise it will simply be HTML with syntax highlighting.

Potential things to add to presentation...

1. How to create template from scratch. Something I definitely cannot overlook is how to create said templates that I mention in the speech.Templates must be in JSON and be in the correct format. However they can use another language like YAML and then convert later into JSON. As well find a way to convert from XML to JSON. Atom is a solid tool to demonstrate.

2. Go over how I handle syntax highlighting. This is a rather significant point in my presentation, so may as well demonstrate how it's done and the many ways I've tried doing so before in NetBeans ( further explains why I switched over ). Especially how it is usable offline now with CSS embedded into page.

Efficient Code Evaluation Tool

• Advancement in Technology calls for increase in automation
• Software Engineers bring about creation and usage of automated tool
• Instructors in charge of breeding new software engineers need automation too
• Code-Glosser is a tool assists in automation of mass grading and giving feedback
• Supports 169 languages
• Allows application or templates
• Allows easy marking up of user code.