fiveham / repeated_phrases Goto Github PK

IO.ENCODING exists, which suggests that sometimes some other value may be the wisest choice for some reason.

Currently, various different pieces of information are connected but have to be connected forcibly so that efficient access from one piece to its connected pieces requires the construction of, for example, Maps from pieces of information of the first type to a list of those of the second type.

Access from one piece of connected information to all pieces connected to it can be organized as a graph where each node can report a list (or lists) of the pieces of information to which it connects.

FileBox maps from Chapter (via filename) to a list of Quotes (phrase-at-Location).
PhraseBox maps from raw phrase (text) to a list of Locations.

files should be read as html novels if and only if they have the names prescribed in HTMLFile.BookData:

AGOT.html
ACOK.html
etc.

• HTMLFile should be split into one class for html books and another class for html chapters, and
• html books should keep track of whether a given operation step has been performed on them so that that step can be skipped if the method for that operation is called again.

If a new Trail, for instance, is specified, linked chapters, for example, linked with a previously used Trail, are returned, saved, etc., instead of that being done with chapters modified to use the data of the new Trail.

Managed data should be lazily created after two checks:

1. If the member is null, or
2. if the inputs for the previous cached result are different than the current inputs

Currently, only the first check is used.

LINK_CHAPTERS says it writes to Folder.LINKED_CHAPTERS but the method that does the operation actually overwrites the original html files from Folder.HTML_CHAPTERS rather than writing to a purpose-specific folder.

Either

• the method used by LINK_CHAPTERS should write its output to a purpose-specific folder, or
• the writeTo folder for LINK_CHAPTERS should be changed to HTML_CHAPTERS, in which case,
  ** the readFrom for SET_TRAIL should be set to HTML_CHAPTERS

For example, the concept of a phrase paired with a location could be called a Quote, while a phrase independent of a location is simply "phrase". It may be a good idea to call the un-located phrase something other than "phrase" since the purpose behind this renaming is to resolve ambiguity surrounding uses of the word "phrase".

This could be resolved by

• integrating the operation steps into one another so that data such as a collection of Chapters can be passed from one operation to the next, or
• integrating the operations and a RepeatedPhrasesApp instance into each other so that data such as a collection of Chapters can be passed from the RepeatedPhrasesApp instance to each operation as the operation demands

Should use an ifless technique instead, such as having a member for each HtmlBook that's assigned in the constructor, which has one of its methods called which chapterizes an arbitrary HtmlBook according to the member's own chapterization rules.

For example

enum Chapterizer{
    AS_NOVEL( /*process for a novel*/ ), 
    AS_NOVELLA( /*process for non-PQ novellas*/ ), 
    AS_PQ( /*process for PQ*/ );
    private final Function<HtmlBook, Collection<HtmlChapter>> func;
    private Chapterizer(Function<...> func){
        this.func = func;
    }

    static Chapterizer get(HtmlBook book){
        /* chooses the right enum element for a book */
    }
    
    static Collection<HtmlChapter> split(HtmlBook h){
        return func.apply(h);
    }
}

public class HtmlBook{
    private final Chapterizer chapterizer;
    public HtmlBook(...){
        ...
        this.chapterizer = Chapterizer.get(this);
    }
    ....
    public Collection<HtmlChapter> splitChapters(...){
        return chapterizer.split(this);
    }
}

Some methods with a header that's one line leave a gap before the first line of code. A single blank line should be left there only if the header is multi-line.

Only the largest phrase size will need to find its next position after each batch of words gets returned. When endpoints update, the value for the longest phrase moves on to be the endpoint for the second longest phrase, and so the data cascades down.

A map from size to endpoint will be needed. An array or an ArrayList could do it well.

The code that uses PhraseProducer would also need to be able to handle getting all the phrases at once like that.

That would keep the methods that split books into chapters separate from the methods that pertain to html chapters.

Ideally it should be possible to call RepeatedPhrasesApp from the command line with some options in order to get results, rather than requiring the use of the GUI.

In addition to script-friendly uses of options in main's args, it'd be nice to have a human-friendly interactive text menu.

That information may be found in a meta tag in the <head> of the html file for the document like so:

<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />

This includes but is not limited to method headers, method calls, parameter types, varargs, array construction, and parameters in lambda expressions.

When this project was first given a GUI, messages which had originally been sent only to the console (System.out) were instead sent to a Consumer which was System.out::println when running the program from the command line and which was a setText method for a JLabel when running the GUI.

Since then, the use of messages has declined, leaving many formerly message-rich behaviors devoid of messages, with the corresponding code not even being given (and not even needing to be given) access to a Consumer for that purpose. However, some code has retained references to a message-handler. This situation is inconsistent and should be made consistent.

Either

• all uses of a message-handler should be removed, or
• all methods that could reasonably need to report on their status to the user should use a message-handler

Initially each Location connects to multiple phrases (except the final Location of any Chapter, which only connects to one phrase). The operations of removing unique phrases, and removing repeated phrases that are subsumed by a larger repeated phrase are simply removing edges that link a Location with a phrase. These operations are complete once each Location connects to at most one phrase.

By generating a runtime graph representing these connections and operating on that graph, the nature of what the program does may be a lot more clear.

Currently, DataManager keeps all the bulk data separate from RepeatedPhrasesApp, which uses the data, but the idea behind DataManager is still not yet realized.

DataManager was supposed to restrict access to members that need to be lazily created when first needed so that the only way to access those members would be through a method that can intelligently create or recreate the data if it does not exist or pertains to different requirements.

While DataManager prevents code in any other class from having the option to access the initialized-null members directly, DataManager itself has direct access to those members since they are part of the class, which requires DataManager to be written more carefully so that the data is only accessed through intelligent methods.

Ideally, there should be a separate manager for each lazily-created member that RepeatedPhrasesApp needs:

• a manager for html chapters,
• a manager for Chapter objects,
• a manager for anchor data, and
• a manager for linked chapters

These phrases are currently discarded. They should instead be retained and incorporated into a pertinent cycle of phrase links.

Wrong:

doMethod("under" 
    + " water " 
    + "basket-" 
    + "weaving");

Right:

doMethod(
    "under" 
    + " water " 
    + "basket-" 
    + "weaving");

Do not leave the first parameter on the same line as the method name with the opening parenthesis.

That way, full parallelization by novel can be achieved instead of having to finish an entire layer before moving on to the next step.

Many calls to IO.scannerHasNonEmptyNextLine(s) should be replaced with s.hasNextLine() && s.hasNext() since that would express the same idea in a more conventional way.

Many or all operations should yield an in-memory result or alter an in-memory object (such as a RepeatedPhrasesApp instance) held in common by all the operations so that time spent on reading and writing can be saved.