videlais / extwee Goto Github PK

I noticed this working with Twee files that didn't have a StoryData passage. It's not hard to work around, but it feels like an odd default.

It'd be nice to optionally allow parsing HTML, for example, to permit missing attributes in <tw-storydata>. I get that you would want a strict mode by default, but it feels a bit overkill to block a story from being parsed if it's missing an IFID, for example.

It would be really helpful to be able to use ext wee -c file.twee -s chapbook-1.2.0 -o file.html where the story format is derived from the '{$npm}/lib/node_modules/extwee/story-formats' directory (using the latest version of a story format if a version number isn't provided, ie: "…-s chapbook …") if a full path (relative or absolute) isn't provided.

Having to always remember to enter the full path to various story-formats is a bit painful.

Currently waiting for approval of Twine 1 TWS documentation in the Twine Specs repo.

Once the approval happens, this will go into Extwee following the pattern of Twine-Utils as TWSParser.js.

Following downloading JSON database file, allow users to search for specific versions of story formats.

In Twine, story formats are saved as part of a story-formats.json file using the following format:

[
  {
  "id": "UUIDv4",
  "name": "Story Format Name",
  "version": "Semantic Version",
  "url": "relative location in the story-formats folder",
  "userAdded": true
  }
]

• Instead of throwing errors, it should produce warnings.
• Testing should verify these warnings are produced.

I noticed that right now the output methods of Passage don't escape things outside of passage text that could cause parsing problems in the output. Some issues I see:

console.log(new Passage('"Test"', '', ['"bad"']).toTwine2HTML());

yields

<tw-passagedata pid="1" name=""Test"" tags=""bad"" ></tw-passagedata>

and on the Twee side:

console.log(new Passage('Where do tags begin? [well', '', ['hmm']).toTwee());

yields

:: Where do tags begin? [well [hmm]

The Twee spec is ambiguous about what to do here... FWIW, Twine outputs this currently:

:: Where do tags begin? \[well [hmm] {"position":"600,275","size":"100,100"}

It'd be nice if I could work with the passages array directly instead of having to use indirect methods like forEachPassage(). One use case (related to #443) was that I wanted to set the start of a story to the first passage in the array (because I didn't care about the starting point; I just wanted to get HTML). I could see myself wanting to use map or reduce on the passages array for other purposes, too.

As noted in Road to Extwee 2.2, Extwee 2.2 (like Twine-Utils) should be able to input and output JSON representations of stories, passages, and story formats for easier inter-process and library communication.

• Add JSONWriter.js with fromStory(), fromPassage() and fromStoryFormat() + tests.
• Add JSONParser.js with toStory(), toPassage(), and toStoryFormat() + tests.

const story = parseTwine2HTML(`<tw-storydata name="test" ifid="test" startnode="1">
  <tw-passagedata name="test" pid="1">&lt;&quot;&apos;</tw-passagedata>
</tw-storydata>`);

console.log(story.getPassageByName('test').text);

yields <"'. I think it should return <"' instead. This came up when I converted HTML output by Twine to Twee by doing essentially parseTwine2HTML(source).toTwee()--I end up with Twee source like this:

> [["We're both exhausted."]]

Passage, Story, and StoryFormat should be well-documented:

• Every property should be included as @property field.
• Every class should have @class and @classdescfields.
• Every class should have @example.

It'd be nice if I could pass a flag when calling Story.toTwine2HTML() to allow publishing without a start point set. In this case, I would expect it to not emit a startnode attribute on tw-storydata.

• Add a branch for de-compiling Twine 1 HTML into Twee 3 via CLI.
• Add the options to the README.

• Once the markdown table is fixed, make an Extwee 2.2 release on GitHub.

Create documentation on the following:

• Compiling Twee 3 + Twine 2 story format into Twine 2 HTML.
• De-compiling Twine 2 HTML into Twee 3.
• Compiling Twee 3 + Twine 1 engine.js + header.html + code.js into Twine 1 HTML.
• De-compiling Twine 1 HTML into Twee 3.

• Use npm publish to push Extwee 2.2 to NPM.

Add documentation on the following:

• Creating passage and story objects directly.
• Exporting a story to Twee 3.
• Exporting a story to Twine 2 JSON.
• Combining a Twine 2 story format + Story into Twine 2 HTML.
• Parsing Twine 2 JSON into Story.
• Parsing Twine 1 HTML into Story (and Twee 3).
• Dynamically creating passages.

When installed via npm (I installed the latest v.2.0.3 globally) running extwee throws a module not found error for 'semver'. Semver is declared as a devDependency, but not as a dependency.

Instead of editing the formats by hand using the '{$npm}/lib/node_modules/extwee/story-formats' directory it would be helpful to be able to list, add, remove, and update the various story formats via the extwee command. Tweego has the ability to list the story-formats currently available in its configured story-format directory which is very helpful as the directory is usually easy to access (as the user has had to specify it) and can be configured via the TWEEGO_PATH environment variable.

Another option would be to provide users with a command that would open a Finder (or Windows and Linux equivalent) window containing the story-formats directory which would enable users to easily manage the various story-formats on their own.

• Add testing of Webpack version of Extwee.
  • Parse (Twine 1 HTML, Twine 2 HTML, Twine 2 Archive HTML, Twine 1 TWS, and Twee)
  • Compile (Twine 1 HTML, Twine 2 HTML, and Twine 2 Archive HTML)
  • Generate Twee and JSON

(Borrow from TwineSpace's use of Jest Puppeteer with per-suite setup.)

Extwee 2.4 should download the Story Format Archives JSON (to be added when the SFA database is completed).

This connects to #335.

new Passage('test', ':: nefarious').toTwee()

This isn't covered by the spec, but FWIW Twine escapes them like so:

:: test
\:: nefarious

Read and write Twine 2 Archive files.

An archive is "one or more" <tw-storydata> elements.

• Add Twine2ArchiveParser.js + tests
• Add Twine2ArchiveWriter.js + tests

It looks like Story.forEachPassage is gone in favor of Story.passages (which I remember requesting :) ) but the type definition still says it's there, and passages isn't. There may be other similar issues--hard to check this.

• Add parseJSON()
• Add parseTwine1HTML()
• Add parseTwine2ArchiveHTML()
• Add compileTwine2ArchiveHTML()
• Add compileTwine1HTML()

Add JSDocs support for generating API documentation.

• Add docs folder.
• Add JSDocs.
• Add clean theme.

Tracking issue for:

• https://github.com/videlais/extwee/security/code-scanning/54

Extwee should support reading and writing Twine 1 HTML (as defined by Twine 1 documentation).

• Add Twine1HTMLParser.js + tests.
• Story has toTwine1HTML() method + tests.
• Twine1HTMLWriter.js + tests

Originally, Extwee 2.0 was going to support a extwee.config.json file for saving compilation options between builds. This should return in Extwee 2.4 as part of SFA support (as it makes more sense to save the story format version from another database).

Every parse() and compile() function should reference specific Twine Specs file using @link{} JSDocs:

• JSON parse
• StoryFormat parse
• Twee parse
• Twine 1 HTML compile
• Twine 1 HTML parse
• Twine 2 HTML compile
• Twine 2 HTML parse
• Twine 2 Archive HTML parse
• Twine 2 Archive HTML compile
• Twine 1 TWS parse

Parsing of StoryFormat JSON data should follow the "soft parsing" rule.

• Instead of throwing errors for missing properties, it should produce warnings.
• Testing should verify these warnings are produced.

Even if you're not particularly in love with TypeScript, adding declarations makes life easier for those (being self-serving here) who do and doesn't force you into a TypeScript workflow. If you're interested in getting help with this, it's something I could probably pitch in with.

The twee3 spec calls the metadata passage StoryData. Extwee seems to be looking for a passage called StoryMetadata from what I can see in the source code. I have not actually tried out Extwee yet, but I plan to, so I don't want to give the impression I did.

The ifid is the only required property of the StoryData passage.

The use of Story.toTwee() should:

• Detect if internal IFID is not in UUIDv4 format, generate a warning, and create a new IFID.
• Avoid generating format, format-version, start, tag-colors, or zoom unless those properties have non-default values.