Git Product home page Git Product logo

opmlpackage's Introduction

opml package

A developer's toolkit for OPML support. Node and browser-based JavaScript code that reads and writes OPML.

What is OPML and why should we use it?

OPML is an XML-based format designed to store and exchange outlines with attributes.

It's been around since the early 2000s, and is widely used in the RSS world to exchange subscription lists.

It's also a standard for interop among outliners. If you support OPML, our products will interop, and our users will be able to use all our products on their data.

Why this package?

I wanted to make it really easy for developers to add basic OPML support to their apps.

So I put the basic code you need to read and write OPML files, code that's debugged, used in other apps, including my own. It's known to work, to respect the format, and be easy to deploy.

There are other ways to read OPML, and that's very good. The more support there is, the more interop and that's the goal. I'm going to accumulate links to resources for OPML developers. If you have something you think they could use, send me an email at the address in the package.json file above.

I recorded a podcast explaining all this. :-)

What's in this package?

JavaScript code to parse and stringify OPML.

  • opml.parse -- turns OPML text into a JavaScript structure representing the OPML.

  • opml.stringify -- takes the JavaScript structure and turns it into OPML text.

  • opml.htmlify -- a simple routine to display outlines in HTML.

  • opml.visitAll -- a routine that visits all the nodes in an outline.

The Node package

It's on NPM, it's called OPML.

Here's a snippet that reads an OPML file, converts it to a JavaScript object, displays it to the console via JSON.stringify.

const fs = require ("fs");
const opml = require ("opml");
fs.readFile ("states.opml", function (err, opmltext) {
	if (!err) {
		opml.parse (opmltext, function (err, theOutline) {
			if (!err) {
				console.log (JSON.stringify (theOutline, undefined, 4));
				}
			});
		}
	});

The full Node example is here.

OPML in the browser

The same routines are available for JavaScript code running in the browser.

See the example. You have to include opml.js in your app, as the example does.

You can run the example right now, without downloading the repo.

The example app includes the Bootstrap Toolkit, the Ubuntu font and jQuery. The OPML parsing and generating code does not depend on the first two, they're just used in the example app. opml.js uses jQuery, but otherwise should be completely self-contained, i.e. it does not depend on any other files.

Other OPML projects

I have a few OPML-related projects on GitHub and on the web.

  • Drummer -- Browser and Electron-based outliner that uses OPML as its native format.

  • OPML Developer checklist. Examples, source code and advice for people adapting outliners to read and write OPML.

  • instantOutlines -- Example code for sharing live-updated outlines between users, using websockets as the notification system.

  • OPML 2.0 spec.

  • etc.

Updates

v0.5.0 -- 10/25/22 by DW

New function -- opml.readOutline. A simple bit of recurring code. Reads an outline over the web returns a standard outline object.

It's time to start a fresh sequence of versions with 0.5.0. No breakage, of course. ;-)

v0.4.24 -- 5/11/22 by DW

New function -- opml.expandIncludes.

Takes two params, an outline that could possibly have include nodes, and a callback, that receives a copy of the outline with includes expanded.

It doesn't stop for errors. This was much-debated internally, but there is linkrot and an outline with includes should work as well as it possibly can, as a blog with one broken link still works.

It is only available in the Node version, but it could be adapted to work in the client if opml.expandInclude (sic) is converted.

BTW, sorry for the closeness in the names, expandInclude and expandIncludes, but it is correct, one is singular and the other is plural.

v0.4.23 -- 3/18/22 by DW

opml.visitAll is now defined in both the Node and browser versions. Previously it was only defined in browser version.

v0.4.22 -- 1/12/22 by DW

New optional param on markdownToOutline, options, an object.

And options.flAddUnderscores, defaults true. If true, we add underscores before attribute names coming from the markdown, so we know to restore them when converting back to markdown.

But sometimes the outline is going to OPML, and on to a processor where you want it to recognize its name without the underscore.

When we're publishing a blog from a LogSeq markdown outline is an example.

v0.4.21 -- 1/8/22 by DW

There was some confusion about whether or not we should try to handle head-level atts in the markdown format, and in the end I decided not to try to do that at this time. Before doing this I have to understand much better what's going on on the other side of the interop. At this stage, whatever I do is going to be wrong, and will have to be grandfathered in for perpetuity. We have a pretty good ability to interop on the content of the outlines, but different products see the file-level metadata very differently. If there ever is an agreement on how this should work it's going to happen later.

However I did fix a problem, if a head-level att does appear as we import, we don't try to attach it to undefined.

v0.4.17 -- 1/4/22 by DW

Added expandInclude in Node package.

v0.4.15 -- 1/4/22 by DW

Last night's release only worked in the client version. The Node package was broken. It should now be fixed.

Also added a new example app that demonstrates the reading and writing of markdown/outline files in a Node app.

v0.4.12 -- 1/3/22 by DW

Two new routines, opml.markdownToOutline and opml.outlineToMarkdown, to read and write markdown files that are used to represent outlines. This is an extended Markdown that LogSeq generates. The format does not have a name at this time, or as far as I know, a spec, but at least now there is JavaScript code that reads and writes the format.

We are using this code in a new version of Drummer in the works.

It is being discussed in this thread.

v0.4.10 -- 9/24/21 by DW

New entry-point in the client, opml.read.

Reads an OPML file, returns a JavaScript object with the outline head and structure.

If options.flSubscribe is true, we ask to be notified when the file changes over a websocket.

We call back to the same routine we called when the file was read, assuming it will do the same thing with the updated OPML.

Questions, comments?

If you have any questions or comments please post an issue here.

opmlpackage's People

Contributors

scripting avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

opmlpackage's Issues

Markdown functions not loaded in node package

If I install the opml package in a node project and call opml.markdownToOutline I get the error:

ReferenceError: markdownToOutline is not defined

I can see that markdownToOutline and outlineToMarkdown are defined in commoncode.js but are not exported.

In opmlpackage.js I can see that markdownToOutline and outlineToMarkdown are exported, but not defined.

I suspect a step was missed while exporting to js.

Tweets in outlines ==> encoding problem

I spent a half-day tracking down a problem that I want to document here.

Outlines are by default encoded as ISO-8859-1. I don't know why, but that's what the code says, and most of my OPML-creating code does that.

But Twitter's API does everything in UTF-8. So when you include text that came from Twitter in an, when you save it, it will create an OPML file that has garbled text where the tweets are. Not a lot of it, which is why we've gotten this far without dealing with it. But enough to make this something that has to be dealt with.

I think what it's going to mean is converting tweet text to UTF-8 as it is inserted into an outline.

Background

The davetwitter package already deals with this in a clumsy way. It changes the content type of the OPML file to UTF-8.

It's not bad in its case when the purpose of the code is to create an outline of tweets, but it isn't a general answer because sometimes you include a tweet in an outline with lots of other text that doesn't come from Twitter.

function tweetsToOpml (screenname, theDay, theTweets) {
	var opmltext = opml.stringify (sortTweets (theTweets));
	opmltext = utils.replaceAll (opmltext, "ISO-8859-1", "UTF-8");
	saveToArchive (screenname, theDay, opmltext);
	return (opmltext);
	}

Today's problem

I am working on tweets.opml.org right now, and this encoding issue shows up there, and I want to fix it.

The clumsy fix above works there too, since the outlines only contain tweets, setting the encoding for the XML to UTF makes everything look nice. An hour ago this was a vexing problem, now it's nicely solved.

But the general problem still exists. I need a simple bit of JS code that converts a bit of UTF-encoded text to ISO-8859-1.

For review: A developer's toolkit for OPML support

A new toolkit, in the form of an NPM package.

https://github.com/scripting/opmlpackage

I wanted to get a review before putting it out there, in case there are any glaring mistakes, omissions and typos. I understand of course it's a national holiday, this just happened to be when it was ready for review. 

If you have any comments or questions or whatever, please add to this thread. Any time in the next few days would be helpful. Thanks! :-)

Convert Symantec MORE to OPML on Mac?

Hi, I've been going back over my old works-in-progress, and I found quite a few Symantec MORE outlines that, of course, I can't open with my current Mac. I'd like to convert them to OPML for use with current software and for archival purposes.

I can't seem to find a way to do this. The closest I came was OmniOutliner, which used to parse MORE files, but removed the feature some time ago.

Is there software available that will convert MORE files into OPML?

Absent that, is there a way to get the details of the MORE file format so I could provide that to someone who would build such a converter?

I'd appreciate any suggestions.

Thanks for considering my request.

opml.js doesn't include filledString() and encodeXml()

It seems like filledString() and encodeXml() are missing from opml.js. As it is now, it isn't self-contained.

I was trying to run the following short code to test the package in the browser:

<!DOCTYPE html>
<html lang="pt-BR">
  <head>
    <meta charset="UTF-8" />
    <title>Example</title>
    <script src="http://scripting.com/code/opmlpackage/client/opml.js"></script>
  </head>

  <body>
    <script>
      var opmlObj = {
        opml: {
          head: {
            title: 'Test',
            ownerName: 'Guilherme'
          },
          body: {
            subs: [
              {
                text: 'Category name',
                title: 'Category name',
                subs: []
              }
            ]
          }
        }
      }

      console.log('object', opmlObj)
      console.log('string', opml.stringify(opmlObj))
    </script>
  </body>
</html>

The console complained about filledString() and encodeXml() being not defined.

After copying the definition of these two functions from the example and pasting before the definition of opmlObj. some XML got printed to the console:

string <?xml version="1.0" encoding="ISO-8859-1"?>
<opml version="2.0">
	<head>
		<title>Test</title>
		<ownerName>Guilherme</ownerName>
		</head>
	<body>
		<outline text="Category name" title="Category name" >
			</outline>
		</body>
	</opml>

Porting the OPML package to non JavaScript environments

The OPML package is pretty stable, and I'm using it in all my JavaScript browser and Node projects. It would be great if people would port it to other environments. Make it easy to do a great job of supporting the format used for outlines and subscription lists.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.