Feature Request: generalize Doc.get_metadata about panflute HOT 10 CLOSED

Just to be clear: right now the metadata is in doc.metadata, and is composed by a bunch of Meta* elements (MetaMap, MetaList, MetaInline, etc.). This is done partly for compat with pandoc and partly because the metadata can contain complex elements (abstracts, paragraphs, etc.)

Are you thinking about a function similar to stringify() that would simplify the metadata into just a bunch of standard python structures? Like dicts + maps + strings?

from panflute.

Are you thinking about a function similar to stringify() that would simplify the metadata into just a bunch of standard python structures? Like dicts + maps + strings?

Yes, essentially that. One slight difference will be that this function should be called once only, not every time the filter's function is walked through. So it might be easier to be pre-computed and passed around. However, some sort of lazy evaluation is needed so it won't be computed when it is not used. Or perhaps just cached the result so even if the function is called every time in a walk it will not be slow?

from panflute.

To elaborate more, using the example from pandoc's manual:

---
title:  'This is the title: it contains a colon'
author:
- Author One
- Author Two
tags: [nothing, nothingness]
abstract: |
  This is the abstract.

  It consists of two paragraphs.
...

pyyaml would parses it to

{'author': ['Author One', 'Author Two'],
'abstract': 'This is the abstract.\n\nIt consists of two paragraphs.\n',
'tags': ['nothing', 'nothingness'],
'title': 'This is the title: it contains a colon'}

It would be nice that the hypothetical new function talked above would parse the panflute AST into the same output (up to reordering), so that it is more predictable and easier to test.

from panflute.

Oops, sorry for a few consecutive messages. Another thing is, it might actually be easier to achieve by manipulating from the JSON directly than from the already parsed panflute AST.

from panflute.

I think manipulating the AST would be simpler; we can modify the stringify function to work well with Meta elements. Also, we can surely cache the results. The only question would be the name.

Right now you already have doc.metadata, what would a good name be for this?

• doc.builtin_metadata
• doc.meta
• doc.simple_metadata
• doc.plain_metadata

?

from panflute.

I see.

How about something like metadata_options, from the existing metadata from doc and options` from yaml_filters.

from panflute.

By the way, what about the get_metadata method of Doc?

http://scorreia.com/software/panflute/code.html#panflute.elements.Doc.get_metadata

It has the advantage of working nicely with nested keys.

from panflute.

I just tried that method. A few questions:

It also returns values of built-in types instead of
:class:.MetaValue elements. EG: instead of returning a MetaBool
it will return True|False.

Let say the YAML in the MD is

---
key1:
  key1-1:
  - value1-1-1
  - value1-1-2
  key1-2:
  - value1-2-1
  - value1-2-2
key2:
  value1-2
...

doc.get_metadata('key1.key1-1') gives [MetaInlines(Str(value1-1-1)), MetaInlines(Str(value1-1-2))]. So it is still not in build-in types?

Another one is a more realistic example:

---
amsthm:
  plain:
  - Theorem: Lemma
  - With Space*
  definition:
  - Definition
  remark:
  - Case
  parentcounter:
  - chapter
...

If I tried doc.get_metadata('amsthm.plain'), it will complain ValueError: too many values to unpack (expected 2).

So in general if there's some unknown keys there (in this case, any levels deeper than the plain/definition/... can be arbitrary, and each entry can either be a simple string or a dictionary), it seems it will be difficult to work with this method?

e.g. the same YAML parsed by pyyaml will returns

{'amsthm': {'definition': ['Definition'], 'plain': [{'Theorem': 'Lemma'}, 'With Space*'], 'parentcounter': ['chapter'], 'remark': ['Case']}}
# pseudo-codes
options = {'amsthm': {'definition': ['Definition'], 'plain': [{'Theorem': 'Lemma'}, 'With Space*'], 'parentcounter': ['chapter'], 'remark': ['Case']}}
amsthm = options['amsthm']
for i in amsthm['plain']:
    if type(i)...

from panflute.

I think the way forward would be to extend get_metadata() to work more generally, even including the case with no argument (where it just returns the entire metadata with built-in types. I'll write down a test case with your examples and see how can I modify it.

from panflute.

Sounds good. I modified the title to reflect that. Thanks!

from panflute.