set up readthedocs so it automatically pulls / builds the docs.

2.7, pypy, 3.4 now fails although it worked in the past (I just re-ran the travis build on 0.9.1 release).

https://travis-ci.org/github/discorporate/flatland/builds/564151940

move the flatland package from ./flatland to ./src/flatland to avoid import issues.

this should be done after working on #11.

pytest is a more advanced testing framework than nose, we should use it.

iirc, the flatland0 fork of mbr has work on that.

start with py27. pypy maybe.

Original report by jek (Bitbucket: jek, GitHub: jek).

The filter from "springy" has been dumped into out/ as-is. It needs to be ported both to flatland and to a modern version of Genshi. (Not sure what version it was targeting, something circa 2006?)

there is a commented/disabled doctest in validation.containers.SetWithKnownFields now (introduced by c3d08d3 ):

from flatland import Dict, Integer
from flatland.validation import SetWithKnownFields
schema = Dict.of(Integer.named('x'), Integer.named('y')).validated_by(SetWithKnownFields())
schema.policy = None
element = schema()
element.set({'x': 123})
assert element.validate()  # assertion error, issue #25, FIXME!    (1)
element.set({'x': 123, 'z': 789})
assert not element.validate()  # no assertion error, but maybe due to #25 also.    (2)

I did a bit of debugging, but I am not familiar enough with the code to fix it, so here is what I found out:

There's a problem in validate():

valid might get set to False early (when encountering the y == None value) and thus validate() will not be able to return True after that. This is why assertion (1) fails although the SetWithKnownFields validator returns True.

    def validate(self, state=None, recurse=True):
        if not recurse:
            down = self._validate(state, True)
            if down is Unevaluated:
                self.valid = down
            else:
                self.valid = bool(down)

            up = self._validate(state, False)
            # an Unevaluated ascent validator does not override the results
            # of descent validation
            if up is not Unevaluated:
                self.valid = bool(up)
            return self.valid

        valid = True
        elements, seen, queue = [], set(), collections.deque([self])

        # descend breadth first, skipping any branches that return All*
        while queue:
            element = queue.popleft()
            if id(element) in seen:
                continue
            seen.add(id(element))
            elements.append(element)
            validated = element._validate(state, True)

            if validated is Unevaluated:
                element.valid = validated
            else:
                element.valid = bool(validated)
                if valid:
                    valid &= validated  # <<<<<<<<<<<<<<<<<< getting False here due to y == None
            if validated is SkipAll or validated is SkipAllFalse:
                continue
            queue.extend(element.children)

        # back up, visiting only the elements that weren't skipped above
        for element in reversed(elements):
            validated = element._validate(state, False)  # <<< SetWithKnownFields gets called, validated = True

            # an Unevaluated ascent validator does not override the results
            # of descent validation
            if validated is Unevaluated:
                pass
            elif element.valid:
                element.valid = bool(validated)
                if valid:  # <<<<<<< as valid is already False, it stays False
                    valid &= validated
        return bool(valid)

  File ".tox/py37/lib/python3.7/site-packages/flatland/schema/base.py", line 355, in path
    return itertools.chain(reversed(list(self.parents)), (self,))
RuntimeError: generator raised StopIteration

Since 3.7 this is a RuntimeError, before it was just a deprecation warning.

after we are finished with #11, we do not need it any more.

after #11 we could add some style checker and fix whatever comes up.

Original report by jek (Bitbucket: jek, GitHub: jek).

__specials__ on the list- and dict-derived containers aren't implemented and currently fail in colorful ways.

when building the docs:

copying static files... WARNING: logo file 'flatland.png' does not exist

Original report by Anonymous.

Flatland is so useful as a form-handling library, but it's desperately lacking a real release on pypi, even just 0.1.

What needs to be completed to get something out there for people to use? I know genshi 0.5 is slated for removal among some other things, but really, it's a useable library that could benefit from some more users who might be willing to give it a try with an actual release.

tw@tux:~/w/flatland$ tox -e py27
GLOB sdist-make: /home/tw/w/flatland/setup.py
py27 inst-nodeps: /home/tw/w/flatland/.tox/.tmp/package/1/flatland-0.8.zip
py27 installed: alabaster==0.7.12,Babel==2.6.0,blinker==1.4,certifi==2018.10.15,chardet==3.0.4,docutils==0.14,flatland==0.8,Genshi==0.7.1,idna==2.7,imagesize==1.1.0,Jinja2==2.10,MarkupSafe==1.0,nose==1.3.7,packaging==18.0,Pygments==2.2.0,pyparsing==2.2.2,pytz==2018.7,requests==2.20.0,six==1.11.0,snowballstemmer==1.2.1,Sphinx==1.8.1,sphinxcontrib-websupport==1.1.0,typing==3.6.6,urllib3==1.24
py27 run-test-pre: PYTHONHASHSEED='265513504'

py27 runtests: commands[0] | nosetests
...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Ran 723 tests in 0.443s

OK

py27 runtests: commands[1] | sphinx-build -b doctest docs/source docs/build
Running Sphinx v1.8.1
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [doctest]: targets for 27 source files that are out of date
updating environment: [] 0 added, 1 changed, 0 removed
reading sources... [100%] validation/builtin

WARNING: autodoc: failed to import class u'string.NANPphone' from module u'flatland.validation'; the following exception was raised:
No module named string
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
running tests...

Document: schema/dicts

1 items passed all tests:
10 tests in default
10 tests in 1 items.
10 passed and 0 failed.
Test passed.

Document: validation/builtin

File "validation/builtin.rst", line ?, in default
Failed example:
import flatland
from flatland.validation import NotDuplicated

validator = NotDuplicated(failure="Please enter each color only once.")
schema = List.of(String.named('favorite_color')).\
              using(validators=[validator])

Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest default[0]>", line 5, in
schema = List.of(String.named('favorite_color')).
NameError: name 'List' is not defined

File "validation/builtin.rst", line ?, in default
Failed example:
from flatland import Dict, Integer
from flatland.validation import SetWithKnownFields

schema = Dict.of(Integer.named('x'), Integer.named('y')).\
              validated_by(SetWithKnownFields())
element = schema()

element.set({'x': 123})
assert element.validate()

element.set({'x': 123, 'z': 789})
assert not element.validate()

Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest default[0]>", line 9, in
assert element.validate()
AssertionError

File "validation/builtin.rst", line ?, in default
Failed example:
from flatland import Dict, Integer
from flatland.validation import SetWithAllFields

schema = Dict.of(Integer.named('x'), Integer.named('y')).\
              validated_by(SetWithAllFields())
element = schema()

element.set({'x': 123, 'y': 456})
assert element.validate()

element.set({'x': 123})
assert not element.validate()

element.set({'x': 123, 'y': 456, 'z': 789})
assert not element.validate()

Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest default[0]>", line 14, in
element.set({'x': 123, 'y': 456, 'z': 789})
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/flatland/schema/containers.py", line 1007, in set
list(mismatch)))
KeyError: "Subset Dict None schema does not allow keys [u'z']"

1 items had failures:
3 of 20 in default
20 tests in 1 items.
17 passed and 3 failed.
Test Failed 3 failures.

Document: schema/traversal

1 items passed all tests:
29 tests in default
29 tests in 1 items.
29 passed and 0 failed.
Test passed.

Document: schema/enums

1 items passed all tests:
8 tests in default
8 tests in 1 items.
8 passed and 0 failed.
Test passed.

Document: schema/declarative

1 items passed all tests:
9 tests in default
9 tests in 1 items.
9 passed and 0 failed.
Test passed.

Document: schema/properties

2 items passed all tests:
7 tests in props
4 tests in pyann
11 tests in 2 items.
11 passed and 0 failed.
Test passed.

Document: patterns/widgets

File "patterns/widgets.rst", line 105, in genshi
Failed example:
from genshi.template import TemplateLoader
from flatland.out.genshi import setup
loader = TemplateLoader(['docs/_fixtures/genshi'], callback=setup)
template = loader.load('form.html')
print template.generate(form=form).render()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest genshi[0]>", line 4, in
template = loader.load('form.html')
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/genshi/template/loader.py", line 246, in load
raise TemplateNotFound(filename, search_path)
TemplateNotFound: Template "form.html" not found

File "patterns/widgets.rst", line 209, in jinja
Failed example:
from jinja2 import Environment, FileSystemLoader
from flatland.out.markup import Generator
loader = FileSystemLoader('docs/_fixtures/jinja')
env = Environment(loader=loader, extensions=['jinja2.ext.do'])
env.globals['form_generator'] = Generator('html')
template = env.get_template('form.html')
print template.render(form=form)
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest jinja[0]>", line 6, in
template = env.get_template('form.html')
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/jinja2/environment.py", line 830, in get_template
return self._load_template(name, self.make_globals(globals))
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/jinja2/environment.py", line 804, in _load_template
template = self.loader.load(self, name, globals)
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/jinja2/loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/jinja2/loaders.py", line 187, in get_source
raise TemplateNotFound(template)
TemplateNotFound: form.html

2 items had failures:
1 of 1 in genshi
1 of 1 in jinja
2 tests in 2 items.
0 passed and 2 failed.
Test Failed 2 failures.

Document: markup

3 items passed all tests:
9 tests in generatorintro
6 tests in transforms1
26 tests in transforms2
41 tests in 3 items.
41 passed and 0 failed.
Test passed.

Document: validation/basic

2 items passed all tests:
18 tests in default
5 tests in signals
23 tests in 2 items.
23 passed and 0 failed.
Test passed.

Document: schema/lists

1 items passed all tests:
17 tests in default
17 tests in 1 items.
17 passed and 0 failed.
Test passed.

Document: validation/api

2 items passed all tests:
3 tests in NoShouting
3 tests in default
6 tests in 2 items.
6 passed and 0 failed.
Test passed.

Document: intro

1 items passed all tests:
7 tests in default
7 tests in 1 items.
7 passed and 0 failed.
Test passed.

Document: validation/custom

1 items passed all tests:
17 tests in default
17 tests in 1 items.
17 passed and 0 failed.
Test passed.

Document: api

1 items passed all tests:
24 tests in default
24 tests in 1 items.
24 passed and 0 failed.
Test passed.

Document: schema/schema

3 items passed all tests:
27 tests in default
3 tests in find
7 tests in fso
37 tests in 3 items.
37 passed and 0 failed.
Test passed.

Document: schema/compound

File "schema/compound.rst", line ?, in default
Failed example:
el2[1].value
Expected:
u'bar'
Got:
u'o'

File "schema/compound.rst", line ?, in default
Failed example:
el2.value
Expected:
u'foo,bar'
Got:
u'f,o,o,,,b,a,r'

File "schema/compound.rst", line ?, in default
Failed example:
schema('a , b,c,d').value
Expected:
u'a, b, c, d'
Got:
u'a, , , ,, , , b, ,, c, ,, d'

1 items had failures:
3 of 14 in default
14 tests in 1 items.
11 passed and 3 failed.
Test Failed 3 failures.

Doctest summary

275 tests
8 failures in tests
0 failures in setup code
0 failures in cleanup code
build finished with problems, 1 warning.

https://github.com/discorporate/flatland/import/authors

there one can (optionally) map the users from the (previously mercurial/bitbucket) commit to github users.

it has a section about "Pre-release", but it is unclear to what that refers.

also, we need to update it with the post-0.8 changes for the 0.8.1 release.

Original report by jek (Bitbucket: jek, GitHub: jek).

#!python

>>> from flatland import String
>>> from flatland.out.markup import Generator
>>> Schema = String.named('choice')
>>> html = Generator(auto_domid=True, auto_for=True)
>>> el = Schema('b')
>>> print html.label(el)
<label for="f_choice" />
>>> print html.input(el, type='radio', value='a')
<input type="radio" name="choice" value="a" id="f_choice_a" />
>>> print html.input(el, type='radio', value='b')
<input type="radio" name="choice" value="b" checked="checked" id="f_choice_b" />

Original report by jek (Bitbucket: jek, GitHub: jek).

needed, in tests/schema/

last python 2.6 release (2.6.9) was 5 years ago and it is unmaintained - so we just drop it now.

about: https://news.ycombinator.com/item?id=25338983

example: https://github.com/borgbackup/borg/blob/master/.github/workflows/ci.yml

• small, totally safe / obvious bug / docs fixes
• no python3 (big change)
• no pytest (big change)

Original report by Anonymous.

Is there plans to support Python 3?

• for versioning based on git tags
• to replace MANIFEST.in maintenance by just using the commited files

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).

it seems like flatland development has come to a halt here about 2y ago.

what I would like to know is whether there are still developers or users of this code out there, who are interested in getting a new release of it and maybe working on bugfixes, porting to python3 etc. or whether flatland has to be abandoned due to lack of interested developers.

Another question is who could lead further development - someone who has done a lot with it or even contributed to it would be for sure better than e.g. me who has only used it a little within another project.

So, if you are interested in a future of this, please leave a comment with what you could and would contribute to it.

see borgbackup project docs for some sample.
or we could just extend the Makefile with some release target?

special to flatland:

• first tag release x.y.z so setuptools_scm generates right version number for docs and pkg
• make tip-sdist will create all the docs and also compile the translations
• python setup.py register sdist upload --identity="Thomas Waldmann" --sign # (or whoever does the release)

https://bitbucket.org/jek/flatland/pull-requests/

they were integrated into this fork: https://github.com/mbr/flatland0 (but not some other recent changesets)

when building the docs:

flatland/docs/source/schema/schema.rst:10: WARNING:
undefined label: containers (if the link has no caption the label must precede a section header)

Original report by Adam Lowry (Bitbucket: robotadam, GitHub: robotadam).

If you instantiate or call set on a Dict element with non-dictlike data, the call will fail. Since we now have .raw to ensure that valid data was passed to it, it seems like this call should behave like List.set, which will ignore return False instead of failing.

I've got a patch + test for this in a fork.

• new repo on github (done)
• python version (done)
• new homepage == readthedocs (can be just linked from github)

Original report by jek (Bitbucket: jek, GitHub: jek).

Find or recreate the output helpers for non-XML template systems (context-sharing tag-building functions, basically.)

we are trying to get things going again in our new home here on github.

goals:

• long-term maintenance
• emphasis on carefully fixing bugs (0.8.x releases, py27 only)
• improving docs
• porting to python 3.x (0.9.x+ releases)
• regularly doing releases

everybody is welcome to help / contribute!

hg needs to get changed to git.

perl to python?

Original report by jek (Bitbucket: jek, GitHub: jek).

Currently hard coded to return True

some translations have a .po file, but the translations are incomplete.

if you speak some of these languages, check the .po file below flatland/i18n/...:

• de
• es
• fr

after #29: the Makefile has some helpers to update the i18n files - use them.