anyblok / anyblok Goto Github PK

Noticed this with the Travis builds of Anyblok / WMS Base falling. Restricting to SQLAlchemy<1.3 does indeed fix them.

I did not investigate further, but I'd be surprised if it were specific to AWB (Anyblok's build seems also to be currently broken on master).

  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok_nose/plugins.py", line 106, in load_registry
    registry = RegistryManager.get(db_name)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/registry.py", line 136, in get
    db_name, loadwithoutmigration=loadwithoutmigration, **kwargs)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/registry.py", line 434, in __init__
    self.load()
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/logging.py", line 92, in f
    return function(*args, **kwargs)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/registry.py", line 946, in load
    raise e
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/registry.py", line 942, in load
    blok2install) or mustreload
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/registry.py", line 1069, in apply_model_schema_on_table
    self.migration.auto_upgrade_database()
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/migration.py", line 985, in auto_upgrade_database
    report = self.detect_changed()
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/migration.py", line 996, in detect_changed
    return MigrationReport(self, diff)
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/migration.py", line 292, in __init__
    if fnct(diff):
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/migration.py", line 179, in init_remove_ck
    self.raise_if_withoutautomigration()
  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/anyblok/migration.py", line 42, in raise_if_withoutautomigration
    raise MigrationException("The metadata and the base structue are "
anyblok.migration.MigrationException: The metadata and the base structue are different, or this difference is forbidden in 'no auto migration' mode

sqlalchemy.exc.ProgrammingError: (MySQLdb._exceptions.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CONSTRAINT anyblok_ck_test__test' at line 1")
[SQL: ALTER TABLE test DROP CONSTRAINT anyblok_ck_test__test]

The issue come from that the View class is not register yet.

Add some action after the commit

$ nosetests anyblok/bloks --with-anyblok-bloks -v -s --with-coverage --cover-package=anyblok
/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/sqlalchemy_utils/utils.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Iterable
/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/orderedmultidict/orderedmultidict.py:15: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import MutableMapping
Loading config file '/etc/xdg/AnyBlok/conf.cfg'
Loading config file '/home/travis/.config/AnyBlok/conf.cfg'
/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/sqlalchemy/pool/impl.py:96: SADeprecationWarning: PoolListener is deprecated in favor of the PoolEvents listener interface. The Pool.listeners parameter will be removed in a future release.
Pool.init(self, creator, **kw)
/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/sqlalchemy/pool/base.py:229: SADeprecationWarning: The Pool.add_listener() method is deprecated and will be removed in a future release. Please use the PoolEvents listener interface.
self.add_listener(l)
/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/sqlalchemy/pool/base.py:290: SADeprecationWarning: PoolListener.connect is deprecated. The PoolListener class will be removed in a future release. Please transition to the @event interface, using @event.listens_for(Engine, 'connect').
interfaces.PoolListener._adapt_listener(self, listener)
/home/travis/build/AnyBlok/AnyBlok/anyblok/registry.py:1173: SADeprecationWarning: The Session.close_all() method is deprecated and will be removed in a future release. Please refer to session.close_all_sessions().
session.close_all()
Traceback (most recent call last):
File "/home/travis/virtualenv/python3.7.1/bin/nosetests", line 11, in
sys.exit(run_exit())
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/core.py", line 121, in init
**extra_args)
File "/opt/python/3.7.1/lib/python3.7/unittest/main.py", line 100, in init
self.parseArgs(argv)
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/core.py", line 179, in parseArgs
self.createTests()
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/core.py", line 193, in createTests
self.test = self.testLoader.loadTestsFromNames(self.testNames)
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/loader.py", line 481, in loadTestsFromNames
return unittest.TestLoader.loadTestsFromNames(self, names, module)
File "/opt/python/3.7.1/lib/python3.7/unittest/loader.py", line 220, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/opt/python/3.7.1/lib/python3.7/unittest/loader.py", line 220, in
suites = [self.loadTestsFromName(name, module) for name in names]
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/loader.py", line 433, in loadTestsFromName
discovered=discovered)
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/loader.py", line 354, in loadTestsFromModule
tests.extend(self.loadTestsFromDir(module_path))
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/loader.py", line 162, in loadTestsFromDir
wanted = self.selector.wantFile(entry_path)
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/selector.py", line 126, in wantFile
plug_wants = self.plugins.wantFile(file)
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/plugins/manager.py", line 99, in call
return self.call(*arg, **kw)
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/nose/plugins/manager.py", line 167, in simple
result = meth(*arg, **kw)
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/anyblok_nose/plugins.py", line 137, in wantFile
self.load_registry()
File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/anyblok_nose/plugins.py", line 106, in load_registry
registry = RegistryManager.get(db_name)
File "/home/travis/build/AnyBlok/AnyBlok/anyblok/registry.py", line 146, in get
db_name, loadwithoutmigration=loadwithoutmigration, **kwargs)
File "/home/travis/build/AnyBlok/AnyBlok/anyblok/registry.py", line 444, in init
self.load()
File "/home/travis/build/AnyBlok/AnyBlok/anyblok/logging.py", line 92, in f
return function(*args, **kwargs)
File "/home/travis/build/AnyBlok/AnyBlok/anyblok/registry.py", line 962, in load
raise e
File "/home/travis/build/AnyBlok/AnyBlok/anyblok/registry.py", line 958, in load
blok2install) or mustreload
File "/home/travis/build/AnyBlok/AnyBlok/anyblok/registry.py", line 1085, in apply_model_schema_on_table
self.migration.auto_upgrade_database()
File "/home/travis/build/AnyBlok/AnyBlok/anyblok/migration.py", line 1047, in auto_upgrade_database
report = self.detect_changed()
File "/home/travis/build/AnyBlok/AnyBlok/anyblok/migration.py", line 1058, in detect_changed
return MigrationReport(self, diff)
File "/home/travis/build/AnyBlok/AnyBlok/anyblok/migration.py", line 297, in init
self.raise_if_withoutautomigration()
File "/home/travis/build/AnyBlok/AnyBlok/anyblok/migration.py", line 44, in raise_if_withoutautomigration
raise MigrationException("The metadata and the base structue are "
anyblok.migration.MigrationException: The metadata and the base structue are different, or this difference is forbidden in 'no auto migration' mode
The command "nosetests anyblok/bloks --with-anyblok-bloks -v -s --with-coverage --cover-package=anyblok" exited with 1.

I was trying to play with the tests of Anyblok / WMS Base under PyPy, with the psycopg2cffi backend, and got an error (see the traceback below). Not sure if this should be reported as a difference in behaviour to psycopg2cffi folks or not…

My versions:

(pypyenv) ~/anyblok/wms/awb $ python --version
Python 3.5.3 (fdd60ed87e941677e8ea11acf9f1819466521bf2, Nov 28 2018, 12:39:44)
[PyPy 6.0.0 with GCC 6.3.0 20170516]
(pypyenv) ~/anyblok/wms/awb $ pip freeze | grep psyco
psycopg2cffi==2.8.1

Traceback (most recent call last):
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
    cursor.execute(statement, parameters)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/psycopg2cffi/_impl/cursor.py", line 30, in check_closed_
    return func(self, *args, **kwargs)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/psycopg2cffi/_impl/cursor.py", line 263, in execute
    self._pq_execute(self._query, conn._async)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/psycopg2cffi/_impl/cursor.py", line 696, in _pq_execute
    self._pq_fetch()
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/psycopg2cffi/_impl/cursor.py", line 757, in _pq_fetch
    raise self._conn._create_exception(cursor=self)
psycopg2cffi._impl.exceptions.InternalError: CREATE DATABASE cannot run inside a transaction block



The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/gracinet/anyblok/wms/pypyenv/bin/anyblok_createdb", line 11, in <module>
    sys.exit(anyblok_createdb())
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/anyblok/scripts.py", line 95, in anyblok_createdb
    create_database(url, template=db_template_name)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy_utils/functions/database.py", line 559, in create_database
    result_proxy = engine.execute(text)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/engine/base.py", line 2075, in execute
    return connection.execute(statement, *multiparams, **params)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/engine/base.py", line 942, in execute
    return self._execute_text(object, multiparams, params)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/engine/base.py", line 1104, in _execute_text
    statement, parameters
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
    raise value.with_traceback(tb)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
    cursor.execute(statement, parameters)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/psycopg2cffi/_impl/cursor.py", line 30, in check_closed_
    return func(self, *args, **kwargs)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/psycopg2cffi/_impl/cursor.py", line 263, in execute
    self._pq_execute(self._query, conn._async)
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/psycopg2cffi/_impl/cursor.py", line 696, in _pq_execute
    self._pq_fetch()
  File "/home/gracinet/anyblok/wms/pypyenv/site-packages/psycopg2cffi/_impl/cursor.py", line 757, in _pq_fetch
    raise self._conn._create_exception(cursor=self)
sqlalchemy.exc.InternalError: (psycopg2cffi._impl.exceptions.InternalError) CREATE DATABASE cannot run inside a transaction block
 [SQL: 'CREATE DATABASE "test_awb_anyblok_wms_base.core" ENCODING \'utf8\' TEMPLATE template1'] (Background on this error at: http://sqlalche.me/e/2j85)

Apart from that, you'll be probably glad to hear that all the tests pass, provided I run anyblok_createdb first with the regular CPython / Psycopg2 stack.

A bad naming convention in the release 0.19.2 already type attribute.
This attribute must be renamed by model_type. It is a better/ understanding name and skyp the collision with a column named type

see in logging

anyblok.migration: DEBUG: Modify column type test.col : DATETIME => DATETIME

This is a kind of proposal to add some consistency between auto migration and upgrade features.

If I properly understood, today auto-migration is done while starting the service whatever you ask for upgrading the database. In that case the update method is not called.

The update method allow to manage data between 2 blok version.

I feel likes changing SQL schema shouldn't be done without asking the administrator. Imagine you start to get a huge project with dependencies, the database is requested by multiple application that aware of the SQL schema for reporting or so on. If the schema changed by a community blok you want to konw as soon as possible that models changed and you'll have to adapt your reporting before deploy in production.

Then if schema is properly update without calling update method I wonder how the administrator will notice that he forgot to run anyblok_upgrade which can let the database in an inconsistency state unexpected in the blok code.

The proposal is to upgrade schema explicitly not silently and make sure update method is called while changing the schema.

Alembic return if a drop/add of one foreign key is done. But if the foreign key already exist with an another configuration, no information is returned.

I would be neat to add a mixins which can conditionaly set a record to readonly.
Something like the actual ReadOnly mixin with an readonly=Bool field

Only Field, Column and RelationShip give this entry, All the fields, columns, relationship have to give their specific options

In AnyblokCore.pre_migration() there's a direct comparison of versions as simple strings:

    def pre_migration(self, latest_version):
      if latest_version is not None and latest_version < '0.4.1':

of course it's wrong. I don't know if it's the proper time to come up with a generic magical solution involving, e.g, the version representation in pkg_resources that surely follows PEP 440 (do we really want to depend on that ?), but at least this case should be fixed.

It can be useful in a class model and it's maybe something we want to specify at a name space level depending on the use case..

Here is the solution we used for the moment to target the db schema :

@Declarations.register(Declarations.Core)
class SqlBase:
    __schema__ = None

    @classmethod
    def define_table_kwargs(cls):
        res = super(SqlBase, cls).define_table_kwargs()
        if cls.__schema__ is not None:
            res.update({'schema': cls.__schema__})
        return res


@Declarations.register(Declarations.Model, tablename='Order')
class Order:
  __schema__ = 'MySchema'

I'm writing a pre_migration() method on a Blok, for a few Model renames (see AnyBlok/anyblok_wms_base#17), was thinking a simple ALTER TABLE … RENAME TO would be enough, but actually the new tables are already created.

So, in short pre_migration() is not really before the migration (unless creation of tables is not considered to be part of migrating a DB?)

I'm trying to use the --with-id and --failed options of nosetests (to replay only failed tests), but it doesn't work. There seems to be a problem reading the environment variables (or more generally the configuration) during the --failed run.

Anyblok version: commit dcf61ba

To reproduce, I'm sabotaging one of the tests of anyblok_wms_base. Here's my diff

$ git diff
diff --git a/anyblok_wms_base/bloks/wms_core/tests/test_goods.py b/anyblok_wms_base/bloks/wms_core/tests/test_goods.py
index 23120e0..3a593d9 100644
--- a/anyblok_wms_base/bloks/wms_core/tests/test_goods.py
+++ b/anyblok_wms_base/bloks/wms_core/tests/test_goods.py
@@ -42,7 +42,7 @@ class TestGoods(BlokTestCase):
                                   reason=self.arrival, location=self.stock)
         self.assertEqual(repr(goods),
                          "Wms.Goods(id=%d, state='future', type="
-                         "Wms.Goods.Type(id=%d, code='MG'))" % (
+                         "Wms.Goods.Type(id=%d, code='MH'))" % (
                              goods.id, gt.id))
         self.assertEqual(str(goods),
                          "(id=%d, state='future', type="

Here's the first nose command (in the proper virtualenv, of course):

ANYBLOK_DATABASE_NAME=test_anyblok_wms_core ANYBLOK_DRIVER=postgresql nosetests anyblok_wms_base/anyblok_wms_base/bloks --with-anyblok-bloks -v -s  --with-id

My altered test fails as it should, and is the only one.

And here's the rerun and its outcome:

======================================================================
ERROR: test suite for <class 'anyblok_wms_base.bloks.wms_core.tests.test_goods.TestGoods'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<venv_path>/lib/python3.5/site-packages/nose/suite.py", line 210, in run
    self.setUp()
  File "<venv_path>/lib/python3.5/site-packages/nose/suite.py", line 293, in setUp
    self.setupContext(ancestor)
  File "<venv_path>/lib/python3.5/site-packages/nose/suite.py", line 316, in setupContext
    try_run(context, names)
  File "<venv_path>/lib/python3.5/site-packages/nose/util.py", line 471, in try_run
    return func()
  File "<venv_path>/anyblok/anyblok/tests/testcase.py", line 333, in setUpClass
    **additional_setting)
  File "<venv_path>/anyblok/anyblok/registry.py", line 128, in get
    db_name, loadwithoutmigration=loadwithoutmigration, **kwargs)
  File "<venv_path>/anyblok/anyblok/registry.py", line 425, in __init__
    self.init_engine(db_name=db_name)
  File "<venv_path>/anyblok/anyblok/registry.py", line 466, in init_engine
    url = Configuration.get('get_url', get_url)(db_name=db_name)
  File "<venv_path>/anyblok/anyblok/config.py", line 79, in get_url
    raise ConfigurationException('No Drivername defined')
anyblok.config.ConfigurationException: No Drivername defined
-------------------- >> begin captured logging << --------------------
anyblok.config: INFO: Configuration.load_config_for_test
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 0 tests in 0.001s

FAILED (errors=1)

sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) ERREUR: la relation « anyblok_uq_user__email » existe déjà
[SQL: 'ALTER TABLE "user" ADD CONSTRAINT anyblok_uq_user__email UNIQUE (email)'] (Background on this error at: http://sqlalche.me/e/f405)
Makefile:30 : la recette pour la cible « setup-dev » a échouée
make: *** [setup-dev] Erreur 1

the constraint is applied twice

I need to find a way of updating a datetime value to now, each time the record values change. Something like a last_modified field. How can I achieve this ? Maybe it can be a new kind of field ?

(see workaround at the end, and I hope to be able to fix this)

While discovering tests to run, nose will walk the directory hierarchy. In each directory, it sorts the directory entries and performs different actions in order for files and directories.

I'll explain by commenting the code of nose.loader I have in my virtual environment:

    def loadTestsFromDir(self, path):
        """Load tests from the directory at path. This is a generator
        -- each suite of tests from a module or other file is yielded
        and is expected to be executed before the next file is
        examined.
        """
        log.debug("load from dir %s", path)
        plugins = self.config.plugins
        plugins.beforeDirectory(path)
        if self.config.addPaths:
            paths_added = add_path(path, self.config)

        entries = os.listdir(path)
        sort_list(entries, regex_last_key(self.config.testMatch))
        for entry in entries:
            # this hard-coded initial-dot test will be removed:
            # http://code.google.com/p/python-nose/issues/detail?id=82
            if entry.startswith('.'):
                continue
            entry_path = op_abspath(op_join(path, entry))
            is_file = op_isfile(entry_path)
            wanted = False
            if is_file:
                is_dir = False
                wanted = self.selector.wantFile(entry_path)

Here self.selector.wantFile calls Anyblok's Nose plugin own wantFile() method, which initializes the registry

            else:
                is_dir = op_isdir(entry_path)
                if is_dir:
                    # this hard-coded initial-underscore test will be removed:
                    # http://code.google.com/p/python-nose/issues/detail?id=82
                    if entry.startswith('_'):
                        continue
                    wanted = self.selector.wantDirectory(entry_path)

self.selector.wantDirectory does not seem to call Anyblok Nose Plugin

            is_package = ispackage(entry_path)

            # Python 3.3 now implements PEP 420: Implicit Namespace Packages.
            # As a result, it's now possible that parent paths that have a
            # segment with the same basename as our package ends up
            # in module.__path__.  So we have to keep track of what we've
            # visited, and not-revisit them again.
            if wanted and not self._haveVisited(entry_path):
                self._addVisitedPath(entry_path)
                if is_file:
                    plugins.beforeContext()
                    if entry.endswith('.py'):
                        yield self.loadTestsFromName(
                            entry_path, discovered=True)
                    else:
                        yield self.loadTestsFromFile(entry_path)
                    plugins.afterContext()
                elif is_package:
                    # Load the entry as a package: given the full path,
                    # loadTestsFromName() will figure it out
                    yield self.loadTestsFromName(
                        entry_path, discovered=True)

And here, if this is called before any regular file has been seen by the loop, we get a direct import before the registry has been initialized, and that's usually an error

For instance, in the current master of anyblok_wms_base, the sorted listing of wms_core has goods.py and location.py before the operation package (__init__.py is discarded, since it starts with an underscore) : the tests pass, but actually, it's lucky.

If I split however goods.py in a package called goods (say, with submodules goods, and type), then goods is the first entry not starting with an underscore in the directory listing of wms_core. It's a package, and I get this error:

======================================================================
ERROR: Failure: AttributeError (type object 'Model' has no attribute 'Wms')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "anyblok_wms/lib64/python3.6/site-packages/nose/failure.py", line 39, in runTest
    raise self.exc_val.with_traceback(self.tb)
  File "anyblok_wms/lib64/python3.6/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "anyblok_wms/lib64/python3.6/site-packages/nose/importer.py", line 48, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "anyblok_wms/lib64/python3.6/site-packages/nose/importer.py", line 95, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/usr/lib64/python3.6/imp.py", line 245, in load_module
    return load_package(name, filename)
  File "/usr/lib64/python3.6/imp.py", line 217, in load_package
    return _load(spec)
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "anyblok_wms/anyblok_wms_base/anyblok_wms_base/bloks/wms_core/goods/__init__.py", line 10, in <module>
    from . import goods
  File "anyblok_wms/anyblok_wms_base/anyblok_wms_base/bloks/wms_core/goods/goods.py", line 32, in <module>
    @register(Model.Wms)
AttributeError: type object 'Model' has no attribute 'Wms'

And that's because the Wms declaration has indeed not been imported. Of course if the registry had been loaded by Anyblok Nose plugin, we wouldn't have this problem.

Workaround: putting a regular file named aaa in that same directory works

(I'll check if implementing wantDirectory would solve this issue for good)

This part is wrong, since version 0.17.0, And must be rewrite

Hi @jssuzanne ,
I had an issue when i tried to use an hybrid_method on an aliased Model.
ipdb> str(query.join(OutputAvatar.reason).join(OutputObj, OutputAvatar.obj).join(OutputType, OutputObj.type).filter(Operation.id == Operation.Move.id, OutputType.query_is_storable())) *** AttributeError: 'NoneType' object has no attribute 'loaded_namespaces'

In some cases, a field might correspond to a mutable Python structure, such as a dict. Changing the contents of that data structure is invisible to SQLAlchemy unless one calls flag_modified().

I'm doing this for example in Anyblok / WMS Base for the flexible JSONB field of Model.Wms.PhysObj.Properties: (taken from https://github.com/AnyBlok/anyblok_wms_base/blob/master/anyblok_wms_base/core/physobj/main.py#L484)

          from sqlalchemy.orm.attributes import flag_modified
          self.flexible[k] = v
          flag_modified(self, '__anyblok_field_flexible')

The problem here is that it makes appicative code (AWB in that case) rely on AnyBlok internals (actual name of the attribute as seen from SQLAlchemy).

Therefore it would be a good thing for AnyBlok to expose its own version of flag_modified, so that I could do simply:

     from anyblok.field import flag_modified
     self.flexible[k] = v
     flag_modified(self, 'flexible')

When you have something incorrect in your entry point blok declarations or blok init file you end up with a message like :

ImportError : No module name 'mymodule'

Because bloks are dynamically loaded, it's hard to understand where the error is (is it in setup.py, in init or elsewhere ?)

Those kind of error is hard to understand especially for beginners that may try to play with anyblok, maybe a better exception can help

sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1005, 'Can't create table anyblok.join_person2_and_address_for_addresses (errno: 121 "Duplicate key on write or update")')
[SQL:
CREATE TABLE join_person2_and_address_for_addresses (
person2_name VARCHAR(64) NOT NULL,
address_id INTEGER NOT NULL,
CONSTRAINT anyblok_pk_join_person2_and_address_for_addresses PRIMARY KEY (person2_name, address_id),
CONSTRAINT anyblok_fk_jpaaf_addresses__address_id FOREIGN KEY(address_id) REFERENCES address (id),
CONSTRAINT anyblok_fk_jpaaf_addresses__person2_name FOREIGN KEY(person2_name) REFERENCES person2 (name)
)

The 'number' field contains the last used value of the sequence, which is certainly useful for applicative
code that would want to read it without incrementing it.

But this isn't true right after creation: it is in that case the first value that will be returned, unless it is 0 in which case the starting value is 1.

>>> sequence = registry.System.Sequence
>>> a = sequence.insert(code='A')
>>> a.number
0
>>> a.nextval()
'1'
>>> b = sequence.insert(code='B', number=3)
>>> b.number
3
>>> b.nextval()
'3'

>>> a.nextval()
'2'
>>> a.number
2
>>> b.nextval()
'4'
>>> b.number
4

The problem is see here is that applicative code has no way to know the correct meaning.

A cleaner way of doing would be to use two fields: start and current, and have current to be None until the first call to nextval()

theses tests are writing only for postgres

Alembic does not detect this change, AnyBlok need to know if the primary key change, because all the relationship can be impacted by that.

An exception must be raised if the primary key change, this action can not be done by AnyBlok, It must be a human action

The default value should be applied on the column

Add in autodoc the options and default value for each fields

AnyBlok check each contraint to know if the it is this own constraint. this is too complex and must be simplified. Because the name is too long and troncated in the database, they does not match with the regex.

Add, all the fonctionnality which does not work with mysql

sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1075, 'Incorrect table definition; there can be only one auto column and it must be defined as a key')
[SQL: ALTER TABLE test DROP PRIMARY KEY ]

An obscure error is raised when a relationship is used in a sql view

In test_column the unit test is skip for Mysql for datetime comparaison

======================================================================
FAIL: test_view_update_method (anyblok.tests.test_view.TestView)

Traceback (most recent call last):
File "/home/jssuzanne/anyblok/AnyBlok/anyblok/tests/test_view.py", line 536, in test_view_update_method
registry.TestView.query().update({'val2': 3})
AssertionError: OperationalError not raised

I have a Blok with required = ['wms-core']

If listed in install_or_update_bloks, anyblok_createdb will indeed install wms-core, but anyblok_updatedb won't upgrade wms-core.

This is not an urgent issue, I'm happy to use --update-all-bloks, but…

• does this mean there's no ordering by requirement ?
• is it planned that required Bloks would also be automatically upgraded ?

It seems that having a many2one with a reverse one2many towards a base polymorphic Model gives rise to problems upon deletions.

This is all happening in a change I'm trying to do within anyblok_wms_base (for its issue #26, with the m2o being from Avatar to Operation:

@register(Model.Wms.PhysObj)
class Avatar:
    (...)
    outcome_of = Many2One(index=True,
                          one2many='outcomes',
                          model=Model.Wms.Operation, nullable=False)

To reproduce, l

• grab https://github.com/gracinet/anyblok_wms_base/tree/outcomes_o2m
• install the wms-core Blok in a fresh database
• launch the tests of anyblok_wms_base/core/operation/tests/test_move.py (there are overall 15 test identical failures in wms-core).

Traceback:

ERROR: test_whole_planned_execute_obliviate (anyblok_wms_base.core.operation.tests.test_move.TestMove)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/gracinet/anybox/projets/Sensee/anyblok_wms/anyblok_wms_base/anyblok_wms_base/core/operation/tests/test_move.py", line 80, in test_whole_planned_execute_obliviate
    move.obliviate()
  File "/home/gracinet/anybox/projets/Sensee/anyblok_wms/anyblok_wms_base/anyblok_wms_base/core/operation/base.py", line 559, in obliviate
    self.obliviate_single()
  File "/home/gracinet/anybox/projets/Sensee/anyblok_wms/anyblok_wms_base/anyblok_wms_base/core/operation/base.py", line 743, in obliviate_single
    self.delete_outcomes()
  File "/home/gracinet/anybox/projets/Sensee/anyblok_wms/anyblok_wms_base/anyblok_wms_base/core/operation/base.py", line 697, in delete_outcomes
    avatar.delete()
  File "/home/gracinet/anybox/projets/Sensee/anyblok_wms/anyblok/anyblok/bloks/anyblok_core/core/sqlbase.py", line 577, in delete
    self.expire_relationship_mapped(mappers)
  File "/home/gracinet/anybox/projets/Sensee/anyblok_wms/anyblok/anyblok/bloks/anyblok_core/core/sqlbase.py", line 531, in expire_relationship_mapped
    field.expire(*rfields)
  File "/home/gracinet/anybox/projets/Sensee/anyblok_wms/anyblok/anyblok/bloks/anyblok_core/core/sqlbase.py", line 552, in expire
    self.registry.expire(self, fields)
  File "/home/gracinet/anybox/projets/Sensee/anyblok_wms/anyblok/anyblok/registry.py", line 1093, in expire
    obj.__class__.get_hybrid_property_columns()
  File "/home/gracinet/anybox/projets/Sensee/anyblok_wms/anyblok/anyblok/common.py", line 121, in wrapper
    return method(*args, **kwargs)
  File "/home/gracinet/anybox/projets/Sensee/anyblok_wms/anyblok/anyblok/bloks/anyblok_core/core/sqlbase.py", line 231, in get_hybrid_property_columns
    fd = cls.fields_description(*pks)
TypeError: fields_description() takes from 1 to 2 positional arguments but 3 were given

I checked with pdb:

-> fd = cls.fields_description(*pks)
(Pdb) l
226  	        inherited model if they come from polymorphisme
227  	        """
228  	        hybrid_property_columns = cls.hybrid_property_columns
229  	        if 'polymorphic_identity' in cls.__mapper_args__:
230  	            pks = cls.get_primary_keys()
231  ->	            fd = cls.fields_description(*pks)
232  	            for pk in pks:
233  	                if fd[pk].get('model'):
234  	                    Model = cls.registry.get(fd[pk]['model'])
235  	                    hybrid_property_columns.extend(
236  	                        Model.get_hybrid_property_columns())
(Pdb) pks
['id', 'id']

and I suspect these are the primary keys for the subclass (Move) as well as the polymorphic parent (Operation). Overall, I imagine theyre'd problems with m2os targetting Models with complex primary keyrs, too.

Hi,

Whenever you try to encrypt an Email field, it raises an Exception Error when you use that value in a query clause.

email = Email(label="email", nullable=False, unique=True, encrypt_key="asecret")

venv/lib/python3.7/site-packages/sqlalchemy/sql/type_api.py:1094: NotImplementedError

The exception is coming from sqlalchemy but I think It should be the responsiblity of Anyblok to implement what is needed to be consistent with others fields.