Git Product home page Git Product logo

wq / django-data-wizard Goto Github PK

View Code? Open in Web Editor NEW
334.0 334.0 53.0 2.39 MB

๐Ÿง™โš™๏ธ Import structured data (e.g. Excel, CSV, XML, JSON) into one or more Django models via an interactive web-based wizard

Home Page: https://django-data-wizard.wq.io

License: MIT License

Python 76.09% Shell 0.24% HTML 4.06% CSS 0.14% JavaScript 19.45%
csv data-import django excel serialization wizard

django-data-wizard's Introduction

wq

wq is a modular framework for field data collection and surveys via offline-capable mobile web apps.

Getting Started

wq can be installed via GitHub, Docker, PyPI, NPM, or a <script> tag via CDN. See the documentation for more information on getting started.

Dev Container (recommended)

  1. Go to https://github.com/wq/wq-docker-template
  2. Select "Use this template" -> "Create a new repository"
  3. Open the dev container via GitHub Codespaces or Docker Desktop

Docker Base Image

docker run ghcr.io/wq/base:main

Python

python3 -m venv venv
. venv/bin/activate
python -m pip install wq
wq create myproject

Node

npm init @wq myproject

CDN

<script type="module">
    import wq from 'https://unpkg.com/wq';
    wq.init({});
</script>

Features

wq is made up of the following submodules, which are maintained as separate packages.

Module Github PyPI npm Description
wq wq/wq wq wq Top level package (specifies submodules as dependencies)
wq.app wq/wq.app wq.app @wq/app A JavaScript+Python library for building robust offline-capable HTML5 data entry apps.
wq.build wq/wq.build wq.build wq command line interface.
wq.create wq/wq.create wq.create @wq/create Project template and scaffolding tools.
wq.db wq/wq.db wq.db Django REST framework extension with design patterns for CRUD APIs.

django-data-wizard's People

Contributors

marianoeramirez avatar sheppard avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

django-data-wizard's Issues

No pyproject.toml (poetry) config file

Projects using poetry for dependency and environment management might appreciate a pyproject.toml file.
I'll issue a PR to add a pyproject.toml.
Also will add another install_editable.sh to pip install --editable from the setup.py file generated by poetry rather than the hardcoded one in the repository.

DataError: integer out of range with Postgres

I am getting an error importing data with both excel and csv. I see only one record being entered into the table. I have IDs as uuid and not as int. My assumption is the library is somewhere storing the id of the entered record and storing it as in Int.

ERROR:root:integer out of range
Traceback (most recent call last):
  File "/home/chirdeep/env/swadeshi-khoj/lib/python3.6/site-packages/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NumericValueOutOfRange: integer out of range

I have also tried with SQLite, same issue. Thanks for your help .

Celery configuation

README said:

# myproject/settings.py

CELERY_RESULT_BACKEND = BROKER_URL = 'redis://localhost:6379/1'

and

# myproject/celery.py
app.config_from_object('django.conf:settings')

So, celery will read RESULT_BACKEND which not set, then you can NOT get the celery task result or status.

Is that why you update the importing status by django.model?

In my opinion, celery.result is elegant and more faster to implement the importing status

Wish you and your project better!

No runtests.sh mentioned in contributing.md and `python -m django test` fails

No runtests.sh script was found so I created one, modeled after the runserver.sh but apparently the unittests are not properly setting up and tearing down the test environment:

(maitag) hobs@frm:~/code/other-peoples/django-data-wizard$ ./runtests.sh
No changes detected
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, data_app, data_wizard, eav_app, naturalkey_app, sessions, source_app, sources
Running migrations:
  No migrations to apply.
Using Backend: data_wizard.backends.threading
Traceback (most recent call last):
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/site-packages/django/__main__.py", line 9, in <module>
    management.execute_from_command_line()
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/site-packages/django/core/management/commands/test.py", line 24, in run_from_argv
    super().run_from_argv(argv)
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/site-packages/django/core/management/base.py", line 414, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/site-packages/django/core/management/base.py", line 460, in execute
    output = self.handle(*args, **options)
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/site-packages/django/core/management/commands/test.py", line 68, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/site-packages/django/test/runner.py", line 993, in run_tests
    self.setup_test_environment()
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/site-packages/django/test/runner.py", line 771, in setup_test_environment
    setup_test_environment(debug=self.debug_mode)
  File "/home/hobs/anaconda3/envs/maitag/lib/python3.9/site-packages/django/test/utils.py", line 125, in setup_test_environment
    raise RuntimeError(
RuntimeError: setup_test_environment() was already called and can't be called again without first calling teardown_test_environment().

Upon installation admin interface error: Reverse for 'app_list' not found. 'app_list' is not a valid view function or pattern name.

Hi,

I just installed django-data-wizard to try it out according to the README.md. However as soon as I add data_wizard to my INSTALLED_APPS I get the error below when trying to access /admin. This is running the latest version from git, but I also tried the pypi version and I tried django 2.2.6. All with the same result?
Did I miss something? Where do I start looking?
Thanks,

Dirkjan

Environment:


Request Method: GET
Request URL: http://localhost:8000/admin/

Django Version: 3.0.2
Python Version: 3.7.3
Installed Applications:
['catalogitem.apps.CatalogitemConfig',
 'viewer.apps.ViewerConfig',
 'import_export',
 'data_wizard',
 'data_wizard.sources',
 'markdown_deux',
 'crispy_forms',
 'pagedown',
 'leaflet',
 'rest_framework',
 'location_field.apps.DefaultConfig',
 'django_filters',
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware']



Traceback (most recent call last):
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/contrib/admin/sites.py", line 249, in wrapper
    return self.admin_view(view, cacheable)(*args, **kwargs)
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/utils/decorators.py", line 130, in _wrapped_view
    response = view_func(request, *args, **kwargs)
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/contrib/admin/sites.py", line 231, in inner
    return view(request, *args, **kwargs)
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/contrib/admin/sites.py", line 499, in index
    app_list = self.get_app_list(request)
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/contrib/admin/sites.py", line 482, in get_app_list
    app_dict = self._build_app_dict(request)
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/contrib/admin/sites.py", line 467, in _build_app_dict
    current_app=self.name,
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/urls/base.py", line 87, in reverse
    return iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs))
  File "/home/dirkjan/Projects/rostma-docker/rostma/venv/lib/python3.7/site-packages/django/urls/resolvers.py", line 677, in _reverse_with_prefix
    raise NoReverseMatch(msg)

Exception Type: NoReverseMatch at /admin/
Exception Value: Reverse for 'app_list' not found. 'app_list' is not a valid view function or pattern name.

Did anyone tried it with django 4.06 python 3.10

Hi all did any one tested it with django 4.06 python 3.10

actually i want to import and export to postgresql where my django ap uses many to many fields and also the foreign keys i tried with django-import-export which prints ID only of foreign key and many to many but not the name. can we achieve the same with django-data-wizard

backend doesn't support absolute paths: Support for non local filesystem

When using data-wizard with S3 as default storage I get the following error.

Django 1.11
Python 2.7

/Users/AEM/venv/lib/python2.7/site-packages/storages/backends/s3boto3.py:282: UserWarning: The default behavior of S3Boto3Storage is insecure and will change in django-storages 2.0. By default files and new buckets are saved with an ACL of 'public-read' (globally publicly readable). Version 2.0 will default to using the bucket's ACL. To opt into the new behavior set AWS_DEFAULT_ACL = None, otherwise to silence this warning explicitly set AWS_DEFAULT_ACL.
  "The default behavior of S3Boto3Storage is insecure and will change "
ERROR:root:This backend doesn't support absolute paths.
Traceback (most recent call last):
  File "/Users/AEM/venv/lib/python2.7/site-packages/data_wizard/backends/base.py", line 39, in try_run_sync
    result = self.run_sync(task_name, run_id, user_id, post)
  File "/Users/AEM/venv/lib/python2.7/site-packages/data_wizard/backends/base.py", line 30, in run_sync
    result = fn(run_id, user_id)
  File "/Users/AEM/venv/lib/python2.7/site-packages/data_wizard/tasks.py", line 72, in wrapped
    return fn(run, user, **kwargs)
  File "/Users/AEM/venv/lib/python2.7/site-packages/data_wizard/tasks.py", line 282, in read_columns
    matched = get_columns(run)
  File "/Users/AEM/venv/lib/python2.7/site-packages/data_wizard/tasks.py", line 303, in get_columns
    return parse_columns(run)
  File "/Users/AEM/venv/lib/python2.7/site-packages/data_wizard/tasks.py", line 387, in parse_columns
    table = run.load_io()
  File "/Users/AEM/venv/lib/python2.7/site-packages/data_wizard/models.py", line 53, in load_io
    self._io_data = loader.load_io()
  File "/Users/AEM/venv/lib/python2.7/site-packages/data_wizard/loaders.py", line 39, in load_io
    return load_file(self.file.path, options=options)
  File "/Users/AEM/venv/lib/python2.7/site-packages/django/db/models/fields/files.py", line 65, in path
    return self.storage.path(self.name)
  File "/Users/AEM/venv/lib/python2.7/site-packages/django/core/files/storage.py", line 111, in path
    raise NotImplementedError("This backend doesn't support absolute paths.")
NotImplementedError: This backend doesn't support absolute paths.
DEBUG:django.template:Exception while resolving variable 'is_popup' in template 'data_wizard/run_columns.html'.
Traceback (most recent call last):
  File "/Users/AEM/venv/lib/python2.7/site-packages/django/template/base.py", line 903, in _resolve_lookup
    (bit, current))  # missing attribute
VariableDoesNotExist: Failed lookup for key [is_popup] in u'[{\'False\': False, \'None\': None, \'True\': True}, {u\'csrf_token\': <SimpleLazyObject: <function _get_val at 0x110e71140>>, \'perms\': <django.contrib.auth.context_processors.PermWrapper object at 0x110d6e950>, \'messages\': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x110d54f50>, u\'request\': <rest_framework.request.Request object at 0x110d53390>, u\'TIME_ZONE\': \'UTC\', u\'STATIC_URL\': \'/static/\', u\'LANGUAGES\': [(u\'af\', u\'Afrikaans\'), (u\'ar\', u\'Arabic\'), (u\'ast\', u\'Asturian\'), (u\'az\', u\'Azerbaijani\'), (u\'bg\', u\'Bulgarian\'), (u\'be\', u\'Belarusian\'), (u\'bn\', u\'Bengali\'), (u\'br\', u\'Breton\'), (u\'bs\', u\'Bosnian\'), (u\'ca\', u\'Catalan\'), (u\'cs\', u\'Czech\'), (u\'cy\', u\'Welsh\'), (u\'da\', u\'Danish\'), (u\'de\', u\'German\'), (u\'dsb\', u\'Lower Sorbian\'), (u\'el\', u\'Greek\'), (u\'en\', u\'English\'), (u\'en-au\', u\'Australian English\'), (u\'en-gb\', u\'British English\'), (u\'eo\', u\'Esperanto\'), (u\'es\', u\'Spanish\'), (u\'es-ar\', u\'Argentinian Spanish\'), (u\'es-co\', u\'Colombian Spanish\'), (u\'es-mx\', u\'Mexican Spanish\'), (u\'es-ni\', u\'Nicaraguan Spanish\'), (u\'es-ve\', u\'Venezuelan Spanish\'), (u\'et\', u\'Estonian\'), (u\'eu\', u\'Basque\'), (u\'fa\', u\'Persian\'), (u\'fi\', u\'Finnish\'), (u\'fr\', u\'French\'), (u\'fy\', u\'Frisian\'), (u\'ga\', u\'Irish\'), (u\'gd\', u\'Scottish Gaelic\'), (u\'gl\', u\'Galician\'), (u\'he\', u\'Hebrew\'), (u\'hi\', u\'Hindi\'), (u\'hr\', u\'Croatian\'), (u\'hsb\', u\'Upper Sorbian\'), (u\'hu\', u\'Hungarian\'), (u\'ia\', u\'Interlingua\'), (u\'id\', u\'Indonesian\'), (u\'io\', u\'Ido\'), (u\'is\', u\'Icelandic\'), (u\'it\', u\'Italian\'), (u\'ja\', u\'Japanese\'), (u\'ka\', u\'Georgian\'), (u\'kk\', u\'Kazakh\'), (u\'km\', u\'Khmer\'), (u\'kn\', u\'Kannada\'), (u\'ko\', u\'Korean\'), (u\'lb\', u\'Luxembourgish\'), (u\'lt\', u\'Lithuanian\'), (u\'lv\', u\'Latvian\'), (u\'mk\', u\'Macedonian\'), (u\'ml\', u\'Malayalam\'), (u\'mn\', u\'Mongolian\'), (u\'mr\', u\'Marathi\'), (u\'my\', u\'Burmese\'), (u\'nb\', u\'Norwegian Bokm\\xe5l\'), (u\'ne\', u\'Nepali\'), (u\'nl\', u\'Dutch\'), (u\'nn\', u\'Norwegian Nynorsk\'), (u\'os\', u\'Ossetic\'), (u\'pa\', u\'Punjabi\'), (u\'pl\', u\'Polish\'), (u\'pt\', u\'Portuguese\'), (u\'pt-br\', u\'Brazilian Portuguese\'), (u\'ro\', u\'Romanian\'), (u\'ru\', u\'Russian\'), (u\'sk\', u\'Slovak\'), (u\'sl\', u\'Slovenian\'), (u\'sq\', u\'Albanian\'), (u\'sr\', u\'Serbian\'), (u\'sr-latn\', u\'Serbian Latin\'), (u\'sv\', u\'Swedish\'), (u\'sw\', u\'Swahili\'), (u\'ta\', u\'Tamil\'), (u\'te\', u\'Telugu\'), (u\'th\', u\'Thai\'), (u\'tr\', u\'Turkish\'), (u\'tt\', u\'Tatar\'), (u\'udm\', u\'Udmurt\'), (u\'uk\', u\'Ukrainian\'), (u\'ur\', u\'Urdu\'), (u\'vi\', u\'Vietnamese\'), (u\'zh-hans\', u\'Simplified Chinese\'), (u\'zh-hant\', u\'Traditional Chinese\')], \'user\': <SimpleLazyObject: <User: sreddy>>, u\'agent\': <django_agent_trust.models.Agent object at 0x110d54a10>, u\'LANGUAGE_CODE\': \'en-us\', \'DEFAULT_MESSAGE_LEVELS\': {\'DEBUG\': 10, \'INFO\': 20, \'WARNING\': 30, \'SUCCESS\': 25, \'ERROR\': 40}, \'srns\': 4254, u\'LANGUAGE_BIDI\': False, u\'MEDIA_URL\': \'https://local-aem-test.s3.amazonaws.com/\'}, {}, {\'record_count\': None, \'content_type_id\': u\'sources.filesource\', u\'LANGUAGE_CODE\': \'en-us\', u\'LANGUAGE_BIDI\': False, \'serializer_label\': u\'Customer\', \'label\': u\'Run for customer\', \'object_id\': 2, \'last_update\': datetime.datetime(2020, 6, 17, 23, 53, 49, 397460, tzinfo=<UTC>), \'object_label\': u\'customer\', \'error\': "NotImplementedError: This backend doesn\'t support absolute paths.", \'loader\': u\'data_wizard.loaders.FileLoader\', u\'id\': 2, \'serializer\': \'data_wizard.registry.CustomerSerializer\'}]'
DEBUG:django.template:Exception while resolving variable 'is_popup' in template 'data_wizard/run_columns.html'.
Traceback (most recent call last):
  File "/Users/AEM/venv/lib/python2.7/site-packages/django/template/base.py", line 903, in _resolve_lookup
    (bit, current))  # missing attribute
VariableDoesNotExist: Failed lookup for key [is_popup] in u'[{\'False\': False, \'None\': None, \'True\': True}, {u\'csrf_token\': <SimpleLazyObject: <function _get_val at 0x110e71140>>, \'perms\': <django.contrib.auth.context_processors.PermWrapper object at 0x110d6e950>, \'messages\': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x110d54f50>, u\'request\': <rest_framework.request.Request object at 0x110d53390>, u\'TIME_ZONE\': \'UTC\', u\'STATIC_URL\': \'/static/\', u\'LANGUAGES\': [(u\'af\', u\'Afrikaans\'), (u\'ar\', u\'Arabic\'), (u\'ast\', u\'Asturian\'), (u\'az\', u\'Azerbaijani\'), (u\'bg\', u\'Bulgarian\'), (u\'be\', u\'Belarusian\'), (u\'bn\', u\'Bengali\'), (u\'br\', u\'Breton\'), (u\'bs\', u\'Bosnian\'), (u\'ca\', u\'Catalan\'), (u\'cs\', u\'Czech\'), (u\'cy\', u\'Welsh\'), (u\'da\', u\'Danish\'), (u\'de\', u\'German\'), (u\'dsb\', u\'Lower Sorbian\'), (u\'el\', u\'Greek\'), (u\'en\', u\'English\'), (u\'en-au\', u\'Australian English\'), (u\'en-gb\', u\'British English\'), (u\'eo\', u\'Esperanto\'), (u\'es\', u\'Spanish\'), (u\'es-ar\', u\'Argentinian Spanish\'), (u\'es-co\', u\'Colombian Spanish\'), (u\'es-mx\', u\'Mexican Spanish\'), (u\'es-ni\', u\'Nicaraguan Spanish\'), (u\'es-ve\', u\'Venezuelan Spanish\'), (u\'et\', u\'Estonian\'), (u\'eu\', u\'Basque\'), (u\'fa\', u\'Persian\'), (u\'fi\', u\'Finnish\'), (u\'fr\', u\'French\'), (u\'fy\', u\'Frisian\'), (u\'ga\', u\'Irish\'), (u\'gd\', u\'Scottish Gaelic\'), (u\'gl\', u\'Galician\'), (u\'he\', u\'Hebrew\'), (u\'hi\', u\'Hindi\'), (u\'hr\', u\'Croatian\'), (u\'hsb\', u\'Upper Sorbian\'), (u\'hu\', u\'Hungarian\'), (u\'ia\', u\'Interlingua\'), (u\'id\', u\'Indonesian\'), (u\'io\', u\'Ido\'), (u\'is\', u\'Icelandic\'), (u\'it\', u\'Italian\'), (u\'ja\', u\'Japanese\'), (u\'ka\', u\'Georgian\'), (u\'kk\', u\'Kazakh\'), (u\'km\', u\'Khmer\'), (u\'kn\', u\'Kannada\'), (u\'ko\', u\'Korean\'), (u\'lb\', u\'Luxembourgish\'), (u\'lt\', u\'Lithuanian\'), (u\'lv\', u\'Latvian\'), (u\'mk\', u\'Macedonian\'), (u\'ml\', u\'Malayalam\'), (u\'mn\', u\'Mongolian\'), (u\'mr\', u\'Marathi\'), (u\'my\', u\'Burmese\'), (u\'nb\', u\'Norwegian Bokm\\xe5l\'), (u\'ne\', u\'Nepali\'), (u\'nl\', u\'Dutch\'), (u\'nn\', u\'Norwegian Nynorsk\'), (u\'os\', u\'Ossetic\'), (u\'pa\', u\'Punjabi\'), (u\'pl\', u\'Polish\'), (u\'pt\', u\'Portuguese\'), (u\'pt-br\', u\'Brazilian Portuguese\'), (u\'ro\', u\'Romanian\'), (u\'ru\', u\'Russian\'), (u\'sk\', u\'Slovak\'), (u\'sl\', u\'Slovenian\'), (u\'sq\', u\'Albanian\'), (u\'sr\', u\'Serbian\'), (u\'sr-latn\', u\'Serbian Latin\'), (u\'sv\', u\'Swedish\'), (u\'sw\', u\'Swahili\'), (u\'ta\', u\'Tamil\'), (u\'te\', u\'Telugu\'), (u\'th\', u\'Thai\'), (u\'tr\', u\'Turkish\'), (u\'tt\', u\'Tatar\'), (u\'udm\', u\'Udmurt\'), (u\'uk\', u\'Ukrainian\'), (u\'ur\', u\'Urdu\'), (u\'vi\', u\'Vietnamese\'), (u\'zh-hans\', u\'Simplified Chinese\'), (u\'zh-hant\', u\'Traditional Chinese\')], \'user\': <SimpleLazyObject: <User: sreddy>>, u\'agent\': <django_agent_trust.models.Agent object at 0x110d54a10>, u\'LANGUAGE_CODE\': \'en-us\', \'DEFAULT_MESSAGE_LEVELS\': {\'DEBUG\': 10, \'INFO\': 20, \'WARNING\': 30, \'SUCCESS\': 25, \'ERROR\': 40}, \'srns\': 4254, u\'LANGUAGE_BIDI\': False, u\'MEDIA_URL\': \'https://local-aem-test.s3.amazonaws.com/\'}, {}, {\'record_count\': None, \'content_type_id\': u\'sources.filesource\', u\'LANGUAGE_CODE\': \'en-us\', u\'LANGUAGE_BIDI\': False, \'serializer_label\': u\'Customer\', \'label\': u\'Run for customer\', \'object_id\': 2, \'last_update\': datetime.datetime(2020, 6, 17, 23, 53, 49, 397460, tzinfo=<UTC>), \'object_label\': u\'customer\', \'error\': "NotImplementedError: This backend doesn\'t support absolute paths.", \'loader\': u\'data_wizard.loaders.FileLoader\', u\'id\': 2, \'serializer\': \'data_wizard.registry.CustomerSerializer\'}]'
DEBUG:django.template:Exception while resolving variable 'has_permission' in template 'data_wizard/run_columns.html'.
Traceback (most recent call last):
  File "/Users/AEM/venv/lib/python2.7/site-packages/django/template/base.py", line 903, in _resolve_lookup
    (bit, current))  # missing attribute
VariableDoesNotExist: Failed lookup for key [has_permission] in u'[{\'False\': False, \'None\': None, \'True\': True}, {u\'csrf_token\': <SimpleLazyObject: <function _get_val at 0x110e71140>>, \'perms\': <django.contrib.auth.context_processors.PermWrapper object at 0x110d6e950>, \'messages\': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x110d54f50>, u\'request\': <rest_framework.request.Request object at 0x110d53390>, u\'TIME_ZONE\': \'UTC\', u\'STATIC_URL\': \'/static/\', u\'LANGUAGES\': [(u\'af\', u\'Afrikaans\'), (u\'ar\', u\'Arabic\'), (u\'ast\', u\'Asturian\'), (u\'az\', u\'Azerbaijani\'), (u\'bg\', u\'Bulgarian\'), (u\'be\', u\'Belarusian\'), (u\'bn\', u\'Bengali\'), (u\'br\', u\'Breton\'), (u\'bs\', u\'Bosnian\'), (u\'ca\', u\'Catalan\'), (u\'cs\', u\'Czech\'), (u\'cy\', u\'Welsh\'), (u\'da\', u\'Danish\'), (u\'de\', u\'German\'), (u\'dsb\', u\'Lower Sorbian\'), (u\'el\', u\'Greek\'), (u\'en\', u\'English\'), (u\'en-au\', u\'Australian English\'), (u\'en-gb\', u\'British English\'), (u\'eo\', u\'Esperanto\'), (u\'es\', u\'Spanish\'), (u\'es-ar\', u\'Argentinian Spanish\'), (u\'es-co\', u\'Colombian Spanish\'), (u\'es-mx\', u\'Mexican Spanish\'), (u\'es-ni\', u\'Nicaraguan Spanish\'), (u\'es-ve\', u\'Venezuelan Spanish\'), (u\'et\', u\'Estonian\'), (u\'eu\', u\'Basque\'), (u\'fa\', u\'Persian\'), (u\'fi\', u\'Finnish\'), (u\'fr\', u\'French\'), (u\'fy\', u\'Frisian\'), (u\'ga\', u\'Irish\'), (u\'gd\', u\'Scottish Gaelic\'), (u\'gl\', u\'Galician\'), (u\'he\', u\'Hebrew\'), (u\'hi\', u\'Hindi\'), (u\'hr\', u\'Croatian\'), (u\'hsb\', u\'Upper Sorbian\'), (u\'hu\', u\'Hungarian\'), (u\'ia\', u\'Interlingua\'), (u\'id\', u\'Indonesian\'), (u\'io\', u\'Ido\'), (u\'is\', u\'Icelandic\'), (u\'it\', u\'Italian\'), (u\'ja\', u\'Japanese\'), (u\'ka\', u\'Georgian\'), (u\'kk\', u\'Kazakh\'), (u\'km\', u\'Khmer\'), (u\'kn\', u\'Kannada\'), (u\'ko\', u\'Korean\'), (u\'lb\', u\'Luxembourgish\'), (u\'lt\', u\'Lithuanian\'), (u\'lv\', u\'Latvian\'), (u\'mk\', u\'Macedonian\'), (u\'ml\', u\'Malayalam\'), (u\'mn\', u\'Mongolian\'), (u\'mr\', u\'Marathi\'), (u\'my\', u\'Burmese\'), (u\'nb\', u\'Norwegian Bokm\\xe5l\'), (u\'ne\', u\'Nepali\'), (u\'nl\', u\'Dutch\'), (u\'nn\', u\'Norwegian Nynorsk\'), (u\'os\', u\'Ossetic\'), (u\'pa\', u\'Punjabi\'), (u\'pl\', u\'Polish\'), (u\'pt\', u\'Portuguese\'), (u\'pt-br\', u\'Brazilian Portuguese\'), (u\'ro\', u\'Romanian\'), (u\'ru\', u\'Russian\'), (u\'sk\', u\'Slovak\'), (u\'sl\', u\'Slovenian\'), (u\'sq\', u\'Albanian\'), (u\'sr\', u\'Serbian\'), (u\'sr-latn\', u\'Serbian Latin\'), (u\'sv\', u\'Swedish\'), (u\'sw\', u\'Swahili\'), (u\'ta\', u\'Tamil\'), (u\'te\', u\'Telugu\'), (u\'th\', u\'Thai\'), (u\'tr\', u\'Turkish\'), (u\'tt\', u\'Tatar\'), (u\'udm\', u\'Udmurt\'), (u\'uk\', u\'Ukrainian\'), (u\'ur\', u\'Urdu\'), (u\'vi\', u\'Vietnamese\'), (u\'zh-hans\', u\'Simplified Chinese\'), (u\'zh-hant\', u\'Traditional Chinese\')], \'user\': <SimpleLazyObject: <User: sreddy>>, u\'agent\': <django_agent_trust.models.Agent object at 0x110d54a10>, u\'LANGUAGE_CODE\': \'en-us\', \'DEFAULT_MESSAGE_LEVELS\': {\'DEBUG\': 10, \'INFO\': 20, \'WARNING\': 30, \'SUCCESS\': 25, \'ERROR\': 40}, \'srns\': 4254, u\'LANGUAGE_BIDI\': False, u\'MEDIA_URL\': \'https://local-aem-test.s3.amazonaws.com/\'}, {}, {\'record_count\': None, \'content_type_id\': u\'sources.filesource\', u\'LANGUAGE_CODE\': \'en-us\', u\'LANGUAGE_BIDI\': False, \'serializer_label\': u\'Customer\', \'label\': u\'Run for customer\', \'object_id\': 2, \'last_update\': datetime.datetime(2020, 6, 17, 23, 53, 49, 397460, tzinfo=<UTC>), \'object_label\': u\'customer\', \'error\': "NotImplementedError: This backend doesn\'t support absolute paths.", \'loader\': u\'data_wizard.loaders.FileLoader\', u\'id\': 2, \'serializer\': \'data_wizard.registry.CustomerSerializer\'}, {\'block\': <Block Node: usertools. Contents: [<TextNode: u\'\\n        \'>, <IfNode>, <TextNode: u\'\\n        \'>]>}]'
DEBUG:django.template:Exception while resolving variable 'result' in template 'data_wizard/run_columns.html'.
Traceback (most recent call last):
  File "/Users/AEM/venv/lib/python2.7/site-packages/django/template/base.py", line 903, in _resolve_lookup
    (bit, current))  # missing attribute
VariableDoesNotExist: Failed lookup for key [result] in u'[{\'False\': False, \'None\': None, \'True\': True}, {u\'csrf_token\': <SimpleLazyObject: <function _get_val at 0x110e71140>>, \'perms\': <django.contrib.auth.context_processors.PermWrapper object at 0x110d6e950>, \'messages\': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x110d54f50>, u\'request\': <rest_framework.request.Request object at 0x110d53390>, u\'TIME_ZONE\': \'UTC\', u\'STATIC_URL\': \'/static/\', u\'LANGUAGES\': [(u\'af\', u\'Afrikaans\'), (u\'ar\', u\'Arabic\'), (u\'ast\', u\'Asturian\'), (u\'az\', u\'Azerbaijani\'), (u\'bg\', u\'Bulgarian\'), (u\'be\', u\'Belarusian\'), (u\'bn\', u\'Bengali\'), (u\'br\', u\'Breton\'), (u\'bs\', u\'Bosnian\'), (u\'ca\', u\'Catalan\'), (u\'cs\', u\'Czech\'), (u\'cy\', u\'Welsh\'), (u\'da\', u\'Danish\'), (u\'de\', u\'German\'), (u\'dsb\', u\'Lower Sorbian\'), (u\'el\', u\'Greek\'), (u\'en\', u\'English\'), (u\'en-au\', u\'Australian English\'), (u\'en-gb\', u\'British English\'), (u\'eo\', u\'Esperanto\'), (u\'es\', u\'Spanish\'), (u\'es-ar\', u\'Argentinian Spanish\'), (u\'es-co\', u\'Colombian Spanish\'), (u\'es-mx\', u\'Mexican Spanish\'), (u\'es-ni\', u\'Nicaraguan Spanish\'), (u\'es-ve\', u\'Venezuelan Spanish\'), (u\'et\', u\'Estonian\'), (u\'eu\', u\'Basque\'), (u\'fa\', u\'Persian\'), (u\'fi\', u\'Finnish\'), (u\'fr\', u\'French\'), (u\'fy\', u\'Frisian\'), (u\'ga\', u\'Irish\'), (u\'gd\', u\'Scottish Gaelic\'), (u\'gl\', u\'Galician\'), (u\'he\', u\'Hebrew\'), (u\'hi\', u\'Hindi\'), (u\'hr\', u\'Croatian\'), (u\'hsb\', u\'Upper Sorbian\'), (u\'hu\', u\'Hungarian\'), (u\'ia\', u\'Interlingua\'), (u\'id\', u\'Indonesian\'), (u\'io\', u\'Ido\'), (u\'is\', u\'Icelandic\'), (u\'it\', u\'Italian\'), (u\'ja\', u\'Japanese\'), (u\'ka\', u\'Georgian\'), (u\'kk\', u\'Kazakh\'), (u\'km\', u\'Khmer\'), (u\'kn\', u\'Kannada\'), (u\'ko\', u\'Korean\'), (u\'lb\', u\'Luxembourgish\'), (u\'lt\', u\'Lithuanian\'), (u\'lv\', u\'Latvian\'), (u\'mk\', u\'Macedonian\'), (u\'ml\', u\'Malayalam\'), (u\'mn\', u\'Mongolian\'), (u\'mr\', u\'Marathi\'), (u\'my\', u\'Burmese\'), (u\'nb\', u\'Norwegian Bokm\\xe5l\'), (u\'ne\', u\'Nepali\'), (u\'nl\', u\'Dutch\'), (u\'nn\', u\'Norwegian Nynorsk\'), (u\'os\', u\'Ossetic\'), (u\'pa\', u\'Punjabi\'), (u\'pl\', u\'Polish\'), (u\'pt\', u\'Portuguese\'), (u\'pt-br\', u\'Brazilian Portuguese\'), (u\'ro\', u\'Romanian\'), (u\'ru\', u\'Russian\'), (u\'sk\', u\'Slovak\'), (u\'sl\', u\'Slovenian\'), (u\'sq\', u\'Albanian\'), (u\'sr\', u\'Serbian\'), (u\'sr-latn\', u\'Serbian Latin\'), (u\'sv\', u\'Swedish\'), (u\'sw\', u\'Swahili\'), (u\'ta\', u\'Tamil\'), (u\'te\', u\'Telugu\'), (u\'th\', u\'Thai\'), (u\'tr\', u\'Turkish\'), (u\'tt\', u\'Tatar\'), (u\'udm\', u\'Udmurt\'), (u\'uk\', u\'Ukrainian\'), (u\'ur\', u\'Urdu\'), (u\'vi\', u\'Vietnamese\'), (u\'zh-hans\', u\'Simplified Chinese\'), (u\'zh-hant\', u\'Traditional Chinese\')], \'user\': <SimpleLazyObject: <User: sreddy>>, u\'agent\': <django_agent_trust.models.Agent object at 0x110d54a10>, u\'LANGUAGE_CODE\': \'en-us\', \'DEFAULT_MESSAGE_LEVELS\': {\'DEBUG\': 10, \'INFO\': 20, \'WARNING\': 30, \'SUCCESS\': 25, \'ERROR\': 40}, \'srns\': 4254, u\'LANGUAGE_BIDI\': False, u\'MEDIA_URL\': \'https://local-aem-test.s3.amazonaws.com/\'}, {}, {\'record_count\': None, \'content_type_id\': u\'sources.filesource\', u\'LANGUAGE_CODE\': \'en-us\', u\'LANGUAGE_BIDI\': False, \'serializer_label\': u\'Customer\', \'label\': u\'Run for customer\', \'object_id\': 2, \'last_update\': datetime.datetime(2020, 6, 17, 23, 53, 49, 397460, tzinfo=<UTC>), \'object_label\': u\'customer\', \'error\': "NotImplementedError: This backend doesn\'t support absolute paths.", \'loader\': u\'data_wizard.loaders.FileLoader\', u\'id\': 2, \'serializer\': \'data_wizard.registry.CustomerSerializer\'}, {\'block\': <Block Node: content. Contents: [<TextNode: u\'\\n\'>, <IfNode>, <TextNode: u\'\\n<table class="mappings">\'>, <For Node: for col in result.columns, tail_len: 7>, <TextNode: u\'\\n  </tbody>\\n</table>\\n\'>, <IfNode>, <TextNode: u\'\\n\'>]>}]'
DEBUG:django.template:Exception while resolving variable 'result' in template 'data_wizard/run_columns.html'.
Traceback (most recent call last):
  File "/Users/AEM/venv/lib/python2.7/site-packages/django/template/base.py", line 903, in _resolve_lookup
    (bit, current))  # missing attribute
VariableDoesNotExist: Failed lookup for key [result] in u'[{\'False\': False, \'None\': None, \'True\': True}, {u\'csrf_token\': <SimpleLazyObject: <function _get_val at 0x110e71140>>, \'perms\': <django.contrib.auth.context_processors.PermWrapper object at 0x110d6e950>, \'messages\': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x110d54f50>, u\'request\': <rest_framework.request.Request object at 0x110d53390>, u\'TIME_ZONE\': \'UTC\', u\'STATIC_URL\': \'/static/\', u\'LANGUAGES\': [(u\'af\', u\'Afrikaans\'), (u\'ar\', u\'Arabic\'), (u\'ast\', u\'Asturian\'), (u\'az\', u\'Azerbaijani\'), (u\'bg\', u\'Bulgarian\'), (u\'be\', u\'Belarusian\'), (u\'bn\', u\'Bengali\'), (u\'br\', u\'Breton\'), (u\'bs\', u\'Bosnian\'), (u\'ca\', u\'Catalan\'), (u\'cs\', u\'Czech\'), (u\'cy\', u\'Welsh\'), (u\'da\', u\'Danish\'), (u\'de\', u\'German\'), (u\'dsb\', u\'Lower Sorbian\'), (u\'el\', u\'Greek\'), (u\'en\', u\'English\'), (u\'en-au\', u\'Australian English\'), (u\'en-gb\', u\'British English\'), (u\'eo\', u\'Esperanto\'), (u\'es\', u\'Spanish\'), (u\'es-ar\', u\'Argentinian Spanish\'), (u\'es-co\', u\'Colombian Spanish\'), (u\'es-mx\', u\'Mexican Spanish\'), (u\'es-ni\', u\'Nicaraguan Spanish\'), (u\'es-ve\', u\'Venezuelan Spanish\'), (u\'et\', u\'Estonian\'), (u\'eu\', u\'Basque\'), (u\'fa\', u\'Persian\'), (u\'fi\', u\'Finnish\'), (u\'fr\', u\'French\'), (u\'fy\', u\'Frisian\'), (u\'ga\', u\'Irish\'), (u\'gd\', u\'Scottish Gaelic\'), (u\'gl\', u\'Galician\'), (u\'he\', u\'Hebrew\'), (u\'hi\', u\'Hindi\'), (u\'hr\', u\'Croatian\'), (u\'hsb\', u\'Upper Sorbian\'), (u\'hu\', u\'Hungarian\'), (u\'ia\', u\'Interlingua\'), (u\'id\', u\'Indonesian\'), (u\'io\', u\'Ido\'), (u\'is\', u\'Icelandic\'), (u\'it\', u\'Italian\'), (u\'ja\', u\'Japanese\'), (u\'ka\', u\'Georgian\'), (u\'kk\', u\'Kazakh\'), (u\'km\', u\'Khmer\'), (u\'kn\', u\'Kannada\'), (u\'ko\', u\'Korean\'), (u\'lb\', u\'Luxembourgish\'), (u\'lt\', u\'Lithuanian\'), (u\'lv\', u\'Latvian\'), (u\'mk\', u\'Macedonian\'), (u\'ml\', u\'Malayalam\'), (u\'mn\', u\'Mongolian\'), (u\'mr\', u\'Marathi\'), (u\'my\', u\'Burmese\'), (u\'nb\', u\'Norwegian Bokm\\xe5l\'), (u\'ne\', u\'Nepali\'), (u\'nl\', u\'Dutch\'), (u\'nn\', u\'Norwegian Nynorsk\'), (u\'os\', u\'Ossetic\'), (u\'pa\', u\'Punjabi\'), (u\'pl\', u\'Polish\'), (u\'pt\', u\'Portuguese\'), (u\'pt-br\', u\'Brazilian Portuguese\'), (u\'ro\', u\'Romanian\'), (u\'ru\', u\'Russian\'), (u\'sk\', u\'Slovak\'), (u\'sl\', u\'Slovenian\'), (u\'sq\', u\'Albanian\'), (u\'sr\', u\'Serbian\'), (u\'sr-latn\', u\'Serbian Latin\'), (u\'sv\', u\'Swedish\'), (u\'sw\', u\'Swahili\'), (u\'ta\', u\'Tamil\'), (u\'te\', u\'Telugu\'), (u\'th\', u\'Thai\'), (u\'tr\', u\'Turkish\'), (u\'tt\', u\'Tatar\'), (u\'udm\', u\'Udmurt\'), (u\'uk\', u\'Ukrainian\'), (u\'ur\', u\'Urdu\'), (u\'vi\', u\'Vietnamese\'), (u\'zh-hans\', u\'Simplified Chinese\'), (u\'zh-hant\', u\'Traditional Chinese\')], \'user\': <SimpleLazyObject: <User: sreddy>>, u\'agent\': <django_agent_trust.models.Agent object at 0x110d54a10>, u\'LANGUAGE_CODE\': \'en-us\', \'DEFAULT_MESSAGE_LEVELS\': {\'DEBUG\': 10, \'INFO\': 20, \'WARNING\': 30, \'SUCCESS\': 25, \'ERROR\': 40}, \'srns\': 4254, u\'LANGUAGE_BIDI\': False, u\'MEDIA_URL\': \'https://local-aem-test.s3.amazonaws.com/\'}, {}, {\'record_count\': None, \'content_type_id\': u\'sources.filesource\', u\'LANGUAGE_CODE\': \'en-us\', u\'LANGUAGE_BIDI\': False, \'serializer_label\': u\'Customer\', \'label\': u\'Run for customer\', \'object_id\': 2, \'last_update\': datetime.datetime(2020, 6, 17, 23, 53, 49, 397460, tzinfo=<UTC>), \'object_label\': u\'customer\', \'error\': "NotImplementedError: This backend doesn\'t support absolute paths.", \'loader\': u\'data_wizard.loaders.FileLoader\', u\'id\': 2, \'serializer\': \'data_wizard.registry.CustomerSerializer\'}, {\'block\': <Block Node: content. Contents: [<TextNode: u\'\\n\'>, <IfNode>, <TextNode: u\'\\n<table class="mappings">\'>, <For Node: for col in result.columns, tail_len: 7>, <TextNode: u\'\\n  </tbody>\\n</table>\\n\'>, <IfNode>, <TextNode: u\'\\n\'>]>}, {}]'
DEBUG:django.template:Exception while resolving variable 'result' in template 'data_wizard/run_columns.html'.
Traceback (most recent call last):
  File "/Users/AEM/venv/lib/python2.7/site-packages/django/template/base.py", line 903, in _resolve_lookup
    (bit, current))  # missing attribute
VariableDoesNotExist: Failed lookup for key [result] in u'[{\'False\': False, \'None\': None, \'True\': True}, {u\'csrf_token\': <SimpleLazyObject: <function _get_val at 0x110e71140>>, \'perms\': <django.contrib.auth.context_processors.PermWrapper object at 0x110d6e950>, \'messages\': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x110d54f50>, u\'request\': <rest_framework.request.Request object at 0x110d53390>, u\'TIME_ZONE\': \'UTC\', u\'STATIC_URL\': \'/static/\', u\'LANGUAGES\': [(u\'af\', u\'Afrikaans\'), (u\'ar\', u\'Arabic\'), (u\'ast\', u\'Asturian\'), (u\'az\', u\'Azerbaijani\'), (u\'bg\', u\'Bulgarian\'), (u\'be\', u\'Belarusian\'), (u\'bn\', u\'Bengali\'), (u\'br\', u\'Breton\'), (u\'bs\', u\'Bosnian\'), (u\'ca\', u\'Catalan\'), (u\'cs\', u\'Czech\'), (u\'cy\', u\'Welsh\'), (u\'da\', u\'Danish\'), (u\'de\', u\'German\'), (u\'dsb\', u\'Lower Sorbian\'), (u\'el\', u\'Greek\'), (u\'en\', u\'English\'), (u\'en-au\', u\'Australian English\'), (u\'en-gb\', u\'British English\'), (u\'eo\', u\'Esperanto\'), (u\'es\', u\'Spanish\'), (u\'es-ar\', u\'Argentinian Spanish\'), (u\'es-co\', u\'Colombian Spanish\'), (u\'es-mx\', u\'Mexican Spanish\'), (u\'es-ni\', u\'Nicaraguan Spanish\'), (u\'es-ve\', u\'Venezuelan Spanish\'), (u\'et\', u\'Estonian\'), (u\'eu\', u\'Basque\'), (u\'fa\', u\'Persian\'), (u\'fi\', u\'Finnish\'), (u\'fr\', u\'French\'), (u\'fy\', u\'Frisian\'), (u\'ga\', u\'Irish\'), (u\'gd\', u\'Scottish Gaelic\'), (u\'gl\', u\'Galician\'), (u\'he\', u\'Hebrew\'), (u\'hi\', u\'Hindi\'), (u\'hr\', u\'Croatian\'), (u\'hsb\', u\'Upper Sorbian\'), (u\'hu\', u\'Hungarian\'), (u\'ia\', u\'Interlingua\'), (u\'id\', u\'Indonesian\'), (u\'io\', u\'Ido\'), (u\'is\', u\'Icelandic\'), (u\'it\', u\'Italian\'), (u\'ja\', u\'Japanese\'), (u\'ka\', u\'Georgian\'), (u\'kk\', u\'Kazakh\'), (u\'km\', u\'Khmer\'), (u\'kn\', u\'Kannada\'), (u\'ko\', u\'Korean\'), (u\'lb\', u\'Luxembourgish\'), (u\'lt\', u\'Lithuanian\'), (u\'lv\', u\'Latvian\'), (u\'mk\', u\'Macedonian\'), (u\'ml\', u\'Malayalam\'), (u\'mn\', u\'Mongolian\'), (u\'mr\', u\'Marathi\'), (u\'my\', u\'Burmese\'), (u\'nb\', u\'Norwegian Bokm\\xe5l\'), (u\'ne\', u\'Nepali\'), (u\'nl\', u\'Dutch\'), (u\'nn\', u\'Norwegian Nynorsk\'), (u\'os\', u\'Ossetic\'), (u\'pa\', u\'Punjabi\'), (u\'pl\', u\'Polish\'), (u\'pt\', u\'Portuguese\'), (u\'pt-br\', u\'Brazilian Portuguese\'), (u\'ro\', u\'Romanian\'), (u\'ru\', u\'Russian\'), (u\'sk\', u\'Slovak\'), (u\'sl\', u\'Slovenian\'), (u\'sq\', u\'Albanian\'), (u\'sr\', u\'Serbian\'), (u\'sr-latn\', u\'Serbian Latin\'), (u\'sv\', u\'Swedish\'), (u\'sw\', u\'Swahili\'), (u\'ta\', u\'Tamil\'), (u\'te\', u\'Telugu\'), (u\'th\', u\'Thai\'), (u\'tr\', u\'Turkish\'), (u\'tt\', u\'Tatar\'), (u\'udm\', u\'Udmurt\'), (u\'uk\', u\'Ukrainian\'), (u\'ur\', u\'Urdu\'), (u\'vi\', u\'Vietnamese\'), (u\'zh-hans\', u\'Simplified Chinese\'), (u\'zh-hant\', u\'Traditional Chinese\')], \'user\': <SimpleLazyObject: <User: sreddy>>, u\'agent\': <django_agent_trust.models.Agent object at 0x110d54a10>, u\'LANGUAGE_CODE\': \'en-us\', \'DEFAULT_MESSAGE_LEVELS\': {\'DEBUG\': 10, \'INFO\': 20, \'WARNING\': 30, \'SUCCESS\': 25, \'ERROR\': 40}, \'srns\': 4254, u\'LANGUAGE_BIDI\': False, u\'MEDIA_URL\': \'https://local-aem-test.s3.amazonaws.com/\'}, {}, {\'record_count\': None, \'content_type_id\': u\'sources.filesource\', u\'LANGUAGE_CODE\': \'en-us\', u\'LANGUAGE_BIDI\': False, \'serializer_label\': u\'Customer\', \'label\': u\'Run for customer\', \'object_id\': 2, \'last_update\': datetime.datetime(2020, 6, 17, 23, 53, 49, 397460, tzinfo=<UTC>), \'object_label\': u\'customer\', \'error\': "NotImplementedError: This backend doesn\'t support absolute paths.", \'loader\': u\'data_wizard.loaders.FileLoader\', u\'id\': 2, \'serializer\': \'data_wizard.registry.CustomerSerializer\'}, {\'block\': <Block Node: content. Contents: [<TextNode: u\'\\n\'>, <IfNode>, <TextNode: u\'\\n<table class="mappings">\'>, <For Node: for col in result.columns, tail_len: 7>, <TextNode: u\'\\n  </tbody>\\n</table>\\n\'>, <IfNode>, <TextNode: u\'\\n\'>]>}]'
INFO:django.server:"GET /datawizard/2/columns/ HTTP/1.1" 200 1697
INFO:django.server:"GET /static/data_wizard/css/base.css HTTP/1.1" 304 0

Debugging

Hi there

This might be a question for DRF instead, but how exactly does one use pdb with this library. If one inserts a set_trace() the output on the django server keeps rolling past and so even though one is able to interact with pdb, the command prompt disappears under a torrent of HTTP requests. Is there any way to pause everything so I can debug ?

Thanks

support non-ERAV data models

The current implementation relies heavily on vera. Generalize this to support three use cases:

  • Populating a single table
  • Populating an EAV structure (e.g. wq.db.patterns.annotate or any EAV implementation)
  • Populating an ERAV structure (i.e. vera)

Obviously, only the ERAV structure will be able to ensure data can be imported without clobbering existing data in the database.

KeyError at /datawizard/ 'content_object'. Full example of simplest configuration.

Hi Andrew! "django-data-wizard" is a great thing and I want to add it to my project for mapping data from recent uploaded spreadsheets but I was trying to send initial POST request to create object with its own id but I've got a strange error and I don't understand how to solve it..Can you please help me with solving this issues and if its really works well like on screenshots, I'll donate few bucks for your project. Thanks, Alexandr. P.S I've add screenshot of all my files, that I've used for initial config on one screenshot of my emacs. So I think that I've done all like in README of your repo, then create view which sends POST request, and this should create object of my model, so I don't understand this moment, I just want to see and configure headers of my excel or csv file, and save them correct in database..
screen shot 2018-01-07 at 11 21 27

screen shot 2018-01-07 at 11 22 02

nested relational tables in a single XML file

does django Data Wizard work with an xml file that has related tables? That is a schema that when properly imported has a at least two related tables with a one to one or more likely a one to many relationship. Is there a away to properly import all of the xml data?

Failing URL lookup

Setting up a fresh install, with data-wizard 2.0.0

After following the setup instructions I am getting lost of URL reverse errors as follows:

image

image

image

I have the urlpatterns configured as per the docs:

image

And the following in INSTALLED_APPS in settings.py:

image

Any suggestions where to start looking on this?

django template based interface

Currently the interface uses Mustache templates designed to be used with applications built with the wq framework. For non-wq users, it would be nice to also provide a default interface based on Django templates, perhaps integrated with the Django admin and/or DRF's built-in API browser.

TypeError at /datawizard/

Hi,

I am trying to import an XML file.

I am getting this error

Request Method: | POST
Request URL: | http://127.0.0.1:8000/datawizard/
Django Version: | 2.0.6
Exception Type: | TypeError
Exception Value: | int() argument must be a string, a bytes-like object or a number, not 'ContentType'
Exception Location: | /Applications/Django/venv/lib/python3.6/site-packages/django/db/models/fields/init.py in get_prep_value, line 947
Python Executable: | /Applications/Django/venv/bin/python
Python Version: | 3.6.4
.....
.....
Error during template rendering
In template /Applications/Django/myvenv3/lib/python3.6/site-packages/rest_framework/templates/rest_framework/horizontal/select.html, error at line 15

` {% if field.allow_null or field.allow_blank %} -------- {% endif %} {% for select in field.iter_options %} # This is line 15 {% if select.start_option_group %} {% elif select.end_option_group %}`

Troubles importing datas when target model has many to many field

Hello,

That's not really an issue or a bug. I am in trouble to get data imported for the following model example :

class IP(models.Model):
    version = models.IntegerField()
    address = models.GenericIPAddressField()
    def __str__(self):
        return "{0}".format(self.address)

class Asset(models.Model):
    hostname = models.CharField(max_length=100, blank=True)
    fqdn = models.CharField(max_length=100, blank=True)
    ip_list = models.ManyToManyField("IP",  blank=True)

My file to import will be the in the following format :

AssetName,AssetIP,FQDN
A0001,192.168.1.1,A0001.int.myorg
A0001,10.1.1.1,A0001.int.myorg
A0002,192.168.1.2,A0002.int.myorg

So the scenario is the following :

  • asset can have many IPs.
  • they are represented as multiple lines.
  • when and unknown IP is found for a known hostname the list should be updated and the IP object created.
    With default configuration, data wizard can not handle this scenario as the Model is expecting a list instead of a string.

Do you think Data Wizard can handle such a scenario ? I read the doc and especially the part on the Custom Serializer but I am not sure to be on a good way.
Maybe I have to build an intermediate model and then report the data into multiple target models.

Should I be able to do this scenario with Data Wizard ?

Thanks for your help

Error 403 in GET /datawizard/3/status/

Hi there. I am trying to set up data-wizard in my greenfield Django site so I can import some data.

To keep the issue short I've created a gist with the following config files:

The wizard UI seems to work fine through the admin until it comes to importing the data, as in this screen in the docs:

image

The progress bar continues forever and the Django log shows repeated 403 errors which continue forever until interrupted.

Forbidden: /datawizard/3/status/
[26/Feb/2020 17:21:07] "GET /datawizard/3/status/?format=json&task=1107fc6f-b995-4de9-9328-fef7c03b4a81 HTTP/1.1" 403 58

This occurred while I was logged in as the admin user, and happened both with and without this setting set:

DATA_WIZARD = {
    'PERMISSION': 'rest_framework.permissions.IsAdminUser',
}

I'm relatively new to Python/Django - do you have any hints for tracking this error down?

Thanks ๐Ÿ™‚

Custom Serializer questions

Hi there

Many thanks for this library, I think once it is configured it will be a big help for us in the team.

I am however having problems creating a Custom Serializer and was hoping you could offer suggestions. Quite simply the excel file we are importing has columns that are not named the same as the corresponding columns in the database. That seems to not be a problem as they can be easily mapped, however I have a column that needs to take the id of the logged in user, which I need to get through the datawizard context according to the documentation.

How do I access the datawizard context please ? Once I have the datawizard context, is it merely a matter of overriding the create() method to add in that extra attribute in the validated_data?

Thanks

Automatically mapping the FK rows

Hey guys,

A question I have a CSV with first column as FK ID - and whenever I do a POST to "auto" endpoint - it stucks saying "input needed" on IDs

I am pretty sure, I am missing something - someone please direct me?

Thanks in Advance

Table A -
name, title

Table B -
FK_A, place, price

importing CSV for table B where FK_A is the foreign key to table A records.

ImportError: cannot import name 'MutableMapping' from 'collections'

hey guys, I use django 4.0.3
after i installed the package from pip , when i tried to run it i crashed
i manage to fix it by editing:
admin.py
from
from django.utils.translation import ugettext_lazy as _
to:
from django.utils.translation import gettext as _

and signals.py

from :
from django.dispatch import Signal
import_complete = Signal(providing_args=['run', 'status'])
new_metadata = Signal(providing_args=['run', 'identifier'])
progress = Signal(providing_args=['run', 'status', 'meta'])

to:
from django.dispatch import Signal
import_complete = Signal()
new_metadata = Signal()
progress = Signal()

the problem i have is :

ImportError: cannot import name 'MutableMapping' from 'collections' (/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/collections/init.py)

Cannot search Natural Key when use unique=True in a NaturalKeyModel

models.py:
class FloorPlan(NaturalKeyModel):
name = models.CharField(max_length=100, null=False, unique=True)
stories = models.CharField(max_length=10, null=True, default=Single)

class FloorPlanFacade(models.Model):
floorplan_name = models.ForeignKey(FloorPlan, on_delete=models.CASCADE, null=False)
name = models.CharField(max_length=100, null=False)
image = models.ImageField(upload_to='floorplanfacade_images/%Y/%m/%d/')

wizard.py
class FloorPlanFacadeSerializer(NaturalKeyModelSerializer):
class Meta:
model = FloorPlanFacade
fields = "all"

Excel spreadsheet, Facade.xls
floorplan_name, name, image

On Admin -> Data Wizard - Sources -> Import via data wizard,
After mapping columns, I have to manually match floorplan_name column with values in FloorPlan, instead of searching natural key automatically.

However, if I added FloorPlan model with unqiue_together, the foreign key values are automatically matched with columns.

class FloorPlan(NaturalKeyModel):
name = models.CharField(max_length=100, null=False, unique=True)
stories = models.CharField(max_length=10, null=True, default=Single)

class Meta:
    unique_together = (('floorplan_name', 'name'),)

Import Number without float point

Than you very much for this library. I am using it to import zip code from an XLS file to a model on Django.

But I have one small issue that it takes the Zipcode like a number and it's imported with the floating-point at the end of the string. For example, the Zipcode says "12345" it imports like "12345.0"

I checked the code but not find any solution to this issue.

'NoneType' object has no attribute 'run'

I set up a wizard to import a CSV file into my project. I used the default settings.
I register a drf serializer not Django model.

Python version: 2.7.13
Django version: 1.8.7
Data Wizard version: 1.2.0
URL: http://localhost:8000/datawizard/7/auto/
Error: {"error": {"message": "'NoneType' object has no attribute 'run'"}}
Settings:

DATA_WIZARD = {
    'BACKEND': 'data_wizard.backends.threading',
    'LOADER': 'data_wizard.loaders.FileLoader',
    'PERMISSION': 'rest_framework.permissions.IsAdminUser',
}

I also tried 'data_wizard.backends.immediate'

I'm assuming the issue with backend.run()

    def run_task(self, name, use_async=False, post=None,
                 backend=None, user=None):
        if not backend:
            backend = data_wizard_backend
        if not user:
            user = self.user
        return backend.run(
            name, self.pk, user.pk, use_async, post,
        )

Empty Identifiers for nullable ForeignKeyFields

Maybe I am missing something obvious here, but there does not seem to be a way to import data which contains "empty" foreign key references to a nullable foreign key field?

Here is a screenshot of what I am talking about:

image

Here the foreign key is to a "part category". An "empty" value in the identifier cannot be mapped to a "null" or "ignore" value. It appears that I am forced to select from an existing "category" - even though the model field in question is nullable.

Question: Are all the dependencies documented?

I followed along the basic setup and it when I tried to runserver I saw that Django rest framework was required.

And now when I try to run an import on a file I have no options when prompted to "Select a format (serializer) to continue."

Is it also required that I fully wire DRF to each model I want to use data-wizard with?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.