sveetch / cookiecutter-bireli Goto Github PK

As from checker implementation:

It should be False, because further apps listing are expected to be properly ordered after the dependencies and the push end mode.

Firstly wrongly reported in cookiecutter-bireli#13

Is your feature request related to a problem? Please describe.

Currently Makefile documentation is manually updated and needs to be maintained when Makefile change.

Describe the solution you'd like

We could automatize this with https://gist.github.com/sveetch/f64e1ed21f3094229ca518a39798bebc, however the current few description addons manually written would be lost. If these addons are important we could move them elsewhere after the automatic build.

Describe alternatives you've considered

Continue to manually maintain Makefile documentation, it allows to enhance description with more text and rst syntax that we could do in Makefile help. However currently there is only two task that needed that.

Additional context

This is a feature that can be added for Bireli documentation and also inside template project doc.

Currently the settings are fixed on "en" as default project language with "Chicago" timezone.

It is obvious we won't never support any possible language and timezone, however Bireli should have small set of common language such as en, fr, de that could be selected from cookiecutter parameter and it is an opportunity to use the new "choice" parameter type from cookiecutter.

Maybe we could look at cookiecutter pre hook if we can pre build choices so we could have many default language, available languages and timezone instead of hardcoded variables in jinja templates.

Finally, the most important is to have cookiecutter variables for this than can be set from local user config, to speed up prompt filling when creating a new project.

Cookiecutter should prompt user to ask if it want basic versioning like 0.1.0 or a bumpversion format on date like 22.0.1 (YEAR + Week number in the year + Release number in the same week)

Then the pyproject.toml template will use a if/else test to push the right version. We probably will need to a custom Jinja filter for this.

Is your feature request related to a problem? Please describe.

Currently from our django-webpack settings we write the stats file (created by webpack-bundle-tracker) in static file directory.

This is not critic but can be used for a malicious attack since it is available for everyone that can guess its url. The file does not share any secret value but have some absolute path and hash.

Describe the solution you'd like

If possible webpack-bundle-tracker, we should write the stats file elsewhere or make it less easy to guess ?

The final solution is still in consideration.

A lot of cms plugins have not been updated for Django>=4.0 which result in a pending migration for a change on related_name, i've fixed it for the ckeditor plugin but at least there is fobi, snippet and djangocms_picture which have this issue in Bireli requirements.

For now we need to disable check-migrations task from quality task.

And then after we will have to pull request these applications to fix the migration issue or wait a possible fix release.

Add them to the backend/frontend requirements, add their makefile task and add them to the quality task.

• Black
• isort import ?
• Add Stylelint
• djLint

We currently add the i18n url just after the previously set ones:

It's fine currently with default composition since the CMS is the only one with i18n urls. However once you want to add another i18n urls from another application it just stack after the CMS. And since the CMS urls are greedy, they have prevalence on everything and trying to reach the other applications i18n urls will finish on the CMS 404 error which don't find any page for it.

I'm not sure if it fail also without any i18n, it needs some tests.

For now i didn't found yet how to resolve this properly in the modular mind of Bireli.

However in a project to fix, the other apps that needs to define i18n have to define them with i18n_patterns(...) + urlpatterns instead of urlpatterns + i18n_patterns(...)

There is actually a lot of Django warnings, almost all coming from the DjangoCMS stack, especially the plugins which are not up to date for Django>=4.0 and they make a lot of noise in tests.

Mute them with corresponding Django settings.

Almost everything can be switchable from the composed application and structure except for the templates.

We are in 2022 and DjangoCMS + sekizai required tags (cms_toolbar and render_block) cannot be inside a Django template "block" tag, they MUST be directly in HTML without any tags around (spaceless, block, etc..). It's AWFUL.

It forces to insert these tags in the skeleton without any condition or something else. For a CMS site it's ok to have the CMS stuff in skeleton, since the site menu will depend on it and the cms toolbar is almost ok to have everywhere, even in non CMS page.

The only remaining way to condition the CMS usage in templates is to make it from a cookiecutter variable. However this would work only at the project creation stage, once created it can be enabled or disable from composed applications. I don't see yet any proper solution.

Also, there is some project parts which are not ready yet for composition, commonly this is because CMS is involved without conditions but opposed to the sekizai problem, these parts can be managed with composer to make the CMS usage optional.

• CMS, Skeleton and sekizai
• CMS and Initial data
• CMS and tests

Since project application creation is not intuitive as simply drop a new project module and edit a monolithic settings files, a helper command is a very nice thing to have.

It won't avoid developers to understand project-composer + django-configurations but at least it will give them a solid base to
start and learn.

The command will probably use cookiecutter and template repository to properly create everything (composition app, django app, etc..).

The GNU command ls is not available on macOS system, so there is some makefile broken tasks because of it.

Actually the ls command is only used to list available apps from django-apps like with check-migration.

Something like this would do the same and be compatible on every Posix system:

find django-apps -maxdepth 1 -mindepth 1 -type d -exec basename {} \;

A list of tasks to achieve before going to beta stage:

• Virtualenv with Python>=3.8, aim support for 3.10;
• Create the Django project structure with Django 4.x;
• Create the frontend structure with Node>=18 + Bootstrap;
• Prepare directory app container;
• Add very basic Bootstrap5 layout;
• Add quality (flake8, black, tox, django-pytest);
• Add ping test on homepage;
• Setting the proper language;
• Try to port mods to django-configurations;
• Include used cookiecutter variable values into a file inside project;
• Prepare styleguide;
• Add a sample app, look how we could benefits from a pluggable cookie template
  dedicated to create new application;
• How can we benefit from proper structure, django-configurations, pseudo mods,
  etc.. to prepare pluggable addons like CMS ?
• Either drop DRF option or implement it correctly;

This directory is missing from generated project/ directory and is required for compilemessages to be run.

Also there is no make po or make mo tasks yet.

Describe the bug

django-recaptcha has released a 4.0.0 version with backward incompatible changes .

Since django-recaptcha is used in Fobi (at least) without an upgrade to it (yet), it is incompatible. But Fobi does not pin its version so it results in error when trying to use Django.

Environment

Describe your environment:

• Plateform: Any
• Python version: 3.10
• Bireli version: Reported from 0.3.7 to 0.3.9

To Reproduce

Steps to reproduce the behavior:

1. Install a project with Fobi enabled
2. Try to run server
3. Django fail on error about unknowed module captcha

Expected behavior

Everything should work well.

We need to pin django-recaptcha to <4.0.0 until Fobi upgrade to the 4.x version.

Is your feature request related to a problem? Please describe.

Sendfile is available and pre configured in composition repository, however there is not any usage sample.

Describe the solution you'd like

There should be a dedicated storage class for protected medias and an usage sample should be documentated to ease implementation of protected media download (as we did in icon and us).

Additional context

Since v4.2, Django implement a new settings to define all storage, it would be a nice thing to directly use instead of the old way, however it needs to upgrade Bireli from Django 4.1 to 4.2, it should be difficult but we have to ensure everything else is compatible (especially the DjangoCMS stack).

Youtube, Vimeo and Codemirror plugins are enabled in CKEditor configurations but are missing from static sources and they are extra plugins that are not included in ckeditor or djangocms-text-ckeditor (maybe codemirror is available in the latter).

We need to get back these plugins or disable them.

Also, Bireli will need a post hook to make symbolic links, since both ckeditor and djangocms-text-ckeditor share the same plugin and config, but depend from different static dirs, we usually put everything in the ckeditor dir and make a symbolic to it for the djangocms one.

There is a lot of settings variables which are a Value type object so we should be able to set their value either from a dotenv file or environment variable.

However it has been badly used on almost all of these settings. We currently do something like this:

SECRET_KEY = values.Value("***TOPSECRET***", environ_name="DJANGO_SECRET_KEY")

The environ_name is causing the issue, if we set DJANGO_SECRET_KEY=foo in dotenv file, the setting value is still ***TOPSECRET***. It seems the code from django-configurations expect a var DJANGO_DJANGO_SECRET_KEY

But if we define it like this:

SECRET_KEY = values.Value("***TOPSECRET***")

When we set DJANGO_SECRET_KEY=foo in dotenv file, the setting value becomes foo.

Not sure actually if adding environ_prefix=None to settings will be enough to fix issue without removing environ_name. However this will add more typo in settings, i tend to prefer to get ride of environ_name because in fact at 90% the current names are the same automatically expected without any environ_** attribute.

Finally, when debugging i found we did some things that are bad practices we should remove:

• The DOTENV setting for the dotenv file path MUST only be either as class attribute or defined in pre_setup BEFORE the super().pre_setup(). In setup or post_setup, it will be ignore.
• Using super().METHOD in setup methods may not been efficient. As the documentation describe it, it should be super(THECLASS, cls).METHOD;

Todo

• Remove all environ_name from Value settings;
• Define DOTENV from pre_setup only;
• Fix all the super().[setup method] everywhere;
• Fix dotenv sample file which use space as separator instead of , as we define in our settings ListValue fields;

We could have a dedicated optional settings that if not empty, will be added to the context.

This would be a non obstructive feature for some project that need to pass some values to the whole template context.

Restore pre configured sitemap builder.

There is not package for this, it would be a new project application that may use project-composer to gather sitemap builder from enabled applications.

This is alike localsettings.py feature we used from previous cookiecutter which allow to override settings, especially to add items to INSTALLED_APPS to install some debugging app on a local install.

Is your feature request related to a problem? Please describe.

The context processor "site_metas" is not endorsed with site metas elements, this is more like project global variables to expose in all templates.

However, the "site metas" name is know involved in a tiny feature with settings.SITE_INDEX_METAS, this is troubling since they are not related.

Describe the solution you'd like

Change the context processor name and all its mention (settings, templates, test, etc..).

Name proposals:

• project_globals
• site_globals
• site_infos
• ...

Recently bumpver has been introduced for a new versionning option but so it can not work properly directly as cookie repo, it needs to be installed.

Even we documentate this or we succeed to make bumpver optional, as it may be difficult because of options are defined in cookiecutter.json

Is your feature request related to a problem? Please describe.

A stable working beta of Diskette has been release.

Describe the solution you'd like

Include Diskette, it will need:

• Add diskette module in composer repository;
• Add diskette requirement >=0.3.2,<0.4.0;
• Add proper diskette settings for bireli project;
• Add disk-init, disk-dump, disk-load tasks to Makefile;
• Add django-apps/project_utils/diskette_definitions.py;
• Update django-apps/project_utils/composer_check.py to include Diskette processor;
• Add diskette to project toml;
• Add disk module for all compose applications that have something to dump;

Also 403 ?

This need to be optional from cookiecutter prompt since some users may need to init repository on their own.

We quickly need unittests to ensure quality since project has largely growed and we don't want to create regression.

The cookie needs tests for itself to ensure generation succeed.

A project needs basic tests, however there is some questions about application tests since we need to be modular, like a CMS test should not be runned if the CMS is not enabled. This is an issue with composer, if we remove the CMS tests from generation, they won't be there anymore if the CMS is enabled further.

We should have everything from template context to know if we are in production or not and this meta tag should conditioned to environment, on default it should disallows for indexation and enable it only for production.

Is your feature request related to a problem? Please describe.

We are actually on Django 4.0, some of our used libraries already dropped support for Django<4.2, we need to upgrade since it is the last LTS.

Describe the solution you'd like

Upgrade to Django 4.2, this will need some review on third applications we support:

• Lotus is known to be ready;
• Django taggit is ready with its last version;
• DjangoCMS is ready since 3.11.3;
• django-webpack is ready since 2.0.1;
• django-filer is ready;
• django-autocomplete-light is ready;
• djangocms-picture is ready since 4.1.0;
• djangocms-snippet is ready;
• crispy form is ready;
• cmsplugin-blocks is ready;
• emencia-django-staticpages has support for >=4.1 since v0.6.0;
• djangocms-admin-style has no support yet, it would work but there is some issues, an upgrade seems on the way: django-cms/djangocms-admin-style#500
• django recaptcha is ready since 4.0.0 but we are tied to the 3.0.0 because of Fobi, and 3.0.0 only have official support for Django 4.0;
• Fobi is currently only tested on 4.1, there is an onwork pull request which adopt 4.2, not merged yet. However we may quickly check with current release and Django 4.2, maybe it could work flawlessly;
• django-ckeditor has no official support yet however it is used on Lotus which have support and tested for Django 4.2;
• django-icomoon has only support and tested for Django 3.1... By the way is is worth to keep it in Bireli ?

Is your feature request related to a problem? Please describe.

Currently, project-composer initialization from project/init.py always print a message:

🚀 Initialize composer

Although it may be helpful to see composer init, this pollute output from commands, so these commands can't be used to output file content directly to a PIPE because the message is included at start.

Describe the solution you'd like

Remove it.

Describe alternatives you've considered

It could be conditioned to an environment variable.

Additional context

Add any other context or screenshots about the feature request here.

Instead of include it in the project scss source, it should be a scss library apart, so it will be more flexible. Also rename it to "Buckle".

Bonus point to make it a npm library.

Composer make some debug logging messages when performing its processes, they are not really useful commonly, so add a logging config in django settings to limit composer logging messages to INFO level at most.

A proper documentation is required before starting to use this project template.

The project init makes it the old way to extract version from project package by using setuptools private API.

Use importlib.metadata instead since we run with Python>=3.8 and all needed stuff is builtin in Python.

Fobi forms don't show any error when submitted with invalid/missing values even though the HTML contains classes and messages. Seems a problem with Bootstrap form error which are wrongly implemented in template.

Is your feature request related to a problem? Please describe.

New cmsplugin-blocks has been released but not upgraded in bireli requirements.

Describe the solution you'd like

Upgrade cms requirement with the last version app version. This was minor changes no need to perform a full validation.

Describe alternatives you've considered

None.

Additional context

A possible new release for cmsplugin-blocks may come soon, we should wait for it if it comes in next week else upgrade to 1.2.0

Generated project lacks of dummy favicon image and meta declaration

Currently each dependency is stored as its own variable in context. It was simple enough but it growed a little and it is not easy to distinguish the ones for bireli, backend or frontend.

New variables _backend_stack and _frontend_stack have been added to resolve this but this not efficient.

A new simpler way have to be done. The best is just to replace current variables with a one for bireli, another for backend and one for frontend.

A search in template(s) for current variables have to be done before to replace them with their new names.

This should results to something like:

"_backend_stack": {
    "python": ">=3.10",
}

And in template files we would use it like {{ cookiecutter._backend_stack.python }}.

Finally, current sphinx directive to output the stack will have to be modified to work with this.

Is your feature request related to a problem? Please describe.

When adding/removing an application from composer we currently need to execute 'requirements' then 'install-backend' tasks. Often, the first one is forgotten and lead to trouble for some devs.

Describe the solution you'd like

Just add a meta task 'update-backend' that will gather everything around updating applications so 'requirements' and then 'install-backend', and another one also for future Diskette definitions implementation.

Since applications are dynamically enabled from project-compose manifest, we can't just have all available applications tests in the tests/ directory or some of them will fail since their application may not be enabled in project.

It does not seems clean to store them in the apps itself, this would explode the structure.

Conditionning test code to enabled application would respect the structure but it may not be the nicest way.

We need to expose current release version in a meta tag generator in skeleton template.

Currently, the composer is initialized each time for each part (settings, urls, ..) it manage.

So actually, when using Django, it initialize first for the settings and then for url. In future, a sitemap part should be added also. This is not efficient.

Since composer is able to manage multiple part processor from the same instance, the composer should be initialized only once somewhere soon in a variable, then the part main controlers could just use the same composer to call processor methods.

If importer does not find the application module from appsdir, it will try to find it else and so it try to discovery from package module with identical name. It could lead to uncanny issues with some packages.

$ .venv/bin/project_composer --mode requirements
01:55:35 - DEBUG - Using manifest: composition-manifest.json
01:55:35 - WARNING - Unable to find module: django_builtins
01:55:35 - WARNING - Unable to find module: webpack
01:55:35 - DEBUG - Unable to find content file from: /home/Projects/the-requirator/.venv/lib/python3.8/site-packages/staticpages/requirements.txt
01:55:35 - WARNING - Unable to find module: styleguide
01:55:36 - DEBUG - Unable to find content file from: /home/Projects/the-requirator/.venv/lib/python3.8/site-packages/icomoon/requirements.txt

It is not really a breaking bug since there is multiple workaround (keep application name unique, add blank files for settings/urls/requirements even when the app don't need it, etc..).

But i think it should be assumed as a bug since we explicitly define a full pythonpath to application modules, it should not look anywhere else.

Is your feature request related to a problem? Please describe.

Currently install process do not compile PO files to MO files and compiled MO files are ignored from Git.

This is commonly and basically not a problem because we don't often translate much things inside projects or at least only in string and templates, they won't be translated but at least won't lead to errors.

However it can be a critical problem with translated URLs from internal application or project urls.py with i18n_patterns and gettext. If the translation does not exists and code or content rely on this translation, it won't work on local install and result to unexpected 404 response.

Describe the solution you'd like

It should be added to additional step to install along the frontend task with as a recommandation.

Describe alternatives you've considered

Add it directly to install task, but it would force a compile for nothing if project does not translate anything, also if it is not documentated this is some sort of magic.

Current implementation has been limited to a tiny stack to ease progression. However in this state the cookiecutter is very limited and even it help to enhance bootstrapping new projects, it lacks of many common applications we use.

Here we list the base applications we should focus to add first:

• DjangoCMS this is probably the more complex to do since it involves many other related apps, it may even need to improve project-compose to manage applications priority for all parts (settings, urls);
• A file browser solution. Filebrowser grapelli is a fork that did have many durability issues, even it's always maintained, something more stable would be nice. django-filer is preferred and will be tried first;
• CKeditor (out of DjangoCMS since other application may need it also);
• #14
• django-apps/project_utils/ probably needs its own app since it may involve migrations and management command;
• Recaptcha;
• Fobi;
• Lotus as the new Blog solution;
• django-sendfile2;
• Crispy forms
• DjangoRest Framework;
• Cmsplugin-blocks but it will need to be updated to fit to Django4 and last cms
• #33

This should use a setup, not a post_setup

The further class may change it also.

We should be able to put something like ?foo=HASH at the end of the CSS static files urls with a templatetag that could read webpack manifest to get the latest bundle hash.

This is to avoid caching issues.

It is required since there is no easy way now to use node-sass as a plugin in Webpack (which could manage bundle hash with webpack-bundle-tracker) because it does only support dart-sass now.

However, the ongoing project Embedded Sass may help to migrate to dart-sass and solve the integration issues with node-sass in webpack environment. (For notice, dart-sass itself is a huge NO since it's the lowest preprocessor we have seen in years).

At the time of working on this ticket, Embedded Sass solution should be audited first if it's fully ready or not to include, else we would fallback to the hacky way with node-sass.

Django filer force "easy-thumbnail" with extra requirement "svg" for SVG support. This means easy-thumbnail will requires to install Reportlab which requires libcairo.

Cairo is not a common library for all OS, especially on Docker Python images.

We don't have choice to requires it so at least we need to put it in install documentation.