Support `mkdocs 1.6.0` about mkdocs-material HOT 41 CLOSED

Released as part of 9.5.19. 4 days from MkDocs' release of 1.6 to us adopting it – not bad 😎 If you find any issues that are related to the MkDocs version bump, please create a new issue here or upstream, depending on what the cause is. If unsure, please create a discussion first. This issue can be considered resolved now.

from mkdocs-material.

As a sponsor of this library, I would be disappointed if the maintainer had given in to the pressure above to YOLO every release instead of testing/verifying it. Thank you for the good tradecraft exhibited.

from mkdocs-material.

@oprypin pip install 'mkdocs>=1.6.0' 'mkdocs-material' is the scenario where someone forces a version of MkDocs that Material has not been tested with. Everyone who installs the version that Material has been tested against will be fine. No point telling people they can use 1.6 when the result of that is as yet unknown. @squidfunk is doing the prudent thing here and has already pledged to look at what the issues might be. I will also have a look as I have an interest in this since I used the new support for generated files (which is cool btw.).

from mkdocs-material.

Please run pip install 'mkdocs>=1.6.0' 'mkdocs-material'

Jup, that is expected.

Did you run this? What is expected?

Is it really expected to you that pip does the following:

Download mkdocs_material-9.5.17. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.16. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.15. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.14. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.13. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.12. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.11. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.10. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.9. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.8. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.7. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.6. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.5. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.4. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.3. Is it compatible with MkDocs 1.6? No
Download mkdocs_material-9.5.2. Is it compatible with MkDocs 1.6? Yes. Great, let's use it!

Is it expected that, until the end of days, users will occasionally install mkdocs_material-9.5.2 because that was the last version that supported all versions of MkDocs?

from mkdocs-material.

No, I did not, because it's nothing we recommend doing on our documentation:

pip install mkdocs-material

This will automatically install compatible versions of all dependencies: MkDocs, Markdown, Pygments and Python Markdown Extensions. Material for MkDocs always strives to support the latest versions, so there's no need to install those packages separately.

I don't think that recommendation is good because using mkdocs-material still requires using the mkdocs CLI, and this CLI could change in an unexpected way (although this should happen only in a major release, but there's no guarantee in any case). Thus, I believe mkdocs should also be a direct dependency, so we can explicitly manage its version as well.

I'm coming from the rather straight forward dependency management in Node.js (which, yes, also has its problems), and I really have trouble with Python's dependency management.

In case you haven't come across it yet, there's a (IMO) fantastic article on Python dependency management with regard to upper bounds: https://iscinumpy.dev/post/bound-version-constraints In contrast to package managers in Node.js, which can install multiple versions of the same library at the same time, Python uses global dependency resolution, so upper bounds can render the dependency tree unresolvable pretty quickly. So, in most cases no upper bounds should be used, temporary upper bounds until a proper fix is available are okay, and upper bounds can always be introduced in userland.

from mkdocs-material.

Users should simply lock dependencies themselves by running pip-compile or equivalent. That's the only way to protect CI and to avoid any unexpected changes.

from mkdocs-material.

The point was:
Users should depend on mkdocs and mkdocs-material because... they depend on mkdocs and mkdocs-material. The approach per documentation may well be the easiest approach but is not the most correct approach.

mkdocs-material clearly depends on mkdocs too, that of course should not be removed. Only all upper bounds should be removed, because it's a library-like project, not an application (not the root of the dependency tree, users' websites are the root of the dependency tree).

from mkdocs-material.

In my view, Material for MkDocs is an application and not a library, as it is not consumed by other Python packages. MkDocs on the other hand is an application and a library, as it exports utilities that are used by plugins, etc. Material for MkDocs is essentially a leaf root dependency, which means: something that is consumed by endusers that are not necessarily developers.

There are no Python packages that have Material for MkDocs as a dependency. Thus, we're not at the root, but at the leaf. I'm also not sure I understand what you mean by "user's websites are the root of the dependency tree". User's websites that are built with MkDocs and Material for MkDocs are an artifact and outside of the dependency tree.

Edit: ignore my words, yes, I'm mistaken, it's a root and not a leaf dependency (got it twisted in my head). Regardless the choice of the wrong word, the rest remains: IMHO, not a library, which is why we should be cautious and prudent about our upstream dependencies.

from mkdocs-material.

Let's agree that we disagree here, i.e., have conflicting views, which is perfectly fine and good to talk about ☺️ I'm of course open to removing all upper bounds again if the majority of our users actually wants to manage dependencies by themselves, I just don't believe this is the case given the demographics of this project. That needs to be balanced against.

As said, I recommend we go back to mkdocs~=1.6 once we consider the upgrade safe. That means we don't run into this problem again until MkDocs releases 2.0. There's still a problem in Insiders discovered by @alexvoss that we need to fix, but we should be good to go in the coming days ✌️

from mkdocs-material.

I just fixed the most apparent error that we discovered in Insiders. Let's short-circuit this discussion by getting to work. Here are two PRs, one for the community edition and one for Insiders! Let's test it for a few days and if we think we caught all problems, release it 🚀 As discussed, I set the version range to ~=1.6.

Community edition

pip install git+https://github.com/squidfunk/mkdocs-material.git@chore/update-to-mkdocs-1.6

• #7082

Insiders

pip install git+https://${GH_TOKEN}@github.com/squidfunk/mkdocs-material-insiders.git@chore/update-to-mkdocs-1.6

• https://github.com/squidfunk/mkdocs-material-insiders/pull/87

from mkdocs-material.

Interesting discussion. Can't help but give my 2 cents 😄

I agree that Material for MkDocs is kinda in between applications and libraries. Application because for end-users, not consumed by other Python packages. Library because it still blends with all other dependencies of any given project using it, in the same resolution algorithm used to resolve these other dependencies.

In my experience though, as someone who glues a lot of Python "tools" together, many projects that consider themselves applications (or CLI tools, or any other name that doesn't imply "library") are actually libraries. In my experience again, all Python packages are libraries. I'll go further: all packages are libraries, whatever the ecosystem. If you're in the ecosystem, you (willingly or not) play by its (resolution) rules, and you should be a kind fellow citizen. If you can depend on an "application", whether you consume it or not, then you can use it as a library. We should never underestimate the imaginative ways developers can consume or depend on something 😉

True applications are projects which are not packaged within this ecosystem. They are above it. True applications simply provide an install script, a requirements.txt file, a ZIP file, or whatever suits the devs of the application and their users, but they do not exist within the underlying ecosystem(s) (ecosystems can be layered).

I see Material for MkDocs as a building block to build websites, not a final product: you install it alongside MkDocs core, as well as many other plugins, Markdown extensions, and various Python tooling to lint, test, build, deploy your docs.

To me, Material for MkDocs is indeed a library 😛 Especially now that it provides many plugins that can be used independently of the theme itself.

Don't get me wrong: I absolutely don't mind temporary pins or upper bounds: I sometimes use them myself, and they do the job perfectly when I want to prevent a known breakage while waiting for / working on a fix, or more generally waiting for / working on supporting the new version of the pinned/capped dependency.

But yeah, the Python ecosystem suffers from this flat packages layout, and so pins and upper bounds are highly discouraged. I really recommend the article mentioned by @sisp: https://iscinumpy.dev/post/bound-version-constraints, it explains it well.

from mkdocs-material.

@sisp Thanks again for your input! We'll re-evaluate this in the near future. Please understand that we're currently very, very heavily working in the background, doing a large refactor of some of the project's features, documentation, working on an entirely new search etc., and other big things we have not communicated yet.

We're also starting to build a team to keep up with the need and demands of enterprises and organizations, and all of this is a lot, lot, lot of work. We're still only 3-4 (mostly part-time) individuals, depending on how you count, with me working full-time on Material for MkDocs.

Given this context, I hope you understand that changing the way we manage dependencies is currently not at the top of our priority list. However, I've heard the criticism and we'll likely offer both options in the future. Probably something like:

pip install mkdocs-material[stable] # bounded
pip install mkdocs-material         # unbounded

The specifics still need to be worked out, and while I understand that you have deep knowledge of Python package management, most Material for MkDocs users don't. They just want it to work, and for this we try to make it as simple as possible. It's after all a strategic decision. As said, we're going to offer more fine-grained options in the future, but for the time being this issue should be resolved.

from mkdocs-material.

I ran into this issue this morning as well.

This is specifically a problem with pip<=22.0.2 (the version shipped with ubuntu:22.04 is 22.0.2)

Upgrading the pip version to pip>=22.0.3 fixes the issue.

Recreation steps:

Launch a new ubuntu 22.04 container:

docker run --rm -it -v .:/code -w /code ubuntu:22.04

Inside the container

apt update
apt install python3 python3-pip
python3 -m pip install mkdocs mkdocs-material

Observe that pypa/pip#10851 occurs

additional information

from mkdocs-material.

Here you go. Don't need to credit me

In here you need to perform this change:

-mkdocs~=1.5.3
+mkdocs>=1.5.3

from mkdocs-material.

I tested all changes against 40 websites using mkdocs-material (and I don't have access to mkdocs-material-insiders), on a continuous basis. Is that enough?
https://github.com/mkdocs/regressions/actions/runs/8638288330

from mkdocs-material.

Have you also verified the resulting sites

There's no diff in HTML in almost all cases so there's no need to even visually inspect it.

There were diffs on this one site but I think the site's build includes randomization or something https://github.com/mkdocs/regressions/actions/runs/8638288330/job/23682412864

from mkdocs-material.

@alexvoss jup, the generated files API looks very promising! We can probably get rid of a few dozens lines of code in our plugins, specifically the blog plugin!

from mkdocs-material.

FYI, I pinned the issue so that users coming to our issue tracker see immediately that we're working on it ☺️

from mkdocs-material.

There is a workaround that could be used to make everyone happy: add a lock extra that adds the constraint while removing it from the default dependencies.

The risk-afraid ones can install mkdocs-material[lock] and the ones that like the 'freedom' can install just mkdocs-material. Some projects can even have pinned dependencies inside this extra, but in the end is up to you to decide if you want ranges or pins.

One extra benefit is that doing this could allow keeping the CI/CD pipelines green, as no new dependencies will randomly break the. Still, I recommend keeping one 'devel' pipeline that is not using the constraints, one that would report on upcoming breakages.

from mkdocs-material.

I'm coming from the rather straight forward dependency management in Node.js (which, yes, also has its problems), and I really have trouble with Python's dependency management. I do not expect users of Material for MkDocs to be experts in how to use pip, since many of them are non-technical, so our goal should be to make it as simple as possible.

IMHO, offering more installation options like mkdocs-material[lock] would add to the confusion 😉 Thus, let's try harder and keep it simple. Additionally, note that we're also recommending to lock dependencies in our installation guide, if you decide to install a specific version of Material for MkDocs:

from mkdocs-material.

I agree, Material for MkDocs is an application and not a library. It should always install on its own successfully, and it should be validated to work with whatever version of MkDocs it "ships with" and not "maybe it will work with whatever is the latest MkDocs, maybe it will have subtle bugs, maybe it will have severe errors". In other words, exactly what's happening now.

from mkdocs-material.

As for the demographics of Material for MkDocs, I don't have anything smart to say. Installing Material for MkDocs is easy, but understanding that there's a compatibility issue, understanding where to search for ways to resolve it, and understanding the suggested ways to resolve it (if there are any) is definitely not as easy. So I understand the need to simplify this for your users 🙂

Would be interesting to know the percentage of non-technical users that manage their Python dependencies themselves vs. those who rely on technical peers for that. But that's for another day 😄

from mkdocs-material.

Yes it is related, see #7076 (comment) mkdocs/mkdocs#3477 (comment)

from mkdocs-material.

@nicfv you can already test a fix, as noted in #7076 (comment) – we'll release it in the coming days ☺️

pip install git+https://github.com/squidfunk/mkdocs-material.git@chore/update-to-mkdocs-1.6

from mkdocs-material.

Our testing shows that we consider it quite safe to upgrade to MkDocs 1.6 🎉 If we hit problems, we'll fix them on the way – as always. The new anchor validation functionality is very useful, and allowed me to fix tons of anchor links in 2585b82 that went stale due to several restructures. Some of the warnings cannot be fixed, because anchors that do exist are not picked up by MkDocs, but that's no problem for us, we'll just live with that.

from mkdocs-material.

Thanks for noting. As said, any help is appreciated!

from mkdocs-material.

For more background regarding reasoning (though that comment of mine is on an unrelated project) see Guts/mkdocs-rss-plugin#137 (comment)

from mkdocs-material.

Thanks @oprypin, I'm aware how to update the MkDocs dependency. I'm just concerned it breaks something in the community edition or in Insiders, which is why we need to test everything before considering the upgrade safe ☺️ We just don't want to be forced into doing a major release, as we had to before 2 (or more, not sure) times.

Thus, if somebody has some time, testing MkDocs 1.6 against the community edition and Insiders is appreciated ✌️ If nobody is up for it, I'll try to find some time on the weekend.

from mkdocs-material.

Sounds good ☺️ Have you also verified the resulting sites? Title handling has undergone significant changes, so I'd be curious if we need to adjust templates or plugins to reflect the upstream changes. Additionally, we need to test Insiders, especially the typeset plugin, and all other plugins that use titles from pages.

As said, I'll try to find some time by the end of the week 💪

from mkdocs-material.

1. I don't know what you're referring to when you say you were forced into doing a major release. MkDocs didn't cause any breakage, especially one that would force dropping older versions of MkDocs. That was done voluntarily to pick up new features.

2. My point isn't about that anyway. By pinning versions, you're trading a possible theoretical breakage for a guaranteed breakage. Please run pip install 'mkdocs>=1.6.0' 'mkdocs-material' and observe what happens. That's kinda nightmarish just by itself.

from mkdocs-material.

I've checked our changelog and it was 2.0 and 3.0. 3.0 turns out be updating to MkDocs 1.0, so all good, long ago, so my memories were probably faded 😅 Regardless, the title handling that came with MkDocs 1.5 was broken for almost a year now, which is why I decided to limit versions to be more careful.

Please run pip install 'mkdocs>=1.6.0' 'mkdocs-material'

Jup, that is expected. As I've outlined in mkdocs/mkdocs#3636 (comment), my opinion is that themes and plugins should define with which version of MkDocs they are compatible. In our installation guide, we do not list in any place that you should install mkdocs explicitly – au contraire, you should install mkdocs-material and let us pick the supported latest version. This way, we can ensure that all plugins work, and yes, we will upgrade to 1.6, but with tens of thousands of projects using our software, we need to be careful about that ✌️

from mkdocs-material.

Again,
version pinning for libraries simply doesn't work well enough in pip to be able to recommend it to anyone. Each library that does this turns the whole stack more and more into a fragile house of cards.

from mkdocs-material.

Again, it is only a temporary solution, until we found the time to test it with the entirety of our code base. Please be patient, we will handle this in the coming days. Thank you for your understanding ☺️

from mkdocs-material.

No, I did not, because it's nothing we recommend doing on our documentation:

pip install mkdocs-material

This will automatically install compatible versions of all dependencies: MkDocs, Markdown, Pygments and Python Markdown Extensions. Material for MkDocs always strives to support the latest versions, so there's no need to install those packages separately.

Is it expected that, until the end of days, users will occasionally install mkdocs_material-9.5.2 because that was the last version that supported all versions of MkDocs?

Nope, as said in #7076 (comment), we're working on it!

from mkdocs-material.

Additionally:

Is it really expected to you that pip does the following:

I agree that the output is rather ugly. That's something that should be reported upstream to the maintainers of pip.

from mkdocs-material.

Thus, I believe mkdocs should also be a direct dependency, so we can explicitly manage its version as well.

So we should remove mkdocs from our requirements.txt? I kindly disagree, because Material for MkDocs ships many plugins that rely on MkDocs APIs. By removing our explicit version requirement, Material for MkDocs might break unexpectedly when MkDocs releases a new version.

Once we upgraded, my suggestion would be to change the requirement to:

mkdocs~=1.6

This would get us all new releases except for the next major release. Setting it to 1.5.3 was just a temporary safety net. Please also understand that we have many, many non-technical users that have no idea how Python version managment works, so essentially, we're trying to make it simpler for them by providing this safety net.

from mkdocs-material.

Fair, maybe it's a rare example of something that doesn't exactly fit into either of the two cases

from mkdocs-material.

I'm not sure if this is related, but I'm getting this error in my mkdocs workflow:

Successfully built paginate
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 165, in exc_logging_wrapper
    status = run_func(*args)
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/usr/lib/python3/dist-packages/pip/_internal/commands/install.py", line 389, in run
    to_install = resolver.get_installation_order(requirement_set)
  File "/usr/lib/python3/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 188, in get_installation_order
    weights = get_topological_weights(
  File "/usr/lib/python3/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 2[76](https://github.com/nicfv/Me/actions/runs/8807702921/job/24175351696#step:2:77), in get_topological_weights
    assert len(weights) == expected_node_count
AssertionError
Error: Process completed with exit code 2.

But it seems like mkdocs 1.6.0 and mkdocs-material 9.5.18 both installed successfully.

from mkdocs-material.

For future reference, all of these really tight dependencies are making it really hard to upgrade packages independently. It's not clear to me there's a high value in doing this.

For example, I'm still blocked because mkdocs-techdocs-core requires a very specific version of mkdocs-material:
https://github.com/backstage/mkdocs-techdocs-core/blob/main/requirements.txt

Obviously that's it's own problem (using exact dependencies).

But anyways, by changing your requirements to be mkdocs ~1.6, you're saying that mkdocs-material is no longer compatible with 1.5. This isn't true and is unnecessarily restricting the allowable solve space.

Instead, it would be better to be mkdocs >= 1.5

Since you all are really insistent on not allowing future versions of mkdocs without a manual step, you could do mkdocs >= 1.5 && mkdocs < 1.7 (and then you can keep bumping both the minimum and maximum support level as necessary)

from mkdocs-material.

@intentionally-left-nil I understand your concerns. However, while the community edition should still be compatible with 1.5, Insiders is definitely not, and we try to keep it synchronized to ease interop and switching back and forth for our sponsors. Our goal is to stay up-to-date with the latest version of MkDocs, as it's currently our most critical dependency.

Additionally ~=1.6 is not really a tight version constraints, as it includes all versions until < 2. All version constraints in our requirements.txt are limited to the current major version, which is absolutely reasonable and standard in the software world. Major versions are meant for breaking changes, so keeping it to the current major version is sensible.

from mkdocs-material.

All version constraints in our requirements.txt are limited to the current major version, which is absolutely reasonable and standard in the software world. Major versions are meant for breaking changes, so keeping it to the current major version is sensible.

Again, I really recommend reading https://iscinumpy.dev/post/bound-version-constraints/ and https://hynek.me/articles/semver-will-not-save-you/. For Python libraries, use only lower bounds whenever possible; for apps (root of the dependency tree), use pins (lock file or even manifest pins + lock file). Clearly, mkdocs-material is not an app under this definition. Limiting to current major versions is common in, e.g., Node.js, but Python isn't Node.js (see the first article), and SemVer is a theoretically great concept but can in practice only serve as a hint at best (see the second article).

@pawamoy also summarized it very well: #7076 (comment)

from mkdocs-material.