3.2.x Release plans

[wshanks]

In #199 (use pyproject.toml) and #200 (drop official support for Python before 3.8), one topic that has come up is what to release next. I don't think either of those PRs requires a new release because they do not change the code, so there should not be much user impact from pulling a new release with those changes in it. That does make me wonder though -- what changes do we want in a new release? Or releases? Do we want a 3.1.8, 3.2.0 or 4.0.0?

#199 does have one change I would like to see in a release which is dropping future as a dependency. I had previously opened #168 to do that. It is not that urgent though because I was able to get future to patch the vulnerability it had reported which was causing environments with uncertainties installed to be flagged as vulnerable by security scanners.

Besides that, I just wanted to freshen up the project generally -- get the CI going, set up more standard packaging/linting, address some deprecation warnings, look into bug fixes, etc. I didn't have any pressing need to add new features or breaking changes.

[jagerber48]

I agree that it would be nice if the next release was “freshen up the package generally”. If that was the change we did I would imagine that to be a minor version bump. But it sounds like we’re going to move to pyproject.toml and possibly break code for Python < 3.7 or 3.8. Breaking code for previously supported versions sounds like a major version bump to me.

That’s a little bit of a shame because I imagine some more of the changes we have in the coming months will also be breaking changes that will be major version bumps. So we could see uncertainties bump up to like 6.y.z or something. If we don’t put more effort towards “bundling” breaking changes. Examples are that we might make breaking formatting changes. We also might deprecate or delete ufloat in favor of UFloat. Not sure what other breaking changes might be planned.

But maybe we shouldn’t consider a few major version bumps to be a big problem and we should just proceed without burdening ourselves with trying to “bundle” the changes.

[newville]

@jagerber48

Ok, that sounds good. so we're not considering breaking python <= 3.5 to warrant a major version bump. Fine. So next version with at least "package clean up" will be 3.2.0.

I think we may not be using words to mean the same thing. I would like to avoid conflict, but this is kind of exhausting.

I am +1 on removing Python2 cruft and using f-strings in the code for 3.2.0. That does not break anything.

Why would we support versions of Python that are past the end of life? Zero people are using a Python version that is past end-of-life and also needs any yet-to-be-released version of uncertainties.

We do not need to intentionally break things for Python versions that are past end-of-life. We certainly can do that, but we should carefully decide if changes would break version N-1 (currently, that would be Python 3.7).

We do not need to make it easy for someone to install into a version of Python that is past end-of-life.

[jagerber48]

I think we may not be using words to mean the same thing.

@newville Sorry, I may be using terms in non-standard ways. I'm still learning standard practices for "releasing" code and putting version numbers on releases so I might be saying non-sensical things. Your last posts made good sense to me and I agree with all of them!

Consider what follows to be me trying to learn, rather than me trying to advocate for any direction of action on uncertainties (and treat it accordingly, e.g. if you don't have time to teach me things then don't! I don't want to exhaust anyone with my questions/learning).

Nowhere am I advocating that we should support python <3.8. I was more wondering whether the act of dropping support (either by changing metadata in setup.py or pyproject.toml or actually including source code that would cause exceptions on those systems) should warrant a MAJOR or MINOR version bump. Reading through e.g. this issue I think some hardcore advocates of semver (people who err on the side of calling smaller changes to be MAJOR version bumps than you might naively think) would consider dropping support for old versions of python to be a MAJOR version bump. Nonetheless, I see no need to listen to those points of view. I think they end up being pretty unpragmatic.

If you are happy to call dropping support for python versions past EOL to be MINOR, rather than MAJOR, version bumps, and no one else has objections, I'm happy to do so also.

We do not need to intentionally break things for Python versions that are past end-of-life. We certainly can do that, but we should carefully decide if changes would break version N-1 (currently, that would be Python 3.7).

This is actually confusing to me. Why do we need to be careful about version N-1? Why not be careful about N-2? I would think that if we support version >= N then we can carelessly make changes that break version N-1.

[newville]

Well, some of those folks over at semver suggest changing the minor version of dependencies requires a major version update. If that were the case, then many releases of many packages would be bumping their major versions, probably multiple times a year. And then all downstream packages would be increasing their major versions multiple times per year.

numpy is at version 1.26. scipy is at 1.12, matplotlib is at 3.8. So, there are at least 26 releases that any "scientific python" package would have to deal with, maybe some could overlap. How is possible that scipy is at 1.12? Pandas is at version 2.2 and scikit-learn is at 1.4? Shouldn't every single scientific/data-science package be at MAJOR version 26 or higher -- and some would likely be at 50. Those fine folks in the semver conversation appear to be working in a different universe.

We do not need to intentionally break things for Python versions that are past end-of-life. We certainly can do that, but we should carefully decide if changes would break version N-1 (currently, that would be Python 3.7).

This is actually confusing to me. Why do we need to be careful about version N-1? Why not be careful about N-2? I would think that if we support version >= N then we can carelessly make changes that break version N-1.

I invite you to look at https://devguide.python.org/versions/ and recognize that Python versions typically come at one-year intervals, and have lifespans of about five years.

As an example, support for Python 3.7 ended in June 2023. Support for Python 3.8 is scheduled to end in October 2024. So, in or shortly after October, many libraries (including this one, I imagine) will stop testing with Python 3.8, just as they stopped testing with Python 3.7 last summer. But, unless there is a truly compelling reason, releasing a version of a library in October 2024 that breaks Python 3.8 would be sort of unkind ... some people are a bit late to make such transitions (or are stuck waiting for IT people to upgrade their systems).

Now, it might be that there truly is a compelling reason to no longer support a version. I think these examples are rare, and when you get to the point that you want to break N-1, are you sure you can't wait a few months after support for it ends?

Similarly, expecting to support the latest Python release one week after it is released is probably a bit much to ask. But if you've gone 6 months and there is no plan for supporting the current version, there is a problem.

[jagerber48]

Thank you very much for all of that information. Yes, I agree some of the semver conversation seems to be happening in a different universe. Or at least in a universe where all projects have dozens of major versions.

And what you say in the second part about python versions is also clarifying. Here's my understanding of what you said. It sounds like the policy would roughly (or exactly) be something like

• uncertainties nominally tests/supports any version of python that is not "end-of-life", however
• uncertainties may hold off on breaking python versions even after they go EOL by a few months or up to a year
• uncertainties may not have testing/support for newly released python versions until a few months after that python version is released.

Both of the two latter points are a intentionally a little loosely defined to allow flexibility. Does this sound right to you?

[wshanks]

Taking a more pragmatic view of versioning than extreme semver, I think it is okay to break compatibility with older versions of Python or another dependency in a minor release. A patch/micro release I would expect to be able to update to without any new errors or problems, but with a minor release I expect possibly new code and so possibly new dependency versions. I would still expect code that worked with X.Y to work with X.(Y+1). Part of why I am okay with the supported dependency versions changing is that I would expect pip to help me with the upgrade -- if the new version drops support for the Python version I am using, I would expect the python-requires in the package metadata to be updated and for pip to refuse to update to that new version, so a new pip install uncertainties should still work in my environment. It might just not install the latest release any more. (And likewise if the minimum version of numpy increased a new pip install would help with that).

[newville]

Thanks @wshanks -- I agree with all of that.

I think we should merge all of #197, #199, #200, and #202. I would probably "squash-and-merge" some of those. I think order does not matter. Any objections to that @wshanks @andrewgsavage @jagerber48? I'll intend to merge in a day or two unless there are objections. If someone else wants to do the merging, that's OK with me.

There were some doc changes folded into #200, but I think we might need a few more doc changes too. I will try to tackle some of those after #200 is merged.

[newville]

@andrewgsavage @wshanks @jagerber48 @lebigot

Sorry for the delay here, and thanks very much to @andrewgsavage for the help and patience with getting the docs updated.

I believe that we have merged all the PRs we listed for version 3.2.0.

Are there any PRs missing or Issues that need to be addressed before tagging and releasing 3.2.0?

[andrewgsavage]

I've put in PRs for changes and numpy2, that's all I can think of before releasing.

[newville]

@andrewgsavage @wshanks @jagerber48 @lebigot. I think the consensus (but please correct if you disagree) that

a) we're really close to releasing 3.2.0, basically all features in place.
b) following PR #223 and #215, we should write the changelog by hand this time to get the release out sooner rather than later and then consider using a better changelog tool in the near future.

If you all agree, I will try to work over the next couple of days on a proposed Changelog for 3.2.0 here (in this conversation) for comments/suggestions that can then get merged as #223.

If you have any (any) comments on the docs as they are currently rendered at https://lmfit.github.io/uncertainties/, it is not too late to change them. I am not super-confident in any part of this (is the theme acceptable, is the user guide still too long, ...).

[andrewgsavage]

I think I've caught all the user facing changes in that single line. There may be some more in the previous commits before we took over, although it looks like not (last changelog update was 44fd4f3

[wshanks]

Minor point on the documentation: It would be nice if the sidebar on the main page had the table of contents instead of current version, installation, development version. Maybe it could still have those things as well. That sidebar is what gets collapsed into the hamburger menu on mobile and it feels weird to expand it and not see access to all the sections of the documentation.

Besides that, I think the docs look good. The theme is much more modern looking than the previous one. Another minor point is that the header has a white background on desktop (on mobile it is all dark). Also, the shadowed text of "uncertainties" in the header looks dated to me, so I would not mind seeing it updated but I don't feel strongly enough to suggest an alternative -- just noting that I am not attached to the current version.

[newville]

Proposed Changelog for 3.2.0:

• Update unumpy test to be compatible with numpy 2
• Mark docstrings with backslashes as raw strings (Mark docstrings with backslashes as raw strings #226)
• Update documentation.
• Move tests to tests folder (Move tests to tests folder #216)
• remove 1to2 and deprecations (remove 1to2 and depreciations #214)
• Moved from setup.py to pyproject.toml (Transition from setup.py to pyproject.toml #199)
• Drop support for Python versions before 3.8, including Python 2 (Drop official support for Python versions before 3.8 #200)
• Moved code development to lmfit organization, with 4 maintainers.

[newville]

@wshanks That looks great to be. I like the idea of starting with:

Version 3.2.0 is the first release of Uncertainties in nearly two years and the first minor release in over five years. It marks the beginning of an effort to refresh and update the project with a new and expanded team of maintainers.

Adding all this to CHANGES.rst would be good -- and the change in length and styling is a hint of the "effort to refresh and update".

I would be okay with linking that to the documentation. I rarely look in the docs for ChangeLog, and expect that to be with the source code, so I would also be okay with leaving it out.

[jagerber48]

This all looks good to me.

I haven't yet had a time to look closely at a docs, but at a quick glance they look fine for release. I view it as pretty low cost to make changes to the docs. There's no backwards compatibility issues or anything so you can just freely make improvements as they are noted.

Yes, a having a good changelog system will help keep us organized about what's been done and what we want to continue to do.

[andrewgsavage]

That looks good. Are you going to make the release @newville ?

[newville]

@andrewgsavage Sorry for the delay. I can and will do that, starting now. To be clear, I do not want to overstep the authority of the "group of maintainers", but am willing to do the release - it seems we all agree. I will (try to) do the following steps:

1. merge changes #223
2. merge add global TOC to doc index page #228
3. add to Changes as from @wshanks above, probably just pushing this to the master branch.
4. tag 3.2.0 and push that to Github, and use Github Releases to tag that (and including the Changes text).
5. build docs locally and push that to the Github-Pages location.
6. build the tar.gz and whl files with python -m build and try to push those to PyPI.

I expect that to all go well. I am not 100% confident how the generation of the readthedocs version of the documentation will go.

If I run into any trouble, I'll post here.

[newville]

OK, merged, but then added a PR to update CHANGES.rst and version in __init__.py. So, #229 needs a review and merge before tagging and releasing 3.2.0.

[newville]

@wshanks @andrewgsavage @jagerber48 OK, I merged and tagged 3.2.0 on Github and pushed releases to PyPI. Thanks everyone!

I don't know how to tell if the automatic doc generation at readthedocs got triggered. I think the keys were set appropriately, but I am not very familiar with readthedocs. Does anyone know how to get the latest docs there?

[andrewgsavage]

The docs haven't changed so looks like not. https://uncertainties-python-package.readthedocs.io/en/stable/

It failed to build 1hr ago https://readthedocs.org/projects/uncertainties/builds/

[andrewgsavage]

is there anything else that needs doing before a 3.2.1 release?

[newville]

I started a tweak to README.rst and pyproject.toml to point to the doc location as http://uncertainties.readthedocs.io.

I will make a PR later today or early tomorrow. With the tarballs now correctly including all of the test codes, I think that could be close to 3.2.1.

[newville]

I think the master branch could be tagged as 3.2.1, released, and pushed to PyPI. I'm away from my desk at the moment. I would be willing to do this tomorrow or Monday if there are no objections. I would not object to someone else doing this either. Preferences?

[andrewgsavage]

sounds good, go for it

[newville]

I tagged 3.2.1, and can install and build but cannot upload packages to PyPI, seeing errors about the "long_description", possibly related to my changes to README.rst.

> twine check uncertainties.3.2.1.tar.gz

is giving me

Checking dist/uncertainties-3.2.1.tar.gz: FAILED
ERROR    `long_description` has syntax errors in markup and would not be rendered on PyPI.
         line 115: Warning: Inline interpreted text or phrase reference start-string without end-string.

which I think I have never seen before. I'll have to investigate.....

[lebigot]

I haven't followed this discussion closely, but I just had a look at the documentation (User Guide): many, many Sphinx/Markdown backticks, which are intended to cite code/functions (cosh, *, etc.) have been transformed into another symbol (apostrophe?), which makes them harder to read, in my opinion, if only because of the additional, useless characters ('*', '**', '-'…). I'll let you decide what you prefer to do. :)

[lebigot]

In particular, but it's in many other places (not everywhere, through, strangely enough). For example:

basic mathematical calculations (‘+’, ‘-’, ‘*’, ‘/’, ‘**’)

Comparison operators (‘==’, ‘!=’, ‘>’, ‘<’, ‘>=’, and ‘<=’)

Etc.

The single quotes are originally backticks (which introduce code, in Markdown/ReST). It's unusual to how single quotes, so I'd either put back the backticks (because all of these are Python code, including the math functions), or I would remove all these new quotes, for legibility (and as per the usual convention).

[newville]

I have no preference for backticks or single quotes to indicate that these are strings representing operators and functions. I think they can not be left bare, as they will be interpreted as formatting instructions.

[lebigot]

The whole idea is to precisely use formatting instructions: I had put backticks around anything found in Python cod, since that's the purpose of Markdown backticks. My vote goes to either go back to this, or to remove quotes altogether (no visible backticks either).

[newville]

@lebigot "The whole idea is to precisely use formatting instructions"

Using ``acos``, ``acosh`` instead of "'acos'", the result is rendered a different colored background, appearing as: acos, acosh , .... Whether that change in background color is more jarring than quotes is a matter of taste, and whether that background color clearly signifies "this is code" compared to the quote signifying "this is a string in code" is a matter of expectation (and maybe sophistication of a screen-reader for those with poor eyesight). I will change to double backticks, though I don't think it actually changes readability much.

More importantly, I would also say that "the whole idea" is actually to communicate effectively, make incremental improvements, being encouraging of others contribution, and being harder on yourself than on others. The risk of being pedantic about others contributions is that you then have to meet those standards too.

The paragraph starting at https://github.com/lmfit/uncertainties/blob/master/uncertainties/__init__.py#L180 in which you wrote a few years back has:

Comparison operations (>, ==, etc.)

while the following paragraph at the paragraph starting at https://github.com/lmfit/uncertainties/blob/master/uncertainties/__init__.py#L186, has:

The result of a comparison operation ("==", ">", etc.)

PR #240 changes the user guide to use double backticks. It also change those code comments to use double backticks uniformly.

When this gets merged, we can tag and release version 3.2.1.

[lebigot]

The legibility of rendered "backtick" code is a fair point: I agree that the result is a matter of taste, and that's why I also suggested to remove quotes (or displayed backticks) altogether.

I only spent time on sharing some feedback because I thought that a clear and consistent use of code indicators could indeed help effective communication with users. Of course, while I always strive for consistency myself, I can make mistakes, and I'm happy that others spot them, because it does help with effective communication; so thank you.

I haven't followed most discussions, in the last few months (that was the whole idea of passing the torch to you guys), so I commented on the only point that fell in my radar, really, but I can see that you worked a lot and delivered quickly. Keep up the good work!