If we choose to do this (which is not something I'm not really convinced of, as I've mentioned in #30559, but I agree it's low cost), then I think the version numbers should just match the Matplotlib release numbers where backend API changes occur and we should just have a table listing changes to the backend API, e.g. the current backend API could be 3.8 if (hypothetically) mpl3.8 was the last release where a breaking change occurred.

I believe we should have a dedicated backend version scheme independent of matplotlib version numbers, because we want to communicate EffVer semantics on the backend. That’s not possible when just taking the matplotlib version number.

My vote would be that going with Matplotlib major versions would be a lot less confusing. But maybe there are other projects that version subsets of their APIs differently from their project version?

When you look at REST APIs, explicit versioning of the API is common and good practice. I'd argure that our backend API is semantically quite the same.

Can you elaborate - I don't see the analogy. We have one backend API, and its versions will only change with Matplotlib versions? Or am I misunderstanding?

The point is that you explicitly version the API. With EffVer (or SemVer) on the API, you have a direct insight on whether and how much the API has changed.
Also, using the matplotlib version would be semantically quite subtle: One matplotlib version supports multiple API versions. For example, we introduced a new optional hatchcolor parameter in 3.11. Does the 3.11 API mean the parameter must be there or is it ok to leave it out? You could technically argue that any API that is accepted by matplotlib 3.11 counts as 3.11 API, but that is less expressive. A backend is then not able to declare that it supports the new hatchcolor parameter. The other solution would be that 3.11 means the newest API supported by 3.11 (here with hatchcolor), but that‘s difficult to define and document clearly.

I am in favor of adding a version on the backends (like we do with on the fontcache). The point is for backends to communicate to us what API they are exposing not what API we are promising.

Some questions:

• do we need full semantic or just a incrementing number
• how can a backend declare to support more than one version?

I will defer to @timhoffm on if you want any elevated review on this.

I think this would be easier to reason about if it follows the MPL version number. I do see the analogy to the font cache that this is for others to inform us about what version they want to use and not our API promise to them.

One question I have is in regards to whether this could be indicated somehow in the backend registration process instead? When a backend wants to register with us, they could declare what features/API they want to agree to during that process and we wouldn't need explicit versioning, but rather some mechanism to indicate feature sets supported by a backend.

I think the font cache is internal to stop us from trying to use an old font cache on a new version of Matplotlib. We could have used any unique ID?

Perhaps this issue would be clarified by a sample of what we think downstream libraries will communicate to their users about these backend versions?

I am in favor of adding a version on the backends (like we do with on the fontcache). The point is for backends to communicate to us what API they are exposing not what API we are promising.

Exactly.

* do we need full semantic or just a incrementing number

just an increasing number is fundamentally enough. However, we may consider using EffVer (macro, meso) to communicate how much change happens between versions, in particular how much effort it will be to migrate to the next version - this is what I've intended with the number scheme in this PR. (Note: micro versions are not needed / don't make sense for an API).

For example, I'd consider the addition of hatchcolor to draw_path_collection() a meso change, whereas e.g. replacing all individual style kwargs by a Style dataclass would be a macro change.

If we think we don't need that distinction a simple increasing number is good enough. But I would then really just count and not use a matplotlib version number.

* how can a backend declare to support more than one version?

They can't, and in general it's not trivial, because to leave us with all degrees of freedom, if a module wanted to provide multiple API versions, they'd have to do each API under a separate namespace.
But IMHO this is not needed. The only benefit would be that a third party backend could support the newest matplotlib and very old versions. But I anticipate that we'll provide extremely long backward compatiblity. Since we have to provide compatibility for multiple API versions anyway, there's little additional effort to have quite long phases of backward compatibility.

One question I have is in regards to whether this could be indicated somehow in the backend registration process instead? When a backend wants to register with us, they could declare what features/API they want to agree to during that process and we wouldn't need explicit versioning, but rather some mechanism to indicate feature sets supported by a backend.

While that is possible, I think this would be over-engineering. We would need to define features or feature-sets marked with distinct specifiers. Essentially, the proposed versioning is a coarse feature set. Every more granualar distinction would need to be defined up-front by use (what can backends opt-into or not).

Looking at this again I still don't see how this really helps.

Right now:

• on mplcairo's side, I know (because I can look at matplotlib's source) what kind of capability sniffing is performed by matplotlib, and I can decide, based on what versions of matplotlib I want to support, how mplcairo's API needs to look like (in theory I could even expose different APIs depending on maptlotlib.__version__, not that I actually bother with that).
• on matplotlib's side, if we decide to change an API while providing a transition period (if we don't care about the transition we can obviously directly break things), we need to perform capability sniffing for a while, which is slightly annoying but can be encapsulated in a single helper function.

With the proposed change here:

• on mplcairo's side, I need to commit to a single API version, so the range of matplotlib versions I can support actually gets (likely) narrower (e.g., until recently mplcairo actually (mostly) supported all matplotlib versions back to mpl2.2 -- which was the original version for which I wrote mplcairo -- because the shims were not that costly to keep).
• on matplotlib's side, API sniffing (if _backend_supports_api(backend): ...) is replaced by checks for backend.MATPLOTLIB_BACKEND_API_VERSION, which admittedly is slightly cleaner, but really not so much in my view. (I would actually support making the APIs more sniffable, e.g. by requiring method signatures to be introspectable.)

(By the way, the claim that the backend API has not changed since mpl1.0 is almost certainly not true.)

I know (because I can look at matplotlib's source) what kind of capability sniffing is performed by matplotlib

IMHO we should communicate our API explicitly. "Look at our source code to understand our API" is not good enough, in particular if you want to support many versions. I minimally require that we have an explicit documentation which API is supported in which version of Matplotlib in one central place. You can do this without versioning, but IMHO attaching an ID to specific states of the API helps with clarity.

Determining how to call the third party backend from our side is a separate topic. Sniffing is fundamentally viable. It's a bit more flexile but OTOH a bit more effort on our side and more fragile - it's comparable to dynamic class attibutes. I claim that we/backends don't need that extra flexibility and life would be easier without sniffing. If you really think we need more flexibility than stating a single API version, I'd rather have the backend decleare support for predfined capabilities and we call it based on that rather than trying to sniff them out.

Look at our source code to understand our API" is not good enough, in particular if you want to support many versions.

In theory that is would certainly be better, but in practice this is simply not the case. There are APIs such as _iter_collection() which are effectively not documented but need to be understood to correctly implement a backend. IIRC there are also some rather obscure points regarding image resolution in vector output.

I'd rather have the backend decleare support for predfined capabilities

Note that there are already some methods that are here to declare capabilities, e.g. option_image_nocomposite() or option_scale_image().

Look at our source code to understand our API" is not good enough, in particular if you want to support many versions.

In theory that is would certainly be better, but in practice this is simply not the case.

That sounds like we should give up on specifying the backend API. IMHO that would be a poor choice. Only small changes would be possible, will be cumbersome (sniffing) and will lack adoption with third party backends (discoverability and documentation).
It‘s not that we‘ll overhaul the whole API tomorrow, but without specifying it and defining compatibility guarantees and migration paths., we‘ll have a hard time in improving aspects such as interactivity, rendering performance, supporting new contexts(tooling integration in IDE/notebooks is a strong selling point by now, but often requires an adapted backend), etc.

t.b.h. I don’t quite understand why declining the backend API and giving it a name in form of a version number sees so much reservation. IMHO there‘s little to loose here, but some potential good gains.
What are the concerns here? Is is not needed? Is it insufficient? Is it harmful?

I'm not saying that we should not better define the API. Quite the contrary, that would be great. (And certainly, trying to fully spec the backend API is not a fun task, and I'm not providing any help in that direction either.)
What I object to is trying to claim "this is the stable mpl backend api v1.0" when that API has never really been specced or made stable, so third-party backends will start adding MATPLOTLIB_BACKEND_API_VERSION, with the cost that the matplotlib versions they are compatible with get restricted, and without actually getting any benefit from that (because things will keep moving under them at every matplotlib release).

What I object to is trying to claim "this is the stable mpl backend api v1.0" when that API has never really been specced or made stable,

I'm happy to leave the past unspecified if we are not sure about past changes.

I would want to give the current state of the API (what is used in 3.10) an ID. - I don't really care on the exact spelling 1.0 or 0.1 or whatever, as long as it's not semantically tied to the library version number. I want this independent ID so that we can state that multiple library versions have the same API, so that it's easier to indicate changes or non-changes on the backend API.

I stating a supported MATPLOTLIB_BACKEND_API_VERSION by 3rd party backends seems overly restrictive, we can defer the discussion on wether we'll always sniff the API or come up with patterns via which 3rd party backends can declare their API.

I would want to give the current state of the API (what is used in 3.10) an ID. - I don't really care on the exact spelling 1.0 or 0.1 or whatever, as long as it's not semantically tied to the library version number. I want this independent ID so that we can state that multiple library versions have the same API, so that it's easier to indicate changes or non-changes on the backend API.

Sure, let's stick to that for now.

I just realized that another possible point of inspiration for designing this interface is the numpy array API standard (NEP 47). Obviously the backend API is not at the same scale, but the idea is the same: there's a central library (here, matplotlib) that provides some objects (canvases, renderers -- compare with numpy arrays) with an API that can be directly used by end users (well almost directly for matplotlib's case, although people do poke at canvases directly) or via helper functions, and a desire that third-party backends can also provide similar objects that can be swapped in to replace the central library's objects.
The array API changes from release to release (see e.g. https://data-apis.org/array-api/latest/changelog.html#breaking-changes) and it may be worth checking how they handle that.

@anntzer is the new wording acceptable to you?

I think this should have at least 2 approvals. So leaving open.