Skip to content

Conversation

@almarklein
Copy link
Collaborator

@almarklein almarklein commented Jul 8, 2025

Context

Previously, Pygfx automatically distinguished between opaque and transparent fragments on a per-fragment level. With the new blending in Pygfx (current main), this distinction will have to be made per object. This means that somewhere a decision will have to be made whether an object is opaque or transparent.

The PyGfx material.alpha_mode is the easy way to control this behavior. The two values that make the most sense for Fastplotlib are:

  • 'auto': applies blending, and writes depth when material.opacity is 1.
  • 'blend': applies blending but does not write depth: relies on the correct render order.

This PR uses 'blend' for all WorldObjects. We can use material.render_queue to control the order of axes, grids, legends etc, with respect to other objects. This means that they do not write to the depth buffer.

Using 'auto' would also work, but self-intersecting objects like noisy lines or scattered points would be prone to show artifacts in semi-transparent regions (e.g. the aa edge).

If we add support for 3D objects (e.g. meshes) these should probably use `alpha_mode='auto' or 'opaque'.

Places where users can introduce transparency

(Let me know if I missed one!)

  • the graphic's alpha argument.
  • alpha in colormaps.
  • alpha in a uniform color
  • alpha in per-point colors.
  • alpha in image pixels.

Ideas (at the start of this PR)

Some ideas, some compatible, some mutually exclusive:

  • The alpha value passed to the graphic could be used to set world_object.material.opacity.
  • Also allow RGB colors, and maybe even promote that a bit over RGBA.
  • Automatically make a graphic transparent when alpha < 1. Easy to do either in fpl or pygfx. Covers quite some use-cases.
    • Somewhat intuitive: people who wonder why their transparent image is opaque may set alpha and see that it works then. Then they can set it to 0.999 to fix it.
  • For other cases (e.g. images or per-point colors), require setting alpha_mode.
    • A bit harder on users, but easy enough to document?
  • Or check the alpha values in images and colorlists, and automatically set the alpha_mode when values < 1 are present.
    • Makes it easier for users, but adds complexity and possible source of bugs.
  • Default to "dither", which always works. Then users can set alpha_mode="blend" for smoother results of transparent objects.

cc @Korijn because this is is a topic that touches both fpl and pygfx

@kushalkolar
Copy link
Member

kushalkolar commented Jul 8, 2025

Thanks! Will check this in detail later today, but I wanted to mention this: #754

We want to make alpha a separate graphic feature and thus an entirely independent Graphic property (in general we're trying to make the constructor as symmetric as possible with the settable properties, just like pygfx) . This would probably be a good time to do that?

@almarklein
Copy link
Collaborator Author

We want to make alpha a separate graphic feature and thus an entirely independent Graphic property

Good to know this was already on the agenda! Yeah I think it makes sense both from an API perspective, but also technically; changing a color is fine, but changing an alpha value might affect how you want to blend things.

@Korijn
Copy link

Korijn commented Jul 11, 2025

Then they can set it to 0.999 to fix it.

This seems a bit awkward. I foresee users would complain about having to do this.

  • Or check the alpha values in images and colorlists, and automatically set the alpha_mode when values < 1 are present.

From your list of options, this seems most straight forward.

  • Default to "dither", which always works. Then users can set alpha_mode="blend" for smoother results of transparent objects.

After thinking about this for a while, I guess what I would expect from a 2D plotting library as a user:

  • All graphics have alpha_mode="blend"
  • Graphics are drawn in the order that I add them to the figure, transparent or not, so the depth buffer is kind of ignored I guess

I'm not sure if fastplotlib also does 3D plotting, but in that case I would expect it to behave as pygfx does naturally.

By the way, I appreciate you calling for my thoughts, so I'm sharing them here, but I really have not been keeping up with fastplotlib's development for a while, so take it with a grain of salt.

@kushalkolar
Copy link
Member

  • Or check the alpha values in images and colorlists, and automatically set the alpha_mode when values < 1 are present.

From your list of options, this seems most straight forward.

I'm leaning towards this as well.

After thinking about this for a while, I guess what I would expect from a 2D plotting library as a user:

  • Graphics are drawn in the order that I add them to the figure, transparent or not, so the depth buffer is kind of ignored I guess

When using an orthographic projection graphics are stacked in z.

I'm not sure if fastplotlib also does 3D plotting, but in that case I would expect it to behave as pygfx does naturally.

Everything is 3D just like pygfx, perspective camera for everything. We just don't have a high level API for meshes yet, I've rarely used meshes myself so I hope someone else would be able to contributre a high level API for that. Or maybe meshes are so specific that the pygfx level of abstraction is the best for it, I don't know yet.

@almarklein
Copy link
Collaborator Author

When using an orthographic projection graphics are stacked in z.

Do you mean that the objects are offset in their ob.local.z?

Is it possible to clearly identify whether the current graphic is in a 2D or 3D scene? If that is the case we could indeed default to blend.

Or maybe meshes are so specific

Haha!. Some Pygfx users consider meshes the "normal object" and think all the non-meshes are special 😉

@Korijn
Copy link

Korijn commented Jul 15, 2025

When using an orthographic projection graphics are stacked in z.

And what determines the stacking order? Insertion order?

@kushalkolar
Copy link
Member

kushalkolar commented Jul 15, 2025

When using an orthographic projection graphics are stacked in z.

Do you mean that the objects are offset in their ob.local.z?

Yup!

When using an orthographic projection graphics are stacked in z.

And what determines the stacking order? Insertion order?

Yup. Though the z can be set when added, or later at any time.

Is it possible to clearly identify whether the current graphic is in a 2D or 3D scene? If that is the case we could indeed default to blend.

If camera.fov == 0

You can switch between them and it's something I do all the time when looking at low dimensional projections with lines and scatters.

@almarklein almarklein mentioned this pull request Sep 2, 2025
4 tasks
@almarklein
Copy link
Collaborator Author

Tests pass again! (or am I missing something?)

@almarklein
Copy link
Collaborator Author

oh lol, all tests are skipped 😆

@almarklein almarklein marked this pull request as ready for review September 2, 2025 10:11
@almarklein almarklein requested a review from clewis7 as a code owner September 2, 2025 10:11
@github-actions
Copy link

github-actions bot commented Sep 2, 2025

📚 Docs preview built and uploaded! https://www.fastplotlib.org/ver/alpha

@Korijn
Copy link

Korijn commented Sep 2, 2025

How are you feeling about this refactor @almarklein, now that you can experience the revamped transparency system as a user?

@almarklein
Copy link
Collaborator Author

How are you feeling about this refactor @almarklein, now that you can experience the revamped transparency system as a user?

I'm feeling pretty good about it in general.

I felt rather disappointed that we cannot abstract the whole transparency thing away from the user. But after having done quite a bit of research on the topic I'm confident that even with the more advanced methods (which would consume more memory and/or be slower) it'd still not be fool proof. That's what it is.

So the only way to go is to shove the problem on the user 😄 , but as gentle as possible, and providing multiple different options to deal with it. I think we've now now done that pretty well, providing pretty powerful options, but also easy-to-use shorthands with alpha_mode.

As for this PR in Fastplotlib, I think it helps a lot if we can determine/assume the scene to be 2D, so we can just use 'blend'. If no object writes depth, Pygfx will not create the depth buffer, so it'd be a bit lighter too.

@kushalkolar
Copy link
Member

if we can determine/assume the scene to be 2D, so we can just use 'blend'. If no object writes depth, Pygfx will not create the depth buffer, so it'd be a bit lighter too

3D is used often, easiest way to check is if camera.fov > 0

This will also need some refactoring of the graphic features, will add more thoughts soon

@almarklein
Copy link
Collaborator Author

I implemented a GraphicsFeature for alpha and alpha_mode. This indeed helps to make the code cleaner (less duplication).

However, setting the alpha / alpga_mode in figure[0, 0].add_scatter(.. , alpha=0.07) does not work. While it does work for visible which is also a graphics feature. Any idea why? I may have missed some init step or something.

@almarklein
Copy link
Collaborator Author

However, setting the alpha / alpga_mode in figure[0, 0].add_scatter(.. , alpha=0.07) does not work. While it does work for visible which is also a graphics feature. Any idea why? I may have missed some init step or something.

Fixed; these were set in _set_world_object()

@almarklein
Copy link
Collaborator Author

Also updated screenshots. Two examples have a diff above the threshold: image_widget_single_video and image_widget_videos. I tried copying the screenshots from the screenshots CI build, but the screenshots are the same (git status is empty). Any ideas?

@kushalkolar
Copy link
Member

Also updated screenshots. Two examples have a diff above the threshold: image_widget_single_video and image_widget_videos. I tried copying the screenshots from the screenshots CI build, but the screenshots are the same (git status is empty). Any ideas?

ImageWidget screenshots aren't being generated, that regeneration action is failing: https://github.com/fastplotlib/fastplotlib/actions/runs/18008358652/job/51234040316?pr=873

Must be because newer wgpu uses newer imgui, so I'm going to try and put the fastplotlib imgui option to just point to "wgpu[imgui]", will see if that works.

@kushalkolar
Copy link
Member

kushalkolar commented Sep 26, 2025

updated to latest imgui, uses the new texture registering method, uses a merged font so regular text and font awesome icons can be used together without push/pop font. @almarklein I think one last thing is to set the render order so that axes lines and points are on top when camera.fov == 0, but not on top of any overlays:

This is examples/gridplot/gridplot.py

image

If it doesn't cause a performance hit you could put a check for camera.fov in Axes.update:

        if self._plot_area.camera.fov == 0:
            # set render order for ruler line, points, text
        else:
            ...

A better way might be if it's possible to add an event handler when camera.fov changes?

@kushalkolar
Copy link
Member

All tests against pygfx@main are passing so I will merge, can do the axes thing in another PR, #903

The tests that are failing are all against the latest release of pygfx, which does not work with the latest imgui so it's fine to skip.

@kushalkolar kushalkolar merged commit 0e75f6c into main Sep 27, 2025
40 of 52 checks passed
@kushalkolar kushalkolar deleted the alpha branch September 27, 2025 06:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants