Skip to content

Conversation

@kushalkolar
Copy link
Member

@kushalkolar kushalkolar commented Nov 19, 2025

This ended up being more than just the cursor tool since I needed to do a few other things to make this work well 😆

Transforms

This PR adds useful functions to Graphic. map_model_to_world() and the inverse map_world_to_model(): https://github.com/fastplotlib/fastplotlib/pull/947/changes#diff-a57994ecff1cde4e05c8175265219f0b5200da144ce8733b9eb560c09bcea7ddR501-R558

This lets you map model/data space <-> world space. It accounts for any transforms (translation, i.e. Graphic.offset, scale and rotation) and maps the actual data position to world space.

Graphic.scale is a new graphic feature.

I added examples that show how all these transforms relate to each other, and how to map between them. They are in examples/space_transforms

I now forgot why I added map model <-> world, but I'm sure they will be useful 😅 .

I also added PlotArea.map_world_to_screen(), which is the inverse of map_screen_to_world(). The Cursor uses this in the following way to display tooltips in other subplots:

  1. Pointer is at a certain location in Subplot 1. We know the world space position of the pointer at this location, draw the Cursor at this location in Subplot 1. Since the pointer is over this location, we can also draw a tooltip here. Tooltips are drawn in the overlay render pass and are therefore always in screen space.
  2. Now we need to draw the cursor at the same world space location in all other subplots, this is simple. But we need the screen space positions for the corresponding world space positions in these other subplots. So we use map_world_to_screen() to know where to draw the tooltip in these other subplots. This is also used to get the pick info at this screen position.

Tooltip

Created TextBox, which is a simple text box object. Tooltip inherits from TextBox and just adds a few uesful properties.

The tooltips are managed by PlotArea, see code review comments for an explanation.

Map WorldObject -> Graphic

I created a dict that maps world object id -> Graphic.

https://github.com/fastplotlib/fastplotlib/pull/947/changes#diff-a57994ecff1cde4e05c8175265219f0b5200da144ce8733b9eb560c09bcea7ddR40-R42

Useful when manually picking the picking Texture to get picking info about a certain screen position (ex: Cursor tool to get pick info other subplots). Also used to get the pick info when updating the tooltip on every render.

Cursor

A simple cursor tool, has two modes "crosshair" which uses 2 infinite lines, and "marker" that uses a pygfx point with marker material.

This just maps the same position in world space across all subplots. I think we should keep it simple this way, unlike the ideas proposed in #662 .

Size, color, etc. are settable properties.

The cursor also sets the tooltip in other subplots that have been added to the cursor! Since we know the world space position in all other subplots, we can map_world_to_screen() and get the pick info for that position in the other subplot, and then display the tooltip in the other subplots with that pick info https://github.com/fastplotlib/fastplotlib/pull/947/changes#diff-dfc9ee2c327058ca7075a70aed1eb973aeb47e1e08e65a67c512a318ed6036aeR254-R257

@kushalkolar kushalkolar requested a review from clewis7 as a code owner November 19, 2025 12:51
@kushalkolar kushalkolar mentioned this pull request Nov 19, 2025
14 tasks
@github-actions
Copy link

github-actions bot commented Nov 19, 2025

📚 Docs preview built and uploaded! https://www.fastplotlib.org/ver/cursor-simpler

Copy link
Member

@clewis7 clewis7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kushalkolar general comments to make sure everything is properly documented/commented etc.

I know you want to merge this today, besides one logic issue with the mode, it looks good to me

The other things are less critical and can be updated in another PR. I just wanted to document them. I went ahead and linted and added the doc updates so everything would pass.

@kushalkolar
Copy link
Member Author

kushalkolar commented Nov 20, 2025

This is gonna need a bit more work.

Display tooltip at corresponding location in other subplots by manually picking at that location to get the info about the graphic at that location.

Basically:

If system cursor is on subplot A, and we want to display tooltip at same world space location in subplot B:

# system cursor over position in subplot A, get world pos
pos = subplot_A.map_screen_to_world(ev)

# get corresponding screen space position in subplot B
x, y = subplot_B.map_world_to_screen(pos)
# get pick info of this screen pixel
pick = subplot.get_pick_info((x, y))

if pick is None:  # no graphic at this screen pixel in subplot B
    tooltips2.visible = False
    return

# display graphic metadata or anyrelevant info in the tooltip in subplot B
info = pick["graphic"].metadata
tooltips2.display((x, y), str(info))
manual_picking-2025-11-20_04.29.45.mp4

@kushalkolar
Copy link
Member Author

@clewis7 I'm thinking about changing who "owns" tooltips. Right now Figure owns tooltips and can auto-display them. I'm thinking of a tooltip being owned by each Subplot instead, so we can have a method:

Subplot.display_tooltip(pos: tuple[float, float], space: Literal["world", "screen"], info: str):
    ...

Tooltips are drawn in the overlay render pass which is in screen space, so the tooltip position in the end always has to be in screen space. However it is useful to think of them in world space since that's where the graphics are. It'll just be much easier to know the position of a graphic in world space where you want to show the tooltip, and the Subplot.display_tooltip() will project that pos to screen space under automatically.

@kushalkolar kushalkolar changed the title new cursor tool, basics work cursor tool and better tooltips Nov 21, 2025
@clewis7
Copy link
Member

clewis7 commented Nov 21, 2025

@clewis7 I'm thinking about changing who "owns" tooltips. Right now Figure owns tooltips and can auto-display them. I'm thinking of a tooltip being owned by each Subplot instead, so we can have a method:

Subplot.display_tooltip(pos: tuple[float, float], space: Literal["world", "screen"], info: str):
    ...

Tooltips are drawn in the overlay render pass which is in screen space, so the tooltip position in the end always has to be in screen space. However it is useful to think of them in world space since that's where the graphics are. It'll just be much easier to know the position of a graphic in world space where you want to show the tooltip, and the Subplot.display_tooltip() will project that pos to screen space under automatically.

And why would tooltips not be owned by the graphic they correspond to? I could also imagine a property that is image_graphic.tooltip.display = False

@kushalkolar
Copy link
Member Author

And why would tooltips not be owned by the graphic they correspond to? I could also imagine a property that is image_graphic.tooltip.display = False

I didn't think of that! This might be a better way to do it, and customization of tooltips per-graphic makes most sense I think.

@clewis7
Copy link
Member

clewis7 commented Nov 21, 2025

I didn't think of that! This might be a better way to do it, and customization of tooltips per-graphic makes most sense I think.

I think then, if you wanted to not display all of the tooltips for graphics in the subplot, you would just need to iterate:

for g in subplot.graphics:
     g.tooltip.display = False 

@kushalkolar
Copy link
Member Author

kushalkolar commented Dec 6, 2025

ok all this stuff works now:

tooltips update on every render, useful when data under the cursor changes:

tooltips-update-2025-12-06_05.14.36.mp4

Tooltips are now enabled on graphics by default, and a cursor can trigger a tooltip to show up in other subplots:

cursor-tooltips-2025-12-06_05.46.04.mp4

Added more mapping functions, these are explained in the guide:

- ``Subplot.map_screen_to_world((x, y))``
- ``Subplot.map_world_to_screen((x, y, z))``
- ``Graphic.map_model_to_world((x, y, z))``
- ``Graphic.map_world_to_model((x, y, z))``

Need to test a bit, add some more examples maybe, and possibly cleanup some of the code if it can be simplified.

Need to add tests to make sure the right default tooltip display comes up for all graphics.

I should add examples that explain all the spaces (model, world, screen) using a variety of graphics. Show a few points on the graphic and put text that indicates model, world and screen space positions.

  • Graphic that isn't rotated or scaled, model and world are same
  • Graphic scaled
  • Graphic rotated
  • Graphic scaled, rotated, translated

Do this with Image, Line and an ellipsoid, should be enough.

@clewis7 you can take a look, but I'm guessing you're deep in CPU design optimizations for the next few days 😆 . I'll summarize everything in this PR later, required quite a bit more than I thought to get this to work.

@kushalkolar
Copy link
Member Author

kushalkolar commented Dec 7, 2025

OK things getting slightly messy, I think this will be better:

  • TextBox that can be used to display a textbox anywhere, just text with a rectangle mesh behind it. Lives in the overlay scene.
  • Tooltip is a subclass of TextBox. One tooltip instance always exists per subplot. Graphic.tooltip merely sets the tooltip in its subplot.

The PlotArea could also update its tooltip with whatever pick info is under a pixel on every render.

Perhaps the renderer "pointer_move" event could just be used to set the tooltips in each subplot by getting the graphic from the pick_info at that position.

@kushalkolar
Copy link
Member Author

kushalkolar commented Dec 7, 2025

I think things are much simpler now.

  • Created TextBox class
  • Tooltip which is a light subclass of TextBox
  • PlotArea has a Tooltip instance as an attribute. "pointer_move" events will set the tooltip.
  • if Tooltip.continuous_mode, then PlotArea will update the tooltip on every render
  • Cursor uses Subplot.get_pick_info() to get the graphic, pick info, and set the tooltip in that subplot.

One thing left out is I need to think of how to set the tooltip w.r.t. the transform when it's on the current subplot with a cursor. Figured it out, let the cursor manage tooltips entirely when a subplot is added to it.

cursor-tooltips-2025-12-07_03.43.05.mp4

@kushalkolar
Copy link
Member Author

kushalkolar commented Dec 7, 2025

~One last thing to figure out, programmatically setting the cursor isn't triggering the tooltip to appear 😩 ~

nevermind it does work, but not on the first render since the canvas is blank

@kushalkolar
Copy link
Member Author

For the examples to demonstrate the different spaces I think I'll make one example for each of these (instead of separate subplots in one Figure) and two sets, one with an image and one with a line:

  • no transforms
  • translation
  • scaling
  • rotation
  • translation + scaling
  • translation + scaling + rotation

Comment on lines +129 to +131
self._tooltip = Tooltip()
self.get_figure()._fpl_overlay_scene.add(self._tooltip._fpl_world_object)
self.renderer.add_event_handler(self._fpl_set_tooltip, "pointer_move")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PlotArea manages a Tooltip instance. Tooltips are draw in the overlay render pass, and are therefore in screen space. The "pointer_move" event determines where it appears (in screen space). Therefore there is only ever 1 tooltip instance (and thus only one mesh, text, and line object required to draw it) per PlotArea. If they're managed by a graphic then things get complicated.

Comment on lines +292 to +293
# need to call int() on it since it's a numpy array with 1 element
# and numpy arrays aren't hashable
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# need to call int() on it since it's a numpy array with 1 element
# and numpy arrays aren't hashable
# unique 32 bit integer id for each world object

if info["world_object"] is not None:
# if this world object is owned by a graphic
if info["world_object"].id in WORLD_OBJECT_TO_GRAPHIC.keys():
info["graphic"] = WORLD_OBJECT_TO_GRAPHIC[info["world_object"].id]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kushalkolar
Copy link
Member Author

@clewis7 finally r4r! And just before midnight here 😂

@kushalkolar
Copy link
Member Author

kushalkolar commented Dec 18, 2025

There is a bug in Volume, will figure out.

Edit: this is a pygfx thing, posted: pygfx/pygfx#1251

@kushalkolar
Copy link
Member Author

Until pygfx/pygfx#1253 is in the next release of pygfx I've kinda disabled tooltips for ImageVolume.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants