Add possibility to perform tests without nicegui plugins and pytest configuration #5380

himbeles · 2025-10-30T10:42:54Z

Motivation

As discussed and proposed in #5377,
this adds a context manager async run(ui_code) as user: ... within the NiceGUI testing framework.
It enables testing of UI code without configuration of the NiceGUI plugin setup proposed in the documentation.

This can be necessary if NiceGUI is used in a setup where the pytest configuration can't be adapted.
As discussed in #5377, this was necessary for UI tests required within the pyinstaller plugin project.

It can also be helpful as another way to get started with testing in a new user's project -- without the need to configure pytest at all for NiceGUI, minimal API overhead in the call sites, and without hidden plugin "magic".

An example of a test function would be:

async def test_button_click():
    def ui_code() -> None:
        ui.button('Click me', on_click=lambda: ui.notify('Hello World!'))

    async with run(ui_code) as user:
        await user.open('/')
        await user.should_see('Click me')
        user.find(ui.button).click()
        await user.should_see('Hello World!')

Implementation

The implementation relies on an async context manager which executes ui.run and points an httpx.AsyncClient as a User to it, in line with the setup and de-setup performed in nicegui.testing.user_plugin.user. The context manager exposes the User for UI tests.

@asynccontextmanager
async def run(ui_code):
    try:
        # simulate user and keep NiceGUI fully headless for tests
        os.environ['NICEGUI_USER_SIMULATION'] = 'true'

        # don't spawn reloader/native window; don't open browser
        ui.run(ui_code, reload=False, native=False, show=False)

        async with core.app.router.lifespan_context(core.app):
            async with httpx.AsyncClient(
                transport=httpx.ASGITransport(core.app),
                base_url='http://test'
            ) as client:
                yield User(client)
    finally:
        os.environ.pop('NICEGUI_USER_SIMULATION', None)
        ui.navigate = Navigate()
        ui.notify = notify
        ui.download = download

Open points in the architecture would be:

Where could this implementation live in the code base?
Suitable name: Candidates would be run, user_run to not risk overloading the very simple run later, user_of, ...
Option to create multiple users for the same UI code. Could yield multiple users in context manager if optional argument n_users>1, for example.

I'm open for discussion and refine the implementation if you consider this useful.

Progress

I chose a meaningful title that completes the sentence: "If applied, this PR will..."
The implementation is complete.
Pytests have been added (or are not necessary).
Documentation has been added (or is not necessary).

evnchn

2 things:

How the import is written, it doesn't match the codebase
I am initially lost over the purpose of this PR. Now, I think this PR is doing, really, "testing without pytest", do you agree?

thanks

nicegui/testing/user_run.py

evnchn · 2025-10-30T11:19:06Z

nicegui/testing/user_run.py

+async def run(ui_code):
+    """A user context manager for the given `nicegui.ui` code.
+
+    Example use in a plain pytest function without user plugin use:


I'm rethinking: The function is just a plain test function now, have very little to do with pytest.

Indeed. You can use run(ui_code) in a pytest test function or with any other test library, or just in regular functions.
But it in the end, it enables you to run pytest tests without using the nicegui pytest plugins.

Minor issue in retrospect.

But still I think the main selling point for this PR is "test without pytest" because that is a superset of "pytest without plugins" and if we describe this PR by "pytest without plugins" we're underselling it.

himbeles · 2025-10-30T12:26:55Z

2 things:

1. How the import is written, it doesn't match the codebase

Fully agreed, of course. I have copied the code from my implementation for the pyinstaller hook, which was not within the nicegui codebase.

Many other things would need to be adapted, also. Like type hints, doc string, etc.

himbeles · 2025-10-30T12:31:01Z

2. I am initially lost over the purpose of this PR. Now, I think this PR is doing, really, "testing without pytest", do you agree?

The main purpose is not to "test without pytest" but to remove the need for the nicegui pytest plugins.
This could make it easier to set up your test environment initially for new users. (Users can use pytest but no special pytest config is required)
Also, in other scenarios, like mine for pyinstaller, you might not have the freedom to configure pytest.

evnchn

Function naming suggestions: run_user or make_user can help hammer down we're getting a User
PR positioning: Though "pytest without plugins" was the intention, suggest keep "test without pytest" as how we communicate because that is the happy-accident and also a superset of the former.
Code duplication: Suggest to refactor instead of copy the code twice.

evnchn · 2025-11-04T18:49:09Z

nicegui/testing/user_run.py

+async def run(ui_code):
+    """A user context manager for the given `nicegui.ui` code.
+
+    Example use in a plain pytest function without user plugin use:


Minor issue in retrospect.

But still I think the main selling point for this PR is "test without pytest" because that is a superset of "pytest without plugins" and if we describe this PR by "pytest without plugins" we're underselling it.

evnchn · 2025-11-04T18:53:43Z

nicegui/testing/user_run.py

+    try:
+        # simulate user and keep NiceGUI fully headless for tests
+        os.environ['NICEGUI_USER_SIMULATION'] = 'true'
+
+        # don't spawn reloader/native window; don't open browser
+        ui.run(ui_code, reload=False, native=False, show=False)
+
+        async with core.app.router.lifespan_context(core.app):
+            async with httpx.AsyncClient(
+                transport=httpx.ASGITransport(core.app),
+                base_url='http://test'
+            ) as client:
+                yield User(client)
+    finally:
+        os.environ.pop('NICEGUI_USER_SIMULATION', None)
+        ui.navigate = Navigate()
+        ui.notify = notify
+        ui.download = download


Some logic is shared with nicegui/testing/user_plugin.py.

If that is the case, could we adhere to the DRY principle and better highlight the (I think quite high) similarity between the user fixture and this new function, either by:

User fixture can simply call this new function?

User fixture and this new function call some common function?

evnchn · 2025-11-04T18:55:13Z

nicegui/testing/user_run.py

+
+
+@asynccontextmanager
+async def run(ui_code):


run may cause confusion with ui.run. And also doesn't show the fact that this outputs a User which then unlocks all of https://nicegui.io/documentation/user

I am proposing get_user or make_user to hammer-down the fact that we're getting a User. Not sure if this name really works out though.

rodja · 2025-11-05T04:18:48Z

Function naming suggestions: run_user or make_user can help hammer down we're getting a User

I like run better because it reads so well in async with run(ui_code) as user and the analogy to ui.run. The few times it might collide with run.io_bound/cpu_bound, people provide another name via as when importing.

PR positioning: Though "pytest without plugins" was the intention, suggest keep "test without pytest" as how we communicate because that is the happy-accident and also a superset of the former.

I find "test without pytest" quite irritating because you still need to run your tests with pytest. It's a more meaningful description is "user testing without using the fixture (which is made available via pytest plugin)". While I like this new feature and shortly thought if this approach could be the new go-to default, I now think that most users are better of using the plugin. By defining the main_file setup is super easy. Modifying the code so that all UI can be build from a single function feels like more hassel.

How the import is written, it doesn't match the codebase

It might be good to do the import via from nicegui.testing.user import run, because a similar feature could be implemented for screen tests.

himbeles · 2025-11-05T06:58:19Z

Thank you a lot for your renewed feedback.
I will have a more in-depth look at all your replies, the API and implementation over the weekend.

In the meantime, I have come up with a slightly modified approach with even simpler user interface. You can now

test code snippets without defining a ui_code callback function

async def test_get_user_context():
    async with get_user() as user:
        ui.label('as')
        await user.should_see('as')

or access multiple pages directly in the context manager

async def test_get_user_context_with_page_definitios():
    async with get_user() as user:
        @ui.page('/')
        def page():
            ui.label('Main page')

        @ui.page('/a')
        def pageA():
            ui.label('Page A')

        await user.open('/')
        await user.should_see('Main page')
        await user.open('/a')
        await user.should_see('Page A')

or access a main file by specifying its path path_to_mainfile

async def test_get_user_context():
    async with get_user(path_to_mainfile) as user:
        await user.should_see('as')

(Implementation is not yet committed to the PR branch.)

What do you think?

evnchn · 2025-11-05T08:30:35Z

When necessary, ignore some of my previous comments to respect the majority vote.

For get_user note the following code is basically equivalent.

async with run(some_code) as user:
    ...

async with get_user() as user:
    some_code()
    ...

So I quite like get_user. But if I'm not mistaken:

With the old run if we want to test a @ui.page function we need to pass an empty function because cannot have page functions inside root function (I think?)
With the new get_user cannot easily test SPAs? (recall root functions are introduced specifically to ease SPA implementation)

So possibly get_user(ui_code=None) can facilitate both use cases?

rodja · 2025-11-06T05:05:51Z

Yes, I like your "slightly modified approach" @himbeles where the context can also provides an enclosure for ui code. As @evnchn suggested the get_user could have an optional root-function parameter, and as you suggested an optional path to main file. That covers most of the use-cases and hence might even become the new default way.

I just struggle with the naming. async with get_user() as user does not read so fluent. What about

async with run_user() as user
async with user_simulation() as user
async with start_simulation() as user
async with run_simulation() as user

First proposal of run(ui_code) for pytest framework

99b6726

evnchn reviewed Oct 30, 2025

View reviewed changes

change to relative imports

318b9c1

falkoschindler added testing Type/scope: Pytests and test fixtures in progress Status: Someone is working on it labels Nov 3, 2025

falkoschindler added this to the 3.x milestone Nov 3, 2025

evnchn reviewed Nov 4, 2025

View reviewed changes

Uh oh!

Add possibility to perform tests without nicegui plugins and pytest configuration #5380

Are you sure you want to change the base?

Add possibility to perform tests without nicegui plugins and pytest configuration #5380

Uh oh!

Conversation

himbeles commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Implementation

Progress

Uh oh!

evnchn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

evnchn Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

himbeles Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

evnchn Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

himbeles commented Oct 30, 2025

Uh oh!

himbeles commented Oct 30, 2025

Uh oh!

evnchn left a comment

Choose a reason for hiding this comment

Uh oh!

evnchn Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

evnchn Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

evnchn Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

rodja commented Nov 5, 2025

Uh oh!

himbeles commented Nov 5, 2025

Uh oh!

evnchn commented Nov 5, 2025

Uh oh!

rodja commented Nov 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

himbeles commented Oct 30, 2025 •

edited

Loading