Skip to content

skip flaky tests due to frequent sequel timeouts #70106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
davidsbailey wants to merge 3 commits into from
Closed

skip flaky tests due to frequent sequel timeouts #70106

davidsbailey wants to merge 3 commits into from

Conversation

@davidsbailey
Copy link
Member

@davidsbailey davidsbailey commented Dec 18, 2025 •
edited
Loading

Multiple dashboard unit tests recently started failing in drone with the following error:

Minitest::UnexpectedError:         Sequel::DatabaseError: Mysql2::Error::TimeoutError: Timeout waiting for a response from the last query. (waited 30 seconds)

This PR disables the 3 worst offenders. Here is the error output for those 3 tests:

ERROR["test_create_User_Level_Interaction_for_project_level", "UserLevelInteractionsControllerTest", 385.328863338]
 test_create_User_Level_Interaction_for_project_level#UserLevelInteractionsControllerTest (385.33s)
Minitest::UnexpectedError:         Sequel::DatabaseError: Mysql2::Error::TimeoutError: Timeout waiting for a response from the last query. (waited 30 seconds)
            test/controllers/user_level_interactions_controller_test.rb:25:in `block in <class:UserLevelInteractionsControllerTest>'
            test/testing/setup_all_and_teardown_all.rb:36:in `run'

ERROR["test_find_channel_owner_finds_channel_owner", "UserTest", 340.3733022749984]
 test_find_channel_owner_finds_channel_owner#UserTest (340.37s)
Minitest::UnexpectedError:         Sequel::DatabaseError: Mysql2::Error::TimeoutError: Timeout waiting for a response from the last query. (waited 30 seconds)
            test/testing/projects_test_utils.rb:25:in `with_storage_id_for'
            test/testing/projects_test_utils.rb:9:in `with_channel_for'
            test/models/user_test.rb:3995:in `block in <class:UserTest>'
            test/testing/setup_all_and_teardown_all.rb:36:in `run'

ERROR["test_undestroy_restores_recently_soft-deleted_projects", "UserTest", 381.173459738]
 test_undestroy_restores_recently_soft-deleted_projects#UserTest (381.17s)
Minitest::UnexpectedError:         Sequel::DatabaseError: Mysql2::Error::TimeoutError: Timeout waiting for a response from the last query. (waited 30 seconds)
            test/testing/projects_test_utils.rb:25:in `with_storage_id_for'
            test/testing/projects_test_utils.rb:9:in `with_channel_for'
            test/models/user_test.rb:2804:in `block (2 levels) in <class:UserTest>'
            test/models/user_test.rb:2802:in `block in <class:UserTest>'
            test/testing/setup_all_and_teardown_all.rb:36:in `run'

root cause

We do not have any great leads for why the tests started failing. From my perspective, I first noticed them failing while running drone on #70001 which adds use_transactional_test_case to more tests which touch the database in setup_all. However, it's not clear why this PR would be the culprit, because none of the offending tests are ones I touched in that PR, and its also possible that the timing was a coincidence.

I thought maybe that the amount of parallelism we are using for dashboard test runs was simply too high, but I believe I have ruled that out by lowering the parallelism from 7 down to 2 without successfully eliminating the errors: #70096

UPDATE: The following PR is confirmed to fix the flakiness:

Testing story

Deployment strategy

Follow-up work

@davidsbailey
Copy link
Member Author

davidsbailey commented Dec 18, 2025 

======= FAIL["test_uses_dashboard_and_pegasus_transactions", "AccountPurgerTest", 200.06286334800006]
 test_uses_dashboard_and_pegasus_transactions#AccountPurgerTest (200.06s)
        [RuntimeError] exception expected, not
        Class: <Sequel::DatabaseError>
        Message: <"Mysql2::Error::TimeoutError: Timeout waiting for a response from the last query. (waited 30 seconds)">
        ---Backtrace---
        test/lib/account_purger_test.rb:95:in `block (2 levels) in <class:AccountPurgerTest>'
        lib/account_purger.rb:63:in `block (3 levels) in really_purge_data_for_account'
        lib/account_purger.rb:61:in `block (2 levels) in really_purge_data_for_account'
        lib/account_purger.rb:60:in `block in really_purge_data_for_account'
        lib/account_purger.rb:59:in `really_purge_data_for_account'
        lib/account_purger.rb:41:in `purge_data_for_account'
        test/lib/account_purger_test.rb:101:in `block (2 levels) in <class:AccountPurgerTest>'
        test/lib/account_purger_test.rb:100:in `block in <class:AccountPurgerTest>'
        test/testing/setup_all_and_teardown_all.rb:36:in `run'
        ---------------
        test/lib/account_purger_test.rb:100:in `block in <class:AccountPurgerTest>'
        test/testing/setup_all_and_teardown_all.rb:36:in `run'

@davidsbailey davidsbailey deleted the disable-flaky-sequel-timeouts branch December 19, 2025 18:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@davidsbailey