Skip to content
This repository was archived by the owner on Apr 1, 2026. It is now read-only.

perf: Reduce CURRENT_TIMESTAMP queries#1114

Merged
tswast merged 4 commits intomainfrom
fast_read_gbq
Oct 31, 2024
Merged

perf: Reduce CURRENT_TIMESTAMP queries#1114
tswast merged 4 commits intomainfrom
fast_read_gbq

Conversation

@TrevorBergeron
Copy link
Copy Markdown
Contributor

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@TrevorBergeron TrevorBergeron requested review from a team and tswast October 24, 2024 21:21
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Oct 24, 2024
Comment thread bigframes/session/time.py Outdated
MIN_RESYNC_SECONDS = 100


class BigquerySyncedClock:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: BigQuery (capital Q)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, could you make some unit tests for this functionality using freezegun to mock out the system monotonic time? Thanks!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added unit tests in new revision

Copy link
Copy Markdown
Contributor

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clever! This should really help things out.

Note: I think the benchmarks use a separate process per query if I remember correctly. If so, we might need to update those to take full advantage of this. Might even be worth some sort of "warm up" query before running the benchmarks proper.

@tswast tswast merged commit 32274b1 into main Oct 31, 2024
@tswast tswast deleted the fast_read_gbq branch October 31, 2024 17:39
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: m Pull request size is medium.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants