Skip to content

Add dependency linting and clean up #13193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 15, 2025
Merged

Add dependency linting and clean up #13193

merged 1 commit into from
Oct 15, 2025

Conversation

@silv-io
Copy link
Member

@silv-io silv-io commented Sep 25, 2025 •
edited
Loading

Motivation

So far it has been difficult to maintain our dependencies and keep them clean. We face two issues:

  1. We nominally depend on packages which we actually don't use in our code
  2. We use packages in our code which we only transitively depend on

To solve these issues we can add a step to our linting which uses https://github.com/fpgmaas/deptry

Changes

  • Add deptry and suitable configuration to indicate what our dev-dependencies are and which code is only used during development
  • Run deptry in the linting make target
  • Define all dependencies that are actually used by our package
    • packages that we use and only depend on transitively are now defined as actual dependencies
    • packages that we define as dependencies and don't use are removed
    • packages that we define as runtime packages but are actually only test packages are moved to the test dependencies
  • Run the dependency upgrade to regenerate the pins correctly.

part of FLC-86

@github-actions
Copy link

github-actions bot commented Sep 25, 2025 •
edited
Loading

Test Results - Preflight, Unit

22 337 tests  ±0   20 587 ✅ ±0   15m 43s ⏱️ - 1m 3s
     1 suites ±0    1 750 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit d6aeff4. ± Comparison against base commit 34a6119.

♻️ This comment has been updated with latest results.

@silv-io silv-io force-pushed the deps/clean-deps branch 2 times, most recently from 9b2e06c to c298624 Compare September 25, 2025 15:22
@github-actions
Copy link

github-actions bot commented Sep 25, 2025 •
edited
Loading

LocalStack Community integration with Pro

    2 files      2 suites   1h 59m 25s ⏱️
4 865 tests 4 516 ✅ 349 💤 0 ❌
4 867 runs  4 516 ✅ 351 💤 0 ❌

Results for commit d6aeff4.

♻️ This comment has been updated with latest results.

@alexrashed alexrashed modified the milestone: 4.9 Sep 26, 2025
bentsku
bentsku reviewed Sep 26, 2025
View reviewed changes
pyproject.toml
"boto3==1.40.35",
# pinned / updated by ASF update action
"botocore==1.40.35",
"awscrt>=0.13.14,!=0.27.1",
Copy link
Contributor

@bentsku bentsku Sep 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awscrt is an optional, transitive dependency of botocore, using C extensions to speed things up around auth, etc.

See https://boto3.amazonaws.com/v1/documentation/api/1.18.31/guide/quickstart.html#using-the-aws-common-runtime-crt

Not sure if we should install boto3/botocore with it or not, and if we should keep this dependency.

Copy link
Member Author

@silv-io silv-io Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding it back in for the time being 👍🏼

Copy link
Member

@alexrashed alexrashed Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought (non-blocking): Should we maybe use the crt extra on boto then instead of defining it here explicitly?

@silv-io silv-io force-pushed the deps/clean-deps branch 3 times, most recently from a91182e to 06b18cd Compare September 29, 2025 13:02
@github-actions
Copy link

github-actions bot commented Sep 29, 2025 •
edited
Loading

S3 Image Test Results (AMD64 / ARM64)

    2 files  ±0    2 suites  ±0   8m 0s ⏱️ -6s
  533 tests ±0  481 ✅ ±0   52 💤 ±0  0 ❌ ±0 
1 066 runs  ±0  962 ✅ ±0  104 💤 ±0  0 ❌ ±0 

Results for commit d6aeff4. ± Comparison against base commit 34a6119.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Sep 29, 2025 •
edited
Loading

Test Results (amd64) - Acceptance

7 tests   5 ✅  3m 20s ⏱️
1 suites  2 💤
1 files    0 ❌

Results for commit d6aeff4.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Sep 29, 2025 •
edited
Loading

Test Results (amd64) - Integration, Bootstrap

    5 files      5 suites   2h 37m 55s ⏱️
5 239 tests 4 730 ✅ 509 💤 0 ❌
5 245 runs  4 730 ✅ 515 💤 0 ❌

Results for commit d6aeff4.

♻️ This comment has been updated with latest results.

@silv-io silv-io marked this pull request as ready for review September 29, 2025 14:19
alexrashed
alexrashed approved these changes Sep 29, 2025
View reviewed changes
Copy link
Member

@alexrashed alexrashed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks a lot for jumping on this and (a) cleaning up, but also (b) making sure that this is continuously checked with a new linting step! Great to see that at least a few dependencies got removed (even though I was hoping for a bit more 😛)! 🥳 🧹

pyproject.toml
"boto3==1.40.35",
# pinned / updated by ASF update action
"botocore==1.40.35",
"awscrt>=0.13.14,!=0.27.1",
Copy link
Member

@alexrashed alexrashed Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought (non-blocking): Should we maybe use the crt extra on boto then instead of defining it here explicitly?

alexrashed
alexrashed reviewed Sep 30, 2025
View reviewed changes
@alexrashed alexrashed mentioned this pull request Sep 30, 2025
Closed
@alexrashed alexrashed modified the milestones: Playground, 4.10 Sep 30, 2025
@silv-io silv-io force-pushed the deps/clean-deps branch 2 times, most recently from ee334da to d1009d6 Compare September 30, 2025 13:50
@silv-io silv-io added review: merge when ready Signals to the reviewer that a PR can be merged if accepted docs: skip Pull request does not require documentation changes notes: skip Pull request does not have to be mentioned in the release notes semver: patch Non-breaking changes which can be included in patch releases labels Oct 3, 2025
@silv-io
Copy link
Member Author

silv-io commented Oct 3, 2025

We can merge this on Monday after 4.9.2 is out (and after rebasing) /cc @k-a-il

k-a-il
k-a-il approved these changes Oct 13, 2025
View reviewed changes
Copy link
Contributor

@k-a-il k-a-il left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@silv-io silv-io force-pushed the deps/clean-deps branch 2 times, most recently from 41e5c10 to 88d0aa4 Compare October 15, 2025 13:18
@silv-io silv-io merged commit 5c97be4 into main Oct 15, 2025
46 checks passed
@silv-io silv-io deleted the deps/clean-deps branch October 15, 2025 20:01
@nik-localstack nik-localstack mentioned this pull request Oct 20, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alexrashed alexrashed alexrashed approved these changes

@bentsku bentsku bentsku left review comments

@k-a-il k-a-il k-a-il approved these changes

@sannya-singal sannya-singal Awaiting requested review from sannya-singal sannya-singal is a code owner

Assignees

No one assigned

Labels

docs: skip Pull request does not require documentation changes notes: skip Pull request does not have to be mentioned in the release notes review: merge when ready Signals to the reviewer that a PR can be merged if accepted semver: patch Non-breaking changes which can be included in patch releases

Projects

None yet

Milestone

4.10

Development

Successfully merging this pull request may close these issues.

5 participants

@silv-io @alexrashed @bentsku @k-a-il