Skip to content

#3134 Update & Review Packages for git-repo-scanner #3316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

#3134 Update & Review Packages for git-repo-scanner #3316

wants to merge 2 commits into from

Conversation

@kart-u
Copy link

@kart-u kart-u commented Oct 10, 2025 •
edited
Loading

Description

Issue

  • The Component: The git-repo-scanner is a custom-written Python tool for scanning Git repositories.
  • The Problem: The scanner's code and its dependencies (the other Python packages it relies on) have not been updated in a while and need attention.
  • The Goal: The main task is to update the Python packages listed in the requirements.txt file to their latest stable versions. A - secondary goal is to perform any other necessary Python-related maintenance to improve the code's health.

Solution

Updated Dpenedencies:

  • Found all dependencies up to date except pytimeparse, which has been inactive since 2018.
  • Replaced pytimeparse with the actively maintained pytimeparse2 (fully compatible).

Corrected Logical errors in github_scanner.py

  • Error :- Inside _process_repos_page method in github_scanner.py handling of start_time and end_time was wrong and break condition was causing inclusion of unwanted repositories in findings when end_time and start_time both are present.
  • Corrected handling of start_time and end_time.
  • added is_over flag to avoid further lookup when when we already out of timeframe

Added new more Test cases for all above code

  • Added more robust test cases in git_repo_scanner_test.py (timeframe related test were absent added variety of those)
  • Added main_test.py file for testing of git_repo_scanner.__main__.py

Testing Results

Unit Testing

original failing
  • Original Code failed when tested with timeframe tests
new_pass
  • Corrected code passed for cases

Manual Testing

'''
cd ./scanners/git-repo-scanner/scanner/
python3 -m git_repo_scanner \
--git-type github \
--organization secureCodeBox\
--access-token $GITHUB_TOKEN \
--activity-since-duration "30d" \
--activity-until-duration "1d" \
--file-output github-results.json
'''

logs1 log2

Closes #3134

Checklist

  • Test your changes as thoroughly as possible before you commit them. Preferably, automate your test by unit/integration tests.
  • Make sure that all your commits are signed-off and that you are added to the Contributors file.
  • Make sure that all CI finish successfully.
  • Optional (but appreciated): Make sure that all commits are Verified.

@netlify
Copy link

netlify bot commented Oct 10, 2025 •
edited
Loading

✅ Deploy Preview for docs-securecodebox ready!

Name Link
🔨 Latest commit 0881ee3
🔍 Latest deploy log https://app.netlify.com/projects/docs-securecodebox/deploys/68f7cb1ee11c1400097cce30
😎 Deploy Preview https://deploy-preview-3316--docs-securecodebox.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@kart-u kart-u force-pushed the updated-git-repo-scanner branch from b8c2f6a to 4944747 Compare October 10, 2025 01:13
@kart-u kart-u changed the title #3134 Update pytimeparse to pytimeparse2,Added more robust tests and correct pagination buggy behavior from tests, Corrected github_scanner.py logic #3134 Update & Review Packages for git-repo-scanner Oct 10, 2025
@Weltraumschaf Weltraumschaf moved this from Triage to To Review in secureCodeBox Oct 10, 2025
@kart-u
Copy link
Author

kart-u commented Oct 11, 2025

Hi @Weltraumschaf , since this PR has been assigned to me, I wanted to clarify the expectations. Should I address the recommendations from @sonarqubecloud, or is my role primarily to review the changes and provide feedback?

@Weltraumschaf
Copy link
Member

Weltraumschaf commented Oct 13, 2025

Hi @Weltraumschaf , since this PR has been assigned to me, I wanted to clarify the expectations. Should I address the recommendations from @sonarqubecloud, or is my role primarily to review the changes and provide feedback?

Hi @kart-u,

thanks for submitting this PR. Your assignement does not mean more than that you work on this PR. We'll first need to review your PR.

@kart-u
Copy link
Author

kart-u commented Oct 13, 2025

Thanks for clarifying! I’ll wait for your review before making any further updates.

@Weltraumschaf Weltraumschaf added the Hacktoberfest https://hacktoberfest.digitalocean.com/ label Oct 13, 2025
twwd
twwd requested changes Oct 15, 2025
View reviewed changes
scanners/git-repo-scanner/scanner/git_repo_scanner/github_scanner.py Outdated
Comment on lines 103 to 110
if start_time and end_time and repo.pushed_at<start_time:
return True
elif start_time and not end_time and repo.pushed_at<start_time:
return True
elif end_time and not start_time and repo.pushed_at>end_time:
return True
else:
continue
Copy link
Contributor

@twwd twwd Oct 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the first glance, these checks seem redundant to the ones in lines 121 til 130.

@malexmave you've wrote the intial code, can you help out here regarding the logic?

Copy link
Author

@kart-u kart-u Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here _check_repo_is_in_time_frame this function is just for checking whether where repos are in correct timeframe or not
and these return True or False if else statements are checking for early exit

@J12934 J12934 removed the Hacktoberfest https://hacktoberfest.digitalocean.com/ label Oct 21, 2025
@kart-u kart-u force-pushed the updated-git-repo-scanner branch from 2926ec3 to 3edbed5 Compare October 21, 2025 15:18
@kart-u
Copy link
Author

kart-u commented Oct 21, 2025

@twwd I’ve addressed your suggestions , could you please review again and let me know if I missed anything?

@kart-u
updated-git-repo-review
0946448 
Signed-off-by: kart-u <pandeykartik715@gmail.com>
@kart-u kart-u force-pushed the updated-git-repo-scanner branch from 7ec3092 to 0946448 Compare October 21, 2025 18:01
@kart-u
Merge branch 'main' into updated-git-repo-scanner
0881ee3 
Signed-off-by: kart-u <pandeykartik715@gmail.com>
@sonarqubecloud
Copy link

sonarqubecloud bot commented Oct 21, 2025

Quality Gate Passed Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@kart-u kart-u requested a review from twwd October 23, 2025 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@twwd twwd Awaiting requested review from twwd

Requested changes must be addressed to merge this pull request.

Assignees

@kart-u kart-u

Labels

None yet

Projects

secureCodeBox
Status: To Review

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Update & Review Packages for git-repo-scanner

4 participants

@kart-u @Weltraumschaf @twwd @J12934