Skip to content

Adding recipe for Listenable Maps for Audio Classifiers #2538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
mravanelli merged 322 commits into from
Jul 3, 2024
Merged

Adding recipe for Listenable Maps for Audio Classifiers #2538

mravanelli merged 322 commits into from
Jul 3, 2024

Conversation

@fpaissan
Copy link
Collaborator

@fpaissan fpaissan commented May 2, 2024

What does this PR do?

  • Implements the work of the ICML'24 paper L-MAC (https://arxiv.org/abs/2403.13086).
  • Refactors exiting L2I and PIQ recipes to unify the interpretability approaches under a single structure. This simplifies code shared among the interpretability pipelines. Also, it minimizes the room for error in the comparison of the techniques.
Before submitting
  • Did you read the contributor guideline?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Does your code adhere to project-specific code style and conventions?

PR review

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified
  • Confirm that the changes adhere to compatibility requirements (e.g., Python version, platform)
  • Review the self-review checklist to ensure the code is ready for review

ycemsubakan and others added 30 commits January 21, 2024 14:39
@ycemsubakan
Merge branch 'icml2024_cemstuff' of github.com:fpaissan/audio_interpr…
f687b60 
…etability into icml2024_cemstuff
@fpaissan
works on cnn14 -- but have a bad checkpoint
a659533
@fpaissan
fixed l2i as well
f632940
@fpaissan
fixed acc in l2i
596b56c
@fpaissan
fix not listenable
163374b
@fpaissan
updated logging for eval
5f30445
@fpaissan
a bit less verbose
1362d62
@fpaissan
printing at sample level
a6b8581
@fpaissan
fix logging - was missing avg
5f7c5cc
@fpaissan
was messing up in the forward
61a2c00
@fpaissan
now running train_piq.py
399fc9f
@fpaissan
minor corrections
c94e788
@fpaissan
fix l2i training with wham!
dd63174
@fpaissan
fixed l2i computation
59f7d5c
@fpaissan
linters
a962410
@fpaissan
add check for wham usage in eval
af5c0a9
@fpaissan
add sample saving during eval
6047a48
@fpaissan
bug fixes
ad8cf66
@fpaissan
added predictions info to the logging
b895803
@fpaissan
fixed id for overlap test
3ddc0b8
@fpaissan
cutting sample before saving
a3ccf71
@fpaissan
fixed l2i sampling rate
bd7aa57
@fpaissan
fixed random seed so eval will match
4bf6815
@fpaissan
running on full set
dbc5b96
@ycemsubakan
faithfulness fix
e77bfc7
@ycemsubakan
remove pdb
f36f3cd
@fpaissan
fix smoothgrad and IG
c891dd5
@fpaissan
fix nmf for pre-training
a5ebd89
@fpaissan
removed nmf reconstructions
2529b85
@fpaissan
truncated gaussian fix for smoothgrad
af88a98
ycemsubakan and others added 28 commits June 29, 2024 14:13
@ycemsubakan
fixed the header in eval.py
7c356ec
@ycemsubakan
added missing l2i command in train_l2i.py
ad1d08e
@ycemsubakan
fixes to train_lmac.py
68e5b85
@ycemsubakan
description for classifier_temp
b332ca9
@ycemsubakan
added comments for pretrained_interpreter and ljspeech_path
cbba482
@ycemsubakan
updated README to have more information on how to use LJSpeech
cce2f9a
@ycemsubakan
added information for piq_vit.yaml and piq_focalnet.yaml
dc74cd9
@ycemsubakan
added more explanation for LJSpeech downloading
e3767db
@ycemsubakan
added missing use_melspectra_log1p attribute to piq_vit.yaml and piq_…
ad7cb57 
…focalnet.yaml
@ycemsubakan
added an assert in eval.py for the pretrained path
5547ada
@ycemsubakan
updates to the readme, added table, updated l2i to print quantus metrics
aba93d5
@mravanelli
Update README.md
9809ecf
@ycemsubakan
added the description of pretained_interpreter in README.md.
d0b132e
@ycemsubakan
Merge branch 'icml2024_lmac' of github.com:fpaissan/speechbrain into …
4a9f128 
…icml2024_lmac
@ycemsubakan
fixed the problem in vit
1a4f25c
@mravanelli
Merge remote-tracking branch 'upstream/develop' into icml2024_lmac
3ed6bff
@ycemsubakan
fixing l2i tests
237817a
@ycemsubakan
Merge branch 'icml2024_lmac' of github.com:fpaissan/speechbrain into …
5a55a57 
…icml2024_lmac
@ycemsubakan
fixed ESC50.csv
2d5e6a1
@ycemsubakan
fixed the yaml tets
9ff7c66
@ycemsubakan
added links to files
25927a8
@ycemsubakan
fixed docstring tests
9e3ec34
@ycemsubakan
bug fix on psi model
99b19d5
@ycemsubakan
removing the classes from PIQ.py
f30e18a
@ycemsubakan
fixes in L2I psi classes
616cc7d
@ycemsubakan
handling sps comp exceptions
fb6480c
@ycemsubakan
added the dropbox links
5283876
@mravanelli
Update README.md
8b59cb2
@mravanelli mravanelli merged commit 2f9e2fd into speechbrain:develop Jul 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mravanelli mravanelli mravanelli approved these changes

@ycemsubakan ycemsubakan Awaiting requested review from ycemsubakan

Assignees

@ycemsubakan ycemsubakan

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@fpaissan @mravanelli @ycemsubakan