Skip to content

Add file check on all recipe tests #2126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
Adel-Moumen merged 37 commits into from
Aug 16, 2023
Merged

Add file check on all recipe tests #2126

Adel-Moumen merged 37 commits into from
Aug 16, 2023

Conversation

@mravanelli
Copy link
Collaborator

@mravanelli mravanelli commented Aug 14, 2023

This PR extends our recipe tests to perform file checks on all the existing recipes.

@mravanelli mravanelli added the enhancement New feature or request label Aug 14, 2023
@mravanelli mravanelli requested a review from Adel-Moumen August 14, 2023 04:28
@mravanelli mravanelli self-assigned this Aug 14, 2023
Adel-Moumen
Adel-Moumen reviewed Aug 14, 2023
View reviewed changes
tests/recipes/DVoice.csv
Copy link
Collaborator

@Adel-Moumen Adel-Moumen Aug 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why removing wer_test from file_exists ? Why is it empty now?

@Adel-Moumen
Copy link
Collaborator

Adel-Moumen commented Aug 15, 2023 •
edited
Loading

Note: Errors with DVoice, fluent-speech-commands

       ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/wer_test_real.txt
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/train_log.txt
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/predictions.jsonl
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/log.txt
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/env.log
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/train.py
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/hyperparams.yaml
        ERROR: The recipe SLURP_row_3 does not contain the expected file tests/tmp/SLURP_row_3/save/SLURM_tokenizer/tokenizer.ckpt
(182/192) Running test for SLURP_row_4...
        ... 71.11s
        ...checking files & performance...
(183/192) Running test for SLURP_row_5...
        ... 5.98s
        ...checking files & performance...
(184/192) Running test for fluent-speech-commands_row_2...
        ... 6.81s
        ERROR: Error in fluent-speech-commands_row_2 (recipes/fluent-speech-commands/direct/hparams/train.yaml). Check tests/tmp/fluent-speech-commands_row_2/stderr.txt and tests/tmp/fluent-speech-commands_row_2/stdout.txt for more info.
        ...checking files & performance...
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/train_log.txt
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/log.txt
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/wer_test.txt
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/env.log
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/train.py
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/hyperparams.yaml
        ERROR: The recipe fluent-speech-commands_row_2 does not contain the expected file tests/tmp/fluent-speech-commands_row_2/save/FSC_tokenizer/tokenizer.ckpt
(185/192) Running test for fluent-speech-commands_row_3...

@Adel-Moumen
Copy link
Collaborator

Adel-Moumen commented Aug 15, 2023

Note: I will rerun everything on a different cluster with a different environment. I think I have some troubles with my current cluster regarding recipe testing.

Adel-Moumen
Adel-Moumen approved these changes Aug 16, 2023
View reviewed changes
Copy link
Collaborator

@Adel-Moumen Adel-Moumen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Tests are passing. Thanks Mirco :)

@Adel-Moumen Adel-Moumen merged commit 5565073 into speechbrain:develop Aug 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Adel-Moumen Adel-Moumen Adel-Moumen approved these changes

Assignees

@mravanelli mravanelli

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mravanelli @Adel-Moumen