How to resolve KeyError: 'spk_dict_mean' in SpeechBrain InputNormalization module? #2927

kychiu1 · 2025-05-26T02:25:42Z

kychiu1
May 26, 2025

Hi,
I'm encountering a KeyError: 'spk_dict_mean' error while loading my custom trained streaming Conformer Transducer model using SpeechBrain's Common Voice recipe with the Cantonese Common Voice dataset. I'm a beginner to SpeechBrain and speech recognition, so I think I may be missing something fundamental in my setup. So far I have attempted both unigram/bpe settings for the tokenizer and this issues relates to both settings.

Steps I have taken to create my own trained model folder

Copy model.ckpt, normalizer.ckpt from the checkpoint folder
adapt the hyperparams.yaml from speechbrain/asr-streaming-conformer-librispeech on huggingface
Load the unigram/bpe model using spm.SentencePieceProcessor() and save the state as tokenizer.ckpt file

My Inference Script

from speechbrain.inference.ASR import StreamingASR
from speechbrain.utils.dynamic_chunk_training import DynChunkTrainConfig
asr_model = StreamingASR.from_hparams(
    source="pretrained_models/my_unigram_model",
    savedir="pretrained_models/my_unigram_model",
    # run_opts = {"device": "cuda:1"}
)

KeyError                                  Traceback (most recent call last)
Cell In[4], line 3
      1 from speechbrain.inference.ASR import StreamingASR
      2 from speechbrain.utils.dynamic_chunk_training import DynChunkTrainConfig
----> 3 asr_model = StreamingASR.from_hparams(
      4     source="pretrained_models/my_unigram_model",
      5     savedir="pretrained_models/my_unigram_model",
      6     # run_opts = {"device": "cuda:1"}
      7 )
      9 # print(asr_model.tokenizer)
     10 asr_model.transcribe_file(
     11     "./common_voice_yue_41050862.wav",
     12     # select a chunk size of ~960ms with 4 chunks of left context
   (...)     16     use_torchaudio_streaming=False,
     17 )

File ~/miniconda3/envs/speechbrain/lib/python3.11/site-packages/speechbrain/inference/interfaces.py:536, in Pretrained.from_hparams(cls, source, hparams_file, pymodule_file, overrides, savedir, use_auth_token, revision, download_only, huggingface_cache_dir, overrides_must_match, local_strategy, **kwargs)
    533 # Load on the CPU. Later the params can be moved elsewhere by specifying
    534 if not download_only:
    535     # run_opts={"device": ...}
--> 536     pretrainer.load_collected()
    538     # Now return the system
    539     return cls(hparams["modules"], hparams, **kwargs)

File ~/miniconda3/envs/speechbrain/lib/python3.11/site-packages/speechbrain/utils/parameter_transfer.py:349, in Pretrainer.load_collected(self)
    345     else:
    346         raise ValueError(
    347             f'Pretrainer has never collected `{name}`, did you forget a call to `collect_files`? Could not fall back to `collect_in`, as it was not specified (default is no longer "model_checkpoints").'
    348         )
--> 349 self._call_load_hooks(paramfiles)

File ~/miniconda3/envs/speechbrain/lib/python3.11/site-packages/speechbrain/utils/parameter_transfer.py:366, in Pretrainer._call_load_hooks(self, paramfiles)
    364 default_hook = get_default_hook(obj, DEFAULT_TRANSFER_HOOKS)
    365 if default_hook is not None:
--> 366     default_hook(obj, loadpath)
    367     continue
    368 # Otherwise find the default loader for that type:

File ~/miniconda3/envs/speechbrain/lib/python3.11/site-packages/speechbrain/processing/features.py:1530, in InputNormalization._load(***failed resolving arguments***)
   1528 device = "cpu"
   1529 stats = torch.load(path, map_location=device)
-> 1530 self._load_statistics_dict(stats)

File ~/miniconda3/envs/speechbrain/lib/python3.11/site-packages/speechbrain/processing/features.py:1476, in InputNormalization._load_statistics_dict(self, state)
   1474 # Loading the spk_dict_mean in the right device
   1475 self.spk_dict_mean = {}
-> 1476 for spk in state["spk_dict_mean"]:
   1477     self.spk_dict_mean[spk] = state["spk_dict_mean"][spk]  # .to(
   1478     #    self.device_inp
   1479     # )
   1480 
   1481 # Loading the spk_dict_std in the right device

KeyError: 'spk_dict_mean'

My Inference Folder:

hyperparams.yaml, train.py, wer_test.txt: https://gist.github.com/kychiu1/bbf4c8b5ad268462f01eacf2e8895628

Questions:

Is this error related to the normalization configuration in my YAML file?

Thanks in advance for any help and guidance

kychiu1 · 2025-05-27T06:54:49Z

kychiu1
May 27, 2025
Author

I have managed to resolve this issue to manually insert the missing keys and replacing my normalizer.ckpt.

import torch  
import os  
  
# Load the checkpoint  
checkpoint = torch.load('pretrained_models/my_bpe_model/normalizer.bak.ckpt', map_location='cpu')  
  
# Check current contents  
print("Current checkpoint keys:", list(checkpoint.keys()))  
  
# Add the missing empty dictionaries  
checkpoint['spk_dict_mean'] = {}  
checkpoint['spk_dict_std'] = {}  
checkpoint['spk_dict_count'] = {}
# Save the updated checkpoint  
torch.save(checkpoint, 'normalizer.ckpt')

Then I ran into some tokenizer issues which I fixed by changing and removing the tokenizer.ckpt and instead reference the .model file in hyperparams.yaml.

tokenizer: !new:sentencepiece.SentencePieceProcessor
   model_file: !ref <save_folder>/5000_bpe.model

Question

These have allowed me to load the model. However I notice the transcription result on the same validation audio does not match the result in wer_test.txt. Could you help me understand what might be causing this discrepancy?

================================================================================
ALIGNMENTS

Format:
<utterance-id>, WER DETAILS
<eps> ; reference  ; on ; the ; first ;  line
  I   ;     S      ; =  ;  =  ;   S   ;   D  
 and  ; hypothesis ; on ; the ; third ; <eps>
================================================================================
common_voice_yue_38885039, %WER 20.00 [ 1 / 5, 0 ins, 1 del, 0 sub ]
你 ; 仲 ; 未 ;   譯   ; 好
= ; = ; = ;   D   ; =
你 ; 仲 ; 未 ; <eps> ; 好
================================================================================

Inference Code

from speechbrain.inference.ASR import StreamingASR
from speechbrain.utils.dynamic_chunk_training import DynChunkTrainConfig
asr_model = StreamingASR.from_hparams(
    source="pretrained_models/my_bpe_model",
    savedir="pretrained_models/my_bpe_model",
    run_opts = {"device": "cpu"}
)
# print(asr_model)
# print(asr_model.tokenizer)
asr_model.transcribe_file(
    "./common_voice_yue_38885039.wav",
    DynChunkTrainConfig(24, 4),
    use_torchaudio_streaming=False,
)

wer_test.txt: 你 ; 仲 ; 未 ; ; 好
inference code: '你家似家拼家進'

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

How to resolve KeyError: 'spk_dict_mean' in SpeechBrain InputNormalization module? #2927

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

How to resolve KeyError: 'spk_dict_mean' in SpeechBrain InputNormalization module? #2927

Uh oh!

Uh oh!

kychiu1 May 26, 2025

Steps I have taken to create my own trained model folder

My Inference Script

Replies: 1 comment

Uh oh!

kychiu1 May 27, 2025 Author

Question

kychiu1
May 26, 2025

kychiu1
May 27, 2025
Author