I am not sure if this is possible without looping.

vad_th = vad_activation + vad_deactivation

is a tensor consisting of 0, 1, 2. The loop after getting a value of 2, changes subsequent 1 into 2. Same for 0: changes 1 into 0.

I can check the device that the input tensor is on and put it back on that device before returning the result.

Could you check if something like this solves your performance issue?

# whether the n-th frame falls below threshold and triggers deactivation
frame_does_not_deactivate = values < deactivation_th

# always start keeping frames over activation threshold activated
vad_th = values >= activation_th

for i in range(1, len(values)):
    vad_th[..., i] |= vad_th[..., i - 1]  # if past frame was activated then keep activated
    vad_th[..., i] &= frame_does_not_deactivate[..., i]  # ... unless the i-th frame is below threshold

It eliminates the batch iteration and should avoid CPU/GPU syncs. It probably fires a ton of kernel launches on CUDA but that might be ok...

Could you check if something like this solves your performance issue?

# whether the n-th frame falls below threshold and triggers deactivation
frame_does_not_deactivate = values < deactivation_th

# always start keeping frames over activation threshold activated
vad_th = values >= activation_th

for i in range(1, len(values)):
    vad_th[..., i] |= vad_th[..., i - 1]  # if past frame was activated then keep activated
    vad_th[..., i] &= frame_does_not_deactivate[..., i]  # ... unless the i-th frame is below threshold

It eliminates the batch iteration and should avoid CPU/GPU syncs. It probably fires a ton of kernel launches on CUDA but that might be ok...

It works! Execution times are very close to my workaround!

Ok, great to hear. I'll sanity check it before merging later but it seemed correct from my testing.

Heads up that I've tested this with speechbrain/vad-crdnn-libriparty and it seems to alter the detected boundaries slightly. I'll look into it.

I have pushed some fixed code and the code now emits the same boundaries as before. There were issues around shape handling.

Could you check if the performance is still OK? It was not iterating properly with the previous code. Sorry for the trouble.

I have pushed some fixed code and the code now emits the same boundaries as before. There were issues around shape handling.

Could you check if the performance is still OK? It was not iterating properly with the previous code. Sorry for the trouble.

I made some measurements for my 6.5h file:
Workaround with offload to cpu: ~29s
First loop elimination: ~31s
Last fix: ~115s

The performance has decreased a bit, but it is still much better than the original ~20minutes :)