BigBlueButton/bigbluebutton-Github
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(recording): Deskshare audio processing hang

Browse Source 
This is a refinement of the fix introduced in 3c1a9cd7c4.

Further investigation of the issue reveals that the cause of the bad
timestamps which result in error messages or hang is when the `apad`
filter does not receive any input audio frames. This happens if the
seekpoint is after the end of audio. The existing seekpoint checks can't
cover this, because in a deskshare file the video stream can continue on
after the audio stream ends.

The `,asetpts=N` filter was added to deal with this problem, and it
works in most cases. But there is a case where it fails - when the
"mismatched length" audio stretching to work around broken recordings
from BigBlueButton 0.81/0.90 kicks in.

The issue there is that the `atrim=start=S` filter (used due to the
difficulty of calculating seeks when stretching) hits the invalid
timestamps and hangs.

I'm working around this issue with a "defense in depth" combination of
two changes:

* Move the `,asetpts=N` filter to be applied before the audio stretch
  filters. This fixes the processing hang.
* Adjust the conditions on the "mismatched length" audio stretching so
  that it only gets applied on audio files likely to be from extremely
  old BigBlueButton versions - those with audio in wav files, or encoded
  to vorbis.

BigBlueButton has been using audio recorded directly to opus by
FreeSWITCH for quite a while, and the handling for gaps or lost packets
is done in current BBB versions by a combination of the libopus decoder
and the use of ffmpeg's `aresample=async=1000` filter to dynamically
stretch, squish, or fill in audio so it becomes a continuous stream
that's locked to the file timestamps. Applying the "mismatched length"
processing on top of that is probably making audio sync issues worse.
This commit is contained in:
Calvin Walton 2023-10-03 14:21:50 -04:00
parent 56edf23a1f
commit d402f519c6
1 changed files with 6 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
record-and-playback/core/lib/recordandplayback/edl/audio.rb
View File

@ -120,7 +120,10 @@ module BigBlueButton
# Check for and handle audio files with mismatched lengths (generated # Check for and handle audio files with mismatched lengths (generated
# by buggy versions of freeswitch in old BigBlueButton # by buggy versions of freeswitch in old BigBlueButton
if entry[:original_duration] && (audioinfo[audio[:filename]][:duration].to_f / entry[:original_duration]) < 0.997 && if (audioinfo[audio[:filename]][:format][:format_name] == 'wav' ||
audioinfo[audio[:filename]][:audio][:codec_name] == 'vorbis') &&
entry[:original_duration] &&
(audioinfo[audio[:filename]][:duration].to_f / entry[:original_duration]) < 0.997 &&
((entry[:original_duration] - audioinfo[audio[:filename]][:duration]).to_f / ((entry[:original_duration] - audioinfo[audio[:filename]][:duration]).to_f /
entry[:original_duration]).abs < 0.05 entry[:original_duration]).abs < 0.05
BigBlueButton.logger.warn " Audio file length mismatch, adjusting speed to #{speed}" BigBlueButton.logger.warn " Audio file length mismatch, adjusting speed to #{speed}"
@ -141,9 +144,9 @@ module BigBlueButton
ffmpeg_filter << "#{FFMPEG_AEVALSRC},#{FFMPEG_AFORMAT}" ffmpeg_filter << "#{FFMPEG_AEVALSRC},#{FFMPEG_AFORMAT}"
end end
ffmpeg_filter << ",atempo=#{speed},atrim=start=#{ms_to_s(audio[:timestamp])}" if speed != 1 ffmpeg_filter << ',asetpts=N'
ffmpeg_filter << ",asetpts=N" ffmpeg_filter << ",atempo=#{speed},atrim=start=#{ms_to_s(audio[:timestamp])}" if speed != 1
else else
BigBlueButton.logger.info " Generating silence" BigBlueButton.logger.info " Generating silence"