Commit Graph

711 Commits

Author SHA1 Message Date
snyk-bot
db42028f5c
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-NOKOGIRI-6228056
2024-04-05 00:31:17 +00:00
snyk-bot
fa07ed9a27 fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-RACK-1061917
2023-11-15 16:31:39 -05:00
Calvin Walton
9d02717b04 fix(recording): Deskshare audio processing hang
This is a refinement of the fix introduced in 3c1a9cd7c4.

Further investigation of the issue reveals that the cause of the bad
timestamps which result in error messages or hang is when the `apad`
filter does not receive any input audio frames. This happens if the
seekpoint is after the end of audio. The existing seekpoint checks can't
cover this, because in a deskshare file the video stream can continue on
after the audio stream ends.

The `,asetpts=N` filter was added to deal with this problem, and it
works in most cases. But there is a case where it fails - when the
"mismatched length" audio stretching to work around broken recordings
from BigBlueButton 0.81/0.90 kicks in.

The issue there is that the `atrim=start=S` filter (used due to the
difficulty of calculating seeks when stretching) hits the invalid
timestamps and hangs.

I'm working around this issue with a "defense in depth" combination of
two changes:

* Move the `,asetpts=N` filter to be applied before the audio stretch
  filters. This fixes the processing hang.
* Adjust the conditions on the "mismatched length" audio stretching so
  that it only gets applied on audio files likely to be from extremely
  old BigBlueButton versions - those with audio in wav files, or encoded
  to vorbis.

BigBlueButton has been using audio recorded directly to opus by
FreeSWITCH for quite a while, and the handling for gaps or lost packets
is done in current BBB versions by a combination of the libopus decoder
and the use of ffmpeg's `aresample=async=1000` filter to dynamically
stretch, squish, or fill in audio so it becomes a continuous stream
that's locked to the file timestamps. Applying the "mismatched length"
processing on top of that is probably making audio sync issues worse.
2023-10-04 13:30:59 -04:00
snyk-bot
2cd88df310
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-ACTIVESUPPORT-3237242
- https://snyk.io/vuln/SNYK-RUBY-ACTIVESUPPORT-3360028
2023-06-23 22:20:37 +00:00
snyk-bot
a6fa4b785f
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-RACK-1061917
2023-05-25 15:01:58 +00:00
snyk-bot
e828923ca9
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-RACK-3237233
- https://snyk.io/vuln/SNYK-RUBY-RACK-3237237
- https://snyk.io/vuln/SNYK-RUBY-RACK-3237240
- https://snyk.io/vuln/SNYK-RUBY-RACK-3356639
- https://snyk.io/vuln/SNYK-RUBY-RACK-3360233
2023-05-25 14:21:38 +00:00
hiroshisuga
2ae085c5e0
Update README 2023-05-21 21:21:38 +09:00
Calvin Walton
08453469b3 recording: Fix video processing incompatibility with Ruby 2.7
The Fcntl::F_SETPIPE_SZ constant was added in Ruby 3.0, but Ubuntu 20.04
still uses Ruby 2.7. Add some error handling so processing doesn't fail
if the constant is not found.
2023-03-31 11:46:09 -04:00
Anton Georgiev
5887effe7c
Merge pull request #17097 from ramonlsouza/issue-17086
fix(whiteboard): Reduce max canvas size
2023-03-28 11:06:10 -04:00
Calvin Walton
8b8fe76bbb recording: Increase quality of libx264 intermediate files
The encoding settings for intermediate files was using -preset veryfast
-crf 30, which resulted in very poor quality video.

After a bit of experimenting, I decided to change this to -preset
veryfast -crf 23. This results in files which are roughly twice the size
of before, but they look significantly better.

There's improvements possible at the same filesize by switching to a
slower encoding mode. But in the case of -preset medium for example,
when normalized to the same output file size, you end up using about 1.5
times as much cpu time to gain only a very small amount of video
quality.
2023-03-27 18:04:59 -04:00
Calvin Walton
e4f9493bd7 recording: Fix handling of demoting a moderator when webcams locked
A comparison was being done against the wrong variable, resulting in the
empty string filename being added to the inactive videos list. This
caused a crash later in the code.
2023-03-21 18:07:45 -04:00
Calvin Walton
057d3abaf5 recording: Fix video processing OOM, Broken pipe errors
In a particular case where you have a large timestamp gap followed by a
frame which re-initializes the filters in the pre-processing ffmpeg
(e.g. due to a resolution change), the fps filter will keep generating
frames to fill this gap even if downstream filters aren't accepting more
frames. Add a trim filter which will eat the frames past the desired end
timestamp to prevent them from getting queued up.

Additionally, in cases with unlucky timing on the filter
re-initialization, the pre-processing ffmpeg can end up generating some
output past the set end time. Since the compositing ffmpeg exits once it
has read enough input, this can cause the pre-processing ffmpeg to fail
with a "Broken pipe" error. To work around this problem, the processing
scripts themselves can open the pipe for reading to hold it open, and
then send a signal to the pre-processing ffmpeg to tell it to exit. This
results in ffmpeg exiting with the return code 255, which can be
distinguished from actual errors.

As a bonus, opening the fifo in the processing script allows increasing
the size of the pipe buffer, which should result in slightly better
performance.
2023-03-21 18:07:45 -04:00
Ramón Souza
3d4d623f16 update bbb version in record-and-playback 2023-03-21 15:07:10 -03:00
Calvin Walton
dc9126b67b recording: Skip EDL area if it is nil
The recording processing would crash if an area was present in the
layout, but was missing in the EDL entry being processed.

This doesn't happen in normal conditions, since most of the methods for
generating an EDL will result in areas being present, even if there are
no videos for that area.

But the EDL cleanup recently added can sometimes cause an EDL entry with
no areas to be processed, so add the code to handle this possibility.
2023-02-21 15:28:24 -05:00
Calvin Walton
54e128f0d3 recording: Don't use tpad to pad start of videos
The tpad filter doesn't actually extend the video backwards (i.e. it
does not create frames with timestamps earlier than the first video
timestamp). Instead, it *delays* the start of the video.

Using it incorrectly was causing audio/video desyncs in the desktop
sharing, and also occasional processing failure if it pushed back the
video enough that the compositing ffmpeg process didn't end up reading
to the end of the input video.

Use the fps filter's "start_time" option instead, which *does* extend
the video backwards to the configured start time by duplicating the
first frame.
2023-02-21 15:23:44 -05:00
Calvin Walton
49cf81f4a0 recording: Enforce minimum length of video segments
The "EDL" provided to the recording video processing can sometimes
contain a series of cuts in very close succession - just milliseconds
between them - purely by chance (e.g. two webcams disconnect at almost
the same time). Right now this can result in segments failing to
process (in some rare cases) or if a segment processes but was detected
to be empty (no frames), it'll get discarded.

There's also some problems which can occur that cause a too-short
recording - just milliseconds between start and stop, or between start
and the meeting end - which also currently fail to process. We've found
it's better for user feedback/support if the recording successfully
processes with no content in this case.

Add a cleanup function that goes through the EDL and consolidates cuts
which are too close together, and ensures that the final output meets
a minimum length.
2023-02-16 11:46:45 -05:00
Calvin Walton
5adbcae7cb recording: Add workaround for webacmsOnlyForModerator typo
There was a brief period during 2.5 development when recordings had the
typo "webacms" instead of "webcams" on one of the event names. I hit
this now and then, so just check for both names in the recording
processing.
2023-02-15 11:50:56 -05:00
Calvin Walton
27e66df19c recording: Fix tests for get_chat_events
The code no longer retrieves chat or avatar color, so stop checking for
those in the tests.

Fix the get_chat_events code to include the sender name when the sender
id is not available (this only happens on *really* old recordings, but
it's a trivial fix).
2023-02-15 10:46:12 -05:00
Anton Georgiev
b5759533d2
Merge pull request #16698 from kepstin/video-recording
recording: video format updates & fixes
2023-02-13 14:41:12 -05:00
Calvin Walton
6379a6f50d recording: video format updates & fixes
A few minor updates and fixes to the video recording format:

* The 'show_moderator_viewpoint' recording setting is now honoured.
* The desktop sharing video replaces the presentation area - it no
  longer hides webcams (it now matches the live meeting).
* The 'playback_protocol' recording setting is now honoured (recording
  links will correctly use https when that's configured).
2023-02-10 12:21:27 -05:00
Calvin Walton
1dd40b8380 recording: Fix handling deskshare with delayed video
When a deskshare stream with combined audio + video starts up, it can
happen that the audio starts before the video - so the first video frame
will be some amount of time after the file start.

If there's a recording processing cut in this gap, the procesing can
crash because it can generate an output video with no video frames.

There are two parts to the fix:
  * Trim input videos with the trim filter, configured to ensure at
    least 1 output frame is generated, even if it would be after the
    end timestamp.
  * Use the tpad filter to pad the *start* of a video stream to make
    sure there's something in the gap.
2023-02-10 11:27:18 -05:00
Calvin Walton
46970162fb Recording: move tpad ffmpeg filter before fps
In cases of extremely short (single frame) input videos, the fps filter
can sometimes generate 0-frame output videos, resulting in the tpad
filter having no input (this breaks it, causing a busy loop).

Move the tpad filter to before the fps filter to solve this problem.
This isn't perfect, since the tpad filter doesn't work well on variable-
framerate video (it generates extremely high framerate video with a lot
of frames that will be discarded), but this only happens between the
tpad and fps filters, and only at the end of an input video (usually
right before a cut) so this seems acceptable.

Since the tpad and fps filter are in the same process, these duplicate
frames don't actually require copying any data (the frame is
reference-counted), and still process reasonably quickly.

Fixes #16407
2023-01-06 10:56:50 -05:00
Anton Georgiev
2841a78848 Merge branch 'v2.5.x-release' of github.com:bigbluebutton/bigbluebutton into port-15552 2022-12-15 13:55:24 +00:00
GuiLeme
af49e25c21 Merge remote-tracking branch 'upstream/v2.5.x-release' into update-gem-tzinfo 2022-12-15 09:42:18 -03:00
GuiLeme
1ba26ea3c3 [update-gem-tzinfo] - update of dependencies - nokogiri, loofah, tzinfo, rack 2022-12-15 09:07:16 -03:00
Calvin Walton
06d0e4d454 Recording: Move tpad filter to after fps filter
The tpad filter is problematic on the variable-framerate webcam files,
and the result can end up being hangs (or, at least, very slow
processing) in the compositing.

Move the tpad filter to the compositing process where it can run after
the fps filter has converted the video to constant framerate. It still
needs to run before the start trimming, so switch to using the trim
filter rather than the fps filter's start_pts feature.
2022-12-06 16:00:39 -05:00
Calvin Walton
5cb32ec088 Recording: Move tpad filter to after fps filter
The tpad filter is problematic on the variable-framerate webcam files,
and the result can end up being hangs (or, at least, very slow
processing) in the compositing.

Move the tpad filter to the compositing process where it can run after
the fps filter has converted the video to constant framerate. It still
needs to run before the start trimming, so switch to using the trim
filter rather than the fps filter's start_pts feature.
2022-12-06 15:59:54 -05:00
Calvin Walton
4127f4ccc7 Recording: Pre-process video files using separate ffmpeg
Even with the filter changes made, there's still some cases where
filter chain hangs can result from filter reconfigurations. To solve the
issue completely, I have split out pre-processing video files to
separate ffmpeg processes, so that the filter chain for compositing will
not ever be reconfigured.

Each input video now has a separate ffmpeg process run for it which
does the scaling, padding, and video extending steps. To avoid issues
with disk space usage and extra cpu usage or quality loss, the output
from these separate processes is sent to the compositing ffmpeg process
as uncompressed video in a pipe. To simplify the setup, named pipes
(special files) are used rather than setting up pipes in the ruby code
programmatically.

The extra ffmpeg processes are configured to log to files, and when
complete their log output is copied to the recording processing log.
Processes are joined to ensure zombie processes are not created, and
the return codes of all the processes are checked so errors can be
detected.

Due to the overhead of transferring video through pipes, this might
be a bit slower than the 2.4 recording processing - but on the other
hand, some of the video decoding and scaling happens in parallel, so it
might balance out.
2022-12-05 11:55:59 -05:00
Calvin Walton
58c4ffd068 Recording: Pre-process video files using separate ffmpeg
Even with the filter changes made, there's still some cases where
filter chain hangs can result from filter reconfigurations. To solve the
issue completely, I have split out pre-processing video files to
separate ffmpeg processes, so that the filter chain for compositing will
not ever be reconfigured.

Each input video now has a separate ffmpeg process run for it which
does the scaling, padding, and video extending steps. To avoid issues
with disk space usage and extra cpu usage or quality loss, the output
from these separate processes is sent to the compositing ffmpeg process
as uncompressed video in a pipe. To simplify the setup, named pipes
(special files) are used rather than setting up pipes in the ruby code
programmatically.

The extra ffmpeg processes are configured to log to files, and when
complete their log output is copied to the recording processing log.
Processes are joined to ensure zombie processes are not created, and
the return codes of all the processes are checked so errors can be
detected.

Due to the overhead of transferring video through pipes, this might
be a bit slower than the 2.4 recording processing - but on the other
hand, some of the video decoding and scaling happens in parallel, so it
might balance out.
2022-12-05 11:49:16 -05:00
Calvin Walton
276e592c01 Recording: Don't use stateful filters in ffmpeg video processing
Because the input videos for BigBlueButton recording processing switch
resolution and aspect ratio, the filter chain gets re-initialized, and
any state in the filters is lost. This causes problems with the
following filters:

`color`: Timestamps restart from 0, rather than continuing at the point
where they left off.
`fps=start_time=12.345`: After reset, the fps filter thinks it's at the
start of the file again, so the next frame it sees gets duplicated
output for timestamps from the `start_time` until it catches back up.
`setpts=PTS-STARTPTS`: The 'STARTPTS' is re-read as the first pts the
filter sees after reinitialization, so timestamp of the next frame is
reset to 0. (In practise, this didn't cause any problems because the
duplicate frames created by the fps filter had the original start time.)

The end result of all of these issues is that a lot of duplicate frames
were created with invalid timestamps, which eventually get discarded
by ffmpeg. But a lot of time is wasted, causing recordings to sometimes
take hours to process when they should be ready in minutes.

The fixes are as follows:

* The `color` filters are used to generate the background and
  substitutes for missing videos. Move them out to separate filter
  chains by using the 'lavfi' input format, which lets you use a filter
  as if it was an input file.
* Rather than use the `fps` filter's `start_time` feature, use the
  `trim` filter to remove early frames.
* The actual start pts is already known by the script, so replace
  `setpts=PTS-STARTPTS` with `setpts=PTS-12.345/TB`, substituting in the
  absolute time.
2022-11-22 15:01:45 -05:00
Calvin Walton
aa1aef6727 Recording: Don't use stateful filters in ffmpeg video processing
Because the input videos for BigBlueButton recording processing switch
resolution and aspect ratio, the filter chain gets re-initialized, and
any state in the filters is lost. This causes problems with the
following filters:

`color`: Timestamps restart from 0, rather than continuing at the point
where they left off.
`fps=start_time=12.345`: After reset, the fps filter thinks it's at the
start of the file again, so the next frame it sees gets duplicated
output for timestamps from the `start_time` until it catches back up.
`setpts=PTS-STARTPTS`: The 'STARTPTS' is re-read as the first pts the
filter sees after reinitialization, so timestamp of the next frame is
reset to 0. (In practise, this didn't cause any problems because the
duplicate frames created by the fps filter had the original start time.)

The end result of all of these issues is that a lot of duplicate frames
were created with invalid timestamps, which eventually get discarded
by ffmpeg. But a lot of time is wasted, causing recordings to sometimes
take hours to process when they should be ready in minutes.

The fixes are as follows:

* The `color` filters are used to generate the background and
  substitutes for missing videos. Move them out to separate filter
  chains by using the 'lavfi' input format, which lets you use a filter
  as if it was an input file.
* Rather than use the `fps` filter's `start_time` feature, use the
  `trim` filter to remove early frames.
* The actual start pts is already known by the script, so replace
  `setpts=PTS-STARTPTS` with `setpts=PTS-12.345/TB`, substituting in the
  absolute time.
2022-11-22 13:35:48 -05:00
GuiLeme
3f9fcc693f [update-gem-tzinfo] - updated resque gem 2022-11-21 17:05:26 -03:00
Ramón Souza
2b0971e2c8 Merge tag 'v2.5.6' into merge-256-26 2022-09-26 09:17:59 -03:00
Calvin Walton
fb43eba355 recording: Fix incorrect seek offsets in video
When converting from using the 'movie' source filter to using separate
ffmpeg command line inputs, it wasn't taken into account that the
'movie' filter passes through timestamps from the source file, while the
ffmpeg input adjusts timestamps so that '0' is the selected seek point.

The easiest fix is to add an option to the ffmpeg command to disable the
timestamp adjustments.

Fixes #15644 (This needs to go into BigBlueButton 2.5 and 2.6)
2022-09-16 12:14:19 -04:00
germanocaumo
d609628047 refactor(recording): remove not used event 2022-08-23 18:39:46 +00:00
GuiLeme
f160efe8fc [update-gem-tzinfo] - update tzinfo, redis-namspace and resque gem 2022-08-17 10:30:28 -03:00
Ramón Souza
ded8493f05 Merge remote-tracking branch 'upstream/v2.5.x-release' into 2526-aug3 2022-08-03 09:53:41 -03:00
germanocaumo
1ae8e5c35d fix(recording): bump bbb_version to 2.6.0 in events.xml
- Update the version to 2.6.0 to ease the detection of old/new whiteboard events
- Fix recording cursor when there is no pan/zoom and annotations in tldraw
- Don't generate slides pngs for 2.6.0, they are not used anymore in playback (svgs instead).
2022-07-29 11:18:30 +00:00
Anton Georgiev
cb1543e1fa
Merge pull request #15149 from kepstin/video-resize
fix: Support deskshare videos that resize
2022-07-25 13:36:00 -04:00
Ramon Souza
db5ac1428a Merge tag 'v2.5.3' into merge25-26-jul14 2022-07-15 11:08:02 -03:00
Anton Georgiev
55dbc34984
Merge pull request #14616 from danimo/bbb-target
build: Introduce bigluebutton.target
2022-07-06 14:42:09 -04:00
Daniel Petri Rocha
ef45c9675f Add cropbox parameter to thumbnails and recording processing step 2022-07-06 18:16:08 +02:00
germanocaumo
c2db91b5f9 Merge branch 'v2.6.x-release' of https://github.com/bigbluebutton/bigbluebutton into tldraw-recording 2022-06-30 14:31:08 +00:00
GuiLeme
be62c6d19a [update-rubocop] - ran bundle install and updated rubocop 2022-06-29 17:38:39 -03:00
Anton Georgiev
61b269eca8
Merge pull request #15171 from GuiLeme/issue-15051
refactor: Removed all traces of Red5
2022-06-29 12:09:02 -04:00
Daniel Molkentin
a7f43ba2b8 Merge remote-tracking branch 'origin/v2.6.x-release' into bbb-target 2022-06-27 17:03:01 +02:00
Anton Georgiev
fa5ea33c6c
Merge pull request #15100 from schrd/fix-issue-10989
Fix(build): add service dependency to redis for several services
2022-06-23 15:40:38 -04:00
germanocaumo
853d0dfd9b fix(whiteboard): tldraw recording processing/publishing
Changed the names of tldraw record events to differentiate from before.
Publish tldraw.json file with all shape information during the meeting to be used in playback.
Adapted cursor.xml and panzoom.xml to store tldraw data.
Publish slides svgs to be used by playback's tldraw component (otherwise we have different image sizes in pngs and thus messing the coordinates).
Retro-compatible with old recordings.
2022-06-23 17:04:09 +00:00
GuiLeme
ac2e68842c Merge remote-tracking branch 'upstream/v2.6.x-release' into issue-15051 2022-06-20 16:12:25 -03:00
GuiLeme
7363c36448 [issue-15160] - fixed error logs in recordings 2022-06-16 09:13:31 -03:00