Commit Graph

739 Commits

Author SHA1 Message Date
Anton Georgiev
9a2a50d2cc
recording: Set playback_protocol to https (#21287)
Changing the default value from http to https. At this point the playback links generated in the recording metadata.xml files should default to httpS protocol
2024-10-07 14:33:30 -04:00
Calvin Walton
4a3050eb30
Fix incorrect merge in recording audio processing (#21149)
The merge of 2.7 code into 3.0 dropped a change which was part of a fix
to prevent a recording processing hang when processing deskshare streams
which contain audio. Re-apply that change.
2024-09-11 11:55:59 -04:00
Calvin Walton
2bf55a0207 Recording: Do a test decode to detect video seek problems
I found a failed recording where a deskshare video file could not be
decoded when seeking was used (oddly enough, it could decode without
seeking). Turns out that remuxing the file was enough to make things
work.

In the video processing code, after checking for other known video file
problems that are fixed by remuxing, also try decoding one frame of the
video with a seek time applied.

Since more videos can end up being remuxed now, I moved the output of
the remux to a new subdirectory. Since more file types than just 'flv'
can be remuxed, I switched the output to the 'nut' container instead.
(This is ffmpeg's generic container format, and can hold any audio/video
which ffmpeg can process, so it works well for temporary files.)
2024-09-10 14:56:26 -04:00
Anton Georgiev
9cee20c55e
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities (#21021)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-REXML-7814166

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-08-29 16:39:55 -04:00
Jesus Federico
6333cc42ff
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities (#20723)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-REXML-7462086

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-07-18 09:39:30 -04:00
Anton Georgiev
7d6a7ef73b
Revert "[Snyk] Fix for 1 vulnerabilities" 2024-05-31 15:34:51 -04:00
snyk-bot
efa8980396
fix: record-and-playback/core/Gemfile to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-NOKOGIRI-7164639
2024-05-31 03:15:29 +00:00
snyk-bot
9909d1b7e9
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-REXML-6861566
2024-05-17 16:21:02 +00:00
Anton Georgiev
23ea53e452
recording: work around invalid locale in caption events (#19682)
This is a workaround for #19178 - but it does not fix the issue. The
caption recording events with invalid empty `<locale/>` are simply
dropped with a warning message, to allow the recording and any valid
caption streams present to be processed.

Co-authored-by: Calvin Walton <calvin.walton@blindsidenetworks.com>
2024-02-23 20:59:34 -05:00
Anton Georgiev
e9ec5ea645
Merge pull request #17884 from Ithanil/nicer_ffmpeg
build: Process recordings with Niceness 19
2024-01-22 11:37:45 -05:00
Daniel Petri Rocha
6c766acc9c
Use semantic versioning
Both bbb-playback and the recording processing scripts fail if the version string is not in the format MAJOR.MINOR.PATCH (3.0.0 in this case instead of 3.0)
2023-12-14 23:10:24 +01:00
danielpetri1
84d341cb94 Bump bbb_version to 3.0 in bigbluebutton.yml 2023-12-12 19:41:09 +01:00
danielpetri1
5fff226d55 Remove leftover diff from merge conflict 2023-11-19 18:43:21 +00:00
André
96d9f71030 v27 changes ported to v30 part 2. 2023-11-10 15:09:45 -03:00
Gustavo Trott
9da3543bc3 Resolve conflicts 2023-09-11 11:25:56 -03:00
prlanzarin
40b642969c fix(recording): handle archive remux failures
Recording archive may fail when remuxing invalid files from KMS or the
new recorder - eg when the raw files are 0-byte sized.

This commit handles the exception raised by EDL::encode so archive keeps
going, logs the issue as a warning and archives the problematic file anyways.
EDL::encode now removes the temporary file when the ffmpeg command execution
fails - this should avoid leaving any stale files around in case of failure.

No specific check for the nature of the error is done - the idea is that
subsequent phases will discard or fix the files if necessary according
to the processing scripts' necessities, making the behavior (in this
specific scenario) similar to what it was before the archive remuxing was
introduced.
2023-09-07 11:57:55 -03:00
Gustavo Trott
5eb04aab91 Resolve conflicts 2023-08-30 15:31:32 -03:00
Gustavo Trott
201b6949a3 Resolve conflicts 2023-08-22 15:07:42 -03:00
snyk-bot
8971f3597b
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-RACK-1061917
2023-08-20 20:05:57 +00:00
prlanzarin
bba4658dd3 fix(recording): remux bbb-webrtc-recorder and KMS files during archive
Kurento may *rarely* generate WebM/MKV files with corrupt or absent
SeekHead sectors. bbb-webrtc-recorder also doesn't generate SeekHead or
even the Cues sectors by default.

While those are are *optional* fields by spec, files need to be seekable
for our recording processing scripts to work.

This commit adds a remuxing step for Kurento and bbb-webrtc-recorder raw
files that is executed during the archive phase. It should re-include
any of the missing fields that make files seekable and restore the Cues
sector in WebM files.
2023-08-17 12:38:52 -03:00
Gustavo Trott
21f1ed9f27 Merge 2.7 into Develop 2023-07-14 17:42:13 -03:00
Anton Georgiev
cd9f93be44 Merge remote-tracking branch 'bbb/v2.6.x-release' into merge-july12 2023-07-12 15:59:46 -04:00
snyk-bot
2cd88df310
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-ACTIVESUPPORT-3237242
- https://snyk.io/vuln/SNYK-RUBY-ACTIVESUPPORT-3360028
2023-06-23 22:20:37 +00:00
Anton Georgiev
70b3220f41 Merge branch 'v2.7.x-release' of github.com:bigbluebutton/bigbluebutton into june5-27 2023-06-05 17:09:35 -04:00
Anton Georgiev
748c7a4a0b Merge branch 'v2.6.x-release' of github.com:bigbluebutton/bigbluebutton into june227 2023-06-02 09:45:34 -04:00
Gustavo Trott
51bdcededc Merge 2.7 into develop 2023-05-26 10:50:39 -03:00
snyk-bot
a6fa4b785f
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-RACK-1061917
2023-05-25 15:01:58 +00:00
snyk-bot
e828923ca9
fix: record-and-playback/core/Gemfile & record-and-playback/core/Gemfile.lock to reduce vulnerabilities
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-RUBY-RACK-3237233
- https://snyk.io/vuln/SNYK-RUBY-RACK-3237237
- https://snyk.io/vuln/SNYK-RUBY-RACK-3237240
- https://snyk.io/vuln/SNYK-RUBY-RACK-3356639
- https://snyk.io/vuln/SNYK-RUBY-RACK-3360233
2023-05-25 14:21:38 +00:00
Anton Georgiev
25af259d2d Merge branch 'v2.6.x-release' of github.com:bigbluebutton/bigbluebutton into merge-may-24 2023-05-24 17:23:00 -04:00
hiroshisuga
71c54860a6
Update gen_webvtt 2023-05-22 12:18:17 +09:00
hiroshisuga
2ae085c5e0
Update README 2023-05-21 21:21:38 +09:00
Jan Kessler
13ddb91398
set bbb-rap-caption-inbox niceness to 19 2023-05-16 07:09:26 +02:00
Jan Kessler
7660171aff
set bbb-rap-resque-worker niceness to 19 2023-05-16 07:06:55 +02:00
Anton Georgiev
0fccb78ee1 build-conf: Remove Kurento dependency; Run on Ubuntu 22.04 2023-04-25 15:08:36 -04:00
prlanzarin
538954a6ae build: add bbb-webrtc-recorder 2023-04-20 16:24:34 -03:00
Calvin Walton
08453469b3 recording: Fix video processing incompatibility with Ruby 2.7
The Fcntl::F_SETPIPE_SZ constant was added in Ruby 3.0, but Ubuntu 20.04
still uses Ruby 2.7. Add some error handling so processing doesn't fail
if the constant is not found.
2023-03-31 11:46:09 -04:00
Anton Georgiev
5887effe7c
Merge pull request #17097 from ramonlsouza/issue-17086
fix(whiteboard): Reduce max canvas size
2023-03-28 11:06:10 -04:00
Calvin Walton
8b8fe76bbb recording: Increase quality of libx264 intermediate files
The encoding settings for intermediate files was using -preset veryfast
-crf 30, which resulted in very poor quality video.

After a bit of experimenting, I decided to change this to -preset
veryfast -crf 23. This results in files which are roughly twice the size
of before, but they look significantly better.

There's improvements possible at the same filesize by switching to a
slower encoding mode. But in the case of -preset medium for example,
when normalized to the same output file size, you end up using about 1.5
times as much cpu time to gain only a very small amount of video
quality.
2023-03-27 18:04:59 -04:00
Calvin Walton
e4f9493bd7 recording: Fix handling of demoting a moderator when webcams locked
A comparison was being done against the wrong variable, resulting in the
empty string filename being added to the inactive videos list. This
caused a crash later in the code.
2023-03-21 18:07:45 -04:00
Calvin Walton
057d3abaf5 recording: Fix video processing OOM, Broken pipe errors
In a particular case where you have a large timestamp gap followed by a
frame which re-initializes the filters in the pre-processing ffmpeg
(e.g. due to a resolution change), the fps filter will keep generating
frames to fill this gap even if downstream filters aren't accepting more
frames. Add a trim filter which will eat the frames past the desired end
timestamp to prevent them from getting queued up.

Additionally, in cases with unlucky timing on the filter
re-initialization, the pre-processing ffmpeg can end up generating some
output past the set end time. Since the compositing ffmpeg exits once it
has read enough input, this can cause the pre-processing ffmpeg to fail
with a "Broken pipe" error. To work around this problem, the processing
scripts themselves can open the pipe for reading to hold it open, and
then send a signal to the pre-processing ffmpeg to tell it to exit. This
results in ffmpeg exiting with the return code 255, which can be
distinguished from actual errors.

As a bonus, opening the fifo in the processing script allows increasing
the size of the pipe buffer, which should result in slightly better
performance.
2023-03-21 18:07:45 -04:00
Ramón Souza
3d4d623f16 update bbb version in record-and-playback 2023-03-21 15:07:10 -03:00
Calvin Walton
dc9126b67b recording: Skip EDL area if it is nil
The recording processing would crash if an area was present in the
layout, but was missing in the EDL entry being processed.

This doesn't happen in normal conditions, since most of the methods for
generating an EDL will result in areas being present, even if there are
no videos for that area.

But the EDL cleanup recently added can sometimes cause an EDL entry with
no areas to be processed, so add the code to handle this possibility.
2023-02-21 15:28:24 -05:00
Calvin Walton
54e128f0d3 recording: Don't use tpad to pad start of videos
The tpad filter doesn't actually extend the video backwards (i.e. it
does not create frames with timestamps earlier than the first video
timestamp). Instead, it *delays* the start of the video.

Using it incorrectly was causing audio/video desyncs in the desktop
sharing, and also occasional processing failure if it pushed back the
video enough that the compositing ffmpeg process didn't end up reading
to the end of the input video.

Use the fps filter's "start_time" option instead, which *does* extend
the video backwards to the configured start time by duplicating the
first frame.
2023-02-21 15:23:44 -05:00
Calvin Walton
49cf81f4a0 recording: Enforce minimum length of video segments
The "EDL" provided to the recording video processing can sometimes
contain a series of cuts in very close succession - just milliseconds
between them - purely by chance (e.g. two webcams disconnect at almost
the same time). Right now this can result in segments failing to
process (in some rare cases) or if a segment processes but was detected
to be empty (no frames), it'll get discarded.

There's also some problems which can occur that cause a too-short
recording - just milliseconds between start and stop, or between start
and the meeting end - which also currently fail to process. We've found
it's better for user feedback/support if the recording successfully
processes with no content in this case.

Add a cleanup function that goes through the EDL and consolidates cuts
which are too close together, and ensures that the final output meets
a minimum length.
2023-02-16 11:46:45 -05:00
Calvin Walton
5adbcae7cb recording: Add workaround for webacmsOnlyForModerator typo
There was a brief period during 2.5 development when recordings had the
typo "webacms" instead of "webcams" on one of the event names. I hit
this now and then, so just check for both names in the recording
processing.
2023-02-15 11:50:56 -05:00
Calvin Walton
27e66df19c recording: Fix tests for get_chat_events
The code no longer retrieves chat or avatar color, so stop checking for
those in the tests.

Fix the get_chat_events code to include the sender name when the sender
id is not available (this only happens on *really* old recordings, but
it's a trivial fix).
2023-02-15 10:46:12 -05:00
Anton Georgiev
b5759533d2
Merge pull request #16698 from kepstin/video-recording
recording: video format updates & fixes
2023-02-13 14:41:12 -05:00
Calvin Walton
6379a6f50d recording: video format updates & fixes
A few minor updates and fixes to the video recording format:

* The 'show_moderator_viewpoint' recording setting is now honoured.
* The desktop sharing video replaces the presentation area - it no
  longer hides webcams (it now matches the live meeting).
* The 'playback_protocol' recording setting is now honoured (recording
  links will correctly use https when that's configured).
2023-02-10 12:21:27 -05:00
Calvin Walton
1dd40b8380 recording: Fix handling deskshare with delayed video
When a deskshare stream with combined audio + video starts up, it can
happen that the audio starts before the video - so the first video frame
will be some amount of time after the file start.

If there's a recording processing cut in this gap, the procesing can
crash because it can generate an output video with no video frames.

There are two parts to the fix:
  * Trim input videos with the trim filter, configured to ensure at
    least 1 output frame is generated, even if it would be after the
    end timestamp.
  * Use the tpad filter to pad the *start* of a video stream to make
    sure there's something in the gap.
2023-02-10 11:27:18 -05:00
Calvin Walton
46970162fb Recording: move tpad ffmpeg filter before fps
In cases of extremely short (single frame) input videos, the fps filter
can sometimes generate 0-frame output videos, resulting in the tpad
filter having no input (this breaks it, causing a busy loop).

Move the tpad filter to before the fps filter to solve this problem.
This isn't perfect, since the tpad filter doesn't work well on variable-
framerate video (it generates extremely high framerate video with a lot
of frames that will be discarded), but this only happens between the
tpad and fps filters, and only at the end of an input video (usually
right before a cut) so this seems acceptable.

Since the tpad and fps filter are in the same process, these duplicate
frames don't actually require copying any data (the frame is
reference-counted), and still process reasonably quickly.

Fixes #16407
2023-01-06 10:56:50 -05:00