Commit Graph

354 Commits

Author SHA1 Message Date
Calvin Walton
9e7cecb033 Split the EDL recording marks generation out to a helper function
This function is useful any place you want the matched recording marks
with timestamps rebased so 0 is the start of the meeting, I've used it
for chat analysis, for example.

There is no functional change here, it only exposes the extra function
for recording scripts or dropin/post scripts to use.
2021-04-14 13:40:01 -04:00
Richard Alam
55a416050c - store status when recording has no start/stop marks 2021-03-26 13:39:27 -07:00
Richard Alam
77b105aea2 Track recording process status
- store recording process status into a redis list. This option
   needs to be enabled in bigbluebutton.yml
2021-03-25 12:25:30 -07:00
Richard Alam
54f4d43bea Merge branch 'develop' of github.com:bigbluebutton/bigbluebutton into keep-recording-status-files 2021-03-25 12:14:36 -07:00
germanocaumo
951700502d Recording: Ignore video in audio processing scripts (fix error with screenshare input) 2021-03-25 16:30:32 +00:00
Calvin Walton
331c63adec Don't print audio info for corrupt audio files
Fixes a crash: undefined method `[]' for nil:NilClass
that happens when a corrupt audio file is hit.
2021-03-24 17:02:47 -04:00
Anton Georgiev
a950c9af53
Merge pull request #11626 from prlanzarin/u23-recsa
screenshare: add rap processing for screen streams with audio (complements #11622)
2021-03-24 14:36:31 -04:00
germanocaumo
ea375d6c5e remove trailing whitespaces 2021-03-24 14:01:17 -03:00
germanocaumo
35104fba47 number of inputs for amix command, remove old java deskshare events 2021-03-24 13:35:00 -03:00
Pedro Beschorner Marin
0c4cf0135d Handle pad's events and Meteor's instances
Since Meteor was split in multiple process and events started to be
filtered by instances, all Etherpad's Redis events were being discarded.

Etherpad has a Redis' publisher plugin that is unaware of BigBlueButton's
existence. All the communication between them is kept simple with minimal
of internal data exchange. The concept of distincts subscribers at Meteor's
side broke part of this simplicity and, now, Etherpad has to know which
instance must receive it's messages. To provide such information I decided
to include Meteor's instance as part of the pad's id. Should look like:

 - [instanceId]padId for the shared notes
 - [instanceId]padId_cc_(locale) for the closed captions

With those changes the pad id generation made at the recording scripts had to
be re-done because there is no instance id available. Pad id is now recorded at
akka-apps and queried while archiving the shared notes.
2021-03-23 18:03:50 -03:00
germanocaumo
136f43eb25 Remove screenshare audio specific ffmpeg codec in recording processing. 2021-03-23 14:49:39 -03:00
germanocaumo
95fabfe8c2 Merge branch 'develop' of https://github.com/bigbluebutton/bigbluebutton into u23-recsa 2021-03-23 14:47:20 -03:00
Richard Alam
7d95fd9189 Keep recording status files
We still use the recording status files to externally monitor the
 progress of the recordings. Let's keep these files for now until
 we figure out a different way to track the status of the recording.
2021-03-22 09:24:02 -07:00
Fred Dixon
3fdb60ffef
Merge pull request #11685 from kepstin/slide-gen-interpolate
Recording: Don't use string interpolation when generating commands for slide conversion
2021-03-18 17:40:04 -03:00
Calvin Walton
7fab16b2ca Recording: Don't use string interpolation generating commands for slide conversion
Just want to make sure that we don't get any bad filenames sneaking in
and causing havoc.
2021-03-18 11:51:40 -04:00
Calvin Walton
0559152dc7 Improvements for audio desync issues
This incorporates only the audio desync related changes from #11626
* Add the aresample filter with async option to fill in timestamp gaps
* Use the libopus decoder for opus audio instead of ffmpeg's builtin
  decoder
2021-03-18 11:18:39 -04:00
Calvin Walton
2a6b0b62c3 Rework the ffmpeg filter generation in the audio rendering.
This gives the following advantages over the previous code:

* The ffmpeg input filters are loaded from a filter "script" file instead
  of passed on the command line. This fixes some cases of recordings
  failing to process because the ffmpeg command line generated for the
  audio processing exceeded the max command line length limit. (Although
  that only really happens due to BBB bugs...)
* Use absolute positions when trimming audio segments for cuts.
  Previously segments were trimmed to the length of the segment, and the
  results were concatenated. There's some possibility of accumulated
  errors in the segment lengths causing audio desync over time. The new
  code incrementally concatenates the segments, and cuts each segment
  end based on the absolute time since the start of the meeting, to
  avoid error accumulation.
2021-03-18 11:17:50 -04:00
germanocaumo
409e3cbc27 read screenshare audio from archived dir insteaf of original path 2021-03-11 21:40:19 +00:00
germanocaumo
5b07244e1e Fix last commit 2021-03-11 21:38:59 +00:00
germanocaumo
4928ca91d1 Mix screenshare audio with mics again to avoid playback autoplay problems in iOS, fixed desync with aresample 2021-03-11 21:36:40 +00:00
germanocaumo
fdbbffb3aa Improve screenshare audio recording sync:
use libopus decoder and encoder, its better than built-in ffmpeg/flac
don't mix screenshare audio with mics, was generating desync with bad audio segments, encode it together with video file (TODO: needs adjustments in playback)
2021-03-11 21:32:02 +00:00
germanocaumo
0d48dd01cb Re-add screenshare audio recording processing 2021-03-11 21:09:07 +00:00
Andrei Bautu
18b3fc2684 Remove unused ffmpeg settings 2021-03-10 13:10:54 +02:00
Andrei Bautu
8cc99ef4ab Include comments about ffmpeg settings 2021-03-10 12:58:59 +02:00
Andrei Bautu
d71df37b8a Fix detection of empty video segments 2021-03-10 11:40:18 +02:00
Andrei Bautu
beb66512a0 Recording processing using MP4/OGG temporary files, stream copy, and customizable (lower) frame rates.
Discussion from https://github.com/bigbluebutton/bigbluebutton/issues/2483
2021-03-08 20:14:48 +02:00
Anton Georgiev
228bad03d5 Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into march4-merge 2021-03-04 21:25:47 +00:00
Pedro Beschorner Marin
c0a7f9cd92 Replace FNV32a pad's id generator with salted SHA1
When managing Etherpad's pads, Meteor makes API calls to initiate the closed captions
and shared notes modules. The pad id was being mapped to a shorter id than the meeting
id because of a Etherpad lenght limitation.

Changed to something less guessable.
2021-02-09 12:59:59 -03:00
Fred Dixon
e08ff14c52
Update events_worker.rb
Fix file existence test.
2020-12-20 18:38:18 -04:00
Anton Georgiev
0c7ead1916 Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into oct16-merge 2020-10-21 14:48:02 +00:00
Calvin Walton
1268753519 Fix io deadlock in recording scripts process execution utilities
The previous implementation of the BigBlueButton.execute method runs the
process with separate stdout and stderr streams. It first reads all of
the output from stdout, then reads all of the output from stderr.

This can cause a deadlock if the process writes a lot of data to stderr.
The IO buffer for stderr could fill, blocking progress. But since it
hasn't closed stdout, the ruby script is still waiting on a read to
stdout.

Switch to an execution method (using IO.popen) that allows combining
stdout and stderr into a single stream, eliminating the issue.
2020-09-23 16:16:57 -04:00
Anton Georgiev
f9d1079b41
Merge pull request #10238 from kepstin/rec-fixes-develop
Events archiver: Strip more invalid characters
2020-08-25 08:36:20 -04:00
hiroshisuga
3cbe9abaf0 events_archiver.rb: Handle an empty meeting_metadata hash from redis
On my server 2.3 alpha, the method metadata_for(meeting_id) gives back {}
(empty Hash). Thus "return if meeting_metadata.nil?" does not occur.

Does @redis.hgetall give {} instead of nil, even though there is a comment in
node_modules/redis/lib/utils.js "hgetall converts its replies to an Object. If
the reply is empty, null is returned"???
2020-08-13 15:46:40 -04:00
hiroshisuga
dc0fc459e2 events_archiver.rb: Handle an empty meeting_metadata hash from redis
On my server 2.3 alpha, the method metadata_for(meeting_id) gives back {}
(empty Hash). Thus "return if meeting_metadata.nil?" does not occur.

Does @redis.hgetall give {} instead of nil, even though there is a comment in
node_modules/redis/lib/utils.js "hgetall converts its replies to an Object. If
the reply is empty, null is returned"???
2020-08-13 15:42:22 -04:00
Calvin Walton
659893051d Events archiver: Strip more invalid characters
Update the list of invalid characters based on what the XML
specification permits and discourages.

Use the ruby string `scrub` method to remove invalid characters that
can't be expressed in the `tr` syntax, like unpaired surrogates and
UTF-8 prefix bytes.
2020-08-11 14:15:29 -04:00
Anton Georgiev
b0e1dc8378 Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into merge-2.2-into-2.3-july-6 2020-07-07 14:35:34 +00:00
Anton Georgiev
420f8bf0a4 Merge branch 'develop' of github.com:bigbluebutton/bigbluebutton into merge-2.2-into-2.3-july-6 2020-07-07 14:13:58 +00:00
Pedro Beschorner Marin
44fc68c07c Skip recording process step for norecord meetings 2020-07-01 12:33:41 -03:00
Pedro Beschorner Marin
526f79e8f2 Merge remote-tracking branch 'bigbluebutton/v2.2.x-release' into bigbluebutton/develop 2020-06-30 18:15:18 -03:00
Calvin Walton
26f5450245 Fix removing reconnect indicator from usernames when counting participants 2020-06-24 11:23:59 -04:00
mw781
f60059c983 Ensure that files are handled safely 2 2020-06-17 11:13:07 +01:00
Anton Georgiev
cc79c4b6ae merge 2.2.10 into 2.3 2020-05-05 19:52:44 +00:00
Pedro Beschorner Marin
d97f413ef6 Fix note id 2020-04-29 13:08:10 -03:00
Calvin Walton
20b150e86b Strip control characters from user-provided metadata in events file 2020-04-01 13:21:54 -04:00
Leonardo Crauss Daronco
5bc3062d58 Fix the paths to the ruby scripts on the rap workers
Was pointing to the wrong directory when loading the ruby scripts. Also
made the directory easier to configure, it's all in a single place now.
2020-02-06 10:43:32 -03:00
Calvin Walton
1a88830a27 RaP: Fix done filenames for segmented recordings
The done filenames all have to contain the break_timestamp to identify
the segment
2020-01-21 17:21:50 -05:00
Calvin Walton
c52a67186e RaP: Clean up resque worker code, add worker for keep_events feature
I've moved the workers code into the `lib` subdirectory with other library-ish
code; this puts it into the ruby load path used by most scripts so referencing
files is easier.

I've applied various style cleanups based on the rubocop config present.

The `events` processing step has been integrated as a new worker `EventsWorker`,
there is no longer a separate `events/events.rb` script. I've reworked the
`rap-starter.rb` script to check for the done files in both the events and
recorded status directories.
2020-01-21 17:14:39 -05:00
Leonardo Crauss Daronco
ac801cfce7 Use journald as the default logger for rap and rap services
Changed only in the main class so journald is the default and in the
scripts related to the processes in resque. Internal scripts might still
be logging to files.
2019-12-06 17:06:32 -03:00
Leonardo Crauss Daronco
5e05b7a8e8 Fixes for the rap scripts after processing a real recording 2019-12-06 12:26:24 -03:00
Leonardo Crauss Daronco
62d62a2a1e Merge remote-tracking branch 'upstream/master' into v2.0-rap-on-resque
# Conflicts:
#	bigbluebutton-config/bin/bbb-conf
#	record-and-playback/core/Gemfile.lock
#	record-and-playback/core/lib/recordandplayback/events_archiver.rb
#	record-and-playback/core/scripts/archive/archive.rb
#	record-and-playback/core/scripts/bigbluebutton.yml
#	record-and-playback/core/scripts/rap-process-worker.rb
#	record-and-playback/core/scripts/rap-publish-worker.rb
#	record-and-playback/core/scripts/rap-sanity-worker.rb
#	record-and-playback/core/scripts/rap-starter.rb
#	record-and-playback/core/scripts/sanity/sanity.rb
#	record-and-playback/core/systemd/bbb-rap-archive-worker.service
#	record-and-playback/core/systemd/bbb-rap-process-worker.service
#	record-and-playback/core/systemd/bbb-rap-publish-worker.service
#	record-and-playback/core/systemd/bbb-rap-sanity-worker.service
#	record-and-playback/core/systemd/bbb-record-core.target
2019-12-04 17:33:44 -03:00