Commit Graph

1354 Commits

Author SHA1 Message Date
Fred Dixon
5743b29eba
Merge pull request #11805 from sebastianberm/patch-2
Updated usage section of analytics script to match actual syntax
2022-01-03 16:38:19 -04:00
Daniel Petri Rocha
14df8d110a
Bug fix: support recordings with no fill attribute
This script is failing for older recordings with no fill attribute. In that case,`event.at_xpath('fill')` is `nil`, failing once `.text` on `nil` is called.
2021-12-14 17:02:19 +01:00
Anton Georgiev
797fc49633 TEMP 2021-12-09 20:37:05 +00:00
Felipe Cecagno
b313c7deeb refactor(record-and-playback): replace trollop by optimist 2021-11-23 23:59:59 -03:00
Felipe Cecagno
2a4cc7b62e chore(record-and-playback): update nokogiri from 1.11.0 to 1.12.5 2021-11-23 23:59:07 -03:00
Daniel Petri Rocha
59c3590d3b Use first- and last_event_timestamps
As requested by @kepstin
2021-11-17 16:11:59 +01:00
Daniel Petri Rocha
3cd16ff7a3 XPATH query optimization: replace // 2021-11-16 00:56:16 +01:00
Daniel Petri Rocha
9be8e9b897 Remove temp dir from deskshare and webcam processing 2021-11-16 00:54:58 +01:00
Daniel Petri Rocha
06a9392af0 Prevent unnecessary file copying in presentation recording processing
Uses archived raw files directly instead of copying them over.
Specifies absolute paths in XPATH expressions for faster query speeds

Linting changes:
Removes useless assignments to ext, metadata_with_playback variables
Use frozen string literals
Use YAML safe load and JSON parse over load
Rescue StandardError instead of Exception
2021-11-16 00:11:20 +01:00
Anton Georgiev
019a0b25b9
Merge pull request #13564 from jfsiebel/remove-unused-property-chat-message
Remove unused properties from chat message event
2021-11-02 15:12:56 -04:00
Anton Georgiev
13aba985cb
Merge pull request #10737 from hiroshisuga/fillshape
feat: Fill shape in whiteboard
2021-11-02 15:03:22 -04:00
Joao Siebel
7a118ed3b8 Remove unused properties from chat message event record. close #11696 2021-10-25 14:46:59 -03:00
Calvin Walton
3c1a9cd7c4 RAP: Use "asetpts=N" in ffmpeg audio processing
There's an issue sometimes where when processing the audio from a
desktop sharing file, the STARTPTS value is invalid somehow, and this
results in bad timestamps in the output stream. Depending on the
selected codec/container combination, this might result in errors such
as spam of the message:

Application provided invalid, non monotonically increasing dts to muxer in stream

or simply a hang during processing.

Since we're already using aresample in async mode to correct timestamp
gaps, we can use "asetpts=N" here to simply set the audio frame pts
values to the sample number directly, giving proper monotonic timestamps
with no gaps.
2021-10-21 11:23:09 -04:00
prlanzarin
fbe1610e1c chore(recording): track mediasoup raw media files locations
I was going to resort to some trickery to make mediasoup raw files end up in the
same directory as KMS to reduce changes, but it ended up being too dirty.
I am adding a third directory (/var/mediasoup) to be tracked by the rap scripts
which is where the new raw files end up in.
2021-10-04 17:33:18 +00:00
Calvin Walton
1654f26084 recording: Anonymize names in chat
Move the handling of chat events into the shared library so it can be
used by multiple recording formats.

The anonymization of names is based on the external user id, if
available, so users have a consistent name through the meeting. Note
that no effort is made to edit chat messages - if someone is mentioned
by name in a chat message, that will still be visible.

Default settings for anonymization can be controlled in
bigbluebutton.yml, and per-meeting overrides can be done using meta
parameters on the create call.
2021-09-02 15:02:27 -04:00
Anton Georgiev
4ecb24b4fa Merge branch 'v2.3.x-release' of github.com:bigbluebutton/bigbluebutton into merge-aug30 2021-08-30 18:11:16 +00:00
Calvin Walton
93422b5f21 Move ffmpeg '-an' and '-vn' options out of the working format args
Disabling audio or video processing isn't really something that's part
of the working format (and at some point we might want to combine
audio+video processing together).

Move the setting of the '-an' and '-vn' options to where they're
required - as a detail of the EDL processing for video-only and
audio-only components of the output.

Honestly, the main reason for this change is that when testing alternate
working formats, I had accidentally dropped the '-vn' option from the
FFMPEG_WF_ARGS variable without noticing.
2021-08-26 11:20:52 -04:00
Calvin Walton
02d5fe14a6 Don't re-parse the events xml for the has_screenshare_audio check 2021-08-19 10:58:04 -04:00
Mikhail Novosyolov
340b0771e5 Avoid race condition between redis and bbb-rap-resque-worker
Redis must be started before bbb-rap-resque-worker.service which will connect to Redis
2021-08-19 02:37:20 +03:00
germanocaumo
4392b0d462 feat(recording): process published external videos
Generate a external_videos.json file at the recordings with an array of
played external videos url and timestamp.

This file will be published along with the other presentation format files
and can be used to display at the playback.
2021-07-07 13:26:24 +00:00
Pedro Beschorner Marin
93f00196a6 feat(recording): process published polls
Generate a poll JSON file at the recordings with an array of published
polling data:
 - type;
 - question;
 - answers;
 - number of responders; and
 - number of respondents

This file will be published along with the other presentation format files
and can be used to display the poll results at the playback.
2021-06-17 18:46:03 -03:00
Pedro Beschorner Marin
41daeca1a2 refactor(recording): callback url getter 2021-05-29 16:52:07 -03:00
Pedro Beschorner Marin
63dca7506e fix(recording): recording ready callback
Fetch for server's security salt at etc's bbb-web properties file.
2021-05-29 16:51:56 -03:00
hiroshisuga
e0a78ffbac
Merge branch 'develop' into fillshape 2021-05-14 20:57:16 +09:00
Pedro Beschorner Marin
3b83bddd86 feat(events): store etherpad events file 2021-05-04 13:06:12 -03:00
Pedro Beschorner Marin
83b48d5ee8 fix(events): keep events using resque workflow 2021-05-04 13:01:32 -03:00
Calvin Walton
9f5832a5d3 Recording: bbb-record --rebuild restarts processing at the sanity step
Previously, bbb-record --rebuild was restarting recording processing
from scratch by creating the .../recording/<meeting_id>.done file. This
causes the recording to be reprocessed starting at the archive step.
However, re-running the archive step for an existing meeting is not
really supported! Ever since the segmented recording code was added, it
shouldn't /corrupt/ the recording files, but it's still not good.

And as a side-effect, re-running the archive step will re-create the
.norecord file for meetings without recording marks, meaning that you
cannot use bbb-record --rebuild to force a recording without marks to be
processed.

Switch bbb-record to restart recording processing at the sanity stage to
match the BBB 2.2 behaviour. Rather than have it insert tasks directly
into resque via redis-cli, it goes through a ruby wrapper that performs
input validation and uses the resque apis.
2021-04-29 11:10:32 -04:00
Calvin Walton
8098b6b3ad Recording: Halt processing after archive step for non-recorded meetings
In the case where a meeting had recording enabled (record=true on create
call) but the presenter did not start recording during the meeting,
recording processing needs to be stopped after the meeting data is
archived, but before the recording formats are processed.

In the current 2.3 code, processing is halted after the "sanity" step.
However, the 2.2 code stopped processing after the "archive" step
instead. The main difference is that the scripts in the "post_archive"
directory (which are actually post_sanity scripts) did not get run on
non-recorded meetings for 2.2. This behaviour should be preserved for
compatibility.

I have added a special exception to trigger halting processing for a
recording job without causing the entire resque job to be marked as
failed. It only causes the `schedule_next_step` method to be skipped, so
following jobs won't get automatically run. This fixes #11877
2021-04-28 15:53:20 -04:00
Calvin Walton
9e7cecb033 Split the EDL recording marks generation out to a helper function
This function is useful any place you want the matched recording marks
with timestamps rebased so 0 is the start of the meeting, I've used it
for chat analysis, for example.

There is no functional change here, it only exposes the extra function
for recording scripts or dropin/post scripts to use.
2021-04-14 13:40:01 -04:00
Sebastian Berm
90f65edef1
Updated usage
Updated usage as this was still copypasted from the other file (post_events.rb), while this is technically not correct.
2021-03-30 14:15:56 +02:00
Richard Alam
55a416050c - store status when recording has no start/stop marks 2021-03-26 13:39:27 -07:00
Richard Alam
77b105aea2 Track recording process status
- store recording process status into a redis list. This option
   needs to be enabled in bigbluebutton.yml
2021-03-25 12:25:30 -07:00
Richard Alam
54f4d43bea Merge branch 'develop' of github.com:bigbluebutton/bigbluebutton into keep-recording-status-files 2021-03-25 12:14:36 -07:00
germanocaumo
951700502d Recording: Ignore video in audio processing scripts (fix error with screenshare input) 2021-03-25 16:30:32 +00:00
Calvin Walton
331c63adec Don't print audio info for corrupt audio files
Fixes a crash: undefined method `[]' for nil:NilClass
that happens when a corrupt audio file is hit.
2021-03-24 17:02:47 -04:00
Anton Georgiev
a950c9af53
Merge pull request #11626 from prlanzarin/u23-recsa
screenshare: add rap processing for screen streams with audio (complements #11622)
2021-03-24 14:36:31 -04:00
germanocaumo
ea375d6c5e remove trailing whitespaces 2021-03-24 14:01:17 -03:00
germanocaumo
35104fba47 number of inputs for amix command, remove old java deskshare events 2021-03-24 13:35:00 -03:00
Pedro Beschorner Marin
0c4cf0135d Handle pad's events and Meteor's instances
Since Meteor was split in multiple process and events started to be
filtered by instances, all Etherpad's Redis events were being discarded.

Etherpad has a Redis' publisher plugin that is unaware of BigBlueButton's
existence. All the communication between them is kept simple with minimal
of internal data exchange. The concept of distincts subscribers at Meteor's
side broke part of this simplicity and, now, Etherpad has to know which
instance must receive it's messages. To provide such information I decided
to include Meteor's instance as part of the pad's id. Should look like:

 - [instanceId]padId for the shared notes
 - [instanceId]padId_cc_(locale) for the closed captions

With those changes the pad id generation made at the recording scripts had to
be re-done because there is no instance id available. Pad id is now recorded at
akka-apps and queried while archiving the shared notes.
2021-03-23 18:03:50 -03:00
Mario Jr
af4f9dbf26 Merge branch 'v2.2.x-release' into merge-2.2-develop 2021-03-23 17:10:28 -03:00
germanocaumo
136f43eb25 Remove screenshare audio specific ffmpeg codec in recording processing. 2021-03-23 14:49:39 -03:00
germanocaumo
95fabfe8c2 Merge branch 'develop' of https://github.com/bigbluebutton/bigbluebutton into u23-recsa 2021-03-23 14:47:20 -03:00
Pedro Beschorner Marin
e48f64b843 Avoid localhost at notes endpoint
Use 127.0.0.1 instead of localhost since Etherpad only listens for ipv4.
2021-03-22 16:41:26 -03:00
Richard Alam
7d95fd9189 Keep recording status files
We still use the recording status files to externally monitor the
 progress of the recordings. Let's keep these files for now until
 we figure out a different way to track the status of the recording.
2021-03-22 09:24:02 -07:00
Fred Dixon
3fdb60ffef
Merge pull request #11685 from kepstin/slide-gen-interpolate
Recording: Don't use string interpolation when generating commands for slide conversion
2021-03-18 17:40:04 -03:00
Calvin Walton
7fab16b2ca Recording: Don't use string interpolation generating commands for slide conversion
Just want to make sure that we don't get any bad filenames sneaking in
and causing havoc.
2021-03-18 11:51:40 -04:00
Calvin Walton
0559152dc7 Improvements for audio desync issues
This incorporates only the audio desync related changes from #11626
* Add the aresample filter with async option to fill in timestamp gaps
* Use the libopus decoder for opus audio instead of ffmpeg's builtin
  decoder
2021-03-18 11:18:39 -04:00
Calvin Walton
2a6b0b62c3 Rework the ffmpeg filter generation in the audio rendering.
This gives the following advantages over the previous code:

* The ffmpeg input filters are loaded from a filter "script" file instead
  of passed on the command line. This fixes some cases of recordings
  failing to process because the ffmpeg command line generated for the
  audio processing exceeded the max command line length limit. (Although
  that only really happens due to BBB bugs...)
* Use absolute positions when trimming audio segments for cuts.
  Previously segments were trimmed to the length of the segment, and the
  results were concatenated. There's some possibility of accumulated
  errors in the segment lengths causing audio desync over time. The new
  code incrementally concatenates the segments, and cuts each segment
  end based on the absolute time since the start of the meeting, to
  avoid error accumulation.
2021-03-18 11:17:50 -04:00
germanocaumo
409e3cbc27 read screenshare audio from archived dir insteaf of original path 2021-03-11 21:40:19 +00:00
germanocaumo
5b07244e1e Fix last commit 2021-03-11 21:38:59 +00:00