Commit Graph

283 Commits

Author SHA1 Message Date
Leonardo Crauss Daronco
c34f3adff0 Add option to configure a different redis server for rap workers
Makes it possible to have a separate redis for workers, which helps when
scaling the processing.
2017-07-13 17:48:02 -03:00
Leonardo Crauss Daronco
076da25a7b Let exceptions be raised on rap resque workers
Let the exceptions be raised so that the job will be put in a "failed"
queue with all information needed.
2017-07-13 14:36:46 -03:00
Leonardo Crauss Daronco
71515d3a0c Make sure bbb-rap-resque-worker starts after boot 2017-07-13 14:36:45 -03:00
Leonardo Crauss Daronco
0252604b16 Add systemd scripts for rap workers on redis 2017-07-13 14:36:45 -03:00
Leonardo Crauss Daronco
13623e96fa Use a hash to pass options to rap workers, makes it easier to extend
With a hash it's easier for customizations to pass new parameters and
even create new types of workers.
2017-07-13 14:36:45 -03:00
Leonardo Crauss Daronco
5488cba3dd Run a different worker for each rap processing format available
Was running all formats inside a single worker, now there is a worker for
each processing script. Each will trigger its own publish worker when
done.
2017-07-13 14:36:44 -03:00
Leonardo Crauss Daronco
27cadca49a Use recording status files for info only, not as triggers
With resque workers we don't need to use status files to trigger each
step of the recording process. So instead of using status files as triggers
they are not used only for information, so that the workers can know if
a e.g. processing script failed or succeeded.
2017-07-13 14:36:44 -03:00
Leonardo Crauss Daronco
1320fcb900 Move common rap functionality to a base class
Moved all workers to a common directory, inheriting from a common base
class that has all common methods used by all workers.
2017-07-13 14:36:44 -03:00
Leonardo Crauss Daronco
72813a908c Move sanity, process and publish rap workers to run in resque
Same as 1bcdc45 but for the other scripts.
2017-07-13 14:36:43 -03:00
Leonardo Crauss Daronco
5aeeee25fd Move rap archive to resque, first step on moving all steps to resque
For now the differences are that the archive worker will run inside
resque (it is a resque worker now) and there is a "rap-trigger" file that
is executed by systemd to detect recordings that need to be archived and
add a job on resque to archive them.

The process os scheduling jobs still needs to be reviewed, but for now will
be done by rap-trigger.
2017-07-13 14:36:43 -03:00
Calvin Walton
da79d1e3b3 rap-process-worker: Write processing_time file only if processing succeeded
If processing didn't succeed, the directory where it is trying to write
the file might not exist, causing an exception in the worker which leads
to it getting stuck in a retry loop.
2017-05-08 16:10:59 -04:00
Fred Dixon
cd491c67b6 Moved the recording systemd scripts into the core directory 2017-03-23 10:36:10 -05:00
Calvin Walton
dd23536c84 record-and-playback: Split rap-worker into separate workers per step.
The purpose of this is to ensure that the recording processing with make
visible forwards progress, by ensuring that different steps in the
recording pipeline don't block each-other.

This fixes:
- recordings being processed prevent archiving new recordings from running
- recordings can't be published until all pending recording processing
  completes

It does allow recording archive, sanity, process, publish steps to run in
parallel with each other, but at most one recording will be in each step at
a time. (I.e. while one recording is being processed, a different recording
can be published). This may potentially increase CPU usage for some users.
If you expect this to be a problem, you can set resource controls (see
`man systemd.resource-control`) on the bbb_record_core.slice systemd unit.
2017-03-22 17:20:29 -04:00
Richard Alam
72c42f9d70 - build get recordings api response 2017-03-13 21:43:39 +00:00
Richard Alam
4e59521268 - store breakout metadata and breakout rooms in events.xml only if they are present. 2017-03-10 19:39:20 +00:00
Richard Alam
71bcd779b7 - process breakout rooms in events.xml and metadata.xml 2017-03-09 23:01:15 +00:00
alexandre
f39874ac9b Returning number of users that participated in each meeting in getRecordings call
Resolves #3623
2017-02-10 16:19:32 -02:00
Calvin Walton
c885fc3ec4 recording: also archive .flv.ser files from deskshare directory 2017-01-20 13:34:16 -05:00
Calvin Walton
f7f3b79ae4 recording: Run the 'ser' fixup tool on deskshare videos, too
Since the deskshare videos are saved in flv files by red5, they can
have the same partially written file issue as seen on webcams.
2017-01-20 13:23:36 -05:00
Jesus Federico
62bd4a587c Merge pull request #3489 from kepstin/gen-webvtt-iterparse
gen_webvtt: Parse entire events file, instead of iterparse
2016-12-07 11:38:14 -05:00
Calvin Walton
b2c1f00356 Merge pull request #3472 from jfederico/fix-recording-preview-alt-error
record-and-playback: Fixed issue with text gathered from slids used o…
2016-12-07 11:30:59 -05:00
Calvin Walton
fb3a913b5b gen_webvtt: Parse entire events file, instead of iterparse
The iterparse mode doesn't correctly handle long multi-byte UTF-8
characters with some versions of lxml library or libxml version.
2016-12-02 14:33:06 -05:00
Calvin Walton
bd8baed660 Fix typo in recording sanity script.
This caused it to think all video files were corrupt.
2016-11-21 15:01:54 -05:00
jfederico
161cab8234 record-and-playback: Fixed issue with text gathered from slids used on preview 2016-11-17 15:59:07 -05:00
Fred Dixon
0b9fc8b143 Merge pull request #3434 from kepstin/recording-cleanup
RnP: Improve error handling in video_info, audio_info functions
2016-11-11 12:14:55 -05:00
Fred Dixon
8b0a3dff50 Update Gemfile.lock for 1.1-dev 2016-11-10 10:03:17 -06:00
jfederico
737697968c record-and-playback: Removed preview extension (thumbnails) from recordings when only has default presentation 2016-11-07 18:50:28 +00:00
jfederico
64a4f4dc08 record-and-playback: Encoded text coming from the slide for being used in thumbnails 2016-10-27 17:35:58 -04:00
Richard Alam
ece6edcb12 - fix check for breakout room
- insert breakout metadata into events.xml
 - delete breakout metadata from redis
2016-10-21 19:05:30 +00:00
jfederico
1a4438d459 record-and-playback: Fixed issue with presentations containing less than 3 slides 2016-09-27 15:19:53 -04:00
jfederico
78cb599a5b record-and-playback: removed text element from extension preview 2016-09-22 13:11:01 -04:00
jfederico
e0b6a9ebf8 recording-and-playback: Removed log used for debugging 2016-09-21 17:31:40 -04:00
jfederico
59c67809d0 recording-and-playback: Added alt text to the preview per slide 2016-09-21 16:39:32 -04:00
jfederico
8adf729e0f recording-and-playback: Added general text to the preview 2016-09-20 16:46:45 -04:00
jfederico
45ba7d85c2 record-and-playback: Filtered presentation files to provide only the first thumbnail for default.pdf 2016-09-20 14:10:28 -04:00
Calvin Walton
ef607f0e44 RnP: Improve error handling in video_info, audio_info functions 2016-09-20 13:22:50 -04:00
Calvin Walton
c730681183 Improve detection of corrupt videos via ffprobe 2016-09-06 18:15:31 -04:00
Calvin Walton
f112be7638 Fix errors in recording scripts.
Logger was apparently imported by the matterhorn stuff ?! so move that
to the main library.

Fix a variable reference in sanity.
2016-09-03 16:08:32 -04:00
Calvin Walton
8ede6d06ee Remove a few matterhorn references that were missed. 2016-09-02 17:45:10 -04:00
Calvin Walton
13b9022282 Update the Gemfile.lock file to match gem changes 2016-09-02 16:40:27 -04:00
Calvin Walton
2cd9565d52 Improve wave length guessing
Instead of assuming that wav files have 44 byte headers (they usually
don't - freeswitch adds some metadata tags), add a super-simple wave
file parser that just finds the offset where the audio data actually
starts.
2016-09-02 16:40:27 -04:00
Calvin Walton
f815fd0d70 Remove a bunch of unused/old audio processing code.
The sanity check script was using one of the functions; it's been
changed to use a function from the EDL audio code instead.
2016-09-02 16:40:27 -04:00
Calvin Walton
0f29a6e5f9 Drop streamio-ffmpeg from required gems
It's no longer used anywhere in the recording processing, and the one
remaining use in the sanity script can be replaced with a (less buggy!)
function from the video EDL code.
2016-09-02 16:40:27 -04:00
Calvin Walton
b4d902cd9a Drop old/broken matterhorn script, and old video processing code. 2016-09-02 16:40:27 -04:00
Calvin Walton
ab0dac1192 Don't appy recording marks twice to audio 2016-08-12 15:53:56 -04:00
Calvin Walton
e4735a12b3 Various updates to the rnp edl handling
Main change is to rewrite the start/stop event handling to do fewer
passes and use simpler logic (it no longer has to track durations).

The temporary work audio codec is switched to flac; slightly lower hard
drive use and it avoids the 4-hour length limit of wav.

Some improvements to the video seeking when processing deskshare and
webcam videos, to avoid video dropouts.

Tweaks to the audio encoding quality settings.

Remove an unused function.
2016-08-12 14:17:51 -04:00
Calvin Walton
bfb987dc01 Make shape rendering in presentation recordings closer to live BBB 2016-08-12 11:53:01 -04:00
Calvin Walton
b589c25831 Fix reference to deleted 'webrtc_deskshare_archiver' 2016-08-02 14:31:20 -04:00
Calvin Walton
d4e7605f42 RnP: Fix use of deprecated IO#lines in logging functions 2016-08-02 10:58:42 -04:00
Calvin Walton
2b8d7a9858 Merge remote-tracking branch 'origin/master' into recording-captions 2016-08-02 10:49:05 -04:00