bigbluebutton-Github/record-and-playback/core/scripts
prlanzarin 40b642969c fix(recording): handle archive remux failures
Recording archive may fail when remuxing invalid files from KMS or the
new recorder - eg when the raw files are 0-byte sized.

This commit handles the exception raised by EDL::encode so archive keeps
going, logs the issue as a warning and archives the problematic file anyways.
EDL::encode now removes the temporary file when the ffmpeg command execution
fails - this should avoid leaving any stale files around in case of failure.

No specific check for the nature of the error is done - the idea is that
subsequent phases will discard or fix the files if necessary according
to the processing scripts' necessities, making the behavior (in this
specific scenario) similar to what it was before the archive remuxing was
introduced.
2023-09-07 11:57:55 -03:00
..
archive fix(recording): handle archive remux failures 2023-09-07 11:57:55 -03:00
post_archive refactor(record-and-playback): replace trollop by optimist 2021-11-23 23:59:59 -03:00
post_events [issue-14819] - Changes in review 2022-05-02 16:41:02 -03:00
post_process refactor(record-and-playback): replace trollop by optimist 2021-11-23 23:59:59 -03:00
post_publish refactor(record-and-playback): replace trollop by optimist 2021-11-23 23:59:59 -03:00
process Move captions.rb to utils directory. 2019-05-10 16:56:29 +01:00
publish
sanity [issue-15051] - Refactored all traces from Red5 2022-06-13 08:19:08 -03:00
utils refactor(record-and-playback): replace trollop by optimist 2021-11-23 23:59:59 -03:00
bbb-0.9-beta-recording-update refactor(record-and-playback): replace trollop by optimist 2021-11-23 23:59:59 -03:00
bbb-0.9-recording-size refactor(record-and-playback): replace trollop by optimist 2021-11-23 23:59:59 -03:00
bbb-1.1-meeting-tag refactor(record-and-playback): replace trollop by optimist 2021-11-23 23:59:59 -03:00
bigbluebutton.yml build: add bbb-webrtc-recorder 2023-04-20 16:24:34 -03:00
cleanup.rb cleanup 2018-07-12 00:06:14 -03:00
rap-caption-inbox.rb edited rap-caption-inbox.rb to get appropriate files in presentation 2019-10-30 13:10:24 -04:00
rap-enqueue.rb Fix rap-enqueue.rb to load gems with bundler 2022-05-31 18:13:33 -04:00
rap-process-worker.rb Sort sanity files so it processes older recordings first 2020-05-23 01:28:43 -03:00
rap-starter.rb bbb-record-core: Update systemd units to use bundler 2022-03-29 12:41:27 -04:00
README Update README 2023-05-21 21:21:38 +09:00

README

These instructions below are for testing by running scripts manually:

1. Create some temp scratch dirs:

mkdir -p ~/temp/log/presentation ~/temp/recording/{process,publish,raw} ~/temp/recording/status/{recorded,archived,processed,sanity} ~/temp/published

2. Edit ~/dev/bigbluebutton/record-and-playback/core/scripts/bigbluebutton.yml and comment the PRODUCTION directories while uncommenting the DEVELOPMENT folders.
The path should match the ones created above.

# For PRODUCTION
# log_dir: /var/log/bigbluebutton
events_dir: /var/bigbluebutton/events
# recording_dir: /var/bigbluebutton/recording
# published_dir: /var/bigbluebutton/published
captions_dir: /var/bigbluebutton/captions
# playback_host: develop.distancelearning.cloud
playback_protocol: https

# For DEVELOPMENT
# This allows us to run the scripts manually
scripts_dir: /home/ubuntu/dev/bigbluebutton/record-and-playback/core/scripts
log_dir: /home/ubuntu/temp/log
recording_dir: /home/ubuntu/temp/recording
published_dir: /home/ubuntu/temp/published
playback_host: 127.0.0.1

3. Stop the recording service:

   systemctl stop bbb-rap-starter
   systemctl stop bbb-rap-resque-worker

4. Create a recording using BigBlueButton. After logging out, it should have created a <meeting-id>.done file in the
   /var/bigbluebutton/recording/status/recorded directory. Make note of this meeting-id as we use that to tell the script
   which recording to process.

5. Before running the scripts, we have to make sure our scripts have the paths set up correctly.
   In your development environment (~/dev/bigbluebutton/record-and-playback), edit

   - /core/scripts/archive/archive.rb
   - /core/scripts/sanity/sanity.rb
   - presentation/scripts/process/presentation.rb
   - presentation/scripts/publish/presentation.rb
   - presentation/scripts/presentation.yml

   and ensure the scripts are using the DEVELOPMENT PATH. We need to do this so the script will be able to find the core library.

   # For DEVELOPMENT
   # Allows us to run the script manually
   require File.expand_path('../../../../core/lib/recordandplayback', __FILE__)

   # For PRODUCTION
   # require File.expand_path('../../../lib/recordandplayback', __FILE__)

   # For PRODUCTION
   # publish_dir: /var/bigbluebutton/published/presentation
   video_formats:
   - webm
   # - mp4

   # For DEVELOPMENT
   publish_dir: /home/ubuntu/temp/published/presentation

6. Now we run the archive step. Go to record-and-playback/core/scripts and type
   
   ruby archive/archive.rb -m <meeting-id>

   If everything went well, you should now have the raw files in ~/temp/recording/raw/<meeting-id>.
   You can also check the logs at ~/temp/log/archive-<meeting-id>.log, and have a .done entry in ~/temp/recording/status/archived.

7. Now we need to do a sanity check to ensure the raw recordings are complete. Again in the scripts folder, type

   ruby sanity/sanity.rb -m <meeting-id>

   Confirm everything went as intended by checking the logs at ~/temp/log/sanity.log and
   that a .done entry in ~/temp/recording/status/sanity exists.

8. Assuming the recording passed the sanity check, it's time to process the recording.

   Set the path of the captions generation script by changing the line

   ret = BigBlueButton.exec_ret('utils/gen_webvtt', '-i', raw_archive_dir, '-o', target_dir)

   to

   ret = BigBlueButton.exec_ret('/home/ubuntu/dev/bigbluebutton/record-and-playback/core/scripts/utils/gen_webvtt', '-i', raw_archive_dir, '-o', target_dir)

   Now, you can run the presentation processing script:

   cd ~/dev/bigbluebutton/record-and-playback/presentation/scripts
   ruby process/presentation.rb -m <meeting-id>

   You can monitor the progress by tailing the log at ~/temp/log/presentation/process-<meeting-id>.log.

   Note that each time one of the scripts fails, the corresponding directories for the recording at

   ~/temp/recording/process/presentation and ~/temp/recording/publish/presentation

   need to be deleted before executing them again.

9. If everything went well, we can now run the publish script. However, we need to cheat a little bit.

   The publish script will be looking for a 'processing_time' file which contains information on how long the
   processing took. Unfortunately, that file is created by the 'rap-worker.rb', a script which we don't run.

   Manually create the file at

      vi ~/temp/recording/process/presentation/<meeting-d>/processing_time

   Enter any number (e.g. 46843) and save the file.

   Once again, the path to the poll generation utility needs to be updated:
   
   Change

   ret = BigBlueButton.exec_ret('utils/gen_poll_svg', ...)

   to

   ret = BigBlueButton.exec_ret('/home/ubuntu/dev/bigbluebutton/record-and-playback/core/scripts/utils/gen_poll_svg', ...)

   Finally, run the publish script:

   ruby publish/presentation.rb -m <meeting-id>-presentation

   Notice we appended "presentation" to the meetingId, this will tell the script to publish using the "presentation" format.

9. Deploy your changes with deploy.sh and restarting the recording-related services:
   
   systemctl restart bbb-rap-starter
   systemctl restart bbb-rap-resque-worker