Commit Graph

259 Commits

Author SHA1 Message Date
Pedro Beschorner Marin
0c4cf0135d Handle pad's events and Meteor's instances
Since Meteor was split in multiple process and events started to be
filtered by instances, all Etherpad's Redis events were being discarded.

Etherpad has a Redis' publisher plugin that is unaware of BigBlueButton's
existence. All the communication between them is kept simple with minimal
of internal data exchange. The concept of distincts subscribers at Meteor's
side broke part of this simplicity and, now, Etherpad has to know which
instance must receive it's messages. To provide such information I decided
to include Meteor's instance as part of the pad's id. Should look like:

 - [instanceId]padId for the shared notes
 - [instanceId]padId_cc_(locale) for the closed captions

With those changes the pad id generation made at the recording scripts had to
be re-done because there is no instance id available. Pad id is now recorded at
akka-apps and queried while archiving the shared notes.
2021-03-23 18:03:50 -03:00
Mario Jr
af4f9dbf26 Merge branch 'v2.2.x-release' into merge-2.2-develop 2021-03-23 17:10:28 -03:00
Pedro Beschorner Marin
e48f64b843 Avoid localhost at notes endpoint
Use 127.0.0.1 instead of localhost since Etherpad only listens for ipv4.
2021-03-22 16:41:26 -03:00
Anton Georgiev
228bad03d5 Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into march4-merge 2021-03-04 21:25:47 +00:00
Calvin Walton
48956ce63f Add some extra docs/setup instructions to the analytics callback script 2021-03-02 16:19:26 -05:00
Pedro Beschorner Marin
c0a7f9cd92 Replace FNV32a pad's id generator with salted SHA1
When managing Etherpad's pads, Meteor makes API calls to initiate the closed captions
and shared notes modules. The pad id was being mapped to a shorter id than the meeting
id because of a Etherpad lenght limitation.

Changed to something less guessable.
2021-02-09 12:59:59 -03:00
Jesus Federico
163d40652a
Update post_events_analytics_callback.rb 2020-12-10 17:23:01 -05:00
Anton Georgiev
0c7ead1916 Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into oct16-merge 2020-10-21 14:48:02 +00:00
Calvin Walton
1866eb7194 Perform captions generation in UTF-16 encoding
The indexes returned in recording events from BBB refer to positions
within a UTF-16 encoded string. Rather than attempt to untangle this in
the server (which might have a performance cost), it's easier to switch
the caption processing code to operate in UTF-16 encoding as well to
make it work consistently.

The PyICU library provides a UnicodeString type which is a UTF-16 string
similar to Java and JavaScript, but which supports all the python
indexing methods. It's fairly straightforwards to swap it in in place of
the types used previously, and works natively as an input to the ICU
line break iterator too.

Fixes #10531
2020-09-25 12:39:18 -04:00
hiroshisuga
5caf825d42
Update rap-starter.rb
probably a simple typo
2020-09-01 22:05:37 +09:00
Jesus Federico
0bbe69b74d
added post_events scripts (example and analytics callback) (#10080)
* added post_events example and analytics callback

* updated dependency
2020-07-17 14:34:57 -04:00
mw781
fea0ff3204 Ensure that files are handled safely
This commit fixes an issue with reading and writing files.
File.open is used which means that a file will remain open
unless explicilty closed or the program exit. This doesn't work
for an NFS mount as the scripts try to "rm -rf" when the file
is still open. This commit fixes that by replacing all .opens
with .reads
2020-06-17 11:09:09 +01:00
Joao Siebel
3e95ed0e4b Merge remote-tracking branch 'upstream/v2.2.x-release' into merge-2.2 2020-06-16 16:40:56 -03:00
Felipe Cecagno
fc95e052a6 Sort sanity files so it processes older recordings first 2020-05-23 01:28:43 -03:00
Anton Georgiev
cc79c4b6ae merge 2.2.10 into 2.3 2020-05-05 19:52:44 +00:00
Pedro Beschorner Marin
d97f413ef6 Fix note id 2020-04-29 13:08:10 -03:00
Pedro Beschorner Marin
beda54fb09 Included Etherpad's data as events to be stored 2020-04-27 10:48:52 -03:00
Calvin Walton
a5a46b2c0a Have the recording archive delete originals of archived media
After the last segment of a recording has been archived, it is safe to
delete the original files that were recorded by the various media
handling components. This patch deletes the freeswitch audio files and
kurento webcam/screenshare files after they have been archived.

It needs to be used in combination with some changes to users/groups and
directory permissions to allow the `bigbluebutton` user to delete files
that it could previously only read.
2020-03-27 15:22:53 -04:00
Leonardo Crauss Daronco
5bc3062d58 Fix the paths to the ruby scripts on the rap workers
Was pointing to the wrong directory when loading the ruby scripts. Also
made the directory easier to configure, it's all in a single place now.
2020-02-06 10:43:32 -03:00
Calvin Walton
c52a67186e RaP: Clean up resque worker code, add worker for keep_events feature
I've moved the workers code into the `lib` subdirectory with other library-ish
code; this puts it into the ruby load path used by most scripts so referencing
files is easier.

I've applied various style cleanups based on the rubocop config present.

The `events` processing step has been integrated as a new worker `EventsWorker`,
there is no longer a separate `events/events.rb` script. I've reworked the
`rap-starter.rb` script to check for the done files in both the events and
recorded status directories.
2020-01-21 17:14:39 -05:00
Calvin Walton
aee7baa76b Merge branch 'v2.0-rap-on-resque' of github.com:daronco/bigbluebutton 2020-01-17 17:40:08 -05:00
Calvin Walton
9907588b5e presentation recording: generate poll svg images without gnuplot
Write a tool that generates the poll svg images directly from the
BBB poll description. This avoids the issues with special characters
in the gnuplot labels, and gives us a lot more flexibility in how
the polls are formatted and styled.
2020-01-13 14:34:30 -05:00
Leonardo Crauss Daronco
95573b2467 Improve the yml for rec steps and run captions always before process 2019-12-10 16:28:13 -03:00
Leonardo Crauss Daronco
ac801cfce7 Use journald as the default logger for rap and rap services
Changed only in the main class so journald is the default and in the
scripts related to the processes in resque. Internal scripts might still
be logging to files.
2019-12-06 17:06:32 -03:00
Leonardo Crauss Daronco
f1d37e8a08 Gracefully stop bbb-rap-starter when the service is stopped 2019-12-06 16:36:25 -03:00
Leonardo Crauss Daronco
835449551d Raise an exception if a rap step fails so resque will list it
Resque captures the exceptions and marks the jobs as failed.
2019-12-06 15:51:47 -03:00
Leonardo Crauss Daronco
29cae33636 Add the format parameter to the examples of post scripts
They are optional but it's better to have them than not. If they are passed
and the script doesn't expect it they break.
2019-12-06 15:23:19 -03:00
Leonardo Crauss Daronco
5e05b7a8e8 Fixes for the rap scripts after processing a real recording 2019-12-06 12:26:24 -03:00
Leonardo Crauss Daronco
d74be7962a Process all .done files when bbb-rap-starter is started
Instead of just running when a .done file is added, it now also goes
through the files already there when the process starts.
2019-12-06 11:50:08 -03:00
Leonardo Crauss Daronco
53c3ed0d7d Change rap-starter to run persistently and watch for file changes
Instead of being executed every 30s by systemd, it's now a service that's
running all the time and will wait for .done files to start the processing
of recordings.
2019-12-05 11:41:13 -03:00
Leonardo Crauss Daronco
d769849ad6 Add resque worker to process the captions of a recording
Moved the code that was after the sanity step to its own worker.
2019-12-04 18:32:14 -03:00
Leonardo Crauss Daronco
62d62a2a1e Merge remote-tracking branch 'upstream/master' into v2.0-rap-on-resque
# Conflicts:
#	bigbluebutton-config/bin/bbb-conf
#	record-and-playback/core/Gemfile.lock
#	record-and-playback/core/lib/recordandplayback/events_archiver.rb
#	record-and-playback/core/scripts/archive/archive.rb
#	record-and-playback/core/scripts/bigbluebutton.yml
#	record-and-playback/core/scripts/rap-process-worker.rb
#	record-and-playback/core/scripts/rap-publish-worker.rb
#	record-and-playback/core/scripts/rap-sanity-worker.rb
#	record-and-playback/core/scripts/rap-starter.rb
#	record-and-playback/core/scripts/sanity/sanity.rb
#	record-and-playback/core/systemd/bbb-rap-archive-worker.service
#	record-and-playback/core/systemd/bbb-rap-process-worker.service
#	record-and-playback/core/systemd/bbb-rap-publish-worker.service
#	record-and-playback/core/systemd/bbb-rap-sanity-worker.service
#	record-and-playback/core/systemd/bbb-record-core.target
2019-12-04 17:33:44 -03:00
Leonardo Crauss Daronco
9e5c386e6f Add option to rap-process-worker to accept a filtering pattern
It will only process recordings which ID matches the pattern. By default
the pattern is nil and it will process all recordings.
2019-12-04 12:05:18 -03:00
Fred Dixon
dd2ae19ca0
Merge pull request #8277 from SilentFlameCR/master
edited rap-caption-inbox.rb to get appropriate files in presentation
2019-11-22 23:11:18 +01:00
SilentFlameCR
b88b70d945 edited rap-caption-inbox.rb to get appropriate files in presentation 2019-10-30 13:10:24 -04:00
Calvin Walton
8283bc16f0 recording archive: Archive freeswitch audio in formats other than wav 2019-10-28 13:47:33 -04:00
Calvin Walton
8bdcea424d Fix typos in archive script 2019-10-23 10:16:29 -04:00
Calvin Walton
cad8995858 recording archive: Compress audio files
The wav files from the freeswitch recordings are huge, uncompressed.
There's no reason to keep them as wav, we can compress them in the
archive step.
2019-10-22 17:27:23 -04:00
Richard Alam
e409190114 - convert tabs to spaces 2019-07-10 13:09:38 -07:00
Richard Alam
ec1343931c - add comments to explain how rap-archive-worker and rap-events-worker work together 2019-07-05 08:57:51 -07:00
Richard Alam
40c6dd8827 - run post events scripts 2019-07-05 08:57:22 -07:00
Richard Alam
61d9223a70 - format spaces to tabs 2019-07-04 11:07:57 -07:00
Richard Alam
82bd5e4a6f - run keep-events worker separate from archive-worker 2019-07-04 11:00:59 -07:00
Calvin Walton
4c1e2ed477 RaP: Reduce ffmpeg output spam in logs of caption inbox 2019-06-06 10:49:27 -04:00
Calvin Walton
7f92b58140 RaP: Dir.glob accepts a block directly, no need for a temp variable 2019-06-06 10:44:47 -04:00
Calvin Walton
620ed367be RaP: glob is a method on Dir, not File 2019-06-06 10:43:40 -04:00
Calvin Walton
94ae7c330b RaP: Use correct variable to get info for new caption track 2019-06-06 10:41:14 -04:00
Calvin Walton
aecb66e28d RaP: Create the recording captions dir before moving the converted vtt file 2019-06-06 10:34:08 -04:00
Calvin Walton
07688593ac caption-inbox: clean up track file if language tag is invalid 2019-06-04 14:28:58 -04:00
Calvin Walton
f693915e99 RaP captions: Fix usage of JSON.parse, use a proc to capture local vars 2019-06-04 14:01:51 -04:00