Commit Graph

477 Commits

Author SHA1 Message Date
Richard Alam
9ac272d1a0 - use thumbnails to find the num pages of presentation instead of swfs as we now turn off generation of
swfs for html5
2019-06-07 13:22:59 -07:00
Richard Alam
a46d0b9680 Merge branch 'master' of github.com:bigbluebutton/bigbluebutton into captions-text-tracks-intermediate-branch 2019-06-06 15:04:53 -07:00
Calvin Walton
19a011cc29 RaP: Fix error message when shared notes could not be archived.
It used to print:

Failed to download file: undefined local variable or method `respose' for BigBlueButton:Module
Did you mean?  response

because the incorrect variable name was used in the error message.

There was no effect other than the message in the log, since the shared notes
couldn't be archived anyways, and the only thing the exception did was ...
prevent the shared notes from being archived.
2019-06-06 10:55:53 -04:00
Calvin Walton
4c1e2ed477 RaP: Reduce ffmpeg output spam in logs of caption inbox 2019-06-06 10:49:27 -04:00
Calvin Walton
7f92b58140 RaP: Dir.glob accepts a block directly, no need for a temp variable 2019-06-06 10:44:47 -04:00
Calvin Walton
620ed367be RaP: glob is a method on Dir, not File 2019-06-06 10:43:40 -04:00
Calvin Walton
94ae7c330b RaP: Use correct variable to get info for new caption track 2019-06-06 10:41:14 -04:00
Calvin Walton
aecb66e28d RaP: Create the recording captions dir before moving the converted vtt file 2019-06-06 10:34:08 -04:00
Calvin Walton
07688593ac caption-inbox: clean up track file if language tag is invalid 2019-06-04 14:28:58 -04:00
Calvin Walton
f693915e99 RaP captions: Fix usage of JSON.parse, use a proc to capture local vars 2019-06-04 14:01:51 -04:00
Calvin Walton
f04bf290af Fix requires for journald-logger 2019-06-04 12:18:46 -04:00
Calvin Walton
983751c9c0 RaP: Have the caption inbox run the caption integration scripts. 2019-06-04 11:03:22 -04:00
Calvin Walton
676fa41f06 Add caption integration script for presentation format. 2019-06-04 11:03:22 -04:00
Calvin Walton
b22c905194 rap-caption-inbox: improve formatting, initial pass at conversion
It now attempts to convert the uploaded caption file to WebVTT and
updates the index file.
2019-06-04 11:03:22 -04:00
Calvin Walton
775c348ddc RaP: Add an optional test group which installs e.g. rubocop 2019-06-04 11:03:22 -04:00
Calvin Walton
c6e09f52a1 RaP Caption inbox: read uploaded caption json file
Some additional validation/normalization is added for the language tag,
and some cleanup has been done for logging.
2019-06-04 11:03:22 -04:00
Calvin Walton
812052a8c7 Change events archiver to read config from relative path
Makes testing easier
2019-06-04 11:03:22 -04:00
Calvin Walton
d74f0044cc Add function that uses ffmpeg to convert subtitles to webvtt
FFmpeg has pretty good format autodetection even if the filename has the
'.txt' extension, so just rely on that. It'll even pull subtitles out of
video files - although I expect we'll have size limits so that doesn't
happen.
2019-06-04 11:03:22 -04:00
Calvin Walton
23da507fa8 Initial outline of the rap worker tool for handling the caption inbox
Rather than running the tool in a loop, I'm using inotify to watch for
new files being created (ideally, the other rap workers will be migrated
to this style in the future). The trigger for processing is the creation
of the .json index file for the uploaded track.
2019-06-04 11:03:22 -04:00
Ghazi Triki
c9194d087e Correctly process captions. 2019-05-13 21:38:51 +01:00
Ghazi Triki
7c5727750f Merge remote-tracking branch 'bigbluebutton/master' into text-tracks 2019-05-13 14:46:20 +01:00
Ghazi Triki
b578aeb243 Move captions.rb to utils directory. 2019-05-10 16:56:29 +01:00
Ghazi Triki
dbd86eb096 Code improve. 2019-05-10 15:51:05 +01:00
Fred Dixon
056ef6ee8c
Merge pull request #7327 from ritzalam/add-redis-password-on-master
Add redis password on master
2019-05-09 16:35:42 -04:00
Calvin Walton
5c09ea5605 Merge branch 'archive-note' of https://github.com/fcecagno/bigbluebutton 2019-05-09 13:43:48 -04:00
Fred Dixon
0d9220051f
Merge pull request #7357 from kepstin/archive-text-cleanup
Recording: Strip control characters in whiteboard text events
2019-05-08 10:10:25 -04:00
Ghazi Triki
17a3af0487 Write playback caption files from the captions file generated for API. 2019-05-07 18:13:09 +01:00
Calvin Walton
548ac7edef Recording: Strip control characters in whiteboard text events
Sometimes when text is pasted into the whiteboard text tool from
external apps, it'll include control characters that mess up later
recording processing scripts.

Run the same cleanup as already used for chat messages on the whiteboard
text as well.

The cleanup has been adjusted to allow newline and tab characters. They
won't really cause issues in chat, and newlines (at a minimum) are
required for the whiteboard.

This is a workaround for #7356
2019-05-07 11:48:20 -04:00
Ghazi Triki
35506cdc44 Generate captions.json for API. 2019-05-01 16:01:15 +01:00
Richard Alam
bf7898225b - minor cleanup 2019-04-30 12:30:59 -07:00
Ghazi Triki
f820446965 Process caption files before processing meeting files. 2019-04-30 19:54:02 +01:00
Ghazi Triki
925f5a2894 Put captions files in the new directory. 2019-04-22 20:28:54 +01:00
Felipe Cecagno
109bb7b658 cleanup 2019-04-15 12:36:05 -03:00
Ghazi Triki
757d92cc29 Raise an error when the note file header does not respond with a HTTPOK status. 2019-04-15 15:07:03 +01:00
Ghazi Triki
92a4bc7e52 Replaced wget and curl commands by net/http module. 2019-04-04 14:09:49 +01:00
Richard Alam
f2098bc376 Merge branch 'master' of github.com:bigbluebutton/bigbluebutton into ritzalam-add-redis-password-on-v2.2-dev 2019-04-02 10:43:46 -07:00
Felipe Cecagno
f70331cbba update notes formats archived 2019-03-18 14:33:29 -03:00
Pedro Beschorner Marin
7e5c5c29ed Avoid publishing empty notes 2019-03-15 17:19:54 -03:00
Felipe Cecagno
51b9b91dae rename note to notes 2019-03-11 23:56:29 -03:00
Felipe Cecagno
3467d912c7 fix typo 2019-03-05 13:22:10 -03:00
Felipe Cecagno
657a3ef013 use etherpad api to archive exported note 2019-03-05 13:17:48 -03:00
Calvin Walton
010153cbb9 Recording: Remove the red5 race workaround that delays processing
This was added to workaround for red5 taking a while to rewrite the
serialized (.ser) data that it streams to disk back to the .flv format.

The workaround is no longer needed, for two reasons:
* The sanity scripts run the red5 code to generate the .flv from the .ser
  if needed, and
* We're expecting more people to be using WebRTC media in the future anyways

This makes recordings available up to 2 minutes earlier than they would have
been otherwise.
2019-02-28 11:13:12 -05:00
Calvin Walton
49a86e4d4e Recording archive: log when events are are out of order and must be sorted
To help us track down exactly why this is happening - and know when we've
properly fixed it - add some logging.
2019-02-26 12:10:21 -05:00
Calvin Walton
abc8284052 Recording events archiver: fix syntax errors
I had renamed prev_event to previous_event, but missed a couple of spots.
2019-02-25 16:43:27 -05:00
Calvin Walton
5d189be062 Sort events when writing the events.xml file
BigBlueButton can sometimes write events out of order - this particularly
seems to affect the final RecordStatusEvent in a meeting which was ended
while recording was still running. This breaks the recording processing
scripts.

As a workaround, sort the events as they're being written into the events.xml
file. We have the following properties:

* The input data is already mostly sorted
* Items in the wrong position will be no more than a couple spots off from where
  they should be
* We should not change the relative order of events with the same timestamp.

The best algorithm to use here is a simple insertion sort. When adding each new
event to the XML structure, it scans backwards through existing events until it
finds the correct position.

For #6035
2019-02-25 12:34:29 -05:00
Calvin Walton
1e76256ea5 Recording: Fix handling of links in chat messages
At some point, BigBlueButton switched from using `"` to `'` in the link tags in
chat messages, which caused the regular expression that was supposed to remove
the `event:` prefix to not match.

I've replaced the error-prone regular expression with an actual HTML parser,
using the "Loofah" HTML transformation/sanitization library based on Nokogiri.

I've removed the code that detected unlinked URLs, since it was broken - and
not needed: current BigBlueButton versions do the link detection in the client.
If someone reprocesses a really old BBB recording with these scripts, URLs in
chat might not be linked in the result. But they wouldn't have been linked in the
client in the original meeting either, so I figure that's ok.

Fixes #6475
2019-02-20 12:03:24 -05:00
Ghazi Triki
99aa2b5fcc Use the right bigbluebutton user for recording directories. 2019-02-08 17:53:57 +01:00
Fred Dixon
566b245e3c
Merge pull request #6385 from pedrobmarin/keep-meeting-events
Created property keepEvents at bbb-web to make redis record the event…
2019-01-24 21:47:50 -05:00
Fred Dixon
b9f9f0b3af
Merge pull request #6635 from kepstin/rec-corrupt-video
Add a check to reject undecodable video streams.
2019-01-24 16:27:54 -05:00
Calvin Walton
52fadaf4a6 Add a check to reject undecodable video streams.
In some cases, ffmpeg will be able to read the file, but the video itself
can't be decoded (missing/corrupt stream headers, for example). In this case,
some of the properties on the stream object will be nil.
Make sure that pix_fmt is present in the probed info, since that's a required
property.
2019-01-24 14:56:48 -05:00