I've moved the workers code into the `lib` subdirectory with other library-ish
code; this puts it into the ruby load path used by most scripts so referencing
files is easier.
I've applied various style cleanups based on the rubocop config present.
The `events` processing step has been integrated as a new worker `EventsWorker`,
there is no longer a separate `events/events.rb` script. I've reworked the
`rap-starter.rb` script to check for the done files in both the events and
recorded status directories.
Found another case where the html5 client was passing through control
characters, in the original presentation name field.
Rather than play whack-a-mole with different fields which may eventually
get poorly sanitized user data, apply the control character filtering
to all properties.
Adjust the character range to do the following:
* Allow horizontal tab (0x09), it's not problematic.
* Disallow control characters in the range 0x1A-0x1F. Probably missed by accident.
FFmpeg has pretty good format autodetection even if the filename has the
'.txt' extension, so just rely on that. It'll even pull subtitles out of
video files - although I expect we'll have size limits so that doesn't
happen.
Sometimes when text is pasted into the whiteboard text tool from
external apps, it'll include control characters that mess up later
recording processing scripts.
Run the same cleanup as already used for chat messages on the whiteboard
text as well.
The cleanup has been adjusted to allow newline and tab characters. They
won't really cause issues in chat, and newlines (at a minimum) are
required for the whiteboard.
This is a workaround for #7356
BigBlueButton can sometimes write events out of order - this particularly
seems to affect the final RecordStatusEvent in a meeting which was ended
while recording was still running. This breaks the recording processing
scripts.
As a workaround, sort the events as they're being written into the events.xml
file. We have the following properties:
* The input data is already mostly sorted
* Items in the wrong position will be no more than a couple spots off from where
they should be
* We should not change the relative order of events with the same timestamp.
The best algorithm to use here is a simple insertion sort. When adding each new
event to the XML structure, it scans backwards through existing events until it
finds the correct position.
For #6035
At some point, BigBlueButton switched from using `"` to `'` in the link tags in
chat messages, which caused the regular expression that was supposed to remove
the `event:` prefix to not match.
I've replaced the error-prone regular expression with an actual HTML parser,
using the "Loofah" HTML transformation/sanitization library based on Nokogiri.
I've removed the code that detected unlinked URLs, since it was broken - and
not needed: current BigBlueButton versions do the link detection in the client.
If someone reprocesses a really old BBB recording with these scripts, URLs in
chat might not be linked in the result. But they wouldn't have been linked in the
client in the original meeting either, so I figure that's ok.
Fixes#6475
In some cases, ffmpeg will be able to read the file, but the video itself
can't be decoded (missing/corrupt stream headers, for example). In this case,
some of the properties on the stream object will be nil.
Make sure that pix_fmt is present in the probed info, since that's a required
property.
Issue #6338
It looks like there was a logic error in the code that was causing it
to break out of the event deletion loop early when deleting events for
the last (or only) segment in a recording. (In this case, last_index
is -1, so i >= last_index is always true).
The trim_events_for call was always succeeding, so the events were
being removed from the event list (meeting:{ID}:recordings key) even
though the events themselves hadn't been deleted in the loop.
I've moved the trim_events_for call to below the event deletion loop
to ensure that if the archive script is interrupted, the events list
will contain all not-yet-deleted events.
The archive_dir can by the raw recording directory in some recording
formats - including, hopefully, presentation at some point (to avoid the
extra copy)
The previous calculation used the video size in pixels, which might lead
to stretched/squashed videos in rare cases where the video has non-square
pixels. The new calculation is correct for all video sizes.
The ffprobe command in ffmpeg 4.0 now omits the aspect ratio fields in the
json output when indeterminate, instead of returning an invalid value with
0 in the numerator or denominator. Handle this correctly.
- some conflicts have been fixed.
The following needs working on:
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/services/ChatCopy.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/services/ChatSaver.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/views/ChatTab.mxml
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/views/ChatWindowEventHandler.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/services/MessageReceiver.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/services/MessageSender.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/views/MediaItemRenderer.mxml
Parking as need to work on something else.
Since the update to the newer red5, seeking in flv files (webcams in
particular are noticable) has been broken, resulting in cameras
appearing to "hang" any time there is a cut in the generated video -
which happens when start/stop button is pushed, or when cameras are
added or removed.
We can detect the problematic video files because the timestamp of the
first frame is large (old red5 versions always set first frame
timestamp to 0.001 seconds). If we see a file like this, having ffmpeg
remux the file - rewriting the timestamps and index - works around the
problem.
This improves the quality of portrait documents, before they were
1200px when landscape documents got 1600px.
Switching to scaling to a square means that we can use the "-scale-to"
option on pdftocaio, which means that it generates images directly
at the desired size. This can save quite a bit of time (and memory)
if a document was uploaded with extremely large page size.
In some cases (due to network issues), the webcam video can be shorter
than the time between the start/stop events. Pad the input video with a
blank video to make sure that there's input to the video tiling filters,
to fix a problem where the video won't render correctly with ffmpeg v3.4
and later.
