Since Meteor was split in multiple process and events started to be
filtered by instances, all Etherpad's Redis events were being discarded.
Etherpad has a Redis' publisher plugin that is unaware of BigBlueButton's
existence. All the communication between them is kept simple with minimal
of internal data exchange. The concept of distincts subscribers at Meteor's
side broke part of this simplicity and, now, Etherpad has to know which
instance must receive it's messages. To provide such information I decided
to include Meteor's instance as part of the pad's id. Should look like:
- [instanceId]padId for the shared notes
- [instanceId]padId_cc_(locale) for the closed captions
With those changes the pad id generation made at the recording scripts had to
be re-done because there is no instance id available. Pad id is now recorded at
akka-apps and queried while archiving the shared notes.
We still use the recording status files to externally monitor the
progress of the recordings. Let's keep these files for now until
we figure out a different way to track the status of the recording.
This incorporates only the audio desync related changes from #11626
* Add the aresample filter with async option to fill in timestamp gaps
* Use the libopus decoder for opus audio instead of ffmpeg's builtin
decoder
This gives the following advantages over the previous code:
* The ffmpeg input filters are loaded from a filter "script" file instead
of passed on the command line. This fixes some cases of recordings
failing to process because the ffmpeg command line generated for the
audio processing exceeded the max command line length limit. (Although
that only really happens due to BBB bugs...)
* Use absolute positions when trimming audio segments for cuts.
Previously segments were trimmed to the length of the segment, and the
results were concatenated. There's some possibility of accumulated
errors in the segment lengths causing audio desync over time. The new
code incrementally concatenates the segments, and cuts each segment
end based on the absolute time since the start of the meeting, to
avoid error accumulation.
use libopus decoder and encoder, its better than built-in ffmpeg/flac
don't mix screenshare audio with mics, was generating desync with bad audio segments, encode it together with video file (TODO: needs adjustments in playback)
When managing Etherpad's pads, Meteor makes API calls to initiate the closed captions
and shared notes modules. The pad id was being mapped to a shorter id than the meeting
id because of a Etherpad lenght limitation.
Changed to something less guessable.
Collects the shared notes' HTML raw data and publishes it along with the other
recording files. The playback will fetch for this file and include an option to
display it's content over the chat.
The indexes returned in recording events from BBB refer to positions
within a UTF-16 encoded string. Rather than attempt to untangle this in
the server (which might have a performance cost), it's easier to switch
the caption processing code to operate in UTF-16 encoding as well to
make it work consistently.
The PyICU library provides a UnicodeString type which is a UTF-16 string
similar to Java and JavaScript, but which supports all the python
indexing methods. It's fairly straightforwards to swap it in in place of
the types used previously, and works natively as an input to the ICU
line break iterator too.
Fixes#10531
The previous implementation of the BigBlueButton.execute method runs the
process with separate stdout and stderr streams. It first reads all of
the output from stdout, then reads all of the output from stderr.
This can cause a deadlock if the process writes a lot of data to stderr.
The IO buffer for stderr could fill, blocking progress. But since it
hasn't closed stdout, the ruby script is still waiting on a read to
stdout.
Switch to an execution method (using IO.popen) that allows combining
stdout and stderr into a single stream, eliminating the issue.
On my server 2.3 alpha, the method metadata_for(meeting_id) gives back {}
(empty Hash). Thus "return if meeting_metadata.nil?" does not occur.
Does @redis.hgetall give {} instead of nil, even though there is a comment in
node_modules/redis/lib/utils.js "hgetall converts its replies to an Object. If
the reply is empty, null is returned"???
On my server 2.3 alpha, the method metadata_for(meeting_id) gives back {}
(empty Hash). Thus "return if meeting_metadata.nil?" does not occur.
Does @redis.hgetall give {} instead of nil, even though there is a comment in
node_modules/redis/lib/utils.js "hgetall converts its replies to an Object. If
the reply is empty, null is returned"???
Update the list of invalid characters based on what the XML
specification permits and discourages.
Use the ruby string `scrub` method to remove invalid characters that
can't be expressed in the `tr` syntax, like unpaired surrogates and
UTF-8 prefix bytes.
This commit fixes an issue with reading and writing files.
File.open is used which means that a file will remain open
unless explicilty closed or the program exit. This doesn't work
for an NFS mount as the scripts try to "rm -rf" when the file
is still open. This commit fixes that by replacing all .opens
with .reads
The "originalFilename" tag is not present in "SharePresentationEvent", and thus presentation_filename would never be "default.pdf". As a consequence, the thumbnails are always generated from the default.pdf.
Playback a presentation does not start on iPad with iPadOS 13 and higher.
The root cause is that `mobileAndTabletCheck` does not detect an iPad as mobile device.
The reasons for this are discussed here: https://github.com/serbanghita/Mobile-Detect/issues/795
A way to detect Safari on IPadOS as mobile device is described here: https://stackoverflow.com/a/60553965
I've added a function `detectLyingiOS13iPad` to work around this problem.
After the last segment of a recording has been archived, it is safe to
delete the original files that were recorded by the various media
handling components. This patch deletes the freeswitch audio files and
kurento webcam/screenshare files after they have been archived.
It needs to be used in combination with some changes to users/groups and
directory permissions to allow the `bigbluebutton` user to delete files
that it could previously only read.
I've moved the workers code into the `lib` subdirectory with other library-ish
code; this puts it into the ruby load path used by most scripts so referencing
files is easier.
I've applied various style cleanups based on the rubocop config present.
The `events` processing step has been integrated as a new worker `EventsWorker`,
there is no longer a separate `events/events.rb` script. I've reworked the
`rap-starter.rb` script to check for the done files in both the events and
recorded status directories.
Write a tool that generates the poll svg images directly from the
BBB poll description. This avoids the issues with special characters
in the gnuplot labels, and gives us a lot more flexibility in how
the polls are formatted and styled.
Changed only in the main class so journald is the default and in the
scripts related to the processes in resque. Internal scripts might still
be logging to files.
Instead of being executed every 30s by systemd, it's now a service that's
running all the time and will wait for .done files to start the processing
of recordings.
Add an example to enable generating an mp4 file (Apple device compatibility)
Switch the webm generation to use the faster single-pass encode by default,
since 2-pass is much slower and more cpu intensive.
Someone on the mailing list had some recordings which were using the 2.0
playback, but were missing the deskshare.xml file (which should always
be present for 2.0… strange). It's safe to continue loading the recording
playback if the deskshare.xml file is not found, the recording will just act
as if there were no deskshare start/stop events.
Previously the setMediaSync function was only called after the deskshare
loaded, but by moving it to run after all media loaded, it now runs even
on recordings that didn't have deskshare. Make it do nothing (return early)
in that case.
If the secondary media loaded before the main media, it would run the
"setMediaSync" function before the main media player was setup. As a
side-effect of setting up the main media player, all of the event handlers
added by the setMediaSync function are detached, and so the secondary
media never starts playing.
Move the call to setMediaSync to after the media-ready events for all
media have fired, so that it can reliably attach the event handlers.
This reverts a bbb-specific customization made in the jquery.acornmediaplayer.js
file: it's restored to what the upstream player did. I can't find any explanation
for why this change was made in the first place? Reverting it doesn't seem to
cause any playback issues (Popcorn still works, in particular).
Found another case where the html5 client was passing through control
characters, in the original presentation name field.
Rather than play whack-a-mole with different fields which may eventually
get poorly sanitized user data, apply the control character filtering
to all properties.
Adjust the character range to do the following:
* Allow horizontal tab (0x09), it's not problematic.
* Disallow control characters in the range 0x1A-0x1F. Probably missed by accident.
It used to print:
Failed to download file: undefined local variable or method `respose' for BigBlueButton:Module
Did you mean? response
because the incorrect variable name was used in the error message.
There was no effect other than the message in the log, since the shared notes
couldn't be archived anyways, and the only thing the exception did was ...
prevent the shared notes from being archived.
FFmpeg has pretty good format autodetection even if the filename has the
'.txt' extension, so just rely on that. It'll even pull subtitles out of
video files - although I expect we'll have size limits so that doesn't
happen.
Rather than running the tool in a loop, I'm using inotify to watch for
new files being created (ideally, the other rap workers will be migrated
to this style in the future). The trigger for processing is the creation
of the .json index file for the uploaded track.
This reverts commit 9ad783fab0.
We ended up not being able to rename the etherpad nginx file, so this
recording format has to go back to using a different name.
Sometimes when text is pasted into the whiteboard text tool from
external apps, it'll include control characters that mess up later
recording processing scripts.
Run the same cleanup as already used for chat messages on the whiteboard
text as well.
The cleanup has been adjusted to allow newline and tab characters. They
won't really cause issues in chat, and newlines (at a minimum) are
required for the whiteboard.
This is a workaround for #7356
This was added to workaround for red5 taking a while to rewrite the
serialized (.ser) data that it streams to disk back to the .flv format.
The workaround is no longer needed, for two reasons:
* The sanity scripts run the red5 code to generate the .flv from the .ser
if needed, and
* We're expecting more people to be using WebRTC media in the future anyways
This makes recordings available up to 2 minutes earlier than they would have
been otherwise.
BigBlueButton can sometimes write events out of order - this particularly
seems to affect the final RecordStatusEvent in a meeting which was ended
while recording was still running. This breaks the recording processing
scripts.
As a workaround, sort the events as they're being written into the events.xml
file. We have the following properties:
* The input data is already mostly sorted
* Items in the wrong position will be no more than a couple spots off from where
they should be
* We should not change the relative order of events with the same timestamp.
The best algorithm to use here is a simple insertion sort. When adding each new
event to the XML structure, it scans backwards through existing events until it
finds the correct position.
For #6035
At some point, BigBlueButton switched from using `"` to `'` in the link tags in
chat messages, which caused the regular expression that was supposed to remove
the `event:` prefix to not match.
I've replaced the error-prone regular expression with an actual HTML parser,
using the "Loofah" HTML transformation/sanitization library based on Nokogiri.
I've removed the code that detected unlinked URLs, since it was broken - and
not needed: current BigBlueButton versions do the link detection in the client.
If someone reprocesses a really old BBB recording with these scripts, URLs in
chat might not be linked in the result. But they wouldn't have been linked in the
client in the original meeting either, so I figure that's ok.
Fixes#6475
The initialization order change to support captions moved the acorn
initialization to after some of the popcorn init had already run. We
have to move that function to run after we create the acorn player
so it references the correct #video element (since acorn replaces/
moves it during init)
In some cases, ffmpeg will be able to read the file, but the video itself
can't be decoded (missing/corrupt stream headers, for example). In this case,
some of the properties on the stream object will be nil.
Make sure that pix_fmt is present in the probed info, since that's a required
property.
Previously, the acorn player was being initialized before the asynchronous
load of captions information completed, meaning that the acorn player didn't
know there were caption tracks, and the CC selector control wasn't available.
Fixes#6463
Issue #6338
It looks like there was a logic error in the code that was causing it
to break out of the event deletion loop early when deleting events for
the last (or only) segment in a recording. (In this case, last_index
is -1, so i >= last_index is always true).
The trim_events_for call was always succeeding, so the events were
being removed from the event list (meeting:{ID}:recordings key) even
though the events themselves hadn't been deleted in the loop.
I've moved the trim_events_for call to below the event deletion loop
to ensure that if the archive script is interrupted, the events list
will contain all not-yet-deleted events.
It was previously trying to get the presentation name from the wrong variable,
which resulted in a nil value (which was treated as a blank string in
filenames). This caused the poll images to not be inside the presentation
subdirectories, and the poll image references in the svg contained a `//`
path, which would break the recording if it was uploaded to e.g. AWS S3.
This fixes font scaling in the presentation area, since that relied on
the <p> element inheriting the font size from the svg <g> that it was
inside of.
This was broken with the switch to the Foundation stylesheet base, which
set a fixed font size on the <p> element.
In some cases, we get DRAW_END events for pencil shapes from the
html5 client that have no dataPoints. The only thing we can really
do here is detect the issue and ignore the shape.
In some cases, this may result in the shape's intermediate drawing
updates being shown, but it'll disappear when the end event happens.
The archive_dir can by the raw recording directory in some recording
formats - including, hopefully, presentation at some point (to avoid the
extra copy)
The previous calculation used the video size in pixels, which might lead
to stretched/squashed videos in rare cases where the video has non-square
pixels. The new calculation is correct for all video sizes.
The ffprobe command in ffmpeg 4.0 now omits the aspect ratio fields in the
json output when indeterminate, instead of returning an invalid value with
0 in the numerator or denominator. Handle this correctly.
- some conflicts have been fixed.
The following needs working on:
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/services/ChatCopy.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/services/ChatSaver.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/views/ChatTab.mxml
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/views/ChatWindowEventHandler.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/services/MessageReceiver.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/services/MessageSender.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/views/MediaItemRenderer.mxml
Parking as need to work on something else.
Since the update to the newer red5, seeking in flv files (webcams in
particular are noticable) has been broken, resulting in cameras
appearing to "hang" any time there is a cut in the generated video -
which happens when start/stop button is pushed, or when cameras are
added or removed.
We can detect the problematic video files because the timestamp of the
first frame is large (old red5 versions always set first frame
timestamp to 0.001 seconds). If we see a file like this, having ffmpeg
remux the file - rewriting the timestamps and index - works around the
problem.
Due to improvements in the recording scripts, most of the stuff the sanity
script was checking for is no longer needed (missing/corrupt video files
are handled by the processing scripts). The version of this script in
master has been cleaned up so that the only things it does are:
- Check that the events.xml exists and is properly formatted xml
- Rebuild flv files from red5 .flv.ser/.flv.info files
The script from master is compatible with the 2.0 code, so just use it
as-is.
This fixes a problem where following the recent red5 upgrade in 2.0 branch,
an flv file is never written for webcam streams where no frames were
received, despite there being recording events.
This improves the quality of portrait documents, before they were
1200px when landscape documents got 1600px.
Switching to scaling to a square means that we can use the "-scale-to"
option on pdftocaio, which means that it generates images directly
at the desired size. This can save quite a bit of time (and memory)
if a document was uploaded with extremely large page size.
There's some cases where you can get 0-duration recordings due to
recording event placement (e.g. a single recording event is the last
event in the events.xml). Detect these cases, and treat them like
no recording marks in the archive script (it will stop the recording
from being automatically processed).
I've also adjusted the sanity script to detect these cases and error
out. The recording processing scripts cannot handle 0-length recordings,
you have to manually edit the events. I've added a message to the
sanity log about this.
In some cases (due to network issues), the webcam video can be shorter
than the time between the start/stop events. Pad the input video with a
blank video to make sure that there's input to the video tiling filters,
to fix a problem where the video won't render correctly with ffmpeg v3.4
and later.
