This is just a bundle of a few things I've been fixing up in the past
while.
= Workaround for BBB 1.1 beta deskshare timestamp bug
This is unlikely to be used, but I have the code for it, might as
well merge it in.
= Rework video tiling code for ffmpeg
Render video using the 'hstack' and 'vstack' filters rather than the
'overlay' filter. This is somewhat faster, particularly with lots of
videos.
= Etc.
- Remove usage of the streamio-ffmpeg gem.
The video rendering code has some stuff to directly read 'ffprobe'
output, so re-use that instead of this gem (which is kind of old and
has issues with newer ffmpeg versions).
- Don't hardcode the deskshare video area size, pull it from the
properties file
- Remove some code that worked around missing video end events.
In some cases this could cause flickering or strange video issues.
It's no longer strictly needed, the new tiling code doesn't break if
the seekpoint is after the end of the video.
The new shapes code, required for handling smooth shape updates & multi-user
whiteboard in the 2.0 BigBlueButton, hits a bug in old recordings where
the pencil tool incorrectly used "line" in its shape names, meaning that
there could be both a pencil mark and a line with the same shape name.
The old recording code didn't rely on the shape name to match shapes, since
there was no chance of concurrent shapes. As this is an incompatible playback
change, we need to make a new playback directory for the updated files.
The old code was very difficult to follow, and I couldn't figure out
a good way to retrofit the BigBlueButton 2.0 undo by shape id and clear
by user id into it - so I rewrote the entire thing instead.
It now generates the shapes.svg, panzooms.xml and cursor.xml all at the
same time during a single pass through the event.xml file. The result
is compatible with the existing recording javascript (at least once a few
minor issues in writing.js were fixed by earlier commits).
The previous code would cause shapes to "blink" during updating if
the updates weren't continuous - in a gap between updates, the shape
would disappear.
Rework the logic for looking up "current" shapes to return the
nearest previous update rather than only exact matching timestamps,
and simplify the logic that decides whether to make a shape visible
or hidden.
Current BBB client is generating invalid indexes when characters are
inserted with an IME - the edit indexes are as if the preedit text was
being removed, but the preedit text was never sent in the first place.
For now, just don't crash if there's an edit that would remove text
which is past the end of known text. The result might be broken, but
it won't prevent the rest of the recording from working.
I've had a chance to see how this script behaves with actual live
caption tracks now, and there's room for improvement. In particular,
it often generates cues that overlap - the next one appears before the
previous one disappears. The browser player position handles this
really poorly, and it's nearly unreadable.
The solution is to, if two cues would overlap, merge them into a
single multiline cue that displays for the full time. This is a lot
easier to read.
Some extra code is added to de-overlap any remaining cues (e.g. if
there's a third cue that would also overlap). This will reduce the
time that the earlier cue gets shown below my preferred minimum, but
not really much we can do about that if people are talking/typing
quickly.
The code can easily be tweaked to set a different number of maximum
lines per cue if desired.
We were getting the duration with a Popcorn temporary instance. This was causing a slowly increase on the cpu usage, which could compromise the performance of long recordings.
- Created function "seySync( )" which sets the sync logic. Basically, we sync all medias when pausing the master video. We pause the master video when any video receives a waiting event, or when the master video is seeked. If necessary, we resume playing after the sync.
- Refactored when 'media-ready' event is fired. We fire this event only when the medias receive a 'canplayall' event.
- Popcorn "onFrame" method runs around 6 times per second. That was unnecessary , causing the CPU work to significantly increase, and preventing the two videos (deskshare and webcam) to run together smoothly
- We only synchronize the medias if one of them fires a "waiting" event. If the user pauses the playback, we synchronize them as well.
If processing didn't succeed, the directory where it is trying to write
the file might not exist, causing an exception in the worker which leads
to it getting stuck in a retry loop.
In BBB 0.81, the slide number was base 0, while the shape page number was
base 1. We already have a conditional for checking 0.9+; use that to
add a workaround for old BBB recordings.
The previous code assumed that if a shape event's timestamp was during the
time when a particular slide was visible, then the shape belonged to that
slide. This isn't always true with the text shapes, since a final close
event can be sent after the slide has switched.
For a more future-proof matching, use the presentation name and page number
fields on the shape events when available to match them with the appropriate
slide image.
The playback had no handling for this type of event. The aspect ratio used to calculate the max-width of the slide div has to be the vbox aspect ratio.
gnuplot's auto-margins don't work well on this graph, so manually specify
the top/bottom (actually left/right - the graph is rotated 90°) to make
the bars fit better, particularly when using wide aspect ratio slides.
The code was previously dividing by the total number of votes rather than
the max number of votes, which meant it was underestimating the width of
the bars. In some cases, this meant that the label for number of votes would
overlap the percentages.
Fixes#3725
The purpose of this is to ensure that the recording processing with make
visible forwards progress, by ensuring that different steps in the
recording pipeline don't block each-other.
This fixes:
- recordings being processed prevent archiving new recordings from running
- recordings can't be published until all pending recording processing
completes
It does allow recording archive, sanity, process, publish steps to run in
parallel with each other, but at most one recording will be in each step at
a time. (I.e. while one recording is being processed, a different recording
can be published). This may potentially increase CPU usage for some users.
If you expect this to be a problem, you can set resource controls (see
`man systemd.resource-control`) on the bbb_record_core.slice systemd unit.
The code was previously passing the message string provided by the user
directly to jQuery - which works okish if the first character is '<' since
it'll parse it as HTML, but the chat messages don't. As a result, it was
sometimes being parsed as a selector, failing, and raising an exception.
The fix is to put the chat message into a DOM node (have the browser parse
the HTML) before doing the jQuery operation to modify the link targets.
Fixes#3670
Chrome frame has been discontinued for quite a while. The playback code actually
does work in IE if you install an extra codec pack or switch the video format used.
Due to the extra delays added by setting up the closed captions, it happens more
often that the writing.js code will attempt to initialize popcorn on the video
element before the video element actually exists on the page.
(This was always a problem, it's just worse now)
Add a synchronization point which will delay the popcorn initialization until the
video element is ready.
- Getting slide dimensions to properly set the panzoom event when deskshare stops.
- Sometimes there's a panzoom event with the deskshare image dimension at time=stop_timestamp+0.1. We remove it.
- Little refactoring
- Properly inserting deskshare panzoom events. We insert a panzoom event with the deskshare video dimensions and the deskshare start timestamp. Immediately, we insert another panzoom event with the previous slide dimensions and the deskshare stop timestamp, deleting any other panzoom event inside this interval of time.
- Handling when deskshare events occur in timestamp = 0.0
- For now, we are limiting the max dimensions to 1280x720
Pending issues:
- Image and ViewBox sizes are hardcoded (1280x720 - a full screen deskshare video). In order to work with region deskshare videos, we must get the flvs resolutions and set shapes.svg and panzooms accordingly.
- We're not considering the deskshare annotations history yet
This is a demo. Some issues:
- When sharing desktop, cursor is still relative to slides, and not to deskshare video;
- Need to show whiteboard canvas on top of deskshare video as well;
- Need more testing (audio/video sync, playback perfomance, multiple presenters and multiple deskshare events).
1. We create a deskshare.xml with start and stop timestamps of each deskshare event
2. We read it in playback e use its information to hide the annotation canvas and cursor accordingly
