In some cases, we get DRAW_END events for pencil shapes from the
html5 client that have no dataPoints. The only thing we can really
do here is detect the issue and ignore the shape.
In some cases, this may result in the shape's intermediate drawing
updates being shown, but it'll disappear when the end event happens.
The archive_dir can by the raw recording directory in some recording
formats - including, hopefully, presentation at some point (to avoid the
extra copy)
The previous calculation used the video size in pixels, which might lead
to stretched/squashed videos in rare cases where the video has non-square
pixels. The new calculation is correct for all video sizes.
The ffprobe command in ffmpeg 4.0 now omits the aspect ratio fields in the
json output when indeterminate, instead of returning an invalid value with
0 in the numerator or denominator. Handle this correctly.
- some conflicts have been fixed.
The following needs working on:
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/services/ChatCopy.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/services/ChatSaver.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/views/ChatTab.mxml
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/views/ChatWindowEventHandler.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/services/MessageReceiver.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/services/MessageSender.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/views/MediaItemRenderer.mxml
Parking as need to work on something else.
Since the update to the newer red5, seeking in flv files (webcams in
particular are noticable) has been broken, resulting in cameras
appearing to "hang" any time there is a cut in the generated video -
which happens when start/stop button is pushed, or when cameras are
added or removed.
We can detect the problematic video files because the timestamp of the
first frame is large (old red5 versions always set first frame
timestamp to 0.001 seconds). If we see a file like this, having ffmpeg
remux the file - rewriting the timestamps and index - works around the
problem.
Due to improvements in the recording scripts, most of the stuff the sanity
script was checking for is no longer needed (missing/corrupt video files
are handled by the processing scripts). The version of this script in
master has been cleaned up so that the only things it does are:
- Check that the events.xml exists and is properly formatted xml
- Rebuild flv files from red5 .flv.ser/.flv.info files
The script from master is compatible with the 2.0 code, so just use it
as-is.
This fixes a problem where following the recent red5 upgrade in 2.0 branch,
an flv file is never written for webcam streams where no frames were
received, despite there being recording events.
This improves the quality of portrait documents, before they were
1200px when landscape documents got 1600px.
Switching to scaling to a square means that we can use the "-scale-to"
option on pdftocaio, which means that it generates images directly
at the desired size. This can save quite a bit of time (and memory)
if a document was uploaded with extremely large page size.
There's some cases where you can get 0-duration recordings due to
recording event placement (e.g. a single recording event is the last
event in the events.xml). Detect these cases, and treat them like
no recording marks in the archive script (it will stop the recording
from being automatically processed).
I've also adjusted the sanity script to detect these cases and error
out. The recording processing scripts cannot handle 0-length recordings,
you have to manually edit the events. I've added a message to the
sanity log about this.
In some cases (due to network issues), the webcam video can be shorter
than the time between the start/stop events. Pad the input video with a
blank video to make sure that there's input to the video tiling filters,
to fix a problem where the video won't render correctly with ffmpeg v3.4
and later.
In some cases (due to network issues), the webcam video can be shorter
than the time between the start/stop events. Pad the input video with a
blank video to make sure that there's input to the video tiling filters,
to fix a problem where the video won't render correctly with ffmpeg v3.4
and later.
In some cases when there is a slight mismatch between audio file
duration and event timestamp difference, and we have a record
status or chapter break event in a certain location, it could
trigger a seek past the end of an audio file. Detect this
condition and just render silence instead.
Also adjust the thresholds for the audio length scaling - they
were being triggered on short recordings that should be correct.
In some cases when there is a slight mismatch between audio file
duration and event timestamp difference, and we have a record
status or chapter break event in a certain location, it could
trigger a seek past the end of an audio file. Detect this
condition and just render silence instead.
Also adjust the thresholds for the audio length scaling - they
were being triggered on short recordings that should be correct.
With the current segment processing, we might be processing one segment
while archiving a different segment from the same recording. To avoid
that the processing scripts see an incomplete events.xml file, write to
a temp file then rename.
Red5 sometimes writes webcam video files with a large offset in the
video frame offsets, sometimes up to 30 or even 60 seconds. However,
the start event in the events.xml file corresponds to the time at
which red5 received the first keyframe (recorded frame) in the video.
The end result is that the video will sometimes appear to be
delayed (out of sync) in the processed recording.
The correction is simple: We're already reading video metadata,
including the timestamp of the first frame, so we just have to apply
a correction during video processing to undo the frame timestamp
offsets in the video file.
Red5 sometimes writes webcam video files with a large offset in the
video frame offsets, sometimes up to 30 or even 60 seconds. However,
the start event in the events.xml file corresponds to the time at
which red5 received the first keyframe (recorded frame) in the video.
The end result is that the video will sometimes appear to be
delayed (out of sync) in the processed recording.
The correction is simple: We're already reading video metadata,
including the timestamp of the first frame, so we just have to apply
a correction during video processing to undo the frame timestamp
offsets in the video file.
When working with the segmented recording format, the events file might
end with a mismatched start video event for an incomplete file. The
sanity script was removing this event, meaning the video didn't show up
in future segments.
Simply drop the code that tries to find invalid video files and removes
them from the events file. The new video processing code is already
robust against missing or corrupt files.
If no audio files were found, it was running rsync with one argument,
which is a bit unexpected. It just printed a file list in this case, but
we can provide a cleaner error message instead.
It previously checked whether any part of the entire meeting was recorded.
Helper functions are added to look up the time of segment start and end
(which handle non-segmented recordings correctly too).
Part of the events handling code was rewritten to reduce the number of times
that the events.xml file gets parsed.
If you're inserting at position 0 (and there was no previous deleted text
from that position), you can't use the timestamp from the previous character
position, since there's no previous character. Use the timestamp of the
following character instead.
In some unusual cases, the recording start can be the last event in the
events file, or at least have the same timestamp as such.
Add some code to check the array bounds and break if needed, so we
don't check the timestamp on the (non-existant) event after the last
event.
The previous code looked for stop events and tried to find their
associated start event. This obviously doesn't work if there was
no stop event. But if there was a start event, we need to show the
deskshare… so rework to code to try to find the matching stop to each
start instead, and use the end of the meeting if no matching stop was
found.
Some old recordings might have invalid or legacy encoding stuff
in the text files. To allow processing to continue, just re-encode int
UTF-8 with the invalid option set to replace, to remove the invalid
characters.
In BBB 2.0, the cursor positions are given relative to the page
size (like annotation positions). Since the recording cursors
aren't actually drawn in the page like annotations, it's more
convenient to have them relative to the visible area (viewbox),
so do that conversion.
While I'm in here - since we switched to new incompatible scripts
for BBB 2.0 anyways - remove an extra factor in the cursor positions
in cursor.xml, and just use a simple ratio of width/height instead.
This is just a bundle of a few things I've been fixing up in the past
while.
= Workaround for BBB 1.1 beta deskshare timestamp bug
This is unlikely to be used, but I have the code for it, might as
well merge it in.
= Rework video tiling code for ffmpeg
Render video using the 'hstack' and 'vstack' filters rather than the
'overlay' filter. This is somewhat faster, particularly with lots of
videos.
= Etc.
- Remove usage of the streamio-ffmpeg gem.
The video rendering code has some stuff to directly read 'ffprobe'
output, so re-use that instead of this gem (which is kind of old and
has issues with newer ffmpeg versions).
- Don't hardcode the deskshare video area size, pull it from the
properties file
- Remove some code that worked around missing video end events.
In some cases this could cause flickering or strange video issues.
It's no longer strictly needed, the new tiling code doesn't break if
the seekpoint is after the end of the video.
The new shapes code, required for handling smooth shape updates & multi-user
whiteboard in the 2.0 BigBlueButton, hits a bug in old recordings where
the pencil tool incorrectly used "line" in its shape names, meaning that
there could be both a pencil mark and a line with the same shape name.
The old recording code didn't rely on the shape name to match shapes, since
there was no chance of concurrent shapes. As this is an incompatible playback
change, we need to make a new playback directory for the updated files.
The old code was very difficult to follow, and I couldn't figure out
a good way to retrofit the BigBlueButton 2.0 undo by shape id and clear
by user id into it - so I rewrote the entire thing instead.
It now generates the shapes.svg, panzooms.xml and cursor.xml all at the
same time during a single pass through the event.xml file. The result
is compatible with the existing recording javascript (at least once a few
minor issues in writing.js were fixed by earlier commits).
The previous code would cause shapes to "blink" during updating if
the updates weren't continuous - in a gap between updates, the shape
would disappear.
Rework the logic for looking up "current" shapes to return the
nearest previous update rather than only exact matching timestamps,
and simplify the logic that decides whether to make a shape visible
or hidden.
Current BBB client is generating invalid indexes when characters are
inserted with an IME - the edit indexes are as if the preedit text was
being removed, but the preedit text was never sent in the first place.
For now, just don't crash if there's an edit that would remove text
which is past the end of known text. The result might be broken, but
it won't prevent the rest of the recording from working.
I've had a chance to see how this script behaves with actual live
caption tracks now, and there's room for improvement. In particular,
it often generates cues that overlap - the next one appears before the
previous one disappears. The browser player position handles this
really poorly, and it's nearly unreadable.
The solution is to, if two cues would overlap, merge them into a
single multiline cue that displays for the full time. This is a lot
easier to read.
Some extra code is added to de-overlap any remaining cues (e.g. if
there's a third cue that would also overlap). This will reduce the
time that the earlier cue gets shown below my preferred minimum, but
not really much we can do about that if people are talking/typing
quickly.
The code can easily be tweaked to set a different number of maximum
lines per cue if desired.
We were getting the duration with a Popcorn temporary instance. This was causing a slowly increase on the cpu usage, which could compromise the performance of long recordings.
- Created function "seySync( )" which sets the sync logic. Basically, we sync all medias when pausing the master video. We pause the master video when any video receives a waiting event, or when the master video is seeked. If necessary, we resume playing after the sync.
- Refactored when 'media-ready' event is fired. We fire this event only when the medias receive a 'canplayall' event.
- Popcorn "onFrame" method runs around 6 times per second. That was unnecessary , causing the CPU work to significantly increase, and preventing the two videos (deskshare and webcam) to run together smoothly
- We only synchronize the medias if one of them fires a "waiting" event. If the user pauses the playback, we synchronize them as well.
If processing didn't succeed, the directory where it is trying to write
the file might not exist, causing an exception in the worker which leads
to it getting stuck in a retry loop.
In BBB 0.81, the slide number was base 0, while the shape page number was
base 1. We already have a conditional for checking 0.9+; use that to
add a workaround for old BBB recordings.
The previous code assumed that if a shape event's timestamp was during the
time when a particular slide was visible, then the shape belonged to that
slide. This isn't always true with the text shapes, since a final close
event can be sent after the slide has switched.
For a more future-proof matching, use the presentation name and page number
fields on the shape events when available to match them with the appropriate
slide image.
The playback had no handling for this type of event. The aspect ratio used to calculate the max-width of the slide div has to be the vbox aspect ratio.
gnuplot's auto-margins don't work well on this graph, so manually specify
the top/bottom (actually left/right - the graph is rotated 90°) to make
the bars fit better, particularly when using wide aspect ratio slides.
The code was previously dividing by the total number of votes rather than
the max number of votes, which meant it was underestimating the width of
the bars. In some cases, this meant that the label for number of votes would
overlap the percentages.
Fixes#3725
The purpose of this is to ensure that the recording processing with make
visible forwards progress, by ensuring that different steps in the
recording pipeline don't block each-other.
This fixes:
- recordings being processed prevent archiving new recordings from running
- recordings can't be published until all pending recording processing
completes
It does allow recording archive, sanity, process, publish steps to run in
parallel with each other, but at most one recording will be in each step at
a time. (I.e. while one recording is being processed, a different recording
can be published). This may potentially increase CPU usage for some users.
If you expect this to be a problem, you can set resource controls (see
`man systemd.resource-control`) on the bbb_record_core.slice systemd unit.
The code was previously passing the message string provided by the user
directly to jQuery - which works okish if the first character is '<' since
it'll parse it as HTML, but the chat messages don't. As a result, it was
sometimes being parsed as a selector, failing, and raising an exception.
The fix is to put the chat message into a DOM node (have the browser parse
the HTML) before doing the jQuery operation to modify the link targets.
Fixes#3670
Chrome frame has been discontinued for quite a while. The playback code actually
does work in IE if you install an extra codec pack or switch the video format used.
Due to the extra delays added by setting up the closed captions, it happens more
often that the writing.js code will attempt to initialize popcorn on the video
element before the video element actually exists on the page.
(This was always a problem, it's just worse now)
Add a synchronization point which will delay the popcorn initialization until the
video element is ready.
- Getting slide dimensions to properly set the panzoom event when deskshare stops.
- Sometimes there's a panzoom event with the deskshare image dimension at time=stop_timestamp+0.1. We remove it.
- Little refactoring
