The ffprobe command in ffmpeg 4.0 now omits the aspect ratio fields in the
json output when indeterminate, instead of returning an invalid value with
0 in the numerator or denominator. Handle this correctly.
- some conflicts have been fixed.
The following needs working on:
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/services/ChatCopy.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/services/ChatSaver.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/views/ChatTab.mxml
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/chat/views/ChatWindowEventHandler.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/services/MessageReceiver.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/services/MessageSender.as
both modified: ../bigbluebutton-client/src/org/bigbluebutton/modules/users/views/MediaItemRenderer.mxml
Parking as need to work on something else.
Since the update to the newer red5, seeking in flv files (webcams in
particular are noticable) has been broken, resulting in cameras
appearing to "hang" any time there is a cut in the generated video -
which happens when start/stop button is pushed, or when cameras are
added or removed.
We can detect the problematic video files because the timestamp of the
first frame is large (old red5 versions always set first frame
timestamp to 0.001 seconds). If we see a file like this, having ffmpeg
remux the file - rewriting the timestamps and index - works around the
problem.
This improves the quality of portrait documents, before they were
1200px when landscape documents got 1600px.
Switching to scaling to a square means that we can use the "-scale-to"
option on pdftocaio, which means that it generates images directly
at the desired size. This can save quite a bit of time (and memory)
if a document was uploaded with extremely large page size.
In some cases (due to network issues), the webcam video can be shorter
than the time between the start/stop events. Pad the input video with a
blank video to make sure that there's input to the video tiling filters,
to fix a problem where the video won't render correctly with ffmpeg v3.4
and later.
In some cases (due to network issues), the webcam video can be shorter
than the time between the start/stop events. Pad the input video with a
blank video to make sure that there's input to the video tiling filters,
to fix a problem where the video won't render correctly with ffmpeg v3.4
and later.
In some cases when there is a slight mismatch between audio file
duration and event timestamp difference, and we have a record
status or chapter break event in a certain location, it could
trigger a seek past the end of an audio file. Detect this
condition and just render silence instead.
Also adjust the thresholds for the audio length scaling - they
were being triggered on short recordings that should be correct.
In some cases when there is a slight mismatch between audio file
duration and event timestamp difference, and we have a record
status or chapter break event in a certain location, it could
trigger a seek past the end of an audio file. Detect this
condition and just render silence instead.
Also adjust the thresholds for the audio length scaling - they
were being triggered on short recordings that should be correct.
With the current segment processing, we might be processing one segment
while archiving a different segment from the same recording. To avoid
that the processing scripts see an incomplete events.xml file, write to
a temp file then rename.
Red5 sometimes writes webcam video files with a large offset in the
video frame offsets, sometimes up to 30 or even 60 seconds. However,
the start event in the events.xml file corresponds to the time at
which red5 received the first keyframe (recorded frame) in the video.
The end result is that the video will sometimes appear to be
delayed (out of sync) in the processed recording.
The correction is simple: We're already reading video metadata,
including the timestamp of the first frame, so we just have to apply
a correction during video processing to undo the frame timestamp
offsets in the video file.
Red5 sometimes writes webcam video files with a large offset in the
video frame offsets, sometimes up to 30 or even 60 seconds. However,
the start event in the events.xml file corresponds to the time at
which red5 received the first keyframe (recorded frame) in the video.
The end result is that the video will sometimes appear to be
delayed (out of sync) in the processed recording.
The correction is simple: We're already reading video metadata,
including the timestamp of the first frame, so we just have to apply
a correction during video processing to undo the frame timestamp
offsets in the video file.
It previously checked whether any part of the entire meeting was recorded.
Helper functions are added to look up the time of segment start and end
(which handle non-segmented recordings correctly too).
Part of the events handling code was rewritten to reduce the number of times
that the events.xml file gets parsed.
In some unusual cases, the recording start can be the last event in the
events file, or at least have the same timestamp as such.
Add some code to check the array bounds and break if needed, so we
don't check the timestamp on the (non-existant) event after the last
event.
The previous code looked for stop events and tried to find their
associated start event. This obviously doesn't work if there was
no stop event. But if there was a start event, we need to show the
deskshare… so rework to code to try to find the matching stop to each
start instead, and use the end of the meeting if no matching stop was
found.
This is just a bundle of a few things I've been fixing up in the past
while.
= Workaround for BBB 1.1 beta deskshare timestamp bug
This is unlikely to be used, but I have the code for it, might as
well merge it in.
= Rework video tiling code for ffmpeg
Render video using the 'hstack' and 'vstack' filters rather than the
'overlay' filter. This is somewhat faster, particularly with lots of
videos.
= Etc.
- Remove usage of the streamio-ffmpeg gem.
The video rendering code has some stuff to directly read 'ffprobe'
output, so re-use that instead of this gem (which is kind of old and
has issues with newer ffmpeg versions).
- Don't hardcode the deskshare video area size, pull it from the
properties file
- Remove some code that worked around missing video end events.
In some cases this could cause flickering or strange video issues.
It's no longer strictly needed, the new tiling code doesn't break if
the seekpoint is after the end of the video.
For now the differences are that the archive worker will run inside
resque (it is a resque worker now) and there is a "rap-trigger" file that
is executed by systemd to detect recordings that need to be archived and
add a job on resque to archive them.
The process os scheduling jobs still needs to be reviewed, but for now will
be done by rap-trigger.
This is a demo. Some issues:
- When sharing desktop, cursor is still relative to slides, and not to deskshare video;
- Need to show whiteboard canvas on top of deskshare video as well;
- Need more testing (audio/video sync, playback perfomance, multiple presenters and multiple deskshare events).
Instead of assuming that wav files have 44 byte headers (they usually
don't - freeswitch adds some metadata tags), add a super-simple wave
file parser that just finds the offset where the audio data actually
starts.
Main change is to rewrite the start/stop event handling to do fewer
passes and use simpler logic (it no longer has to track durations).
The temporary work audio codec is switched to flac; slightly lower hard
drive use and it avoids the 4-hour length limit of wav.
Some improvements to the video seeking when processing deskshare and
webcam videos, to avoid video dropouts.
Tweaks to the audio encoding quality settings.
Remove an unused function.
FreeSWITCH writes wav files >4GB long with incorrect values in the length
field in the header. Recalculate the length based on file size, and ensure
that ffmpeg reads the complete audio file rather than stopping.
FreeSWITCH writes wav files >4GB long with incorrect values in the length
field in the header. Recalculate the length based on file size, and ensure
that ffmpeg reads the complete audio file rather than stopping.
If the video EDL merge function is applied to an EDL that has already
been edited to apply recording marks, the merge function will change the
video start times to incorrect values on edit points during the timestamp
recalculation.
The fix is pretty simple; just pull in the timestamps from the entry that's
being merged in and apply them to the relevant videos.
This bug doesn't currently cause any issues in the BigBlueButton recording
scripts, since the existing code does the merge before the start/stop marks
are applied. But to avoid surprises later, it would be good to fix this.
Makes it handle possibly corrupted video files with less chance of
breakage. Use features from newer ffmpeg release to simplify the video
trimming - in particular, using -ss as an input option is accurate now.
This is the point at which the start/stop events for the audio are
correctly matched up with eachother; doing it later can give
incorrect results if an 'end recording' event was missing from
the events file.
The previous version may have the beginning of the audio off by a bit,
since it was seeking in the audio file to find the start before the
stretch was applied.
There's some fairly major changes here, including:
* All audio is resampled to 48kHz stereo on input, allowing files with
non-standard or varying rates to be mixed
* The audio processing is now done in a single pass; ffmpeg reads all
inputs, concatenates them, and outputs one file.
The 'mkclean' tool reorganizes the encoded webm file to optimize it for
streaming. In particular, it moves the index to the start of the file.
This fixes streaming in Chrome, which otherwise had a very long delay
before playback started since it downloaded until it saw the index
before it started playback.
This needs a new dependency added to the bbb-record-core package to
pull in the mkclean tool.
We now use ghostscript to output pngs directly from the original pdf,
rather than using convert on the split pages. This should make corrupt
or strange pdfs less likely to cause issues.
As well, if a pdf page conversion fails (for any reason, including that
the original pdf is missing...) it will be logged, and a blank page
generated, and processing will continue.
I've been working on this for a while, and it's adapted from code that
has been fairly well-tested on a wide variety of recordings. I've found
it to do a more accurate job of combining multiple webcam files, and it
should be more accurate in the audio as well.
Another key feature is that it does fewer re-encoding steps during video
processing, which should both speed it up and hopefully improve quality.
The settings on the VP8 encoder have been tuned somewhat as well.
For BigBlueButton 0.81, we just require that you install ffmpeg 0.11.x
for recordings to function correctly. I'll also add a check to bbb-conf
to verify that the correct ffmpeg is available.
also, the output video resolution is modified to the highest screen resolution shared during the conference, independently on the output video resolution set by the user; it was done to preserve the quality of the desktop sharing image; if there's no desktop sharing, the output video resolution is respected
Refactored the code that generates the video file used in the presentation playback. The problem with the sync of video sources was related to problems of precision inside FFmpeg (sometimes the script runs FFmpeg to trim video files, and the output video file trimmed sometimes was 3+ seconds bigger or smaller than the requested).
Also it's a little more modular, so it will be easier to expand functionality. The processing steps are more explicit now, and the logs are better now to understand the final output.
A new parameter on presentation.yml was added to enable the admin to add an offset for the audio stream - this is mainly useful when the session ran with h264, because the video takes longer for the client to encode, and then in the recording we can see the huge gap between audio and video.
