Firefox doesn't fire the ended evt/onended callback for live
MediaStreamTrack(s). We rely on that event.
Manually emit the ended event which works with the onended callback
when a track is stopped
kurento-utils is unmaintained. It's served us well, but its age
shows. We need to transition to something else if we want to
have better maintainability and include simulcast, multistream, ...
This introduces a simplified/leaner wrapper kit that's almost
API-compatible with what we use right now - so widespread changes
are minimal). It's easier to maintain/read/transition from. This
can be read as an intermediate step to transitioning to
something definitive (ie mediasoup-client).
Under some scenarios, cameras are freezing when the virtual background
code is running due to runPostProcessing(_renderMask) throwing
NS_ERROR_FAILURE - mainly on Firefox - consequently preventing
subsequent TimerWorker ticks from being scheduled.
Cases where I've seen that happen are:
- conferences running under an iframe where the iframe is briefly
stalled for some reason
Address the issue with a try-catch and a log for debugability (it's high
frequency, hence why not error level). We should probably remove the log
entirely once we figure out why the post-processing method is failing.
Use the built-in getLocalStream from the peer wrapper instead (which
relies on getSenders - the proper, spec-compliant way).
Two different issues are addressed:
- RTCPeerConnection.getLocalStreams is a pre-1.0 WebRTC spec which is
deprecated and not recommended.
- Fixed an issue where a switch from full audio to listen only could
cause the latter to be rejected with an error 1004 in rare scenarios.
There could be a race condition where the local getDisplayMedia stream ends
(eg via Chrome`s stop sharing toast) while the broker hasn't finished starting.
That would lead to a scenario where the broker wouldn't emit an end event,
causing screen sharing to be flagged as started with a blank/invalid stream.
- Remove the old listen only bridge (kurento.js), superseded by the equivalent
and equally stable (AS FAR AS LISTEN ONLY IS CONCERNED) sfu-audio-bridge
- Rename FullAudioBridge.js -> sfu-audio-bridge.js
* A more generic name that better represents the capabilities and
the nature of the bridge
* The bridge name identifier in configuration is still the same
('fullaudio')
- Remove the FreeSWITCH listen only fallback
- Temporarily disable the "trickle ICE" pair gathering feature used
in SIP.js (which was always experimental, nonstandard and disabled
by default)
- Updates to settings.yml keys in places where relevant
"default" is not an universally valid default value for deviceIds which was causing issues with Firefox and Safari in some specific scenarios where exact deviceId constraints were being used
Seems to have been introduced by a partial merge commit
There were a bunch of style changes introduced by that partial commit as well; I kept those changes to avoid introducing further conflicts between v2.4-2.5...
- forceRelayOnFirefox: whether TURN/relay usage should be forced to work
around Firefox's lack of support for regular nomination when dealing with
ICE-litee peers (e.g.: mediasoup).
* See: https://bugzilla.mozilla.org/show_bug.cgi?id=1034964
- iOS endpoints are ignored from the trigger because _all_ iOS browsers
are either native WebKit or WKWebView based (so they shouldn't be affected)
When joining breakouts, we now wait for the bridge to be loaded before
automatically start user's audio.
This problems happens only on fullaudio bridge
This commit allows user to join/leave audio using the fullaudio bridge.
This is still under development, but to use this now we must set values of
skipCheck to false, and defaultFullAudioBridge to fullaudio. This
depends on newest version of bbb-webrtc-sfu
ICE lite servers (eg mediasoup) dont need candidates signaled out-of-band; neither does KMS in certain scenarios
Disable their signaling saves us some ticks in bbb-webrtc-sfu and some bandwidth all around
Restored the old behavior when ending breakout rooms while user is in the
breakout audio transfer, which is to the trigger the reconnection to the audio
in the main room.
This behavior could be improved by (instead of reconnecting) transfering user
back to the main room, but this requires some changes in akka-apps/fsesl
which can be treated in a different issue.
Closes#13242
Undefined by default means that the governing configuration is in bbb-webrtc-sfu
Also add some inline docs in settings.yml about the media server adapter configs
Applies to video, listen only and screen sharing
New metadata values: media-server-video, media-server-listenonly, media-server-screenshare; parameter is a String
For browsers that don't support headerBytesSent in RTCOutboundRtpStreamStats
neither headerBytesReceived in RTCInboundRtpStreamStats, we are now able
to calculate upload and download rates.
We are also able to get transportStats information for browsers that
don't support iceTransport attribute of RTCDtlsTransport.
Added support for getStats in screenshare's service. This works similar
to the getStats for video provider, and the information retrieved from
screenshare is added to the video information for cameras.
Changes (maybe not a complete list):
- Disable virtualbgs by default
- Move the virtualbg selector in video-preview to the side below the
profile selection
- Restore old video-preview sizes
- Add a wrapper class for MediaStreams (BBBVideoStream)
- Centralize virtualbg services and business logic code into BBBVideoStream
- Refactor and centralize virtualbg constant fetching
- Refactor and centralize virtualbg config fetching
- Organize virtualbg type definitions
- Remove added states in video-provider to prevent further bloat
- Remove added states in video-preview to prevent further bloat
- Lock virtual bg switching while video-preview itself is locked
- Add proper virtualbg error surfacing via toasts
- Refactor iOS availability detection to use centralized UA checker
- Avoid calling gUM when toggling virtualbgs on/off
- Make virtualbg video-list-item action a toggle instead of a
state-aware action
- Make virtualbg switching work in video-preview for cameras that are
already shared. Especially useful when there are multiple source
cameras, and will be important in the near future
- Add Derivative Work notices in files that are partially copied from
jitsi-meet
- Simplify track replacing in video-provider
- Split video-preview UI code for virtualbgs into a separate functional component
Here's what we do when user activates mic:
1 - When we do something similar to listenonly's joining process
until we find a valid candidate-pair. The information about this
local candidate is store.
2 - We then start a new userAgent, and as soon as browser finds
a candidate with the same local ip address, we leave only this
candidate in the SDP and send this to FreeSWITCH. SDP should
contain only a single candidate.
3 - The rest of signaling process is basically the same.
Remove parts of a previous connection monitor.
To add some context (as far as my memory goes) to the multiple connection
monitor features the product has, `stats` (currently named `connection status`)
was introduced at the Flash client back in ~2016. @fcecagno and I did it
as a BigBlueButton's Summit activity. Our work was squashed into a single
commit in 92554f8b3e :).
I'm not sure about the whole story behind `network information` (the late
connection monitor added to the HTML5 client) but I assume it should work
as a collector for a bunch of different connectivity monitors. I remember
when it was introduced but I don't know why it wasn't adopted. My best guess
would be because of some performance issues the `user list` had back then.
To follow on why `connection status` replaced `network information` at the
HTML5 client, when I did the `multiple webcams` feature I had to refactor
a big chunk of the `video provider` (#8374). Something that wasn't really
helping there was the adaptation of `stats` that was made to show local
feedback for each webcam connection. Although this feature wasn't being
used anymore, `network information` did rely on that to build up data. With
this monitor gone I assumed it was my responsibility to provide an alternative
so I promoted Mconf's port of the Flash `stats` monitor to BigBlueButton's
HTML5 client (#8579).
Well, that's my perspective on how things went for those features. If
anyone would like to correct me on something or add something else on
that history I would appreciate to know.
This commit contains three fixes: one already reported and two detected
during the investigation of the solution.
This started as a fix for firefox (#12023), but i also fixed the muted
alert/banner when device changes: the banner wasn't detecting device changes,
unless audio was deactived/actived.
There's another fix for the microphone stream: we now keep sender's track
disabled if it was already disabled for the sender's track of the previous
selected device.
Also did small refactor for eslint checking.
Some technical information: in sip bridge (bridge/sip.js), setInputStream and
liveChangeInputDevice function were both fully turned into promises, which
guarantees we have everything ready when it resolves to the respective values.
This helps AudioManager (audio-manager/index.js) to sequentially sets and
tracks the state of the current microphone stream (inputStream), when calling
liveChangeInputDevice function: we first set the current stream to null,
creats a new one and then set it to the newly created value - this is needed
because MutedAlert (muted-alert/component.jsx) can then gracefully
allocate/deallocate the cloned stream when it is set to a non-null/null value
(the cloned stream is used for speech detection with hark).
In MutedAlert we also make sure to enable the cloned stream's audio
tracks, just in case the user change the device when muted (audio track is
disabled in this case), which also leaves the cloned stream muted (we then
enable the track to allow speech detection).
Closes#12023
Firefox doesn't create a device called 'default' and we were trying
to set this when user is joining the room. We don't do this anymore, letting
devices to be changed when there's some user request.
Moved outputDeviceId inputDeviceId information to be managed in bridge
(just like we do with inputDeviceId), we don't store this duplicated
information in audio container anymore.
Fixed the eslint warning in "playAlertSound(url) { ..."
We are safe to let users try to change input/output devices because the
device list is retrieved from enumerateDevices.
Allow listenonly users to change output devices
Fixed dynamic audio device change for firefox
Fixed shortcuts for audio join/leave
Show (with a bold font) the current selected device
[performance] Prevent calling mediaDevices.enumerateDevices every time we render
the selector. This adds a delay (~200ms, on my chrome setup) to render this component
[performance] Do not call enumerateDevices to search for new devices, instead we listen on mediaDevices.deviceChange event
Small refactoring and fixed a few errors that were being throw in browser's console
Fixed device selection when this is done in audio-settings modal
Fallback to default device when current device is removed
Truncate device name length
Renamed "Input","Output" labels to "Microphone","Speakers", respectively
Update eslint rule for accessKey
The underlying webkit autoplay issues were properly tackled a long time ago now; this thing isnt needed anymore
Also took the liberty to remove the whole create listen only stream thing because it`s useless
Currently this information is lost everytime breakout-room component is
unmounted, causing the panel to shows wrong information during next renders
Fixes#11333
After ending the notification playback, we set the ".src" property to null, which immediately stop the internal player of mobile browser (tested on Chrome for Android - device list is on #11458).
For the specific list of devices, this prevents the internal buffer error "-61" described in #11458.
Fixes#11458.
After audio reconnection, a muted user would have it's microphone unmuted by default, unless muteOnStart is set to true. This fix this problem.
Fixes#9016
Done to avoid false positives where the stream would transition to the unhealthy UI state (or flicker between states) due to disconnected also triggering it, which is not fatal. Disconnected has to be used alongside getStats heuristics, which I removed due to issues with it, so I´m hardening the transition to fatal states only
As explained in #11143, disabling audio filters is desired in some scenarios.
This basically adds an option for user to disable default constraints.
When user doesn't change this value in Settings > Application, the default
value for each audio constraints is retrieved from settings.yml.
When user changes this value in Settings > Application, audio
filters (AGC, Noise Supression and Echo Cancellation) are all set to
true/false, according to the value selected in the Settings GUI.
To start it simple, we decided to not to add a different setting in the GUI for
each audio contraint. This may be added in the future, though (perhaps in a
dedicated Audio Settings tab)
This is related to #4873
As explained in #11143, disabling audio filters is desired in some scenarios.
This basically adds an option for user to disable default constraints.
When user doesn't change this value in Settings > Application, the default
value for each audio constraints is retrieved from settings.yml.
When user changes this value in Settings > Application, audio
filters (AGC, Noise Supression and Echo Cancellation) are all set to
true/false, according to the value selected in the Settings GUI.
To start it simple, we decided to not to add a different setting in the GUI for
each audio contraint. This may be added in the future, though (perhaps in a
dedicated Audio Settings tab)
This is related to #4873
Moved bbb-webrtc-sfu utilitaries to properly named folder
Logging improvements to base broker
Added onerror/onstart/onended callback interfaces to base broker
Extracts most of the common bbb-webrtc-sfu WebSocket setup, handshaking and message broker procedures that was scattered among HTML5 components (video, screenshare and listen only) into a base class suitable for inheritance
When refusing ("thumbs down" button) echo test, user is able to select a different input device. This should work fine for chrome, firefox and safari (once user grants permission when asked by html5client).
For output devices, we depend on setSinkId function, which is enabled by default on current chrome release (2020) but not in Firefox (user needs to enable "setSinkId in about:config page). This implementation is listed as (?) in MDN.
In other words, output device selection should work out of the box for chrome, only.
When selecting an outputDevice, all alert sounds (hangup, screenshare , polling, etc) also goes to the same output device.
This solves #10592
This considerably changes the way we process audio signaling and start audio elements in user's browser.
We now avoid using AudioContext element for both microphone and listenonly calls, once it is unstable for some iOS devices (cracky audio, user stops hearing audio after a while).
Increased default value for listenOnlyCallTimeout: this avoids activating FreeSWITCH's fallback when ICE negotiation takes longer than 15sec (tested on DO).
Increased listenonly logs.
This fixes#8133#10388
If an ejected user tries to enter in the meeting using the current url
html5 client keep trying to validate that user, but without success
causing a validateAuthToken message spam until the connection times out.
The mic mute is done client side via the track`s enabled flag, which generates silent when false. This still tracks the server/freeswitch mute state, so server-side mutes will be reproduced in the client
We noticed that the mute action button wasn't being consistent when
using muteOnStart as meeting configuration.
Included the addition of the voice user object to the collection as
an event to also be observed.
