There are still a bunch of edge cases and issues with reconnection
scenarios for video:
- Signaling socket refuses to reconnect once maxRetries expire
- Race conditions on local stream attachment: local camera wouldn't be
correctly rendered _if_ the attached stream existed _without_ video
tracks yet
- Video tracks leak on local streams when replacing them (virtual bgs)
- Completely ignoring Meteor state when trying to reconnect cameras
- Streams aren't proactively stopped when the signaling socket dies
- Outbound request queues aren't isolated by stream nor are they
flushed when a newer peer with the same ID is created
- Server originated negotiation errors won't trigger a local peer
cleanup - thus leaving dangling peers that take way too long to
reconnect
This commit fixes or improves all of the aforementioned issues, +:
- Remove unused arguments in the peer (client->SFU) 'start' request
- Prevent crashes when trying to render video-list-items without user
data (which might happen on re-connections)
Reconnection timers are far too long for abrupt failures because we
are waiting the original timeouts to elapse (30-60s) before trying it
again - even if a connection worked N-sessions back in that session's
history. The ideal thing to have is another intermediate, smaller and
fixed reconnection timer for sessions that had a working screen share
at least once.
The UI is also not being updated to the reconnecting state on negotiation
failures.
* Add an intermediate reconnection timer for abrupt failures set to 8s.
This should improve reconnection times.
* Lower default connection timers values (base 20s down from 30s, max
25s down from 60s)
* Set screen share UI to reconnecting on abrupt failures as well - we
were only tracking ICE states prior to this, not negotiation errors
video-provider's current ping-pong is as good as nothing in 2.5+. We
were counting on Meteor's (and consequently the component's mount state)
before 2.5 to act as a "heartbeat" as far as the socket is concerned.
The ping-pong served only to sustain traffic for finnicky,
traffic-dependant firewall.
Since 2.5, the component's state is _kind of_ detached from Meteor's -
which means it won't unmount when Meteor disconnects. That causes the
video-provider websocket to lose its borrowed heartbeat and leads to a
bunch of reconnectiong inconsistencies, the worst of them being a stuck,
useless signaling socket that will cause cameras not to work until a
client refresh.
This commit does the following:
- Implements actual heartbeat checks to trigger signaling socket
reconnects when necessary, all within the scope of video-provider
- Remove borked, eons old 'offline'/'online' event handlers: they were
causing unnecessary camera drops AND causing video-provider to
generate a stuck signaling socket
- Properly catch WebSockets.send errors
There may be other bridges may not need to force relay traffic on
firefox as @prlanzarin pointed out. So set the default to a
configuration that works out of the box but leave other choices for the
operator.
The option is moved from kurento namespace to media next to the general
forceRelay option.
Firefox has a buggy ICE implementation and needs WebRTC media traffic to
be routed through a turn server to work reliably with mediasoup.
Use the information fetched by the STUN API to determine if the operator
has configured a turn server. If there is one force firefox to use it.
Closes#16164
When BBB is run as a single node or in a scaleout setup with a cluster
proxy in front (see https://docs.bigbluebutton.org/admin/clusterproxy.html) it
is useful to store client settings in browser localStorage instead of
sessionStorage. If localStorage is configured then the client will keep seetings
like notifications for user joining, chat etc across meetings.
It is not advisable to set the setting to `local` in a setup of multiple BBB
nodes without a cluster proxy in front of it because this would lead to
unexpected behaviour at users. The browser would store settings for each server
and for users it would look like BBB is sometimes store the settings and
sometimes not.
It adds the new setting
```yaml
public:
app:
userSettingsStorage: (session|local)
```
RTCRTPSender exposes DSCP marking via `networkPriority` in the encodings
configuration dictionaries. That should allow us to control
QoS priorities for different media streams, eg audio with higher network
priority than video. The only browser that implements that right
now is Chromium.
To use this, the public.app.media.networkPriorities configuration in
settings.yml. Audio, camera and screenshare priorities can be controlled
separately. For further info on the possible values, see:
- https://www.w3.org/TR/webrtc-priority/
- https://datatracker.ietf.org/doc/html/rfc8837#section-5
Mobile endpoints are flaky with the WebSpeechAPI:
- iOS versions that support it are borking our outbound audio when it's
enabled
- Android speech recognition has flaky locale detection and speech
transcription
Additionally: the support check is not checking the WebSpeechAPI
availability properly, so older devices (eg iOS 12) are flagged as
supported even though they aren't.
This commit adds a configuration flag (public.audioCaptions.mobile) to
control transcription availability on mobile. False by default.
Also extends the setSpeechVoices support check and
hasSpeechRecognitionSupport method to prevent false positives.
Adds two new flags to the settings file which change the way the locale
flag is used:
- forceLocale: (true/false) => If true, enforces the transcription
language to be the locale content field and jumps the language
selector
in audio modal.
- defaultSelectLocale: (true/false) => If true, the default selected
value in the dropdown language selector in audio modal will be defined
by the locale content field.
In any case, if the locale flag holds an invalid value, it defaults to
disabled.
Move the language collection to the HTML settings file. This data defines
the available languages available for the speech API.
These language tags are used to filter SpeechSynthesis' API `getVoices`
result. Tags must use BCP 47 format.
https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesisVoice/lang
FreeSWITCH has mDNS resolution capabilities as of 1.10.7. Having the filtering
configurable in the client allows us to field trial whether we should keep that
on or off. The default is still to filter them out because FreeSWITCH does not
resolve mDNS candidates by default (ice_resolve_candidate in switch.conf.xml).
- Remove the old listen only bridge (kurento.js), superseded by the equivalent
and equally stable (AS FAR AS LISTEN ONLY IS CONCERNED) sfu-audio-bridge
- Rename FullAudioBridge.js -> sfu-audio-bridge.js
* A more generic name that better represents the capabilities and
the nature of the bridge
* The bridge name identifier in configuration is still the same
('fullaudio')
- Remove the FreeSWITCH listen only fallback
- Temporarily disable the "trickle ICE" pair gathering feature used
in SIP.js (which was always experimental, nonstandard and disabled
by default)
- Updates to settings.yml keys in places where relevant
mediasoup is the default media server in v2.5, so we don't need ICE candidates
to be signaled to bbb-webrtc-sfu. Disabling it saves resources (client and server).
public.media.showVolumeMeterInSettings => public.media.showVolumeMeter
public.media.simplifiedEchoTest => public.media.localEchoTest.enabled
Initial hearing state can be configured in public.media.localEchoTest.initialHearingState
New features:
- A simplified echo test mode that only does a local loopback (instead of
going to FS and back)
- A volume meter for microphone streams to the AudioSettings view
Those two features are experimental and disabled by default; see
public.app.media.simplifiedEchoTest and public.app.media.showVolumeMeter configs
Collateral changes:
- fix: localize fallback device strings in AudioSettings/DeviceSelector
- Refactor on some media stream utils to be re-usable across components
- Refactor in AudioSettings to keep gUM #uses stable.
* TODO: need to pass streams through AudioManager to avoid the surplus gUM.
- fix(audio): drop ScriptProcessorNode usage (deprecated)
* Used in volume meter for tracking - use hark instead
Rationale: the important thing here is bitrate. Disabling constraints should have no meaningful on 1) client-side bw 2) client-side cpu 3) server-side bw/cpu - while it will also guarantee seemingly smoother streams
Tries to mitigate too-rapidly-switching camera profiles causing video freezes
due to encoder resets. Excluding constraints might not help a lot since
the thing that actually restarts the encoder is the bitrate change, but
they're not really important in the context of dynamic profiles.
We can't get rid of bitrate changes, though, since it's what does the actual
quality constraining.
The camera profile change debounce timer is 2.5s by default (which is
the same timer used for floor changes).
Also fixed an issue with camera profile backfiring due to badly defined peers
Set height to 720 in those profiles so that aspect ratio has a better chance to
be cohesive in all environments. Having only width set was causing certain
devices to output videos with non-standard ARs
Set height to 720 in those profiles so that aspect ratio has a better chance to
be cohesive in all environments. Having only width set was causing certain
devices to output videos with non-standard ARs
Since bbb-pads the html client Etherpad redis' channel subscription isn't
needed anymore. All communication between Etherpad and BigBlueButton goes
through bbb-pads.
Move all Etherpad's access control from Meteor to a separated [Node application](https://github.com/bigbluebutton/bbb-pads).
This new app uses [Etherpad's API](https://etherpad.org/doc/v1.8.4/#index_overview)
to create groups and manage session tokens for users to access them. Each group
represents one distinct pad at the html5 client.
- Removed locked users' access to pads: replaced readOnly pad's access with a new pad's content sharing routine
- Pad's access is now controlled by [Etherpad's API](https://etherpad.org/doc/v1.8.4/#index_overview)
- Closed captions edited content now reflects at it's live feedback
- Improved closed caption's dictation mode live feedback
- Moved all Etherpad's API control from Meteor to a separated [app](https://github.com/bigbluebutton/bbb-pads)
- Included access control both in akka-apps and bbb-pads
I have growing concerns about gain node`s effect on audio quality the way it
was implemented, so I opted to fall back to HTMLMediaElement`s volume control
for the time being until we can gauge quality impacts properly later on
Add a new configuration flag enableVolumeControl, false by default while the
feature undergoes a field trial
- forceRelayOnFirefox: whether TURN/relay usage should be forced to work
around Firefox's lack of support for regular nomination when dealing with
ICE-litee peers (e.g.: mediasoup).
* See: https://bugzilla.mozilla.org/show_bug.cgi?id=1034964
- iOS endpoints are ignored from the trigger because _all_ iOS browsers
are either native WebKit or WKWebView based (so they shouldn't be affected)
Add the <ButtonEmoji/> to the webcam button. Now it
is possible to change the webcam settings and change
your settings without having to turn off sharing.
Modifies the <ButtonEmoji/> component so that it can
receive an onClick per props.
It is possible to disable this feat in the settings.yml
The idea is to run a loadbalancer node which maps each BBB node to a
path. That way each user gets only one gUM permission query for a
cluster. The loadbalancer node only serves the html5 client, each BBB
node will serve its own API and handle the websockets for freeswitch and
bbb-webrtc-sfu.
Configuring a cluster setup
===========================
* let bbb-lb.example.com be the loadbalancer node
* let bbb-01.eaxmple.com be a BBB node
Loadbalancer
------------
On the loadbalancer node add an nginx configuration similar to this one
for each BBB node:
```
location /bbb-01/html5client/ {
proxy_pass https://bbb-01.example.com/bbb-01/html5client/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
```
BBB Node
--------
On the BBB node add the following options to
`/etc/bigbluebutton/bbb-web.properties`:
```
defaultHTML5ClientUrl=https://bbb-lb.example.com/bbb-01/html5client/join
presentationBaseURL=https://bbb-01.example.com/bigbluebutton/presentation
accessControlAllowOrigin=https://bbb-lb.example.com
```
Add the following options to `/etc/bigbluebutton/bbb-html5.yml`:
```
public:
app:
basename: '/bbb-01/html5client'
bbbWebBase: 'https://bbb-01.eaxmple.com/bigbluebutton'
learningDashboardBase: 'https://bbb-01.eaxmple.com/learning-dashboard'
media:
stunTurnServersFetchAddress: 'https://bbb-01.eaxmple.com/bigbluebutton/api/stuns'
sip_ws_host: 'bbb-01.eaxmple.com'
presentation:
uploadEndpoint: 'https://bbb-01.eaxmple.com/bigbluebutton/presentation/upload'
```
Create the following unit file overrides:
* `/etc/systemd/system/bbb-html5-frontend@.service.d/cluster.conf`
* `/etc/systemd/system/bbb-html5-backend@.service.d/cluster.conf`
with the following content:
```
[Service]
Environment=ROOT_URL=https://127.0.0.1/bbb-01/html5client
```
Change the nginx `$bbb_loadbalancer_node` variable to the name of the
load balancer node in `/etc/bigbluebutton/nginx/loadbalancer.nginx` to
allow CORS requests:
```
set $bbb_loadbalancer_node https://bbb-lb.example.com
```
Prepend the mount point of bbb-html5 in all location sections except
from the `location @html5client` section in
`/etc/bigbluebutton/nginx/bbb-html5.nginx`
```
location @html5client {
...
}
location /bbb-01/html5client/locales {
...
}
```
Previously we were using the same for listenonly, which for default
environments points to kurento, and didn't make any difference in media server
selection.
This could be problematic in those environments where meta param
for listeonly media server is set , though.
Fullaudio has now it's own meta param that can be passed through API:
"meta_media-server-fullaudio"
Adds prometheus client to collect html5 server metrics.
Only default Node.js metrics in this initial version.
Enable via configs in private.app.prometheus
ICE lite servers (eg mediasoup) dont need candidates signaled out-of-band; neither does KMS in certain scenarios
Disable their signaling saves us some ticks in bbb-webrtc-sfu and some bandwidth all around
Undefined by default means that the governing configuration is in bbb-webrtc-sfu
Also add some inline docs in settings.yml about the media server adapter configs
Changes (maybe not a complete list):
- Disable virtualbgs by default
- Move the virtualbg selector in video-preview to the side below the
profile selection
- Restore old video-preview sizes
- Add a wrapper class for MediaStreams (BBBVideoStream)
- Centralize virtualbg services and business logic code into BBBVideoStream
- Refactor and centralize virtualbg constant fetching
- Refactor and centralize virtualbg config fetching
- Organize virtualbg type definitions
- Remove added states in video-provider to prevent further bloat
- Remove added states in video-preview to prevent further bloat
- Lock virtual bg switching while video-preview itself is locked
- Add proper virtualbg error surfacing via toasts
- Refactor iOS availability detection to use centralized UA checker
- Avoid calling gUM when toggling virtualbgs on/off
- Make virtualbg video-list-item action a toggle instead of a
state-aware action
- Make virtualbg switching work in video-preview for cameras that are
already shared. Especially useful when there are multiple source
cameras, and will be important in the near future
- Add Derivative Work notices in files that are partially copied from
jitsi-meet
- Simplify track replacing in video-provider
- Split video-preview UI code for virtualbgs into a separate functional component
Remove parts of a previous connection monitor.
To add some context (as far as my memory goes) to the multiple connection
monitor features the product has, `stats` (currently named `connection status`)
was introduced at the Flash client back in ~2016. @fcecagno and I did it
as a BigBlueButton's Summit activity. Our work was squashed into a single
commit in 92554f8b3e :).
I'm not sure about the whole story behind `network information` (the late
connection monitor added to the HTML5 client) but I assume it should work
as a collector for a bunch of different connectivity monitors. I remember
when it was introduced but I don't know why it wasn't adopted. My best guess
would be because of some performance issues the `user list` had back then.
To follow on why `connection status` replaced `network information` at the
HTML5 client, when I did the `multiple webcams` feature I had to refactor
a big chunk of the `video provider` (#8374). Something that wasn't really
helping there was the adaptation of `stats` that was made to show local
feedback for each webcam connection. Although this feature wasn't being
used anymore, `network information` did rely on that to build up data. With
this monitor gone I assumed it was my responsibility to provide an alternative
so I promoted Mconf's port of the Flash `stats` monitor to BigBlueButton's
HTML5 client (#8579).
Well, that's my perspective on how things went for those features. If
anyone would like to correct me on something or add something else on
that history I would appreciate to know.
Add the `camera` icon in the user list for whoever is sharing,
in order to improve the understanding of who is sharing the webcam.
It is possible to enable/disable this indication in the settings.yml
This allows showing (or hiding) the option from users in the conference.
Default value is true (show this option to users).
Fixed some eslint warnings.
When listenOnlyMode=false, skipCheck=true and skipCheckOnJoin=true, the
audio tries to start a session more than one time, causing it to fail
at the first one (and reconnect after that).
Now we check if user is already connecting before trying to start a new
audio session.
Added some info in settings.yml for the options related to this commit
Closes#12190
