This is an initial, experimental implementation of the feature proposed in
https://github.com/bigbluebutton/bigbluebutton/issues/14021.
The intention is to phase out the explicit listen only mode with two
overarching goals:
- Reduce UX friction and increase familiarity: the existence of a separate
listen only mode is a source of confusion for the majority of users
Reduce average server-side CPU usage while also making it possible for
having full audio-only meetings.
The proof-of-concept works based on the assumption that a "many
concurrent active talkers" scenario is both rare and not useful. With
that in mind, this including two server-side triggers:
- On microphone inactivity (currently mute action that is sustained for
4 seconds, configurable): FreeSWITCH channels are held (which translates
to much lower CPU usage, virtually 0%). Receiving channels are switched,
server side, to a listening mode (SFU, mediasoup).
* This required an extension to mediasoup two allow re-assigning producers
to already established consumers. No re-negotiation is done.
- On microphone activity (currently unmute action, immediate):
FreeSWITCH channels are unheld, listening mode is deactivated and the
mute state is updated accordingly (in this order).
This is *off by default*. It needs to be enabled in two places:
- `/etc/bigbluebutton/bbb-webrtc-sfu/production.yml` ->
`transparentListenOnly: true`
- End users:
* Server wide: `/etc/bigbluebutton/bbb-html5.yml` ->
`public.media.transparentListenOnly: true`
* Per user: `userdata-bbb_transparent_listen_only=true`
In peer.js offer generation method, the provided constraitns are only
being type checked. It needs a value check when the constraint is a
boolean.
This commit should prevent useless transceivers from being added when
explictly specifying audio/video as `false`
There's an edge case in finnicky networks where ALG-like firewalls
tamper with USE-CANDIDATE STUN packets and, consequently, bork ICE-lite
connectivity establishment. The odd part is that client-side gathering
seems to complete if intermediate STUN bindings work (before the final
USE-CANDIDATE), which may cause the peer not to generate relay
candidates == connectivity fails.
This adds the `public.kurento.gatheringTimeout` option to forcefully extend
the candidate gathering window in peers that act as offerers. The
behavior is as follows: if the flag is set (ms), the peer will wait
either the gathering completed stage or, _at most_,
public.kurento.gatheringTimeout ms before proceeding with calls chained
to setLocalDescription.
This option is disabled by default and intentionally ommited from the
base settings.yml file as to not encourage its use. Don't use it unless
you know what you're doing :).
There are still a bunch of edge cases and issues with reconnection
scenarios for video:
- Signaling socket refuses to reconnect once maxRetries expire
- Race conditions on local stream attachment: local camera wouldn't be
correctly rendered _if_ the attached stream existed _without_ video
tracks yet
- Video tracks leak on local streams when replacing them (virtual bgs)
- Completely ignoring Meteor state when trying to reconnect cameras
- Streams aren't proactively stopped when the signaling socket dies
- Outbound request queues aren't isolated by stream nor are they
flushed when a newer peer with the same ID is created
- Server originated negotiation errors won't trigger a local peer
cleanup - thus leaving dangling peers that take way too long to
reconnect
This commit fixes or improves all of the aforementioned issues, +:
- Remove unused arguments in the peer (client->SFU) 'start' request
- Prevent crashes when trying to render video-list-items without user
data (which might happen on re-connections)
BBBVideoStream 'inactive' event currently does not send the
MediaStream id as the payload for handlers. This can cause theoretical
race conditions due to media stream mismatches in places where
'inactive' handling is done globally (eg video-preview).
This adds the MediaStream id to BBBVideoStream's 'inactive' event
and uses it in video-preview to avoid such race conditions.
RTCRTPSender exposes DSCP marking via `networkPriority` in the encodings
configuration dictionaries. That should allow us to control
QoS priorities for different media streams, eg audio with higher network
priority than video. The only browser that implements that right
now is Chromium.
To use this, the public.app.media.networkPriorities configuration in
settings.yml. Audio, camera and screenshare priorities can be controlled
separately. For further info on the possible values, see:
- https://www.w3.org/TR/webrtc-priority/
- https://datatracker.ietf.org/doc/html/rfc8837#section-5
Firefox doesn't fire the ended evt/onended callback for live
MediaStreamTrack(s). We rely on that event.
Manually emit the ended event which works with the onended callback
when a track is stopped
kurento-utils is unmaintained. It's served us well, but its age
shows. We need to transition to something else if we want to
have better maintainability and include simulcast, multistream, ...
This introduces a simplified/leaner wrapper kit that's almost
API-compatible with what we use right now - so widespread changes
are minimal). It's easier to maintain/read/transition from. This
can be read as an intermediate step to transitioning to
something definitive (ie mediasoup-client).
Changes (maybe not a complete list):
- Disable virtualbgs by default
- Move the virtualbg selector in video-preview to the side below the
profile selection
- Restore old video-preview sizes
- Add a wrapper class for MediaStreams (BBBVideoStream)
- Centralize virtualbg services and business logic code into BBBVideoStream
- Refactor and centralize virtualbg constant fetching
- Refactor and centralize virtualbg config fetching
- Organize virtualbg type definitions
- Remove added states in video-provider to prevent further bloat
- Remove added states in video-preview to prevent further bloat
- Lock virtual bg switching while video-preview itself is locked
- Add proper virtualbg error surfacing via toasts
- Refactor iOS availability detection to use centralized UA checker
- Avoid calling gUM when toggling virtualbgs on/off
- Make virtualbg video-list-item action a toggle instead of a
state-aware action
- Make virtualbg switching work in video-preview for cameras that are
already shared. Especially useful when there are multiple source
cameras, and will be important in the near future
- Add Derivative Work notices in files that are partially copied from
jitsi-meet
- Simplify track replacing in video-provider
- Split video-preview UI code for virtualbgs into a separate functional component
