The current Vosk CC provider does not support stereo mic streams
(pending investigation as to why).
This commits makes sure stereo is forcefully disabled via SDP munging
only when transcription is active and using Vosk. Having it disabled
in the server side (FreeSWITCH) is not enough because the stereo parameter
is client mandated and replicated by FS on its answer. So we need to
make sure it's always disabled for the time being.
SFU audio does munging server side (and stereo is always off), so no changes
needed there.
The rest of the providers (except WebSpeech) need to be validated against
stereo audio as well.
This is also intended to be temporary - ideally this needs to be fixed in
mod_audio_fork/Vosk/wherever this is breaking.
Audio state callback and remote media setup both depend on FS's state
(comes through Meteor) and the ICE state (local, peer connection). The
caveat: FS's state can come delayed on reconnection scenarios because
Meteor's websocket generally takes significantly longer to re-connect than
the peer connection, which means the ICE state gets completed way before FS
is flagged as ready.
The practical issue: while outbound audio (client -> FS) will work, inbound
audio (FS -> client) won't _just because it wasn't played_ (even though
data is coming through).
This commit decouples the remote media setup step from the state
through:
- Setup remote media when ICE state is completed
- Run the state callback only after FS is flagged as ready. This
should maintain the UI states consistent across client-server.
Keep in mind the assumption that if FS is ready, ICE is completed by
consequence.
Mostly benign, but exitAudio/forceExitAudio was throwing an unhandled
error when called on sessions with no active audio because the
underlying bridge methods did not check whether there was an active
session to stop beforehand.
There are some situations where previously set deviceIds (
local/session storage) may become stale. This causes an unexpected
behavior where audio is temporarily borked until the user clears their
local storage.
This issue has been seen more recently on Safari endpoints when switching
back-and-forth breakout rooms in environments running under iframes.
Also seen randomly on endpoints with virtual input devices.
This centralizes audio gUM calling into a single method that retries the
gUM procedure without pre-set deviceIds only if the initial call fails
due with an OverconstrainedError - hopefully circumventing the issue.
There's no rollback procedure in case a device switch fails right now,
nor does the code entrypoints that call the switching procedures wait
for resolution or failure before marking the new device as chosen. That
may cause inconsistent states in a couple of ways:
- No rollback: switch fails, audio is still on but no actual
microphone input is being transmitted
- Not waiting for resolutions: inconsistent chosen devices on failures
Device switching errors are also not surfaced to the end user
This commit:
- Adds device rollback and proper resolution/failure response
awaits to try and make the state a bit more consistent.
- Centralizes the input device switching code to be reused between
different bridges
- Centralizes device ID state management in audio-manager to try and
mantain them a bit more consistent across the board
- Surface device switching failures to the end user
- Guarantee device IDs are set to the session storage on all
appropriate scenarios
Sometimes the handler that listens for the state change in the callState is
not updated correctly.
In these rare cases, the state of the callstate changes directly to in_conference,
not taking the expected path: call_started -> in_echo_test -> in_conference
FreeSWITCH has mDNS resolution capabilities as of 1.10.7. Having the filtering
configurable in the client allows us to field trial whether we should keep that
on or off. The default is still to filter them out because FreeSWITCH does not
resolve mDNS candidates by default (ice_resolve_candidate in switch.conf.xml).
"default" is not an universally valid default value for deviceIds which was causing issues with Firefox and Safari in some specific scenarios where exact deviceId constraints were being used
The idea is to run a loadbalancer node which maps each BBB node to a
path. That way each user gets only one gUM permission query for a
cluster. The loadbalancer node only serves the html5 client, each BBB
node will serve its own API and handle the websockets for freeswitch and
bbb-webrtc-sfu.
Configuring a cluster setup
===========================
* let bbb-lb.example.com be the loadbalancer node
* let bbb-01.eaxmple.com be a BBB node
Loadbalancer
------------
On the loadbalancer node add an nginx configuration similar to this one
for each BBB node:
```
location /bbb-01/html5client/ {
proxy_pass https://bbb-01.example.com/bbb-01/html5client/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
```
BBB Node
--------
On the BBB node add the following options to
`/etc/bigbluebutton/bbb-web.properties`:
```
defaultHTML5ClientUrl=https://bbb-lb.example.com/bbb-01/html5client/join
presentationBaseURL=https://bbb-01.example.com/bigbluebutton/presentation
accessControlAllowOrigin=https://bbb-lb.example.com
```
Add the following options to `/etc/bigbluebutton/bbb-html5.yml`:
```
public:
app:
basename: '/bbb-01/html5client'
bbbWebBase: 'https://bbb-01.eaxmple.com/bigbluebutton'
learningDashboardBase: 'https://bbb-01.eaxmple.com/learning-dashboard'
media:
stunTurnServersFetchAddress: 'https://bbb-01.eaxmple.com/bigbluebutton/api/stuns'
sip_ws_host: 'bbb-01.eaxmple.com'
presentation:
uploadEndpoint: 'https://bbb-01.eaxmple.com/bigbluebutton/presentation/upload'
```
Create the following unit file overrides:
* `/etc/systemd/system/bbb-html5-frontend@.service.d/cluster.conf`
* `/etc/systemd/system/bbb-html5-backend@.service.d/cluster.conf`
with the following content:
```
[Service]
Environment=ROOT_URL=https://127.0.0.1/bbb-01/html5client
```
Change the nginx `$bbb_loadbalancer_node` variable to the name of the
load balancer node in `/etc/bigbluebutton/nginx/loadbalancer.nginx` to
allow CORS requests:
```
set $bbb_loadbalancer_node https://bbb-lb.example.com
```
Prepend the mount point of bbb-html5 in all location sections except
from the `location @html5client` section in
`/etc/bigbluebutton/nginx/bbb-html5.nginx`
```
location @html5client {
...
}
location /bbb-01/html5client/locales {
...
}
```
Restored the old behavior when ending breakout rooms while user is in the
breakout audio transfer, which is to the trigger the reconnection to the audio
in the main room.
This behavior could be improved by (instead of reconnecting) transfering user
back to the main room, but this requires some changes in akka-apps/fsesl
which can be treated in a different issue.
Closes#13242
We now let audio-manager log errors, using the specified error code.
These errors are still logged in bridge layer, but as a warning.
This commit doens't change error codes behavior, they are still being
logged as errors and with the same code numbers.
Here's what we do when user activates mic:
1 - When we do something similar to listenonly's joining process
until we find a valid candidate-pair. The information about this
local candidate is store.
2 - We then start a new userAgent, and as soon as browser finds
a candidate with the same local ip address, we leave only this
candidate in the SDP and send this to FreeSWITCH. SDP should
contain only a single candidate.
3 - The rest of signaling process is basically the same.
When joining breakout audio, the output device selected in the main room is
used in breakout.
When returning from breakout rooms, the output audio device previously set in
the main room is restored.
Some specific info:
SIPSession doesn't handle Storage anymore, we do this in SIPBridge, since
it has more info about the current selected device and it doesn't depend of
a session being oppened.
We also now pass the output device ID when joining audio sessions. We can
then keep this information in the Storage.
Closes#11663
This commit contains three fixes: one already reported and two detected
during the investigation of the solution.
This started as a fix for firefox (#12023), but i also fixed the muted
alert/banner when device changes: the banner wasn't detecting device changes,
unless audio was deactived/actived.
There's another fix for the microphone stream: we now keep sender's track
disabled if it was already disabled for the sender's track of the previous
selected device.
Also did small refactor for eslint checking.
Some technical information: in sip bridge (bridge/sip.js), setInputStream and
liveChangeInputDevice function were both fully turned into promises, which
guarantees we have everything ready when it resolves to the respective values.
This helps AudioManager (audio-manager/index.js) to sequentially sets and
tracks the state of the current microphone stream (inputStream), when calling
liveChangeInputDevice function: we first set the current stream to null,
creats a new one and then set it to the newly created value - this is needed
because MutedAlert (muted-alert/component.jsx) can then gracefully
allocate/deallocate the cloned stream when it is set to a non-null/null value
(the cloned stream is used for speech detection with hark).
In MutedAlert we also make sure to enable the cloned stream's audio
tracks, just in case the user change the device when muted (audio track is
disabled in this case), which also leaves the cloned stream muted (we then
enable the track to allow speech detection).
Closes#12023
Firefox doesn't create a device called 'default' and we were trying
to set this when user is joining the room. We don't do this anymore, letting
devices to be changed when there's some user request.
Moved outputDeviceId inputDeviceId information to be managed in bridge
(just like we do with inputDeviceId), we don't store this duplicated
information in audio container anymore.
Fixed the eslint warning in "playAlertSound(url) { ..."
We are safe to let users try to change input/output devices because the
device list is retrieved from enumerateDevices.
Allow listenonly users to change output devices
Fixed dynamic audio device change for firefox
Fixed shortcuts for audio join/leave
Show (with a bold font) the current selected device
[performance] Prevent calling mediaDevices.enumerateDevices every time we render
the selector. This adds a delay (~200ms, on my chrome setup) to render this component
[performance] Do not call enumerateDevices to search for new devices, instead we listen on mediaDevices.deviceChange event
Small refactoring and fixed a few errors that were being throw in browser's console
Fixed device selection when this is done in audio-settings modal
Fallback to default device when current device is removed
Truncate device name length
Renamed "Input","Output" labels to "Microphone","Speakers", respectively
Update eslint rule for accessKey
Without 'exact' match, the browser fallbacks to the default inputDeviceId
This prevents the error (input device error) when breakout is ended and we try
to skipCheck the microphone when user returns to main room (assuming the
user had the microphone active before joining breakout room).
Without 'exact' match, the browser fallbacks to the default inputDeviceId
This prevents the error (input device error) when breakout is ended and we try
to skipCheck the microphone when user returns to main room (assuming the
user had the microphone active before joining breakout room). Related
to the feature c451666d52
As explained in #11143, disabling audio filters is desired in some scenarios.
This basically adds an option for user to disable default constraints.
When user doesn't change this value in Settings > Application, the default
value for each audio constraints is retrieved from settings.yml.
When user changes this value in Settings > Application, audio
filters (AGC, Noise Supression and Echo Cancellation) are all set to
true/false, according to the value selected in the Settings GUI.
To start it simple, we decided to not to add a different setting in the GUI for
each audio contraint. This may be added in the future, though (perhaps in a
dedicated Audio Settings tab)
This is related to #4873
