Commit 325887e325 split the local echo audio
element from the main audio element to allow concurrent playback without the
risk of interfering with one another.
This introduced a regression where local echo doesn't track output device
changes. The main audio element (i.e. the meeting's audio) is not affected by
this regression.
This commit ensures local echo reacts to output device changes as needed.
UI team suggested a few adjustments to the audio settings modal:
- Larger (24px/1.5rem) margin between content and headers
- Rephrasing of modal title, subtitle and volume indicator label
- Change the "audio feedback" button to an outline or link styled
button (there are currently two primary buttons and we want users to
focus on the "Join audio" one)
Implement the suggested changes. The approach for the audio feedback
button is link-styled.
When going from "no mic" -> mic via the unmute action, the client isn't
unmuting itself after confirming the change. This is caused by not
waiting the liveChangeInputDevice method (which is a Promise) to be
fully executed before unmounting the AudioSettings modal -- the one
responsible for triggering the unmute. Since it unmounts before the
device is changed, the unmute action will be ignored because the device
is still "listen-only" (no mic).
Properly unmute audio when transitioning from "no mic" -> "mic" via the
unmute trigger by waiting for liveChangeInputDevice to resolve.
Additionally, some general improvements to UI/UX:
- Display the AudioSettings modal title when gUM is on prompt mode
- Add specific subtitles to the AudioSettings modal to 1) warn that no
mic is selected 2) Give a hint that the user can test their devices
- Always honor settings.yml's "initialHearingState" state (whether
local echo feedback should be played by default in AudioSettings)
We are missing a way to select transcription languages in some
scenarios, e.g.: listenOnlyMode=false. The audio settings UI is also not
handling item disposition very well on smaller devices.
This commit does the following to improve those blind spots:
- Add the transcription language selector to it whenever applicable
- Add proper styling to the transcription selector
- Handle small screens by changing the disposition of elements to
portrait mode
- Improve how elements are disposed to a more familiar view: Mic ->
Activity Indicator; Speaker -> Speaker test. This is more in line
with how other platforms do audio configuration/pre flight screens.
This is a rework of the audio join procedure whithout the explict listen
only separation in mind. It's supposed to be used in conjunction with
the transparent listen only feature so that the distinction between
modes is seamless with minimal server-side impact. An abridged list of
changes:
- Let the user pick no input device when joining microphone while
allowing them to set an input device on the fly later on
- Give the user the option to join audio with no input device whenever
we fail to obtain input devices, with the option to try re-enabling
them on the fly later on
- Add the option to open the audio settings modal (echo test et al)
via the in-call device selection chevron
- Rework the SFU audio bridge and its services to support
adding/removing tracks on the fly without renegotiation
- Rework the SFU audio bridge and its services to support a new peer
role called "passive-sendrecv". That role is used by dupled peers
that have no active input source on start, but might have one later
on.
- Remove stale PermissionsOverlay component from the audio modal
- Rework how permission errors are detected using the Permissions API
- Rework the local echo test so that it uses a separate media tag
rather than the remote
- Add new, separate dialplans that mute/hold FreeSWITCH channels on
hold based on UA strings. This is orchestrated server-side via
webrtc-sfu and akka-apps. The basic difference here is that channels
now join in their desired state rather than waiting for client side
observers to sync the state up. It also mitigates transparent listen
only performance edge cases on multiple audio channels joining at
the same time.
The old, decoupled listen only mode is still present in code while we
validate this new approach. To test this, transparentListenOnly
must be enabled and listen only mode must be disable on audio join so
that the user skips straight through microphone join.
Output device changes aren't working in 2.6's echo test when artifical delay
is on due to the fact that the feedback audio is being played via the WebAudio
context rather the the HTMLMediaElement. Since output device change works
via HTMLMediaElement's setSinkId, it's basically a no-op.
This commit fixes the issue by piping the AudioContext destination
through the main audio element rather than using WebAudio directly for
playback. An additional stub media element (muted) is added to circumvent one
of Chrome's WebAudio issue.
The alternative would be to use AudioContext's setSinkId, but it isn't
supported by Firefox (setSinkId enabled) and Chrome < 110.
This should work with FF (setSinkId enabled) and a wide array of Chromium
versions.
If BBB 2.6 is used without headphones, the audio test works differently
than in 2.5. In 2.5 audio traffic is routed to freeswitch and then
returned to the browser. This adds usually some latency which makes it
easy to hear you audio quality. In 2.6 there is a local loopback. As
there is almost no latency, it is either difficult or even impossible to
check your own audio quality as echo cancellation of the browser will
filter out your own signal.
This patch adds a delay node to the audio loopback test, which makes is
easier to check your quality.
public.media.showVolumeMeterInSettings => public.media.showVolumeMeter
public.media.simplifiedEchoTest => public.media.localEchoTest.enabled
Initial hearing state can be configured in public.media.localEchoTest.initialHearingState
New features:
- A simplified echo test mode that only does a local loopback (instead of
going to FS and back)
- A volume meter for microphone streams to the AudioSettings view
Those two features are experimental and disabled by default; see
public.app.media.simplifiedEchoTest and public.app.media.showVolumeMeter configs
Collateral changes:
- fix: localize fallback device strings in AudioSettings/DeviceSelector
- Refactor on some media stream utils to be re-usable across components
- Refactor in AudioSettings to keep gUM #uses stable.
* TODO: need to pass streams through AudioManager to avoid the surplus gUM.
- fix(audio): drop ScriptProcessorNode usage (deprecated)
* Used in volume meter for tracking - use hark instead
