I/O device IDs are logged in some specific logCodes, but they aren't too
useful on their own without the rest of the MediaDeviceInfo object. We
need that extra data (label, group) to be able to better investigate
incorrect device issues and NotFoundError occurrences.
Register full I/O device info whenever the client fetches them and add
those, unfiltered, to the following logCodes:
- audiomanager_error_getting_device
- audiomanager_error_device_not_found
- audiomanager_error_unknown
- audio_joined
- audio_ended
- audio_failure
- audiomanager_input_live_device_change_failure
- audiomanager_output_device_change_failure
We are missing a way to select transcription languages in some
scenarios, e.g.: listenOnlyMode=false. The audio settings UI is also not
handling item disposition very well on smaller devices.
This commit does the following to improve those blind spots:
- Add the transcription language selector to it whenever applicable
- Add proper styling to the transcription selector
- Handle small screens by changing the disposition of elements to
portrait mode
- Improve how elements are disposed to a more familiar view: Mic ->
Activity Indicator; Speaker -> Speaker test. This is more in line
with how other platforms do audio configuration/pre flight screens.
This is a rework of the audio join procedure whithout the explict listen
only separation in mind. It's supposed to be used in conjunction with
the transparent listen only feature so that the distinction between
modes is seamless with minimal server-side impact. An abridged list of
changes:
- Let the user pick no input device when joining microphone while
allowing them to set an input device on the fly later on
- Give the user the option to join audio with no input device whenever
we fail to obtain input devices, with the option to try re-enabling
them on the fly later on
- Add the option to open the audio settings modal (echo test et al)
via the in-call device selection chevron
- Rework the SFU audio bridge and its services to support
adding/removing tracks on the fly without renegotiation
- Rework the SFU audio bridge and its services to support a new peer
role called "passive-sendrecv". That role is used by dupled peers
that have no active input source on start, but might have one later
on.
- Remove stale PermissionsOverlay component from the audio modal
- Rework how permission errors are detected using the Permissions API
- Rework the local echo test so that it uses a separate media tag
rather than the remote
- Add new, separate dialplans that mute/hold FreeSWITCH channels on
hold based on UA strings. This is orchestrated server-side via
webrtc-sfu and akka-apps. The basic difference here is that channels
now join in their desired state rather than waiting for client side
observers to sync the state up. It also mitigates transparent listen
only performance edge cases on multiple audio channels joining at
the same time.
The old, decoupled listen only mode is still present in code while we
validate this new approach. To test this, transparentListenOnly
must be enabled and listen only mode must be disable on audio join so
that the user skips straight through microphone join.
Mobile users have significant trouble figuring out correct audio I/O
devices according to feedbacks. The potential absence of echo test after
having set an initial device in the first join cycle might complicate
things even further if they got it wrong.
Ignore the skipCheckIfPreviousDevice flag in mobile endpoints. They'll
always go through the echo test if no other skip flag is set, even if
they had a previously configured device.
* refactor(storage): replace Tracker.Dependency with observer hook
* fix(storage): set initial value
* refactor(storage): stop using Meteor's Session singleton
The audio troubleshooting modal has very microphone-specific strings,
which might confuse users trying to join listen only.
Review the Help screen so that listen only scenarios are more generic.
As a bonus, review the unknownError locale with a more actionable text.
- Adds a new Help view for unknown error codes
- Correctly detect NotAllowedError (permissions) - they are currently
being treated like unknown errors in the Help modal
- Rephrase NotAllowedError help text; make it more succint and direct
- Rephrase the unknown error help text; make it more succint and direct
- Add error code and message to that view
- Add public.media.audioTroubleshootingLinks to allow referencing KB
links on the Help modal
- See inline docs
There's no rollback procedure in case a device switch fails right now,
nor does the code entrypoints that call the switching procedures wait
for resolution or failure before marking the new device as chosen. That
may cause inconsistent states in a couple of ways:
- No rollback: switch fails, audio is still on but no actual
microphone input is being transmitted
- Not waiting for resolutions: inconsistent chosen devices on failures
Device switching errors are also not surfaced to the end user
This commit:
- Adds device rollback and proper resolution/failure response
awaits to try and make the state a bit more consistent.
- Centralizes the input device switching code to be reused between
different bridges
- Centralizes device ID state management in audio-manager to try and
mantain them a bit more consistent across the board
- Surface device switching failures to the end user
- Guarantee device IDs are set to the session storage on all
appropriate scenarios
public.media.showVolumeMeterInSettings => public.media.showVolumeMeter
public.media.simplifiedEchoTest => public.media.localEchoTest.enabled
Initial hearing state can be configured in public.media.localEchoTest.initialHearingState
New features:
- A simplified echo test mode that only does a local loopback (instead of
going to FS and back)
- A volume meter for microphone streams to the AudioSettings view
Those two features are experimental and disabled by default; see
public.app.media.simplifiedEchoTest and public.app.media.showVolumeMeter configs
Collateral changes:
- fix: localize fallback device strings in AudioSettings/DeviceSelector
- Refactor on some media stream utils to be re-usable across components
- Refactor in AudioSettings to keep gUM #uses stable.
* TODO: need to pass streams through AudioManager to avoid the surplus gUM.
- fix(audio): drop ScriptProcessorNode usage (deprecated)
* Used in volume meter for tracking - use hark instead
We are now leaving the check for the minBrowserVersions object in settings.yml
If the settings enables chrome iOS, audio should allow users to be joining
with audio.
This is related to recent Chrome update (iOS 14.3+) that now allows
camera/microphone to be captured. We are looking for enabling this for
Chrome 93 in iOS (chromeMobileIOS version in settings.yml)
When listenOnlyMode=false, skipCheck=true and skipCheckOnJoin=true, the
audio tries to start a session more than one time, causing it to fail
at the first one (and reconnect after that).
Now we check if user is already connecting before trying to start a new
audio session.
Added some info in settings.yml for the options related to this commit
Closes#12190
When joining/returning breakouts, audio would always connect
with full audio. This can lead to a performance problem, once
all listenonly users would join full audio, increasing the
number of streams in FreeSWITCH.
We now have a consistent behavior, which is:
1 - The choice made by the user in the main room is predominant:
if mic is active in main room, user will automatically
join mic in breakout room. When returning from breakout
room, user will also join with mic again.
2 - Changes made in breakout room won't have effect when
returning to the main room. This means if user, for example,
change from listenonly to mic in breakout room, the returning
will consider the option choosen previously (listenonly) and
listenonly will be active again in the main room.
3 - If user didn't join audio in the main room, the audio modal
will be prompted when joining the breakout room (this is
a special case of (1))
The following is some technicall information:
InputStreamLiveSelector (component.jsx) now calls
'handleLeaveAudio' function, which is the default
function when user leaves audio (also used when
dynamic devices are inactive).
We now store information about user's choice (mic or listenonly)
using local storage, instead of the previous cookie method (this
was triggering some warnings in browser's console).
Also did a small refactoring to match eslint rules.
Fixes#11662.
* add param to force echo test when user joins audio after init
* fix UI stuck on connecting when userdata-bbb_auto_join_audio=false
* fix conditions for joinFullAudioImmediately and joinFullAudioEchoTest | remove old format
* remove extra param in getItem
* recover audioLocked | only set getEchoTest if doesnt exist
