Commit Graph

205 Commits

Author SHA1 Message Date
Mario Jr
65e6e70d24 update audio-manager docs for getStats and related functions 2021-08-13 09:46:19 -03:00
Mario Jr
cf366e5090 Get user and audio data, shows it and allows it to be copied to clipboard
Using getStats api and peer information to retrieve upload/download rates
and transport information
2021-08-12 16:39:04 -03:00
Mario Jr
4e1eb7318d chore(audio): add audio join time in extraInfo of server log 2021-07-09 13:43:43 -03:00
Mario Jr
c4abd274e8 feat(audio): add option for enabling/disabling kms's trickle for mic 2021-07-02 12:51:00 -03:00
Mario Jr
287c4eb682 feat(audio): use kurento's trickle-ice to improve mic negotiation
Here's what we do when user activates mic:
1 - When we do something similar to listenonly's joining process
until we find a valid candidate-pair. The information about this
local candidate is store.
2 - We then start a new userAgent, and as soon as browser finds
a candidate with the same local ip address, we leave only this
candidate in the SDP and send this to FreeSWITCH. SDP should
contain only a single candidate.
3 - The rest of signaling process is basically the same.
2021-07-01 16:26:44 -03:00
Anton Georgiev
36b67971ee
Merge pull request #12054 from mariogasparoni/fix-12023-and-other-two-fixes
fix: mic selection (firefox/all browsers) and muted alert when mic is changed
2021-04-20 13:40:54 -04:00
Anton Georgiev
5566941ad1
Merge pull request #12003 from ramonlsouza/issue-11985
Fix join webcam breakout room
2021-04-16 10:46:04 -04:00
Mario Jr
e8d59ed14a fix: mic selection (firefox/all browsers) and muted alert when mic is changed
This commit contains three fixes: one already reported and two detected
during the investigation of the solution.
This started as a fix for firefox (#12023), but i also fixed the muted
alert/banner when device changes: the banner wasn't detecting device changes,
unless audio was deactived/actived.

There's another fix for the microphone stream: we now keep sender's track
disabled if it was already disabled for the sender's track of the previous
selected device.

Also did small refactor for eslint checking.

Some technical information: in sip bridge (bridge/sip.js), setInputStream and
liveChangeInputDevice function were both fully turned into promises, which
guarantees we have everything ready when it resolves to the respective values.
This helps AudioManager (audio-manager/index.js) to sequentially sets and
tracks the state of the current microphone stream (inputStream), when calling
liveChangeInputDevice function: we first set the current stream to null,
creats a new one and then set it to the newly created value - this is needed
because MutedAlert (muted-alert/component.jsx) can then gracefully
allocate/deallocate the cloned stream when it is set to a non-null/null value
(the cloned stream is used for speech detection with hark).
In MutedAlert we also make sure to enable the cloned stream's audio
tracks, just in case the user change the device when muted (audio track is
disabled in this case), which also leaves the cloned stream muted (we then
enable the track to allow speech detection).

Closes #12023
2021-04-16 10:45:40 -03:00
Ramon Souza
32dec42264 fix join webcam breakout room 2021-04-12 15:59:43 -03:00
Ramon Souza
d90e889394 fix active poll refresh crash 2021-04-12 13:57:02 -03:00
Mario Jr
951fc0ade8 fix: MEDIA_ERROR when joining room using Firefox
Firefox doesn't create a  device called 'default' and we were trying
to set this when user is joining the room. We don't do this anymore, letting
devices to be changed when there's some user request.

Moved outputDeviceId inputDeviceId information to be managed in bridge
(just like we do with inputDeviceId), we don't store this duplicated
information in audio container anymore.

Fixed the eslint warning in "playAlertSound(url) { ..."

We are safe to let users try to change input/output devices because the
device list is retrieved from enumerateDevices.
2021-04-01 15:53:43 -03:00
Mario Jr
60378e8c63 Merge branch 'develop' into merge-update-pr-10129 2021-03-29 19:07:27 -03:00
Mario Jr
cddca95ad1 Fixes and improvements on pr #10129 - dynamic change audio device
Allow listenonly users to change output devices
Fixed dynamic audio device change for firefox
Fixed shortcuts for audio join/leave
Show (with a bold font) the current selected device
[performance] Prevent calling mediaDevices.enumerateDevices every time we render
the selector. This adds a delay (~200ms, on my chrome setup) to render this component
[performance] Do not call enumerateDevices to search for new devices, instead we listen on mediaDevices.deviceChange event
Small refactoring and fixed a few errors that were being throw in browser's console
Fixed device selection when this is done in audio-settings modal
Fallback to default device when current device is removed
Truncate device name length
Renamed "Input","Output" labels to "Microphone","Speakers", respectively
Update eslint rule for accessKey
2021-03-29 18:55:17 -03:00
Mario Jr
cec88c996d Merge remote-tracking branch 'tainan/issue-9723' into merge-update-pr-10129 2021-03-29 18:52:12 -03:00
Mario Jr
af4f9dbf26 Merge branch 'v2.2.x-release' into merge-2.2-develop 2021-03-23 17:10:28 -03:00
Anton Georgiev
e673fb2cf4
Merge pull request #11630 from prlanzarin/u23-sculptor
listenonly,video-provider,screenshare: remove ICE gathering pre-flight check
2021-03-12 12:58:43 -05:00
prlanzarin
da0a3c36af listen only: remove the old silent audio autoplay workaround
The underlying webkit autoplay issues were properly tackled a long time ago now; this thing isnt needed anymore

Also took the liberty to remove the whole create listen only stream thing because it`s useless
2021-03-12 03:39:12 +00:00
prlanzarin
ab17b10c38 listenonly,video-provider,screenshare: remove ICE gathering pre-flight check 2021-03-12 02:51:45 +00:00
Mario Jr
3b6b9523fc Update breakout audio transfer states when user leave mic
When leaving audio/reconnecting , we must update breakout audio transfer
state.

Related to the issue #11333
2021-03-08 15:01:10 -03:00
Mario Jr
1c7c1d5076 Change breakout-audio-transfer-status function signature
We now update breakout'a audio transfer status by using an object
2021-03-08 14:55:15 -03:00
Mario Jr
ff6cba8d53 Store information about user status during breakout room audio transfer
Currently this information is lost everytime breakout-room component is
unmounted, causing the panel to shows wrong information during next renders
Fixes #11333
2021-03-08 14:55:05 -03:00
Mario Jr
39780c6a50 Do not try to recover mute state when joining listenonly
Prevent toggleMute api call to be called for listenonly
This has no effect to the end user, but avoids unnecessary server calls
2021-03-05 00:23:20 -03:00
Mario Jr
c0625e2ee1 Cleanup joinedAudioOnly state when user disconnects or reconnect microphone
Fixes #11490
2021-03-05 00:06:19 -03:00
Mario Jr
e16566de9d Recover mute state for moderator when returning from breakout's audio transfer
Fixes #11489
2021-03-05 00:02:31 -03:00
Anton Georgiev
228bad03d5 Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into march4-merge 2021-03-04 21:25:47 +00:00
Mario Jr
fc1401ab53 Do not try to recover mute state when joining listenonly
Prevent toggleMute api call to be called for listenonly
This has no effect to the end user, but avoids unnecessary server calls
2021-02-26 15:52:11 -03:00
Mario Jr
ad23e9c0af Cleanup joinedAudioOnly state when user disconnects or reconnect microphone
Fixes #11490
2021-02-26 15:05:17 -03:00
Mario Jr
404d65d7a4 Recover mute state for moderator when returning from breakout's audio transfer
Fixes #11489
2021-02-25 15:38:41 -03:00
Mario Jr
2e92de8010 Clean up HTMLAudioElement used for notification sounds
After ending the notification playback, we set the ".src" property to null, which immediately stop the internal player of mobile browser (tested on Chrome for Android -  device list is on #11458).
For the specific list of devices, this prevents the internal buffer error "-61" described in #11458.
Fixes #11458.
2021-02-23 11:12:37 -03:00
Anton Georgiev
55e8de4357 Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into feb18-merge 2021-02-18 20:33:19 +00:00
Mario Jr
b7216dd100 Preserve mutestatus when disconnecting/reconnecting microphone
After audio reconnection, a muted user would have it's microphone unmuted by default, unless muteOnStart is set to true. This fix this problem.
Fixes #9016
2021-02-11 23:55:34 -03:00
Mario Jr
b582b1ca78 Correctly return MediaStream object when calling audio-manager's inputStream
This was missed in recent audio/sip.js changes.
MediaStream is now returned from SIP.js
Related to #10733
2021-01-29 19:05:51 -03:00
Mario Jr
f2de2806eb Add setting for enabling/disabling microphone audio filters
As explained in #11143, disabling audio filters is desired in some scenarios.
This basically adds an option for user to disable default constraints.
When user doesn't change this value in Settings > Application, the default
value for each audio constraints is retrieved from settings.yml.
When user changes this value in Settings > Application, audio
filters (AGC, Noise Supression and Echo Cancellation) are all set to
true/false, according to the value selected in the Settings GUI.

To start it simple, we decided to not to add a different setting in the GUI for
each audio contraint. This may be added in the future, though (perhaps in a
dedicated Audio Settings tab)
This is related to #4873
2021-01-29 14:18:15 -03:00
Mario Jr
97c76900cb Add setting for enabling/disabling microphone audio filters
As explained in #11143, disabling audio filters is desired in some scenarios.
This basically adds an option for user to disable default constraints.
When user doesn't change this value in Settings > Application, the default
value for each audio constraints is retrieved from settings.yml.
When user changes this value in Settings > Application, audio
filters (AGC, Noise Supression and Echo Cancellation) are all set to
true/false, according to the value selected in the Settings GUI.

To start it simple, we decided to not to add a different setting in the GUI for
each audio contraint. This may be added in the future, though (perhaps in a
dedicated Audio Settings tab)
This is related to #4873
2021-01-22 16:30:42 -03:00
Anton Georgiev
3e3f3236cc correct path to silence.mp3 2021-01-06 15:55:01 +00:00
Anton Georgiev
c1ffced27d Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into develop 2020-12-17 21:14:29 +00:00
prlanzarin
218c6a6549 [audio-manager] Add bridge name to callback logging and fix listen only audio exit 2020-12-01 18:24:24 +00:00
Anton Georgiev
f34826718d include instanceId in resources url 2020-11-30 16:09:35 +00:00
Anton Georgiev
0c7ead1916 Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into oct16-merge 2020-10-21 14:48:02 +00:00
Mario Jr
a7ab00e3e9 Fix wrong skipCheck handling
We now let sip.js handle getUserMedia calls, same should happen when skipCheck is active.
Solves #10652.
2020-10-14 23:30:08 -03:00
Mario Jr
e9e436378a Correctly set audio input/output devices
When refusing ("thumbs down" button) echo test, user is able to select a different input device. This should work fine for chrome, firefox and safari (once user grants permission when asked by html5client).
For output devices, we depend on setSinkId function, which is enabled by default on current chrome release (2020) but not in Firefox (user needs to enable "setSinkId in about:config page). This implementation is listed as (?) in MDN.
In other words, output device selection should work out of the box for chrome, only.
When selecting an outputDevice, all alert sounds (hangup, screenshare , polling, etc) also goes to the same output device.
This solves #10592
2020-10-06 20:37:55 -03:00
Mario Jr
619ffa0ec1 Port SIP.js to 0.17.1 release
This considerably changes the way we process audio signaling and start audio elements in user's browser.
We now avoid using AudioContext element for both microphone and listenonly calls, once it is unstable for some iOS devices (cracky audio, user stops hearing audio after a while).
Increased default value for listenOnlyCallTimeout: this avoids activating FreeSWITCH's fallback when ICE negotiation takes longer than 15sec (tested on DO).
Increased listenonly logs.
This fixes #8133 #10388
2020-09-25 20:26:22 -03:00
Anton Georgiev
377dc27a8d Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into 09-16-merge 2020-09-17 14:37:28 +00:00
prlanzarin
b0ccaf330f audio-manager: shortcut on setSenderTrackEnabled when in listen only mode 2020-09-11 23:06:56 +00:00
prlanzarin
e994680e14 audio-manager: refactor client-side mute/unmute 2020-09-10 14:50:50 +00:00
prlanzarin
2315208887 Remove explicit mute/unmute calls from toggle and leave them only for the voice state tracker
Should increase client-server mute state consistency
2020-09-09 18:08:11 +00:00
prlanzarin
59221530df Add a client-side mute via the sender track corresponding to the mute state
The mic mute is done client side via the track`s enabled flag, which generates silent when false. This still tracks the server/freeswitch mute state, so server-side mutes will be reproduced in the client
2020-09-09 17:03:27 +00:00
Pedro Beschorner Marin
0971edfcb6 Include voice user's upsert to changes observer
We noticed that the mute action button wasn't being consistent when
using muteOnStart as meeting configuration.

Included the addition of the voice user object to the collection as
an event to also be observed.
2020-08-26 18:37:04 -03:00
Anton Georgiev
5be5aed1d9 Merge branch 'v2.2.x-release' of github.com:bigbluebutton/bigbluebutton into merge-2020-08-12 2020-08-12 17:12:58 +00:00
prlanzarin
dac3259c48 Merge remote-tracking branch 'pedrobmarin/bbb-multiple-webcams' into upstream-2.2-vpeg-base 2020-08-05 14:20:58 +00:00
Tainan Felipe
0ba6ff5cf6 Merge remote-tracking branch 'upstream/develop' into issue-9723 2020-07-27 17:02:44 -03:00
Tainan Felipe
2c61d5ee75 Add input/output dynamic audio change 2020-07-27 16:49:26 -03:00
prlanzarin
1dbafffa26 audio: make listen only call timeout configurable 2020-07-09 18:02:18 +00:00
Anton Georgiev
ff1357cbbd
Revert "Improve audio reconnection for muted users for 2.2" 2020-06-22 16:28:22 -04:00
Anton Georgiev
16189780c8
Update bigbluebutton-html5/imports/ui/services/audio-manager/index.js 2020-06-19 16:58:41 -04:00
Tainan Felipe
9fa1986611 Add implicit toggle for toggle voice call 2020-06-19 17:07:11 -03:00
Tainan Felipe
e540d6e46d set user as muted in audio reconnection 2020-06-05 09:41:48 -03:00
Pedro Beschorner Marin
1103663d86 Monitoring audio WebRTC stats 2020-01-28 10:07:21 -03:00
Pedro Beschorner Marin
5c71e2d0bb Included new getStats monitor for audio and video 2019-12-10 09:07:06 -03:00
Chad Pilkey
7218f36e44 listen only logging improvements 2019-12-02 14:15:46 -08:00
Anton Georgiev
2d2c09e914 Replaced "Audio Error" with a template literal of the error 2019-11-27 13:47:40 +00:00
prlanzarin
b52214f0b5 Make some imports use absolute paths
Some more linting as well
2019-11-14 19:07:35 +00:00
prlanzarin
215f79f383 Account for mDNS candidates on gUM fallback for recvonly peers
Also added some client logging for those cases

Moved video-provider gUM fallback detection to be used only on recvonly streams
2019-11-14 16:35:56 +00:00
KDSBrowne
4fd06509e4 fix chrome on ios error being displayed incorrectly 2019-10-01 14:28:06 +00:00
KDSBrowne
89853dc026 move microphone error codes to const 2019-09-30 14:54:34 +00:00
KDSBrowne
636f49cd5d add error messages for getUserMedia failing with mic 2019-09-27 13:52:29 +00:00
Chad Pilkey
f471b1e2b9
Merge pull request #8072 from prlanzarin/2.2-abort-play
Fixes to client-side screenshare/listen only code and log improvements
2019-09-06 18:37:22 -04:00
prlanzarin
c8a88148e1 Improve log message for screenshare/listen only/autoplay and harden media play with retries 2019-09-06 18:58:22 +00:00
Chad Pilkey
35595a18ba fix microphone failure code 2019-09-06 09:35:55 -07:00
Chad Pilkey
304bb2c103
Merge pull request #8030 from antobinary/chat-ids-2
Reduced client load via slight refactor
2019-09-05 13:23:37 -04:00
Chad Pilkey
b33e247f44 change audio error code log property 2019-09-04 11:16:14 -07:00
Anton Georgiev
892336632f added "fields:{...}", removed most mapUser 2019-08-28 16:32:55 -04:00
prlanzarin
8d65ce565d Add UI prompts when autoplay is blocked for video/screenshare/listen only 2019-08-02 21:32:42 +00:00
KDSBrowne
afcc2095a4 remove capitalization from webkitAudioContext 2019-07-26 14:52:39 +00:00
KDSBrowne
4fba9978d3 add audio logs (join/leave/mute/unmute) / fix lint issues 2019-07-25 18:41:24 +00:00
Anton Georgiev
b74d3d9bb0 correct logger.warning to .warn 2019-07-02 16:54:10 +00:00
Anton Georgiev
3fcb95ab44 log structure {logCode, extraInfo}, comment 2019-06-28 21:45:50 +00:00
Fred Dixon
2c617fed7b
Merge pull request #7610 from capilkey/ipv4-audio-fallback
IPv4 FS audio fallback, auto-reconnect, and fixed input select
2019-06-12 19:41:15 -04:00
Chad Pilkey
cc7513279e IPv4 FS audio fallback, auto-reconnect, and fixed input select 2019-06-12 21:01:20 +00:00
Chad Pilkey
854aa8019b bump audio leave log level 2019-06-04 11:16:43 -07:00
Anton Georgiev
12eefb7785 pass additional properties in logline 2019-04-17 19:40:08 +00:00
KDSBrowne
fcbbd0dab5 add default icon parameter to notify() 2019-04-12 13:55:14 +00:00
KDSBrowne
b91229e393 add fix for issue #7096 2019-04-11 22:53:57 +00:00
Anton Georgiev
8380875e70 remove Verto code 2019-03-08 19:41:19 +00:00
Chad Pilkey
89b8189087 html5 sipjs bridge has useful errors now 2019-02-20 13:58:37 -08:00
Maxim Khlobystov
b86a990612 Add logCode and build number to log messages 2019-02-01 14:12:06 -05:00
gustavotrott
c21bb693b4 Store in session if can generate ice candidates, change timeout to 5secs, display error msgs 2019-01-14 22:45:32 -02:00
Gustavo Trott
a875b0b386 Undo changes in askDevicesPermissions 2019-01-14 12:41:44 -02:00
Gustavo Trott
fdf50b3ab7 Validade if Safari can send ICE Candidates before open RTC to receive video 2019-01-10 21:24:25 -02:00
Gustavo Trott
219cad3234 Verify if Safari can generate iceCandidates else askDevicesPermission 2019-01-10 19:25:23 -02:00
Gustavo Trott
6d4cca9306 Fix audio in ios webview 2018-12-21 15:14:05 -02:00
Anton Georgiev
452a12287b improve signal for audio join/leave when in iframe 2018-11-28 21:42:24 +00:00
Anton Georgiev
fea8c267cc improve on listeners for state outside iframe 2018-11-13 17:14:30 +00:00
Ghazi Triki
ff80b30df1 Merge remote-tracking branch 'bigbluebutton/master' into merge-2.x-release-into-master
# Conflicts:
#	bigbluebutton-html5/imports/ui/services/audio-manager/index.js
2018-11-02 21:02:02 +01:00
Tainan Felipe
1bc8ae0b21 add join audio only, and imporov UX 2018-10-23 14:18:09 -03:00
prlanzarin
2741022cb2 Updated kurento-utils WebRTC calls, added permission request on listen only for safari
Also fixed a resource leak on listen only fallback and transitioned the SFU listen only from a tag-based approach to MediaStream
2018-10-03 18:39:55 +00:00
prlanzarin
fe9835b5f7 Add SIP.js listen only as fallback when SFU is enabled 2018-09-11 17:09:29 +00:00
prlanzarin
dc767d9bdb Fixed server-side error handling for listen only via SFU, correctly handling KMS abrupt shutdown 2018-08-30 02:41:02 +00:00
prlanzarin
b4052f7052 Resolving listen only stream correctly 2018-08-29 19:59:33 +00:00
prlanzarin
a7bfb985d3 Improvements to Safari/Chrome autoplay policies with placeholder buffer on audiocontext and bogus audio + upstarting audio tag as muted 2018-08-29 19:12:34 +00:00