ICE lite servers (eg mediasoup) dont need candidates signaled out-of-band; neither does KMS in certain scenarios
Disable their signaling saves us some ticks in bbb-webrtc-sfu and some bandwidth all around
Restored the old behavior when ending breakout rooms while user is in the
breakout audio transfer, which is to the trigger the reconnection to the audio
in the main room.
This behavior could be improved by (instead of reconnecting) transfering user
back to the main room, but this requires some changes in akka-apps/fsesl
which can be treated in a different issue.
Closes#13242
Applies to video, listen only and screen sharing
New metadata values: media-server-video, media-server-listenonly, media-server-screenshare; parameter is a String
Added support for getStats in screenshare's service. This works similar
to the getStats for video provider, and the information retrieved from
screenshare is added to the video information for cameras.
We now let audio-manager log errors, using the specified error code.
These errors are still logged in bridge layer, but as a warning.
This commit doens't change error codes behavior, they are still being
logged as errors and with the same code numbers.
Scenario: presenter`s client could crash when the presenter changed while they were sharing their screen
That is due to a race condition on the stop procedure in the bridge: two stops can be triggered (one from the server-side websocket tear off and another from the client itself detecting the presenter change)
That could create a scenario where the broker was cleaned in one stop procedure after the second had checked its availability, causing an attribute access of a null member
Here's what we do when user activates mic:
1 - When we do something similar to listenonly's joining process
until we find a valid candidate-pair. The information about this
local candidate is store.
2 - We then start a new userAgent, and as soon as browser finds
a candidate with the same local ip address, we leave only this
candidate in the SDP and send this to FreeSWITCH. SDP should
contain only a single candidate.
3 - The rest of signaling process is basically the same.
Code style fixes and remove translations other than en to avoid conflicts with transifex.
Co-authored-by: Pedro Beschorner Marin <pedrobmarin@gmail.com>
Remove parts of a previous connection monitor.
To add some context (as far as my memory goes) to the multiple connection
monitor features the product has, `stats` (currently named `connection status`)
was introduced at the Flash client back in ~2016. @fcecagno and I did it
as a BigBlueButton's Summit activity. Our work was squashed into a single
commit in 92554f8b3e :).
I'm not sure about the whole story behind `network information` (the late
connection monitor added to the HTML5 client) but I assume it should work
as a collector for a bunch of different connectivity monitors. I remember
when it was introduced but I don't know why it wasn't adopted. My best guess
would be because of some performance issues the `user list` had back then.
To follow on why `connection status` replaced `network information` at the
HTML5 client, when I did the `multiple webcams` feature I had to refactor
a big chunk of the `video provider` (#8374). Something that wasn't really
helping there was the adaptation of `stats` that was made to show local
feedback for each webcam connection. Although this feature wasn't being
used anymore, `network information` did rely on that to build up data. With
this monitor gone I assumed it was my responsibility to provide an alternative
so I promoted Mconf's port of the Flash `stats` monitor to BigBlueButton's
HTML5 client (#8579).
Well, that's my perspective on how things went for those features. If
anyone would like to correct me on something or add something else on
that history I would appreciate to know.
To avoid re-rendering of the poll answers when the poll panel is opened and closed,
this commit uses the currentPoll object to determine wether results should be rendered.
This patch adds a new option to hide the individual answers to the presenter
when starting a poll. This includes hiding the answers for the presenter and
informing the other users that the current poll is anonymous.
See #9524
Refactor the external videos collection, moving the logic and functionalities
outside of /imports/api/meetings to a new location in /external-videos/server/modifiers
in order to decrease the coupling between the functionalities, favoring
the maintenance.
BigBlueButton v2.3 uses an instance prefix while generating padIds. Closed
captions was missing this information when updating pad's content for dictation
mode.
Add PollPublishedRecordEvent with all infos (also fix duplicated handlePollStoppedEvtMsg)
Add answer text to UserRespondedToPollRecordEvent (useful for custom answers)
Add type in PollStartedRecordEvent
When joining breakout audio, the output device selected in the main room is
used in breakout.
When returning from breakout rooms, the output audio device previously set in
the main room is restored.
Some specific info:
SIPSession doesn't handle Storage anymore, we do this in SIPBridge, since
it has more info about the current selected device and it doesn't depend of
a session being oppened.
We also now pass the output device ID when joining audio sessions. We can
then keep this information in the Storage.
Closes#11663
This commit contains three fixes: one already reported and two detected
during the investigation of the solution.
This started as a fix for firefox (#12023), but i also fixed the muted
alert/banner when device changes: the banner wasn't detecting device changes,
unless audio was deactived/actived.
There's another fix for the microphone stream: we now keep sender's track
disabled if it was already disabled for the sender's track of the previous
selected device.
Also did small refactor for eslint checking.
Some technical information: in sip bridge (bridge/sip.js), setInputStream and
liveChangeInputDevice function were both fully turned into promises, which
guarantees we have everything ready when it resolves to the respective values.
This helps AudioManager (audio-manager/index.js) to sequentially sets and
tracks the state of the current microphone stream (inputStream), when calling
liveChangeInputDevice function: we first set the current stream to null,
creats a new one and then set it to the newly created value - this is needed
because MutedAlert (muted-alert/component.jsx) can then gracefully
allocate/deallocate the cloned stream when it is set to a non-null/null value
(the cloned stream is used for speech detection with hark).
In MutedAlert we also make sure to enable the cloned stream's audio
tracks, just in case the user change the device when muted (audio track is
disabled in this case), which also leaves the cloned stream muted (we then
enable the track to allow speech detection).
Closes#12023
Video streams can be sorted by voice floor activity in the client according to FreeSWITCH´s floor events. The feature works together with pagination, essentially giving an Last-N like experience while not disrupting too much
Made video stream sorting extensible in a way. The sorting modes for pagination and unbounded can be configured in settings.yml and new sorting modes can be added to the stream sorting util under video-provider. Inline docs explain how to do that
Changed how the stream ID attribute from video-streams collection was passed to downstream components; we had an array map that was executed every change just to map stream to cameraId, which is bizarre. So I changed the cameraId usage in downstream components to be conformat with the collection attributes and shaved off the map where it wasnt needed
Add better selectors to video-list-item container´s VoiceUser fetch
Etherpad has a limitation of 50 characters for it's pad ids. Although
our SHA1 hash function generates 40 characters length values, after
prefixing (Meteor's instance id) and suffixing (closed captions support)
the pad id we can reach an invalid size.
Firefox doesn't create a device called 'default' and we were trying
to set this when user is joining the room. We don't do this anymore, letting
devices to be changed when there's some user request.
Moved outputDeviceId inputDeviceId information to be managed in bridge
(just like we do with inputDeviceId), we don't store this duplicated
information in audio container anymore.
Fixed the eslint warning in "playAlertSound(url) { ..."
We are safe to let users try to change input/output devices because the
device list is retrieved from enumerateDevices.
Allow listenonly users to change output devices
Fixed dynamic audio device change for firefox
Fixed shortcuts for audio join/leave
Show (with a bold font) the current selected device
[performance] Prevent calling mediaDevices.enumerateDevices every time we render
the selector. This adds a delay (~200ms, on my chrome setup) to render this component
[performance] Do not call enumerateDevices to search for new devices, instead we listen on mediaDevices.deviceChange event
Small refactoring and fixed a few errors that were being throw in browser's console
Fixed device selection when this is done in audio-settings modal
Fallback to default device when current device is removed
Truncate device name length
Renamed "Input","Output" labels to "Microphone","Speakers", respectively
Update eslint rule for accessKey
- Removed the connection-status history from the user list's gear
icon and now is opened by the connection-status button. Moderators will
render the same modal as before and viewers will only have access to their
own data.
- Added data-savings shortcut at the connection-status modal.
- Added websocket round-trip time.
Since Meteor was split in multiple process and events started to be
filtered by instances, all Etherpad's Redis events were being discarded.
Etherpad has a Redis' publisher plugin that is unaware of BigBlueButton's
existence. All the communication between them is kept simple with minimal
of internal data exchange. The concept of distincts subscribers at Meteor's
side broke part of this simplicity and, now, Etherpad has to know which
instance must receive it's messages. To provide such information I decided
to include Meteor's instance as part of the pad's id. Should look like:
- [instanceId]padId for the shared notes
- [instanceId]padId_cc_(locale) for the closed captions
With those changes the pad id generation made at the recording scripts had to
be re-done because there is no instance id available. Pad id is now recorded at
akka-apps and queried while archiving the shared notes.
Modified the previous implementation of the whiteboard individual access to remove
multiple Collections dependency on this feature. Multi-user whiteboard is now an
array instead of a boolean value and most of the access control can be synchronized
and handled by akka-apps.
Moderators are able to send a message to the meeting's guest lobby. This new
event reaches bbb-web and is sent to the guest user with her/his status response
while polling. All guest users that are waiting for acceptance will be able to
read this message.
enableGuestLobbyMessage is disabled by default.
Without 'exact' match, the browser fallbacks to the default inputDeviceId
This prevents the error (input device error) when breakout is ended and we try
to skipCheck the microphone when user returns to main room (assuming the
user had the microphone active before joining breakout room).
Without 'exact' match, the browser fallbacks to the default inputDeviceId
This prevents the error (input device error) when breakout is ended and we try
to skipCheck the microphone when user returns to main room (assuming the
user had the microphone active before joining breakout room). Related
to the feature c451666d52
After audio reconnection, a muted user would have it's microphone unmuted by default, unless muteOnStart is set to true. This fix this problem.
Fixes#9016
Associate pads with meetings so session validation is restricted to the
meeting's valid session tokens.
Meteor will dispatch new redis events on shared notes and closed captions
pads creation. This event will go through apps and reach web to populate
a new meeting's pad collection that contains all valid pad id's for that
session. Nginx will use this collection to check if the user's session token
belongs to the pad's authorized users.
Besides these modifications, an extra change will be needed at notes.nginx.
Location /pad/p/ needs to change it's auth_request:
from /bigbluebutton/connection/checkAuthorization;
to /bigbluebutton/connection/validatePad;
When managing Etherpad's pads, Meteor makes API calls to initiate the closed captions
and shared notes modules. The pad id was being mapped to a shorter id than the meeting
id because of a Etherpad lenght limitation.
Changed to something less guessable.
* add param to force echo test when user joins audio after init
* fix UI stuck on connecting when userdata-bbb_auto_join_audio=false
* fix conditions for joinFullAudioImmediately and joinFullAudioEchoTest | remove old format
* remove extra param in getItem
* recover audioLocked | only set getEchoTest if doesnt exist
As explained in #11143, disabling audio filters is desired in some scenarios.
This basically adds an option for user to disable default constraints.
When user doesn't change this value in Settings > Application, the default
value for each audio constraints is retrieved from settings.yml.
When user changes this value in Settings > Application, audio
filters (AGC, Noise Supression and Echo Cancellation) are all set to
true/false, according to the value selected in the Settings GUI.
To start it simple, we decided to not to add a different setting in the GUI for
each audio contraint. This may be added in the future, though (perhaps in a
dedicated Audio Settings tab)
This is related to #4873
As explained in #11143, disabling audio filters is desired in some scenarios.
This basically adds an option for user to disable default constraints.
When user doesn't change this value in Settings > Application, the default
value for each audio constraints is retrieved from settings.yml.
When user changes this value in Settings > Application, audio
filters (AGC, Noise Supression and Echo Cancellation) are all set to
true/false, according to the value selected in the Settings GUI.
To start it simple, we decided to not to add a different setting in the GUI for
each audio contraint. This may be added in the future, though (perhaps in a
dedicated Audio Settings tab)
This is related to #4873
In some scenarios, there's no need for the browser to apply such audio filters. For example, when user's microphone already does audio filtering (echo cancellation, noise supression ...).
This commit doens't change the current behavior (filters still follow browser's default config): admins need to uncomment/set these values if disabling/enabling specific filters if desired.
This is related to #4873
Audio client logs already cover audio session progress the way we need.
This avoids keepAlive and other unnecessary messages to be logged in browser's console.
If setting is not present, default value is set to false.
This was added as an option (websocketKeepAliveInterval), which is the interval to send keep alive messages.
Setting websocketKeepAliveInterval to 0 disables the keep alive, producing the same old behavior.
This helps avoid websocket disconnection due to socket inactivity, preventing it to unnecessarily reconnect.
Also, sometimes reconnect fails and error 1005 is triggered.
Fixes problems reported in #10985.
Also reduces occurrences of error 1005.
Added new SFU broker for screen sharing
Removed kurento-extension entirely
Added inbound and outbound reconnection procedures
Improve UI responsiveness when sharing
Add reconnection UI states
Redo error handling
Refactor actions-bar screen share components. Make it smarter with less prop drilling and less re-rendering. Also more readable. Still work to do in that I think
Add a connection retry procedure for screen presenters when they are sharing; try a configurable amount of times when failure is triggered, with configurable min and max reconn timeouts and timeout increase factor
Make local preview attachment smarter
ADD PARTIAL SUPPORT FOR AUDIO SHARING VIA SCREEN SHARING WITH GET DISPLAY MEDIA, RECORDING STILL NOT SUPPORTED!!!
Fixed listen only reconnection handling
Added proper error handling; now all errors have proper mapped codes which are funneled through to audio-manager logger and should be easier to gauge types of errors
Fixed botched reconnection error rejection, audio modal shouldnt be stuck anymore when it fails
Remove every tie that listen only bridge had to kurento-extension
Instead of sending using rfc4733 standard, we use INFO message for all transfers
INFO message was used in older SIP.js version. Although this is not a standard for sending DTMF tones, this has more reliability (once it sent over TCP)
This might reduce occurrences of 1008
This is the same behavior we used to have on older sip.js version code
By doing this we reduce errors when user try to perform join/hangup during an websocket reconnection
This happens because FreeSWITCH is not able to parse the "From" header when it has multiple occurrences of ':'. So user is not able to join audio.
To fix, we now changed the "callerId" to use the base64 value of the user name, instead of directly using user's input (the callerId format keeps being a triple like this: <user_id>-bbbID-<base64_encoded_name>).
Once this callerIdName is encoded at the same point it is generated, there shouldn't be server side effects for changing this value; except for those places where the callerName is retrieved by splitting this triple (such as the voice talking-indicator, as described below).
Updated the talking-indicator to retrieve the username from User's object, instead of retrieving from the one username generated by splitting the callerId triple.
This problem also happens in versions <= 2.2.26.
This adds the possibility to configure the SIP Via header to plain WS to allow reverse proxying from WSS to WS, internally, to work around a bug in freeswitch where the WSS stack would get deadlocked due to a still unidentified bug in there that has to do with SSL termination
Although Chrome's default is now unified plan, Chrome <-> FreeSWITCH ICE connection fails for some Chrome installations (specially those running on Windows).
FS ICE fails when Chromes's SDP has "a=mid:<index>" (instead of "a=mid:audio").
This fixes Error 1010 and situations where echo test takes too long.
This fixes#6414 regression, once we do the same older version of SIP.js used to do.
We now use both peer's connectionstatechange and iceconnectionstatechange to monitor ICE state for audio sessions.
The same way we did with old sip.js version, we leave iceconnectionstate trigger audio actions , such as connect, disconnect, reconnect.
We still listen for 'failed' state for connectionstatechange event, because chrome triggers this (tested on 86+).
This should reduce the audio error 1010 ocurrences, once some browsers (specially Chrome/Android) don't trigger connectionstatechangeevent.
This might reduce problems reported in #10708, which still needs more investigation though.
Maps WebSocket's 1006 error to BBB's 1002, the same way it was done with old sip.js version
Set user agent's number of reconnection attempts to the same value as older sip.js version
Changed the maximum attempts of the UserAgent reconnection (this should be changed when binding audio's websocket to meteor's connection state).
Added a log to monitor WS reconnect attempts.
When closing/reloading tab with active microphone, audio exits successfully but a wrong log-error (1005) is shown.
We now process closing/reloading tab the same way we do when user hangup the call.
For some reason (still investigating), using turn/coturn on 443/tcp makes firefox's iceGathering process (during echo test) takes 12+ seconds (tested on webrtc's trickle page with multiple instances).
This was found when testing the current default (15s) on production with a private turn/coturn server on port 443/tcp. For default bbb setup (stun only), echo test still runs fast.
To avoid adding extra delay to iceGathering on this scenario (Firefox + turn on 443/tcp), i am just setting the default value back to the 5s (old default).
So , for those who wants to reduce the 1004 occurrences, increasing the iceGatheringTimeout could help (just be aware this adds delay on the mentioned scenario).
Added a default 'MEDIA' option: iceGatheringTimeout. This option allows admin to set a higher ICE gathering timeout, which can help when getting ICE errors during audio negotiation (eg 1004)
Default value set to 15s (current default is 5s).
Sometimes, when user already joined audio session, RTCPeerConnection may
find new ICE candidates, which triggers 'connected' state for peer's
'onconnectionstatechange' event. When this happens we process this
new state the same way when user is not running an audio session, which
makes html5client popup an annoying 'Audio Connected' message.
The audio keeps working fine, but this can make user think that there's a
connection issue, or the audio is reconnecting, while audio is ok.
When getting disconnected with 1001 ("websocket closed unexpectedly" error) we were creating a new SIP session, therefore a new FreeSWITCH channel.
While reconnecting the socket, instead of closing the SIP session, we keep it alive during reconnection (audio should keep working in the meantime). When reconnected we keep using this same session (avoiding the creation of an extra one).
We also better handle WebSocket error codes from SIP.js.
FF immediately closes websocket when unloading page, so we now to stop user agent when 'beforeunload' event is triggered, to avoid leaving open sessions in FreeSWITCH when user leaves page.
When refusing ("thumbs down" button) echo test, user is able to select a different input device. This should work fine for chrome, firefox and safari (once user grants permission when asked by html5client).
For output devices, we depend on setSinkId function, which is enabled by default on current chrome release (2020) but not in Firefox (user needs to enable "setSinkId in about:config page). This implementation is listed as (?) in MDN.
In other words, output device selection should work out of the box for chrome, only.
When selecting an outputDevice, all alert sounds (hangup, screenshare , polling, etc) also goes to the same output device.
This solves #10592
This considerably changes the way we process audio signaling and start audio elements in user's browser.
We now avoid using AudioContext element for both microphone and listenonly calls, once it is unstable for some iOS devices (cracky audio, user stops hearing audio after a while).
Increased default value for listenOnlyCallTimeout: this avoids activating FreeSWITCH's fallback when ICE negotiation takes longer than 15sec (tested on DO).
Increased listenonly logs.
This fixes#8133#10388
If an ejected user tries to enter in the meeting using the current url
html5 client keep trying to validate that user, but without success
causing a validateAuthToken message spam until the connection times out.
This adds bbb_record_video userdata that, when is set to false, signals
bbb-webrtc-sfu to avoid generating the stream recording assets for a
particular user.
By default, bbb_record_video is true.
