If the autoplay block is triggered in listen only, the connection timer
keeps ticking even if the user correctly accepts the audio play prompt.
That causes an audio re-connect once the timeout expires.
Clear the connection timer if the audio bridge starts with
NotAllowedError as a soft error. For connection purposes, the audio join
procedure worked. The autoplay thing is at the UI/UX level, not WebRTC.
This is an initial, experimental implementation of the feature proposed in
https://github.com/bigbluebutton/bigbluebutton/issues/14021.
The intention is to phase out the explicit listen only mode with two
overarching goals:
- Reduce UX friction and increase familiarity: the existence of a separate
listen only mode is a source of confusion for the majority of users
Reduce average server-side CPU usage while also making it possible for
having full audio-only meetings.
The proof-of-concept works based on the assumption that a "many
concurrent active talkers" scenario is both rare and not useful. With
that in mind, this including two server-side triggers:
- On microphone inactivity (currently mute action that is sustained for
4 seconds, configurable): FreeSWITCH channels are held (which translates
to much lower CPU usage, virtually 0%). Receiving channels are switched,
server side, to a listening mode (SFU, mediasoup).
* This required an extension to mediasoup two allow re-assigning producers
to already established consumers. No re-negotiation is done.
- On microphone activity (currently unmute action, immediate):
FreeSWITCH channels are unheld, listening mode is deactivated and the
mute state is updated accordingly (in this order).
This is *off by default*. It needs to be enabled in two places:
- `/etc/bigbluebutton/bbb-webrtc-sfu/production.yml` ->
`transparentListenOnly: true`
- End users:
* Server wide: `/etc/bigbluebutton/bbb-html5.yml` ->
`public.media.transparentListenOnly: true`
* Per user: `userdata-bbb_transparent_listen_only=true`
The refactoring in 838accf015 incorrectly
replaced the wrong parseMessage function in addBulkGroupChatMsgs.js
This bug is only triggered when the option public.chat.bufferChatInsertsMs != 0.
SFU based audio is missing connection timers, which means the join
procedure can go on indefinitely in a couple of scenarios.
Refactor the connection timers added for re-connections in the SFU audio
bridge and make them valid for the first try as well.
Make 1010 errors (connection timeout) retriable when retryThroughRelay
is enabled.
1007 errors are still a large fraction of our overall audio join error
rate. This usually indicates some sort of firewall block or UDP issues
carrier networks. I can't figure out why some scenarios won't trickle
down to relay candidates though - I'm leaning to scenarios where STUN
packets with USE-CANDIDATE are being mangled/lost along the way or
something else that borks the (already fragile) conn checks for ICE-lite
implementations.
Add a new feature called retryThroughRelay which triggers a retry with
iceTransportPolicy=relay whenever audio fails to join with a 1007 error.
The goal is to force relay usage to try and bypass 1007s scenarios that
still happen.
Disabled by default.
* Refactor: Make all chat area use graphql
* Fix: large space between welcome msg and chat list
* Fix: missing file
* add pending status and fix system messages
* Add: mark messages as seen in chat
* Refactor: Move char opening logic to inside of chat panel
* Refactor message and mark as seen
* Add Recharts to package.json and fix miss data
* Implements clear-chat function on graphql
* Make system message sticky
* Add clear message support and fix user is typing
* FIx chat unread and scroll not following the tail
* Change: make unread messages be marked by message and fix throttle
* Don't show restore welcome message when the welcome message isn't set
* Fix: scroll not following the tail properly
* Fix: previous page last sender not working
* Fix: scroll loading all messages
* Fix messaga not marked as read
---------
Co-authored-by: Gustavo Trott <gustavo@trott.com.br>
Currently, the reactions are cleared on mongo based on the expire time.
So to sync its state with akka, an observer is atattched to the user
reactions collection and whenever a reaction is removed because of the
expiration time, akka is notified with message 'UserReactionTimeExpiredCmdMsg'.
Adds two new messages which clear the users emoji statuses. This messages
enables this task to be done in a single call, instead of triggering one
method call for each user.
'ClearAllUsersEmojiCmdMsg' is sent from meteor to akka and updates all the
emoji states in the users model.
'ClearedAllUsersEmojiEvtMsg' is sent from akka to meteor. This triggers the
mongo collection update.
This commit makes the messages of the timer feature to be proxied by
akka-apps and also adds a timer model that is updated based on these
messages.
Moving the timer panel opening logic to the timer button component in
the navigation panel was a consequence of these changes.
Changes the timer auto-stop threshold from 90% to 100% of user count
to prioritize the timer alert sounding for all users over the timer state
being consistent.
Also, puts the user count fetch back to where it was to avoid a race
condition where the number of users when setting the observer is
different than when the timer ends(i.e. users've joined or left the
meeting while the timer was running) causing the timer not to stop or
stop prematurely.
There was a legacy attribute being used to find active users in the meeting.
This wrong attribute caused the returned number of users to be 0 which
makes the timer stop prematurely and, possibly, not to issue the timer beep.
Also adds a missing argument to updateTimer and moves the find call to
the Users collection to a external function, so it doesn't get executed
every time an user notifies timer has ended.
Filters users collection by 'online' connection status and describes when/how
server detects that the timer has ended to automatically stop it.
Also, fixed a corner case that when timer alarm was disabled, clients didn't
notify that the timer ended.
When user enters meeting with music already playing, an event listener is set
to play the music only after user interaction.
Also, to keep timer more cohesive and reduce complexity of the condition for
playing music, timer automatically stops after 90% of users notify that the
timer has expired.
Adds a music player for ambient sound which can be turned on and off using a
toggle located inside timer panel. When stopwatch is selected, this toggle is
automattically turned off.
Turns the screenshare component into a generic component, so that it can be
used both for screenshare and camera as content fetures.
Also changes specific locales and icons for the camera as content feature.
Interactions button is a button that stays in action bar.
Integrates several features in just one place: user-reaction, raise hand,
user-status (away/not away), toggle question panel.
add user-reaction collection
add emoji picker for user reaction in the user list
add options to enable/disable user-reaction
add a way to pass style to emoji-picker component
Fixes a case where the locale selector don't show up in Chrome when using
'webspeech' provider.
And adds missing fields to the webspeech transcription messages, after the
addition of some new parameters to those messages with the open
transcription server.
The current Vosk CC provider does not support stereo mic streams
(pending investigation as to why).
This commits makes sure stereo is forcefully disabled via SDP munging
only when transcription is active and using Vosk. Having it disabled
in the server side (FreeSWITCH) is not enough because the stereo parameter
is client mandated and replicated by FS on its answer. So we need to
make sure it's always disabled for the time being.
SFU audio does munging server side (and stereo is always off), so no changes
needed there.
The rest of the providers (except WebSpeech) need to be validated against
stereo audio as well.
This is also intended to be temporary - ideally this needs to be fixed in
mod_audio_fork/Vosk/wherever this is breaking.
Audio state callback and remote media setup both depend on FS's state
(comes through Meteor) and the ICE state (local, peer connection). The
caveat: FS's state can come delayed on reconnection scenarios because
Meteor's websocket generally takes significantly longer to re-connect than
the peer connection, which means the ICE state gets completed way before FS
is flagged as ready.
The practical issue: while outbound audio (client -> FS) will work, inbound
audio (FS -> client) won't _just because it wasn't played_ (even though
data is coming through).
This commit decouples the remote media setup step from the state
through:
- Setup remote media when ICE state is completed
- Run the state callback only after FS is flagged as ready. This
should maintain the UI states consistent across client-server.
Keep in mind the assumption that if FS is ready, ICE is completed by
consequence.
The voice user ejection callback tethered to Meteor's socket
disconnection seems broken (since its introduction). The VU selector
uses an invalid field (requesterUserId) - so no VU is ever returned.
Since I'm unaware of the original goal behind this code and there's
already ejections in place in other components (akka-apps, for
instance), this is basically a revert of #9888.
There's an edge case in finnicky networks where ALG-like firewalls
tamper with USE-CANDIDATE STUN packets and, consequently, bork ICE-lite
connectivity establishment. The odd part is that client-side gathering
seems to complete if intermediate STUN bindings work (before the final
USE-CANDIDATE), which may cause the peer not to generate relay
candidates == connectivity fails.
This adds the `public.kurento.gatheringTimeout` option to forcefully extend
the candidate gathering window in peers that act as offerers. The
behavior is as follows: if the flag is set (ms), the peer will wait
either the gathering completed stage or, _at most_,
public.kurento.gatheringTimeout ms before proceeding with calls chained
to setLocalDescription.
This option is disabled by default and intentionally ommited from the
base settings.yml file as to not encourage its use. Don't use it unless
you know what you're doing :).
Reconnection timers are far too long for abrupt failures because we
are waiting the original timeouts to elapse (30-60s) before trying it
again - even if a connection worked N-sessions back in that session's
history. The ideal thing to have is another intermediate, smaller and
fixed reconnection timer for sessions that had a working screen share
at least once.
The UI is also not being updated to the reconnecting state on negotiation
failures.
* Add an intermediate reconnection timer for abrupt failures set to 8s.
This should improve reconnection times.
* Lower default connection timers values (base 20s down from 30s, max
25s down from 60s)
* Set screen share UI to reconnecting on abrupt failures as well - we
were only tracking ICE states prior to this, not negotiation errors
The reconnect routine is stopping for viewers if a broker cannot
re-connect in the first try. That is wrong: viewers should try to
reconnect as long as there'sigaling data that mandates so.
The reconnect trigger is changed from broker's started attribute to the
presence of a scheduled reconnection timeout - if there isn't one (not
schedule), always re-schedule it.
Mostly benign, but exitAudio/forceExitAudio was throwing an unhandled
error when called on sessions with no active audio because the
underlying bridge methods did not check whether there was an active
session to stop beforehand.
Several improvements to tldraw whiteboard:
- Only send the shape diff on shape updates (reduce a lot the message traffic)
- Shape permissions (don't allow others to select/edit unless you are presenter/moderator)
- This required some changes in akka model
- Tldraw state patch changes to improve stability with fast updates (fix several crashes)
We were calling upsert in the Annotations collection for the same annotation in all frontend instances, this could lead to the same annotation being inserted
multiple times with different ids due to concurrency.
Added the html5InstanceId of the original request to the redis message so we can use it to only call upsert in one instance.
