The media monitor responsible for triggering the reconnecting view in
the screen sharing component was maintaing the previous state (eg
flowing) in cases where the peer just failed before media stopped
flowing. That triggered an error in the bps calculations that caused the
previous state to be preserved - eg stuck in flowing while it should be
not_flowing.
These changes make it so that if there's not peer to fetch stats from,
them the bps calculations will correctly return 0 (which translates to
not_flowing).
Smart layout (et al) presumes screen sharing will always use 100%
width of the media area. That causes cameras to always be positioned on
top, which is not always the optimal position depending on the viewport
and stream aspect ratio/resolution - so space is wasted.
This commit uses the actual screen sharing video size as provided by
HTMLVideo's videoWidth/videoHeight properties. The calculation uses the
same logic as the one used for presentation/slides, which should make it
a bit familiar.
There's also a handler for HTMLVideo's `resize` event for those browsers
that support it - which enables handling of variable-sized screen
sharing streams. That handler is debounced at 500 ms to prevent
excessive CPU use.
Extra testing is needed with the widest range possible of
browsers/environments and feature combinations.
There could be a race condition where the local getDisplayMedia stream ends
(eg via Chrome`s stop sharing toast) while the broker hasn't finished starting.
That would lead to a scenario where the broker wouldn't emit an end event,
causing screen sharing to be flagged as started with a blank/invalid stream.
Screen streams were only deemed unhealthy when the transport's ICE state
transitioned to failed. That was as good as nothing because the stream would
stay frozen with no visual UI feedback until it reconnected. Bad UX.
This commit addresses that issue via two changes:
- A stream is deemed *potentially* unhealthy now if the transport's
state becomes disconnected
- If a stream is deemed potentially unhealthy, a monitor probe is
started to check whether there is media/packet flow (every 500ms).
If there's no packet flow, the stream is flagged is factually unhealthy and
UI feedback about that is rendered.
It's still not as good as it could be - relying on disconnected still
leaves a couple of seconds of silence to be dealt with. For that to be
addressed the prober would have to run nonstop, but that's for later.
I have growing concerns about gain node`s effect on audio quality the way it
was implemented, so I opted to fall back to HTMLMediaElement`s volume control
for the time being until we can gauge quality impacts properly later on
Add a new configuration flag enableVolumeControl, false by default while the
feature undergoes a field trial
Splits screenshare stream into video and audio and adds gain node to audio
stream in order to permit volume control by the user. Volume is normalized
between [0, 2](muted and 2x boost).
Added support for getStats in screenshare's service. This works similar
to the getStats for video provider, and the information retrieved from
screenshare is added to the video information for cameras.
isFullscreen is a state property and is being fetched from the component
props. I think this is something I messed up while resolving a cherry-pick
conflict.
Change the previous loading spinner to the new loading dots, ensuring
a bit more UI consistency.
Add the "unhealthy stream" filter to make the dots a little more visible and
remove the loading spinner.
Shave off the number of calls in video-preview and video-provider by
using a stream storage
We don´t call an upfront gUM in video-preview anymore to lift the
fingerprinting barrier on device labels and IDs. Flow has been reversed:
upfront enumerate, load first preview, then check if previous
enumeration was obfuscated.
Add a stream storage in video-preview`s service to avoid re-fetching
them in video-providerj
Remove some unneeded video-preview container props
Improve some of video-preview`s error locales
Add the shrink/expand button in the screenshare layout to reduce the mirror/tunnel
effect, when the presenter is sharing the same screen as the application is.
If the presenter gets changed while the user kept the getDisplayMedia browser picker open, accepting the getDisplayMedia prompt might get the previous presenter´s client-side state inconsistent as if they were sharing. The screen sharing is a no-op though
This adds a check after gDM resolves to see whether the user is still the presenter; if they aren`t, just clean up the gDM stream and make it a full no op
Adapted the breakout room screen sharing shutdown to use the new refactor calls
Also changed the behaviour a bit to only trigger the stop code for presenters/sharers, mimicking the camera behaviour and not leaving viewer with stuck screens
Mainly: track unhealthy fatal states instead of supposedly healthy states
Avoids unwarranted UI reconnecting states or flickering on low end devices or struggling networks
Also added a version of isVideoBroadcasting for presenters which only accounts for the server state, so that we can deal with the presnter`s sharing state with more fidelity in the UI
Added new SFU broker for screen sharing
Removed kurento-extension entirely
Added inbound and outbound reconnection procedures
Improve UI responsiveness when sharing
Add reconnection UI states
Redo error handling
Refactor actions-bar screen share components. Make it smarter with less prop drilling and less re-rendering. Also more readable. Still work to do in that I think
Add a connection retry procedure for screen presenters when they are sharing; try a configurable amount of times when failure is triggered, with configurable min and max reconn timeouts and timeout increase factor
Make local preview attachment smarter
ADD PARTIAL SUPPORT FOR AUDIO SHARING VIA SCREEN SHARING WITH GET DISPLAY MEDIA, RECORDING STILL NOT SUPPORTED!!!
