Commit Graph

12380 Commits

Author SHA1 Message Date
Pedro Beschorner Marin
d553ca65cf feat(captions): use navigator language
If not set to use the default language, try to select the navigator
language as speech default locale.
2022-07-20 17:20:52 +00:00
Pedro Beschorner Marin
116c0d9a49 fix(captions): filter duplicated languages
Avoid multiple instances of the same language at the voices data.
2022-07-20 17:20:52 +00:00
Pedro Beschorner Marin
51eeb092b3 refactor(captions): configurable languages
Move the language collection to the HTML settings file. This data defines
the available languages available for the speech API.

These language tags are used to filter SpeechSynthesis' API `getVoices`
result. Tags must use BCP 47 format.

https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesisVoice/lang
2022-07-20 17:20:52 +00:00
Pedro Beschorner Marin
b52c67d7a7 feat(captions): first pass on recording
Add the main server-side adapter for using the legacy closed captions
recording process with the audio captions data.
2022-07-20 17:20:52 +00:00
Pedro Beschorner Marin
aa7f2abae2 feat(captions): talking users' floor
Associate the floor status with the talking users to improve feedback
on automated closed captions.
2022-07-20 17:20:51 +00:00
Pedro Beschorner Marin
1064aab84d style(captions): add background alpha
Add some transparency to the background color.
2022-07-20 17:20:51 +00:00
Pedro Beschorner Marin
fb48e61d6d feat(captions): add talking indicator feedback
Inform other users about who are the current talkers with the speech
recognition enabled.
2022-07-20 17:20:51 +00:00
Pedro Beschorner Marin
df184b542c feat(captions): add unsupported warning
Add a disclaimer for users on browsers that do not provide speech synthesis'
voices.
2022-07-20 17:20:51 +00:00
Pedro Beschorner Marin
307ab6ea40 feat(captions): add author avatar
Include captions' author avatar at the live feedback to improve identification.
2022-07-20 17:20:50 +00:00
Pedro Beschorner Marin
d00909751a refactor(captions): change getVoices routine
In some cases, `getVoices` returns an empty array even if the browser's vendor
has full support for speech synthesis. Add a trigger call to initiate the
voices fetching process.

As drafted, `getVoices` can be an asynchronous call and monitoring it
depends on the support of a `voiceschanged` event. Although many of the
main vendors support voices, this event is not (yet) by Safari.

https://wicg.github.io/speech-api/#dom-speechsynthesis-getvoices
https://wicg.github.io/speech-api/#eventdef-speechsynthesis-voiceschanged
https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesis/voiceschanged_event
2022-07-20 17:20:50 +00:00
Pedro Beschorner Marin
d6dc66f57e feat(captions): language selector
Replace the checkbox with a selector up with 3 languages: en-US, es-ES and pt-BR.

Add setting option to enable by default with predetermined locale.
2022-07-20 17:20:50 +00:00
Pedro Beschorner Marin
5671bd7d3c fix(captions): check for voices
Avoid enable audio transcription if the browser's vendor does not provide
voices data.

This should prevent false positives for browsers such as Chromium and
Brave.
2022-07-20 17:20:49 +00:00
Pedro Beschorner Marin
e131925370 style(captions): font-family fallback
Add extra `sans` alternatives
2022-07-20 17:20:49 +00:00
Pedro Beschorner Marin
d688a2572f fix(captions): remove padding when no content
Avoid rendering the captions wrapper's padding when there's no data to
be displayed at the live feedback.
2022-07-20 17:20:49 +00:00
Pedro Beschorner Marin
f726cb0ad5 refactor(captions): UX improvements
Parse the audio transcript before broadcasting it's content back to the
client and the recording actor. Limiting by 8 words per line and max of
2 lines to avoid CPU intensive operations over this recurring event.

Replace Calibri font family with Verdana to improve character spacing,
add relative sizing to the text content and a background padding.
2022-07-20 17:20:49 +00:00
Pedro Beschorner Marin
75969ec93c feat(captions): audio captions app
Add a server-side app for the audio captions feature and record proto-events
for this data.

As it is, only behaves as a pass-through module. The idea is to include all
the business intelligence in this app.
2022-07-20 17:20:48 +00:00
Pedro Beschorner Marin
0bc730b3e3 refactor(captions): improve recovery
Use the user's talking state to trigger a speech API recovery after long
periods of silence.
2022-07-20 17:20:48 +00:00
Pedro Beschorner Marin
944edf2ccf feat(captions): web speech prototype
Hardcoded pt-BR prototype for closed captions generated by the browser's
WebSpeech API.
2022-07-20 17:20:48 +00:00
Ramón Souza
c3dd17f913
Merge pull request #15406 from KDSBrowne/v2.6.x-fix-fs-cursor
Cursor Position Update For Video Focus Layout
2022-07-20 17:10:30 +01:00
KDSBrowne
100bfcfb86 update fullscreen element check 2022-07-19 10:51:59 +00:00
KDSBrowne
ed55d58b01 handle cursor positions for video focus 2022-07-19 03:37:47 +00:00
KDSBrowne
47303757de handle cursor position for fullscreen presentation 2022-07-19 01:23:10 +00:00
Localization Server
1539f402c7 chore: Pulled the latest 2.6 HTML5 locales from Transifex 2022-07-18 16:58:19 +00:00
Paulo Lanzarin
a156db25e4
Merge pull request #15358 from frankemax/fix-audio-infinite-joining
fix(audio): prevent race condition when joining audio
2022-07-15 15:24:14 -03:00
Paulo Lanzarin
383fe89653
Merge pull request #15364 from prlanzarin/u26/refactor/kutils-kboom
refactor: remove kurento-utils-js
2022-07-15 14:53:10 -03:00
prlanzarin
d06ae5ce40 fix: add default gUM factory to new peer wrapper
For scenarios such as preview-less camera sharing, old echo test
2022-07-15 17:07:15 +00:00
Anton Georgiev
f2e25768c7
Merge pull request #15390 from ramonlsouza/merge25-26-jul14
chore: Merge v2.5.3 into v2.6
2022-07-15 11:43:17 -04:00
Ramon Souza
83d9feb9c0 fix missing import in audio controls 2022-07-15 11:41:11 -03:00
prlanzarin
cd9b8ccef4 fix: handle Firefox stream cleanup edge case (peer.js)
Firefox doesn't fire the ended evt/onended callback for live
 MediaStreamTrack(s). We rely on that event.

 Manually emit the ended event which works with the onended callback
 when a track is stopped
2022-07-15 14:25:37 +00:00
Ramon Souza
d08ff18add Merge remote-tracking branch 'upstream/v2.6.x-release' into merge25-26-jul14 2022-07-15 11:12:25 -03:00
Ramon Souza
db5ac1428a Merge tag 'v2.5.3' into merge25-26-jul14 2022-07-15 11:08:02 -03:00
prlanzarin
72c0b23b9d refactor: remove kurento-utils from compatibility
Phased out.
2022-07-15 14:00:12 +00:00
prlanzarin
45049cbd65 refactor: swap kurento-utils for new peer wrapper in screen sharing and audio 2022-07-15 14:00:12 +00:00
prlanzarin
6a4ceac61f refactor(webcam): swap kurento-utils for the new peer wrapper 2022-07-15 14:00:12 +00:00
prlanzarin
7281475271 feat: add new peer wrapper to phase out kurento-utils
kurento-utils is unmaintained. It's served us well, but its age
shows.  We need to transition to something else if we want to
have better  maintainability and include simulcast, multistream, ...

This introduces a simplified/leaner wrapper kit that's almost
API-compatible with what we use right now - so widespread changes
are minimal). It's easier to maintain/read/transition from. This
can be read as an intermediate step to transitioning to
something definitive (ie mediasoup-client).
2022-07-15 13:51:03 +00:00
Ramón Souza
267a6f5c9c
Merge pull request #15203 from ramonlsouza/improve-ghost-button-border
style: improve ghost buttons border in low density display
2022-07-15 12:06:59 +01:00
Ramón Souza
da848e990a
Merge pull request #15320 from gabriellpr/dark-light-buttons
Styling light and dark buttons
2022-07-13 21:03:46 +01:00
gabriellpr
1c2f9e9cd9
Update styles.js 2022-07-13 16:35:02 -03:00
Paulo Lanzarin
3e8ccd370d
Merge pull request #15375 from prlanzarin/u26/fix/local-echo-gUM-block
fix(audio): block audio join while gUM isn't resolved (local echo)
2022-07-13 15:57:02 -03:00
gabriellpr
49f12c43af
Merge branch 'v2.6.x-release' into dark-light-buttons 2022-07-13 15:36:31 -03:00
prlanzarin
27896e74e6 fix(audio): block audio join while gUM isn't resolved (local echo)
The new local echo view doesn't block the "Join audio" button while
awaiting for getUserMedia permission to be granted/denied. That may
cause unexpected behavior when unattentive users just click "Join audio"
without granting or denying gUM.

This commit accounts for gUM resolution when deciding whether to block
the "Join audio" button. It also includes an extra "isConnecting" check
to it to avoid spam-clicking issues.
2022-07-13 14:35:37 +00:00
Ramón Souza
04386dfcd5
Merge pull request #15212 from gabriellpr/styling-buttons
style: Revised styles for our secondary buttons
2022-07-13 15:22:49 +01:00
Ramón Souza
a0f80e0efb
Merge pull request #15353 from gabriellpr/contrast-record-indicator
Improving contrast on recording indicator
2022-07-13 15:20:36 +01:00
prlanzarin
7b76d2e88f fix: change local echo Stop hearing locale to Stop audio feedback 2022-07-13 13:59:49 +00:00
Anton Georgiev
1393e4a7de
Merge pull request #15352 from germanocaumo/tldraw-z-f
fix(tldraw): initial/viewer zoom fixes + presToolbar height
2022-07-13 09:42:36 -04:00
Anton Georgiev
70c084bfdb
Merge pull request #15300 from JoVictorNunes/fix-focus-loss-Jun-30
fix: disable Tldraw autofocus
2022-07-13 09:39:06 -04:00
Paulo Lanzarin
64af188f1b
Merge pull request #15371 from prlanzarin/u26/fix/webcam-oninactive
fix(webcam): handle stream inactivation/gUM revocations
2022-07-13 10:37:36 -03:00
Anton Georgiev
20762cfac6
Merge pull request #15310 from germanocaumo/tldraw-production-fix
fix(tldraw): fix shapes not persisting in production +
2022-07-13 09:36:44 -04:00
prlanzarin
029c957b22 fix(webcam): handle stream inactivation/gUM revocations
The 'inactive' event is fired whenever the stream gets inactive (ie it
cannot be used anymore), and there are scenarios where that is
unexpected behavior and must be handled accordingly.
The main example of that is when gUM permissions are revoked by the user
via the browser's permission management panel.
Since MediaStream/Track inactive events aren't being handled in such
scenarios, what actually happens is that the camera just freezes without
further indication why.

This commit handles those scenarios in both video-preview and
video-provider by:
  - 1) correctly stopping the camera (provider)
  - 2) surfacing a toast (provider) or error indication (preview)
2022-07-12 22:03:01 +00:00
prlanzarin
e93440e15a refactor(webcam): fix linter errors in BBBVideoStream 2022-07-12 17:50:16 +00:00