Commit Graph

46 Commits

Author SHA1 Message Date
Tainan Felipe
fb0b98a6af Hotfix: Audio captions not working properly 2023-10-06 18:13:55 +00:00
Anton Georgiev
cd9f93be44 Merge remote-tracking branch 'bbb/v2.6.x-release' into merge-july12 2023-07-12 15:59:46 -04:00
Gabriel Porfirio
8f52121d5c changes on the speech-recognition test 2023-06-30 17:16:22 -03:00
Paulo Lanzarin
171c7a6587
Merge branch 'v2.7.x-release' into v2.7.x-release 2023-05-26 09:27:14 -03:00
KDSBrowne
1b51377147 fix getContentAnchorEl console error 2023-05-10 01:31:48 +00:00
Arthurk12
3b871e5ca2 fix(captions): "not supported" in chrome
Fixes a case where the locale selector don't show up in Chrome when using
'webspeech' provider.
And adds missing fields to the webspeech transcription messages, after the
addition of some new parameters to those messages with the open
transcription server.
2023-04-25 10:25:20 -03:00
Arthurk12
eafa0f200e feat(captions): no longer writes in the pad
This feature was too coupled to the old closed captions' pads.
(e.g. the old closed captions feature should be enabled for this
 to work properly)
Some things were hardcoded and others didn't make sense from the
user experience perspective.
Reverts #876d8aa.
Partially reverts #802964f, removes changes to make closed captions'
pads compatible with live-transcription but keeps provider settings.
2023-04-25 10:24:48 -03:00
prlanzarin
54b6578b03 fix(audio): forcefully disable stereo when using Vosk transcription
The current Vosk CC provider does not support stereo mic streams
(pending investigation as to why).

This commits makes sure stereo is forcefully disabled via SDP munging
only when transcription is active and using Vosk. Having it disabled
in the server side (FreeSWITCH) is not enough because the stereo parameter
is client mandated and replicated by FS on its answer. So we need to
make sure it's always disabled for the time being.
SFU audio does munging server side (and stereo is always off), so no changes
needed there.

The rest of the providers (except WebSpeech) need to be validated against
stereo audio as well.
This is also intended to be temporary - ideally this needs to be fixed in
mod_audio_fork/Vosk/wherever this is breaking.
2023-04-25 10:10:39 -03:00
Lucas Fialho Zawacki
fee6ff026a feat(captions): Use setUserSpeechLocale as an akka event and catch it in the transcription manager 2023-04-25 09:54:34 -03:00
Lucas Fialho Zawacki
6979432c36 feat(transcription): Server side open source transcriptions 2023-04-24 18:23:34 -03:00
Ramón Souza
d181eba1c2 replace lodash.throttle with native function 2023-03-02 10:25:08 -03:00
Ramón Souza
41c187d93e Merge remote-tracking branch 'upstream/v2.6.x-release' into lodash-radash 2023-03-01 15:19:12 -03:00
Ramón Souza
0a622eff32 replace lodash throttle with standalone package 2023-03-01 15:13:29 -03:00
Ramón Souza
dd710aa96f replace lodash uniq and uniqBy 2023-02-23 10:44:29 -03:00
Gabriel Porfirio
907589cef1 disabled features test 2023-02-21 15:28:53 -03:00
GuiLeme
d566d1fd1a [patch-issue-16375] - add semicolon 2023-01-17 15:27:04 -03:00
GuiLeme
065e7f17e7 [patch-issue-16375] - Patch for SonarCloud warning 2023-01-17 14:50:10 -03:00
GuiLeme
bdc5a71967 [issue-16375] - not showing anything when liveTranscription is disabled 2023-01-16 18:10:21 -03:00
GuiLeme
b212a17c55 [issue-16375] - Change message for live transcription not being activated 2023-01-12 15:33:56 -03:00
GuiLeme
f67f530b32 [disabled-transcription] - Renamed audioCaptions to liveTranscription (for disabledFeatures) 2023-01-09 10:47:22 -03:00
GuiLeme
b4afec689e [disabled-transcription] - Created new disabledFeature audioCaptions 2022-12-16 17:04:14 -03:00
Ramón Souza
9b0cff367a fix conditional hooks - captions select 2022-11-22 09:20:14 -03:00
Ramón Souza
f2a6fc7694 fix conditional hooks - captions button 2022-11-22 09:16:26 -03:00
Arthurk12
0ede0fa0e9 feat(captions): change transcription language on the fly
Adds a chevron button that allows enabling/disabling and changing the
language of the automatic transcription.
2022-08-24 15:40:03 +00:00
Ramón Souza
9e219d9cb3 adjust cc icon position 2022-08-02 17:33:53 -03:00
prlanzarin
6c8b097eba fix: add option to disable transcription in mobile, extend support check
Mobile endpoints are flaky with the WebSpeechAPI:
  - iOS versions that support it are borking our outbound audio when it's
    enabled
  - Android speech recognition has flaky locale detection and speech
    transcription
Additionally: the support check is not checking the WebSpeechAPI
availability properly, so older devices (eg iOS 12) are flagged as
supported even though they aren't.

This commit adds a configuration flag (public.audioCaptions.mobile) to
control transcription availability on mobile. False by default.
Also extends the setSpeechVoices support check and
hasSpeechRecognitionSupport method to prevent false positives.
2022-07-20 17:20:54 +00:00
Arthurk12
e93b9739de style(captions): adds border to toggle button 2022-07-20 17:20:53 +00:00
Arthurk12
c96b53093c feat(captions): adds locale settings
Adds two new flags to the settings file which change the way the locale
flag is used:

- forceLocale: (true/false) => If true, enforces the transcription
  language to be the locale content field and jumps the language
selector
  in audio modal.
- defaultSelectLocale: (true/false) => If true, the default selected
  value in the dropdown language selector in audio modal will be defined
  by the locale content field.

In any case, if the locale flag holds an invalid value, it defaults to
disabled.
2022-07-20 17:20:53 +00:00
Arthurk12
da9adca229 fix(captions): talking indicator icon
Prevents the speech recognition from being initialized when the closed
captions feature is disabled.
2022-07-20 17:20:52 +00:00
Pedro Beschorner Marin
d553ca65cf feat(captions): use navigator language
If not set to use the default language, try to select the navigator
language as speech default locale.
2022-07-20 17:20:52 +00:00
Pedro Beschorner Marin
116c0d9a49 fix(captions): filter duplicated languages
Avoid multiple instances of the same language at the voices data.
2022-07-20 17:20:52 +00:00
Pedro Beschorner Marin
51eeb092b3 refactor(captions): configurable languages
Move the language collection to the HTML settings file. This data defines
the available languages available for the speech API.

These language tags are used to filter SpeechSynthesis' API `getVoices`
result. Tags must use BCP 47 format.

https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesisVoice/lang
2022-07-20 17:20:52 +00:00
Pedro Beschorner Marin
b52c67d7a7 feat(captions): first pass on recording
Add the main server-side adapter for using the legacy closed captions
recording process with the audio captions data.
2022-07-20 17:20:52 +00:00
Pedro Beschorner Marin
1064aab84d style(captions): add background alpha
Add some transparency to the background color.
2022-07-20 17:20:51 +00:00
Pedro Beschorner Marin
fb48e61d6d feat(captions): add talking indicator feedback
Inform other users about who are the current talkers with the speech
recognition enabled.
2022-07-20 17:20:51 +00:00
Pedro Beschorner Marin
df184b542c feat(captions): add unsupported warning
Add a disclaimer for users on browsers that do not provide speech synthesis'
voices.
2022-07-20 17:20:51 +00:00
Pedro Beschorner Marin
307ab6ea40 feat(captions): add author avatar
Include captions' author avatar at the live feedback to improve identification.
2022-07-20 17:20:50 +00:00
Pedro Beschorner Marin
d00909751a refactor(captions): change getVoices routine
In some cases, `getVoices` returns an empty array even if the browser's vendor
has full support for speech synthesis. Add a trigger call to initiate the
voices fetching process.

As drafted, `getVoices` can be an asynchronous call and monitoring it
depends on the support of a `voiceschanged` event. Although many of the
main vendors support voices, this event is not (yet) by Safari.

https://wicg.github.io/speech-api/#dom-speechsynthesis-getvoices
https://wicg.github.io/speech-api/#eventdef-speechsynthesis-voiceschanged
https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesis/voiceschanged_event
2022-07-20 17:20:50 +00:00
Pedro Beschorner Marin
d6dc66f57e feat(captions): language selector
Replace the checkbox with a selector up with 3 languages: en-US, es-ES and pt-BR.

Add setting option to enable by default with predetermined locale.
2022-07-20 17:20:50 +00:00
Pedro Beschorner Marin
5671bd7d3c fix(captions): check for voices
Avoid enable audio transcription if the browser's vendor does not provide
voices data.

This should prevent false positives for browsers such as Chromium and
Brave.
2022-07-20 17:20:49 +00:00
Pedro Beschorner Marin
e131925370 style(captions): font-family fallback
Add extra `sans` alternatives
2022-07-20 17:20:49 +00:00
Pedro Beschorner Marin
d688a2572f fix(captions): remove padding when no content
Avoid rendering the captions wrapper's padding when there's no data to
be displayed at the live feedback.
2022-07-20 17:20:49 +00:00
Pedro Beschorner Marin
f726cb0ad5 refactor(captions): UX improvements
Parse the audio transcript before broadcasting it's content back to the
client and the recording actor. Limiting by 8 words per line and max of
2 lines to avoid CPU intensive operations over this recurring event.

Replace Calibri font family with Verdana to improve character spacing,
add relative sizing to the text content and a background padding.
2022-07-20 17:20:49 +00:00
Pedro Beschorner Marin
75969ec93c feat(captions): audio captions app
Add a server-side app for the audio captions feature and record proto-events
for this data.

As it is, only behaves as a pass-through module. The idea is to include all
the business intelligence in this app.
2022-07-20 17:20:48 +00:00
Pedro Beschorner Marin
0bc730b3e3 refactor(captions): improve recovery
Use the user's talking state to trigger a speech API recovery after long
periods of silence.
2022-07-20 17:20:48 +00:00
Pedro Beschorner Marin
944edf2ccf feat(captions): web speech prototype
Hardcoded pt-BR prototype for closed captions generated by the browser's
WebSpeech API.
2022-07-20 17:20:48 +00:00