mediasoup workers are currently for general use,
regardless of stream type. This makes it difficult to give
different scheduling priorities for audio workers or prevent
noise from video streams, when our goal is to give higher
priority to audio in all ends of the system.
Set `mediasoup.dedicatedMediaTypeWorkers.audio` to `auto`. This will spin up
`ceil((min(nproc, 32) * 0.8) + (max(0, nproc - 32))` mediasoup workers
dedicated to handling audio streams.
* Refactor: Make bundle using webpack
* Fix: restore after install codes and a few settings
* Fix: build script folder permission
* Refactor: Remove support to async import on audio bridges
* Upgrade npm using nvm
* Avoid questions on npm ci execution
* Let npm ci install dev dependencies (as we need the build tools here)
* Fix: enconding
* Fix: old lock files
* Remove: bbb-config dependency to bbb-html5 service, bbb-html5 isn't a service anymore
* Fix: TS errors
* Fix: eslint
* Fix: chat styles
* npm install with "lockfileVersion": 3 (newer npm)
* build: allow nodejs 22
* node 22; drop meteor from CI and bbb-conf
* TEMP: use bbb-install without mongo but with node 22 and newer image
* build: relax nodejs condition to not trip 22.6
* build: ensure dir /usr/share/bigbluebutton/nginx exists
* init sites-available/bbb; drop disable-transparent-
* nginx complaining of missing file and ;
* TMP: print status of services
* WIP: tweak nginx location to debug
* Fix: webcam widgets alignments
* akka-apps -- update location of settings.yml
* build: add locales path for nginx
* docs and config changes for removal of meteor
* Fix: build encoding and locales enpoint folder path
* build: set wss url for media
* Add: Enable minimizer and modify to Terser
* Fix: TS errors
---------
Co-authored-by: Tiago Jacobs <tiago.jacobs@gmail.com>
Co-authored-by: Anton Georgiev <anto.georgiev@gmail.com>
Co-authored-by: Anton Georgiev <antobinary@users.noreply.github.com>
We currently use a simple producer round-robin algorithm to distribute
elements between mediasoup workers. This works for most scenarios but
fails in some edge cases, such as:
- 1-to-N scenarios where N >= ~600-800 (sample number, varies by
single-core performance). This is due to subscribers being pinned to a
producer's worker.
- Poor distribution results from round-robin.
Enable the following new features in bbb-webrtc-sfu via after-install by
default:
- `mediasoup.workerBalancing.strategy: least-loaded`: Replaces
round-robin with load scoring. Workers are selected based on which is
least loaded.
- `mediasoup.enableWorkerTransposing: true`: Allows media streams to be
bridged between workers through internal RTP pipes. This, along with a
per-worker stream limit, enables seamless offloading of streams between
workers (whether publishers or subscribers). The per-worker stream limit
is still under review.
These changes should address the issues mentioned. They are enabled via
after-install because the SFU version is shared with previous BBB versions
where these features are not desirable yet.
- Set bbb-webrtc-recorder as the default `recordingAdapter` in bbb-webrtc-sfu by default
- Set `kurento` in SFU's config to an empty array; Kurento isn't provided by 2.8+ by default
After node-config was bumped to 3.3.9 (from 3.3.6), it started throwing errors if
configurations are mutated without the ALLOW_CONFIG_MUTATIONS env var set.
We mutate some configs directly, but I every time I added one of those I made sure that
they are always deep cloned.
However, we hit an issue with kurento-client mutating a config input, which is an indirect mutation.
So, to prevent further surprises I'm allowing mutations on production while prohibiting them in dev
envs until I'm 100% sure nothing, direct on indirect, improperly mutates configuration values.
bbb-webrtc-sfu (and mediasoup) are running in the CFS scheduler which
means it has to compete with (much) lower priority tasks like
presentation conversion, recording processing, [...]
Since it encompasses an RTC application which also handles audio, it
should be _at least_ on the same scheduling policy as FS/bbb-html5 - and
that should be safer now with mediasoup which has a lower footprint
(and generates lower CPU noise overall).
This commit puts bbb-webrtc-sfu in the FIFO scheduling policy (same as
bbb-html5). Also bumps bbb-html5 nice level up to 18 and sets SFU to
nice 19 (so bbb-html5 has some advantage when push comes to shove).
This can be improved further by using per-process priorities in SFU.
Ideally we'd want mediasoup audio workers and mcs-core to be the same
priority as FS (so higher than bbb-html5), but the rest of them
(video/screen workers) to be the same or lower than bbb-html5. For
future reference:
- https://github.com/bigbluebutton/bbb-webrtc-sfu/commit/3e245122dfa155ecb77b536eeadac1e4607cee
- 66d443d204
Audio's callerId depends on the user name and there isn't
an "on-demand" way of fetching that field internally, making callerId
assembly with trusted attributes (server-side generated) impossible in
bbb-webrtc-sfu.
The new extra header (User-Name, mapped to user_name in the proxied
connection) allows fetching the user name field in a cheap way and
consequently provides a cheap+safe way of assembling the callerId.
Alternatives I've considered but discarded:
- a new akka-apps req-resp pair for fetching the user name (+overhead)
- a new akka-apps req-resp pair for generating the callerId (+overhead)
- piggybacking on GetMicrophonePermissionReq/Resp to generate the
callerId (same overhead, but mixing responsabilities)
* fix unit name: the unit name on Ubuntu is `redis-server.service`
* services which need a working redis require both After= and Wants=
See the description in the `systemd.unit` man page.
yq package is now provided in the BigBlueButton support PPA for BBB 2.5,
so we can depend on the package now. Ensure the dependency is specific
to avoid an incompatible yq version 4 from being installed.
The old 6h values seem far too large and I cant recall nor find any good
justification for them to be that way
Reducing the timeouts to more sane values allow resources (WebSockets) to be
cleaned up faster
The heartbeat routine in bbb-webrtc-sfu runs every 20s. The heartbeat
routine in SIP.js/FS runs every 30(+10)s. The new timeouts are those values
multiplied by 3.
Remaining, to be handles separately:
bbb-html5 before-remove and after-install -- sip.nginx needs to be
handled in bbb-conf
bbb-freeswitch-core -- /tmp/vars xml, etc. -- not sure how to handle
