CAF: Working With Cmc Mixers
Audio Mixers Definition
Audio mixers are used to mix audio from multiple call legs and provide a mixed output to each call leg. At any time during a call flow the application can join call legs to one (or multiple) audio mixers.
TMedia concepts and density
Please read the following page for general information about TMedia Mixers: Audio Mixers
An audio mixer is represented by an instance of the CTBCMCMixer class. Call legs (CTBCMCLeg) can be joined to this audio mixer as speakers, listeners, or both.
In addition to call legs joining/unjoining, the CTBCMCMixer class allows some media functions such as playing audio files to all listeners of the mixer, or recording all active speakers of the mixer.
Other member functions are available to retrieve (and change in some cases) the mixer attributes, and the way different call legs are joined to the mixer (speaker/listener)
It is important to note that this class is protocol-agnostic and can handle any type of supported call legs (e.g. SIP/VOIP, ISDN, SS7, Media only, etc).
- Do not confuse the base class CTBCMCMixer with the class CTBCAFMixer. The CTBCAFMixer class is an implementation class specialized for use by the ITBCAFCallFlow interface when dealing with multiple legs and mixers.
Command Flow for Mixer Actions in CAF
All actions requested on a mixer are executed asynchronously. For each action DoSomething() on a mixer, a corresponding OnSomethingResponse() event will be received on the mixer once Toolpack starts processing the command. If the OnSomethingResponse() function does not handle the response (the default implementation is empty) and the result of the action was a failure, the default error handling function OnMixerError() will be called. In some cases, when the action is completed, an event will be received on the mixer (OnSomethingDone() for example).
For example, to play an audio file on the mixer (heard by all legs listening to this mixer), you call MixerPlayStream(). It will almost immediately receive the OnMixerPlayStreamResponse() event indicating that Toolpack has received the command. At the same time this event is called, Toolpack will start configuring the resources to play on the mixer. Once the play has started, the user will receive an event OnMixerStreamPlayingStarted(), and when it is finished playing (or failed to start), the user will receive an OnMixerStreamPlayingDone() event.
Creating a mixer is always done through the definition of a mixer attribute. The values entered in the mixer attribute will define how the mixer behaves, and will influence resource reservation.
Preparing Mixer Attributes
The class CTBCMC_MIXER_ATTRIBUTE defines available attributes to configure an audio mixer. Default parameters are provided for standard usage, though parameters may be changed to control the behavior more precisely. Example mixer attributes are:
- SetReserveListenerSlot / IsReserveListenerSlot
Choose if a 'listener-only' slot must be reserved on this mixer (allocated even when no call leg is joined as listener-only). If disabled, the listener-only slot is added dynamically upon joining of first 'listener-only' leg (note that only one listener-only slot is required if there are more than one listener-only legs).
Default is TBX_FALSE (no listener-only slot reservation).
- SetNbReservedActiveSpeakerSlots / GetNbReservedActiveSpeakerSlots
Choose the number of speaker slots to reserve on this mixer. The mixer will always have a minimum number of speaker slots ready, though it can be more if joining more than the number of speaker call legs.
Default is 0 (no reserved active speaker slot).
Maximum is 12 active speakers (though there can be unlimited number of "inactive speakers", waiting voice activity detection to take the place of the oldest active speaker among the 12).
- SetResizeDownTimer / GetResizeDownTimer
Choose the delay before reducing the size of a mixer (removing unnecessary IVR or TDM slots).
Smaller delay reduces MIPs usage faster, but increases the number of times mixers need to be re-sized, and thus increases CPU usage on the TMedia Unit.
A larger delay allows, for example, resizing the mixer only once if multiple users leave the conference (un-joined from the mixer) almost at the same time.
Default is 15,000 (15 seconds).
- SetNbDominantSpeakers / GetNbDominantSpeakers
Choose the maximum number of active "dominant" speakers (maximum number of audio sources mixed together at a given time, based on loudest speakers).
Default is 3. Max is 12.
- SetNoiseFloor / GetNoiseFloor
Choose the noise floor (for voice activity of inactive speakers, before they are considered active speakers).
Default is -30dbm. Min / max is -94dbm / 0dbm.
- SetNoiseSuppressionLimit / GetNoiseSuppressionLimit
Choose the noise suppression limit (for removal of background noise from active speakers).
Default is 30dbm. Min / max is 0dbm / 95dbm.
- SetTargetPower / GetTargetPower
Choose the target power for conference output. Conference output level will be adjusted (amplified up to GetGainLimit, dampened up to GetLossLimit) to match the target power.
Default is -5dbm. Min / max is -50dbm / 0dbm.
- SetLossLimit / GetLossLimit, and SetGainLimit / GetGainLimit
Choose the loss limit / gain limit (limiting the audio level adjustments allowed from speaker legs to reach the target power)
Default loss limit is -10dbm. Min / max loss limit is -23dbm / 0dbm.
Default gain limit is 5dbm. Min / max gain limit is 0dbm / 23dbm.
- SetReserveIvrSlot / IsReserveIvrSlot
Choose if a IVR slot must be reserved on this mixer (allocated even when not playing/recording). If disabled, IVR slot is add it dynamically when needed.
Default is TBX_FALSE (no IVR slot reservation).
- SetRecordPlayback / IsRecordPlayback
Choose if mixer recording will "hear" the mixer audio playback or not.
Enabling this options will cause the mixer to use 2 IVR slots.
Disabling this option will cause the mixer to use 1 IVR slot for both play and record, but record won't include playback audio.
Default is TBX_FALSE (record don't include playback, use only 1 IVR slot)
Note: Please refer to header file tbcmc_mixer.hpp for a more complete documentation on all available mixer attributes.
Once a mixer is created, Toolpack will manage tge reservation of DSP resources on a TMedia, and will manage connecting legs to the mixer (listeners, or speakers). When call legs are joined or unjoined from the mixer, Toolpack will automatically resize the DSP resource if appropriate (with a minimum size determined by the 'reserve' mixer attributes); the application does not have to deal with resource management distribution among TMedia units.
For more information about the capacity of TMedia units (in regard to the number of conference "slots"), please refer to this page: Audio Mixers -> Resource usage
The following code snippet demonstrates how to build the attributes.
PTRCTBCMC_MIXER_ATTRIBUTE ptrMixerAttribute; ptrMixerAttribute = tbnew CTBCMC_MIXER_ATTRIBUTE(); ptrMixerAttribute->SetReserveIvrSlot( TBX_TRUE ); ptrMixerAttribute->->SetReserveListenerSlot( TBX_FALSE ); ptrMixerAttribute->SetNbReservedActiveSpeakerSlots( 3 ); ptrMixerAttribute->SetNbDominantSpeakers( 3 ); ptrMixerAttribute->SetNoiseFloor( TBCMC_VAD_NOISE_FLOOR_LEVEL_MINUS_30_DBM ); ...
Creating the Mixer
Once the mixer attributes have been filled, creating the mixer in Toolpack is only a matter of creating a CTBCMCMixer object and calling MixerCreate() on it. At that moment a message is sent to Toolpack to allocate the mixer on the hardware.
pCallMixer = new CTBCMCMixer( 0 /* or MixerId from OnMixerSync() */, ptrMixerAttribute, this, &mLegMutex ); pCallMixer ->MixerCreate();
Once the mixer has been allocated on hardware, the function OnMixerCreated() will be called.
Terminating a mixer is a 2-step process:
- Call MixerTerminate()
- Ask for mixer object free upon OnMixerTerminated() callback function
When MixerTerminate() is called, the Toolpack framework will start destroying the mixer (including unjoining any call leg that was joined to it). Once the mixer hardware resources have been deallocated, Toolpack will call OnMixerTerminated(). Receiving that event means that the mixer no more exists in Toolpack, and the object can thus be freed. Any time after OnMixerTerminated() (or within that function), the application is responsible to call method FreeMixer(), which will eventually cause the mixer's ITBCMCFreeListener interface function Free to be called. The ITBCMCFreeListener to use can be specified either in the CTBCMCMixer constructor and by calling SetFreeListener(). The mixer can be it's own FreeListener, in which it just 'delete' itself. An application specific FreeListener can be used for cases where other action needs to be taken before the mixer is deleted. An example could be the case where the mixer object was allocated from a pool and it should be returned to that pool instead of being deleted.
In some situations, it is possible that mixer termination be initiated by Toolpack. This can happen in the event of sudden media resource unavailability on hardware, or if the application gets disconnected from the Toolpack engine. In that cases, the callback OnMixerTerminated() is called. On reception of this event, the application should also acknowledge termination by calling FreeMixer().
Mixer Synchronization on Failover
When a Toolpack application is restarted, or when a standby application is activated, it goes through a synchronization phase with the Toolpack framework. During this phase, Toolpack informs the application of all the already allocated call legs, links and mixers:
- Callback OnCallLegSync() lists all legs that were retrieved from Toolpack
- Callback OnMixerSync() lists all mixers that were retrieved from Toolpack, along with list of legs joined to each mixer.
- Callback OnLinkSync() lists all joined leg pairs that were retrieved from Toolpack
- Callback OnMixerLinkSync() lists all legs joined to mixers, and mixers joined to mixers.
The course of action to take when receiving a leg or mixer synchronization event depends on the ability of the application to resynchronize or rebuild its own states.
A typical resynchronization scenario would be similar to:
- Application receives all OnCallLegSync() callbacks, for each a new object of type CTBCMCLeg is created
- Application receives all OnMixerSync() callbacks, and for each a new object of type CTBCMCMixer is created.
- Application receives all OnLinkSync() callbacks, and for each re-builds the association between the CTBCMCLeg by calling function SetJoined() of one of the legs.
- Application receives all OnMixerLinkSync() callbacks, and for each re-builds the association between legs and mixers by calling function MixerSetJoined().
- Any leg, link or mixer that the application does not wish to keep is destroyed (RefuseLeg, RefuseLink, RefuseMixer, or RefuseMixerLink)
When all legs, links and mixers have been synchronized, Toolpack sends the OnCmcLibReady() event indicating that active resources synchronization is complete. Event OnSyncDone() is also called on each re-synchronized leg, and OnMixerSyncDone() on each re-synchronized mixer.