Actions

ProSBC Troubleshooting Guide

Hide TOC

Call Drops Problems

My call dropped while I was talking, I might hear fast busy or just dead air.

  • Call dropping within the first minute – missing ACK in Invite. Check the NAT configuration. NAT
  • Maximum Call time exceeded. Many service providers set a limit on the maximum duration for any call passing through their system. Double-check with your service provider.

Calls dropping at specific intervals (10 minutes, 30 minutes)

    • Check the SIP session timers. [[SIP_session_timers|SIP Session timers]
    • Uncheck “Use Session Timer” from SIP configuration. SIP -> Select the name from SIP Configuration menu -> Session Timers

  • Call dropping within the first minute – missing ACK in Invite. Check the NAT configuration. NAT
  • Maximum Call time exceeded. Many service providers set a limit on the maximum duration for any call passing through their system. Double-check with your service provider.

Calls dropping because of Congestions

The calls can be dropped because of the congestion too. You can check the NAP status for congestions.

If you are receiving a congestion error,

  • You first need to check the concurrent call counts. If your concurrent call count exceeded the license, you would have a congestion error. Please check your license.
  • Check if you set any Call Rate Limiting in the NAP. Go to NAPS -> Select The NAP -> Advanced Parameters -> Call Rate Limiting
  • For other congestion problems please contact support@telcobridges.com

Log procedure for Signaling/Routing problems

  • Set trace levels to 1 using by tbx_cli_tools_remote command
    • Connect to ssh and use tbx_cli_tools_remote command
    • Please check the following link for how to set log levels on the application Application_trace_level
    • Select the applications (application names will be given by Telcobridges Support. Mostly gateway, toolpack_engine, and tbsyslog application logs are needed) one-by-one
    • Type "T" and "1" to set the log level
    • When you finished the test call please disconnect from the application, and press Escape twice.
    • Do this for all the applications given by Telcobridges support.
    • You can get more detail about how to use tbx_cli_tools_remote from the following link How_to_use_tbx_cli_tools_remote_program
  • Run tbsigtrace capture. For more details please check the following link Toolpack_Debug_Application:Tbsigtrace
  • Make a test call or simulate the problem
  • Stop the tbsigtrace capture
  • Export the call trace. For more details please check the following link Toolpack:Retrieving_Call_Trace_C
  • Generate a tbreport for the date/time you made the test call For more details please check the following link TBReport

Log procedure for Voice/RTP problems

  • Set trace levels to 1 using by tbx_cli_tools_remote command
    • Connect to ssh and use tbx_cli_tools_remote command
    • Select the application (application names will be given by Telcobridges Support. Mostly gateway, toolpack_engine, and tbsyslog application logs are needed)
    • Please check the following link for how to set log levels on the application Application_trace_level
    • Type "T" and "1" to set the log level
    • When you finished the test call please disconnect from the application, and press Escape twice.
    • Do this for all the applications given by Telcobridges support.
    • You can get more detail about how to use tbx_cli_tools_remote from the following link How_to_use_tbx_cli_tools_remote_program
  • Run tbrouter capture. For more details please check the following link VoIP_Ethernet_Capture_FreeSBC_A
  • Make a test call or simulate the problem
  • Stop the tbrouter capture
  • Export the call trace. For more details please check the following link Toolpack:Retrieving_Call_Trace_C
  • Generate a tbreport for the date/time you made the test call For more details please check the following link TBReport


Sub-optimal config sbc list warning

The reason for this is that some Nics (Azure, vmxnet3, e1000, ixgbe vf) report a RETA size of 0 while reporting multiple rx queues, meaning they can not support multiple CPUs. However TBRouter is not aware of this limit on those Nics then initiate the Cores on the Nics

Please check the following link for how to solve this issue Toolpack: How to Get Rid of Sub Optimal

Scheduling Problems

You may see the SBC tab in the general status as yellow. When you select it:

Status -> SBC

You may see “Scheduling problem sbc list” with the hostname of the ProSBC. If you select it, you may see “Scheduling problem alarm” set to “true”
Status containing “scheduling” problem may point to:

  • Non-dedicated (or not ‘pinned’) CPU [Open-stack]
  • Not enough CPU reservation [Vmware]
  • Memory is not dedicated to the Virtual Machine (VM)
  • Too many active virtual machines fighting for resources on the host
  • On KVM based installations (proxmox,virtmanager) CPU type must set to “host” and numa must be disabled

You need to be sure other VMs are not taking resources from ProSBC VM instance. Follow requirements shown here: ProSBC requirements

Registration Errors

Endpoint sends the registration request to ProSBC however the ProSBC is not forwarding it to the registrar

  • Check if SIP domain configured correctly

Creating a SIP domain

  • Check the SIP domain Status. Be sure the domain registrar can be reached by ProSBC
 Go to Status -> SIP -> SIP Domain -> Status -> SIP Registration Domains
  • Check if the SIP client sends the correct Domain name to ProSBC. You can capture a SIP trace (see Signaling trace capture tool ) and use Wireshark to analyze the trace. Look at the “To:” SIP header: it must match what is in the Sıp Domain configuration of the ProSBC.

ProSBC forwards incoming registration messages to the registrar but the registrar returns an error

  • Check if SIP domain configured with correct registrar setting

Creating a SIP Registrar

  • Check if Registrar NAP configured with a correct IP address

Allocating a SIP NAP

  • Check if your client configured with the correct username and password
  • Check the forwarding modes on ProSBC, and select the correct one

Forwarding Modes

ProSBC always modifies the contact URI in SIP register requests to remain on the path between SIP User Agents and registrars. ProSBC supports two different SIP registration forwarding modes (i.e. "Contact Remapping" or "Contact Passthrough").

  • The "Contact Passthrough" forwarding mode makes the contact username portion of the Contact URI in SIP register requests to pass through unchanged.
  • The "Contact Remapping" forwarding mode modifies the contact username portion of the Contact URI in SIP register requests and makes it unique.

Forward original headers from incoming request

If your registrar wants to receive original headers from incoming requests, Forward the domain without any modification in From/To/Contact/P-Asserted-Identity SIP headers. If you want to forward incoming SIP from/to header domain to the outgoing leg you need to upgrade your ProSBC to the minimum Release 3.0.114. With Release 3.0.114 we added the forward_sip_domain script to our routing scripts.

This filter is used to forward the domain name (or IP address and port) from the incoming call to the outgoing call, for the following SIP headers:

  • from (update call attribute :calling by appending :calling_sip_host and calling_sip_port)
  • to (update call attribute :called by appending :called_sip_host and called_sip_port)
  • P-asserted-identity (update call attribute :private_address by appending :private_address_sip_host and private_address_sip_port)
  • To set up a Filter, the main script needs to be modified. The main script can be either simple_routing.rb, or any other script.

First, go to the routing script section of the Web portal

Gateway -> Routing scripts -> Example Scripts -> simple_routing.rb [Edit]

Three things need to be added. At the start of the script:

require 'forward_sip_domain' unless defined?(ForwardSipDomain)

In the main class:

include ForwardSipDomain
route_remap :method => :forward_sip_domain


The final script will look like this (with possibly other filters also included):

require 'base_routing'
require 'forward_sip_domain' unless defined?(ForwardSipDomain)

class SimpleRouting < BaseRouting
  include ForwardSipDomain
  
  route_match :call_field_name => :called
  route_match :call_field_name => :calling
  route_match :call_field_name => :private_address
  
  route_match :call_field_name => :nap
  route_remap :call_field_name => :called, :route_field_name => :remapped_called
  route_remap :call_field_name => :calling, :route_field_name => :remapped_calling
  route_remap :call_field_name => :private_address, :route_field_name => :remapped_private_address
  route_remap :call_field_name => :nap, :route_field_name => :remapped_nap
  route_remap :method => :forward_sip_domain
  

end
  • This script requires the routes to have a custom column named "forward_sip_domain", type boolean.
Gateway -> Routes -> Create New Route Column

Name: forward_sip_domain
Type attributes: boolean

See ProSBC uses case: Remote Workers

Note: This script is coming as default after Rls. 3.0.116. You just need to set the forward_sip_domain route column as enabled.

One Way Audio/ No Audio Problems

  • Can place a call, but one way or no audio

One-way audio is a common VOIP problem. It is one of the most frequent support questions we receive. There are many possible causes;

  • Outdated firmware in routers, VOIP phones, Firewalls, etc. can cause one-way audio. Ensure you have the latest updates for all the devices in the call path.
  • Firewall mistakenly blocking RTP, be sure firewalls are configured correctly.
  • Particularly if Network Address Translation (NAT) is involved in the call path, configuration of the various devices may be a problem. Please check the link for NAT
  • Another reason for one-way audio is having your system set to offer unsupported codecs within your other SIP systems.
  • Just as each side of a call must send RTP within the same codec, each side must also have the same phase timing (or ptime value). See SDP Description
  • Configured RTP port ranges can cause a problem too. Check ProSBC and endpoints (Clients, SIP Trunks) are using correct RTP ports. See Creating an IP Port Range
  • High one-way packet loss. If sufficient packet loss occurs in one direction on a call, that half of the conversion may break down, but not cause the entire call to drop. Packet loss can occur due to a number of reasons:
    • High utilization on a link with no QoS.
    • Misconfigured interface: Half-duplex or duplex mismatch.
    • Underperforming network devices.
    • Cabling faults.
  • Wrong threshold configuration under SBC Advanced parameter can cause ProSBC to block RTP-Audio by the firewall due to thresholds. You can see the following error in your call trace;


Please double check your thresholds configuration from;

SBC -> Advanced parameters
  • For Two-way no audio, sometimes phone/phone system is behind the external firewall, the user may find that if they just establish the call, two-way audio is fine, but if the call is placed on hold for a certain time, and when resuming the call, there is no more two-way audio. This could be due to the external firewall is set to shutdown the RTP path after a certain time of being idle (silence) as in the case of call on hold. The solution is to optimize this setting at an external firewall to prevent it from shutting down the RTP over a reasonable or defined time.

DTMF Problems

If you can see DTMF received from one leg and sent to the other leg but not working, you need to check the UDP Checksum errors. To set Wireshark to detect UDP Checksum Errors you need to Enable Validate the UDP checksum if possible in the Preferences. To enable that parameter to go to

Edit -> Preferences -> Protocols -> UDP



After enabling this parameter you must see the checksum errors if it exists. To solve this problem you need to add a string/ or modify the existing string in the nic.json file. nic.json file is located under the folder /usr/lib/tb/toolpack/setup/12358/3.1/. Before doing any change please make a copy of the existing file.

You can open the file with vim editor and add the following string to dpdk controlled nics, voip nics.

"software_ip_checksum": true,

before the strings;

"tx_fifo_size": 8192,
"tx_kni_coalesing_loops": 1024,


If you already have software_ip_checksum parameter and it is set to false, please set it as true.

After this change, you need to restart the tbrouter application or reboot the server.


SBC accept entries and fragmentation

When a network has a lot of fragmented packets (data too large to fit in one packet), it may lead to issues with the ProSBC mechanism to prevent DDOS attacks.
The effect is that some valid packets are dropped because the list of valid entries is full.
tbrouter log files can show how many Accept Entries are currently in use. You can search for the following terms in the logs:

 "Accept entries"

Or if a lot of fragmented packets are received. You can search for the following terms in the logs:

 "nbFragPkt"
 "Frag entries"


You can change this value here in the configuration:

 SBC -> Advanced Parameters -> List sizes -> Maximum accept entries: 8000

This setting will only affect memory used, so it can be increased as necessary. It will use approximately 1MB per 1000 entries.
(reference trk#24706)

List full SBC

This is seen on the main status page of the web portal, SBC section. The "list full sbc list" happens when either:

  • The accept entries are full
 This is the maximum number of different sources that can connect to the system simultaneously.
 The default maximum "accept" entries is 4000. This can be changed in SBC -> Advanced Parameters -> List Sizes
  • The RTP entries are full
 This is the maximum number of RTP connections that you can have on the system.
 This limit is set with the number of sessions licence.
  • The drop entries are full
 Drop entries are used to protect against a DDOS attack. These sources of information are flagged as "drop" when they do not match any valid entry from the system. If this limit is reached and the CPU usage is high, the DDOS mode will be activated.
 The default maximum "drop" entries is 2000. This can be changed in SBC -> Advanced Parameters -> List Sizes


(reference trk#24277)