VoLTE KPIs

I have never planned to talk about such an operator specific matter as KPIs. But since I posted NEWS: Telco Monitoring I’m receiving many questions related to this topic and I guess we can discuss at least the basic principles.

Inside AT&T’s Network Operations Center by PCWorldVideos

If you have read the VoLTE standards such as GSMA IR.92 or VoLTE Service Description and Implementation Guidelines, you probably noticed that performance monitoring is more or less ignored. And at the same time all operators are asking about it. What KPIs to watch, how, what are the guidelines?

Btw. I always enjoy being in NOC (Network Operation Centre) or war or crisis rooms. Especially during events like NYE. However mostly it is not allowed to take any photos there, so instead I’ve linked some youtube videos showing the scene. Respect to you bros working day and night to keep the network running!

 

The basic 3GPP specification defining KPI templates is the 3GPP 32.410. This and some other documents define, what we are supposed to understand as “KPI”. So KPI stands for Key Performance Indicator. Each (telco) system provides some measurements/counters which help us to identify the performance and condition of it. These KPIs can be of several types:

• RATIO: This KPI indicates the percentage of a specific case occurrence to all the cases.
• MEAN: This KPI shows a mean measurement value based on a number of sample results.
• CUM: This KPI reflects a cumulative measurement which is always increasing.

The performance is then measured in several categories:

•    Serveability
•    Accessibility
•    Retainability
•    Integrity
•    Availability
•    Reliability
•    Maintainability
•    Utilization
•    Mobility

When a KPI is defined, it is classified into one of the listed category. More in the document ITU-T Recommendation E.800: “Terms and definitions related to quality of service and network performance including dependability”.

Finally KPIs can be calculated for different telco subsystems – e.g.  UTRAN, GERAN, CS/PS Core, IMS.

• TS 32.401: Performance Management (PM); Concept and requirements
• TS 52.402: Performance Management (PM); Performance measurements – GSM
• TS 32.404: Performance Management (PM); Performance measurements – Definitions and template
• TS 32.405: Performance Management (PM); Performance measurements UTRAN
• TS 32.406: Performance Management (PM); Performance measurements Core Network (CN) PS domain
• TS 32.407: Performance Management (PM); Performance measurements Core Network (CN) CS domain
• TS 32.408: Performance Management (PM); Performance measurements Teleservice
• TS 32.409: Performance Management (PM); Performance measurements IMS
• TS 38.913: 5G KPIs

The specification 3GPP 32.409 contains definitions of IMS related KPIs. Howto count a particular KPI is then described in 3GPP 32.454. And these two documents are probably the ones you are looking for.

So let’s have an example. The first KPI described in 32.454 is the Initial Registration Success Rate of S-CSCF (IRSR). This KPI is useful to evaluate accessibility performance provided by IMS and overall network performance. It is defined as a number of Successful Initial Registrations divided by Attempted Initial Registrations.

Initial Registration Success Rate

Sounds logical. However from the practical point of view it has several implications. Firstly a particular S-CSCF not always directly provides such a KPI (differs vendor to vendor). But it has to provide two cumulative counters for Successful Initial Registrations and Attempted Initial Registrations. The exact definition of  these counters is then in the 32.409.

Initial Registration Success Rate Measurements

The list of standardized IMS KPIs:

Accessibility KPIs

• Initial Registration Success Rate of S-CSCF
  • Useful to evaluate accessibility performance provided by IMS and network performance.
• Session Setup Time (Mean)
  • Indicates accessibility performance provided by IMS and network performance. This KPI can influence the users’ satisfaction directly and reflect network transaction performance.
• Session Establishment Success Rate
  • Important in order to evaluate the session establishment performance including user behaviour factors. It is necessary to define session establishment success rate (SESR) with differentiating originating and terminating, otherwise when the operator wants to consider SESR from whole IMS network perspective, the value of SESR will be much bigger than real situation due to one success session being counted twice (both at originating side and terminating side).
    
• Third Party Registration Success Rate
  • It is defined to fulfil the need of the operator to evaluate the service and network accessibility performance.
• Re-registration Success Rate of S-CSCF
  • This KPI is useful for evaluate accessibility of the IMS network, including the user behaviour factors.
• Session Setup Time Originated from IMS (Mean)
• Session Setup Time Originated from CS (Mean)
• Immediate Messaging Success Rate
• Session Establishment Network Success Rate 
  • KPI is focusing on network view different from the KPI Session Establishment Success Rate. Therefore the measurements on the number of release before ringing, UE not found, UE address incomplete and UE busy should be excluded from the failed measurements. This KPI is helpful to evaluate the real network session establishment success rate.

Retainability KPI

• Call Drop Rate of IMS Sessions
  • It is important in order to evaluate retainability performance of IMS including user causes.

Utilization KPI

•  Mean Session Utilization 
  • The mean number of simultaneous online and answered sessions together with Maximum Number of Sessions can reflect system resource utilization. If the value of this KPI is very high, it indicates system capacity is not sufficient and should be increased.

Inside Verizon’s Super Bowl Command Center by IDG.tv

 The standardized KPIs are useful if we need to compare two solutions from different vendors. But in daily practice we don’t limit ourselves only to standard set for KPIs and counters. Each VoLTE/VoWifi/RCS/…  solution is very specific, each operator does use a different set of features, has a different network architecture, different infrastructure etc. Some best practices or recommendations are provided by ITU’s Telecommunication Standardization Sector (ITU-T) – e.g. the ITU-T G.1028 gives us End-to-end quality of service for voice over 4G mobile networks recommendations, including high-level troubleshooting guidelines.

VoLTE QoS measurement points – monitoring as per ITU-T

Anyway to go a more in detail we need to understand a particular network, we have to know what are the supported and expected callflows, applied functionalities and other operator specific stuff. That’s why the monitoring is often designed or customized directly by mobile operators/service providers. They have to decide the importance of individual counters, KPIs (as well as billing/log records used as inputs for statistics). What is also extremely important, the data is stored in database and operators can base their analysis/decisions on comparison of actual values with the numbers from the past (e.g. to compare Registration Success Rate with the data from yesterday, previous week, previous month, ..). The values can be also used as feeds for Artificial Intelligence Systems providing decision making support.

Related posts:

• Multimedia in VoLTE
• News: Telco Monitoring
• IMS/WebRTC Tracing and Test Tools
• Challenges of Automated Testing (for telcos)