ITU-T G.1052 PDF

This Recommendation specifies an automated laboratory-based (testbed) scoring framework for various types of human-machine or machine-machine applications over a series of varying use cases, network conditions, terminals and measurement domains. All parts of the testbed are described in this Recommendation, including its components, the calibration process, the testing methodology, the most relevant use cases and network scenarios, and the scoring methodology.

The framework consists of a single laboratory-controlled cellular network testbed and a set of test cases to automatically evaluate application (app) performance under various network conditions or user equipment (UE) and provide a single quality score based on the measured key performance indicators (KPIs). The score is computed by weighting the results obtained while testing the different uses cases for each app, for the different aspects relevant to the user, and under the network scenarios pertinent to the app. All KPI measurements are available to the user. The framework incorporates specific quality of service (QoS) to quality of experience (QoE) estimations based primarily on standardized models. In the absence of standardized models, subjective experiments may be used to determine fixed coefficients for the calibration of the testbed.

This Recommendations describes components and methodology to obtain stable and reproducible results, so that different parties implementing such a testbed would obtain comparable results. This can be useful to device vendors, mobile network operators, app developers, test and measurements companies, among others.

Existing and in-progress Recommendations (sufficiently mature) are referenced for QoE estimation purposes, with accommodation for the inputs needed provided. The proposed use cases are for further study.