ITU-T P.1311 PDF

Modern immersive telemeeting systems have enabled remote meeting experiences that begin to resemble in-person meetings, with rapid turn-taking and frequent occurrences of multiple people speaking at once. The shift to these natural behaviours is possible because telemeeting systems are increasingly capable of delivering contextual cues that allow interlocutors to stay oriented in a complex virtual meeting environment, much as they do in in-person meetings.

Telemeeting systems also are under pressure to allow access from a variety of endpoints and environments, including mobile endpoints, desks, home offices, conference rooms and dedicated telemeeting rooms, and to interconnect users serviced by different networks. All of these endpoints and networks potentially have different capabilities to capture, carry, and reproduce the contextual cues that make engaging and interactive conferences possible. The industry is in need of a standard method to quantify the intelligibility of multiple concurrent talkers that these systems provide and this Recommendation defines such a method.

This Recommendation¹ describes a method for obtaining an objective measure of how well a telemeeting system allows users to follow a conversation when talk spurts of several talkers coincide. The method comprises a listening-only test that involves listeners observing several concurrent talkers, identifying one of them, and reporting what that talker said. The method is applicable to audio and audiovisual telemeeting systems with three or more users in two or more locations. The test outcome is determined by the degree to which the system under test (SuT) preserves the independence of voices and limits perceptual interference between voices. For example, in a system that spatially virtualizes different talkers, the method is sensitive to changes in the perceived angular separation of talkers and can be used to differentiate between alternative implementations of system components such as sound field capturing microphones or virtual spatial auditory displays. Implementing the method does not require access to internal components of the system under test, so that the method of this Recommendation can be used to test both system components and deployed, live systems. Practitioners should be aware that telemeeting systems can and should be evaluated on several metrics. Tests conducted using the method of this Recommendation provide an objective measure of how well users follow a conversation during times of concurrent talk. This is an important aspect of meetings, but it is not the only aspect that determines the acceptance of and user satisfaction with a telemeeting system. Practitioners must evaluate which other aspects, if any, are important for a complete evaluation of a system under test and are referred to [ITU-T P.1301] and other Recommendations currently in preparation for guidance on selecting measurement procedures. Practitioners should further be aware that the method specified in this Recommendation is applicable only to the evaluation of two or more concurrent talkers; it is not applicable to the assessment of intelligibility under conditions where only one talker is active at any one time. To evaluate the intelligibility of a single talker over a communication system practitioners are referred to [b-ANSI S3.2 (R2014)] In particular, the method does not aim at any representativeness of the phonetic representation of a chosen language.

¹ This Recommendation includes an electronic attachment containing speech samples.