Top Page | English | 简体中文 | 繁體中文 | 한국어 | 日本語
Monday, 17 October 2016, 11:16 HKT/SGT

Source: Fujitsu Ltd
Fujitsu Technology Uses Conversational Speech to Identify Customer Satisfaction
Enables automatic response evaluation in customer interactions in various situations

TOKYO, Oct 17, 2016 - (JCN Newswire) - Fujitsu Laboratories Ltd. today announced the development of voice-analysis technology that can automatically identify times when customers feel satisfied or dissatisfied in their conversations with service personnel.

Figure 1: Service personnel training using voice analysis

Figure 2: Quantification of voice cheerfulness with this technology

Figure 3: Structure of this technology

Service provided by personnel in support of customers, such as at call centers and bank teller windows, directly relates to the company's image, making training critical. Given this, in the past, efforts have been made to convert conversations with customers to text with speech recognition technology in order to grasp a customer's sense of satisfaction. However, there are cases where the same words can convey either satisfaction or dissatisfaction, depending upon the tone of voice. For this reason, based on word content alone, it was impossible to fully grasp the customer's sense of satisfaction.

Now, using a method that takes into account not only the average pitch of the voice and the degree of variation, but also the characteristic changes at relative points within voice data that covers multiple words - namely at the start or end of speech - Fujitsu has succeeded in a highly accurate quantification of voice cheerfulness. In addition, using machine learning in conjunction with customer-service evaluations, Fujitsu has developed a technology that can automatically identify times in a conversation when a customer is satisfied or dissatisfied with roughly 70% accuracy, as compared with results of a determination made by the human ear.

In a field trial using this technology in Fujitsu Limited and Fujitsu FSAS Inc. call centers, Fujitsu confirmed increases in the efficiency of training, such as monitoring and evaluation of support personnel and feedback on the results, reducing the time required by about 30%. Moreover, there was a greater degree of acceptance by both the evaluator and the person being evaluated due to increased objectivity.

Going forward, Fujitsu Laboratories intends to incorporate this technology into Human Centric AI Zinrai, the AI technology of Fujitsu Limited, and offer it as a product for use in customer-service evaluations and in the training of service personnel in a variety of enterprises that emphasize communication, such as banks and retail stores.

Developmental Background

In communication, both remotely, as in call centers and distance learning services, as well as in such face-to-face situations as bank teller windows, the service provided to customers by service personnel is directly related to the company's image. This means that training for service personnel is seen as extremely important.

Until now, information was sometimes gathered through customer surveys and used in training, but in many cases, the only data that could be obtained was the overall evaluation of the service. This made it difficult to determine what service personnel did wrong in the conversation. Also, in addition analyzing purchasing history for individual customers, surveys and other marketing activities, there is a need to discern, based on the customer's actual voice, such as can be heard in call centers, what the customer wants with regard to specific products or services, for example, what the customer's requirements are or what points need improvement. In so doing it is hoped that data from conversations with customers, gathered from service situations where personnel interact directly with customers, could be used in such areas as evaluating the quality of service, training employees, and marketing strategies.

Figure 1: Service personnel training using voice analysis


Previously, there were efforts to grasp customer feelings by converting conversations with customers to text using voice recognition software, but not only do actual conversations not always follow ordinary grammar, but they can also be impacted by surrounding noise. As a result, there were many technological difficulties in converting conversations to text through voice recognition, such as many instances where the conversion was incorrect. In addition, even if people use the same words, the meaning may change in a variety of ways depending on their emotions, so it was difficult to correctly interpret the customer's emotions using methods that analyze speech that has been converted to text.

About the Technology

Fujitsu has now developed technology to automatically determine times in a conversation when customers feel satisfied or dissatisfied based on features of their voices in relation to the way they are speaking with service personnel. With this technology, customer service personnel and service providers can rapidly evaluate service content and improve support methods. Features of the newly developed technology are as follows.

Technology to quantify "voice cheerfulness" from patterns of changes in voice pitch

A cheerful voice is ordinarily one with a high tone, or one in which the voice's tone and volume change a great deal. Moreover, Fujitsu Laboratories has determined through proprietary research that cheerful voices have unique characteristics of change at the beginnings and endings of statements. As a result, by using a method that takes into account unique changes at relative positions in voice data across multiple words, in addition to analyzing a voice's average pitch and changes, Fujitsu Laboratories has succeeded in accurately quantifying the cheerfulness of a voice (Figure 2).

Figure 2: Quantification of voice cheerfulness with this technology

In addition, because there is a highly correlated relationship between perceived voice cheerfulness and degree of satisfaction, Fujitsu Laboratories has quantified the sense of satisfaction in a conversation, using the quantified cheerfulness of a voice in a proprietary conversion formula based on survey results (Figure 3). By combining this with customer-service evaluations, and using machine learning to find a threshold point between satisfaction and dissatisfaction, Fujitsu Laboratories has developed technology to automatically identify times during a conversation when the customer is satisfied or dissatisfied. With the learning, the customer can adopt the results of the response evaluation on site, and fixed standards can also be customized for each location.

Figure 3: Structure of this technology


With this technology, Fujitsu Laboratories was able to determine times of satisfaction and dissatisfaction in conversations in customer service situations with about 70% accuracy compared with results of a determination made with the human ear. This has made it easier, in training customer service personnel, for the person receiving the results to understand which parts of the conversation were good or bad, leading to efficient improvement in service skills.

This technology was field tested in three of Fujitsu and Fujitsu FSAS's call centers. In addition to this technology, the trial used evaluation tools that also implemented technology to identify problem areas in customer interaction, such as when service personnel spoke over customers or paused too long before responding. As a result, the field test confirmed that not only were there great increases in the efficiency of training through monitoring and evaluating service personnel and providing feedback on the results, with a 30% reduction in time required, but the acceptance of the evaluation by both the evaluator and person being evaluated was significantly increased because of the increased objectivity of the evaluation.

This technology can be expected to lead to improvements in customers' sense of satisfaction and training efficiency due to increased service skills in a variety of customer contact situations, including call centers.

Future Plans

By combining this technology with Fujitsu Limited's digital solutions and services, such as automatic interactive answering services, Fujitsu Laboratories is working on initiatives to extract information on highly satisfactory features from products customers offer for use in marketing analysis, as well as service that adapts on the fly to customers' sense of dissatisfaction. Plans call for commercialization as a call center-related service, offered by Fujitsu and Fujitsu FSAS, by the end of March 2017.

In addition, by using this technology in a way that strengthens the sensory-media technology that is part of Human Centric AI Zinrai, Fujitsu Limited's AI technology, Fujitsu Laboratories aims to commercialize it for use in a variety of situations, such as bank teller windows, retail, healthcare and training, by the end of March 2019.

About Fujitsu Laboratories

Founded in 1968 as a wholly owned subsidiary of Fujitsu Limited, Fujitsu Laboratories Ltd. is one of the premier research centers in the world. With a global network of laboratories in Japan, China, the United States and Europe, the organization conducts a wide range of basic and applied research in the areas of Next-generation Services, Computer Servers, Networks, Electronic Devices and Advanced Materials. For more information, please see:

About Fujitsu Ltd

Fujitsu is the leading Japanese information and communication technology (ICT) company offering a full range of technology products, solutions and services. Approximately 130,000 Fujitsu people support customers in more than 100 countries. We use our experience and the power of ICT to shape the future of society with our customers. Fujitsu Limited (TSE:6702) reported consolidated revenues of 3.9 trillion yen (US$35 billion) for the fiscal year ended March 31, 2020. For more information, please see

Fujitsu Laboratories Ltd.
Media Processing Laboratory
E-mail: [email protected]

Fujitsu Limited
Public and Investor Relations
Tel: +81-3-3215-5259

Oct 17, 2016 11:16 HKT/SGT
Source: Fujitsu Ltd

Fujitsu Ltd (TSE: 6702)

Topic: Press release summary
Sectors: Electronics
From the Asia Corporate News Network

Copyright © 2020 ACN Newswire. All rights reserved. A division of Asia Corporate News Network.

Fujitsu Ltd Links

Fujitsu Ltd
June 1, 2020 11:01 HKT/SGT
Fujitsu Signs Partnership Agreement with Scaled Agile, Inc., Driving Large-Scale Agile for Digital Transformation
May 28, 2020 10:26 HKT/SGT
Fujitsu Accelerates Digital Transformation with Flexible, Scale-out Software-Defined Storage ETERNUS Data Services Platform
May 26, 2020 10:10 HKT/SGT
Fujitsu Launches 14 New Models of Enterprise Notebooks, Tablets and Workstations Optimized for Remote Work
May 26, 2020 09:34 HKT/SGT
Fujitsu Develops AI-Video Recognition Technology to Promote Hand Washing Etiquette and Hygiene in the Workplace
May 21, 2020 08:00 HKT/SGT
Fujitsu RunMyProcess Drives Next Phase of Digital Transformation in Business Processes with DigitalSuite ProcessModeler
May 15, 2020 09:11 HKT/SGT
Fujitsu and Accenture Collaboration Accelerates Reliable Blockchain Interoperability
May 14, 2020 14:53 HKT/SGT
Fujitsu Sets Out the 'Fujitsu Technology and Service Vision 2020'
May 14, 2020 14:46 HKT/SGT
Notice of Agreement for the Merger through Absorption (Simplified Absorption-type Merger) of a Consolidated Subsidiary of Fujitsu Limited
May 12, 2020 09:08 HKT/SGT
Fujitsu Offers Free Access to Intellectual Property Rights to Support Fight Against COVID-19
May 8, 2020 09:11 HKT/SGT
Fujitsu and Qualcomm Complete Multi-Gigabit Data Call Using 5G Carrier Aggregation
More news >>
 News Alerts
Copyright © 2020 ACN Newswire - Asia Corporate News Network
Home | About us | Services | Partners | Events | Login | Contact us | Privacy Policy | Terms of Use | RSS
US: +1 800 291 0906 | Beijing: +86 400 879 3881 | Hong Kong: +852 2217 2912 | Singapore: +65 6304 8926 | Tokyo: +81 3 6859 8575

Connect With us: