Top Page | English | 简体中文 | 繁體中文 | 한국어 | 日本語
Wednesday, 21 September 2016, 13:10 HKT/SGT

Source: Fujitsu Ltd
Fujitsu Doubles Deep Learning Neural Network Scale with Technology to Improve GPU Memory Efficiency

KAWASAKI, Japan, Sept 21, 2016 - (JCN Newswire) - Fujitsu Laboratories Ltd. today announced development of technology to streamline the internal memory of GPUs to support the growing neural network scale that works to heighten machine learning accuracy. This development has enabled neural network machine learning of a scale up to twice what was capable with previous technology.

Recent years have seen a focus on technologies that use GPUs for high-speed machine learning to support the huge volume of calculations necessary for deep learning processing. In order to make use of a GPU's high-speed calculation ability, the data to be used in a series of calculations needs to be stored in the GPU's internal memory. This, however, creates an issue where the scale of the neural network that could be built is limited by memory capacity.

Fujitsu Laboratories has now developed technology to improve memory efficiency, implementing and evaluating it in the Caffe open source deep learning framework software. Upon commencement of learning, the technology analyzes the structure of the neural network, and optimizes the order of calculations and allocation of data to memory, so that memory space can be efficiently reused. With AlexNet and VGGNet(1), image-recognition neural networks widely used in research, this technology was confirmed to enable the scale of learning of a neural network to be increased by up to roughly two times that of previous technology, thereby reducing the volume of internal GPU memory used by over 40%.

This technology makes it possible to expand the scale of a neural network that can be learned at high speed on one GPU, enabling the development of more accurate models. Fujitsu Laboratories aims to commercialize this technology as part of Fujitsu Limited's AI technology, Human Centric AI Zinrai, to work with customers in the use of AI.

Details of this technology were announced at MLSP (IEEE Machine Learning for Signal Processing 2016), an international conference held in Salerno, Italy from September 13 to 16.

Development Background

In recent years, deep learning has been gaining attention as a machine learning method that emulates the structure of the human brain. In deep learning, the more layers there are in a neural network, the more accurate it performs tasks, such as recognition or categorization. In order to increase accuracy, the scale of neural networks has been growing, but this lengthens learning times. Along with this, more attention is being placed on GPUs that execute computations with large volumes of data, and technology that accelerates the process by using multiple GPUs in parallel, as with supercomputers.


One method of increasing the scale of deep learning is to distribute a single neural network model across multiple computers and do the computations in parallel, but the volume of data that must be transmitted in exchanges between computers then becomes a bottleneck, greatly reducing learning speed. In order to take full advantage of the GPU's high-speed calculation capability, to the extent possible the data to be used in series of calculations needs to be stored in the GPU's internal memory. However, as GPU memory is usually smaller than that of an ordinary computer, there had been the issue of limitations in scale of neural networks capable of high-speed learning.

About the Newly Developed Technology

Now Fujitsu Laboratories has developed technology to streamline memory efficiency to expand the scale of a neural network for computations with one GPU, without using parallelization methods that greatly reduce learning speed. This technology reduces the volume of memory by enabling the reuse of memory resources; it takes advantage of the ability to independently execute both calculations to generate the intermediate error data from weighted data, and calculations to generate the weighted data error from intermediate data. When learning begins, the structure of every layer of the neural network is analyzed, and the order of calculations is changed so that memory space in which larger data has been allocated can be reused.


Fujitsu Laboratories implemented this newly developed technology into the Caffe open source deep learning software framework and measured the usage of GPU internal memory. In evaluations using AlexNet and VGGNet, which are widely used in research fields, it achieved reductions in memory usage volume of over 40% compared with before the application of this technology, enabling the scale of learning on a neural network for each GPU to be increased by up to roughly two times.

This will enable high-speed learning calculations using the full capability of a GPU, even with a large-scale neural network that requires complicated processing, accelerating the development of more accurate models.

Future Plans

Fujitsu Laboratories aims to commercialize this newly developed technology as part of Fujitsu Limited's AI technology, Human Centric AI Zinrai, by March 31, 2017. In addition, it plans to combine this technology with its already announced high-speed technology to process deep learning through GPU parallelization(2), and further improve these technologies.

(1) AlexNet and VGGNet: Multi-layered neural networks for image recognition.
In 2012 AlexNet received top honors in a competition for image classification, and in 2014 VGGNet received top honors in a competition for position detection, each achieving the highest recognition accuracy in the world. Today they each form the basis of image recognition neural networks.

(2) High-speed technology to process deep learning through GPU parallelization
"Fujitsu Develops High-Speed Technology to Process Deep Learning," press release dated August 9, 2016:

About Fujitsu Laboratories

Founded in 1968 as a wholly owned subsidiary of Fujitsu Limited, Fujitsu Laboratories Ltd. is one of the premier research centers in the world. With a global network of laboratories in Japan, China, the United States and Europe, the organization conducts a wide range of basic and applied research in the areas of Next-generation Services, Computer Servers, Networks, Electronic Devices and Advanced Materials. For more information, please see:

About Fujitsu Ltd

Fujitsu is the leading Japanese information and communication technology (ICT) company, offering a full range of technology products, solutions, and services. Approximately 159,000 Fujitsu people support customers in more than 100 countries. We use our experience and the power of ICT to shape the future of society with our customers. Fujitsu Limited (TSE:6702; ADR:FJTSY) reported consolidated revenues of 4.7 trillion yen (US$41 billion) for the fiscal year ended March 31, 2016. For more information, please see

* Please see this press release, with images, at:

Fujitsu Laboratories Ltd.
Computer Systems Laboratory

Fujitsu Limited
Public and Investor Relations
Tel: +81-3-3215-5259

Sept 21, 2016 13:10 HKT/SGT
Source: Fujitsu Ltd

Fujitsu Ltd (TSE: 6702)

Topic: Press release summary
Sectors: Electronics, Science & Research
From the Asia Corporate News Network

Copyright © 2016 ACN Newswire. All rights reserved. A division of Asia Corporate News Network.

Figure 1: Technology to improve memory efficiency
View Image
Fujitsu Ltd Links

Fujitsu Ltd
Sept 29, 2016 16:49 HKT/SGT
Fujitsu Strengthens Organization to Expand its Digital Business and Presence in Global Markets
Sept 28, 2016 12:21 HKT/SGT
Fujitsu to Partner with Tealium in Digital Marketing Business
Sept 28, 2016 11:32 HKT/SGT
Fujitsu to Exhibit at "ITS World Conference 2016" in Melbourne
Sept 28, 2016 11:12 HKT/SGT
Fujitsu Opens Test Environment for IoT Systems with Actual Hardware, Accelerates Co-Creation in the IoT Space
Sept 21, 2016 13:15 HKT/SGT
Fujitsu ICT to Support the Future of Basketball in Japan
Sept 20, 2016 11:45 HKT/SGT
Fujitsu and Fraunhofer HHI Develop First-Ever Simultaneous Wavelength Conversion Technology with No Wavelength Restriction in Next-Generation Optical Networks
Sept 20, 2016 11:25 HKT/SGT
Fujitsu Introduces ATMS PMV, Bringing Efficiency to Increasingly Complex Intellectual Property Management
Sept 14, 2016 10:17 HKT/SGT
Fujitsu Chosen for the Dow Jones Sustainability World Index for 17th Time
Sept 12, 2016 10:22 HKT/SGT
Fujitsu Expands Lineup of Palm Vein Authentication Products for Embedding into Equipment
Sept 7, 2016 10:46 HKT/SGT
Fujitsu Receives Order for Super-Kamiokande Experiment Computer System from University of Tokyo
More news >>
 News Alerts
Copyright © 2016 ACN Newswire - Asia Corporate News Network
Home | About us | Services | Partners | Events | Login | Contact us | Privacy Policy | Terms of Use | RSS
US: +1 800 291 0906 | Beijing: +86 10 8405 3688 | Hong Kong: +852 2217 2912 | Singapore: +65 6304 8926 | Tokyo: +81 3 5791 1818

Connect With us: