12.09.2022
| Research |
On the search for the right ML-accelerator
The use of deep learning in IoT and Edge Computing devices creates new challenges. How can strict requirements regarding performance and power consumption already be taken into account during the design stage? With an advanced ML-method, the power consumption of graphic cards can be determined during the development phase.
In cooperation with Mehran Goli (DKFI) under the direction of Rolf Drechsler, our research associate Christopher Metz has developed a new approach enabling a more precise determination of the power consumption of GPGPUs when running Convolutional Neural Networks (CNNs) in early phases of development. The authors discuss this new method in their paper titled “Towards Neural Hardware Search: Power Estimation of CNNs for GPGPUs with Dynamic Frequency Scaling”.
The method is based on earlier works of the authors (DDECS we reported
»hier, CODES+ISSS we reported
»hier, DATA Friday Workshop SLOHA we reported
»hier) and enhances previous approaches regarding the dynamic scaling of a graphic card’s frequency. Usually, power consumption can be reduced by lowering the frequency of computing units. With this approach, however, the expected electricity consumption can be estimated in advance. Here an error of only 5,03% is achieved.
Over the last years the power consumption of graphic cards has been increasing steadily. While the Nvidia P100 only required 200 Watt (W), newer models like the V100 and A100, for instance, need 250 and 350 W. This trend is continuing. The upcoming Nvidia H100 can require up to 700 W. Regarding the use of machine learning (especially deep learning) in IoT and Edge Computing, limited electricity consumption plays an important role during the design of these systems. In order to automatise future design processes the authors introduce the concept of “Neural Hardware Search” (NHS), a term that describes the search for the best hardware for neural networks. In contrast to already existing approaches of Software/Hardware-Co-Designs, Neural Hardware Search does not design new hardware. Instead, the most cost-efficient ML-accelerators (e.g. graphic cards) from an already existing set are being selected. The approach of power consumption estimation is combined with other prediction models (e.g. for performance, storage, etc.), to test different graphic card configurations without having to build a large number of prototypes.
Moreover, this approach can also be utilised to efficiently reduce the power consumption when working with cloud providers. For this purpose, the electricity consumption for a CNN is estimated for different frequencies, to determine the frequency with the lowest usage. Afterwards, the previously determined frequency can then be set for all CNNs in the cloud.
This approach was developed in cooperation with the working group Cyber-Physical Systems of the DFKI in Bremen. It was presented during the 4. ACM/IEEE Workshop on Machine Learning for CAD on the 12th and 13th of September in Snowbird Utah.
About the MLCAD Workshop:
The workshop dealt with machine learning, focusing on aspects of CAD as well as the design of electronic systems. It was sponsored by the ACM Special Interest Group on Design Automation (SIGDA) as well as the IEEE Council on Electronic Design Automation (CEDA). The workshop was comprised of different presentations as well keynotes
« back
12.09.2022 | Research
On the search for the right ML-accelerator
The use of deep learning in IoT and Edge Computing devices creates new challenges. How can strict requirements regarding performance and power consumption already be taken into account during the design stage? With an advanced ML-method, the power consumption of graphic cards can be determined during the development phase.
In cooperation with Mehran Goli (DKFI) under the direction of Rolf Drechsler, our research associate Christopher Metz has developed a new approach enabling a more precise determination of the power consumption of GPGPUs when running Convolutional Neural Networks (CNNs) in early phases of development. The authors discuss this new method in their paper titled “Towards Neural Hardware Search: Power Estimation of CNNs for GPGPUs with Dynamic Frequency Scaling”.
The method is based on earlier works of the authors (DDECS we reported
»hier, CODES+ISSS we reported
»hier, DATA Friday Workshop SLOHA we reported
»hier) and enhances previous approaches regarding the dynamic scaling of a graphic card’s frequency. Usually, power consumption can be reduced by lowering the frequency of computing units. With this approach, however, the expected electricity consumption can be estimated in advance. Here an error of only 5,03% is achieved.
Over the last years the power consumption of graphic cards has been increasing steadily. While the Nvidia P100 only required 200 Watt (W), newer models like the V100 and A100, for instance, need 250 and 350 W. This trend is continuing. The upcoming Nvidia H100 can require up to 700 W. Regarding the use of machine learning (especially deep learning) in IoT and Edge Computing, limited electricity consumption plays an important role during the design of these systems. In order to automatise future design processes the authors introduce the concept of “Neural Hardware Search” (NHS), a term that describes the search for the best hardware for neural networks. In contrast to already existing approaches of Software/Hardware-Co-Designs, Neural Hardware Search does not design new hardware. Instead, the most cost-efficient ML-accelerators (e.g. graphic cards) from an already existing set are being selected. The approach of power consumption estimation is combined with other prediction models (e.g. for performance, storage, etc.), to test different graphic card configurations without having to build a large number of prototypes.
Moreover, this approach can also be utilised to efficiently reduce the power consumption when working with cloud providers. For this purpose, the electricity consumption for a CNN is estimated for different frequencies, to determine the frequency with the lowest usage. Afterwards, the previously determined frequency can then be set for all CNNs in the cloud.
This approach was developed in cooperation with the working group Cyber-Physical Systems of the DFKI in Bremen. It was presented during the 4. ACM/IEEE Workshop on Machine Learning for CAD on the 12th and 13th of September in Snowbird Utah.
About the MLCAD Workshop:
The workshop dealt with machine learning, focusing on aspects of CAD as well as the design of electronic systems. It was sponsored by the ACM Special Interest Group on Design Automation (SIGDA) as well as the IEEE Council on Electronic Design Automation (CEDA). The workshop was comprised of different presentations as well keynotes
Please contact us if you have any questions:
Christopher Metz
Research Associate
+49 (421) 218 - 63942
cmetz@uni-bremen.de
« back