Zurich - Researchers from the Swiss Federal Institute of Technology in Zurich (ETH) have developed a software solution to accelerate the training process for neural networks. This takes a huge amount of computing time and costs a great deal of money. This new predictive software could now make this process between two and potentially more than five times faster.

Two researchers from the Scalable Parallel Computing Lab at the Swiss Federal Institute of Technology in Zurich (ETH) have developed a software solution to rapidly speed up the training of deep learning applications. This is important as this process is the most resource-​demanding and costly step of all, ETH Zurich writes in a press release. It accounts for up to 85 percent of the training time. For example, a single training run of a sophisticated voice recognition model can cost around 10 million US dollars.

Neural networks learn faster using ETH software
Many tasks are performed better by neural networks than by conventional algorithms. ETH computer scientists developed software that accelerates the training of artificial intelligence on the supercomputer "Alps" at CSCS.

The new software named NoPFS was developed by Roman Böhringer and Nikoli Dryden. It uses the concept of clairvoyance: “It exploits a pseudo-​random and therefore predictable process in training more effectively than other tools have to date”, ETH Zurich writes in the press release. NoPFS is able to predict the sequencing of data samples. Nikoli Dryden explains that this knowledge means that “the individual data items can be loaded in advance based on the frequency they will be accessed”, which allows the data items to be “prefetched to the respective processors, which speeds up training”.

Depending on the respective supercomputer, the number of graphics processors used and the size of the data set, NoPFS was able to speed up the training of neural networks by a factor of between 2.1 and 5.4. With AI-capable systems, such as those used in the new Alps supercomputer, the training efficiency increase may be somewhat lower. In spring 2023, Alps is set to replace the previous flagship among Swiss supercomputers, Piz Daint, at CSCS, the Swiss National Supercomputing Center that is connected to ETH Zurich.

Information technology

Information technology

Greater Zurich: where tech invests

The Greater Zurich Area is a leading location for information technologies in Europe - just ask global tech giants like Apple, Disney, Google, and IBM that have found success in the region. Companies that locate information technology operations in the Greater Zurich Area benefit from wide availability of skilled talent, opportunity for cooperative efforts with top universities and innovative start-ups, and a stable, business friendly atmosphere.

Contact us

Can we put you in touch with a peer company or research institute? Do you need any information regarding your strategic expansion to Switzerland's technology and business center? Give us a call or drop us a line. 

+41 44 254 59 59