- Talk Title : Wafer-Scale Processors for AI and HPC

- Abstract :
   Dennard scaling is over, and Moore’s Law is coming to an end. Now, with the rise of deep learning and a host of beyond-exascale challenges in HPC, we need more performance. To extend the growth of performance in HPC, architectural specialization has returned, providing a significant boost for computational science. At the hardware level, the development by Cerebras Systems of a viable wafer-scale compute platform opens new opportunities.
   Cerebras has created the first commercial wafer-scale computer. The CS-2, with a 7nm wafer having over 800,000 powerful processors and an equally powerful memory and interconnect, is the second generation system. Because single-wafer integration provides remarkable memory and interconnect bandwidth and latency, wafer-scale processing has breakthrough implications across high-performance use cases, from inference and training in deep neural networks, to computational chemistry, to computational physics. The memory and communication walls in single-chip processors impose delay when off-processor-chip data is needed. By changing the scale of the chip by two orders of magnitude, we pack a small, powerful, mini-supercomputer on one piece of silicon, and eliminate much of the off-chip traffic for applications that can fit in the available memory. The elimination of most off-chip communication also cuts the power per unit performance, a key performance determining parameter.
   For training the emerging large-scale neural networks, the use of the wafer-scale processor means that less scale out and smaller batches are needed, improving usability and efficiency and reducing the pressure on the data storage service. For HPC use cases like finite difference methods, Fourier methods, dense matrices, and examples involving irregular and dynamic interprocessor communication, we are demonstrating supercomputer performance levels or better at kilowatt rather than megawatt power and single rack rather than datacenter cost. The fine-grained nature of the architecture allows strong scaling and reduces time to solution rather than boosting flops for enormous problems.
   The future holds great promise for the wafer-scale approach, and I will sketch some areas for extending our basic technology with innovations at the hardware and the architecture level.

- Bio :
    Rob Schreiber is a technical fellow at Cerebras Systems, Inc., where he works on architecture and programming of systems for AI and for computational science. Before Cerebras he taught at Stanford and RPI and worked at NASA, at startups, and at HP. Schreiber’s research spans sequential and parallel algorithms for matrix computation, compiler optimization for parallel languages, and high performance computer design. With Moler and Gilbert, he developed the sparse matrix extension of Matlab. He created the NAS CG parallel benchmark. He was a designer of the High Performance Fortran language. Rob led the development at HP of a system for synthesis of custom hardware accelerators. He has help pioneer the exploitation of photonic signaling in processors and networks. He is an ACM Fellow, a SIAM Fellow, and was awarded, in 2012, the Career Prize from the SIAM Activity Group in Supercomputing.

- Talk Title : Tiny Machine Learning (TinyML) for Domain-Specific Systems

- Abstract :
   Tiny machine learning (TinyML) is a fast-growing and emerging field at the intersection of machine learning (ML) algorithms and low-cost embedded systems. It enables on-device sensor data analysis for vision, audio, IMU, etc., at ultra-low-power consumption. Moving machine learning computing close to the sensor(s) allows for an expansive new variety of always-on ML use-cases aptly suited for domain-specific computing. This talk introduces the vision behind TinyML and focuses on some exciting new domain-specific applications that TinyML is enabling for low-cost IoT solutions. Although TinyML has rich possibilities, there are still numerous technical challenges. Tight onboard processor, memory and storage constraints, coupled with embedded software fragmentation, and a lack of relevant large-scale sensor datasets and benchmarks pose a substantial barrier to developing novel applications. The talk touches upon the myriad research opportunities for unlocking the full potential of this emerging field, spanning from algorithm design to automatic hardware synthesis for TinyML.

- Bio :
    Vijay Janapa Reddi is an Associate Professor at Harvard University, VP and a founding member of MLCommons (mlcommons.org), a nonprofit organization aiming to accelerate machine learning (ML) innovation for everyone. He also serves on the MLCommons board of directors and is a Co-Chair of the MLCommons Research organization. He co-led the MLPerf Inference ML benchmark for datacenter, edge, mobile and IoT systems. Before joining Harvard, he was an Associate Professor at The University of Texas at Austin in the Electrical and Computer Engineering department. His research sits at the intersection of machine learning, computer architecture and system software. He specializes in building computing systems for tiny IoT devices, as well as mobile and edge computing. Dr. Janapa-Reddi is a recipient of multiple honors and awards, including the National Academy of Engineering (NAE) Gilbreth Lecturer Honor (2016), IEEE TCCA Young Computer Architect Award (2016), Intel Early Career Award (2013), Google Faculty Research Awards (2012, 2013, 2015, 2017, 2020), Best Papers at the 2020 Design Automation Conference (DAC), 2005 International Symposium on Microarchitecture (MICRO), 2009 International Symposium on High-Performance Computer Architecture (HPCA), IEEE’s Top Picks in Computer Architecture honorable mentions & awards (2006, 2010, 2011, 2016, 2017, 2021). He has been inducted into the MICRO and HPCA Hall of Fame (in 2018 and 2019, respectively). He is passionate about widening access to applied machine learning for STEM, Diversity, and using AI for social good. He designed the Tiny Machine Learning (TinyML) series on edX, a massive open online course (MOOC) that sits at the intersection of embedded systems and ML that thousands of global learners can access and audit free of cost. He was also responsible for the Austin Hands-on Computer Science (HaCS) deployed in the Austin Independent School District for K-12 CS education. Dr. Janapa-Reddi received a Ph.D. in computer science from Harvard University, an M.S. from the University of Colorado at Boulder and a B.S from Santa Clara University.

Invited Talk III

- Speaker : Moo-Kyoung Chung, Chief Technology Officer, SAPEON Korea

Invited Talk IV

- Speaker : Euicheol Lim, Research Fellow ,SK Hynix