Google new scientific article on V4 TPU for AI | | | | Turtles AI

#Google just published a #scientific #article about its v4 #TPU, claiming to be ahead of #NVIDIA #A100 (but please remember that the latest and greatest NVIDIA product is the #H100, which had almost a twofold increase in performance compared to A100) in #AI #inference and #training. You can find the full article here. Below is a short summary. In the world of supercomputing, Google's TPU v4 is the latest and greatest, boasting not one, but two major architectural features that give it a significant edge over the competition. The first of these is the SparseCore, a revolutionary piece of technology that accelerates the embeddings of #DLRM models by an impressive 5x-7x. The secret to its success is a dataflow sea-of-cores architecture that allows embeddings to be placed anywhere in the 128 TiB physical memory of the TPU v4, providing unparalleled flexibility and speed. But the advantages of the TPU v4 don't stop there. The other major architectural feature is the OCSes and underlying optical components, which are surprisingly inexpensive at less than 5% of the overall cost and less than 3% of the overall power consumption. Despite their low cost, these components offer an impressive array of benefits, including scalability, improved availability, modularity, higher performance, diminished power usage, simplified scheduling, faster deployment, and enhanced security. Replacing the OCS and ICI with Infiniband may seem like a tempting option, but it comes at a cost. Not only does it increase costs and power consumption, but it also degrades performance. When compared to contemporary DSA chips made using similar technologies, the TPU v4 is faster and has lower power, offering a significant edge in the world of supercomputing. In fact, the TPU v4 is so powerful that it has become the go-to choice for large language models (LLMs) like LaMDA, MUM, and PaLM. Its performance, scalability, and availability make it the workhorse of the supercomputing world, allowing the 540B parameter PaLM model to sustain an impressive 57.8% of the peak hardware floating point performance over 50 days while training on TPU v4 supercomputers. Google has already deployed dozens of TPU v4 supercomputers for both internal and external use via Google Cloud, and its reliance on OCSes looks even more prescient given the recent surge in enthusiasm for LLMs. Computer architects' advances in warehouse-scale computing (WSCs) are also helping to reduce the carbon footprint of ML, making the TPU v4 an even more attractive option for those looking to deliver on the amazing potential of ML in a sustainable manner. All in all, the TPU v4 is a game-changing piece of technology that is revolutionizing the world of supercomputing. Its innovative features and impressive performance make it the clear choice for those looking to take their computing capabilities to the next level.