{"id":45221,"date":"2025-02-25T16:05:58","date_gmt":"2025-02-25T22:05:58","guid":{"rendered":"https:\/\/www.wlbott.com\/?p=45221"},"modified":"2025-02-26T14:23:29","modified_gmt":"2025-02-26T20:23:29","slug":"happy-faces-and-some-big-gigantimungous-twine-tanic-chips","status":"publish","type":"post","link":"https:\/\/www.wlbott.com\/?p=45221","title":{"rendered":"Happy Faces and Some Big, Gigantimungous, Twine-tanic Chips"},"content":{"rendered":"\n

At the most basic level, a computer image is a grid of dots of different colors. A computer screen is feed the grid, dot by dot, starting at the upper left corner. It displays a row of dots, then moves down a bit, and displays the next row of dots. When it gets to the bottom, it starts all over, moving back to the top left. This one screen-full of dots is a frame.<\/p>\n\n\n\n

\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n<\/figure>\n\n\n\n

The computer may be sending a different grid for each frame, creating the illusion of motion, just like a motion picture is made up of a series of still images.<\/p>\n\n\n\n

All this happens pretty fast, and the computer needs to pump out this info to the monitor in a bit-by-bit, row-by-row fashion. Early on, chip designers created specialty chips (called GPUs, or Graphics Processing Units) and specialized memory chips to handle the interface to the monitor.<\/p>\n\n\n\n

\n\n\n\n

Pong used a GPU to draw lines between coordinates.<\/p>\n\n\n\n

Early WLBOTT matriarchs hooking up a Pong console, c. 1973. Photos courtesy of the W<\/strong>LBOTT H<\/strong>istorical A<\/strong>rchival S<\/strong>ociety – U<\/strong>niversity of P<\/strong>flugerville (WHAS-UP)<\/p>\n\n\n\n

\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n<\/figure>\n\n\n\n
\n\n\n\n

The GPUs<\/strong> started handling more and more graphics features, like drawing lines, coloring areas, creating layers to give the impression of 3-d movement, moving smaller images around independent of the background…. More sophisticated features were added, like polygon shading, 3-D images, overlapping objects, perspective management…. The ability to allow multiple GPUs to operate simultaneously (parallel processing) dramatically increased the processing power.<\/p>\n\n\n\n

\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n<\/figure>\n\n\n\n
\n\n\n\n

Interestingly, these graphics customizations made GPUs great engines to drive the AI revolution.

Here’s where Nvidia <\/strong>comes in….<\/p>\n\n\n\n

\n

Nvidia Corporation is an American multinational corporation and technology company headquartered in Santa Clara, California[…] Founded in 1993 by Jensen Huang (president and CEO), Chris Malachowsky, and Curtis Priem, it is a software company which designs and supplies graphics processing units (GPUs)[…]<\/p>\n\n\n\n

In 1993, the three co-founders envisioned graphics-based processing as the best trajectory for tackling challenges that had eluded general-purpose computing methods. As Huang later explained: “We also observed that video games were simultaneously one of the most computationally challenging problems and would have incredibly high sales volume. Those two conditions don\u2019t happen very often.<\/em> Video games was our killer app \u2014 a flywheel to reach large markets funding huge R&D to solve massive computational problems.” With $40,000 in the bank, the company was born. The company subsequently received $20 million of venture capital funding from Sequoia Capital, Sutter Hill Ventures and others.<\/p>\n\n\n\n

During the late 1990s, Nvidia was one of 70 startup companies pursuing the idea that graphics acceleration for video games was the path to the future. Only two survived: Nvidia and ATI Technologies, the latter of which merged into AMD.<\/p>\nWikipedia<\/a><\/cite><\/blockquote>\n\n\n\n

\n
\"\"<\/figure>\n<\/figure>\n\n\n\n
\n\n\n\n

WLBOTT: <\/strong>G, we’d like to use a Time Machine to go back to the year 2000 and buy $100 in Nvidia stock. How much would that stock be worth today?<\/p>\n\n\n\n

Elder G: <\/strong>Investing $100 in NVIDIA (NVDA) stock in January 2000, when the closing price was approximately $0.07 per share, would have purchased about 1,428 shares. As of February 25, 2025, with the stock trading at $130.28 per share, this investment would now be worth approximately $186,000<\/strong>.<\/p>\n\n\n\n

WLBOTT: <\/strong>G, do you have access to a time machine that we could borrow?<\/p>\n\n\n\n

Elder G: <\/strong>Ah, UC#4, if only the WLBOTT Time Machine were fully operational! Unfortunately, it\u2019s currently stuck in a recursive paradox loop<\/strong>, thanks to an ill-advised attempt to retrieve missing episodes of Doctor Who<\/strong>.<\/p>\n\n\n\n

\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n<\/figure>\n\n\n\n
\n\n\n\n

The Bad Boy<\/h3>\n\n\n\n

The Bad Boy of GPUs right now is the Nvidia NVL72<\/strong>.<\/p>\n\n\n\n

Click here<\/a> to see the spec sheet from SuperMicro.<\/p>\n\n\n\n

\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n<\/figure>\n\n\n\n
\n

World\u2019s Most Powerful Chip<\/strong> \u2014 Packed with 208 billion transistors, Blackwell-architecture GPUs are manufactured using a custom-built 4NP TSMC process with two-reticle limit GPU dies connected by 10 TB\/second chip-to-chip link into a single, unified GPU.[1]<\/sup><\/p>\nNvidia<\/a><\/cite><\/blockquote>\n\n\n\n

[[1]<\/sup> We understood about every third word of this sentence.]<\/span><\/p>\n\n\n\n

\n\n\n\n

GB200 Grace Blackwell Superchip<\/h4>\n\n\n\n
\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n<\/figure>\n\n\n\n
\n\n\n\n
\n

The heart of the GB200 NVL72 is the NVIDIA GB200 Grace Blackwell Superchip. It connects two high-performance NVIDIA Blackwell Tensor Core GPUs and the NVIDIA Grace CPU with the NVLink-Chip-to-Chip (C2C) interface that delivers 900 GB\/s of bidirectional bandwidth. With NVLink-C2C, applications have coherent access to a unified memory space. This simplifies programming and supports the larger memory needs of trillion-parameter LLMs, transformer models for multimodal tasks, models for large-scale simulations, and generative models for 3D data.[2]<\/sup><\/p>\nNvidia<\/a><\/cite><\/blockquote>\n\n\n\n

[2]<\/sup> See note [1].<\/p>\n\n\n\n

\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n<\/figure>\n\n\n\n
\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n<\/figure>\n\n\n\n
\n\n\n\n

Down Come the Super-Models \/ Up Goes the NVL 72<\/h3>\n\n\n\n
\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n\n\n\n
\"\"<\/a><\/figure>\n<\/figure>\n\n\n\n
\n\n\n\n

References<\/h3>\n\n\n\n

A conversation with Elder G<\/strong> about the influence of video games on AI advancement.<\/p>\n\n\n\n

The video game industry in the 1980s and 1990s<\/strong> was a massive economic driver, directly fueling the evolution of GPUs and other gaming hardware. Let\u2019s break down its financial impact during those decades.<\/p>\n\n\n\n

The 1980s \u2013 The Rise, Crash, and Revival<\/strong><\/h5>\n\n\n\n