Ep. 42 | Groq CEO and Ex-Googler Jonathan Ross on the Petaflop AI Chip and First Ever TPU

Moore's Lobby: Where engineers talk all about circuits

In this episode of Moore’s Lobby, Daniel speaks with Jonathan Ross whose journey as a hardware and software engineering expert is revolutionary and something to behold. Ross is a self-described “successful college dropout” with entrepreneur DNA who designed Google’s first TPU (tensor processing unit) and was a $10B idealist for X - the moonshot factory. After believing he was done with chips, Ross founded Groq and is now their fearless CEO leading the creation of some of the most ambitious AI hardware in the world.

What could you accomplish if you could dedicate 20% of your job to new ideas? While at Google, Ross was allowed to take on a “20% project”—an initiative that could take 20% of his time, if he thought it could benefit the company in some way. His 20% project ended with the creation of the first Tensor Processing Unit (TPU), a key hardware component on which Google built their world-changing data center processing (and what he affectionately calls “a successful disaster”).

Now at Groq, Ross is leading a groundbreaking AI and high-performance computing company that’s tackling AI hardware with some surprising tactics. Their chips boast hundreds of times the processing power of current-gen processors. The kicker? It isn’t about using the most advanced transistors! According to Ross, the secret is in the chip architecture.

Ross speaks to how “no one ever thinks of solving their problems with hardware”, yet he believes hardware can solve a plethora of technical problems that software can’t. Daniel and Ross go down the rabbit hole of comparing Groq’s chip to CPUs and GPUs.

“People still use CPUs because they take less time to give an answer than a GPU, but they give fewer. And what we've (Groq) done is, we've actually built a chip that does both. It gives you the answers quicker than even a CPU does, but it actually gives you the throughput of the parallel performance of a GPU, or better.”

He also brings a unique approach to new projects including the design of their unique processor. “If you want to design a car, do you ask a bunch of mechanics or do you ask drivers?”

Some of Groq’s groundbreaking technology is powering autonomous vehicles, computing, data centers, drug discoveries, and even nuclear fusion. And, with investment topping $350 million, Groq stands to make big waves in the industry.

Learn from Ross’ fountain of wisdom on chip architecture and large-scale industry trends. End enjoy lots of engineering jokes that are actually funny and notable stories including:

-The evolution of a Google project from ”sea star” to “squirrel brain” and “superfluid”, before finally settling on something better

-The domain Elon Musk refuses to sell to Google

-Why Ross encourages everyone to accept a job at X - the moonshot factory aiming for $10B business ideas (yes, that’s with a B); but why it felt too much like retirement for him

-Hear Ross’ definition of machine learning that Daniel thinks is “the best he’s ever heard”

-A somewhat stinging reminder that a human hasn’t beat a computer in chess in decades

-Learn what “Groq” (a play on grok) means as a term and as a company

-And, how Ross believes in balancing products with talent: “The better the product, the better the talent; the better the talent, the better the product. Because the best talent wants to work on the best products, and the best products are built by the best talent.”

Do you want to learn more about Groq or real-time AI, ML, & HPC solutions? Register HERE for GroqDay coming up fast this Thursday, March 31st, 2022.

The second GroqDay is a series of virtual events bringing together enthusiasts and early adopters in the communities of artificial intelligence, machine learning, and high-performance computing.