Abhas Kumar Sinha

> The cache has now consumed way more transistors than the cpu it serves.
For your gpu/AI accelerator, you’ll need lots of high speed sram for intermediate storage and cache. I’d suggest you search for some micro photographs of modern cpus and you should see vast tracks of regular patterns which tend to be memory.

Yes. I understand that, I've no problems with cache consuming more transistors than computation till it is in the ratio of 10:1. What are some of the strategies to avoid or optimize that?

*Hypothetical Device Specs*

1. Based on RISC-V specs.
2. RISC-V has Vector Extension (RVV) that we can implement for SIMD units.
3. RISC-V has HCM Extension that can be used to implement multiple *Harts* or SIMT Units. (I don't know how Harts compare to threads tho).
4. Get high L2 memory into it.
5. As far I understand RAMs are faster because they are able to update a lot of memory at once, rather than serial data transfer like USBs. So, Create a special soft of SRAM that works on two way - One side goes into CPU Motherboard and other side goes into our hypothetical machine and it enables…

One **A100** NVIDIA GPU specs:

1. It has 9k SIMT units (or cores or CUDA units) where 9,000 threads get processed simultaneously.
2. It has 512-bit registers supporting SIMD units, where at a time a single operation can be performed to 512 float numbers simultaneously.
3. It has 40 mb of L2 Cache and 80 G of Static RAM memory which is very high.
4. It at the same time has optimized CUDA software libraries like cuDNN, etc. which has a lot of software-based optimization done from the perspective of Deep Learning and ML.