This isn't ai slop. It's fascinating and well written.
But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.
A lot of neural engine, particularly in the embedded domain (ARM/RISC MCUs) have the same problem. Designing other models means on top of this means a lot of profiling to get convolution blocks right to get good speedups. (We optimized this in the past e.g. using Neural Architecture Search on super networks)
Also, just imagine being the group at Apple responsible for designing this section of the chip, starting probably almost a decade back – under the constant uncertainty of not knowing what direction ML workloads would develop in…
But I learned something really basic - i didn't know that the ANE (and the data pipeline around it) was designed for CNN rather than transformers. It's always been an open loop in my head, wondering why the ANE was less impactful than i understood it should be.
Same for me!
Also, just imagine being the group at Apple responsible for designing this section of the chip, starting probably almost a decade back – under the constant uncertainty of not knowing what direction ML workloads would develop in…
excels!