So you basically dispose of cache for the memory region used? I wonder what the offsets of the cache misses is going to be in practice (the article addresses it but there is no solution/impact given by samsung).
So you basically dispose of cache for the memory region used? I wonder what the offsets of the cache misses is going to be in practice (the article addresses it but there is no solution/impact given by samsung).
> So you basically dispose of cache for the memory region used? I wonder what the offsets of the cache misses is going to be in practice (the article addresses it but there is no solution/impact given by samsung).
This is a temporary issue. JEDEC's LPDDR6-PIM is going to add defined commands for Processing-in-Memory operations. Once there are standardised commands, it will be possible for the CPU vendors to make the CPU cache aware of what is happening.
Of course, that doesn't solve it for this generation of the technology. But I think this generation is more of a demo for early adopters to gain experience with it. It will likely take a few years for all these issues to be solved, but there is no principled reason why they can't be.
I guess you also get very high bandwidth that way? I‘m not sure that would come for free though.
It comes at a cost of a fragmented memory space, which is fine for some applications, like LLMs
if your working set fits in cache PIM is irrelevant
Caches are important for cpu core performance even when the the working set doesn't fit.