The other interesting detail here is the memory sizing problem. If I can consume all my RAM with caches and buffers, how much RAM do I need? The answer (as always) depends on what you're optimizing for. For performance, bigger is better. For cost, energy, etc you're going to want some way to calculate whether adding more RAM (and so having bigger caches) is worth the cost, heat, power, etc.
Gray and Putzolu's classic "The 5 minute rule for trading memory for disc accesses" (https://dl.acm.org/doi/pdf/10.1145/38713.38755) from 1987 is probably one of the most important CS systems papers of all time. In it, they lay out a way of thinking about memory and cache sizing by comparing the cost of holding cache to the cost of access (this isn't the first use of that line of thinking, but is a very influential statement of it). Back then, they found that storing 4kB in RAM for 5 minutes costs about the same as reading it back from storage. So if you're going to access something again within 5 minutes you should keep it around. The constants have change a lot (RAM is way cheaper, IOs are way cheaper, block sizes are typically bigger) since then, but the logic and way of thinking are largely timeless.
The 5 minute rule is a quantitative way of thinking about the size of the working set, an idea that dates back at least to 1968 and Denning's "The working set model for program behavior" (https://dl.acm.org/doi/10.1145/363095.363141).
Back to marginal costs - the marginal cost of converting empty RAM to cache is zero in the minute, but only because the full cost has been borne up front when the machine is purchased. It's not zero, just pre-paid.
Huh, never heard of that before. An interesting paper!
Running the numbers - assuming 4k record size instead of 1k, ignoring data size changes, ignoring cache, ignoring electricity and rack costs, selecting a $60 Samsung 980 with 4xPCIe and a $95 set of 2x16GB DDR5-6400 DIMMs...I get $0.003/disk access/second/year and $0.0000113 for 4k of RAM, a ratio of 264.
That is remarkably close to the original paper's ratio of 400, even though their disks only got 15 random reads per second, not 20,000, and cost $15,000, and their memory cost $1000/MB not $0.002/MB.
I'm not sure the "Spend 10 bytes of memory to save 1 instruction per second" works equally well, especially given that processors are now multi-core pipelined complex beasts, but working naively, you could multiply price, frequency, and core count to calculate ~$0.01/MIP (instead of $50k). $0.01 is about the cost of 3 MB of RAM. Dividing both by a million you should spend 3 bytes, not 10 bytes, to save 1 instruction per second.
If this is a Hetzner machine then yes, but enterprise SSDs costs more, especially from enterprise vendors. But this only drives the storage cost up.
More so, if you tend to send some big amount of data every 5 minutes and you are somewhat constrained by memory (32 / 1000 x 100 = 3.2%) then it would be easier to just read it from the storage again. If you are not constrained by storage bandwidth, of course.
And by the way, the latest gaming consoles (at least PlayStation?) is designed around this concept - they trade having big amount of RAM (which in case of PS5 is shared between GPU and the OS) to just loading assets from the storage extremely fast 'just in time'. Which works fine for games.
> Back to marginal costs - the marginal cost of converting empty RAM to cache is zero in the minute, but only because the full cost has been borne up front when the machine is purchased. It's not zero, just pre-paid. .
Or, in other words, you get to fully use what you paid for.
I think the OPs point was that people tend to buy more RAM than they actually need because they have no idea how much RAM they actually need, because it's always used, and so err on the side of caution
Gray and Putzolu's classic "The 5 minute rule for trading memory for disc accesses" (https://dl.acm.org/doi/pdf/10.1145/38713.38755) from 1987 is probably one of the most important CS systems papers of all time. In it, they lay out a way of thinking about memory and cache sizing by comparing the cost of holding cache to the cost of access (this isn't the first use of that line of thinking, but is a very influential statement of it). Back then, they found that storing 4kB in RAM for 5 minutes costs about the same as reading it back from storage. So if you're going to access something again within 5 minutes you should keep it around. The constants have change a lot (RAM is way cheaper, IOs are way cheaper, block sizes are typically bigger) since then, but the logic and way of thinking are largely timeless.
The 5 minute rule is a quantitative way of thinking about the size of the working set, an idea that dates back at least to 1968 and Denning's "The working set model for program behavior" (https://dl.acm.org/doi/10.1145/363095.363141).
Back to marginal costs - the marginal cost of converting empty RAM to cache is zero in the minute, but only because the full cost has been borne up front when the machine is purchased. It's not zero, just pre-paid.