How much memory should a program get? Tonight, a quick note on sizingfor garbage-collected heaps. There are a few possible answers,depending on what your goals are for the system.
Sometimes you build a system and you want to study it: to identify itsprincipal components and see how they work together, or to isolate theeffect of altering a single component. In that case, what you want is afixed heap size. You run your program a few times and determine a heapsize that is sufficient for your problem, and then in future run theprogram with that new fixed heap size. This allows you to concentrateon the other components of the system.
A good approach to choosing the fixed heap size for a program is todetermine the minimum heap size a program can have by bisection, thenmultiplying that size by a constant factor. Garbage collection is aspace/time tradeoff: the factor you choose represents a point on thespace/time tradeoff curve. I would choose 1.5 in general, but this isarbitrary; I'd go more with 3 or even 5 if memory isn't scarce and I'mreally optimizing for throughput.
Note that a fixed-size heap is not generally what you want. It's notgood user experience for running ./foo at the command line, forexample. The reason for this is that program memory use is usually afunction of the program's input, and only in some cases do you know whatthe input might look like, and until you run the program you don't knowwhat the exact effect of input on memory is. Still, if you have a teamof operations people that knows what input patterns look like and hasexperience with a GC-using server-side process, fixed heap sizes couldbe a good solution there too.
On the other end of the spectrum is the average user. You just want torun your program. The program should have the memory it needs! Not toomuch of course; that would be wasteful. Not too little either; I cantell you, my house is less than 100m², and I spend way too much timeshuffling things from one surface to another. If I had more space Icould avoid this wasted effort, and in a similar way, you don't want tobe too stingy with a program's heap. Do the right thing!
Of course, you probably have multiple programs running on a system thatare making similar heap sizing choices at the same time, and therelative needs and importances of these programs could change over time,for example as you switch tabs in a web browser, so the right thingreally refers to overall system performance, whereas what you arecontrolling is just one process' heap size; what is the Right Thing,anyway?
My corner of the GC discourseagrees thatsomething like the right solution was outlined by Kirisame, Shenoy, andPanchekha in a 2022 OOPSLApaper,in which the optimum heap size depends on the allocation rate and the gccost for a process, which you measure on an ongoing basis.Interestingly, their formulation of heap size calculation can be made byeach process without coordination, but results in a whole-systemoptimum.
There are some details but you can imagine some instinctive results: forexample, when a program stops allocating because it's waiting for someexternal event like user input, it doesn't need so much memory, so itcan start shrinking its heap. After all, it might be quite a whilebefore the program has new input. If the program starts allocatingagain, perhaps because there is new input, it can grow its heap rapidly,and might then shrink again later. The mechanism by which this happensis pleasantly simple, and I salute (again!) the authors for identifyingthe practical benefits that an abstract model brings to the problemdomain.
Hoo, friends-- I don't know. I've seen some things. Not to exaggerate,I like to think I'm a well-balanced sort of fellow, but there's somesuspicion too, right? So when I imagine a background thread determiningthat my web server hasn't gotten so much action in the last 100ms andthat really what it needs to be doing is shrinking its heap, kicking offadditional work to mark-compact it or whatever, when the whole point ofthe virtual machine is to run that web server and not much else, only tohave to probably give it more heap 50ms later, I-- well, again, Iexaggerate. The MemBalancer paper has a heartbeat period of 1 Hz and asmoothing function for the heap size, but it just smells like danger.Do I need danger? I mean, maybe? Probably in most cases? But maybe itwould be better to avoid danger if I can. Heap growth is usually bothnecessary and cheap when it happens, but shrinkage is never necessaryand is sometimes expensive because you have to shuffle around data.
So, I think there is probably a case for a third mode: not fixed, notadaptive like the MemBalancer approach, but just growable: grow the heapwhen and if its size is less than a configurable multiplier (e.g. 1.5)of live data. Never shrink the heap. If you ever notice that a processis taking too much memory, manually kill it and start over, or whatever. Default to adaptive, of course, but when you start to troubleshoot a high GC overhead in a long-lived proess, perhaps switch to growable to see its effect.
There is some heuristic badness that one cannot avoid: even with the adaptive MemBalancer approach, you have to choose a point on the space/time tradeoff curve. Regardless of what you do, your system will grow a hairy nest of knobs and dials, and if your system is successful there will be a lively aftermarket industry of tuning articles: "Are you experiencing poor object transit? One knob you must know"; "Four knobs to heaven"; "It's raining knobs"; "GC engineers DO NOT want you to grab this knob!!"; etc. (I hope that my British readers are enjoying this.)
These ad-hoc heuristics are just part of the domain. What I want to say though is that having a general framework for how you approach heap sizing can limit knob profusion, and can help you organize what you have into a structure of sorts.
At least, this is what I tell myself; inshallah. Now I have told you too. Until next time, happy hacking!