That tiny gadget I've been mentioning has a pair of 100µF ceramic chip capacitors (1206 footprint, X6T dielectric, rated 6.3V).
These are across the output power bus. In operation, they get charged up to 4.2V from the battery bus when the load switch is turned on.
I've just noticed something. If I do some fiddling with the load disconnected, put the gadget into power-down mode (which turns off the load switch, natch), disconnect the battery and the external power source, set the board down all by its little lonesome, go fix dinner, eat dinner, watch a couple of vids, then pick up the board after it's been sitting for an hour or so, and plug in the load...
...There's still enough charge in those caps to get quite an impressive blink out of the LEDs on the load.
I'd say technology has improved dramatically since I was a kid, but then I remember that, back in those days, the phrase "100µF ceramic capacitor" would have been a ticket to the looney bin. There were 2.2µF ceramic capacitors. They were quite bulky, and lossy. (Audibly lossy.) And now we have these wonderful things that cost (checks last DigiKey invoice) 67.4¢ each in quantity ten. And a 41.2¢ (qty. 10) load switch: rated 3.5A maximum, 1.1V to 5V, logic-level control input, 42mΩ ON resistance, and OFF resistance high enough not to be worth trying to measure, all in a 1.2x1.6mm package.