So I ask again: Why use a small capacitor to bypass power on these types of
circuits? Why isn't the 1 uF tantalum a good solution, albeit a few cents more?

Good analysis.

Monolithic Ceramic costs less, won't catch fire, scream, smell, explode, short (no fun at all :-)).

Now, if that was a solid aluminium cap you were recommending I may agree :-)

RM