For handguns, regular .22 is as cheap as it gets. For a few more cents a bullet, .22lr has much more power and accuracy.
Ruger makes some excellent .22lr handguns.
Same with rifles. .22lr is a great cartridge, and there are numerous .22lr rifles out there in the $200 to $500 range.