I've learned a few things since I started handloading in 1968, and the advice given so far is spot-on. Get one of the reloading manuals from Nosler, Hornady, Speer or any other bullet manufacturer and READ the introduction, process and safety concerns related to the pastime thoroughly until you have a firm grasp on the process. That's probably the most important aspect of starting out.
After digesting the entire process, you'll be better prepared to decide just how much you want to invest to get started. The single stage press advice is sound, as that's where almost all of us got started. I actually wore out my old Rockchucker press if that tells you anything, and still use one for my precision rifle loads.
Start slow with the caliber of choice, keep all loads within 'book' recommendations, and you'll do just fine.
But.....here's what this hobby/pastime (sickness) can and sometimes does result in....complete "reloading brain" as my wife calls it, and at least one room in your home will be sacrificed.
This is how things can turn out if it "catches on" so to speak.
The upside is that you can burn through a couple hundred rounds of your caliber of choice, and go home and have fun putting them all back together again, all the while listening to your favorite tune.
You'll also find that building your own ammo, shooting your own ammo is quite rewarding in terms of personal satisfaction. You won't really save money over time, because you'll find yourself shooting more, becoming a better shot, and just getting out more. It's all good and is really worth the time and investment. The key word though, is "fun!"