I also prefer hormone-free birth control, so I opted for the ParaGard IUD (or IUC, as the company now calls it). You can have it removed at any time you choose. You don't have to leave it in for the long term. I would honestly look into the ParaGard if I were you. If you're not looking to have a baby for at least 2 years, IUDs/IUCs are actually very good options. If you're a good candidate, it's actually cheaper than almost any other form of birth control when used for at least 2 years, and the ParaGard company provides payment plans for those who don't have insurance (or those whose insurance doesn't cover it). Mine was covered completely by my insurance, so I didn't pay a penny, fortunately.
I've had it for almost a year, and I love it. No hormones, no need for barrier methods like condoms (I'm in a long term, mutually monogamous relationship), and the side effects have been minimal for me. I don't even notice it at all anymore. I've saved a lot of money, and it's actually a more environmentally friendly birth control method as well. We also enjoy being able to have spontaneous sex without putting something on or in first.
You can see their site for more info: http://www.paragard.com/home.php
There's also a great LiveJournal group called IUD Divas. You can find a lot of info there as well: http://community.livejournal.com/iud_divas Check the FAQ section at the left first: http://community.livejournal.com/iud_divas/218168.html
Also, FYI, the patch, Nuvaring, and Depo all contain hormones like those the pill.
If you're still not interested in the ParaGard, and you want hormone-free birth control (it's likely the hormones in the pill that were messing with your heart rhythm, so you should probably avoid all hormonal birth control), your options are fairly limited. You can always use barrier methods: condoms, diaphragms, sponges, or cervical caps. Those are hormone-free, although they should be used with spermicides, and some women are sensitive to the chemicals in spermicides. They also take a lot of the spontaneity out of sex, and in the heat of the moment, a lot of couples decide to forgo the use of a barrier method so as not to disrupt the moment (I'm not necessarily saying that you or your significant other would do this, but it's a definite downside to barrier methods).
There's also FAM, NFP, and other similar methods. FAM stands for "fertility awareness method", and NFP stands for "natural family planning". Basically, this means that you'd have to monitor your fertility signs and ovulation signals very, very closely in order to avoid having sex during fertile days. I do know many couples who have used these methods with great success, but they require a lot of dedication. You have to be militant about monitoring your ovulation and fertility signs, and you either have to avoid sex on fertile days, or use a barrier method on those days. It also takes a lot of education and preparation to learn how to read your body's signals accurately, and one little slip up, and you can end up pregnant. It's a very involved system.
You can read more about it here: http://www.suite101.com/article.cfm/birth_control/102539 If you're interested, I'd also look into getting the book "Taking Charge of Your Fertility".
There's a good birth control comparison chart here: http://www.womentowomen.com/sexualityandfertility/birthcontrolmethods.aspx
And another here: http://www.birth-control-comparison.info/