One of the main reasons would be the mental scars (and trauma?) of eating another human being. Seriously, think about it. You'd say 'No' pretty quick, and some of that would be Morality/other emotions (I think?) kicking in and giving you a jolt of "that person was like me, and I'm eating her/him, am I a freaking monster?"
Also, if you were in a state of starvation, you'd probably be dehydrated, too. If you eat when you're dehydrated, your hunger goes away, and your dehydration becomes worse. (I think that's how it goes.)
This is different than animals because of sentenence. (How do you spell that?)
Also, yeah, there's sickness, ect, but really, even if you willed yourself to, if you hadn't been brought up a cannibal, then you'd probaby reconsider very quickly.
The reason why it varies is because if you're born into it, then you're (most likely) in a society of people who are also cannibals, so it wouldn't feel as bad as if you weren't - because you were raised that way. But then again, what do I know? I'm not a cannibal.
Please excuse any inaccuracy, it's pretty late over here.