"A man truly free is under no obligation to obey any injunction, human or divine."
-Ragnar Redbeard

I can choose to act "positively" and help woman (by calling police, intervening, etc.). I can choose to act "negatively" and harm woman (by attacking her, warning attacker if somebody is approaching, etc.), or I can choose to act neutrally and do nothing. In this case I am demonstrating that I am neither for nor against whatever is happening. That I wish to keep "my world", "my system", as much separate and independent from the rest of the world as possible. Those claiming that by isolating yourself from event, and thus not affecting it in any meaningful way, somehow makes you good or bad are simply biased towards one or another.

So why I live in society? Because I have clear and mutually beneficial contract with society. It's called "law". I work, provide for society, pay my taxes, don't harm anyone, etc. and government ("representatives" of society), in return, allows me to engage in trade on their territory, provides health care, etc. Law states full extent of what society expects of me, of my responsibilities, and helping those in need is not amongs them. I am simply playing by your rules, society. Rules, according to which I am keeping my end of the bargain. I am doing what you wanted, so you lost all rights to call me bad.