Freedom versus license

I think you misunderstand freedom: in a free society, you would be free to do whatever you want, and so would everyone else.

Your freedom to swing your fist ends where my face begins.

Iron Marxist stated the same thing that I recently read in a book called "For Us, the Living" by Robert Heinlein:
"In a free society, the government has no right to interfere with the actions of a citizen unless those actions present a clear and present danger to another human being."

Justice is an abstract concept; there is no thing in the space-time-matter continuum to which you can point and say: "This is Justice."

Damage, however, is real and tangible; it can be measured. No action or life-style may be forbidden other than that which would harm another person, and violate the concept that freedom to live your life is enjoyed by everyone in society, not merely those strong enough to get away with it.

"...everybody would be doing what they want."
Yes, providing they respect the rights of others to enjoy that same freedom.

Anything else is kindergarten ethics.