The first amendment doesn't go THAT far

Something I heard on a podcast reminded me of a heckler that yelled "health care for everyone!" during the speech by Ron Paul at ASU. Honestly, he was being loud and obnoxious, but I ignored him and kept listening to the speech. Someone sitting in the seats that were set up for the event, however, did not agree and asked him kindly to stop yelling, to which he replied "freedom of speech bro".

This is the most common misconception with regards to the freedom of speech, and the first amendment, that you hear nowadays. Let it be clear that the first amendment does NOT defend you against someone asking you to shut the hell up. The first amendment is there to prevent the GOVERNMENT from violating speech. It only applies to the government-individual relationship, not the individual-individual relationship. As much as you have the right to speak freely, I have the right to tell you to shut the hell up.

I thought the response by the heckler was rude, and using the first amendment to do it just isn't right.