The new rules prevent ISPs from limiting broadband access to rival content, online video or other forms of Internet traffic. It also would require providers to give more data on Internet speeds and service.
The FCC voted to impose fines and bring injunctions against offenders.
The new rules follow several years of debate over net neutrality. Following the 2008 order against Comcast for peer-to-peer throttling, regulators proposed transparent and non-discriminatory rules.
But when the U.S. Circuit Court of Appeals for the Federal Circuit struck down the Comcast decision earlier this year, the FCC proposed considering ISPs as partial common carriers.
The five-member FCC panel voted along strict party lines, with the three Democratic members endorsing net neutrality guidelines and two Republican commissioners dissenting.
"Today, for the first time, we are adopting rules to preserve basic Internet values," FCC Chairman Julius Genachowski said during the hearing. "These rules will increase certainty in the marketplace; spur investment both at the edge and in the core of our broadband networks, and contribute to a 21st century job-creation engine in the U.S."
While the net neutrality rules were approved, it could be tough to implement. The FCC will face opposition from Republican members of Congress and obvious court challenges.
