I'm all for fighting terrorists, but what scares people is the fact it has the potential to be misused ala Section 215. What's section 215? quoted from the slate article:
"What it does: Section 215 modifies the rules on records searches. Post-Patriot Act, third-party holders of your financial, library, travel, video rental, phone, medical, church, synagogue, and mosque records can be searched without your knowledge or consent, providing the government says it's trying to protect against terrorism."
However, the 2257 inspections are just a misguided attempt at battling conduct that is festering elsewhere other than the porn industry, but the reasonable fear about this portion of the Patriot Act is that these new powers have the potential to be used for the wrong purpose. I agree with you most people are just having irrational knee-jerk reactions to the Patriot Act w/o ever realizing exactly why or even reading the damn thing!