A mod_rewrite defense

mrG — Wed, 08 Dec 2004 16:26:44 -0500

A mod_rewrite defense against comment-spam found on Drupal.org, posted by candygenius ...

They are using indexers to find the comment links. I blocked the indexers as well as the originating spammer and cut down on the spam 99%. This block in .htaccess got rid of the latest and most persistent one I have seen.
RewriteCond %{HTTP_REFERER} 12\.163\.72\.13 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Fetch\ API\ Request) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (Microsoft\ Scheduled\ Cache\ Content\ Download\ Service) [NC,OR]
RewriteRule .* - [F]
Doesn't stop Google or anyone legitimate from indexing.

That code didn't work for me directly, but once I removed the NC, in each line, it worked just fine.

The same basic technique is applicable to a whole range of abuse-thwarting Apache rewrite rules; I picked up a list at Dive Into Mark to detect and defeat all sorts of suspect referrers and User-Agents with no business being in your site (spambots/spybots/offline downloaders) -- the list was probably overkill, banning long since forgotten offenders (I expect they evolve like flu virii) but still instructive enough to use as a template for blocking today's versions.

Blog-Comment SpamBots

mrG — Wed, 01 Oct 2003 17:07:29 -0400

Ok, it's real and it's official and it's here today: Automated blog-comment spambots are a reality.

Today I was hit by two test posts, and the odd grammar of it made it simple to test in feedster where I found the same odd and pointless test many times ... right down to the exact same mispelled URL used as the spammer's homepage link.

I posted the following to another victim's comments, but it bears repeating:

Looks like we have cause to alert people -- I was hit twice and Feedster shows a few others were hit too, and since the message is almost identical and I'll bet the URL too (in the one's I got, the URL didn't resolve, so Verisign picks it up)

And no, I don't see the value except for one thing: As a marker. I have heard that spammers who find open comment blogs will leave a marker that can be found with a subsequent search (Feester?) and that may explain why the previous blog-comment spam I've received always goes to the same three blog posts. I used to edit their comments to remove the URLs and then ridicule them, but maybe this is a mistake, maybe they just want that highly recognizable string to stay there so they can find it later ... or (as I believe is the case today) where robots can find them.

I got another today, simply some guy's name "rules" with a link to the same name dot com, nothing more, and that link was bogus as was the name@aol ... it must be some sort of marker they are leaving to bootstrap some other planned deployment. That's the only explanation I can think of that makes sense.

end of an era here folks. Open blog comments: 1997-2003.

A Proposal for Blogcrafters

I've written previously on the problems with some of the obvious solutions such as the bubble-obscured pass key; any viable solution has to be open to all platforms, including those used by blind surfers and low-tech browsers. The solution has to preserve the casual conversation, in situo to retain the forum sense of a thread, and it has to thwart the blog spammers by making each comment somehow accountable ...

I propose an HTTP-based comment moderation or author confirm scheme.

Just as we do for automated subscription confirmations, I propose we still allow our blog engines to accept posts from anyone, but before the post is displayed someone must confirm the message. The blog engine might send the comment back to the author with a hash-code URL they must click to have the comment confirmed and displayed.

Blog owners would still retain the power to override this comment-hold manually through the control panel, or perhaps they too are sent the email and can confirm or reject it. A whitelist of trusted parties wouldn't be sufficient since the spammer could just borrow emails they find on the same thread of comments.

Since many, probably most webhosts can't accept email, email reply to confirm is probably impractical. The simplest hack for a blog like MovableType would be a comment setting for moderated and/or confirmed comments (maybe this already exists?) where email alerts it already sends on new comments are extended to include the confirm (or instant delete?) URL in the message.

Is this possible as an mt-plugin? Looks like another job for Lazyweb

TeledyN - Blog-Comment SpamBots - Comments

A mod_rewrite defense

Blog-Comment SpamBots

A Proposal for Blogcrafters