FeedWordPress Duplicate Post Filter

This is a filter for the FeedWordPress plugin. If you don’t use FeedWordPress this plugin will not be useful to you.

I wrote this filter after seeing the same post in my database 32 times. Maybe you’ve had the same problem, and I hope this filter helps you solve it.

The filter works by hooking the “syndicated_post” action of FeedWordPress and the “save_post” action in the core of WordPress.

For each potential post from a feed, the plugin attempts to find an identical SHA1 hash of the first 1024 non-whitespace characters (stripped of HTML tags) of the post content.
If it finds a match the post is skipped. If not, the post is processed and a hash is inserted into the post’s metadata. (Stored as key _dpf.)

NOTE: No filter is going to be 100% accurat…