Keyword Filtering Function in PHP

[Andy]

This is something I started coding today as part of a project so I thought it would be an idea to pose the challenge of coding things to others and then post my solution once I ironed it out.

The challenge here is to code a PHP function that accepts a string of what is supposed to be a comma delimited list of keywords/phrases and outputs a list of useable keywords in lower case separated by commas.

So the function sanitizes the input and removes duplicate keywords. Then it returns the keyword list.

The input can be any sequence of characters.

Here is what I coded so far but it is not finished yet:

Code:

<?php
function clean_keywords($keywords) {
$keywords = ereg_replace('[^, a-z]', '', trim(strtolower($keywords))); //remove unwanted chars
$keywords = ereg_replace(' +', ' ', $keywords); //reduce long spaces to one space
$keywords = ereg_replace(',+ *', ',', $keywords); //after the above reductions condense any wiped out entries
$keywords = trim($keywords, ","); //get rid of stragglers at the left and right extremities
$ka = explode(',',$keywords); // separate out the remaining keywords
// Next we will remove duplicate keywords
// If exists $newkey which is $value then remove the current - thinking out loud now ...
foreach($ka as $key=>$value) echo strpos($keywords,$value); //if (strpos($keywords,$value) !== false) unset($ka[$key]);

// Remove short words

// Remove offensive words

// Spell check the remaining words

$keywords = implode(', ',$ka); // Convert the array of words back into a string
return $keywords;
}
?>

As you can see, it is maybe trickier than you first thought when you have to imagine every possible scenario of what your website visitors may enter to your web form and submit.

Further thoughts: the keywords should be limited to words that are longer than say 2 characters. Also, should this apply to the individual words of a phrase?

Then, there should be a limit maybe to the number of keywords in the list and only the best keywords retained. This is because the keyword list will be used in a meta tag and used to form a url stub for a web page.

Also, the words should be passed through a bad words filter to remove offensive words.

[Andy]

Here is the finished version of the code that I came up with:

Code:

<?php
function word_ok($w) {
global $bad_words; // This is an array of bad words that is defined outside of the function.
foreach ($bad_words as $bad) if (strpos($w,$bad)!== false) return false;
return true;
}

function clean_keywords($keywords) {
$keywords = ereg_replace('[^, a-z]', '', trim(strtolower($keywords)));
$keywords = ereg_replace(' +', ' ', $keywords);
$keywords = ereg_replace(',+ *', ',', $keywords);
$keywords = trim($keywords, ",");
$ka = explode(',',$keywords);
$words = array();
foreach($ka as $value) if (strlen($value) > 2) if (word_ok($value)) $words[$value] = $value; // Reject short words, bad words and avoid duplicating words
$keywords = implode(', ',$words);
return $keywords;
}
?>

In this code I use arrays to easily split and combine the list of keywords.

I used the keyword as an index key for a temporary array, so if there are duplicate words, they are not added as new array elements. This is much simpler than counting how many times words appear in the list and so on.

To filter out bad words, I added another function word_ok()

Related to this, I developed a function to create keyword-rich URLs. It takes the keyword list or if this is empty, it uses the title of the web page for the words (in this case they are passed to the keywords script to clean them up).

Then the software checks for an already existing URL like this in the database and appends a number to it to make it unique. In Wordpress, this is done too when you choose to have keyword rich URLs, however I did not examine their code since it can be very complicated tracing through all the functions that are involved. also, I wanted to have my own solution to avoid having to apply the GPL licence.

This is all aimed at automated SEO of user-entered data.

The other thing I will do is to write some Javascript to filter the characters as they are typed on the keyboard. This helps to force/educate users to enter the correct format data. The PHP is more complex since it has to assume that any kind of data was entered such as by hackers trying to break the script.