[php/mysql]search queries and handling - Tilted Forum Project Discussion Community

Outpour · 09-07-2004, 07:34 PM

I'm having some problems implementing my concept of searching on my blog [www.binaryelysium.com].

This is the way I am attempting to run a query. Searcher submits terms ie: "large red clowns."

What I want to do is give results in the following order (top with highest priority):

results that include all terms
results that include all terms minus one
results that include all terms minus two
and so on until one term remains this one is displayed at the very end

So... for the above example the user would get:

posts with the the striing "large red clowns"
posts with the string "large red"
posts with the string "large clowns"
posts with the string "red clowns"
posts with the string "large"
posts with the string "red"
posts with the string "clowns"

But the last three ordered in such a manner of being most recent.
Does anyone have any idea how I could go about doing it?

bacon · 09-09-2004, 04:36 AM

You could seperate the search string into words, and then generate a really bigass where clause with ANDs and ORs....

Or, seperate the searches into multiple queries and use mysql's UNION operation to concat the results together.

rukkyg · 09-09-2004, 11:26 AM

When I was writing my first search engine, I thought of doing this. Then I realized that Google only includes results that have all terms. The difference is that they show results that are pages that link to pages that have your search terms. Basically, you don't need to do what you're saying, because you can just construct people to search more generally (less terms).

ratbastid · 09-09-2004, 04:20 PM

There are really two issues here. Querying matches from the database is one. The other is ranking the responses by (let's call it) keyword density.

The first is easy--just split your query terms on spaces and do an OR query for each of them.

Then run a post-query ranking process where you test each record returned from your query for the presence of each of your search terms, and scores them. Then sort your responses by score, and bob's yer uncle.

Like this:

Code:

//Let's say that array $responses has strings of response values.

foreach ($responses as $record) {
   $result[$i++][record] = $record;
   foreach ($keyword as $word) {
       if (ereg($word, $record)) {
           ++$result[$i][score];
       }
    }
}

function user_sort ($a, $b) {
     if ($a[score] > $b[score]) {
         return 1;
      } elseif ($b[score] > $a[score]) {
         return -1;
      } else { return 0; }
}

sort($result, 'user_sort');

Note that I didn't actually run this code--this is strictly off-the-top-of-my-head. YMMV greatly.

Outpour · 09-12-2004, 01:50 PM

So to do the actual query would you do seperate queries for each term?

PHP Code:


			
//Let $_GET['q'] be the search terms

$search = explode(" ",$_GET["q"]);

foreach ($search as $term) {

   $query = "SELECT DISTINCT * FROM table WHERE field LIKE '%$term%' OR field2 LIKE '$term%' OR field3 LIKE '%$term%'";

}

Obviously that itself won't work.. $query would be over written each time the foreach loops. How would you do a query on all the terms and combine them into one large array?

Outpour · 09-12-2004, 02:06 PM

egh ugly highlight colors.

Anyways i figured out how to combine all the queries:

Code:

$search = explode(" ",$_GET["q"]);
$query = '';
foreach ($search as $term) {
	$query .= "SELECT DISTINCT * FROM table WHERE field LIKE '%$term%' OR field2 LIKE '$term%' OR field3 LIKE '%$term%' UNION ";
} 
$query = substr($query, 0, -6)
$searchquery = mysql_query($query,$db);

Outpour · 09-12-2004, 03:34 PM

Quote:

Originally Posted by ratbastid

--snip--
Then run a post-query ranking process where you test each record returned from your query for the presence of each of your search terms, and scores them. Then sort your responses by score, and bob's yer uncle.

Like this:

--snip--

Note that I didn't actually run this code--this is strictly off-the-top-of-my-head. YMMV greatly.

I am not sure exactly how to implement your example.. what is the purpose of returning 1, 0 or -1 on the sorting function.

Here is your code with some modifications for my situation:

Code:

$search = explode(" ",$_GET["q"]);
$query = 'SELECT * FROM posts WHERE 0=0 OR ';
foreach ($search as $term) {
	$query .= "'Body' LIKE '%$term%' OR 'intro' LIKE '%$term%' OR ";
} 
$query = substr($query, 0, -3);
$searchquery = mysql_query($query,$db);
while ($results = mysql_fetch_array($searchquery)) {
	foreach ($results as $i => $record) {
			 $result[$i++][record] = $record;
			   foreach ($search as $word) {
				  if (ereg($word, $record)) {
					  ++$result[$i][score];
				  }
			}
		}		
}

//The user_sort function here

$sorted = usort($result, 'user_sort');
print_r($sorted);

How do I go about outputting results in a sorted manner?

09-07-2004, 07:34 PM	#1 (permalink)
Outpour Upright	[php/mysql]search queries and handling I'm having some problems implementing my concept of searching on my blog [www.binaryelysium.com]. This is the way I am attempting to run a query. Searcher submits terms ie: "large red clowns." What I want to do is give results in the following order (top with highest priority): results that include all terms results that include all terms minus one results that include all terms minus two and so on until one term remains this one is displayed at the very end So... for the above example the user would get: posts with the the striing "large red clowns" posts with the string "large red" posts with the string "large clowns" posts with the string "red clowns" posts with the string "large" posts with the string "red" posts with the string "clowns" But the last three ordered in such a manner of being most recent. Does anyone have any idea how I could go about doing it?

09-09-2004, 04:20 PM	#4 (permalink)
ratbastid Darth Papa Location: Yonder	There are really two issues here. Querying matches from the database is one. The other is ranking the responses by (let's call it) keyword density. The first is easy--just split your query terms on spaces and do an OR query for each of them. Then run a post-query ranking process where you test each record returned from your query for the presence of each of your search terms, and scores them. Then sort your responses by score, and bob's yer uncle. Like this: Code: //Let's say that array $responses has strings of response values. foreach ($responses as $record) { $result[$i++][record] = $record; foreach ($keyword as $word) { if (ereg($word, $record)) { ++$result[$i][score]; } } } function user_sort ($a, $b) { if ($a[score] > $b[score]) { return 1; } elseif ($b[score] > $a[score]) { return -1; } else { return 0; } } sort($result, 'user_sort'); Note that I didn't actually run this code--this is strictly off-the-top-of-my-head. YMMV greatly.

09-12-2004, 01:50 PM	#5 (permalink)
Outpour Upright	So to do the actual query would you do seperate queries for each term? PHP Code: `//Let $_GET['q'] be the search terms $search = explode(" ",$_GET["q"]); foreach ($search as $term) { $query = "SELECT DISTINCT * FROM table WHERE field LIKE '%$term%' OR field2 LIKE '$term%' OR field3 LIKE '%$term%'"; }` Obviously that itself won't work.. $query would be over written each time the foreach loops. How would you do a query on all the terms and combine them into one large array? __________________ Nihilism is the organic, reasoned response to artificial chaos, the intentional chaos manufactured by government, religion and mass-media.

09-12-2004, 02:06 PM	#6 (permalink)
Outpour Upright	egh ugly highlight colors. Anyways i figured out how to combine all the queries: Code: $search = explode(" ",$_GET["q"]); $query = ''; foreach ($search as $term) { $query .= "SELECT DISTINCT * FROM table WHERE field LIKE '%$term%' OR field2 LIKE '$term%' OR field3 LIKE '%$term%' UNION "; } $query = substr($query, 0, -6) $searchquery = mysql_query($query,$db); __________________ Nihilism is the organic, reasoned response to artificial chaos, the intentional chaos manufactured by government, religion and mass-media.

09-09-2004, 04:36 AM	#2 (permalink)
bacon Upright	You could seperate the search string into words, and then generate a really bigass where clause with ANDs and ORs.... Or, seperate the searches into multiple queries and use mysql's UNION operation to concat the results together.

09-09-2004, 11:26 AM	#3 (permalink)
rukkyg Loser Location: RPI, Troy, NY	When I was writing my first search engine, I thought of doing this. Then I realized that Google only includes results that have all terms. The difference is that they show results that are pages that link to pages that have your search terms. Basically, you don't need to do what you're saying, because you can just construct people to search more generally (less terms).