Tilted Forum Project Discussion Community - View Single Post

tuckdiddy · 04-08-2004, 08:13 PM

Does anyone know the basics of how a general search engine works?

I'm curious as to whether a SQL database could be used for such a thing. For instance, say a simple search engine matches the keywords you type in using boolean AND (it matches all of the keywords). That part is simple enough to do in SQL, but the ranking of the results is where I get stumped.

Google has a complex algorithm that ranks a page based on it's popularity: it ranks higher if more pages link to that site.

What if I just wanted to make a somewhat search engine using a SQL database that would rank the results based on the number of times a keyword appears in the searched text.

Does any have an idea how this would be accomplished? It seems difficult to find general theory about search engines on the internet. That might be because if someone comes up with a good idea, they patent the technology?

Another question is, would your standard database server (Oracle, MySQL, SQL Server) be able to do text searches through millions of records? If not, what kind of technology would be used here?

04-08-2004, 08:13 PM	#1 (permalink)
tuckdiddy Upright	How to build a search engine? Does anyone know the basics of how a general search engine works? I'm curious as to whether a SQL database could be used for such a thing. For instance, say a simple search engine matches the keywords you type in using boolean AND (it matches all of the keywords). That part is simple enough to do in SQL, but the ranking of the results is where I get stumped. Google has a complex algorithm that ranks a page based on it's popularity: it ranks higher if more pages link to that site. What if I just wanted to make a somewhat search engine using a SQL database that would rank the results based on the number of times a keyword appears in the searched text. Does any have an idea how this would be accomplished? It seems difficult to find general theory about search engines on the internet. That might be because if someone comes up with a good idea, they patent the technology? Another question is, would your standard database server (Oracle, MySQL, SQL Server) be able to do text searches through millions of records? If not, what kind of technology would be used here?