Let Google do the search indexing for us - Tilted Forum Project Discussion Community

BentNotTwisted · 12-08-2003, 09:20 AM

With the temporary server issues and search being disabled, I was wondering if it would be possible to set up the site so Google can index the TFP pages. This would have the benefit of relieving the TFP server of the overhead of creating search indexes. I have seen Google do this with other sites (techrepublic.com) where it will return search results for members only sites, but you have to log into the site if you want to view the page. One other possible benefit is it might bring in some new members who found us via Google.
Your thougths?

Dilbert1234567 · 12-17-2003, 02:14 AM

im not sure on this but i think it would be jsut as bad, if not worse if google had to talk to the server to search it would just compound the problem rught?

BentNotTwisted · 12-17-2003, 07:09 AM

Quote:

Originally posted by Dilbert1234567
im not sure on this but i think it would be jsut as bad, if not worse if google had to talk to the server to search it would just compound the problem rught?

I'm no expert by any means, but I believe Google would hit each thread once and then use its own indexing algorithms to allow searches. After that it would occasionally come back for a refresh.

cheerios · 12-17-2003, 07:46 AM

bent: think of how often it'd have to hit us to stay current...

BentNotTwisted · 12-17-2003, 07:49 AM

Quote:

Originally posted by cheerios
bent: think of how often it'd have to hit us to stay current...

True, that had not occured to me. Google would have to hit the entire site several times a day to have reasonably current indexes.
Nevermind, it was a dumb idea. I was just hoping we could use some of Googles advanced search features.

cowudders14 · 12-20-2003, 03:28 PM

The biggest problem I can see with this is that, AFAIK, google can't see the site. You have to login to be able to read anything, and google, surprise surprise, can't login!

I take your point about other sites having had access set up, but I do wonder if Halx wants to go through that process. I've no idea how complicated it would be.

Hmmm, interesting idea....

MSD · 12-23-2003, 10:03 PM

When the forums on ultimatemetal.com were indexed by google, the views column hit 10 digits in a matter of hours

The average thread was viewed by the search spider 800,000 times per day.

Our server would have a heart attack. Death by Google.

12-08-2003, 09:20 AM	#1 (permalink)
BentNotTwisted Optimistic Skeptic Location: Midway between a Beehive and Centennial	Let Google do the search indexing for us With the temporary server issues and search being disabled, I was wondering if it would be possible to set up the site so Google can index the TFP pages. This would have the benefit of relieving the TFP server of the overhead of creating search indexes. I have seen Google do this with other sites (techrepublic.com) where it will return search results for members only sites, but you have to log into the site if you want to view the page. One other possible benefit is it might bring in some new members who found us via Google. Your thougths? __________________ IS THAT IT ???!!! Do you even know what 'it' is? When the last man dies for just words that he said... We Shall Be Free

12-17-2003, 02:14 AM	#2 (permalink)
Dilbert1234567 Devils Cabana Boy Location: Central Coast CA	im not sure on this but i think it would be jsut as bad, if not worse if google had to talk to the server to search it would just compound the problem rught? __________________ Donate Blood! "Love is not finding the perfect person, but learning to see an imperfect person perfectly." -Sam Keen

12-20-2003, 03:28 PM	#6 (permalink)
cowudders14 Bringer of good Moos... Location: Midlands, UK	The biggest problem I can see with this is that, AFAIK, google can't see the site. You have to login to be able to read anything, and google, surprise surprise, can't login! I take your point about other sites having had access set up, but I do wonder if Halx wants to go through that process. I've no idea how complicated it would be. Hmmm, interesting idea.... __________________ Moo! I'm mooey!

12-17-2003, 07:46 AM	#4 (permalink)
cheerios Banned Location: 'bout 2 feet from my iMac	bent: think of how often it'd have to hit us to stay current...

12-23-2003, 10:03 PM	#7 (permalink)
MSD The sky calls to us ... Super Moderator Location: CT	When the forums on ultimatemetal.com were indexed by google, the views column hit 10 digits in a matter of hours The average thread was viewed by the search spider 800,000 times per day. Our server would have a heart attack. Death by Google.