Can't you make it so that the searches have a much lower priority on the server's CPU than replies and page views?

___________________
I'm the trouble starter, fuckin' instigator.
I'm the fear-addicted, danger illustrated.

quote:

Originally posted by Cosmic Fur Can't you make it so that the searches have a much lower priority on the server's CPU than replies and page views?

Unfortunately, no. It locks certain database tables during the search and then all the other queries just wait in queue - I'd set it to dump any search that doesn't return a result within 20 seconds but you can't do that with mysql.

___________________

"In a world of illusion you only see what you feel"

MySQL blah blah ... Text files dammit!

___________________
[alk]

quote:

Originally posted by Swamper Unfortunately, no. It locks certain database tables during the search and then all the other queries just wait in queue - I'd set it to dump any search that doesn't return a result within 20 seconds but you can't do that with mysql.

Oh, I see the picture now. Yeah, no easy way of getting around that, except maybe creating two identical databases, one for searching and one fo browsing, but that's a retarded way of doing it.

___________________
I'm the trouble starter, fuckin' instigator.
I'm the fear-addicted, danger illustrated.

quote:

Originally posted by Cosmic Fur Oh, I see the picture now. Yeah, no easy way of getting around that, except maybe creating two identical databases, one for searching and one fo browsing, but that's a retarded way of doing it.

Actually it is a smart way, I could do it across two servers - the problem lies here:

DB Server 1 - master - has all the current forum data
DB Server 2 - slave - has older version of the forums (used for search)

Now you're on DBServer1, if I throw you over to DBServer2 for the search then it has to talk back to DBServer1 with whatever it finds - the problem being that what it sends back is uncompressed - and considering how many searches are done here a day that is a ton of wasted bandwidth.

Alternative -> Allow you to request a search from here over to DBServer2 and have the results fed back to you from a web server running on DBServer2 - but - when you click a thread/post it would redirect you to DBServer1 so you could post your reply and everyone could see it

I tried having an alternate copy of the database on a whole other SCSI drive but while that helped in certain regards it was still tying things up while seeking it out

It'll be hectic for a bit.. eventually I need 2 or 3 opteron servers.. 1 for web.. 1 for forums and 1 for search.. with the ones for forums/search linked by crossover cable and with raid 10

(The built-in Mysql clustering doesn't work too well with Vbulletin/MyIsam tables)

>>> Outside of all this - the search page needs to be revamped and I need to throw a Google box in there too since it is much easier searching that way and half the forums are already in there.

___________________

"In a world of illusion you only see what you feel"

Last edited by Swamper on Sep-29-2005 at 22:38

nerds

___________________
Deviant Art - Smuncky
My Photos on Flickr
If anyone wants to share their clubbing/nightlife pics go to Toronto Nightlife
"The best car-related innovation we have is not to improve the car, but eliminate the need to drive it everywhere we go.” -Alex Steffen

more info then i needed to know

at least I understand why though.

___________________
Trance in my pants

it's never slow for me. (well very seldom). But I do get frustrated by the error page at certain times of the day.

Maybe it's not TA, it's your computer. You should stop downloading porn torrents the whole day.

ahahaa. Yeah I know. Thing is I dont have this issue on other boards I frequent, so the pr0n cant be creating that many issues

___________________
Trance in my pants

quote:

Originally posted by Euphorica ahahaa. Yeah I know. Thing is I dont have this issue on other boards I frequent, so the pr0n cant be creating that many issues

I suspect that very few other boards you frequent have close to 5 million posts - and those that do - restrict their search to registered users because they know it's a huge bottleneck.

___________________

"In a world of illusion you only see what you feel"

Why not just filter out some of the really stupid search phrases?

I know it won't stop the problem, but it might at least reduce it somewhat. Just don't let people search for "trance" or "tiesto" or "armin", etc.

___________________
My party schedule:
2009-02-21 - DJ Attention @ I'm So Popular
2009-06-18 - DJ Annoying @ People Need To Know Where I'll Be
2012-11-32 - DJ Insufferable ɸ Or At Least the Stalkers I Complain About
2048-06-66 - Spastic & Whocares ¶ Although I'm Actually Flattered
9999-45-81 - Tweaker Gimp ☼ I Probably Won't Even Go To This But I Have To Make Sure I Fill Up All The Available Space Here