|
Something I don't quite understand about TA's problems with traffic
|
View this Thread in Original format
| Lira |
TA uses PHP and so does Yahoo, for example - but here we often get that "too many users connected to database" error, meanwhile this never happened in Yahoo (as I recall it). Is this a problem with mysql then? I reckon it is but, why? I'd like to know how reliable the PHP + MySQL combo, which TA seems to be using, is for great traffic.
ps.: I'd like to also thank those who have been helping me in this matter so far, thank you very much indeed :) |
|
|
| jonSun |
| I thought this thread was gonna be about Tiesto's Traffic. |
|
|
| robin |
| quote: | Originally posted by jonSun
I thought this thread was gonna be about Tiesto's Traffic. |
+1 :o
Curious though |
|
|
| loconet |
It has little to do with php itself so you can't compare it to Yahoo using php as well... From what I've seen and from what swpamz has mentioned, the bottle-neck is (as with most web apps) the db (in this case MySQL). Remember.. This is even more so in the case of TA's db which is several gigs in size, rich in dynamic data (lots of db queries - reads and writes on the same box?) and full of refresh whores.
At work, I setup a cluster of servers to serve a site that at times gets 800+ requests / second with lots of db traffic but unlike TA, the traffic is mostly reads so the data doesn't change as often. This allows me to replicate dbs, cache to server disks as well as getting a good mysql cache hit rate -- on TA this more challanging.
On top of that, the boards probably have SQL everywhere which I'm sure Swampz has fun optimizing during his spare time to avoid situations where the db is so slow it maxes out in connections - ie: user posts searching :P
The number of users might not matter depending on what it is that the users are doing ie: is the page they are hitting doing dozens 1s+? queries on the db, if so .. 300 users is enough to shake up the db.. Assuming this puppy is running innodb, to avoid the whole table locking issue, there must be a lot of db hugging sql considering how often we see the max connections error, but then again..it might be just a hardware problem :P.
In any case, I've said it before, Del needs to get off his ass and go with flat files. This sql is overrated :D |
|
|
| Massive84 |
Actually more the none specefic searches.
Like...
Tiesto
Many people do this and..well it's better to put it this way..how many topics do NOT contain the name tiesto... |
|
|
| robin |
| quote: | Originally posted by Massive84
Actually more the none specefic searches.
Like...
Tiesto
Many people do this and..well it's better to put it this way..how many topics do NOT contain the name tiesto... |
This one does now :o |
|
|
| Swamper |
The problem isn't with traffic exactly, it's with database bottlenecks.
vBulletin uses MyISAM tables (not InnoDB) - so table locking issues arise. When you have so many people doing searches a lot of problems come up when the resultset is large (terms like 'Tiesto' or 'tracklist' just kill things). I've tried various things, some of which even vBulletin developers barely heard of - but it's not enough to handle the load.
Josh your posts are annoying as all hell because they have a tone of me not giving a about the error messages and/or not having adequate hardware to handle the current userbase. Search in vB is flawed as I've mentioned in prior posts regarding this issue - ALL LARGE vB forums have search issues - which is why if you go to big-boards.com and search for the biggest vB boards you'll see most of them do not allow unregistered users to search. There are ways around it - one way is to set up another dedicated server solely for search but the problem there is increased inter-server bandwidth usage as the requesting client goes to the main TA forum/web server to do the search and then from there it goes out to another server to return the resultset (whose content is NOT compressed) - at least now all stuff you get from TA is 1/6th its original size thanks to gzip compression.
Live with it for now. |
|
|
| jonze234 |
| quote: | Originally posted by josh4
create a forum where you and moderators can post would be good places to start. |
i think they already have it |
|
|
| Jocker |
| quote: | Originally posted by Lira
TA uses PHP and so does Yahoo, for example |
i SERIOUSLY doubt that. maybe in their first days they were. neither php not mysql are ready for enterprise. mysql doesn't even have stored procedures. in fact, due to its simplicity, mysql is very fast (probably the fastest database) for small number of users. but when the number of concurrent connections grow - the problems start occurring.
moreover, you can't compare yahoo to ta. yahoo literally has server farms: when you type www.yahoo.com in your browser or search, the load balancer sends your request to one of several hundreds of web-servers, which, in turn, connect to one of hundreds of database servers which run in one huge cluster utilizing a fully-redundant and freakink fast SAN setups, which allow network storage to perform at speeds very close of speeds of the local scsi harddrives.
what ta needs to alleviate the problems is either a complete makeover (with switching to asp.net or java technology with more stable rdbms like oracle or ms sql), which is probably not feasible at current state, or to go into a cluster, with putting two-three servers to load-balance or at least to move the search database to a completely different volume (with its own scsi controller and disks). |
|
|
| Swamper |
| quote: | Originally posted by josh4
Thats what it seems like because even if you are making these supposed posts around the forums not everyone sees them. |
I wasn't talking about other people, I was talking about YOU - and the thread in question that I replied in previously was in response to YOU. And what is this 'supposed' business - if I said I replied to something previously it's because I did - why must your posts have such an anal tone?
jocker - vBulletin only runs on MySQL so moving to another server isn't the answer - it'd be nice if it was innodb since table locking is the main issue here. As for moving search to another volume/scsi drive - that was done for about 2 months and there wasn't much of a difference bottleneck wise. As it stands now there are two 10k drives in a raid-0 config. 15k will be next upgrade. |
|
|
| nchs09 |
| quote: | Originally posted by josh4
Thats what it seems like because even if you are making these supposed posts around the forums not everyone sees them. When you don't explain whats going on where everyone can see all thats left is for people to draw their own conclusions.
Making regular (key word: regular) updates on the status of TA (and this 2.0 we keep hearing nothing about) would help to alleviate this problem. More than 2 sentences on the front site_news section or even create a forum where you and moderators can post would be good places to start. |
dude ya for real. he does it for free.... so leave him alone . go swapmah |
|
|
| d-miurge |
| quote: | Originally posted by nchs09
go swapmah |
Le Webmonstah for president! :wtf: |
|
|
|
|
