sangiro 26 #1 April 11, 2003 Greetings! I know you've all probably been bugged by those annoying Proxy Error messages. I'm obviously aware of that and will be taking some steps to see if I can improve the performance again. At this time we're simply using more CPU power than my Dual 1.2 GHz processors can deliver. I've attached a graphic (for the geeks!) of last weeks server load. A load average of 2 means the 2 processors are both running at 100% capacity. As you can see we average at 1,93 and there are sustained periods where it stays between 4 and 6! So, I need to do a few things to see if we can lighten the load. In the near future we will: move to a new server with 15k scsi drives upgrade to MySQL 4 (query caching in MySQL 4 greatly improves performance) upgrade to ReiserFS file system (a high performance file system)This will require one to two hours of downtime to move the server over. As always I'll post a warning so the post-whores in our midst don't have too much angst. After this is done we'll monitor what happens and take further steps if needed. A few other notes: this is simply a product of high traffic. Lots of server requests and processes high bandwidth (data transfer) usage is not the cause of these errors. While not causing the performance issues, it costs money. there's a lot of storage space left, no need to delete emails or PMs, although it's good discipline to clean up a bit anyway. Thanks for your patience. Peace out! Safe swoops Sangiro Quote Share this post Link to post Share on other sites
RevJim 0 #2 April 11, 2003 Thanks for the heads up, HH. Now, for the nitty gritty, read between the lines stuff. . . . Quote This will require one to two hours of downtime Arrrhhh! Noooo! I'll never catch Clay at this rate! Quote * this is simply a product of high traffic Read as: This site is getting realy popular. Major Coolio! Quote * high bandwidth (data transfer) usage is not the cause of these errors. Post whoring with mega attachments does not directly cause errors, but: Quote While not causing the performance issues, it costs money. Cutting back aon the constant attachments of stuff you have found to Talk Back would save mucho bucks. Learn thy clicky, and LINK to the stuff, and the world will be a happier place. Quote clean up a bit anyway Full mailboxes bounce emails. Empty the SPAM people!It's your life, live it! Karma RB#684 "Corcho", ASK#60, Muff#3520, NCB#398, NHDZ#4, C-33989, DG#1 Quote Share this post Link to post Share on other sites
PhreeZone 20 #3 April 11, 2003 NOICE server name there HH! Let all of us know what we can do to make things work smoother.Yesterday is history And tomorrow is a mystery Parachutemanuals.com Quote Share this post Link to post Share on other sites
sunshine 2 #4 April 11, 2003 Quote Let all of us know what we can do to make things work smoother. Kicking my post whore ass off the forums would prolly help. ___________________________________________ meow I get a Mike hug! I get a Mike hug! Quote Share this post Link to post Share on other sites
smiles 0 #5 April 11, 2003 ... cannot imagine what you are up against, hope all goes well. Hey- dz.com rocks !! Only prob I have after posting--get "page has expired" when returning back to forum. Smiles Quote Share this post Link to post Share on other sites
ManBird 0 #6 April 11, 2003 Ouch! Not in MySQL 4 yet? Actually... XML has come through big time in SP1 of version 4. Of course, you aren't going to switch this site over, but you should look into it. Intel and Microsoft have worked together to release the "NewParser" property and XQL language for XML. XML has always had this great structure, but it sort of lived in theory, as it really slowed down if you got over even 1 MB. Now, a P4, Xeon, or Itanium will do all the work, and XQL allows node selection without traversion. After converting a 100 MB SQL Server database (post compacting) to XML, time between data requests and display decreased eightfold (ie, half a second instead of eight). Plus the structure is really easy to work with... no relations needed in a tree (for the most part). I'm loving it. Good stuff. This coming from someone that has been hardcore ODBC/SQL (and even MySQL) for years."¯"`-._.-¯) ManBird (¯-._.-´"¯" Click Quote Share this post Link to post Share on other sites
meatmissile 0 #7 April 11, 2003 On the graph, do you call those sharp spikes post-whore spikes ? Or do they occur after someone posted a few interesting attachments? -- ZZZzzzz.... Quote Share this post Link to post Share on other sites
jtval 0 #8 April 11, 2003 Im not computer genius but (sarcasm) why is there a lull on the weekends?(sarcasm off)My photos My Videos Quote Share this post Link to post Share on other sites
Muenkel 0 #9 April 11, 2003 Thanks for the translation Reverend. The only word I understood in the boss' post was "Greetings!".Chris _________________________________________ Chris Quote Share this post Link to post Share on other sites
hooked 0 #10 April 11, 2003 Quote Thanks for the translation Reverend. The only word I understood in the boss' post was "Greetings!".Chris Well said Chris, that's what I thought ! J -------------------------------------- Sometimes we're just being Humans.....But we're always Human Beings. Quote Share this post Link to post Share on other sites
jmpnkramer 0 #11 April 11, 2003 I have not had any issues, but Thank you for the info. I probably will not even notice the down time. Thank You much for all that you do. Sincerely, Kraaaaamer The REAL KRAAAAAAAAAAAAAAAAAMER! "HESITATION CAUSES DEATH!!!" "Be Slow to Fall into Friendship; but when Thou Art in, Continue Firm & Constant." - SOCRATES Quote Share this post Link to post Share on other sites
RevJim 0 #12 April 11, 2003 Not only am I a geek that understands geekeese, I am able to translate much to non geekeese which is mostly understandable by the non geeks. It's your life, live it! Karma RB#684 "Corcho", ASK#60, Muff#3520, NCB#398, NHDZ#4, C-33989, DG#1 Quote Share this post Link to post Share on other sites
DivaSkyChick 0 #13 April 11, 2003 Quote Not only am I a geek that understands geekeese, I am able to translate much to non geekeese which is mostly understandable by the non geeks. HUH?? --- www.facebook.com/mandyhamptonfitch Quote Share this post Link to post Share on other sites
SkyDekker 1,465 #14 April 11, 2003 Quote Actually... XML has come through big time in SP1 of version 4. Of course, you aren't going to switch this site over, but you should look into it. Intel and Microsoft have worked together to release the "NewParser" property and XQL language for XML. XML has always had this great structure, but it sort of lived in theory, as it really slowed down if you got over even 1 MB. Now, a P4, Xeon, or Itanium will do all the work, and XQL allows node selection without traversion. After converting a 100 MB SQL Server database (post compacting) to XML, time between data requests and display decreased eightfold (ie, half a second instead of eight). Plus the structure is really easy to work with... no relations needed in a tree (for the most part). I'm loving it. Good stuff. This coming from someone that has been hardcore ODBC/SQL (and even MySQL) for years. You might as well be speaking Chinese my friend.... Quote Share this post Link to post Share on other sites
Guest #15 April 11, 2003 Looks like you need to build a Beowulf supercomputer cluster like Google has, in order to handle the load... "The mouse does not know life until it is in the mouth of the cat." Quote Share this post Link to post Share on other sites
RevJim 0 #16 April 11, 2003 QuoteNow, a P4, Xeon, or Itanium will do all the work, and XQL allows node selection without traversion. And what happens to those using older Pentiums, Celerons, or AMD chips? How does XML make up for the bottleneck on older systems and/or servers?It's your life, live it! Karma RB#684 "Corcho", ASK#60, Muff#3520, NCB#398, NHDZ#4, C-33989, DG#1 Quote Share this post Link to post Share on other sites
MattM 0 #17 April 11, 2003 Quote Ouch! Not in MySQL 4 yet? Actually... XML has come through big time in SP1 of version 4. Of course, you aren't going to switch this site over, but you should look into it. Intel and Microsoft have worked together to release the "NewParser" property and XQL language for XML. XML has always had this great structure, but it sort of lived in theory, as it really slowed down if you got over even 1 MB. Now, a P4, Xeon, or Itanium will do all the work, and XQL allows node selection without traversion. After converting a 100 MB SQL Server database (post compacting) to XML, time between data requests and display decreased eightfold (ie, half a second instead of eight). Plus the structure is really easy to work with... no relations needed in a tree (for the most part). I'm loving it. Good stuff. This coming from someone that has been hardcore ODBC/SQL (and even MySQL) for years. Geeeeeeek! Matt Quote Share this post Link to post Share on other sites
dreamsville 0 #18 April 11, 2003 Is XML in the front end, with SQL Server still in the back end? Since we don;t have high traffic sites, Active Server Page with vbscript/javascript has worked OK. I just want to investigate, since I have only checked out XML in passing. Also, I want to say that I appreciate having this web site. It seems like it probably makes little, if any money, but provides a great outlet. When you figure in the tech and admin time, it proabably loses money. |I don't drink during the day, so I don't know what it is about this airline. I keep falling out the door of the plane. Harry, FB #4143 Quote Share this post Link to post Share on other sites
kevin922 0 #19 April 11, 2003 Funny to see the traffic go to almost nothing on the weekends.. where could everyone be??? :) Quote Share this post Link to post Share on other sites
beowulf 1 #20 April 11, 2003 ReiserFS coooool dz.com is linux shop Quote Share this post Link to post Share on other sites
hooked 0 #21 April 11, 2003 Quote Not only am I a geek that understands geekeese, I am able to translate much to non geekeese which is mostly understandable by the non geeks. Ummm...........okay.J -------------------------------------- Sometimes we're just being Humans.....But we're always Human Beings. Quote Share this post Link to post Share on other sites
skymama 37 #22 April 11, 2003 Quote Proxy Errors - Performance Issues Hopefully for you, your only performance issues are with your proxy. She is Da Man, and you better not mess with Da Man, because she will lay some keepdown on you faster than, well, really fast. ~Billvon Quote Share this post Link to post Share on other sites
jfields 0 #23 April 11, 2003 Quote Hopefully for you, your only performance issues are with your proxy. Proxy? I thought he said he was having performance issues with his doxy. Quote Share this post Link to post Share on other sites
RevJim 0 #24 April 11, 2003 Quote Quote Hopefully for you, your only performance issues are with your proxy. Proxy? I thought he said he was having performance issues with his doxy. No, he's so busy, he needs to distribute his performance By Proxy. Now accepting applications to be Sangiro's Proxy in performance related issues. It's your life, live it! Karma RB#684 "Corcho", ASK#60, Muff#3520, NCB#398, NHDZ#4, C-33989, DG#1 Quote Share this post Link to post Share on other sites
pilotdave 0 #25 April 11, 2003 QuoteIm not computer genius but (sarcasm) why is there a lull on the weekends?(sarcasm off) Skydivingmovies.com gets the same thing. About half as many hits on Saturdays as mondays or wednesdays. The hits peak on wednesday, drop steadily until saturday, then begin climbing back up on sunday, and shoot up on monday to near the wednesday peak. I kinda guess that internet traffic in general dies down on weekends (nobody's at work surfing the net), but I wonder if it's more extreme on skydiving sites cause skydivers tend to have their weekends occupied away from the computer. Dave Quote Share this post Link to post Share on other sites
