Throughput is one thing, seeks are another. The sps and mps stats in iostat can give you an idea if that's what you're waiting on. If this is abnormally high, even caching writes won't help much since the BBWC will just delay a bunch of seeks. What type of filesystem are you using?
I've always had a heck of a time troubleshooting these issues on linux, tools in the dtrace toolkit on solaris make troubleshooting issues like this dead simple. Perhaps an admin with more linux experience can chime in with how to figure out which process is tying up the most disk IO. After I've got a better idea on what's tying up the IO, knowing where to make tweaks is easier.
With most of the complaints on the list regarding performance always coming back to IO, I think I'll keep the better way to skin the RRD cat in the back of my mind today.
Sent from my Verizon Wireless BlackBerry
-----Original Message-----
From: Rupert Ogilvie <
[hidden email]>
Date: Fri, 6 Nov 2009 04:11:09
To: General OpenNMS Discussion<
[hidden email]>
Subject: Re: [opennms-discuss] Opennms with RAMSan = iowait?
Hi Duncan,
I've tried running the databases on SAS RAID with BBWC and on the PCI-E RAMSan drive, but the IOWait when on the RAID is even worse than with the RAMSan! I don't think the bottleneck is with the write to drive as the RAMSan can handle ~700MBs throughput and the iostat results show its not getting close to this (I think it intermittently peaks at around 200MBs). Any suggestions you've got would be great,
Cheers
Rupert
-----Original Message-----
From: Duncan Hill [mailto:
[hidden email]]
Sent: 06 November 2009 09:34
To:
[hidden email]
Subject: Re: [opennms-discuss] Opennms with RAMSan = iowait?
Quoting Rupert Ogilvie <
[hidden email]>:
> Hi
> Thanks for your replies. Apologies for the vagueness - it?s
> whichever memory is shown by the top command, I think that?s OS
> memory?
Used is a combination of actively used application memory, buffers and cache.
Linux will throw all of your RAM at caching disk activity until that
RAM is needed for applications. Perfectly normal beast.
> The IOWait seems to be due to thread limitations, we?ve got 8 cores
> but only 2 are ever being used. Is there a way of enabling
> multithreading? I?ve turned on the multithreading garbage collection
Odd. Unfortunately, I don't have an 8-core machine to run our ONMS
install (it runs on a DL380 G4) to see if I can replicate that.
I will say that we used to see hardly any I/O after we put in a BBWC
on the RAID card. I just looked at we're back to 90% wait on one CPU
when we get a mass RRD/SQL update going. Need more disks in that
server.
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.
http://p.sf.net/sfu/bobj-july_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQopennms-discuss mailing list
To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss--------------------------------------------------------
Intergence Systems is a limited company registered in England and Wales. Registered number: 04667187. Registered office: Quern House, Mill Court, Great Shelford, CB22 5LD.
The content of this message and any attached file are confidential and/or privileged and are intended for the recipient only. If you are not the intended recipient, any unauthorised review, use, re-transmission, dissemination, copying, disclosure or other use of, or taking of any action in reliance of this information is strictly prohibited. If you receive this message in error please contact the sender immediately and then delete this email from your system. Copyright in this email and attachments created by us belongs to Intergence Systems Ltd. Any attachment with this message should be checked for viruses before it is opened. Intergence Systems Ltd cannot be held responsible for any failure by the recipient to test for viruses before opening any attachments. Should you communicate with anyone at Intergence Systems Ltd by email you consent to us monitoring and reading any such correspondence.
--------------------------------------------------------
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.
http://p.sf.net/sfu/bobj-july_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQopennms-discuss mailing list
To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.
http://p.sf.net/sfu/bobj-july_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQopennms-discuss mailing list
To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss