IDS Forum

Re: rebuild making system run SLOOOooowwww...

Posted By: Jonathan Leffler
Date: Friday, 4 November 2005, at 2:11 a.m.

In Response To: RE: rebuild making system run SLOOOooowwww... (Konikoff, Rob \(Contractor\))

On 11/3/05, Konikoff, R.... <rob.konikoff@us.army.mil> wrote:
>
> The number one question was a tie between:
> "Did they build it as RAID5?"
> "Did you run UPDATE STATISTICS?"
>
> The answer is: UPDATE STATISTICS runs every morning at about 4AM
> production time and NO RAID5! (www.BAARF.com <http://www.BAARF.com>)
>
> From what I'm told, they did NOT change the actual db portion of raw space
> and spindle configuration.
>
> A lot of feed back from a lot of people on what to check... THANKS! I've
> elected to reply to Jonathan Leffler simply because he included the ids
> forum in his line and reply all included it. :)




I take it that the problem hasn't gone away? And any extra help would be
useful...

You're likely to need to explain more about the disk setup. Do you have any
statistics on database performance from before the changes? It really is
something every production system should have - a basic log of the
performance data (onstat output, maybe sar output) that is logged routinely.
Then you can backtrack in situations like this and see what's different.

OK - it's easy to be wise after the event.

So, you have 4x74GB disk drives - in an array? So RAID is in use? Which RAID
level? Previously, you were only using 32 GB of each disk. Now all 74 GB are
in use. What is happening on the other 42 GB? If there is disk intensive
activity occurring on the other section, the performance of IDS will suffer.

Previously, how was IDS given the space to work with? Were you using a LVM?
Are you now using an LVM? Were the devices raw (character devices) or cooked
(plain files, or block devices). Are you using sections of a single disk?
Given that IDS is 7.23, you can only be using 2GB chunks; how many of them
are you using? Any mirroring? Are the devices the same now as they were
before? Have you tracked through any and all symbolic links to ensure you
know what IDS is using? Have you got any measurements of the disk
performance (ignoring IDS) before and after the change? Could there be a
wonky connection as a result of moving something - something that slows
things down because of retries but doesn't fail over time?

Have you got KAIO configured? Should you? Have you got enough AIO VPs
configured?

Are you positive the only change was in the disk i/o subsystem?


________________________________________
> From: Jonathan Leffler [mailto:jleffler.iiug@gmail.com]
> On 11/2/05, Konikoff, R.... <rob.konikoff@us.army.mil> wrote:
> We have old stuff here.
>
> INFORMIX-OnLine Version 7.23.UC11
> HP-UX 11.0
>
> We just updated one of our systems to use an array of four 74GB disks.
> We had only been using 32GB each of our 74GB disks. We didn't change
> the operating system, just rebuilt the foot print to allocate the unused
> space.
>
> The engine still uses RAW space. Is it common to experience a MAJOR
> slowdown in performance?
>
> What was taking about an hour for a workhorse query now takes about four
> hours. Captive User Interface scripts that used to take about five
> seconds to build and display user data now takes about 45 seconds and
> you can actually watch it put data up on the screen one piece at a
> time...
>
> This is killing me. The developers/OS engineers are telling me this is
> normal and to basically (insert expletive) shut up.
>
> QUESTION: Is this normal? (about the slowdown, not the abuse)
>
>
> No - it shouldn't be normal. But if you just rebuilt your systems, did you
> remember to run UPDATE STATISTICS? (How often does that answer -
> counter-question - come up?).
>
> Another possibility is you're using block devices instead of raw
> (character) devices. However, that shouldn't be order-of-magnitude
problems;
> UPD STATS really can make that much difference. (Unless they did some RAID
5
> -- quiet Art; it's only a possibility at this point :-) -- behind the
> scenes.)
>



--
Jonathan Leffler #include <disclaimer.h>
Email: jleffler@earthlink.net, jleffler@us.ibm.com
Guardian of DBD::Informix v2005.02 -- http://dbi.perl.org/

Messages In This Thread

• rebuild making system run SLOOOooowwww...
  Konikoff, Rob \(Contractor\) -- Wednesday, 2 November 2005, at 4:13 p.m.
  • Re: rebuild making system run SLOOOooowwww...
    Jonathan Leffler -- Thursday, 3 November 2005, at 12:16 a.m.
    • RE: rebuild making system run SLOOOooowwww...
      Robert Roussey \(MIS\) -- Thursday, 3 November 2005, at 8:35 a.m.
  • RE: rebuild making system run SLOOOooowwww...
    Simmons, Keith -- Thursday, 3 November 2005, at 3:23 a.m.
  • RE: rebuild making system run SLOOOooowwww...
    Konikoff, Rob \(Contractor\) -- Thursday, 3 November 2005, at 1:15 p.m.
    • Re: rebuild making system run SLOOOooowwww...
      Jonathan Leffler -- Friday, 4 November 2005, at 2:11 a.m.

IDS Forum is maintained by Administrator with WebBBS 5.12.