Join IIUG
 for   
 

Informix News
18 Nov 13 - ZDNet - Top 20 mobile skills in demand... Read
09 Sep 13 - telecompaper - Shaspa and Tatung have shown a new smart home platform at Ifa in Berlin. Powered by the IBM Informix software... Read
06 Sep 13 - IBM data magazine - Mission Accomplished - Miami, Florida will be the backdrop for the 2014 IIUG Informix Conference... Read
01 Feb 13 - IBM Data Magazine - Are your database backups safe? Lester Knutsen (IBM Champion) writes about database back up safety using "archecker"... Read
14 Nov 12 - IBM - IBM's Big Data For Smart Grid Goes Live In Texas... Read
3 Oct 12 - The Financial - IBM and TransWorks Collaborate to Help Louisiana-Pacific Corporation Achieve Supply Chain Efficiency... Read
28 Aug 12 - techCLOUD9 - Splunk kicks up a SaaS Storm... Read
10 Aug 12 - businessCLOUD9 - Is this the other half of Cloud monitoring?... Read
3 Aug 12 - IBM data management - Supercharging the data warehouse while keeping costs down IBM Informix Warehouse Accelerator (IWA) delivers superior performance for in-memory analytics processing... Read
2 Aug 12 - channelbiz - Oninit Group launches Pay Per Pulse cloud-based service... Read
28 May 12 - Bloor - David Norfolk on the recent Informix benchmark "pretty impressive results"... Read
23 May 12 - DBTA - Informix Genero: A Way to Modernize Informix 4GL Applications... Read
9 Apr 12 - Mastering Data Management - Upping the Informix Ante: Advanced Data Tools... Read
22 Mar 12 - developerWorks - Optimizing Informix database access... Read
14 Mar 12 - BernieSpang.com - International Informix User Group set to meet in San Diego... Read
1 Mar 12 - IBM Data Management - IIUG Heads West for 2012 - Get ready for sun and sand in San Diego... Read
1 Mar 12 - IBM Data Management - Running Informix on Solid-State Drives.Speed Up Database Access... Read
26 Feb 12 - BernieSpan.com - Better results, lower cost for a broad set of new IBM clients and partners... Read
24 Feb 12 - developerWorks - Informix Warehouse Accelerator: Continuous Acceleration during Data Refresh... Read
6 Feb 12 - PRLOG - Informix port delivers unlimited database scalability for popular SaaS application ... Read
2 Feb 12 - developerWorks - Loading data with the IBM Informix TimeSeries Plug-in for Data Studio... Read
1 Feb 12 - developerWorks - 100 Tech Tips, #47: Log-in to Fix Central... Read
13 Jan 12 - MC Press online - Informix Dynamic Server Entices New Users with Free Production Edition ... Read
11 Jan 12 - Computerworld - Ecologic Analytics and Landis+Gyr -- Suitors Decide to Tie the Knot... Read
9 Jan 12 - planetIDS.com - DNS impact on Informix / Impacto do DNS no Informix... Read
8 Sep 11 - TMCnet.com - IBM Offers Database Solution to Enable Smart Meter Data Capture... Read
1 Aug 11 - IBM Data Management Magazine - IIUG user view: Happy 10th anniversary to IBM and Informix... Read
8 Jul 11 - Database Trends and Applications - Managing Time Series Data with Informix... Read
31 May 11 - Smart Grid - The meter data management pitfall utilities are overlooking... Read
27 May 11 - IBM Data Management Magazine - IIUG user view: Big data, big time ( Series data, warehouse acceleration, and 4GLs )... Read
16 May 11 - Business Wire - HiT Software Announces DBMoto for Enterprise Integration, Adds Informix. Log-based Change Data Capture... Read
21 Mar 11 - Yahoo! Finance - IBM and Cable&Wireless Worldwide Announce UK Smart Energy Cloud... Read
14 Mar 11 - MarketWatch - Fuzzy Logix and IBM Unveil In-Database Analytics for IBM Informix... Read
11 Mar 11 - InvestorPlace - It's Time to Give IBM Props: How many tech stocks are up 53% since the dot-com boom?... Read
9 Mar 11 - DBTA - Database Administration and the Goal of Diminishing Downtime... Read
2 Feb 11 - DBTAs - Informix 11.7 Flexible Grid Provides a Different Way of Looking at Database Servers... Read
27 Jan 11 - exactsolutions - Exact to Add Informix Support to Database Replay, SQL Monitoring Solutions... Read
25 Jan 11 - PR Newswire - Bank of China in the UK Works With IBM to Become a Smarter, Greener Bank... Read
12 Oct 10 - Database Trends and Applications - Informix 11.7: The Beginning of the Next Decade of IBM Informix... Read
20 Sep 10 - planetIDS.com - ITG analyst paper: Cost/Benefit case for IBM Informix as compared to Microsoft SQL Server... Read
20 Jul 10 - IBM Announcements - IBM Informix Choice Edition V11.50 helps deploy low-cost scalable and reliable solutions for Apple Macintosh and Microsoft Windows... Read
20 Jul 10 - IBM Announcements - Software withdrawal: Elite Support for Informix Ultimate-C Edition... Read
24 May 10 - eWeek Europe - IBM Supplies Database Tech For EU Smart Grid... Read
23 May 10 - SiliconIndia - IBM's smart metering system allows wise use of energy... Read
21 May 10 - CNET - IBM to help people monitor energy use... Read
20 May 10 - ebiz - IBM Teams With Hildebrand To Bring Smart Metering To Homes Across Britain... Read
19 May 10 - The New Blog Times - Misurare il consumo energetico: DEHEMS è pronto... Read
19 May 10 - ZDNet - IBM software in your home? Pact enables five-city smart meter pilot in Europe... Read
17 March 10 - ZDNet (blog) David Morgenstern - TCO: New research finds Macs in the enterprise easier, cheaper to manage than... Read
17 March 2010 - Virtualization Review - ...key components of Big Blue's platform to the commercial cloud such as its WebSphere suite of application ser vers and its DB2 and Informix databases... Read
10 February 2010 - The Wall Street Journal - International Business Machines is expanding an initiative to win over students and professors on its products. How do they lure the college crowd?... Read


End of Support Dates

IIUG on Facebook IIUG on Twitter

[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

IDS Forum

Re: New RFE entry needs votes

Posted By: Madison Pruet
Date: Thursday, 11 January 2018, at 6:31 p.m.

In Response To: Re: New RFE entry needs votes (Fernando Nunes)

Does the problem exist with buffered logging or only with unbuffered logging? If only with unbuffered logging, then that implies that the risk of the problem occurring increases as the efficiency of the log flush decreases. By efficiency, I mean the number of log records per log flush being higher. Buffered logging does increase the efficiency as is evident from the average number of pages per flush IO from onstat -l. Generally speaking unbuffered logging has far fewer pages per flush IO than buffered logging. Not only will that decrease the efficiency of the log flush, but it also increases the number of IO per minute.
So in order to improve performance on the active transactional system, it would seem that the key would be to make the log flushing more efficient by increasing the log records per flush and decreasing the required number of IO. Mind you, Im not suggesting telling the customer to switch to buffered logging - just suggesting that some of the characteristics of buffered logging address some of the issue.
I would also not suggest trying to implement exactly what buddy bunching commits that other DBMS have implemented. The archecture of IDS allows for very lite weight cooperation amongst threads that other DBMS dont readily have.

Now for the HDR issue.
Like the log flush, HDR has a single point bottleneck. With the log flush it is the amount of time that it takes to perform the physical IO and with HDR it is amount of time that it takes to send the HDR buffer across the network. By increasing the number of HDR transmit buffers, you not only increased the number of buffers on the primary, but also the number of buffers on the secondary. We did a lot of work in later versions of the server to improve the performance on the secondary so that we were not waiting so long for an IO completion to apply a log record. I dont know if the customer had that version of the server, but having additional buffers on the secondary would decrease the probability of a bottleneck that we had seen in receiving the log buffers. Since I didnt work the case, I dont know if the problem that the customer saw was a result of backflow caused by not being able to apply the log records or if it was a problem caused by the network transmission being too sluggish. Both cases would have resulted in the primary being unable to find an empty buffer.
Just before I retired, we had addressed a problem with network transmission with RSS. This allowed us to have parallel transmission streams between primary and RSS secondary. This resulted in a significant improvement in synchronization after a long disconnect - on the order of 6-10 times faster. Just before I retired, I was thinking through the possibility of switch HDR so that it no longer used the half-duplex model that it used and instead using SMX so that we could move data to the secondary much quicker. But instead I focused on parallel shard queries.
But then again, I no longer have acces to the code and cant experiment with a solution. So instead, Im currently boondocking in the Arizona desert... ;-)

Sent from Yahoo Mail for iPad

On Thursday, January 11, 2018, 1:37 AM, Fernando Nunes <domusonline@gmail.com> wrote:

Trying to argue with two "monsters" (rest assured this is a compliment) is
really a bold move. Please take this as food for thought, and not exactly
as an educated opinion:

Although the message misses some "newline" changes, and I didn't get the
direct reply from Madison, I believe what he's suggesting is that the
suggestion I made on the other RFE (group commit) would be a better
solution.
I was totally convinced about that, but a recent discussion with a person
that was somehow involved with DB2 development (and mentioned that group
commit caused issues...) raised some doubts.

Apart from this, and considering Madison's points are very relevant (and
didn't came to my mind innitially) I wonder if just increasing the number
of buffers wouldn't help.
Taking is example of flushing buffer "5" and how that would require
previous flushes of buffers 1-4, we should not forget that it's the current
situation we have with buffer "2" and "1", with the (big?) difference that
currently we get into this as soon as we trigger a flush on buffer 3. So we
only have one buffer before we get into the situation. With the considered
scenario we would allow applications to "fill/flush" buffers 6-10 before
getting into the issue. I wonder if that wouldn't help....
I do recall an ancient ATLAS case where from the description, a patch was
created for an European customer to add more "HDR" buffers. And apparently
that helped. It's not the same situation, but I wonder if the effect
wouldn't be the same.
But I guess that the folks from HCL reading this may consider that.... as
long as there is a stress test, it shouldn't be too difficult to try...

As to John's statement that "if it was easy, it would have been done
before"... That causes me some "issues". I have a list of things that sound
"easy" and were never implemented. I have two explanations for this:

1- My concept of "easy" doesn't always match reality. So good feedback like
the one provided by Madison helps us, ignorant folks to understand why
sometimes things are not that simple. And that feedback should be provided
through the RFE system/process

2- Many times there's a huge gap between customer requirements and what's
important for them, and the development priorities. This happens for a few
reasons: The user's voice is not loud enough (and that's mainly their
fault, but there's also a communication channel issue...). Development most
of the times is concerned with what big customers want, and with what may
gain us a new customer. I'm not criticizing this, but it has its
disadvantages.

3- Many times, people that deal with big customers (mainly in the USA) have
a privilege access to the development teams and managers, and they
sometimes manage to use some "workarounds" that are "good enough".
This is an obscure point, and a very personal "complaint" but let me
illustrate with an example: What's the use for the common user of the
"onstat -g pqs"? However for internal people the information may be very
relevant.

Regards.

On Thu, Jan 11, 2018 at 7:31 AM, John Miller <jmiller@jfmiii.com> wrote:

> =0AI would agree with my esteemed learned colleague. Increasing the=0Anumb=
> er of log buffers will not help performance at all. If it would
> have=0Athis=
> would be an easy change which would have been done long ago. There=0Aare m=
> any =0Aissues in dealing with the log buffer and Madison bring up one, but=
> =0Athere =0Aare several more.=0A=0A=0AJohn=0A=0A-------- Original Message
> -=
> -------=0ASubject: Re: New RFE entry needs votes [40481]=0AFrom: "Madison
> P=
> ruet" <madison_pruet@yahoo.com>=0ADate: Wed, January 10, 2018 10:09
> pm=0ATo=
> : ids@iiug.org=0A=0AIve been doing some thinking about this and would
> like =
> to make some =0Asuggestions. You can take it for what its worth. =0AIts
> bes=
> t to make a request for change to describe a problem and not so=0Amuch a =
> =0Asolution. I dont think that just having more log buffers will
> resolve=0A=
> any =0Aproblem because the order that any buffer can be reused is
> dependent=
> on=0Athe =0Apreceding buffers being reusable. To explain - suppose that yo=
> u had 10=0Abuffers =0Aand commit log records had been placed in all 10.
> Sin=
> ce the commit log=0Arecord =0Awould have started the log flush of all 10
> lo=
> g buffers, then there would=0Abe no =0Afree log buffers available for any
> s=
> ubsequent log records. This would be=0A=0Aidentical to the situation that
> w=
> e have today. =0AAlso lets suppose that log buffer 5 completed the log
> flus=
> h. Fine, but=0Athe =0Athreads which were attached to log buffer 5 could
> sti=
> ll not complete the=0A=0Acommit until log buffers 1-4 were complete. Why?
> B=
> ecause the=0Atransactions =0Aissueing the commit in log buffer 5 could
> have=
> log records in buffers=0A1-4. If =0Athe system were to be restarted before=
> all of the log buffers 1-5 were=0Afully =0Aflushed, then you would lose th=
> e transaction having the commit log=0Arecord in =0Abuffer 5 and that would
> =
> break the ACID rules of transactions. =0AAlso, if there are a large number
> =
> of log buffers and there is no control=0Aof =0Athe IO execution, then all
> t=
> hat youve done is to increase the IO=0Acontention on =0Athe log files, and
> =
> that would decrease performance. =0ASo the solution is probably not so
> much=
> just increasing the number of=0Alog =0Abuffers, but rather increasing the =
> number of commits per log buffer=0Awhile =0Astill maintaining good
> performa=
> nce and functioning similar to unbuffered=0A=0Alogging. Perhaps not
> allowin=
> g the start of IO on buffer 2 until buffer 1=0Ais =0Acompletely flushed
> AND=
> allowing additional log records to be placed into=0A=0Abuffer 2 until the =
> IO completion of log buffer 1. Yes - those=0Atransactions =0Athat placed a
> =
> commit log record into buffer 2 would have to be suspended=0Auntil =0Athe
> I=
> O completion of the IO on buffer 2 AND the IO would not be=0Ainitiated
> =0Au=
> ntil the completion of the IO on log buffer 1. =0AI dont know the
> solution.=
> I no longer have access to the source code to =0Aexperiment on a good solu=
> tion to the problem. But again, Im not sure=0Athat the =0Aproblem is
> stated=
> in the RFE either. =0A=0ASent from Yahoo Mail for iPad =0A=0AOn Sunday, Ja=
> nuary 7, 2018, 1:00 PM, Art Kagel <art.kagel@gmail.com>=0Awrote:
> =0A=0AFOlk=
> s: =0A=0AI submitted the following RFE to the IBM RFE Community site. If
> yo=
> u have=0Aan =0Ainterest in this feature/enhancement, please add your vote:
> =
> =0A=0A------------------------------------------------------------
> =0A-----=
> --------------------------------------------------------- =0ANotification
> g=
> enerated at: 07 Jan 2018, 02:57 PM Eastern Time (ET) =0A=0AID: 114905
> =0AHe=
> adline: Need the ability to increase =0Athe number of logical log buffers
> f=
> rom 3 to handle peak transaction=0Aloads =0ASubmitted on: 07 Jan 2018,
> 02:5=
> 7 PM Eastern Time =0A(ET) =0ABrand: Analytics Platform =0AProduct:
> Informix=
> Servers =0AComponent: Product functionality =0A=0ALink: http://www.ibm.com
> =
> / =0Adeveloperworks/rfe/execute?use_case=3DviewRfe&CR_ID=3D114905
> =0A------=
> ------------------------------------------------------
> =0A-----------------=
> --------------------------------------------- =0A=0AArt =0A=0AArt S.
> Kagel,=
> President and Principal Consultant =0AASK Database Management =0Awww.askdb=
> mgt.com =0A=0ABlog: http://informix-myview.blogspot.com/
> =0A=0ADisclaimer: =
> Please keep in mind that my own opinions are my own opinions=0A=0Aand do
> no=
> t reflect on the IIUG, nor any other organization with which I=0Aam
> =0Aasso=
> ciated either explicitly, implicitly, or by inference. Neither do =0Athose
> =
> opinions reflect those of other individuals affiliated with any =0Aentity
> w=
> ith which I am affiliated nor those of the entities themselves.
> =0A=0A=0A**=
> ************************************************************
> ***************=
> **=0A=0AForum Note: Use "Reply" to post a response in the discussion
> forum.=
> =0A=0A=0A***************************************************
> **************=
> **************=0A=0A Forum Note: Use "Reply" to post a response in the
> disc=
> ussion forum.
>
>
> ************************************************************
> *******************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>

--
Fernando Nunes
Portugal

http://informix-technology.blogspot.com
My email works... but I don't check it frequently...

*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.

Messages In This Thread

[ View Thread ] [ Post Response ] [ Return to Index ] [ Read Prev Msg ] [ Read Next Msg ]

IDS Forum is maintained by Administrator with WebBBS 5.12.