IDS Forum

Re: HDR and ER Running Together

Posted By: Keith Simmons
Date: Wednesday, 20 June 2012, at 4:43 a.m.

In Response To: Re: HDR and ER Running Together (Madison Pruet)

Madison

Thanks for this explanation, it makes logical sense and ties in with
what I see occasionally. However I got the following errors a couple
of days ago:
Primary:
13:40:25 DR: Cannot connect to secondary server
13:40:36 DR: Primary server connected
13:40:36 DR: Send error
13:40:36 ASF Echo-Thread Server: asfcode = -25580: oserr = 32: errstr =
: System error occurred in network function.
System error = 32.
13:40:36 DR: Failure recovery error (2)
13:40:37 DR: Turned off on primary server
13:40:37 DR: Cannot connect to secondary server
13:40:48 DR: Primary server connected
13:40:49 DR: Secondary server needs failure recovery

Secondary:
13:40:35 DR: Received connection request from remote server when DR is
not Off

[Local type: Secondary, Current state: ?]

[Remote type: Primary]
13:40:42 DR: ping timeout
13:40:42 DR: Receive error
13:40:42 ASF Echo-Thread Server: asfcode = -25582: oserr = 4: errstr =
: Network connection is broken.
System error = 4.
13:40:44 DR: Turned off on secondary server
13:40:47 DR: Secondary server connected
13:40:49 DR: Secondary server needs failure recovery
13:40:49 DR: Failure recovery from disk in progress ...
13:41:24 DR: Received connection request from remote server when DR is
not Off

[Local type: Secondary, Current state: ?]

[Remote type: Primary]
13:41:36 DR: Received connection request from remote server when DR is
not Off

[Local type: Secondary, Current state: ?]

HDR failed to reconnect for some 40 minutes during which time ER failed to
send transactions. HDR only restarted when the secondary engine was
restarted.
Thoughts would be appreciated, but please tell me if this is better as a PMR
rather than discussion here.

Keith

On 19 June 2012 17:10, Madison Pruet <mpruet@us.ibm.com> wrote:
> Let's get the perspective.
>
> 1) If ER has a txn which it needs to send, but can't because the secon=
> dary
> has not receive the log buffer containing the commit, it will force the=
> HDR
> transmit buffers to be immediately transmitted. Generally this means a=
> bout
> a 1/4 sec delay in the ER transmission.
>
> 2) If ER is not able to send the HDR transmit buffer and/or the ACK of=
> the
> transmission is not received within a couple of seconds, then you get t=
> he
> message.
>
> 3) If the HDR ping timeout kicks in and we mark HDR as down, then ER
> doesn't wait because the HDR secondary is no longer a viable failover
> server.
>
> So the max delay is based on the HDR ping timeout.
>
> There is no distinction between sync and async HDR mode since ER will p=
> ush
> out the HDR buffer as soon as it is blocking on that buffer.
>
> M.P.
>
> From: "Art Kagel" <art.kagel@gmail.com>
> To: ids@iiug.org,
> Date: 06/19/2012 08:22 AM
> Subject: Re: HDR and ER Running Together [27392]
> Sent by: ids-bounces@iiug.org
>
> Gotcha. It was a thought. Always heartening when you find someone using=
>
> the right tool for the job and disheartening when it doesn't work perfe=
> ctly
>
> anyway. <sigh>
>
> Madison: Your response does not indicate whether ER would wait for the
> secondary in the case where the secondary is running in asynchronous mo=
> de.
> Since the primary only waits for the secondary to ack the log buffer
> packets and not for the local commit, would the ER send also not wait f=
> or
> the secondary to commit?
>
> Art
>
> Art S. Kagel
> Advanced DataTools (www.advancedatatools.com)
> Blog: http://informix-myview.blogspot.com/
>
> Disclaimer: Please keep in mind that my own opinions are my own opinion=
> s
> and do not reflect on my employer, Advanced DataTools, the IIUG, nor an=
> y
> other organization with which I am associated either explicitly,
> implicitly, or by inference. Neither do those opinions reflect those of=
>
> other individuals affiliated with any entity with which I am affiliated=
> nor
>
> those of the entities themselves.
>
> On Tue, Jun 19, 2012 at 10:02 AM, Keith Simmons <smiley73@gmail.com> wr=
> ote:
>
>> Art
>>
>> Thanks, but not an option, those ER targets are subsets of the tables=
>
>> with specific indexing to support that application and are very diffe=
> rent
>
>> from the OLTP application on the HDR pair. Also the ER targets are on=
>
>> Solaris with the HDR on AIX.
>>
>> Keith
>>
>> On 19 June 2012 14:53, Art Kagel <art.kagel@gmail.com> wrote:
>> > An alternative would be to use an RSS secondary for the
>> > monitoring/feedback/reporting server. That would also allow you to
>> upgrade
>> > the RSS secondary to an HDR secondary if one of the HDR servers goe=
> s
>> down.
>> >
>> > Art
>> >
>> > Art S. Kagel
>> > Advanced DataTools (www.advancedatatools.com)
>> > Blog: http://informix-myview.blogspot.com/
>> >
>> > Disclaimer: Please keep in mind that my own opinions are my own
> opinions
>> > and do not reflect on my employer, Advanced DataTools, the IIUG, no=
> r
> any
>> > other organization with which I am associated either explicitly,
>> > implicitly, or by inference. Neither do those opinions reflect thos=
> e of
>
>> > other individuals affiliated with any entity with which I am affili=
> ated
>
>> nor
>> > those of the entities themselves.
>> >
>> > On Tue, Jun 19, 2012 at 9:42 AM, Keith Simmons <smiley73@gmail.com>=
>
>> wrote:
>> >
>> >> Thanks Madison,
>> >>
>> >> So am I correct in thinking that while HDR is stopped for any reas=
> on
>> >> ER will also pause, regardless of whether HDR is in synchronous or=
>
>> >> asynchronous mode ?
>> >>
>> >> I'm trying to get my head around the implications of this.
>> >>
>> >> We use the target(s) of the ER as part of our monitoring and feed-=
>
>> >> back system and also as a read-only access for some customer
>> >> facing web-pages to allow them to track orders in 'almost' real ti=
> me.
>> >>
>> >> It would seem that if either the HDR Secondary or HDR Primary fail=
>
>> >> for any reason and require an extended portion of down-time (but
>> >> not long enough to cycle the logs and require a complete re-
>> >> instantiation of HDR) then we will not be able to maintain these
>> >> targets in step. The alternative I think, therefore, is, during an=
> y
>> failure
>> >> likely to take more than 3-4 hours to resolve, to switch the remai=
> ning
>
>> >> HDR server to stand- alone and then restart HDR using an archive/
>> >> restore (not ideal for a 1Tb instance on each end of a 100 mile WA=
> N
>> >> link).
>> >>
>> >> Please can you confirm this reasoning and conclusion.
>> >>
>> >> Keith
>> >>
>> >> On 19 June 2012 13:38, Madison Pruet <mpruet@us.ibm.com> wrote:
>> >> > If a replicated transaction is replicated via ER prior to ER kno=
> wing
>
>> th=3D
>> >> > at
>> >> > it has made it to the HDR secondary, then there would be
>> inconsistency =3D
>> >> > in
>> >> > the event of a failover. So we don't allow ER to replicate the
>> transac=3D
>> >> > tion
>> >> > until we know that the secondary has the transaction.
>> >> >
>> >> > From: "Keith Simmons" <smiley73@gmail.com>
>> >> > To: ids@iiug.org,
>> >> > Date: 06/19/2012 02:58 AM
>> >> > Subject: HDR and ER Running Together [27383]
>> >> > Sent by: ids-bounces@iiug.org
>> >> >
>> >> > Lets try again, this may be easier to read !!
>> >> >
>> >> > Set up is as follows:
>> >> >
>> >> > IDS 9.4 on AIX 5.3 --> ER over LAN --> IDS 10 on Solaris 2.8
>> >> > | |->
>> >> > IDS 11 on Solaris 2.10
>> >> > V
>> >> > Sync HDR over WAN
>> >> > |
>> >> > V
>> >> > IDS 9.4 on AIX 5.3
>> >> >
>> >> > Note ER is Primary to Secondary (one way).
>> >> >
>> >> > With this setup I am seeing lines in the error log of the HDR/ER=
>
>> >> > Primary as follows:
>> >> > ER can not send replicated transactions due to HDR hang
>> >> > ER resume transmission of ER/HDR replicated transactions
>> >> > Also a few days ago the HDR secondary lost contact with the prim=
> ary
>> >> > and couldn't automatically re-establish (brief network glitch). =
> I
>> stopp=3D
>> >> > ed
>> >> > and restarted the secondary and this HDR re-established, but whi=
> le
>> >> > HDR was not running (and until the logs had shipped and applied =
> up
>> >> > to date) ER was frozen.
>> >> > I am trying to understand why ER keeps stopping (albeit briefly)=
> and
>
>> >> > will not run while HDR is not connected. My suspicion is that it=
> is
>> rel=3D
>> >> > ated
>> >> >
>> >> > to the fact I have HDR in Synchronous mode but would like this
>> >> > confirmed. Also, if this is the case, is it possible to switch
> between
>> >> > Synchronous and Asynchronous modes without restarting instances =
> ?
>> >> >
>> >> > Many Thanks in anticipation
>> >> >
>> >> > Keith
>> >> >

Messages In This Thread

• HDR and ER Running Together
  Keith Simmons -- Tuesday, 19 June 2012, at 4:58 a.m.
  • Re: HDR and ER Running Together
    Madison Pruet -- Tuesday, 19 June 2012, at 8:38 a.m.
    • Re: HDR and ER Running Together
      Keith Simmons -- Tuesday, 19 June 2012, at 9:42 a.m.
      • Re: HDR and ER Running Together
        Art Kagel -- Tuesday, 19 June 2012, at 9:53 a.m.
        • Re: HDR and ER Running Together
          Keith Simmons -- Tuesday, 19 June 2012, at 10:02 a.m.
          • Re: HDR and ER Running Together
            Art Kagel -- Tuesday, 19 June 2012, at 10:19 a.m.
            • Re: HDR and ER Running Together
              Madison Pruet -- Tuesday, 19 June 2012, at 12:10 p.m.
              • Re: HDR and ER Running Together
                Keith Simmons -- Wednesday, 20 June 2012, at 4:43 a.m.
                • Re: HDR and ER Running Together
                  JACQUES RENAUT -- Wednesday, 20 June 2012, at 8:58 a.m.
                  • Re: HDR and ER Running Together
                    Keith Simmons -- Monday, 27 August 2012, at 8:50 a.m.
                    • Re: HDR and ER Running Together
                      JACQUES RENAUT -- Monday, 27 August 2012, at 11:01 a.m.
      • Re: HDR and ER Running Together
        Madison Pruet -- Tuesday, 19 June 2012, at 11:46 a.m.

IDS Forum is maintained by Administrator with WebBBS 5.12.