Autoconfig Error : ORA-01422: exact fetch returns more than requested number of rows

Issue: Autoconfig on EBS 12.2 environment fails while running script txkGenADOPWrapper.pl

Below is the error message  from logfile

=======================

Script Name    : txkGenADOPWrapper.pl

Script Version : 120.0.12020000.3

ERROR DESCRIPTION:

(*******FATAL ERROR*******

PROGRAM : (/test/app/fs1/inst/apps/test_mtnode1/admin/install/txkGenADOPWrapper.pl)

TIME    : Wed Nov 25 04:22:34 2020

FUNCTION: TXK::SQLPLUS::_doExecute [ Level 3 ]

MESSAGES:

SQLPLUS error: buffer=

 

SQL*Plus: Release 10.1.0.5.0 - Production on Wed Nov 25 04:22:33 2020

 

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

 

SQL> SQL> Connected.

SQL> SELECT ad_zd_adop.get_node_type('mtnode1') FROM DUAL

       *

ERROR at line 1:

ORA-01422: exact fetch returns more than requested number of rows

ORA-06512: at "APPS.AD_ZD_ADOP", line 3010

=======================

Cause:

FND_NODES table has incorrect node information.

Found an extra row in FND_NODES table where node_name is null after clone.

Solution:

1. Clean up FND_NODES table

Login to database as apps user

sqlplus apps/<password>

    

sql>exec FND_CONC_CLONE.SETUP_CLEAN;

2.Run Autoconfig on Database node(s).

cd $ORACLE_HOME/appsutil/bin

adconfig.sh

[Provide Database Context file  and apps Password]

3. Run Autoconfig on apps tier

Login to Application node 

Source the RUN filesystem environment file.

cd $AD_TOP/bin

adconfig.sh

[Provide Applictions Context file and apps Password]

What is Split Brain Syndrome in Oracle RAC?

Split brain syndrome occurs when the Oracle RAC nodes are unable to communicate with each other via private interconnect, but the communication between client and RAC node is maintained. This can cause data Integrity issues  when the same block is read or updated by two nodes and changes done from one node are overwritten by the other node because the block being changed is not locked.

When a node fails, the failed node is prevented from accessing all the shared disk devices and groups. This methodology is called I/O Fencing, Disk Fencing or Failure Fencing.

The node which first detects that one of the node is not accessible will evict that node from the RAC cluster group.This problem is solved by configuring the heartbeat connections through the same communication channels that are used to access the clients.