我们的文章会在微信公众号IT民工的龙马人生和博客网站( www.htz.pw )同步更新 ,欢迎关注收藏,也欢迎大家转载,但是请在文章开始地方标注文章出处,谢谢!
由于博客中有大量代码,通过页面浏览效果更佳。
Oracle案例:SOLARIS RAC平台模拟节点crash后强制删除与增加
本次测试来至于跟朋友一次聊天,关于10G RAC中crash节点的删除与重新增加,已经N久没有做过10G RAC的操作,并且原来的操作记录也没有找到,悲剧的曾经的笔记全掉了。
在这次测试过程中,遇到一个原来重要没有遇到的过的问题。
欢迎大家加入ORACLE超级群:17115662 免费解决各种ORACLE问题
本次是通过手动删除节点1来模拟节点1crash后,在节点2上清除节点1的信息。官方文档见:
Steps to Remove Node from Cluster When the Node Crashes Due to OS/Hardware Failure and cannot boot up (文档 ID 466975.1)
1,当前集群资源的状态
www.htz.pw $ crs_stat -tName Type Target State Host ————————————————————ora….SM1.asm application ONLINE ONLINE sol1 ora….L1.lsnr application ONLINE ONLINE sol1 ora.sol1.gsd application ONLINE ONLINE sol1 ora.sol1.ons application ONLINE ONLINE sol1 ora.sol1.vip application ONLINE ONLINE sol1 ora.sol10g.db application ONLINE ONLINE sol2 ora….g1.inst application ONLINE ONLINE sol1 ora….g2.inst application ONLINE ONLINE sol2 ora….SM2.asm application ONLINE ONLINE sol2 ora….L2.lsnr application ONLINE ONLINE sol2 ora.sol2.gsd application ONLINE ONLINE sol2 ora.sol2.ons application ONLINE ONLINE sol2 ora.sol2.vip application ONLINE ONLINE sol2
2,删除节点1中数据库与crs的信息
www.htz.pw # /oracle/app/oracle/product/10.2.0/db_1/bin/crsctl stop crsStopping resources. This could take several minutes.Successfully stopped CRS resources.Stopping CSSD.Shutting down CSS daemon.Shutdown request successfully issued.www.htz.pw $ crs_stat -tName Type Target State Host ————————————————————ora….SM1.asm application ONLINE OFFLINE ora….L1.lsnr application ONLINE OFFLINE ora.sol1.gsd application ONLINE OFFLINE ora.sol1.ons application ONLINE OFFLINE ora.sol1.vip application ONLINE ONLINE sol2 ora.sol10g.db application ONLINE ONLINE sol2 ora….g1.inst application ONLINE OFFLINE ora….g2.inst application ONLINE ONLINE sol2 ora….SM2.asm application ONLINE ONLINE sol2 ora….L2.lsnr application ONLINE ONLINE sol2 ora.sol2.gsd application ONLINE ONLINE sol2 ora.sol2.ons application ONLINE ONLINE sol2 ora.sol2.vip application ONLINE ONLINE sol2删除节点1相当的信息www.htz.pw # rm /etc/init.d/init.cssdrm /etc/init.d/init.crswww.htz.pw # rm /etc/init.d/init.crswww.htz.pw # rm /etc/init.d/init.crsdwww.htz.pw # rm /etc/init.d/init.evmdwww.htz.pw # rm /etc/rc3.d/K96init.crs/etc/rc3.d/K96init.crs: No such file or directorywww.htz.pw # rm /etc/rc3.d/S96init.crswww.htz.pw # rm -Rf /var/opt/oracle/scls_scrwww.htz.pw # rm -Rf /var/opt/oracle/oprocdwww.htz.pw # rm /etc/inittab.crswww.htz.pw # cp /etc/inittab.orig /etc/inittabwww.htz.pw # rm -rf /var/tmp/.oracle/*www.htz.pw #www.htz.pw # rm -rf /tmp/.oracle/*www.htz.pw # rm -rf /oracle/app
3,节点2,CRS中削除节点1的信息
www.htz.pw # pwd/oracle/app/oracle/product/10.2.0/crs_1/binwww.htz.pw # ./oifcfg getife1000g0 192.168.111.0 global publice1000g1 192.168.112.0 global cluster_interconnect在oifcfg中清除节点1的信息www.htz.pw # ./oifcfg delif -node sol1PROC-4: The cluster registry key to be operated on does not exist.PRIF-11: cluster registry error在ons中清除节点1的信息www.htz.pw # cat $CRS_HOME/opmn/conf/ons.configlocalport=6100remoteport=6200loglevel=3useocr=onwww.htz.pw # $CRS_HOME/bin/racgons remove_config sol1:6200racgons: Existing key value on sol1 = 6200.racgons: sol1:6200 removed from OCR.www.htz.pw # $CRS_HOME/bin/crs_stat -tName Type Target State Host ————————————————————ora….SM1.asm application ONLINE OFFLINE ora….L1.lsnr application ONLINE OFFLINE ora.sol1.gsd application ONLINE OFFLINE ora.sol1.ons application ONLINE OFFLINE ora.sol1.vip application ONLINE ONLINE sol2 ora.sol10g.db application ONLINE ONLINE sol2 ora….g1.inst application ONLINE OFFLINE ora….g2.inst application ONLINE ONLINE sol2 ora….SM2.asm application ONLINE ONLINE sol2 ora….L2.lsnr application ONLINE ONLINE sol2 ora.sol2.gsd application ONLINE ONLINE sol2 ora.sol2.ons application ONLINE ONLINE sol2 ora.sol2.vip application ONLINE ONLINE sol2在CRS中清除节点1的信息www.htz.pw # $CRS_HOME/bin/srvctl remove instance -d sol10g -i sol10g1Remove instance sol10g1 from the database sol10g? (y/[n]) ywww.htz.pw # $CRS_HOME/bin/srvctl remove asm -n sol1www.htz.pw # $CRS_HOME/bin/srvctl remove nodeapps -n sol1Please confirm that you intend to remove the node-level applications on node sol1 (y/[n]) yPRKO-2108 : Node applications are still running on node: sol1www.htz.pw # $CRS_HOME/bin/srvctl remove nodeapps -n sol1Please confirm that you intend to remove the node-level applications on node sol1 (y/[n]) yPRKO-2108 : Node applications are still running on node: sol1# $CRS_HOME/bin/crs_stat -tName Type Target State Host ————————————————————ora….L1.lsnr application ONLINE OFFLINE ora.sol1.gsd application ONLINE OFFLINE ora.sol1.ons application ONLINE OFFLINE ora.sol1.vip application ONLINE ONLINE sol2 ora.sol10g.db application ONLINE ONLINE sol2 ora….g2.inst application ONLINE ONLINE sol2 ora….SM2.asm application ONLINE ONLINE sol2 ora….L2.lsnr application ONLINE ONLINE sol2 ora.sol2.gsd application ONLINE ONLINE sol2 ora.sol2.ons application ONLINE ONLINE sol2 ora.sol2.vip application ONLINE ONLINE sol2 www.htz.pw # $CRS_HOME/bin/crs_stop -f ora.sol1.vipTarget set to OFFLINE for `ora.sol1.LISTENER_SOL1.lsnr`Attempting to stop `ora.sol1.vip` on member `sol2`Stop of `ora.sol1.vip` on member `sol2` succeeded.www.htz.pw # $CRS_HOME/bin/srvctl remove nodeapps -n sol1Please confirm that you intend to remove the node-level applications on node sol1 (y/[n]) yPRKO-2112 : Some or all node applications are not removed successfully on node: sol1# $CRS_HOME/bin/crs_stat -tName Type Target State Host ————————————————————ora….L1.lsnr application OFFLINE OFFLINE ora.sol1.vip application OFFLINE OFFLINE ora.sol10g.db application ONLINE ONLINE sol2 ora….g2.inst application ONLINE ONLINE sol2 ora….SM2.asm application ONLINE ONLINE sol2 ora….L2.lsnr application ONLINE ONLINE sol2 ora.sol2.gsd application ONLINE ONLINE sol2 ora.sol2.ons application ONLINE ONLINE sol2 ora.sol2.vip application ONLINE ONLINE sol2www.htz.pw # $CRS_HOME/bin/crs_stat|grep lsnNAME=ora.sol1.LISTENER_SOL1.lsnrNAME=ora.sol2.LISTENER_SOL2.lsnrwww.htz.pw # $CRS_HOME/bin/crs_unregister ora.sol1.LISTENER_SOL1.lsnrwww.htz.pw # $CRS_HOME/bin/olsnodes -nsol1 1sol2 2删除节点1的信息www.htz.pw # ./rootdeletenode.sh sol1,1CRS-0210: Could not find resource ‘ora.sol1.LISTENER_SOL1.lsnr’.CRS-0210: Could not find resource ‘ora.sol1.ons’.CRS-0210: Could not find resource ‘ora.sol1.vip’.CRS-0210: Could not find resource ‘ora.sol1.gsd’.CRS-0210: Could not find resource ora.sol1.vip.CRS nodeapps are deleted successfullyclscfg: EXISTING configuration version 3 detected.clscfg: version 3 is 10G Release 2.Successfully deleted 14 values from OCR.Key SYSTEM.css.interfaces.nodesol1 marked for deletion is not there. Ignoring.Successfully deleted 5 keys from OCR.Node deletion operation successful.‘sol1,1’ deleted successfullywww.htz.pw # $CRS_HOME/bin/olsnodes -nsol2 2更新inventory文件www.htz.pw $ $ORA_CRS_HOME/oui/bin/runInstaller -updateNodeList ORACLE_HOME=$ORA_CRS_HOME “CLUSTER_NODES=sol2” CRS=TRUEStarting Oracle Universal Installer…No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.The inventory pointer is located at /var/opt/oracle/oraInst.locThe inventory is located at /oracle/app/oracle/oraInventory‘UpdateNodeList’ was successful.www.htz.pw $ $ORACLE_HOME//oui/bin/runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME “CLUSTER_NODES=sol2”Starting Oracle Universal Installer…No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.The inventory pointer is located at /var/opt/oracle/oraInst.locThe inventory is located at /oracle/app/oracle/oraInventory‘UpdateNodeList’ was successful.更新后的值www.htz.pw $ cat inventory.xml<?xml version=”1.0″ standalone=”yes” ?><!– Copyright (c) 2008 Oracle Corporation. All rights Reserved –><!– Do not modify the contents of this file by hand. –><INVENTORY><VERSION_INFO><SAVED_WITH>10.2.0.4.0</SAVED_WITH><MINIMUM_VER>2.1.0.6.0</MINIMUM_VER></VERSION_INFO><HOME_LIST><HOME NAME=”OraCrs10g_home” LOC=”/oracle/app/oracle/product/10.2.0/crs_1″ TYPE=”O” IDX=”1″ CRS=”true”><NODE_LIST><NODE NAME=”sol2″/></NODE_LIST></HOME><HOME NAME=”OraDb10g_home1″ LOC=”/oracle/app/oracle/product/10.2.0/db_1″ TYPE=”O” IDX=”2″><NODE_LIST><NODE NAME=”sol2″/></NODE_LIST></HOME></HOME_LIST></INVENTORY>
4,增加节点
www.htz.pw $ pwd/oracle/app/oracle/product/10.2.0/crs_1/oui/binwww.htz.pw $ lsaddLangs.sh addNode.sh attachHome.sh detachHome.sh lsnodes ouica.bat ouica.sh resource runConfig.sh runInstaller runInstaller.shwww.htz.pw $ ./addNode.shStarting Oracle Universal Installer…No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.Oracle Universal Installer, Version 10.2.0.4.0 ProductionCopyright (C) 1999, 2008, Oracle. All rights reserved这里点next就报错,从报错信息中可以找到下面内容,oui的默认日志路径见log文件位置INFO: Username:oracleINFO: Install area Control created with access level 1INFO: Oracle Universal Installer version is 10.2.0.4.0INFO: Setting variable ‘ORACLE_HOME’ to ‘/oracle/app/oracle/product/10.2.0/crs_1’. Received the value from the command line.INFO: Setting variable ‘PREREQ_CONFIG_LOCATION’ to ”. Received the value from variable association.INFO: Setting variable ‘FROM_LOCATION’ to ‘/oracle/app/oracle/product/10.2.0/crs_1/inventory/ContentsXML/comps.xml’. Received the value from a code block.INFO: Setting variable ‘ROOTSH_LOCATION’ to ‘/oracle/app/oracle/product/10.2.0/crs_1/root.sh’. Received the value from a code block.INFO: Setting variable ‘ROOTSH_STATUS’ to ‘3’. Received the value from a code block.INFO: Setting variable ‘ORACLE_HOME’ to ‘/oracle/app/oracle/product/10.2.0/crs_1’. Received the value from the command line.INFO: Setting variable ‘PREREQ_CONFIG_LOCATION’ to ”. Received the value from variable association.INFO: Setting variable ‘FROM_LOCATION’ to ‘/oracle/app/oracle/product/10.2.0/crs_1/inventory/ContentsXML/comps.xml’. Received the value from a code block.INFO: Setting variable ‘ROOTSH_LOCATION’ to ‘/oracle/app/oracle/product/10.2.0/crs_1/root.sh’. Received the value from a code block.INFO: Setting variable ‘ROOTSH_STATUS’ to ‘3’. Received the value from a code block.INFO:*** Welcome Page***INFO: Unable to read inventory information for the home: /oracle/app/oracle/product/10.2.0/crs_1.INFO: Unable to read inventory information for the home: /oracle/app/oracle/product/10.2.0/crs_1.
下面部分是在点next之后产生的
/************************************************************************************INFO: Setting variable ‘ORACLE_HOME_NAME’ to ‘OraCrs10g_home’. Received the value from a code block.INFO: Unable to read inventory information for the home: /oracle/app/oracle/product/10.2.0/crs_1.SEVERE: Abnormal program termination. An internal error has occured. Please provide the following files to Oracle Support :“/oracle/app/oracle/oraInventory/logs/addNodeActions2014-05-08_06-05-16PM.log”“/oracle/app/oracle/oraInventory/logs/oraInstall2014-05-08_06-05-16PM.err”“/oracle/app/oracle/oraInventory/logs/oraInstall2014-05-08_06-05-16PM.out***************************************************************************************/
/oracle/app/oracle/oraInventory/logs/oraInstall2014-05-08_06-05-16PM.err中我们可以发现下面的报错信息
org.xml.sax.SAXParseException: <Line 889, Column 9>: XML-20210: (Fatal Error) Unexpected EOF.at oracle.xml.parser.v2.XMLError.flushErrorHandler(XMLError.java:415)at oracle.xml.parser.v2.XMLError.flushErrors1(XMLError.java:284)at oracle.xml.parser.v2.XMLReader.popXMLReader(XMLReader.java:540)at oracle.xml.parser.v2.NonValidatingParser.parseElement(NonValidatingParser.java:1339)at oracle.xml.parser.v2.NonValidatingParser.parseRootElement(NonValidatingParser.java:326)at oracle.xml.parser.v2.NonValidatingParser.parseDocument(NonValidatingParser.java:293)at oracle.xml.parser.v2.XMLParser.parse(XMLParser.java:209)at oracle.sysman.oii.oiii.OiiiInstallXMLReader.readComps(OiiiInstallXMLReader.java:271)at oracle.sysman.oii.oiii.OiiiInstallInventory.getCompOHListElement(OiiiInstallInventory.java:1663)at oracle.sysman.oii.oiii.OiiiAreaInventory.getAllCompsVect(OiiiAreaInventory.java:1052)at oracle.sysman.oii.oiii.OiiiAreaInventory.getTopLevelComps(OiiiAreaInventory.java:1872)at oracle.sysman.oii.oiii.OiiiInstallInventory.setOHProperties(OiiiInstallInventory.java:6064)at oracle.sysman.oii.oiif.oiifp.OiifpContentsTabPanel.addHomes(OiifpContentsTabPanel.java:777)at oracle.sysman.oii.oiif.oiifp.OiifpContentsTabPanel.fillInventoryTree(OiifpContentsTabPanel.java:691)at oracle.sysman.oii.oiif.oiifp.OiifpContentsTabPanel.refreshTree(OiifpContentsTabPanel.java:1508)at oracle.sysman.oii.oiif.oiifp.OiifpContentsTabPanel.prepareInvTree(OiifpContentsTabPanel.java:2253)at oracle.sysman.oii.oiif.oiifd.OiifdInventoryDialog.doModal(OiifdInventoryDialog.java:457)at oracle.sysman.oii.oiif.oiifw.OiifwWizDialog.onViewPrivate(OiifwWizDialog.java:863)at oracle.sysman.oii.oiif.oiifw.OiifwWizDialog.access$000(OiifwWizDialog.java:330)at oracle.sysman.oii.oiif.oiifw.OiifwWizDialog$PrepareInventoryTree.run(OiifwWizDialog.java:1778)at java.lang.Thread.run(Thread.java:534)
这里报XML结果,非法的文件结局,中途以为是inventory配置错误,使用opatch -lsinventory结果显示正常,这里怀疑是某个XML文件损坏导致的。
通过truss来查看runInstaller访问了那些xml文件
www.htz.pw $ truss -aefo /tmp/123.log ./addNode.shStarting Oracle Universal Installer…No pre-requisite checks found in oraparam.ini, no system pre-requisite checks will be executed.Oracle Universal Installer, Version 10.2.0.4.0 ProductionCopyright (C) 1999, 2008, Oracle. All rights reserved.
可以看到打开了如下的XML文件
115 1728 23646/1: open(“/oracle/app/oracle/product/10.2.0/crs_1/oui/jlib/xmlparserv2.jar”, O_RDONLY|O_LARGEFILE) = 6119 1776 23646/1: open(“/oracle/app/oracle/product/10.2.0/crs_1/oui/jlib/xml.jar”, O_RDONLY|O_LARGEFILE) = 61382 15600 23646/1: open(“/oracle/app/oracle/oraInventory/ContentsXML/inventory.xml”, O_RDONLY|O_LARGEFILE) = 151385 15915 23646/1: open(“/oracle/app/oracle/product/10.2.0/crs_1/inventory/ContentsXML/oraclehomeproperties.xml”, O_RDONLY|O_LARGEFILE) = 151387 15974 23646/1: open(“/oracle/app/oracle/product/10.2.0/db_1/inventory/ContentsXML/oraclehomeproperties.xml”, O_RDONLY|O_LARGEFILE) = 151401 18121 23646/15: open(“/oracle/app/oracle/oraInventory/ContentsXML/comps.xml”, O_RDONLY|O_LARGEFILE) = 191403 18217 23646/15: open(“/oracle/app/oracle/product/10.2.0/crs_1/inventory/ContentsXML/comps.xml”, O_RDONLY|O_LARGEFILE) = 191421 19159 23646/15: open(“/oracle/app/oracle/product/10.2.0/crs_1/inventory/ContentsXML/comps.xml”, O_RDONLY|O_LARGEFILE) = 201444 22099 23646/1: open(“/oracle/app/oracle/product/10.2.0/crs_1/inventory/ContentsXML/comps.xml”, O_RDONLY|O_LARGEFILE) = 21
最后发现只有comps.xml有889行
这里下载了一个XML编辑器,不能正常编辑COMPS.XML文件,说明文件有异常。
从其它的环境是CP一个COMPS.XML过来一对比,发现出错的XML文件下了很多的内容
正常的comps.xml文件的结构如下:
www.htz.pw $ cat comps.xml<?xml version=”1.0″ standalone=”yes” ?><!– Copyright (c) 2008 Oracle Corporation. All rights Reserved –><!– Do not modify the contents of this file by hand. –><PRD_LIST><TL_LIST></TL_LIST><COMP_LIST></COMP_LIST><ONEOFF_LIST></ONEOFF_LIST></PRD_LIST>www.htz.pw $ pwd/oracle/app/oracle/oraInventory/ContentsXML
其实这里我们也可以通过opatch来验证XML文件结构是否正确
www.htz.pw $ $ORA_CRS_HOME/OPatch/opatch util LoadXML -xmlInput /oracle/app/oracle/product/10.2.0/crs_1/inventory/ContentsXML/comps.xml.backInvoking OPatch 10.2.0.4.2Oracle Interim Patch Installer version 10.2.0.4.2Copyright (c) 2007, Oracle Corporation. All rights reserved.UTIL sessionOracle Home : /oracle/app/oracle/product/10.2.0/db_1Central Inventory : /oracle/app/oracle/oraInventoryfrom : /var/opt/oracle/oraInst.locOPatch version : 10.2.0.4.2OUI version : 10.2.0.4.0OUI location : /oracle/app/oracle/product/10.2.0/db_1/ouiLog file location : /oracle/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2014-05-08_20-39-18PM.logInvoking utility “loadxml”UtilSession failed: Unable to parse the xml file.OPatch failed with error code 73
由于这里是测试环境,所以我直接使用的mv方式,如果是生产环境,建议从其它相当环境CP一个文件过来
www.htz.pw # mv /oracle/app/oracle/product/10.2.0/crs_1/inventory/ContentsXML/comps.xml /oracle/app/oracle/product/10.2.0/crs_1/inventory/ContentsXML/comps.xml.back
再次执行addNode.sh终于见到了
clip_image001
一路next下去,一切正常,直到出现下面的图片
clip_image002
这里我们选择yes,因为OCR是由root用户执行的,日志属主是root,不影响addNode.sh操作
在执行addNode.sh操作的主机上面执行rootaddnode.sh报下面的错误
www.htz.pw # /oracle/app/oracle/product/10.2.0/crs_1/rootaddnode.shclscfg: EXISTING configuration version 3 detected.clscfg: version 3 is 10G Release 2.Attempting to add 1 new nodes to the configurationUsing ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.node <nodenumber>: <nodename> <private interconnect name> <hostname>node 3: sol1 sol1-priv sol1Creating OCR keys for user ‘root’, privgrp ‘root’..Operation successful./oracle/app/oracle/product/10.2.0/crs_1/bin/srvctl add nodeapps -n sol1 -A %s_nodevips%/255.255.255.0/e1000g0 -o /oracle/app/oracle/product/10.2.0/crs_1PRKO-2109 : Invalid address string: %s_nodevips%/255.255.255.0/e1000g0www.htz.pw # /oracle/app/oracle/product/10.2.0/crs_1/bin/srvctl config nodeapps -n sol2 -aVIP exists.: /sol2-vip/192.168.111.49/255.255.255.0/e1000g0www.htz.pw # grep “s_nodevips|CRS_NEW_NODEVIPS” /oracle/app/oracle/product/10.2.0/crs_1/rootaddnode.shCRS_NEW_NODEVIPS=%s_nodevips%www.htz.pw # grep “CRS_NEW_NODEVIPS” /oracle/app/oracle/product/10.2.0/crs_1/rootaddnode.sh CRS_NEW_NODEVIPS=%s_nodevips%NODE_VIP=`$ECHO $CRS_NEW_NODEVIPS | $CUT -d’,’ -f$Ni`
手动修改rootaddnode.sh脚本内容,
/oracle/app/oracle/product/10.2.0/crs_1/rootaddnode.shNi=1for i in `$ECHO $NODES_LIST`doNODE_NAME=$iNODE_VIP=`$ECHO $CRS_NEW_NODEVIPS | $CUT -d’,’ -f$Ni`NODEVIP=$NODE_VIP/$NETMASK/$NETIFs$ECHO $CH/bin/srvctl add nodeapps -n $NODE_NAME -A $NODEVIP -o $CH$CH/bin/srvctl add nodeapps -n $NODE_NAME -A $NODEVIP -o $CHNi=`expr $Ni + 1`done
更改后
Ni=1for i in `$ECHO $NODES_LIST`doNODE_NAME=$i#NODE_VIP=`$ECHO $CRS_NEW_NODEVIPS | $CUT -d’,’ -f$Ni`NODE_VIP=192.168.111.48NODEVIP=$NODE_VIP/$NETMASK/$NETIFs$ECHO $CH/bin/srvctl add nodeapps -n $NODE_NAME -A $NODEVIP -o $CH$CH/bin/srvctl add nodeapps -n $NODE_NAME -A $NODEVIP -o $CHNi=`expr $Ni + 1`donewww.htz.pw # /oracle/app/oracle/product/10.2.0/crs_1/rootaddnode.shclscfg: EXISTING configuration version 3 detected.clscfg: version 3 is 10G Release 2.Node sol1 is already assigned nodenum 3.Aborting: No configuration data has been changed.clscfg -add -nn nameA,numA,nameB,numB,… -pn privA,numA,privB,numB,…[-hn hostA,numA,hostB,numB,…] [-t p1,p2,p3,p4]-nn specifies nodenames in the same fashion as -nn in -install mode-pn specifies private interconnect names as -pn in -install mode-hn specifies hostnames in the same fashion as -hn in -install mode-t specifies port numbers to be used by CRS daemons on the new node(s)default ports: 49895,49896,49897,49898WARNING: Using this tool may corrupt your cluster configuration. Do notuse unless you positively know what you are doing./oracle/app/oracle/product/10.2.0/crs_1/bin/srvctl add nodeapps -n sol1 -A 192.168.111.48/255.255.255.0/e1000g0 -o /oracle/app/oracle/product/10.2.0/crs_1
下面是在1节点执行
www.htz.pw # /oracle/app/oracle/product/10.2.0/crs_1/root.shWARNING: directory ‘/oracle/app/oracle/product/10.2.0’ is not owned by rootWARNING: directory ‘/oracle/app/oracle/product’ is not owned by rootWARNING: directory ‘/oracle/app/oracle’ is not owned by rootWARNING: directory ‘/oracle/app’ is not owned by rootWARNING: directory ‘/oracle’ is not owned by rootChecking to see if Oracle CRS stack is already configuredOCR LOCATIONS = /dev/rdsk/c2t0d0s0Setting the permissions on OCR backup directorySetting up NS directoriesOracle Cluster Registry configuration upgraded successfullyWARNING: directory ‘/oracle/app/oracle/product/10.2.0’ is not owned by rootWARNING: directory ‘/oracle/app/oracle/product’ is not owned by rootWARNING: directory ‘/oracle/app/oracle’ is not owned by rootWARNING: directory ‘/oracle/app’ is not owned by rootWARNING: directory ‘/oracle’ is not owned by rootclscfg: EXISTING configuration version 3 detected.clscfg: version 3 is 10G Release 2.Successfully accumulated necessary OCR keys.Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.node <nodenumber>: <nodename> <private interconnect name> <hostname>node 1: sol1 sol1-priv sol1node 2: sol2 sol2-priv sol2clscfg: Arguments check out successfully.NO KEYS WERE WRITTEN. Supply -force parameter to override.-force is destructive and will destroy any previous clusterconfiguration.Oracle Cluster Registry for cluster has already been initializedStartup will be queued to init within 30 seconds.Adding daemons to inittabExpecting the CRS daemons to be up within 600 seconds.CSS is active on these nodes.sol2sol1CSS is active on all nodes.Waiting for the Oracle CRSD and EVMD to startOracle CRS stack installed and running under init(1M)Running vipca(silent) for configuring nodeappsCreating VIP application resource on (0) nodes.Creating GSD application resource on (0) nodes.Creating ONS application resource on (0) nodes.Starting VIP application resource on (2) nodes1:CRS-1002: Resource ‘ora.sol1.vip’ is already running on member ‘sol1’CRS-0223: Resource ‘ora.sol1.vip’ has placement error.Check the log file “/oracle/app/oracle/product/10.2.0/crs_1/log/sol1/racg/ora.sol1.vip.log” for more details…Starting GSD application resource on (2) nodes1:CRS-0233: Resource or relatives are currently involved with another operation.Check the log file “/oracle/app/oracle/product/10.2.0/crs_1/log/sol1/racg/ora.sol1.gsd.log” for more details…Starting ONS application resource on (2) nodes1:CRS-0233: Resource or relatives are currently involved with another operation.Check the log file “/oracle/app/oracle/product/10.2.0/crs_1/log/sol1/racg/ora.sol1.ons.log” for more details…Done.
这里注意报了很多错误,但是不影响。
这里看到各个节点都正常
www.htz.pw $ crs_stat -tName Type Target State Host ————————————————————ora.sol1.gsd application ONLINE ONLINE sol1 ora.sol1.ons application ONLINE ONLINE sol1 ora.sol1.vip application ONLINE ONLINE sol1 ora.sol10g.db application ONLINE ONLINE sol2 ora….g2.inst application ONLINE ONLINE sol2 ora….SM2.asm application ONLINE ONLINE sol2 ora….L2.lsnr application ONLINE ONLINE sol2 ora.sol2.gsd application ONLINE ONLINE sol2 ora.sol2.ons application ONLINE ONLINE sol2 ora.sol2.vip application ONLINE ONLINE sol2www.htz.pw # ifconfig -alo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1inet 127.0.0.1 netmask ff000000e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2inet 192.168.111.46 netmask ffffff00 broadcast 192.168.111.255ether 0:c:29:5a:e5:7ae1000g0:1: flags=1040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4> mtu 1500 index 2inet 192.168.111.48 netmask ffffff00 broadcast 192.168.111.255e1000g1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3inet 192.168.112.46 netmask ffffff00 broadcast 192.168.112.255ether 0:c:29:5a:e5:84
下面就是在oracle用户下面增加节点
www.htz.pw $ ./addNode.sh
这里很顺利,无任何报错
配置监听服务,这里可以使用手动的方式来配置,由于是测试环境,我这里在正常的节点上面通过netca来配置的。在配置过程中监听服务需要被中断。
这里中途需要停listener
www.htz.pw $ crs_stat -tName Type Target State Host ————————————————————ora….L1.lsnr application ONLINE ONLINE sol1 ora.sol1.gsd application ONLINE ONLINE sol1 ora.sol1.ons application ONLINE ONLINE sol1 ora.sol1.vip application ONLINE ONLINE sol1 ora.sol10g.db application ONLINE ONLINE sol2 ora….g2.inst application ONLINE ONLINE sol2 ora….SM2.asm application ONLINE ONLINE sol2 ora….L2.lsnr application ONLINE ONLINE sol2 ora.sol2.gsd application ONLINE ONLINE sol2 ora.sol2.ons application ONLINE ONLINE sol2 ora.sol2.vip application ONLINE ONLINE sol2在正常的节点上面执行dbca来增加实例dbca很正常,会自动增加ASM实例的信息。www.htz.pw $ crs_stat -tName Type Target State Host ————————————————————ora….SM3.asm application ONLINE ONLINE sol1 ora….L1.lsnr application ONLINE ONLINE sol1 ora.sol1.gsd application ONLINE ONLINE sol1 ora.sol1.ons application ONLINE ONLINE sol1 ora.sol1.vip application ONLINE ONLINE sol1 ora.sol10g.db application ONLINE ONLINE sol2 ora….g1.inst application ONLINE ONLINE sol1 ora….g2.inst application ONLINE ONLINE sol2 ora….SM2.asm application ONLINE ONLINE sol2 ora….L2.lsnr application ONLINE ONLINE sol2 ora.sol2.gsd application ONLINE ONLINE sol2 ora.sol2.ons application ONLINE ONLINE sol2 ora.sol2.vip application ONLINE ONLINE sol2www.htz.pw $ crs_stat|grep asmNAME=ora.sol1.ASM3.asmNAME=ora.sol2.ASM2.asm
这里注意到我们的ASM实例变成了ASM3,是由于自动增加的原因,我们可以使用增加增加ASM实例来解决问题
www.htz.pw $ srvctl stop instance -d sol10g -i sol10g1www.htz.pw $ srvctl stop asm -n sol1www.htz.pw $ crs_unregister ora.sol10g.sol10g1.instwww.htz.pw $ crs_unregister ora.sol1.ASM3.asmwww.htz.pw $ cat /oracle/app/oracle/admin/+ASM/pfile/init.ora##############################################################################www.htz.pw # Copyright (c) 1991, 2001, 2002 by Oracle Corporation#########################################################################################################################www.htz.pw # Cluster Database###########################################asm_diskgroups=’DATA’background_dump_dest=/oracle/app/oracle/admin/+ASM/bdumpcluster_database=TRUEcore_dump_dest=/oracle/app/oracle/admin/+ASM/cdumpinstance_type=asmlarge_pool_size=12582912remote_login_passwordfile=EXCLUSIVEuser_dump_dest=/oracle/app/oracle/admin/+ASM/udump+ASM2.instance_number=2+ASM1.instance_number=1www.htz.pw $ export ORACLE_SID=+ASM1www.htz.pw $ sqlplus / as sysdbaSQL*Plus: Release 10.2.0.4.0 – Production on Thu May 8 23:14:50 2014Copyright (c) 1982, 2007, Oracle. All Rights Reserved.Connected to an idle instance.SQL> create spfile from pfile;File created.www.htz.pw $ srvctl remove asm -n sol1www.htz.pw $ srvctl add asm -n sol1 -i +ASM1 -o $ORACLE_HOME -p $ORACLE_HOME/dbs/spfile+ASM1.orawww.htz.pw $ srvctl start asm -n sol1www.htz.pw $ srvctl start instance -d sol10g -i sol10g1www.htz.pw $ srvctl start instance -d sol10g -i sol10g1
修改一下依赖性
www.htz.pw $ srvctl modify instance -d sol10g -i sol10g1 -s +ASM1www.htz.pw $ srvctl stop asm -n sol1www.htz.pw $ srvctl start instance -d sol10g -i sol10g1
下切正常
www.htz.pw $ crs_stat -tName Type Target State Host ————————————————————ora….SM1.asm application ONLINE ONLINE sol1 ora….L1.lsnr application ONLINE ONLINE sol1 ora.sol1.gsd application ONLINE ONLINE sol1 ora.sol1.ons application ONLINE ONLINE sol1 ora.sol1.vip application ONLINE ONLINE sol1 ora.sol10g.db application ONLINE ONLINE sol2 ora….g1.inst application ONLINE ONLINE sol1 ora….g2.inst application ONLINE ONLINE sol2 ora….SM2.asm application ONLINE ONLINE sol2 ora….L2.lsnr application ONLINE ONLINE sol2 ora.sol2.gsd application ONLINE ONLINE sol2 ora.sol2.ons application ONLINE ONLINE sol2 ora.sol2.vip application ONLINE ONLINE sol2
整个增加过程结束,在增加过程中,遇到了一点小麻烦。
------------------作者介绍-----------------------
姓名:黄廷忠
个人博客: (http://www.htz.pw)
CSDN地址: (https://blog.csdn.net/wwwhtzpw)
博客园地址: (https://www.cnblogs.com/www-htz-pw)