RAC由于归档表空间满而无法启动实例的解决

今天想测试点东西，登录测试库；发现实例是关闭的；

sql> startup
oracle instance started.

Total System Global Area 2722467840 bytes
Fixed Size                  2231472 bytes
Variable Size            1476395856 bytes
Database Buffers         1241513984 bytes
Redo Buffers                2326528 bytes
Database mounted.
ORA-03113: end-of-file on communication channel
Process ID: 23281
Session ID: 1 Serial number: 5


查看日志

SUCCESS: diskgroup ORAARCH was mounted
ARCH: Error 19504 Creating arcHive log file to '+ORAARCH'
Errors in file /u01/app/oracle/diag/rdbms/xhdb/xhdb1/trace/xhdb1_ora_23565.trc:
ORA-16038: log 5 sequence# 345 cannot be archived
ORA-19504: failed to create file ""
ORA-00312: online log 5 thread 1: '+ORADATA/xhdb/onlinelog/group_5.272.857422319                    '
USER (ospid: 23565): terminating the instance due to error 16038
System state dump requested by (instance=1, osid=23565), summary=[abnORMal insta                    nce termination].
System State dumped to trace file /u01/app/oracle/diag/rdbms/xhdb/xhdb1/trace/xh                    db1_diag_23451.trc
Dumping diagnostic data in directory=[cdmp_20141218102056], requested by (instan                    ce=1, osid=23565), summary=[abnormal instance termination].
Instance terminated by USER, pid = 23565


查看(fdisk)查看磁盘都在；切换到grid集群也在运行，asm磁盘组都在；可能是归档空间不够了；把节点1的数据库实例启动到挂载状态下；



SQL> startup mount;
ORACLE instance started.

Total System Global Area 2722467840 bytes
Fixed Size                  2231472 bytes
Variable Size            1476395856 bytes
Database Buffers         1241513984 bytes
Redo Buffers                2326528 bytes
Database mounted.
SQL> archive log list;
Database log mode              Archive Mode
Automatic archival             Enabled
Archive destination            +ORAARCH
Oldest online log sequence     345
Next log sequence to archive   345


SQL> select group_number,name,state,total_mb,free_mb from v$asm_diskgroup;

GROUP_NUMBER NAME                           STATE         TOTAL_MB    FREE_MB
------------ ------------------------------ ----------- ---------- ----------
           1 CRS                            MOUNTED            998        600
           2 ORAARCH                        MOUNTED           8189         11
           3 ORADATA                        CONNECTED        20473      16198



发现是oraarch 归档空间不够了,节点1登录rman,把归档日志删除；在节点1和节点2启动数据库实例，正常；

RMAN> crosscheck archivelog all;

RMAN> delete archivelog all; 


SQL> select group_number,name,state,total_mb,free_mb from v$asm_diskgroup;

GROUP_NUMBER NAME                           STATE         TOTAL_MB    FREE_MB
------------ ------------------------------ ----------- ---------- ----------
           1 CRS                            MOUNTED            998        600
           2 ORAARCH                        CONNECTED         8189       5123
           3 ORADATA                        CONNECTED        20473      16198

SQL> alter database open;

Database altered.





节点2 也能正常打开
[oracle@rac2 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Thu Dec 18 10:56:54 2014

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area 2722467840 bytes
Fixed Size                  2231472 bytes
Variable Size            1191183184 bytes
Database Buffers         1526726656 bytes
Redo Buffers                2326528 bytes
Database mounted.
Database opened.