[PaulClayton]

Hi,

We got a server running AIX 5.3, that rebooted the other day, and now keeps on reporting hdisk errors althouth the errors are of type temp.
The disk are configured in a vpath, and I noted that some phantom disks have appeared in the vpath config showing no physical size.
Looking for some recommendations, as I believe these phantom drives may be causing the disk errors. Below some output.

---------------------------------------------------------------------------
LABEL: SC_DISK_ERR4
IDENTIFIER: DCB47997
Date/Time: Wed Dec 12 13:09:28 USAST 2007
Sequence Number: 1883368
Machine Id: 00502A6A4C00
Node Id: metpol9
Class: H
Type: TEMP
Resource Name: hdisk9
Resource Class: disk
Resource Type: 1750
Location: U0.1-P2-I3/Q1-W500507630E8203F0-L4011402900000000
VPD:
Manufacturer................IBM
Machine Type and Model......1750500
Serial Number...............13399501129
EC Level.....................954
Device Specific.(Z0)........10
Device Specific.(Z1)........0101
Device Specific.(Z2)........013
Device Specific.(Z3)........27005
Device Specific.(Z4)........08
Device Specific.(Z5)........00
Description
DISK OPERATION ERROR
Probable Causes
MEDIA
DASD DEVICE
User Causes
MEDIA DEFECTIVE
Recommended Actions
FOR REMOVABLE MEDIA, CHANGE MEDIA AND RETRY
PERFORM PROBLEM DETERMINATION PROCEDURES
Failure Causes
MEDIA
DISK DRIVE
Recommended Actions
FOR REMOVABLE MEDIA, CHANGE MEDIA AND RETRY
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
0
SENSE DATA
0A00 2A00 01C0 18F0 0000 0804 0000 0000 0000 0000 0000 0000 0200 0200 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 1283 0000 7740 0000 0001 0000 0000 0000 0000 0000 0000 0000
0000 0034 001D
---------------------------------------------------------------------------
LABEL: SC_DISK_ERR4
IDENTIFIER: DCB47997
Date/Time: Wed Dec 12 11:21:13 USAST 2007
Sequence Number: 1883367
Machine Id: 00502A6A4C00
Node Id: metpol9
Class: H
Type: TEMP
Resource Name: hdisk9
Resource Class: disk
Resource Type: 1750
Location: U0.1-P2-I3/Q1-W500507630E8203F0-L4011402900000000
VPD:
Manufacturer................IBM
Machine Type and Model......1750500
Serial Number...............13399501129
EC Level.....................954
Device Specific.(Z0)........10
Device Specific.(Z1)........0101
Device Specific.(Z2)........013
Device Specific.(Z3)........27005
Device Specific.(Z4)........08
Device Specific.(Z5)........00
Description
DISK OPERATION ERROR
Probable Causes
MEDIA
DASD DEVICE
User Causes
MEDIA DEFECTIVE
Recommended Actions
FOR REMOVABLE MEDIA, CHANGE MEDIA AND RETRY
PERFORM PROBLEM DETERMINATION PROCEDURES
Failure Causes
MEDIA
DISK DRIVE
Recommended Actions
FOR REMOVABLE MEDIA, CHANGE MEDIA AND RETRY
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
0
SENSE DATA
0A00 2A00 0268 5C38 0000 0804 0000 0000 0000 0000 0000 0000 0200 0200 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 127C 0008 E700 0000 0001 0000 0000 0000 0000 0000 0000 0000
0000 0034 001D


--------------------------------------------------------------------------

lsvpcfg
vpath0 (Avail pv dsvg01) 13399501028 = hdisk6 (Avail ) hdisk8 (Avail ) hdisk10 (Avail ) hdisk12 (Avail )
vpath1 (Avail pv dsvg02) 13399501129 = hdisk7 (Avail ) hdisk9 (Avail ) hdisk11 (Avail ) hdisk13 (Avail )
vpath2 (Avail pv dsvg02) 13399501022 = hdisk14 (Avail ) hdisk16 (Avail ) hdisk18 (Avail ) hdisk20 (Avail )
vpath3 (Avail pv dsvg01) 13399501023 = hdisk15 (Avail ) hdisk17 (Avail ) hdisk19 (Avail ) hdisk21 (Avail )

-------------------------------------------------------------------------
datapath query adapter
Active Adapters :2
Adpt# Name State Mode Select Errors Paths Active
0 fscsi0 NORMAL ACTIVE 6233073 0 8 8
1 fscsi1 NORMAL ACTIVE 291731344 0 8 8

--------------------------------------------------------------------------
datapath query device
Total Devices : 4

DEV#: 0 DEVICE NAME: vpath0 TYPE: 1750500 POLICY: Optimized
SERIAL: 13399501028
================================================== ========================
Path# Adapter/Hard Disk State Mode Select Errors
0 fscsi0/hdisk6 OPEN NORMAL 0 0
1 fscsi0/hdisk8 OPEN NORMAL 0 0
2 fscsi1/hdisk10 OPEN NORMAL 47841707 0
3 fscsi1/hdisk12 OPEN NORMAL 47849190 0
DEV#: 1 DEVICE NAME: vpath1 TYPE: 1750500 POLICY: Optimized
SERIAL: 13399501129
================================================== ========================
Path# Adapter/Hard Disk State Mode Select Errors
0 fscsi0/hdisk7 OPEN NORMAL 3117495 0
1 fscsi0/hdisk9 OPEN NORMAL 3115666 0
2 fscsi1/hdisk11 OPEN NORMAL 0 0
3 fscsi1/hdisk13 OPEN NORMAL 0 0
DEV#: 2 DEVICE NAME: vpath2 TYPE: 1750500 POLICY: Optimized
SERIAL: 13399501022
================================================== ========================
Path# Adapter/Hard Disk State Mode Select Errors
0 fscsi0/hdisk14 OPEN NORMAL 0 0
1 fscsi0/hdisk16 OPEN NORMAL 0 0
2 fscsi1/hdisk18 OPEN NORMAL 90927468 0
3 fscsi1/hdisk20 OPEN NORMAL 90924399 0
DEV#: 3 DEVICE NAME: vpath3 TYPE: 1750500 POLICY: Optimized
SERIAL: 13399501023
================================================== ========================
Path# Adapter/Hard Disk State Mode Select Errors
0 fscsi0/hdisk15 OPEN NORMAL 0 0
1 fscsi0/hdisk17 OPEN NORMAL 0 0
2 fscsi1/hdisk19 OPEN NORMAL 7095055 0
3 fscsi1/hdisk21 OPEN NORMAL 7094537 0
--------------------------------------------------------------------------

[Frans]

Follow the serial back to the diskstarage device an make sure you have not remove the device or deleted the mapping

[duke900ssd]

I think the 0200 0200 is a transport fault - out in SAN land.

Why did it reboot, what errors were posted at that time?

[PaulClayton]

The reboot was finger trouble, but it did high light the issue. I confirmed with our storage guys today, and the server is connected via fabric switch.
The server is in a zone with a P570 and the storage. So the zone config looks like this

DS6800 -A0
DS6800 -B3
P570
P650

I would like to see the the 570 and 650 seperated, as this may be a cause.