For those of you who aren’t nerds ignore this. I wanted to put up a note that could be found via a web search on an issue we had here at work.One that even the IBM GPFS support guys were scratching their heads over.

When using dm multipath on linux and then mounting gpfs filesystems you MUST make a modification to the /etc/mmfs/etc/nsddevices file. Either modify the existing file or copy the sample file from /usr/lpp/mmfs/samples/nsddevices.sample to /etc/mmfs/etc/nsddevices.

The modification that must be made is to change the linux section of the file to


if [[ $osName = Linux ]]
then
$awk ‘                                                          \
/dm-/   { if (NF > 3 && $3 > 1) print $4 ” dm-mpio” ; next }  \
‘ /proc/partitions 2>$STDERR
fi

and uncomment the return part of the following section


# To bypass the GPFS disk discovery (/usr/lpp/mmfs/bin/mmdevdiscover),
return 0

The reason that you have to do this is that the dm multipath functionality of linux creates a device /dev/dm-# and gpfs will not automagically look for this. It looks for sd* and hd* devices only. (well emcpower and vpath also to be picky). That and it uses the wrong device type of “generic” instead of “dm-mpio” Without both of these, finding the right device, and using the write driver type, gpfs will not work correctly.

For those interested gpfs uses the script mmdevdiscover to find usable devices with which to build NSD’s on. Here is the subsection of the script that does this functionality for Linux


elif [[ $osName = Linux ]]
then
# Get the names of all disks, vpaths, etc.
$awk ‘                                                                 \
/emcpower/   { if (NF > 3 && $3 > 1) print $4 ” powerdisk” ; next }  \
/vpath/      { if (NF > 3 && $3 > 1) print $4 ” vpath”     ; next }  \
/[sh]d/      { if (NF > 3 && $3 > 1) print $4 ” generic”   ; next }  \
‘ /proc/partitions 2>$STDERR

Using the /var/mmfs/etc/nsddevices file modifies the way gpfs will discover the necessary devices and will thus find the /dev/dm-# devices. I imagine you could also just modify the /usr/lpp/mmfs/bin/mmdevdiscover file directly, but if you do this BEWARE any gpfs upgrades my overwrite the file without you knowing it until your file systems go away. It’s much safer to just modifiy the nsddevices file and call it good.

Enough said. Good luck and may the force be with you. (-:

Advertisements