[npinney]

Hopefully someone will be able to help us here as we are very new to setting up VIO servers and partitions so are wondering if we have done something very wrong.

We have built 3 x AIX 5.3 partitions on a VIO server but the performance is dreadful.

Even a simple copy of a directory makes the disks go 100% busy and takes minutes to complete instead of seconds.

I am not sure what is causing this so was looking for some help.

I am also not sure what info to provide here to help people help me!

Any suggestions please or what info can I provide that may help?

Thanks in advance.

[ross.mather]

How did you configure the VIO Server?

Did you use 1 Virtual I/O server or 2?
What type of disks are allocated to the AIX servers? Local SCIS or SAN disks
Did you use the logical volume manager on the Virtual I/O server?
What CPU and Memory allocation have you yused for the VIO?

[npinney]

Hi Ross

Thanks for the reply.

Just one question before I collate all the info for you...

The disks are local SCSI connected.

We are wondering if a PP size of 32 is too small for rootvg - this was the default that it set up?

Just an idea that we came across this morning when we found something on another site - can't send you the link here I am afraid but it said ...

Hi,

I kinda don't understand the problem -- yes, DS4300 will outperform
internal disks in 510/520 and the rest. I've seen it many times.
Internal disks with bs less than 64K aren't making anyone happy. IBM
is aware about it but wouldn't do sh* about it. WHY? I don't know.
Especially it surprising since same disks are used in SUN boxes and
they outperform disks in IBM p5 systems 2-to-3 times. My guess is IBM
wants everyone to use DS storages and not internal disks ( well,
other than for rootvg maybe )


Do you have any thoughts on that ?

[ross.mather]

Personally the only local disks I use now are the rootvgs for the Virtual I/O servers - and that is only so that I can do some SAN debugging if LPARS won't start.

For everything else I want SAN storage.

For systems with High throughput I allocate the rootvg via the VIO and then the remainder of the disks via Fibre Cards in the LPAR - again for SAn debugging.

If all the data is on the SAN you save overheads on Logical Volume Management of the mirroring, and 72 Gb for the rootvg is overkill.

[npinney]

Ross

Unfortunately we are stuck with the local disks - we have 4 of them on the VIO server. We do have a NAS and all the filesystems are NFS mounted but we do also make use of a /Disk1 filesystem that will be on local disks.

The cp command that I was doing was from /tmp

Do you still need all the config or do you think we have another problem here that is maybe not something you have seen before seeing as you use (rightly so) a SAN solution?

[ross.mather]

What I want isn't always what I get - but if you drop the info I'll see if I can see anything thats causing the bottleneck.

I do quite often use just local disks for small test environments, so I do have experience there.

What may help you out yourself is to use the topas command on the VIo server to see how the disks are behaving.

[npinney]

We have one VIO server. It has 2 x 73Gb (that are only used locally and mirriored) and 2 x 146Gb disks. These are the backing devices for the adapters.

There are 3 partitions created from this setup.

Did you use the logical volume manager on the Virtual I/O server? - Sorry not quite sure what you mean by that? We used commands such as:

mklv -lv rootvg_dbsrv rootvg_clients 10G

Is that what you mean?

What CPU and Memory allocation have you used for the VIO?

It has the following:

The 3 figures are for Minimum, Desired and Maximum

Memory - 128Mb 512Mb 128Mb
Processor units - 0.2 0.5 0.8
Virtual Processors - 1 2 2

It is uncapped at a weight of 255.

We have run topas and nmon and all looks ok in terms of cpu and memory but the disks show as being up to 100% busy once we do a copy (for example).

I will attach some output files from iostat and vmstat if that helps?

Thanks in advance

[ross.mather]

Ok commands such as mklv are front ends to the Logical Volume manager on the Virtual I/O server. Effectively on the 148Gb disks you have a Logical Volume Manager running on topof a disk with the Logical Volume Manager.

This means that doing a copy on one server will influence the copy on the other.

The only way you can really help this is to allocate one 146Gb disk to one server and the other one to the second LPAR. If you allocate the whole disk that way - not using the logical volume commands - and then create all the file systems on the LPAR it will improve the performance some what.

If you are using disks this way for rootvgs its usually OK as there isn't much writing to the rootvg so you don't often get the problem. Unfortunately you do and I don't have a solution.

[npinney]

Hi Ross

We can't allocate the disks in that way I am afraid cos these are not LPAR's in a true sense.

Anyway .....

I got a call back from IBM and this performance is "normal" apparently.

We were comparing the timings with a Solaris server where a script ran in 30 seconds yet on the AIX server it took 2.5 hours.

We copied some data on another AIX 5.3 server and the performance was just as bad (good???).

So it looks like there is nothing wrong with our setup as such just that the servers are this much slower that the Solaris ones we have.

IBM said they would try and find some tuning parameters we could tweak to improve the performance but not sure what as yet.

Will let you know when I hear more.

Thanks for your help with this anyway.

[ross.mather]

Do remember that you have the overhead of 2 x a Logical Volume Manager in this setup.

I'm curious if your Solaris setup has the same overhead?

Also - how are your filesystems configured? jfs2 with inline logging? Or do you have a log logical volume?