[Graham]

I am new to VIO recieving a P550 with Vio Server only 2 weeks ago.
I need to have the machine live running an LPAR with a new system
in 2 weeks. I pretty much know what Im doing with AIX but am new to VIOS
It all seems strait forward enough, I Just need some reassurance im setting this all up the right way:

I have one P550 with:
a tape drive,
CD drive
2 x fibre cards (2Gigabit) for Clariion attachment
3 x Gigabit NICs.
3 x Internal disk. 73Gb
The machine has no HMC

Im considering the best way to configure it to run 3 LPARs (A live environment, A UAT/Test environment and a standby LIVE environment for another machine.)


I had an IBM consultant in to set up the machine quickly due to time constraints.

IBM consultant left the VIO server on one disk, not mirrored.
Allocated the other 2 disks to a client_rootvg Volume group, then created a storage pool over client_rootvg volume group and created two LPARs
within the storage pool. As I had asked for my rootvgs to be mirrored
in my spec consultant then seems to have used oem_setup_env to go behind the scenes to mirrorvg the client_rootvg volume group to place the
LPARs across both disks. For my SAN storage I insisted that both LPARS use both fibre cards, Consultant seemed to suggest that I install Powerpath on each LPAR but was very unsure. This didnt seem right to me. I had consultant install POwerpath and Clarion support to the VIO server.
So the powerhdisk devices appeared to the VIO server and could then be allocated to the LPARS as single disks. I saw this as a benefit because I
had never heard of storage pools and could not see the point in them. I thought and still think that the client LPAR root volume groups would be better residing entirely on the SAN. Installing Powerpath on the VIO allows the client LPAR root VG to use both fibre cards. I then had consultant migrate one of the LPARS onto a SAN disk. We then rebooted the LPAR and proved ot worked fine. Regsarding Ethernet. I accept I need to use one NIC dedicated to the VIO server. I had thought the other 2 nics could be used as truncs with each LPAR being able to specify its own type C subnet. Annoyingly as I dont have a HMC I was told this was not possible because of some bizare IBM restriction. I had to set up one NIC on our live secure network and the other on the relaxed TEST system network. Then I allocate a virtual NIC to LIVE server LPARS from one physical NIC and virtiual NICs on the TEST subnet from the other physical NIC. Not quite what I wanted but this will survice for almost any eventuallity we may have.
Regarding backups I was advised that I backup the VIO direct using mksysb. I now find not actually mksysb but "ioscli backupios -tape /dev/rmt0"
I was then told to backup the LPARs using mksysb to file then ftp them to the vio and dd them to tape!!

Im not entirely happy with what I currently have and advice Ive been given.

I plan to migrate all my LPAR rootvgs to my SAN, ditch the storage pool and mirror the vio rootvg across 2 internal disks leaving one unused internal disk as a spare. I have crontab'd a backupios weekly under the padmin vio account which was easy to do but question whether its possible or necessary to schedule creation of a Partition Configuration Backup (profile.bak) file too.
Regarding backing up the LPAR AIX operating systems, I am currently configuring an old Pseries as a NIM/Backup server, I am NFS mounting directories from the NIM server to create mksysb file backups of the LPARs. I will also use this old machine to mount snap mirror copies of the large SAN application and Oracle file systems so I can backup the data to tape without any impact on the new LPARs. I was strongy advised to buy a HMC but Im coming to conclusion that with only one vio/lpared server and a plan to have no more that 2 or 3 long term thers really no need.

Im the only person in my organisation with any AIX and Oracle experience so its a little hard keeping up with everything. I want to keep in line with the general best practices without straying to far from the norm. OK I can read the Red books but they are full of you can do this or this or this. I just want to do whats safe and normal practice.

I'm just hoping someone can give me feedback regarding what Im trying to do - Powerpath on VIO server not LPAR,
LPAR rootvg on SAN RAID5 storage,
backup mksysb to NIM server for LPARS?

It was expensive to get a consultant in for a couple of days, I had hoped for some skills transfer to become familiar with vios best practices but consultant was fresh of the course themselves and was agast with horror when realised I had no HMC.

Just some comment, what other people do and what Im doing would be most apreciated. Thanks Graham

[ross.mather]

OK a big long story - and of course I would never criticise a fellow IBMer....

Installing the Clarion device driver in the VIO is absolutely the best way to go. That is in fact the IBm recommended way - then you pass the whole of each LUN through to your LPAR and off you go. As you mentioned you want to boot from SAN, and use the internal disks only for the VIO Server. There is a command mirrorios to mirror the rootvg of the Virtual I/O server.

You can make an mksysb from the VIO server to your NIM server - but NIM will need to be AIX 5.3 if you ever want to put it back. Its always worthwhile backing up all your profiles of LPARS and also the disk mappings that you have. (a combination of lsmap -all and lspv is usually enough) That way if the whole machine dies you know which disk belongs to which LPAR from the physical volume id.

You don't normally have to dedicate an whole ethernet card to the VIO server but I guess this is becaiuse you need the IVM functionality of the VIO, so you can't break your initial connection. Be warned I've never used IVM so I could be mistaken about this one.

To be honest your gut feeling is pretty much the way I certainly do things, just remember keep your virtual slot numberings sensible, otherwise you can get confused. For Example I always make the slot id of a Virtual Ethernet Adpater the same as its VLAN id on all LPARS. like wise with pairs of Virtual SCSI Server and client slots.

If I missed anything you want answered then let me know.

[cdelgadop]

Hi

I wouldn't let VIOS install disk without mirroring. IBM disks are good, but you never know.

Well. Let me see if i can help you. When configuring VIOS based enviroments the goal is to maximize the use of all resources in the server. You can create a VIOS partition using an HMC station
or using the IVM web interface. I think the IBM specialist has used the IVM method, if so, there're some limitations but still VIOS enviroment can be done.

I would install VIOS SW on one internal disk, configure (ip address for the VIOS so the IVM web appl can be configure from a browser, hmmm, you can even configure it without ip address and link to the loopback, provide you have a graphical monitor) and install patches according to the procedure. Then create a mirror of this VIOS disk on another internal disk (either using VIOS commands or AIX commands), please reboot.

Login into VIOS and check for websphere appl server is running, open a browser to the ip address configured and use the web appl to create new partitions. It´s supposed that your 2 HBA are already zoned to the Clarrion and you can see disks in the SAN. You need to install in the VIOS partition the EMC2 driver support as well as the PowerPath software (if available) from AIX enviroment (oem_setup_env) in order to provide load balancing and fault recovery. Those powerpath disks configured are going to be the storage pools used to host the VIO clients so you need. Note that storage protection is at SAN level, ussually RAID5 disks, and throught the use of PowerPath you're protecting against HBA or Switches failures.

Create the storage pools using the web interface. Then size and create the Logical Volumes (LV) you need. Remember that these LV you've just created will be the disks used for your clients. I think that rootvg disks should be LVs of at most 36GB each and create LVs for your Oracle File Systems. Exercise some naming convention for the LVs for different VIO Clients.

Ethernet: your server has 3 Giga adapters. Hmm. If you need VIOS remote connectivity then you will have only two adapters to use. Since using IVM you cannot use the SEA (Shared Ethernet Adapter) feature (hmmm, i have my doubts about this, i think SEA can be configured from the VIOS CLI) so you only are able yo use Bridging concept which, please correct me if i'm wrong, allows for connecting one bridge to one adapter (one to one relation) so if using VLANs those have to be configured in the Ethernet Switches in access mode.

Then create the client partitions using the wizard and assign resources (CPU, RAM, Storage as Logical Volumes, and Ethernet bridges) as planned. Take a backup using the backupios command at VIOS enviroment and keep it updated (some recovery test at this level and once you hace documented all your work would probe to be very usefull). Test starting both VIOS and VIOS clients and install AIX on each client. As you can see, Powerpath should be installed in the VIOS server, not in the clients, which only see some Logical Volumes as conventional hdisk devices from the VIOS. About your NIM enviroment, nice idea.

Folliwng links can help.

IBM Wikis - Linux on POWER - IVM

Use the Integrated Virtualization Manager with Linux on POWER

Hope this helps

[Graham]

Thanks guys for your advice.
Ross I think you summed up what I wanted to hear pretty well.

cdelgadop, I think your suggesting that I should still use storage pools but create storage pools on the san disks before I allocate to the LPARs, I think this could be a good idea if in future I need to create a number of aix environments for test, ones I may well ditch in the future, just allocate 70-80Gb of SAN to a storage pool and allocate bits of it as necessary from the storage pool. Regarding systems I plan to use for production I think I'll just pass through One san disk = one vg on the LPAR

Regarding the ethernet, this is quite complicated. Apparently under HMC its all really easy and you can trunc ethernet adaptors properly.

But just with the VIOS and its web interface you are restricted to 4 subnets on a trunc where the class C part of the arress must be 1 ,2,3 or 4 ???
So I can have a trunc on 10.192.x.x with numerous virtual nics on 10.192.1.xxx thru 10.192.4.xxx but as We have already allocated 1,2,3,4 for other purposes this is no use to me.
If I have the 2 Nics set up with a class C mask on the VIO I can create numerous Virtual Nics on each. So I can set one NIC for 10.192.5.XXX and the other for 10.192.6.xxx and have as many interfaces as I like on my LPars so long as I use adresses on these 2 subnets. As I said this will surfice for my needs but it wasnt quite what I originally intended.
As Im already using the 2 NICs not allocated to the VIOS and have users developing, and migrating apps on both my LPARS I cant really start mucking about trying things out with NICs. I'll have to wait till another machine becomes spare and I can rebuild it with VIOS for that.

PS, if I manage to get a HMC, (which I am able to borrow for a few months), can I use it on a machine I started configuring/have already configured with VIOS without messing it up and making the machine then reliant on the HMC.

Eg if I in future really nead the TRunc ethernet, can I plug in a HMC sort it out and unplug the HMC leaving myself with something I can still support.

[ross.mather]

Once you've gone to an HMC solution you cannot go back to IVM - unfortunately.

I will admit - for my two cents worth - that I don't like Logical Volumes on SAN LUNs. If I want 3 x 20 Gb (which is usually big enough for yor rootvg) then I want the SAN device to allocate those. Also if you want you can move LPARS to another machine you can then simply move the LUNS and restart the LPAR. There are a small bit of rework on the Ethernet adapters and the like - but its pretty quick.

[Graham]

[quote]
if you want you can move LPARS to another machine you can then simply move the LUNS and restart the LPAR. [quote]

Interesting, Ill have a dig around on this site for anything regarding this.
This is exactly what we do under VMware with our linux environments
We can move virtual machines between VNware servers, I had been under the impression that because LPARs were less virtual than VMware environments that this was a real ball ache to do. Lots of changing MAC addresses etc and wasnt advised. Sounds like people are doing this. Ive got another machine that I should be able to use soon when I get it moved from or test suit to computer room, Its only a 2 processor P615 but should be OK
for

[ross.mather]

Oh you'll need to reconfigure your routing info once the LPAR comes back up - but thats not exactly rocket science.

The Vmoton effect for ESX servers will be available on Power6 hardware for virtual LPARS - then it will be fun.