HMC and FSP

Flexible Service Processor Basics and Connecting a Pseries system to HMC

Flexible Service Processor is the component which resides as part of the system board or as a card in your P-series system which provides diagnostics, initialization, configuration, run-time error detection and correction.
It contains the firmware which boots as sooon as AC Power is given and handles the startup of LPARs communication to HMC etc in a partitioned environment.

HMC

The HMC is a dedicated desktop PC workstation that provides several functions for configuring and operating pSeries servers functioning either partitioned or in
the Full System Partition, using the graphical user interface1(GUI) or command line interface2 (CLI). The functions provided by HMC include:

Logical partitioning management

The HMC provides a set of tasks that are necessary to manage logical
partitions. These tasks include:

– Starting, stopping, resetting, and shutting down a partition.

– Opening a virtual console for each partition or connected pSeries server
system.

– Creating partition profiles that define the processor, memory, and I/O
resources allocated to an individual partition.

Connecting FSP (Or the managed system) to HMC

When the FSP is in it's factory default configuration, it's Ethernet ports are configured as DHCP clients. When AC power applied to the system unit, the
FSP will automatically begin it's boot process. Assuming that a network link on either of the FSP's Ethernet ports is in an active
state, The FSP will issue a DHCP request on the active port(s).

Note that active means that the link light is illuminated which indicates that the FSP is communicating with another device on the
network (but NOT necessarily an HMC) The DHCP request will be issued during the first 60 second period after AC power is
applied however the completion of the boot process to FSP standby will take 3 to 5 min. The DHCP request process will be
repeated every time one of the following actions is performed:

• AC power is cycled.
• The FSP is reset (rebooted) via the Operator Panel pinhole reset method
• The ASMI Reset Service Processor menu option is utilized

Notes:
1) A common misconception is that if a Managed System has had AC power applied to it before the HMC is configured
that extensive recovery steps (flipping FSP Dip switches, removing time of day battery etc) must be performed to allow
the FSP to communicate correctly. This is NOT the case!
2) In addition to the 3 actions listed above, numerous other ASMI options exist that will cause the FSP to reboot when
certain configuration changes have been made.
3) There is one other condition under which the FSP will generate a DHCP request. If the FSP Ethernet link is not
active (no cable connected) during the FSP boot process and then an active connection is made after the FSP is
booted to standby, a DHCP request will now be generated. Keep in mind that this operation will occur as soon as the
link goes active, regardless of what device is on the other end of the link.
Example: If an Ethernet switch is in the path between the FSP and the HMC, the connection between the switch and
the HMC needs to be established before the connection between the FSP and the switch otherwise, the DHCP request
will not be seen by the HMC.
4) Future Firmware releases beginning with the 01SF225 release will also allow a DHCP request to be generated
anytime the network link is dropped for approximately 60 seconds and then re-established.
Now we’ve established how and when a DHCP request is generated. What happens next depends on whether the FSP receives
a response to it's request:


Scenario #1: Response to DHCP request is NOT received

The FSP will assign the following default IP addresses to it's ports:
HMC1 - 192.168.2.147
HMC2 - 192.168.3.147
The assignment of these default addresses allows the user to configure and attach a network capable device such as a PC or
Laptop, connect it directly to one of the FSP network ports and access the FSP's ASMI interface. Note that these addresses are
NOT permanent, (static) but are just defaults which the FSP continues to use until the next DHCP request cycle is performed and
a response is received.


Scenario #2: Response to DHCP request IS received

In this case, the FSP simply accepts the IP address it was administered by the DHCP server however this new IP address now
replaces the default IP address the FSP would have assigned itself above in scenario #1. The new IP address now becomes the
new default IP address for that port however just like in scenario #1, the DHCP request process will take place again on the next
power or FSP reset cycle. If a DHCP response containing a different IP address is received, the FSP will accept it AND replace
the previous default IP address with the new one. A few examples of how this can happen are listed below:
• System (FSP) is attached to a different HMC (using the same FSP port that was previously configured)
• If the HMC's disk drive is reloaded or replaced, the address may change, particularly if multiple systems are attached
and are discovered in a different order.
• An additional DHCP server such as another HMC configured as a DHCP server appears on the network
As long as no configuration changes are made, the HMC should always assign the same IP address to a respective Ethernet
port. It does this by keeping track of the network mac (hardware) addresses assigned to each Ethernet port. These mac
addresses are unique for every Ethernet device.

Connecting a Managed System to an HMC

The information contained in this section assumes that DHCP as been chosen as the desired method of assigning IP addresses
and that the HMC itself has been configured as the DHCP server. Prior to discussing troubleshooting options, we will describe
the expected behavior.
The HMC should be configured as a DHCP server prior to connection of the system unit. For this example, we'll assume that the
HMC is set up properly, the Managed System has been attached and AC power has been applied. As discussed in the last
section, the DHCP address request and assignment process takes place within the first 60 seconds of the FSP's boot process.
The HMC should now automatically detect the Managed System.
On the HMC, Click Server and Partitions > Server management to view the status of your managed system. An object
representing the Managed System with an IP address listed next to it should be displayed within a few minutes. If the Managed
System has never been configured on this OR another HMC, the state of the system will be Pending
Authentication. This indicates the that the HMC's access password (to the FSP) has not yet been set.
When this password is set, it is written to the FSP and also stored in a file on the HMC. This password
establishes secure connection between the HMC and the FSP.
At this point, the HMC access password must now be set as well as the ASMI admin and general user's
passwords. This can be done by selecting the object with the left mouse buttons and then right clicking on
the object and choosing the "Update Managed System Password" option from the menu.


Actual passwords chosen are up to you. If not specified the common passwords are as below :

HMC Access: abc123
Admin user: admin
General user: general


If the state of the system is Failed Authentication, it indicates that the FSP already has a stored HMC
access password which is not known by the HMC. This condition can occur if the Managed System was previously connected to
a different HMC or it was previously connected to the current HMC and it's connection was then logically removed using the
Reset or Remove connection option. When the connection is removed the HMC deletes it’s copy of the password.
If the password is known, it can be input now. If the password is NOT known, a couple of options are available. These options
are covered in the Common Symptoms and Solutions section which appears later in this document.
Once the passwords have been entered, the system will go to a Power Off state and the IP address in
the icon will be replaced with the machine type and serial number of the system (or a customized setting
for systems that were previously installed)


What if no Managed System object appears?
The first step is to establish whether or not the FSP is requesting an address and if so, is the HMC
responding?
The easiest way to do this is to open a rshterm on the HMC and run the command "lshmc -n". The output
from the command will look something like the following:


Example 1:
hscroot@sqhmc:~> lshmc -n
hostname=sqhmc,domain=austin.ibm.com,"ipaddr=10.0.0.1,9.3.253.230","networkmask
=255.255.255.0,255.255.255.192",gateway=9.3.253.193,"nameserver=9.0.7.1,9.0.6.11
",domainsuffix=,ipaddrlpar=9.3.253.230,networkmasklpar=255.255.255.192,"clients=
10.0.0.254"

Note the last field in the above example: "clients=10.0.0.254" This field of information contains all IP
addresses that have been assigned by the HMC's DHCP server. If multiple systems are attached,
multiple addresses will appear in this field.

Example 2:
hscroot@sqhmc:~> lshmc -n
hostname=sqhmc,domain=austin.ibm.com,"ipaddr=10.0.0.1,9.3.253.230","networkmask
=255.255.255.0,255.255.255.192",gateway=9.3.253.193,"nameserver=9.0.7.1,9.0.6.11
",domainsuffix=,ipaddrlpar=9.3.253.230,networkmasklpar=255.255.255.192,"clients=
10.0.0.251,10.0.0.254,10.0.0.254"


In the example above you can see that there are 2 instances of the same IP address (10.0.0.254) Why?
When the HMC DHCP server assigns an IP address, it creates a DHCP lease. Each instance of an IP
address represents a DHCP lease. All unexpired leases will be displayed in by the lshmc -n command. If
the DHCP server receives additional DHCP requests from the same MAC address, it will create a new
lease for that MAC address despite the fact that an unexpired lease currently exists. This process will be
repeated every time the FSP requests an address. Once the 2 hour lease period expires on any inactive
leases, they will be automatically deleted and will not appear in the lshmc -n command output. Also note
that address 10.0.0.251 appears in the above example. In this case, our HMC is controlling 2 Managed
Systems.
Armed with this information, we can now establish whether or not an FSP and the HMC have established
basic communication.
For example: A Managed System (FSP) is cabled up to an HMC. An object for this managed system does
not appear. Now what? Run the lshmc -n command and check to see if any IP addresses appear in the
"clients" field.
Assuming that the answer to this question is yes, then how do we know if the IP address belongs to the
system we are currently trying to configure or, in the event that multiple systems are attached, how do you make out which IP address belongs to which system?

Make a note of how many IP addresses appear and also how many instances of each IP exists, then
force the FSP to request an address again using one of the methods described in Section 1 above. If the
FSP and the HMC are communicating properly, then another IP address or an additional instance of one
of the previously existing IP addresses should appear. Compare the example below to example 2 above
and note that address 10.0.0.254 now appears a third time as a result of an FSP being reset. If you reset
the FSP again, a 4th instance will appear and so on.


Example 3:
hscroot@sqhmc:~> lshmc -n
hostname=sqhmc,domain=austin.ibm.com,"ipaddr=10.0.0.1,9.3.253.230","networkmask
=255.255.255.0,255.255.255.192",gateway=9.3.253.193,"nameserver=9.0.7.1,9.0.6.11
",domainsuffix=,ipaddrlpar=9.3.253.230,networkmasklpar=255.255.255.192,"clients=
10.0.0.251,10.0.0.254,10.0.0.254,10.0.0.254"
On V4R3 and later HMC code levels, the command lssysconn –r all provides an additional method for
viewing which systems are connected. This command will show all systems and frames managed by the
HMC including those that are not DHCP clients.


Example 4:
hscroot@sqhmc:~> lssysconn -r all
resource_type=sys,type_model_serial_num=9117-570*109C4DD,sp=primary,ipaddr=10.0.0.251
,state=Connected
resource_type=sys,type_model_serial_num=9117-570*100079A,sp=primary,ipaddr=10.0.0.248
,state=Connected


The next example shows the same command run on an HMC with 9119 -595 attached. Note that 4 IP addresses appear, 1 for each of the FSP’s and 1 for each of the Bulk Power Controllers.

Example 5:
hscroot@sqhmc2:~> lssysconn -r all
resource_type=sys,type_model_serial_num=9119-595*02ACC3C,sp=secondary,ipaddr=192
.168.254.255,state=Connected
resource_type=sys,type_model_serial_num=9119-595*026ACC3C,sp=primary,ipaddr=192.1
68.255.252,state=Connected
resource_type=frame,type_model_serial_num=9458-100*9112345,side=a,ipaddr=192.168
.255.253,state=Connected
resource_type=frame,type_model_serial_num=9458-100*9112345,side=b,ipaddr=192.168
.254.254,state=Connected


The above exercise established that the FSP has requested an IP and the HMC has issued one. The next
step is to open a rshterm and perform a ping test to the IP address you obtained in the previous step.
Assuming that the ping test was successful, we have now established that a good communication path
exists between the FSP and the HMC so the problem is most likely NOT with the FSP. Continue with
"Manually Adding a Managed System" below.
What if "clients= " (blank)? This indicates that effective 2 way communications between the FSP and the
HMC have NOT occurred! There are numerous potential causes for this condition including but not limited
to:

• The HMC is not correctly configured as a DHCP server.(The DHCP server is not running or
another HMC problem exists)
• A networking problem exists in the path between the FSP and the HMC:
o Is the Ethernet cable connected to the correct port on the HMC?
o Is there a malfunctioning or misconfigured Switch or Hub in the network path?
o Are all the link lights illuminated?
• The FSP's Ethernet port has been configured with a static IP address To verify, connect to
ASMI and view the Network Configuration settings. (Unless the system has been previously
installed in another environment, this is unlikely)


Manually Adding a Managed System

Now that we know the address of the Managed System which we are trying to configure and verified that
we can ping it, we can use the Add Managed System option to manually add the system. For Systems
configured as DHCP clients we normally we would not have to perform this step however there are few
known HMC problems that will cause the object not to appear automatically. A list of known problems that
produce this behavior are listed in the "Known HMC Connectivity Issues" section.
There is one common cause for this behavior that is by design:

Example: A Managed System which has had it's IP address dynamically assigned by the HMC. It's connection is then removed
using the Reset or Remove connection option. When this occurs, the HMC stores the removed IP address in an "exclusion
file. This prevents the HMC from polling the removed IP address despite the presence of a valid DHCP lease. To resolve this
problem, the Managed system can simply be added manually. On V4R3 and later HMC levels, the command "mksysconn -o
auto" will remove the contents of the exclusion file which will allow the Managed System object to appear automatically.
In the event that the customer does not wish to utilize DHCP to assign IP addresses, the desired
addresses can be assigned manually via the ASMI menus. Once the IP addresses are known, the
follow below steps to add the managed system.

1) Select the "add managed system" option from the Server Management menu:
2) Input the address of the Managed System you are trying to add and click "next". The password is the
HMC’s access password for the FSP. It can be left blank as this time.
3) On the next panel, click "Finish"

----------------------------------------------------------------------------------------------------------------------------------------------------------------