Tuesday, March 27, 2012

ucs first time setup

Cisco UCS steps :-

First Time setup :
1)Configure the Fabric InterconnectA and floating IP
2)Configure Fabric Interconnect B ip
3)Chasis Discovery Policy(1-uplink,2-uplink,4uplink) configure your minimum uplink policy
4)Power Policy (n+1 or grid)
5)MAC Address Table Aging.(mostly deafult)
6) DNS serevr and time serevrs and syslog----------Pending
7)Maintence Polices
8)Network Polices

9)Configure Storage uplinks---------Pending
10)COnfigure Server ports
11)Configure Network Ports
12)Boot Policy
13)Syslog---------------Pending
14)Settings in Admin Tab
15)Call Home-----------------Pending
16)





Step 2 :-

Pools Creation :-
Reset Cisco UCS Blade
Create EX-Mgmt Pool
Create UUID Pool
Create MAC Pool for server profile
FAB A
FAB B
Create WWN pools
FAB A
FAB B
Create  WWPN Pools
FAB A
FAB B
Create Global Vlans

Create GLoal VSan

Assign VSAN to FC Uplink

Create vNIC Template

Create vHBA Template

Define Boot Policy

Define Polices

Create Servive Profile for single Blade

pair of Nexus 7K vPC


Uplink Connectivity :-
UPlink Config from interconnect to Core Network Switches.
1) Atleast two uplinks from interconnect to Core Network Switches
2) or Use VPC(Virtual port channel  if these are nexsus 7000 series)
3) Configure interface in trunk mode
4) Configure Sapnning tree portfast trunk(So that failover happen faster)
5)Specify Vlans  to trunk
6)CreaTE PORT CHANNELS AND ADD INTERFACES .Port channel must be lacp active mode.
7)uplinked Switches must be trunk vlans to UCS.
8)Use pin gourps for DMZ atleeast UCSM 2.0



FC SAN COnnectivity :-
NPIV should be enabled on FC switch
NPV should be enabled on FI
FC switch sees FI as bunch of WWPN and WWNN names

FI commads

UCLI COmmands :
?
show
scope
top/up
where
connect
SSH to cluster IP :
#? shows all your commands
#where (shows your peresnt location )
#show ? (will show all your operation you can use)
#scope (Zoom down to particular item and dig into  it )
#scope chassis 1 (will get you in chasis one to look into details)
#up (goes back to pervious state )
#top (goes all the way back to start of ssh ip )
#connect (Connect to NX os or frabirc to do failover and others)


UCS-A# connect local-mgmt (for performing ping and local operations on that fabric)
UCS-A(local-mgmt) # enable cluster virtual-ip-addr(enable Cluster Mode)



Changing Mgmt subnet of a Cluster :

UCS-A # scope fabric-interconnect a
        set out-of-bandip ip-address netmask  ip-netmask gw gateway-Ip-Addr
UCS-A # scope fabric-interconnect b
        set out-of-bandip ip-address netmask  ip-netmask gw gateway-Ip-Addr
UCS-A /fabric-interconnect # scope system
UCS-A /sytem #set virtual-ip ip-add
UCS-A /system #commit-buffer


Set end Host mode :-
UCS-A#scope eth-uplink
UCS-A /eth-uplink #set mode end-host
UCS-A /eth-uplink #commit-buffer

FI will reboot after this


Set system Name :-
UCS-A#scope system
UCS-A /system #set name DOmainname
UCS-A /system #commit-buffer


Move Primary to secondary for Maintence :-

UCS-A#connect local-mgmt
UCS-A(local-mgmt)#cluster lead B(name of FI to want to make as primary)
UCS-A(local-mgmt)#show cluster extendedstate

UCS-A(Local-mgmt)#Cluster force B(When you want to chnage the FI and force it)




-------------------------------------------------


Split Brain in FI :Both FI think that they are the primary FI."Partition in  space " to FI can not speak with each other.

Partition in time :time mismatch in FI who ever wirtes seeprom.

Wednesday, March 14, 2012

UCS Techsupport operations

Interconnect : reboot command will reboot the interconnect.

I/O module Reset : Hot Pull and Plug back the  I/O module.

Equipment ->Chasis -> IO Module -> IO Module 1

BMC: Baseboard Managment Controllers for out of band management on each blade and monioters the blades.

CMC : Chasis Management Controller -->:lives of FEX and performs monitoring of the chasis

I/O Adapters on blades out put can redirected to CSV

Commands to check status:

connect local-mgmt

Show tech-support ucsm brief

show tech-support ucsm detail

Dir workspace:///what ever

copy workspace://what ever  scp://dir/what ever

show tech-support chasis ---gives information on chasis,Blades,fex,I/O modules and BMC can be filtered.

Setting Threshold Polices:-

We can set thresolds on each

all sorts if ethernet port stats
FC port
vNIC Port

Service Profiles

Service Profiles :
Two models

1)Basic Opt-in Model :-Traditainal blade servers in that hardware defaults are used no logical pools.

2)Logical Server Pot-IN :- use pools for indentifiers and other information.When you move a profile
all the identifiers intact move as well.

3)Make sure your pools are designed  first since this requires some planning before.

Service Profile Templates  :-
Service profile Templates allow you to spawn new profiles.They are two templates.
Intial Template  :
If you modify the template the changes do not get passed to those porfiles
created from the templates

Updating Template:
If you make a change to the template that change gets pushed down to the profiles created from the
template.Need to set Mantinaece policy so that Blades do not reboot with notice.you can set user Ack.
---------------------------------------------------------------------------------------------
Native Vlan frames comming and out are not tag . Examples  Windows Machine and it need only one
vlan .

IPMI : Help DPM in case of VMware to power down server and bring them up when the need arise .

Autodeploy : Is boot from nothing ,it does a pix boot to a boot server and pool everyhting from Vcenter builds the image and runs on memmory .

Sunday, March 11, 2012

UCS Pools



Pools:-

Management IP Address Pool:Pool of IP addresses used to access a server via the CIMC(Cisco Intergarted Managment Controller)

Examples:KVM,serial over Lan.Mgmt IP are accessed through Mgmt Ports on FI and connections are

distributed half & Half .If you lose an FI you will lose half of the KVM access.No taging on MGMT IP's.


Can be added  to :Serive profiles,Servers and thorugh Template pool oh MGMT


Check if the Management IP on FI are same a Managemtn pools ?

UUID Pools:for servers, Similar to Serial Number,32 hex digit,only sufix is changed

MAC Address Pools:-LAN ->Pools--MAC Pools,00:25:B5 is cisco MAC start (Organization Unique identifier),Cannot chnage MAC pool name once created.

WWNN Pools:have 00:25:B5 total address 20:OA:00:25:XX:XX:XX gerally start with NAA -20 NAA-50 reservered for sttroage arrays

WWPN Pools:

Server Pools

Server Pool Memebership

Tuesday, March 6, 2012

UCS Pools


Pools:-
Management IP Address Pool:Pool of IP addresses used to access a server via the CIMC(Cisco Intergarted Management Controller)
Examples:KVM,serial over Lan.Mgmt IP are accessed through Mgmt Ports on FI and connections are
distributed half & Half .If you lose an FI you will lose half of the KVM access.No taging on MGMT IP's.
Can be added  to :Serive profiles,Servers and thorugh Template pool oh MGMT


Check if the Management IP on FI are same a Managemtn pools ?

UUID Pools:for servers, Similar to Serial Number,32 hex digit,only sufix is changed
MAC Address Pools:-LAN ->Pools--MAC Pools,00:25:B5 is cisco MAC start (Organization Unique identifier),Cannot chnage MAC pool name once created.
WWNN Pools:have 00:25:B5 total address 20:OA:00:25:XX:XX:XX gerally start with NAA -20 NAA-50 reservered for sttroage arrays
WWPN Pools:
Server Pools
Server Pool Memebership

UCS SAN Config


UCS uses the concept of service proflies for flexibility and mobility .To utilize stateless
ness of the blades you have to use boot from  SAN using FC or iSCSi protocals from ucsm 2.0.
Reason why iSCSI or FC should be used is becoz of block level protocal.We can not use NFS since this file based stroage protocal.

FIber Channel:
Operates in 1,2,4 ,8 and 16 Gb speeds ,Block Level Acess to stroage,servers use HBA to connect to storage Array using optical cables.

Servers are called initiators.Storage array is usually known as targets
To allow the two to comminicate we perform Zoning.You zone one or more initators to one or more targets.
Zoening is done on switches (MDC,Brocade,Macdata).Once you Zone you should also make sure server see the LUN is wants to and mask all the other Lun
using LUN making this is supported by stroage array.

HBA has two part A world Wide Node Name  for HBA and depending on the number of ports on HBA it would have one or two World Wide Port Name.
Example DUAL PORT HBA would have one WWNN and Two WWPN names. When you zone you zone them to WWPN not WWNN and make sure one port goes to one frabric
and other ports goes to another fabric.

Ports types:
N_Ports- Node ports,a port on the end node (Server or stroage array)
E-ports - expanssion Ports ,usaually connect to switches.
F-ports - Fabric ports,ports on a switch that connect N-Ports
Cisco UCS uses a technology known as N_Port ID Virtulization (NPIV).
NPIV allows a single N-Port to have many initators behind it.NPIV is enabled on FC Switch this lets
the FC hosts to see the FI as bunch of WWPN and WWNN like networking End host mode.FI use NPV not on the FC switch.

FC Switching Modes :

FC End host host mode: Uses NPIV,appers to Host as bunch of WWPN numbers.
FC Switching Mode
Fiber channel Port -Channels and Trunking:
Similar to Ethernet port Channels,bind multiple uplinks together if one fails other take over and better
load dsitribution only supported on MDS and Nexus FC Switch.
Trunking is smilar to ethernet Trunks: Allows for multiple VSANs over links.

6100 -- FC ports are located on Expansion Module
6200 -- use slider to ajust the port port on left are ethernet,right are FC and expasion module
FCoE: Fibre Channel encapsulated in ethernet.

Friday, March 2, 2012

UCS Comminication


STP is a protocal which detects from the formation of loops by blocking the ports when they see the same traffic

comming back.This typically happens during boardcast strom.

-----------------------------------------------------------------------------------

IF A blade1 in Chasis 1 connected to port 1 is sending a packet to Blade 2 in chasis 1 port 1 them packet is sent to FI and

switched back again if this is in same Vlan.If the traffic is detianed for outside the FI or another vlan or to a port

on other FI the traffic is sent out an uplink port.

Curretly 6200 are not layer 3 ready but this is going to happen in future.
-----------------------------------------------------------------------------------------------------------------

Traffic recived by the FI for a server is forwarded on  if the traffic arrives on the pinned uplink or port channel.

Reverese  Path forwarding

If traffic is recived on a diffrent uplink than one pinned to the server it is dropped

Deja vu Check
-----------------------------------------------------------------------------------------------------------------


UPlink Config from interconnect to Core Network Switches.

1) Atleast two uplinks from interconnect to Core Network Switches

2) or Use VPC(Virtual port channel  if these are nexsus 7000 series)

3) Configure interface in trunk mode

4) Configure Sapnning tree portfast trunk(So that failover happen faster)

5)Specify Vlans  to trunk

6)CreaTE PORT CHANNELS AND ADD INTERFACES .Port channel must be lacp active mode.

7)uplinked Switches must be trunk vlans to UCS.

8)Use pin gourps for DMZ atleeast UCSM 2.0

Fabric Interconnect Modes and Vlan config


Configureing to LAN :-
Basic Concept if one blade in cisco ucs need to communicate with other blade in same chasis or a diffrent chassis communication does not happen through fabric extenders internal ,commincation is passed thorugh FEX which query fabric interconnect for sending the information from one blade to another.

Fabric interconnect operate in two modes :

1)Ethernet End Host Mode (default) network sees this as host with bunch of MAC addresses of blades and VM's,no spanning tree

2)Ethernet Switching Mode you have to be carefull about spanning tree and other network issues.

Ethernet End host mode :-
FI only learn MAC address from the servers and VM's in Chasis they donot learn MAC addresses from outside network,server MAC addresses can move between uplinks and FI during failover.

Ethernet Switching Mode :-
stp is introduced and uplinks will be blocked not recommaneded.Port channels should be use LACP.

PIN Groups :
UCS by default will round robin the servers to all the uplinks.But if you want a specify a port and PIN it to a server you can use pin groups.
Example :DMZ ports

Saturday, February 18, 2012

UCS First setup


First Time setup :
1)Configure the Fabric InterconnectA and floating IP
2)Configure Fabric Interconnect B ip
3)Chasis Discovery Policy(1-uplink,2-uplink,4uplink) configure your minimum uplink policy
4)Power Policy (n+1 or grid)
5)MAC Address Table Aging.(mostly deafult)
6) DNS serevr and time serevrs and syslog

Cisco UCS UCLI

Service profile is like a VMX file in Virtulization world which holds all logical information like number of WWN,MAC,number of Cpu,amount of memoery
and infromation without extracting it from hardware.
----------------------------------------------------------------------
UCLI COmmands :
?
show
scope
top/up
where
connect
SSH to cluster IP :
#? shows all your commands
#where (shows your peresnt location )
#show ? (will show all your operation you can use)
#scope (Zoom down to particular item and dig into  it )
#scope chassis 1 (will get you in chasis one to look into details)
#up (goes back to pervious state )
#top (goes all the way back to start of ssh ip )
#connect (Connect to NX os or frabirc to do failover and others)
------------------------------------------------------------------

Tuesday, February 7, 2012

Cisco UCS Project California


Cisco UCS :Project california

UCSM  Embedded ,UCS Fabric Iterconnect,UCS FAbric Exterder (40Gbps or 80 Gbps), UCS Blade server chasis,UCS Virtual Adapters,4 Power supply (N+1 or Grid redendency);8 Fans modules 1 front and 1 Back total 16 fans.


UCS Fabric Iterconnect:-

1 st gen 6100 6120(20 port+one module slot comes with 8 port license)
2 cluster port in each Fabric interconnect ,Management Console port,20 Ethernet port,expandle module SAN port and two power supplies.

and 6140(40 port+two module slot comes with 16 port license) and 2 nd gen 6200 6248UP 48 unified ports supports port channels

Fabric interconnect do not pass data .If data need to pass from one fabric to another it need to go thorugh the core in the deafault configuration fabric interconnect can act as switches if configurations are changed.

Fabric interconnect A and B are active/passive from management point.But from LAN and SAN I/O are active on both fabirc interconnects.

UCS FAbric Exterder (40Gbps or 80 Gbps)
FEX :1 gen 2100 series 2104 (4 fabric port 10 Gb ETH/FCoe)
and 2 gen 2200 series gen 2208(8 fabric prots 10 gb eth/ fcoe)

UCS Blade server chasis:
B200M2 half wdith blade 12 DIMM slots for 192GB /16gb Dimms each,2 scoket 6 cores, 1 Mezzannine Adapter
B250M2 FUll Width Blade 48 DIMM solts for 384 GB /8GB Dimms each ,2 Scket 6 cores ,2 Mezzene card
B230M2 half width Blade 32 DIMM solts for 512 Gb/16 GB dimms, 2 socket 10 cores,1 Mezzanine card, with SSD

UCS Virtual Adapters:
Support feature of hardware failure if one fails all ports fail and move to surveying fabric also called CNA since they support Fabir cannel and ethernet

M81KR (Palo) Both FCoe and Etehrnet,support 56 each  HBA and NIC Virtual Interface
VIC1280 Both FCoe and Ethernet,support 116 total   Virtual Interface reqire 2208 FEX back fo chasis
M72KR Emulex/QLogic does not support Virtual interface)

Cisco Connectivty :
2* 1 Link 20 Gbps
2* 2 Link 40 Gbps
2* 4 Link 80 Gbps
2* 8 Link 160 Gbps