How to Create a VMWare lab on your machine

I have seen lots of posts by newly certified VCPs to know how they can create labs for testing and to enhace the knowledge they gained after certification. Creating VM ware "labs" is an expensive thing and not every one can afford. This post of mine is to help those who don't want to spend too much but still want to have Lab where they can create/break/troubleshoot things as they want and experiment things learnt in VCP course.

I started working on VMWare esxi host when 3.0 was about to come in market and after gaining few years of experience I thought of going for certification. This was the time when esxi host 4.5 was much talked about. I would like to thank Mr. Abdullah who was instructor for delivering such good lactures. After completing my certification I wanted to do lot of experiments with HA, vMotion, FT etc. but unfortunately could not find environement. The IT consultant company for which I am working has lots of virtualisation environment but at no client site we are using vCentre. So I made up my mind to buy a new system which can support virtualisation. Boght a new system with following configurations. 

Processor - Intel Core i7 2600

M.Board  - ASUS Extreme GENE-Z

HDD - 512 gb SATA

NAS device for creating shared storage.

Here is how I created my LAB with two esxi host 4.5 and a vCenter.

-Installed win7 utltimate on 100gb partition.

-Created two primary partition of remaining disks.

-Installed VM ware workstation 8 on one partition.

-Created 3 VMs in VMware Workstation. 1 for windows 2003 with 64 bit VM and 2 for esxi VMs.

-Installed windows server 2003 64 bit on the windows machine.

-Installed esxi 4.5 on each of the other VM. Configured IP and host name for both the hosts.

-Kept documenting stuff and I advise same to you too.

-Once both the Esxi hosts were accessible from another machine LAN, created 2 windows VM in each host.

-Installed vCenter on the windows 2003 server 64 bit which was created on vmware workstation 8.

-Accessed vCenter over the network and added both the esxi host to it.

-Here is your vmware lab ready with two esxi hosts which are being managed from vCentre.

You can do as much as experimentation as you like. If you already have a machine with mother board and processor supporting VT, you can go ahead with creating above lab. If  this article of mine is helpful to you do drop your feedback. Please experiment and share your findings which might be helpful for all. Thanks for reading.

Virtual Machine

A virtual machine is a set of hardware on which a supported guest oprating system and its applications run. The virtual machine is a set of discrete files. The virtual machine's configuration file describes the virtual machine's configuration , which includes the virtual hardware, such as CPU, memory, disk, network interface, Optical Drive and floppy drives etc. While naming virtual machine avoid use of special characters including spaces etc.

Files that make up a Virtual Machine

The table lists the files that make upa VM. Except for the log files, the name of each file starts with the VM's name . A virtual machine consists of the following files

• 
  
  Configuration file (.vmx)


• 
  
  One or more virtual disk files. The first virtual disk has files .vmdk and -flat.vmdk.


• 
  
  A file contiaining the VM's BIOS (.nvram).


• 
  
  A VM log file (.log) and a set of files used to archive old log entries (-#.log). Six of these files are maintained at any one time.


• 
  
  A swap file (.vswp).


• 
  
  A snapshot description file (.vmsd). This file is empty if the virtual machine has no snapshots.

If the virtual machine is converted to a template, a virtual machine template configuration file (.vmtx) replaces the virtual machine configuration file (.vmx).

If the virtual machine has more than on disk file, the file pair for the second disk file and later is named (VM_name>_#.vmdk and (VM_name>_#-flat.vmdk, where # is the next number in the sequence, starting with 1.

Six of the archive log files are maintained at any one time. e.g. -1.log to -6.log might exist at first. The next time an archive log file is created e.g. when the virtual machine is powered off and powered back on -2.log to -7.log are maintained (-1.log is deleted), then -3.log to -8.log and so on.

A VM can have other files e.g. if one or more snapshots were taken or if raw device mappings were added. A VM will have an additional lock file if it resides on an NFS datastore. A VM will have a change tracking file if it is backed up with the VMware data recovery appliance.

The configuration files of a VM can be displayed by going to Configuration > Datastore; right click datastore and browse. The another option to VM configuration files is by going to Configuration > Datastore > View and select 'Show All Virtual Mahine Files'

Virtual Machine Hardware

A VM uses virtual hardware. Each guest OS sees ordinary hardware devices. It does not know that these devices are virtual. All virtual machines have uniform hardware (except for a few variations that the system Administrator can apply). Uniform hardware makes virtual machines portable across VMware virtualization platforms. We can add and configure VM's CPU, Memory, HDD, CD/DVD, Floppy, SCSI devices etc. But we can't add video devices, but can configure existing video devices. A USB connected to physical host can be made available to one VM at a time. To make it available to another VM it should first be disconnected from existing VM.

CPU and Memory: VMware Virtual SMP allows you to take advantage of configuring a VM with upto 8 vCPU and Memory upto 255 gb.

Virtual Disk: The VM should have atleast one virtual disk. Adding the first virtual disk implicitly adds a virtual SCSI adapter for it to be connected. The ESX/ESXi host offers a choice of adapters: BusLogic Parallel, LSI Login Parallel, LSUI Logic SAS, and VMware Paravirtual. The VM creation wizrd in the vSphere client selects the type of virtual SCSI adapter, based on the choice of guest OS. By default Virtual disk is thick provisioned but this can be changed to thin-provisioned disk if needed. There is special disk mode call independent. The independent mode has two options: persistent and non-persistnet. If we want changes to disk written immediately and permanently then persistent is chosen. Non-persistnent option is chosen to discard changes when the VM is powered off or reverted to a snapshot. In most cases there is no need to change the disk mode for virtual disks.

vNetwork Interface Card: The network cards available for Virtual machine are Flexible (vlance & vmxnet), e100, vmxnet2 (enhanced vmxnet) and vmxnet3.

Other Devices: A VM can have CD/DVD drive or Floppy drive on the physical host. The CD-ROM, DVD drives either connect to physical drive or to an ISO file. We can also add generic SCSI devices, such as tape libraries to VM. These devices can be connected to the virtual SCSI adapters on a VM.

Virtual Machine Console: From console we can send power changes to a VM. The VM's guest OS can be accessed from it. The Ctr+Alt+Del is sent from console. If VM tolls are not installed then to release pointer VM's Console Ctr+Alt.

Networking

Before we go for understaning vNetwork and vSwitch, let us understand the importance of vmware ESX/ESXi Network. The networking feature in the ESX/ESXi host allow virtual machines to communicate with each other as well as VMs on other host, IP based storage and physical machines. This means it is as good as configuring a physical network but the difference is that we don't have to deal with lots of networking cables. If the network is not configured properly it become difficult to manage VMs, IP storage etc.

In VMware ESX/ESXi network there are two important components to be understood; 1) vNetwork and 2) vSwitch.

Virtual Network provides networking for VMWare ESX/ESXi hosts and VMs. The fundamental component of a vNetwork is vSwitch.

Virtual Switch

-It directs network traffic between virtual machines and links to external networks. All network communications handled by a host passes through one or more virtual switches. A virtual switch provides connections for virtual machines to communicate with one another, whether they are on the same host or on a different host. A virtual switch allows connections for the management network on ESXi hosts and for the service console on the ESX host.

-We use vSwitch to combine the bandwidth of multiple network adapters and balances traffic among them. It can also handle physical network interface card (NIC) failover.

-Models a physical Ethernet switch e.g. VM's NIC can connet to a port and each uplink adapter uses one port.

-Virtual switches work at layer 2 of the OSI model. You can't have two virtual switches mapped to the same NIC but you can have two or more NICs mapped to the same virtual switch.

-When two or more virtual machines are connected to the same virtual switch, network traffic among them is routed locally. If an uplink adapter is attached to the virtual switch, eacj VM can access the external network that the adapter is connected to.

Let us now understand virtual switch in more details for ESXi hosts only.

Uplink Ports: Its number of Physical adapters (NIC) available.

Port Groups: A vSwitch can be subdivided into smaller units called port groups. There are three typs of port groups and each port group is related to different type of traffic: a) Virtual machines Port Group; b)Service Console port group (only ESX); c)VMKernel port group (used for vMotion, FT, IP Storage etc.)

A virtual switch provides three types of conection types to hosts and VMs.

• 
  
  Connecting VMs to the phycial netork.


• 
  
  Connecting VMkernel to physical network. VMkernel services include aces to IP storage, such as NFS or iSCSI, vMotion, access to the managment network on ESXi host.


• 
  
  Providing networking for the service console for ESX only.

Separate IP stacks are configured for each VMkernel port and the ESXi management netork port . Each ESXi management network port and each VMkernel port must be configured with tis own IP address, netwmask and gateway. All three port groups connect to outside world through physical adaptors assigned to the vSwitch. We can place all networks on a single vSwitch or multiple vSwitches can be opted which depends upon the situation.

There are two types of vSwitches; vNetwork Standard switch and vNetwork distributed switch. The vNetwork standard switch is configured at host level. We can have maximum of 4088 vSwitch ports per standard switch and 4096 vSwitch ports per host. The vNetwork distribued switch is configured at vCenter. Its components are same as vNetwork Standard switch but it functions as a single virtual switch across all associated hosts. This allows virtual machines to maintain consistent network configuration as the migrate across multiple hosts.

The virtual switches will be discussed in more details in succeeding posts.

ESXi - Storage

One of the very important section of ESXi is to understand Storage. Though big companies have Storage Administrators to take care of Storage but a VMware administrator has lot to understand and take care of this section.

Depending on type of storage used, datastores can have these two file systems; VMFS & NFS. The datastores are logical containers like file systems that hide specifices of each storage device and provide a uniform model for storing virtual machine files. Datastores can also be used for storing ISO images, VM templates etc. Several storage technologies are supported by VMware ESX/ESXi hosts in the VMware vSphere environment. These are locally attached storage, /fibre Channel, iSCSI and NAS (Network attached Storage). Let us understand Datastores, VMFS, NFS in more details before proceeding further.

DATASTORES

A Datastore is a logical storage unit that can use disk space on one physical device or one disk partition or can span across several physical devices. A VM is stored as a set of files in its own directory in the datastore. The datastore can be a VMFS or NFS datastore. The datastores can be used to store ISO images, floppy images, virtual machines and templates. A VMFS datastore can also hold Raw Device Mapping (RDM) which is used by a VM to access its data.

RDM (Raw Device Mapping) : The VMs running on ESX/ESXi host, instead of storing VM data in virtual disk file, you can store the data directly on a raw LUN. Storing the data this way is useful if you are running applications in your VMs that must know the physical characteristics of the storage device. Mapping a raw LUN allows you to use existing SAN commands to manage storage for the disk. The RDM is used to map to the raw LUN. RDM is a special file in a VMFS datastore that acts as a proxy for a raw LUN. It maps a file in a VMFS data store to a raw volume. A VM then references the RDM, which in turn points to the raw volume holding the VM's data. The raw device mapping is recommended when a its is must that VM should interact with a real disk on the SAN. e.g. when you make disk array snapshots or when you have a large amount of data that you do not want to move onto a virtual disk. When there is requirement to use VM in cluster using MSCS then Raw Device Mapping is very important.

Virtual Machine File System (VMFS) : The datastore reconised by VMFS which is created by assigning unpartitioned disks space. Once it is assigned an ESXi host automatically discover VMFS volume. This datastore is a clustered file system that allows multiple physical servers to read/write to the same storage device simulataneously. The cluster of file system enables unique virtualization-based services, which include:

Live migration of running VMs across physical servers.

Automatic restart of a failed VM on another physical server.

Clustering of VM across different physical servers.

The VMFS allows IT organisations to greatly simply virtual machines provisioning by efficiently storing the entire machine state in a central location. VMFS sllows multiple ESX/ESXi hosts to concurrently access shared virtual machine storage. The size of a VMFS datastore can be increased dynamically while VMs residing on it are powered on. A VMFS datastore efficiently stores both large and small files belonging to a VM. A VMFS datastore can be configured to use an 8mb block size, to support large, virtual disk files up to 2 TB in size. A VMFS datastore uses sub-block addressing to make efficient use of storage for small files. VMFS provides block-level, distributed locking to ensure that the same virtual machine is not powered on by multiple servers at the same time. If a physical server fails, the on-disk lock for each virtual machine can be released so that virtual machines can be restarted on other physical servers. The VMFS can be deployed as locally attached storage, FCS SAN, iSCSI storage and these appear to VM as mounted SCSI device. The virtual disk hides the physical storage layer from the virtual machine's OS. For the OS on a VM, VMFS preserves the internal file system semantics. The semantics ensure correct application behavior and data integrity for applications running in VMs. To view existing ESXi VMware datastore, login in to host using VMware vSphere client, Click on configuration >> Storage. Click on any datastore whose detail is needed. It will show; Volume label (datastore..), Device, Capacity, Free, file system etc. To add another datastore, Click add storage from here, leave default option Disk/LUN select >> Click next which will bring to screen where you can un-assigned LUNs. Selected required storage and click next >> review layout and click next >> Give a suitable name to it click next >> here is the section where you can choose appropriate block size. The Virtual disk upto 256gb will require block size 1 mb, 512gb - 2mb, 1 TB - 4 mb, 2 TB - 8 mb. >> Click finish. To understand Block size in respect of disk size in more details please refer http://www.vmware.com/pdf/vi3_301_201_config_max.pdf . Though this pdf talks about host 3.1 and before but good to take a look.

LOCAL AND SHARED STORAGE

All VMware ESX and ESXi installables are installed at Local storage. Also it is ideal for small environments. When all VMs are located on local sotrage management become easier. The usage of shared storage is

>vMotion.

>fast central repository for virtual machine templates

>recovery of VMs on another host incase of a host failure.

>Clustering of VM across hosts.

>Allocate large amounts of storage.

IP Storage

The ESX & ESXi hosts support two types of IP storage; iSCSI and NAS. iSCSI is used to hold one or more VMFS datastores whereas NAS is user to hold one or more NFS datastores. Both are used to hold virutal machines, ISO images and templates. vMotion, HA, DRS etc. features are supported on these datastores. The ESX & ESXi hosts support upto 64 NFS datastores and iSCSI and NAS run over 10 Gbps which provides increased storage performance.

iSCSI Components

Let us take a scenario where an iSCSI SAN storage system has more than one LUNs and 2 Storage processors. The communication between host and storage happen over a TCP/IP network. The components for this scenario will be as below:

1. iSCSI Storage consists of, number of physical disks under number of LUNs and storage processors connecting to TCP/IP network.

2. TCP/IP Network

3. Physical servers with software or hardware iSCSI initiators (HBA) connecting to TCP/IP network.

An iSCSI initiator trasmits SCSI commands over the IP network. A target receives SCSI commands from the IP network. You can have multiple initiators and targets in your iSCSI network. iSCSI is SAN oriented because the initator finds one or more targets, a target presents LUNs to the initiator and the initiator sends it SCSI commands. An initiator resides in the ESX/ESXI host. LUN masking is also available in iSCSI and works as it does in Fibre Channel. Ethernet switches do not implement zoning like Fibre Channel switches. Instead you can use VLANs to create zones.

Let us discuss iSCSI initiators; Software & hardware. The Software iSCSI initiator is VMware code built in to the vKernal. It allows host to connect to the iSCSI storage device through standard network adapters. The software iSCSI initiator handles iSCSI processing while communicating with the network adapter. With the software iSCSI initiator you can use iSCSI technology without purchasing specialised hardware.

Hardware iSCSI - A hardware iSCSI initiator is a specialised thierd-party adapter capable of accessing iSCSI storage over TCP/IP. Hardware iSCSI initiators are divided further into two categories: dependent and independent hardware. A dependent hardware iSCSI initiator or adapter depends on VMware networking and on iSCSI configuration and management iterfaces provided by VMware. This type of adapter, such as a broadcom 5709 NIC presents a standard network adaptor and iSCSI off-load functionality for the same port. To make this type of adapter functional, one must setup networking for the iSCSI traffic and bind the adapter and an appropriate vKernel iSCSI port.

An independent hardware iSCSI adapter handles all iSCSI and network processing and management for the host. The QLogic 4062C is an example of an independent hardware iSCSI.

Install and Configure VM ware ESXi 4.1 host

As it was stated in previous post that ESXi 4.1 software has a footprint as small as 70 mb, just like firmware insalled on the system. It comes as embedded in a server's physical hardware by vendor or burnt onto an external USB key. But remember that moving ESXi embedded on USB external key from one server to another is not supported. The ESXi software also comes as an installable CD/DVD.

Let's now proceed with the installation. We are considering Dell server which meets all pre-rquisits, hardware requirements etc. and we are installing from a DVD. To know more about hardware requirements please visit below link: http://kb.vmware.com/kb/1022101

Installation

Turn on the server, insert the DVD and boot from disk. Once installation is initiated, it will take few minutes to complete without any further keyboard inputs and you will be presented with following screen after installation is complete.

Configuration

Next step is to configure your ESXi 4.1 host. Press F2 to go to configuration screen. You will be prompted for user name password. By default admin user is 'root' and password is blank. Once you login you will reach to configuration console know as (DCUI) Direct Console User Interface. The DCUI looks as below:

First thing is to configure root password. It is important to configure root password to be able to login to vSphere client.

-Configure Host Name
-Configure IP Address, Subnet mask, Gateway
-Configure DNS servers.

Once above configuration is done, press F2 to exit the DCUI and F12 to reboot your host. Make sure you add a host record to your DNS server for this ESXi host. Now you are ready to create virtual machines on your ESXi 4.1 host. Open browser on any machine in the same subnet in which you have your host. Type the host name or IP address of your ESXi 4.1 host and you will be presented with below screen:

Click Download vSphere Client, once download completes continue with the istallation. Once installation is complete launch vSphere client from Start > All programmes >> enter the IP address or Host Name of your ESXi 4.1 host >> user name - root and password.

You are now in virtualisation world, explore all settings and my next post will be about creating virtual machines.

What is Virtualisation

The ability to run multiple machines with separate Operating system and Application on a single machine is Virtualisation. In other words the technology with which we can transform hardware into software is Virtualisation. Virtualisation uses either Hosted Architecture or Bare-Metal hypervisor Architecture.

HOST BASED VIRTUALISATION
It requires an Operating System to be installed on a computer. For example a computer which is running Windows 7/xp or any linux Operating system, VMWare server or workstation Virtualisation layer is installed on it. This is called host based Virtualisation. Once VM ware server/workstations version is installed, virtual machines can be created and deployed. This virutal machines will have their own Operating Systems and set of Applications. Though this is good solution in testing environment and production environment to certain extent but this might not be an idle solution for production environment. There are few drawbackups e.g. rebooting hosting OS means all virtual machines will be rebooted too and many more.

VIRTUALISATION USING A BARE-METAL HYPERVISOR
A Bare-metal hypervisor system does not require an operating system. The Hypervisor itself is an Operating System. It installs the virutalisation layer directly on a clean x86-based system. Because a bare-metal hypervisor architecture has direct access to the hardware resources, rather than going through the Operating System which make it more efficient than hosted architecture. This delivers greater scalability, robustness and performance. A hypervisor is the primary component of virtualisation that enables basic computer system partitioning i.e. simple partitioning of CPU, memory and I/O. VMware ESX/ESXi employs a bare-metal hypervisor architecture on certified hardware for datacenter-class performance.

WHAT IS A VIRTUAL MACHINE
From a user's perspective, a virtual machine is a software platform that like a physical computer runs an operating system and applications.
From Hypervisor's perspective, a virtual machine is a discrete set of files. The main files are as below;

- Configuration file
-Virtual disk file
- NVRAM settings file
-Log file

WHY VIRTUALISATION
Virtual machines have many advantages over physical machines.
Virtual machines are easy to move and copy across different hosts.

Encapsulated into files.

Independent of Physical hardware.

Isolated from othr virtual machines.

Insulated from hardware changes.

Provides the ability to support legacy applications.

Allows server to be consolidated.

VMWare Basic things to start with

Those who already know basics about VM Ware or already working on this technology can skip reading this article. But if you are aspirant to learn this technology and want to make your careers then here are few basic things.

It is very important that one should be comfortable with networking, subnetting type of storage and should have worked System Engineer on windows, Linux platforms. Though these are not the madatory requirements but these will make learning easier. If you are comfortable with above mentioned things then it will be very much helpful to do your homework before enrolling for classroom training. Because VMWare Certification is not possible without minimum 5 days of classroom training from an authorised VMWare training partner. As 5 days classroom training is approx. 45000/- (US $ 1000), so home work before going for classroom training become very important. The first thing you can do is join VM Ware community http://communities.vmware.com/. The community will let you know what experts are talking about. The next thing you can do is little hands on. The trial software can be downloaded from VM Ware website. http://www.vmware.com/support. You will find Various products listed on VM Ware's official website but the one we will be discussing about is Esxi 4.x, the one which will be used in data center. All relevant documents can be found on VM ware website.

VMWare vSpher Introduction

VMWare vSphere leverages the power of virtualisation to transform datacenters into simplified cloud computing infrastructures and enables IT organisations to deliver flexible and reliable IT services. VMWare vSphere virtualises and aggregates the underlying physical hardware resources across multiple systems and provides pools of virtual resources to the datacenter. As a cloud oprating system. VMWare vSphere manages large collections of infrastructure, such as CPUs, Storage and networking, as a seamless and dynamic oprating environment and also manages the complexity of a datacenter. The following component layers make up VMWare vSphere

---->>Infrastructure Services,

---->>Application Services,

---->>VMware vCenter Server

---->>Clients

The very important thing about VMWare esxi 4.x is that this is plateform independent which means it is directly installed on baremetal. The previously VMware had ESX which came with service console but with lauch of esxi 4.x VMware had obselete the ESX which means no more service console. The ESXi 4.1 foot print is as small as 32 mb. You initiate the installation and no keyboard input required. The hardware vendors like Dell (I will talk about Dell because am skilled on Dell PE servers) has esxi 4.1 embedded. You create vritual machines running Services like DC, AD. Exchange servers, file/print servers, db server on top of this 32 mb footprint.