CCIE R&S 5.0 Virtual LAB Setup via VMware and 3560 Switches and more

If you are looking for a STEP by STEP of initial install please see the excellent link below:

http://ithitman.blogspot.com/2014/11/ccie-v5-ine-home-lab-part-1-configuring.html

What you need:

1. Any PC with 32 gig of RAM [ more ram if you want to make more routers. 2.5 Gig per router is recommended although you can get away with 2Gig. ]

2. 500 Gig of hard-disk [ would suggest raid 5 if you can afford it and 3 disks of 500 gig. Typically 8 to 10 Gig per router, 110 Gig for Linux host ]

3. CPU: anything quad core, or dual core with 2.1 GHz CPU clock rate should do.

4. FOUR 3560 Switches [ The X versions is better, but too expensive, so no X will do here :-) ]

5. At least TWO Ethernet ports. [ I used 6 just to make the routers reachable via public net and create scenarios like each switch having it's own 'trunk' connection or  access vlan setup for each router, etc, to really simulate complex L2 scenarios beyond the simple trunking stories in the practice labs; hay gotta push the limits!  This is CCIE]

6. CD/DVD/USB/PXE or any external device that the PC can be booted from for installing OS on the bare bone box the first time.

7. VMware ESXi version 5.5 or later.

8. VMware Vsphere to manage the virtual hosts from a PC running Windows.

9. Linux OS in CD or ISO format for tcpdump/wireshark, tftp, telnet, ssh, tclsh, VNC and other tools that come with the OS for FREE.
You can use Windows OS if you prefer that, but you need to pay for the OS and hunt the other software yourself.

Summary of steps for the impatient:
---------------------------------------
1. install ESXi on the PC; free download.  Assign an IP, user/pass.
https://www.vmware.com/go/get-free-esxi

2. install Vsphere on a Windows machine that can reach the ESXi, aka Hypervisor box using the Assigned IP in previous step.
I believe it is licensed, but you can use a trial version for few weeks before buying it.
https://my.vmware.com/group/vmware/info?slug=datacenter_cloud_infrastructure/vmware_vsphere/5_5

3. download Cisco CSR images for VMware, aka .ova file; you'd need to have a registered user account to download this. The file name below, do a search on it to learn more and pick the version you want.  csr1000v-universalk9.03.13.01.S.154-3.S1-ext.ova

Old document with *some* relevant info, just in case you have time to look at a lot of redundant details ;-).
http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/installesxi.html

4. put it all together; using Vsphere client application:

- Install the OVA file, aka CSR image for VMware guest OS. Repeat for as many routers as you want or have hardware to support it.
- Create a Virtual Switch and put the 2nd NIC card as the Gige 1 interface/vmnetwork2.
- Connect the 2nd NIC physically to one of the 3560 switches; enable cdp, dot1q trunk on the switch.
- Enable 'promiscuous' mode and 'all vlans' in Virtual Switch so everyone can see everybody. 


- I used 4 ports to connect to 4 switches for various scenarios.
- Planning to use 'ifconfig alias' on Linux interfaces so each Ethernet port on Linux can have multiple ip addresses; limit is 255 I think.  Will post any gotchas.

The final install should look something like so:

SW1#sh ver | i image
System image file is "flash:c3560-ipservicesk9-mz.150-2.SE6"
SW1#
SW1#sh int gi0/1 trunk

Port        Mode             Encapsulation  Status        Native vlan
Gi0/1       on               802.1q         trunking      1

Port        Vlans allowed on trunk
Gi0/1       1-4094

Port        Vlans allowed and active in management domain
Gi0/1       1,1000-1001,2000

Port        Vlans in spanning tree forwarding state and not pruned
Gi0/1       1,1000-1001,2000
SW1#
SW1#sh arp            
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  155.1.37.7              -   001d.a1f2.73c1  ARPA   GigabitEthernet0/3
Internet  192.1.1.1              40   000c.29d8.ba5a  ARPA   Vlan1
Internet  192.1.1.2              56   000c.2987.d997  ARPA   Vlan1
Internet  192.1.1.3              30   000c.29ee.68e2  ARPA   Vlan1
Internet  192.1.1.4             142   000c.29b7.9a6b  ARPA   Vlan1
Internet  192.1.1.5             131   000c.2993.b3c8  ARPA   Vlan1
Internet  192.1.1.6              65   000c.29a7.e3ef  ARPA   Vlan1
Internet  192.1.1.7              17   000c.2918.3a40  ARPA   Vlan1
Internet  192.1.1.8             132   000c.29fc.3bf6  ARPA   Vlan1
Internet  192.1.1.9              29   000c.2984.b56b  ARPA   Vlan1
Internet  192.1.1.10            228   000c.2923.17a7  ARPA   Vlan1
Internet  192.1.1.11             22   000c.2995.10b0  ARPA   Vlan1
Internet  192.1.1.12            140   000c.29e8.6db3  ARPA   Vlan1
Internet  192.1.1.13             58   000c.2964.64a4  ARPA   Vlan1
Internet  192.1.1.14            137   000c.29ec.800d  ARPA   Vlan1
Internet  192.1.1.15            130   000c.2991.9b22  ARPA   Vlan1
Internet  192.1.1.16            146   000c.2958.33f9  ARPA   Vlan1
Internet  192.1.1.17              1   000c.2922.7a53  ARPA   Vlan1
Internet  192.1.1.18              1   000c.299c.7e2b  ARPA   Vlan1
Internet  192.1.1.19              8   000c.2927.d87b  ARPA   Vlan1
Internet  192.1.1.20              2   000c.290e.b4c9  ARPA   Vlan1
Internet  192.1.1.21              -   001d.a1f2.73c0  ARPA   Vlan1
Internet  192.1.1.22            125   001d.e628.a240  ARPA   Vlan1
Internet  192.1.1.23            122   001d.a170.3c40  ARPA   Vlan1
Internet  192.1.1.24            142   001d.a1f2.7540  ARPA   Vlan1
Internet  192.1.1.254             1   000c.2963.d85a  ARPA   Vlan1
SW1#

Some output to match when you run into an issue:

R1#sh plat      
Chassis type: CSR1000V           

Slot      Type                State                 Insert time (ago)
--------- ------------------- --------------------- -----------------
R0        CSR1000V            ok, active            2d08h        
F0        CSR1000V            ok, active            2d08h        

R1#
R1#sh ver | i CSR
Cisco IOS Software, CSR1000V Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 15.5(1)S, RELEASE SOFTWARE (fc5)
cisco CSR1000V (VXE) processor (revision VXE) with 785444K/6147K bytes of memory.
R1#

Here is capture from the router which is exported to the Linux host over the same Gig1.

R1#monitor capture 1 control-plane both int gig1 both    
Control plane already attached in IN, changing direction to BOTH

R1#monitor capture 1 match any
R1#monitor capture 1 start
R1#
%BUFCAP-6-ENABLE: Capture Point 1 enabled.
R1#
R1#sh monitor cap 1 buffer
 buffer size (KB) : 10240
 buffer used (KB) : 128
 packets in buf   : 1017
 packets dropped  : 0
 packets per sec  : 4

R1#
R1#monit cap 1 export tftp://brick/tftp/iospcaps/control-plane-capture.pcap  
!
Exported Successfully

R1#
R1#exit
Connection closed by foreign host.
bastion:/data/tftp/iospcaps/$ ls -l
total 188
-rw-rw-rw- 1 opentftp root 188701 Jan 10 22:12 control-plane-capture.pcap
bastion:/data/tftp/iospcaps/$
bastion:/data/tftp/iospcaps/$ ifconfig eth1 | grep Bcast
          inet addr:192.1.1.254  Bcast:192.1.1.255  Mask:255.255.255.0
bastion:/data/tftp/iospcaps/$ 

 

Customizing configs

- Linux host at 192.1.1.254
- R1 through R20 at 192.1.1.1-20
-SW1 through 4 at 192.1.1.21-24
-tftp server at 192.1.1.254/tftp/*

Here is a quick 5 minutes script to convert the initial configs for various scenarios into what I need; namely, Gig1 needs an ip to be reachable from the Linux host.  Note that if Gig1 has no IP address, you lose your connection to the router which obviates running Vsphere and console connection....

You can use Virtual Serial console on ESXi, but needs a license, so telnet/ssh will do.

- using Gig1 as the management interface.
- adding my configs like ntp, ip hosts, history size, etc, [ hey Unix guy doesn't type if he/she doesn't have to ;-). ]
bastion:/data/tftp/ineconfigs/advanced.technology.labs/basic.bgp.routing/$ cat ~/bin/reconfig.sh

#!/usr/local/bin/bash
#
# add ip address for gige one so telnet would work, remove 'no ip add' and 'end' words.
# append my nifty configs
#
echo " running dos2unix removing Windowz thingy \r\n"
/usr/bin/dos2unix ./R*

echo " running sed to add vty ip address, remove no ip and add myconfig below"

MYCONFIG="
clock timezone PST -8 0
logging buffered 10000
no ip domain lookup
ip host muck 192.1.1.254
ip host brick 192.1.1.254
ip host bastion 192.1.1.254
!
ipv6 unicast-routing
!
no ip domain lookup
ip domain name cisco.com
!
line con 0
 exec-timeout 0 0
 privilege level 15
 logging synchronous
 stopbits 1
line vty 0 4
 exec-timeout 0 0
 privilege level 15
 logging synchronous
 no login
 length 33
 width 0
 history size 256
!
ntp server 192.1.1.254
"
# Note the zero in sed command to replace ONLY the first occurrence; man took me some searchin ;-).

for i in `seq 10`;
    do
    sed -i -e "0,/GigabitEthernet1/s//GigabitEthernet1\n ip address 192\.1\.1\.$i 255\.255\.255\.0/" -e "s/no ip address//" -e "s/end//" R$i.txt;
    echo "$MYCONFIG" >> R$i.txt;
    done

echo " cleaning the directory and moving the fixed up configs to /data/tftp/iosconfigs/cur"
ls -l /data/tftp/iosconfigs/cur/*
rm /data/tftp/iosconfigs/cur/*
cp ./R* /data/tftp/iosconfigs/cur/
ls -l /data/tftp/iosconfigs/cur/*
pwd
#put some error checking when have time and make it more flexible!



On the router to replace the config with mine.  IOS is funny with copy command unlike XR which has much much better facilities for config management.

R10#config repl tftp://brick/tftp/iosconfigs/cur/R10.txt
This will apply all necessary additions and deletions
to replace the current running configuration with the
contents of the specified configuration file, which is
assumed to be a complete configuration, not a partial
configuration. Enter Y if you are sure you want to proceed. ? [no]: yes
Loading tftp/iosconfigs/cur/R10.txt from 192.1.1.254 (via GigabitEthernet1): !
[OK - 5040 bytes]

Loading tftp/iosconfigs/cur/R10.txt from 192.1.1.254 (via GigabitEthernet1): !
Enter configuration commands, one per line.  End with CNTL/Z.
Warning: The input license udi(CSR1000V:9Y9VCZ8B841) differs from the platform udi(CSR1000V:97F7HDB8UH1).% use 'write' command to make license boot config take effect on next boot
... output snipped ...

How to access everything:

- Install VNC SERVER,  on the Linux Virtual host.
- Install VNC VIWER on your laptop/desktop/access computer and your rack is just couple of clicks away.
- Start working on the lab scenarios and when you get tired, just close the vnc, the next time you come back to the vnc, it has kept EVERYTHING where you left it off.