BGP Email Alert from Router using Cisco EEM

Here was a fun thing.  A customer wanted to get email alerts when BGP changed status on their routers.  I used Cisco Embedded Event Manager (EEM) Scripting to achieve this.

I was able to run this on IOS 15.1.(4).

router bgp xx
bgp log-neighbor-changes

event manager environment _email_to customeremail@customer.com
event manager environment _email_server customer-mail-server.com
event manager environment _email_from Router-Hostname@customer.com
event manager applet BGP-Alert
event syslog pattern "%BGP-5-ADJCHANGE:*"

action 1.0 mail server "$_email_server" to "$_email_to" from "$_email_from" subject "$_event_pub_time:" body "$_syslog_msg"

action 1.5 syslog priority notifications msg "BGP Message - Mail Sent"

Configure the BGP process to log neighbor changes.
When the router matches a syslog entry for %BGP-5-ADJCHANGE: an email will be generated using the environment paramaters you set.
The router will also log a message in syslog that the mail was sent.

Keep in mind the IP's for the interfaces will need to be allowed to send mail on your mail server.  If this isn't working for you, run a debug on eem.  If the problem is the mail server rejecting the mail, it is pretty easy to decipher.

Sample down message:

From: Router-Hostname@customer.com

2883577: .Apr 10 02:45:59.928 EDT: %BGP-5-ADJCHANGE: neighbor 5.5.5.5 Down Interface flap

Cisco EEM Best Practices:
https://supportforums.cisco.com/docs/DOC-12757

Cisco EEM Basic Overview and examples:
https://supportforums.cisco.com/docs/DOC-27996

Ubuntu DNS slow in 12.04

I noticed recently that I had slowness in Ubuntu.  It was taking forever to surf the web.  No problems using IP addresses but pinging with dns was very slow.

I run Ubuntu 12.04 (yes I enjoy Unity!) and as a consultant I go to many different customer sites.  I had recently run updates so I thought something got screwed up on my machine or it was slowness at a site.  After a couple of days, updates, and different sites I noticed no change so I investigated.

In 12.04, DNS was changed quite a bit and uses something called resolvconf now.  This actually works much better with some VPN networks I connect to in terms of split tunneling so I've been happy with it. I had to change some scripts I use on my machine for stats but that wasn't a big deal.  No more /ets/resolv.conf or so I thought.

I checked my DNS stats and everything checked out.


nmcli dev list | grep DNS
IP4.DNS[1]:                             208.67.222.222
IP4.DNS[2]:                             208.67.220.220

I finally looked at resolv.conf and found entries in the file for a site I had been to a few days ago.  Clearly my machine was trying to hit those DNS servers, timing out, and finally using my real ones.  I manually edited resolv.conf to remove those servers and I'm blazing again.

Not sure why the reslov.conf file wasn't clearing out but something to keep an eye on.

Check Cisco md5 checksum on Ubuntu

Here's a quick tip.  You may have noticed when downloading files from certain locations such as Cisco.com, MD5 checksums are listed.

Something like this:

MD5 Checksum:

04f9d3f5a91a8b80397773a265a88848

MD5 checksums verify integrity of files you have downloaded.

In Ubuntu (and most other distros), md5sum is a program automatically included that allows you to check MD5 Checksums.

Simply run md5sum from terminal.

md5sum waas-universal-4.5.1.2-k9.bin
04f9d3f5a91a8b80397773a265a88848  waas-universal-4.5.1.2-k9.bin

One more tip.  Depending on the device and IOS image, you can verify MD5 on Cisco routers as well.  

Simply run

verify /md5 flash:iosimage.bin

Router#verify /md5 flash:c2800nm-spservicesk9-mz.124-25f.bin

.....Done!

verify /md5 (flash:c2800nm-spservicesk9-mz.124-25f.bin) = 09322ffb9111bce00a8eef38165f30e4

Cisco UCS KVM key problem with Ubuntu Linux

I was building a new VMware 5 ESXi machine on a UCS C series server via KVM.  I get to the point where I need to use the arrow keys to make a selection. 

Unfortunately, the arrow keys were not responding.  Nothing.

I'm working from my Ubuntu laptop (12.04 if you want to know and yes I do like Unity!) which doesn't have a number pad.  Not the obvious one anyway.  I finally remembered that I can use the function key and some standard keys to simulate the number pad and this allowed me to use the 'arrow keys'.  My keyboard doesn't show the arrows but its 8 for up and 2 for down on a keyboard with a number pad.

Install complete.

Link aggregation - Port Channeling - (Etherchannel)

When connecting redundant paths to servers or other network gear such as switches, it is often ideal to connect multiple links for redundancy.  Enabling link aggregation allows all connected links to pass traffic in an active/active fashion.  Failure on 1 port does not affect the link as traffic just continues flowing on the remaining ports.  If Etherchannel is set up on 1 side (server, switch, whatever) it must be set up on the the side of the link!!

Multiple options exist for this feature.
EtherChannel - Cisco proprietrary standard.
Port Aggregation Protocol (PAgP) - Cisco proprietary - Allows for the dynamic creation of Fast Etherchannel links.  This is not supported on all Cisco devices and is going away.  Like the ISL of trunking :)
Link Aggregation Control Protocol (LACP) - IEEE standard 802.3ad

Some things to keep in mind about Port Channeling.
It is considered best practice to hard code the port channeling method and not rely on dynamic protocols.  PAgP isn't even supported on some Cisco devices such as NX-OS and IOS routers.
Check the load balancing method deployed on each configured device, defaults are not the same.

Switch IOS Configuration:


Port config:
Samples show trunks but can be access.

Interface GigabitEthernetX/X/X
 switchport trunk encapsulation dot1q
 switchport mode trunk                                                        
 channel-group 1 mode on
Interface GigabitEthernetX/X/Y
 switchport trunk encapsulation dot1q
 switchport mode trunk                                                        
 channel-group 1 mode on
Interface Port-Channel 1
 switchport trunk encapsulation dot1q
 switchport mode trunk

Check load-balance method:
show etherchannel load-balance
Recommended changes in config mode:

port-channel load-balance src-dst-ip !(to enable IP-based load balancing)
port-channel load-balance src-mac !(to enable MAC-based load balancing)

Port channel options:
switch (config-if)#channel-group 1 mode ?

  active     Enable LACP unconditionally
  auto       Enable PAgP only if a PAgP device is detected
  desirable  Enable PAgP unconditionally
  on         Enable Etherchannel only
  passive    Enable LACP only if a LACP device is detected


Check port channel config settings:
switch#show etherchannel 1 summary

Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator

        M - not in use, minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port


Number of channel-groups in use: 1
Number of aggregators:           1

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)          -        Te2/0/2(P)  Te3/0/2(P)

NX-OS:
The commands are similar but different.  IOS uses etherchannel, NX-OS uses port-channel syntax.


Port-Config
Interface EthernetX/X
switchport mode trunk
 channel-group 1

Interface GigabitEthernetX/Y
switchport mode trunk
 channel-group 1
Interface Port-Channel 1
 switchport mode trunk

Check load-balance method:

show port-channel load-balance

The default should be source-dest-ip and you should be good to go here.

Port channel options:

Nexus(config-if)# channel-group 1 mode ?
 active   Set channeling mode to ACTIVE

  on       Set channeling mode to ON

  passive  Set channeling mode to PASSIVE

Check port channel config settings:

Nexus# show port-channel summary 

Flags:  D - Down        P - Up in port-channel (members)

        I - Individual  H - Hot-standby (LACP only)

        s - Suspended   r - Module-removed

        S - Switched    R - Routed

        U - Up (port-channel)

--------------------------------------------------------------------------------

Group Port-       Type     Protocol  Member Ports

      Channel

--------------------------------------------------------------------------------

1     Po1(SU)     Eth      NONE      Eth1/9(P)    Eth1/10(P)

ASA (New in 8.4!)

No Global load-balance command.  It goes under the port-channel.  When using etherchannel, you'll probably want to start using trunking since you'll be using up more interfaces.

Port-Config

interface Ethernet0/1
channel-group 1 mode on

interface Ethernet0/2

channel-group 1 mode on

interface port-channel1

port-channel load-balance src-dst-ip-port

IOS Routers:

Etherchannel on IOS routers looks to be supported on 2811 and above it looks like.  Not all modules are supported!  Load balancing can not be configured.  IOS routers are set to source and destination IP load balancing.  Only 802.1Q trunking is supported.  Who uses ISL anymore!?

Port-Config

interface Port-channel1

ip add 192.168.0.1 255.255.255.0

int GigabitEthernet0/0

channel-group 1

int GigabitEthernet0/1

channel-group1

VMware server:

To configure vSwitch properties for load balancing:
Source: VMware KB

1. Click the ESX host.
2. Click the Configuration tab.
3. Click the Networking link.
4. Click Properties.
5. Click the virtual switch in the Ports tab and click Edit.
6. Click the NIC Teaming tab.
7. From the Load Balancing dropdown, choose Route based on ip hash.
8. Verify that there are two or more network adapters listed under Active Adapters.
   

<

Note: The only load balancing option for vSwitch or vDistributed Switch that can be used with EtherChannel is IP HASH.

• Do not use beacon probing with IP HASH load balancing.
• Do not configure standby or unused uplinks with IP HASH load balancing.
• VMware support only one EtherChannel per vSwitch or vNetwork Distributed Switch (vDS).

Access Network Load Balancing (NLB) server and WLC

I ran in to a problem where users on the wired LAN could access a Microsoft cluster IP but not through the Cisco Wireless LAN.  You could even ping the virtual NLB IP from the WLC itself.  I found that the you need to enable a bogus multicast address on the Wireless LAN Controller under general settings and enabling multicast on the global settings.

Controller - Multicast - Enable Multicast Mode

and

Controller - General - AP Multicast Mode to Multicast and define an IP.

This should be enough if wireless users and the servers are on the same VLAN.  Of course, if the servers are on a different VLAN, you need to enable multicast on your switches or routers.

Multicast with Wireless LAN Controllers (WLCs) and Lightweight Access Points (LAPs) Configuration Example

Five handy secure shell tips and tricks

Five handy secure shell tips and tricks

UCS Links

Some UCS Links - blog.scottlowe.org - The weblog of an IT pro specializing in virtualization, storage, and servers

Some UCS Links