dimanche 17 mai 2015
jeudi 14 mai 2015
Ping vs. Traceroute vs Pathping
One of the biggest misconceptions of all time in networking is the use of a traceroute to determine that your communication with a server has high latency. On windows, traceroute is the same command as tracert.
Many people beleive that when they see high latency such as 250ms+ in a single hop of a trace-route that it means that that device in the transit path is responsible for the degraded network performance when in fact it could not be more further from the truth.
First lets look at how ping works.
PING, is an application based on the ICMP protocol which is used to send echo packets to a destination and expecting to receive an echo response and it calculates the RTT (Round Trip Time) from when the packet was sent to when the echo response was received. Generally when using PING on a LAN network you can trust that what it is saying is accurate unless you have foreknowledge of network devices in the transit path that prioritize ICMP over mission critical TCP/UDP Traffic. This however is very common in networks that utilize unified communications, meaning voice and data on the same network. This is because QoS Policies are put in place to ensure voice traffic and other mission critical traffic is prioritized over ICMP thus indirectly affecting the RTT time of an ICMP ping test.
Trace-route is another method commonly used by technicians and engineers to diagnosis latency in the transit path however any engineer that has studied how trace-route works would know that its results are nearly always misleading.
Trace-route works in a manner similar to ping however it uses the TTL feature to make each successive hop in the transit path respond with an ICMP TTL Expired packet. Thus gives you the ability to determine which network devices the ICMP packet is traversing.
When you dig deeper into the operation of traceroute you will see that traceroute utilizes 3 probe packets for each successive hop by default unless you specify other wise. Each probe packet indirectly measures the latency between the source and the device where the TTL is declared expired. This latency calculation is a by product of its true intended purpose. Keep in mind even if you send probes to a device that is five hops away, random latency spikes in any four devices prior to the fifth hop can result in the fifth hop looking like it has high latency.
Also note that any Control Plane Policing policy enforced on any device in the transit path could result in ICMP being prioritized to the control plane of the transit device. ICMP is processed switched by most devices whereas TCP/UDP is express forwarded.
An example below of a traceroute on Windows 7;
C:\>tracert www.google.com -dTracing route to www.google.com [74.125.225.113]over a maximum of 30 hops: 1 1 msYou can see from the trace route shown above that there is 3 probes per hop between the source and destination and that it does not appear to have latency until traffic traverses 64.69.97.217
The whole point of this blog is to teach you how to interpret such data. Just because you see a spike in latency on the 5th hop does not mean that the 5th hop is causing latency. It can easily mean that the control plane in the device on the fifth hop is under marginal load and that the processor does not respond to the ICMP immediately due to other processes with priority.
Just because you see potential latency with trace-route, you should never expect that to be an accurate representation of latency for TCP/UDP traffic because ICMP and TCP/UDP traffic is treated completely different when it comes to the routers control/forwarding planes.
Most ISP’s use control-plane policing (CoPP) to prevent overwhelming ICMP flooding to a devices control plane. This type of flood prevention mechanism can also result in skewed data in trace routes.
Shown below is a simple CoPP Policy which can result in skewed trace route data.
!class-map match-all Catch-All-IP match access-group 124class-map match-all Management match access-group 121class-map match-all Normal match access-group 122class-map match-all Undesirable match access-group 123class-map match-all Routing match access-group 120!policy-map RTR_CoPP class Undesirable police 8000 1500 1500 conform-action drop exceed-action drop class Routing police 1000000 50000 50000 conform-action transmit exceed-action transmit class Management police 100000 20000 20000 conform-action transmit exceed-action drop class Normal police 50000 5000 5000 conform-action transmit exceed-action drop class Catch-All-IP police 50000 5000 5000 conform-action transmit exceed-action drop class class-default police 8000 1500 1500 conform-action transmit exceed-action transmit!access-list 120 permit tcp any gt 1024 10.0.1.0 0.0.0.255 eq bgpaccess-list 120 permit tcp any eq bgp 10.0.1.0 0.0.0.255 gt 1024 establishedaccess-list 120 permit tcp any gt 1024 10.0.1.0 0.0.0.255 eq 639access-list 120 permit tcp any eq 639 10.0.1.0 0.0.0.255 gt 1024 establishedaccess-list 120 permit tcp any 10.0.1.0 0.0.0.255 eq 646access-list 120 permit udp any 10.0.1.0 0.0.0.255 eq 646access-list 120 permit ospf any 10.0.1.0 0.0.0.255access-list 120 permit ospf any host 224.0.0.5access-list 120 permit ospf any host 224.0.0.6access-list 120 permit eigrp any 10.0.1.0 0.0.0.255access-list 120 permit eigrp any host 224.0.0.10access-list 121 permit tcp 10.0.2.0 0.0.0.255 10.0.1.0 0.0.0.255 eq telnetaccess-list 121 permit tcp 10.0.2.0 0.0.0.255 eq telnet 10.0.1.0 0.0.0.255 establishedaccess-list 121 permit tcp 10.0.2.0 0.0.0.255 10.0.1.0 0.0.0.255 eq 22access-list 121 permit tcp 10.0.2.0 0.0.0.255 eq 22 10.0.1.0 0.0.0.255 establishedaccess-list 121 permit udp 10.0.2.0 0.0.0.255 10.0.1.0 0.0.0.255 eq snmpaccess-list 121 permit tcp 10.0.2.0 0.0.0.255 10.0.1.0 0.0.0.255 eq wwwaccess-list 121 permit udp 10.0.2.0 0.0.0.255 10.0.1.0 0.0.0.255 eq 443access-list 121 permit tcp 10.0.2.0 0.0.0.255 10.0.1.0 0.0.0.255 eq ftpaccess-list 121 permit tcp 10.0.2.0 0.0.0.255 10.0.1.0 0.0.0.255 eq ftp-dataaccess-list 121 permit udp 10.0.2.0 0.0.0.255 10.0.1.0 0.0.0.255 eq syslogaccess-list 121 permit udp 10.0.3.0 0.0.0.255 eq domain 10.0.1.0 0.0.0.255access-list 121 permit udp 10.0.4.0 0.0.0.255 10.0.1.0 0.0.0.255 eq ntpaccess-list 122 permit icmp any 10.0.1.0 0.0.0.255 echoaccess-list 122 permit icmp any 10.0.1.0 0.0.0.255 echo-replyaccess-list 122 permit icmp any 10.0.1.0 0.0.0.255 ttl-exceededaccess-list 122 permit icmp any 10.0.1.0 0.0.0.255 packet-too-bigaccess-list 122 permit icmp any 10.0.1.0 0.0.0.255 port-unreachableaccess-list 122 permit icmp any 10.0.1.0 0.0.0.255 unreachableaccess-list 122 permit pim any anyaccess-list 122 permit udp any any eq pim-auto-rpaccess-list 122 permit igmp any anyaccess-list 122 permit gre any anyaccess-list 123 permit icmp any any fragmentsaccess-list 123 permit udp any any fragmentsaccess-list 123 permit tcp any any fragmentsaccess-list 123 permit ip any any fragmentsaccess-list 123 permit udp any any eq 1434access-list 123 permit tcp any any eq 639 rstaccess-list 123 permit tcp any any eq bgp rstaccess-list 124 permit tcp any anyaccess-list 124 permit udp any anyaccess-list 124 permit icmp any anyaccess-list 124 permit ip any any!control-plane service-policy input RTR_CoPP!If you examine the CoPP policy in detail you will notice that all ICMP destined to the control plane is limited to 50000bps as shown below. It can bust up to 5000bps and if it conforms to the policy it is transmited, if it exceeds the policy the ICMP is dropped.
class Catch-All-IP police 50000 5000 5000 conform-action transmit exceed-action dropWith this in mind you should always use trace route for its intended purpose which is to determine the route traffic takes when traversing the transit path and that latency shown on the per hop probe basis is to be taken with at grain of salt when traversing public devices.
The intended purpose of the 3 probe count is to determine if the traffic traverses multiple routed paths due to route engineering, not to determine the latency 3 times.
I will conclude this blog with the pathping command. This command found on windows is a command similar to traceroute but it combines traceroute with ping to give you a better understanding of latency in the transit path.
Pathping works first by doing a traceroute to the destination then it uses ICMP to ping each hop in the transit path 100 times. This is used to verify latency between the source and destination via icmp echo per each hop. But remember what I said earlier, you cannot rely ICMP when public devices are involved. So you can run into cases where you see ICMP pings destined to one hop in the transit drop 40% of the traffic whereas the next hop has 100% success rate. This is due to CoPP.
Pathping in general is a much better tool to diagnosis latency from a specific source to destination with a relative degree of accuracy. Note that I said Relative, this is because latency is ALWAYS relative to your location on the network.
Shown below is an example of pathping in the works;
C:\>pathping www.google.com -nTracing route to www.google.com [74.125.225.116]over a maximum of 30 hops: 0 10.100.38.162 1 10.100.38.2 2 209.51.231.145 3 64.65.234.204 4 64.69.98.171 5 64.69.99.238 6 165.121.238.178 7 64.214.141.253 8 67.16.132.174 9 72.14.218.13 10 72.14.238.232 11 72.14.236.206 12 216.239.46.215 13 72.14.237.132 14 209.85.240.150 15 74.125.225.116Computing statistics for 375 seconds... Source to Here This Node/LinkHop RTT Lost/Sent = Pct Lost/Sent = Pct Address 0 10.100.38.162 0/ 100 = 0% | 1 1ms 0/ 100 = 0% 0/ 100 = 0% 10.100.38.2 0/ 100 = 0% | 2 0ms 0/ 100 = 0% 0/ 100 = 0% 209.51.231.145 0/ 100 = 0% | 3 4ms 0/ 100 = 0% 0/ 100 = 0% 64.65.234.204 0/ 100 = 0% | 4 6ms 0/ 100 = 0% 0/ 100 = 0% 64.69.98.171 0/ 100 = 0% | 5 22ms 0/ 100 = 0% 0/ 100 = 0% 64.69.99.238 0/ 100 = 0% | 6 10ms 0/ 100 = 0% 0/ 100 = 0% 165.121.238.178 0/ 100 = 0% | 7 34ms 0/ 100 = 0% 0/ 100 = 0% 64.214.141.253 0/ 100 = 0% | 8 37ms 0/ 100 = 0% 0/ 100 = 0% 67.16.132.174 0/ 100 = 0% | 9 35ms 0/ 100 = 0% 0/ 100 = 0% 72.14.218.13 0/ 100 = 0% | 10 --- 100/ 100 =100% 100/ 100 =100% 72.14.238.232 0/ 100 = 0% | 11 --- 100/ 100 =100% 100/ 100 =100% 72.14.236.206 0/ 100 = 0% | 12 --- 100/ 100 =100% 100/ 100 =100% 216.239.46.215 0/ 100 = 0% | 13 --- 100/ 100 =100% 100/ 100 =100% 72.14.237.132 0/ 100 = 0% | 14 --- 100/ 100 =100% 100/ 100 =100% 209.85.240.150 0/ 100 = 0% | 15 36ms 0/ 100 = 0% 0/ 100 = 0% 74.125.225.116Trace complete.C:\>As you can see from the pathping shown above there are some hops in the transit path that completely drop ICMP. You can also notice that the latency to hop 5 is higher then the latency is to hop 6. This shows that either Control Plane Policing is used on 64.69.99.238 or the process utilization on hop 5 is relatively higher.
You should know that there are other tools out there that are extremely useful when trying to diagnosis latency related problems. Most of these tools rely on ICMP and your decision to trust them is based on your understanding the transit path. One of these tools being Ping Plotter. There are several useful tools included in the Solarwinds Engineers Toolset however this toolset is extremely expensive. You can download a trial and check it out at Solarwinds Engineers Toolset
The most accurate tools depend on TCP however since TCP is a connection oriented protocol, both the source and destination must be willing to participate in the testing. Some tools are hardware based such as the Fluke Network EtherScope which cost several thousand dollars.
So in conclusion, your decision to trust and use data from ICMP based troubleshooting should be based on your relative understanding of the transit path. You should never take a traceroute that has high latency on it and say its a network issue just because hope 7 has latency greater then 250ms. This is no different the a doctor telling you your spleen is the result of your headaches without factual basis.
If you do not have clear factual data when diagnosing a problem and you blame the network because of a traceroute, you may very well be completely missing the root cause of the problem. Think of it as getting tunneled vision when sh!t hits the fan and management is expecting answers and the first thing you notice is high latency on a traceroute. With out completely understanding traceroute you may be fixating on an issue that is really not an issue at all.
Cisco ASA Pre-8.3 NAT & Post-8.3 NAT
When Cisco made changes to the Cisco ASA software in 8.3, it completely shook the ASA engineering community. Major changes overhauled the operating system in how the Cisco ASA handles Network Address Translation.
Even today, these changes still surprise people when upgrading from 8.2 to 8.3 or later and many people have a hard time understanding these changes. However, these changes are actually a good thing as it gives you more granular control over the NAT function(s) that your Cisco ASA performs.
I have compiled a list of differences as shown below to help you understand the configurational differences between pre-v8.3 and post-v8.3 NAT configurations.
First we will start with STATIC NAT which is translation from one IP Address on the outside interface (203.0.113.20) to an IP Address on the inside interface (10.1.1.6)
Regular Static NAT 8.2 & Earlier
static (inside,outside) 203.0.113.20 10.1.1.6 netmask 255.255.255.255Regular Static NAT 8.3 & Later
object network obj-10.1.1.6 host 10.1.1.6 nat (inside,outside) static 203.0.113.20Next up is the Static PAT where we translate port 80 on the outside interface IP Address of 203.0.113.20 to inside IP 10.1.1.15 port 8080.
Regular Static PAT 8.2 & Earlier
static (inside,outside) tcp 203.0.113.20 80 10.1.1.15 8080 netmask 255.255.255.255Regular Static PAT 8.3 & Later
object network obj-10.1.1.15 host 10.1.1.16 nat (inside,outside) static 203.0.113.20 service tcp 8080 wwwNow we’ll take a look at Static Policy NAT where if the host IP Address 10.1.2.3 attempting to get to the subnet 10.75.7.0/27 gets NAT’d to 192.168.100.100 on the outside interface.
Static Policy NAT 8.2 & Earlier
access-list NET1 permit ip host 10.1.2.3 10.75.7.0 255.255.255.224!static (inside,outside) 192.168.100.100 access-list NET1Static Policy NAT 8.3 & Later
object network obj-10.1.2.3 host 10.1.2.3object network obj-192.168.100.100 host 192.168.100.100object network obj-10.75.7.0 subnet 10.75.7.0 255.255.255.224nat (inside,outside) source static obj-10.1.2.3 obj-192.168.100.100 destination static obj-10.75.7.0 obj-10.75.7.0CCIE Poaching and Sniping…
The CCIE Certification is an extremely sought after certification not just from an engineering perspective but also a Company perspective. In order for any company to become a Silver or Gold Certified Partner with Cisco, they must have 2 CCIE’s full time for Silver and 4 CCIE’s full time for Gold. This requirement cannot be waived and this makes the CCIE valuable to companies because when companies have Silver or Gold partnership status they get certain benefits from Cisco such as discounts, access and etc…
If you just passed the CCIE Lab exam there are several things you should know prior to jumping into the job market looking for that big fish job offer. Many people go after the CCIE certification for the six digit paycheck, some people do it for personal goals and the big challenge and some people just do it just because they can. In any case there are two terms that you should know as a CCIE prior to job hunting and you should also know how to protect yourself from companies that want to leech on your talent and asset(s).
The Anti-Poaching clauses that Cisco have implemented into their policy causes some unintended problems in the industry when it comes to talent acquisition. Cisco has put in a clause to the CCIE agreement and Partnership agreement that gives companies the ability to continue to use your CCIE number for partnership status for 12 months after you leave a company. This clause is vague and does not specify any reasoning for your departure.
So with all that said lets take a look at the two most important things you should know as a CCIE when job hunting.
What is CCIE Poaching?CCIE poaching is an industry term used to define the method at which companies attempt to recruit CCIE engineers from other Cisco Certified Silver and Gold Partners with the intent of becoming a Cisco Certified Silver or Gold partner themselves. The Anti-Poaching clause was added to the partnership agreement stating that companies had the right to use the CCIE number of any engineer departing the company for a period of 12 months.
This type of language makes it impossible for companies to recruit CCIE’s from other companies for the sole purpose of the CCIE number. However, it does not prevent the recruiting of talent but it prevent the use of the CCIE number in a partnership status.
As a CCIE, the big part of getting a job is because you have the CCIE and companies want your CCIE number to use for partnership eligibility. While this is not always the case, its typically the case when it comes to companies paying extremely big bucks for a CCIE because its something that need, not because its something they want.
Some companies hire CCIE’s just because they need them to become a silver or gold partner. Silver and Gold Cisco Certified Partnership gives the company several benefits such as discounts. If a company can hire four CCIE’s at 600k a year and they save 10 million a year in hardware/software cost because of the partnership than it is stupid not to hire them right? The company would make money just by employing those people and having them sit on their ass watching television but of course managers would never allow that to happen because of the greedy nature of humans in general. Ultimately the company gets a huge deal by having the Cisco Partnership and the expert level talent at their disposal while effectively not paying a penny if they balance their business right.
What is CCIE Sniping?CCIE Sniping is a term loosely used in the industry when it comes to the involuntary termination of a CCIE. Because of the anti-poaching clause in the Partnership Agreement, there is nothing to protect the engineer from malpractice from companies who are greedy and just want to get a leg up in the industry.
So what is to prevent a company from hiring a freshly minted CCIE and using their CCIE number for the silver or gold partnership status, only to let them go after the partnership status as been established? Because of the Cisco agreement, the newly minted CCIE cannot use his CCIE number at another company for another 12 months thus this puts the CCIE in a bad position as it prevents him from obtaining any jobs that REQUIRE the CCIE certification. Indirectly this method devalues the CCIE certification and it’s the fastest way to piss off a CCIE.
It is because of this industry malpractice, you must protect yourself from greedy snipers.
IE Sniping is not a common practice in the United States however it does happen on occasion. This type of business malpractice is more common in developing nations. In any case you need to protect your digits.
How to protect your digits!If you have recently passed the CCIE Lab exam or you have the CCIE number that has never been registered with a partner then prior to accepting a job you need to ensure that there is protection in the employment agreement for your CCIE number.
For example, the easiest way to protect yourself is to ensure that the employment agreement states that the company will relinquish the use of the CCIE number for partnership status in the event of a non-performance related involuntary termination. This is the safest way to ensure that you’re number is not at risk if the company decides to terminate you because you don’t like the color blue or because they just want to save money. If this type of agreement does not exist than your CCIE number can be tied up for 12 months regardless of the termination reason.
You should note that this does NOT protect you if you are terminated for performance related reasons. Performance related termination is an extremely fine line and its best to have performance based termination defined in the employment agreement.
For example, just because you’re late to work, does not mean that you’re not meeting performance requirements. As long as you complete all work and achieve all work related goals on time than legally speaking you have met the conditions of the agreement.
This type of agreement also protects you against non-performance related terminations. For example you’re terminated for inappropriate language, sexual harassment, stealing, so on and so forth. Rather or not the company policy was the reason for your termination, it is not a valid reason for continued use of the CCIE number as set forth in the employment agreement based strictly on performance therefore the company would be legally required to relinquish the use of the CCIE number. While I would advise you never to break company policy because that is just common sense, there are many cases where people have said something that was offensive to others that could be classified as sexual harassment and could lead to termination. For example, telling a racist or dirty joke at work.
You would not want your company to terminate your position because of your performance and not give you a warning to correct your performance related issues. Before accepting any position that requires the CCIE certification, you should make sure that any performance based termination requires a 30 day written warning prior to termination in the employment agreement so you have the ability to correct your problems and meet performance expectations. If the company terminates you without warning because of performance related reasons then the company must immediately relinquish your CCIE number from the partnership. This plugs up the loophole that exist by the
You must look at your employment agreement as a legal document and as such you must protect your own ass as much as the next guy.
The safest way to protect your CCIE number is to require that upon termination, voluntary or involuntary, the Company is required to relinquish your CCIE number from any partnership status. This however may make it harder to get a job requiring the the risk to the employer is higher and it indirectly shows that you’re more concerned about yourself then the job opportunity however this is your decision and this should be negotiated at the time of the employment offer and prior to signing the employment agreement with any company.
Opengear and the Stub Lab
The Stub Lab is a fully functional Cisco Lab provided as a free service to the community by Free CCNA Workbook. All you need to do to gain access to a top notch Cisco Lab that has ISR’s, Multi-layer switches, Cisco ASA’s and more is sign up at the Stub Lab Schedule Portal and reserve lab time. When your lab time comes you can log into the lab devices remotely using Telnet and have access to real Cisco hardware to perform educational labs on for the purposes of obtaining Cisco certifications.
For years though Free CCNA Workbook has received countless emails requesting information on how the Stub Lab operates. How does Free CCNA Workbook provide free lab time? How does the Schedule Portal work? How do you detect abusive commands and kick users from the lab? This blog entry is going to be very helpful for anyone looking to build their own publicly accessible Cisco Lab.
This blog entry will address all the questions previously asked regarding how all this cool stuff works… So lets get started shall we?
Stub Lab Schedule PortalFirst, lets start off with the Schedule Portal. The Stub Lab Schedule Portal is a custom designed PHP application that runs on top of RedHat 6 and integrates with MySQL and NET-SNMP. From the user perspective the schedule portal allows for users to register and set their timezone where they can then reserve lab time on the Stub Lab. When a user schedules a lab session the schedule portal then places that information into the back-end database for the schedule portal but also modifies the Free RADIUS database which also resides in MySQL.
Free RADIUS Controls the user authentication for the Stub Lab console server which is an OpenGear IM4216. Free RADIUS stores authentication information regarding the username, SHA1 password hash, date and time in which the lab session is permitted.
The back-end of the Schedule Portal, the Admin portal allows for more administrative control over the Lab operations. The Admin portal allows for staff members to control power for the device via SNMP, modify and/or delete schedule lab sessions, disable and/or delete user accounts and post announcements visible to everyone who logs into the portal.
Unfortunately Stub Lab Schedule Portal is a proprietary software development and is not available for free download.
Opengear Console Server OverviewThe Opengear IM4216 is the bread and butter of the Stub Lab. This console server manufactured by Opengear makes everything possible. One of the biggest limitations of the lab is to ensure that individuals that use the lab cannot perform specific commands that could be harmful to the lab such as format flash: or delete flash:/image.bin. Commands like this would make devices inoperable for other users and would require manual intervention to fix the problems created by users with malicious intent.
While the Stub Lab is free for anyone to use, there are individuals out there with malicious intent that want to ruin the lab experience for everyone else.
The Opengear Console server gives the ability to detect specific serial string patterns received by the Lab device and when received they will execute a specific function such as disconnect the user and run a script which will break the abusive command using the CTRL+C function then powering off the device.
Because the Opengear console server is based on linux, the console server can perform cron jobs which ultimately can execute scripts. From the Stub Lab’s perspective, a Cron job is executed once every 3 hours to disconnect all users and power off all devices to conserve electricity. Another cron job is executed 10 minutes before the end of each session warning users that are currently logged into the devices that the lab session will end in 10 minutes.
Another unique feature of the Opengear console server platform is the ability to control APC smart PDU’s which control the power to each lab device. This gives users of the lab the ability to power on and off devices remotely to perform password-recovery and any other lab that requires a power cycle. Once you log into the lab device, you can access the power menu by typing ~p. This will bring up a menu that looks like this;
Power Commands: O - Power ON P - Power OFF R - Power cycle off then on again s - Show current power status . - Exit power menu ? - Show this message[R1] Power >From this menu you can power control power to each lab device individually. To exit the power menu you would type period (.) and press enter.
Stub Lab Console Server Deep DiveSo now that you have a basic understanding of how the Stub Lab works, lets take a deep dive into the operations of the Opengear console server. In this section we will discuss how you can setup your very own Opengear Console Server to provide free public access to users around the world.
First we’ll start off with Serial Port and RPC Configuration then Bash scripting and followed by Auto-Response.
Serial Port and Remote Power Control (RPC) ConfigurationAfter you have configured the basics of the Opengear Console Server such as the IP Address, authentication, time settings (NTP) and so on you will the need to configure the Serial Ports. The Serial Port configuration gives you the ability to specify a Name on a per port basis and the operating parameters of each port such as the baud rate, data bits, parity, stop bits and flow control. Other parameters that can be configured in this section is the serial port operational mode. There are 5 modes that the port can operate in. The first “Console Server Mode” which is what you’ll use to access Lab devices allows for users to remotely access the console ports of a connected device remotely via TCP/IP using Telnet and/or SSH.
Device Mode, can be used to specify exactly what type of device is connected to the console server to allow for more advanced controls of the connected device such as a Battery Backup (USP), or Smart PDU, known as an RPC (Remote Power Controller) or an environmental sensor used to detect and report environmental statistics such as humidity and temperature.
SDT (Secure Desktop Tunneling) Mode if a feature developed by Opengear which allows for Secure desktop tunneling of VNC or RDP through the use of an SSH Tunnel over the internet to authenticated endpoints.
Terminal Server Mode gives the console server the ability to interconnect to a terminal server using the configured terminal type.
And lastly, Serial Bridged mode is used to bridge serial over TCP/IP and is commonly used in hospital environments where specialized label printers operate using Serial ports and not parallel or native TCP/IP.
When building your own Stub Lab you will be using the Console Server mode. If you have an APC 7901 or 7902 you can then use the Console Server to control the power to this device by configuring the RPC via SNMP in Network Host(s) first. Once you have defined the RPC Network Host you must then define the RPC Connection settings. This is where you’ll name the RPC and provide the SNMP community string and label the outlets. Shown below is a screenshot of the Network Host configuration page.
Once you have the RPC defined and configured you must then define each “Managed Device”. This is where you will pair up the Serial Port with the RPC Outlet. You must doe this for every device in the lab. Once completed you then have the ability to enable the “Power Menu” function on the Serial Port configuration page.
After all the configuration has been completed properly you’ll be able to control the power of each device independently through Telnet and/or SSH.
Auto-Response ConfigurationOne of the obstacles in the way of providing free access to a Cisco lab is the ability to prevent people from executing harmful commands such as format flash. Fortunately, Opengear gives you the ability to prevent users from executing commands using the “Auto-Response” function. The Auto-Response function is very powerful as in it has the ability to recognize serial pattern strings and execute a specific action once the string has been detected.
The Stub Lab makes use of this function by preventing users from executing abusive commands. When a user executes a command that is harmful to the lab environment, the console server will automatically disconnect the user immediately before they can continue to execute the command. After the user has been disconnected it will then execute a bash script which will be discussed in the next section which sends the CTRL+C command to the port which the user executed the abusive command on and finally powering off the device.
When it comes to the Stub Lab, the Auto-Repsonse is used to detect Rx (Receive data) serial strings and execute a function upon being triggered. Because all abusive commands when executed will prompt you for additional confirmation, we can use this to our advantage and trigger an action when detected by the console server. Below is a list of (Rx) strings that are used.
Abusive CommandRx StringDebug AllThis may severely impact network performance.Format OperationFormat operation mayErase Flash:Erasing the flash filesyIOS Upgrade AttemptSource filename \[.*\]:Delete BIN FileDelete filename \[.*\.bin\]\?Save File to FlashDestination filename \[(?!(running|startup)-config\]).*\?ASA Erase FlashErase operation may take a while. Continue\?No SVC Password RecoveryExecuting this command will disable password recWhen the Auto-Response is triggered based on Rx string detection it will execute a trigger function. On the stub lab, trigger executes a customized bash script that is provided in the next section. However the configuration of the trigger must pass an Argument to the bash script so the bash script knows which port the abusive command was executed on and can take action on that specific port only.
Provided below is a screenshot;
The Argument $AR_DEV_REF is a very special argument which passes the port configuration label into the bash script. You can of course pass your own defined arguments but this one is used to define which port the auto-response was triggered on.
Bash ScriptingBecause the Opengear console servers are built on top of Linux, you have the ability to execute Bash scripts which make the console extremely powerful. The Stub Lab makes use bash scripting to help with the management of the lab. Provided below is a summary of each script and the script its self to help you build your own stub lab.
The following script “abusive_cmd.sh” is used by the Auto-Response to send 3x CTRL+C’s to break any abusive command executed. This script also powers off the device.
#!/bin/sh#The $AR_DEV_REF variable feeds the #port info from the config location #config.ports.port# into $1#Strip down to just the port number get_port=$(echo $1 | sed 's/[^0-9]*//g')#Pad the 1-9 ports with a 0 number=`printf "%02d" $get_port`#Execute three CTRL+C to Cancel Command and display message.printf "\003\003" > /dev/port$number#Power Off Device After Command Canceledpmpower -r 172.16.28.8 -c private -o $get_port offUp next is the “10min_warning.sh” Script which executes a set of echo commands to all ports in the Bash array warning the user the lab session will end in 10 minutes;
#!/bin/bash### WHICH PORTS TO ECHO 10 MINUTE WARNING ON ###PORTS=( 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 );### EXECUTE LOOP TO ECHO WARNING ON EACH PORT IN $PORTS ARRAY ###for PORT in "${PORTS[@]}"doecho -en " ################################################### #### YOUR LAB SESSION WILL END IN 10 MINUTES #### #### PLEASE ERASE ALL LAB DEVICE CONFIGS #### ###################################################" > /dev/port$PORTdoneThe next script used is the “end_session.sh” script which will disconnect all current telnet and ssh sessions on the lab devices defined in the bash array.
#!/bin/bash### DEFINE WHICH PORTS TO EXECUTE SESSION TERMINATION ON ###PORTS=( 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 17 )### EXECUTE SESSION TERMINATION LOOP ON EACH PORT DEFIEND IN $PORTS ARRAY ###for PORT in "${PORTS[@]}"do### Find process ID(s) of pmshell for port, if any ###procs=`ps aux | grep pmshell | grep port$PORT | awk '{print $1}'`### KILL PIDS ###for pid in $procsdokill $piddonedone### EXECUTE KILL POWER SCRIPT ###/bin/sh /etc/config/scripts/kill_power.shThe “kill_power.sh” script which is used by the “end_session.sh” script makes use of the pmpower command on the Opengear to send an SNMP SET query to the RPC to turn off the outlet(s).
#!/bin/shpmpower -r 172.16.28.8 -c private -o 1 offpmpower -r 172.16.28.8 -c private -o 2 offpmpower -r 172.16.28.8 -c private -o 3 offpmpower -r 172.16.28.8 -c private -o 4 offpmpower -r 172.16.28.8 -c private -o 5 offpmpower -r 172.16.28.8 -c private -o 6 offpmpower -r 172.16.28.8 -c private -o 7 offpmpower -r 172.16.28.8 -c private -o 8 offpmpower -r 172.16.28.8 -c private -o 9 offpmpower -r 172.16.28.8 -c private -o 10 offpmpower -r 172.16.28.8 -c private -o 11 offpmpower -r 172.16.28.8 -c private -o 12 offpmpower -r 172.16.28.8 -c private -o 13 offpmpower -r 172.16.28.8 -c private -o 14 offpmpower -r 172.16.28.8 -c private -o 15 offpmpower -r 172.16.28.8 -c private -o 16 offThe next script which is executed automatically by the opengear console server when a user successfully authenticates to a lab device is the “/etc/config/pmshell-start.sh” script. This script is designed to prevent ROOT access to the console lines and it will also display a welcome msg and login banner to authenticated users;
#!/bin/sh################################################################## ## OPENGEAR CONSOLE SERVER (ver 3.10.0) PMSHELL START SCRIPT ## Published by: Matthew George, Free CCNA Workbook ## #################################################################### DECLARE VARIABLES ###PORT="$1"USER="$2"LABEL=$(config -g config.ports.port$PORT.label | cut -f2- -d' ')### PROHIBIT ROOT USER AUTHENTICATION ON CONSOLE PORTS ###if [ "$USER" == "root" ]; thenecho "Permission denied for Super User"exit 1fi### DISPLAY SWITCH BANNER PASSWORD WARNING AFTER AUTHENTICATION ###if [[ $PORT = [6-9] ]]; thenecho ""echo ""echo "####################################################################"echo "# #"echo "# WARNING: LEAVING PASSWORDS ON LAB SWITCHES WILL RESULT IN A BAN #"echo "# #"echo "####################################################################"fi### DISPLAY WELCOME AFTER AUTHENTICATION ###if [ -z "$LABEL" ]; thenecho ""echo ""echo "Welcome $USER, you are connected to Port $PORT"echo ""echo ""elseecho ""echo ""echo "Welcome $USER, you are connected to Port $PORT ($LABEL)"echo ""echo ""fiAnd the last script which is not really script is the CRONTAB configuration. The Stub Lab is provides free lab access using 3 hour sessions. Every 3 hours the CRON process will execute a script. First it will execute the “/etc/config/scripts/10min_warning.sh” script informing users that they have 10 minutes left before their lab session ends. Then once the lab session ends, it will execute the “/etc/config/scripts/end_session.sh” script which will disconnect the user from all lab devices and power off the entire lab to save energy.
### SESSION 1 ###50 2 * * * /etc/config/scripts/10min_warning.sh0 3 * * * /etc/config/scripts/end_session.sh### SESSION 2 ###50 5 * * * /etc/config/scripts/10min_warning.sh0 6 * * * /etc/config/scripts/end_session.sh### SESSION 3 ###50 8 * * * /etc/config/scripts/10min_warning.sh0 9 * * * /etc/config/scripts/end_session.sh### SESSION 4 ###50 11 * * * /etc/config/scripts/10min_warning.sh0 12 * * * /etc/config/scripts/end_session.sh### SESSION 5 ###50 14 * * * /etc/config/scripts/10min_warning.sh0 15 * * * /etc/config/scripts/end_session.sh### SESSION 6 ###50 17 * * * /etc/config/scripts/10min_warning.sh0 18 * * * /etc/config/scripts/end_session.sh### SESSION 7 ###50 20 * * * /etc/config/scripts/10min_warning.sh0 21 * * * /etc/config/scripts/end_session.sh### SESSION 8 ###50 23 * * * /etc/config/scripts/10min_warning.sh0 0 * * * /etc/config/scripts/end_session.shWith the following scripts provided you should have enough to get your own Opengear Console Server configured to allow for free remote access without having to worry about people screwing up your hardware.
While some of the scripts are not perfect, they are indeed functional and work as intended. I have however made a feature request to Opengear requesting that Opengear build a function that would allow for the console server to receive a RADIUS A/V Pair such as “Session-End” = “Date/Time” which would automatically execute a script in /etc/config/scripts/ to provide the functionality of allowing for more diverse lab session management without using the CRON function.
Our Appreciation!Last but not least, Free CCNA Workbook and on the behalf of all the registered users of the Stub Lab would like to thank Opengear for sponsoring the Stub Lab by providing us with a free Opengear IM4216. Without this gracious sponsor, free access to the Stub Lab would not be possible.
If you are interested in learning more about Opengear products such as the Console Server or Infrastructure Manager, please check out Opengear Products page. If you are interested in inquiring about products for educational classroom purposes you can reach out to the Opengear VP of Sales, Todd Rychecky at 303-346-6853. Please let him know that Matthew George from Free CCNA Workbook referred you.
How to make a T1 Crossover Cable
For years people have used the old and untrusty DB60 DCE to DB60 DTE cables to simulate WAN connectivity between Cisco Routers using WIC-1T modules; and what I mean by untrusty is that these cables have a tendency to go bad after you move them around a lot in a lab environment.
While the WIC-1T cards are extremely cheap, the DB60 DTE to DB60 DCE cables can cost anywhere between $5.00 to $15.00 each which can add up really fast in a lab that has more than 5 routers.
Another common alternative is to use the WIC-1DSU-T1 which is dirt cheap and supported in the older Cisco 1700, 2600, 3600 series routers however WIC-1DSU-T1 is NOT supported in the newer ISR routers such as the 1800, 2800 and 3800 Series. For the newer routers you’ll need the second generation card known as the WIC-1DSU-T1-V2 which can also be commonly found on eBay for under $10.00
As for the WIC-1DSU-T1-V2, it will also work in the older routers with the latest Cisco IOS software.
Shown below is a picture of the WIC-1DSU-T1-V2
The difference between the WIC-1T and the 1DSU cards is that the 1DSU cards have an integrated DSU and does not require an external box to be connected via synchronous serial. The 1DSU cards can be cabled using standard unshielded twisted pair cable (Cat3, Cat5, Cat6, etc..)
Unfortunately a simple ethernet cable will not magically make the WIC-1DSU-T1’s cards talk to each other because the pin-out schematic is different than a standard patch or crossover.
You can make your very own T1 crossover for pennies on the dollar however you will need 2 RJ-45 jacks, some Cat5 cable and a crimper tool; all of which can be picked up at your local hardware store such as Lowe’s, Home Depot, Ace Hardware, etc… (If you’re in the United States).
And now for a fun fact… The most common color you will see used for T1’s in the industry is either white or yellow.
Before you start, it’s a good idea to modify the RJ-45 Jacks a little bit. Using a small flat head screw driver or pocket knife remove pins 3, 6, 7 and 8 from the jack so that it looks like this;
This way only the wires that are needed to be connected to the T1 WIC card are actually connected. Other wires in the RJ45 connector are not used for anything and this will also help you easily identify the type of cable in the future. Keep in mind this is not a mandatory requirement to make the cable, its just good practice.
Once the RJ-45 connectors have been modified you can then proceed to building the cable, one side at a time of course. In a standard T1 cable, wires 1,2 and 3,4 will be swapped at each end of the cable looking like the following;
As for the color of the wires that go into pins 3, 6, 7 and 8; these can be any color you choose. Sometimes its easier to make the cable using certain colors in different spots.
Once the cable has been successfully crimped on both ends you can test it out between two WIC-1DSU-T1’s. If the cable was made correctly the CD (Carrier Detect) LED should light up on both WIC’s. At that point you can assign IP addresses to the serial interfaces and verify IP connectivity between the two WIC’s.
Below I have provided an example verification;
R1#show interface serial0/0Serial0/0 is up, line protocol is up Hardware is PQUICC with Fractional T1 CSU/DSU Internet address is 10.255.212.1/24 MTU 1500 bytes, BW 1536 Kbit/sec, DLY 20000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation HDLC, loopback not set Keepalive set (10 sec) Last input 00:00:00, output 00:00:00, output hang never Last clearing of "show interface" counters never Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec 1651 packets input, 107752 bytes, 0 no buffer Received 605 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 1661 packets output, 109224 bytes, 0 underruns 0 output errors, 0 collisions, 7 interface resets 0 unknown protocol drops 0 output buffer failures, 0 output buffers swapped out 3 carrier transitions DCD=up DSR=up DTR=up RTS=up CTS=up R1#R1#ping 10.255.212.2Type escape sequence to abort.Sending 5, 100-byte ICMP Echos to 10.255.212.2, timeout is 2 seconds:!!!!!Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 msR1#Hope this blog was helpful ^_^ Please be sure to share!
Poll: Which WAN Interface Card (WIC) Do you prefer to use in your Cisco lab? WIC-1T WIC-2T WIC-1DSU-T1 WIC-1DSU-T1-V2View Results Loading …
WAN Technologies and Link Speed Charts
The following blog has been compiled to provide information for the different type(s) of WAN Technologies. Keep in mind that Ethernet is a fast growing WAN technology with companies moving to deploy VPLS, MPLS and TLS Circuits for WAN connectivity. As a Cisco network engineer you will still encounter legacy WAN connections.
Digital Subscriber Line (DSL) technologies are fairly common in the United States and other developing nations that provide broadband to customers over existing telephone wire infrastructure. The is however the limitation of distance which basically means the subscriber can be so far from the DSLAM (DSL Access Multiplexer) which is commonly known as the “central office”.
Common NameStandards NameDownstream RateUpstream RateExt. DownEst. UpaDSLG.992.1 G.DMT12Mbit/s1.3Mbit/s1.5MB/s166KB/saDSL+ITU G992.4/524Mbit/s1.0Mbit/s3MB/s128KB/sHDSLG.991.12.048Mbit/s2.048Mbit/s262KB/s262KB/sSHDSLG.991.22.265Mbit/s2.265Mbit/s290KB/s290KB/sVDSLG.933.152Mbit/s19.2Mbit/s6.5MB/s2MB/sVDSL2G.991.2250Mbit/s250Mbit/s21.25MB/s21.25MB/sIDSLANSI T1.601144Kbit/s144Kbit/s18KB/s18KB/sRADSL*ANTI TR598Mbit/s8Mbit/s1MB/s1MB/sNOTE: Bandwidth is shown at maximum theoretical speed using the latest standard.*RADSL – Rate-Adaptive Asymmetric Digital Subscriber Line – Adjusts the upstream/downstream bandwidth in a tradeoff.Integrated Services Digital Networks (ISDN)
ISDN Lines have seen their glory in the 90’s and is now considered a legacy technology however you may encounter this technology in third world countries where the hardware is extremely cheap to acquire and deploy.
Link TypeLink BandwidthEst. ThroughputAdditional InformationBRI128Kbit/s16KB/s2x 64Kb B Channels & 1x 16k D ChannelPRI – NA (AKA: T1)1.544Mbit/s197KB/s23x 64Kb B Channels & 1x 64k D ChannelPRI – EU (AKA: E1)2.048Mbit/s262KB/s30x 64Kb B Channels & 1x 64k D ChannelT CarriersLink TypeLink BandwidthEst. ThroughputAdditional InformationT1 (DS1)1.544Mbit/s197KB/s24 DS0 ChannelsE1 (DS1EU)2.048Mbit/s262KB/s30 Channels + 2DT1c (DS1c)3.152Mbit/s403KB/s48 ChannelsT2 (DS2)6.312Mbit/s808KB/s96 Channels (uncommon)T3 (DS3)44.736Mbit/s5.59MB/s672 ChannelsT4 (DS4)274.176Mbit/s34.3MB/s4032 ChannelsOptical CarriersLink TypeLink BandwidthEst. ThroughputAdditional InformationOC-1 / STS-151.84Mbit/s6.48MB/sOC-3 / STS-3155.52Mbit/s19.44MB/sOC-9 / STS-9*466.56Mbit/s58.32MB/sOC-12 / STS-12622.08Mbit/s77.76MB/sOC-18 / STS-18*933.12Mbit/s116.64MB/sOC-24 / STS-24*1244.16Mbit/s155.52MB/sOC-36 / STS-36*1866.24Mbit/s233.28MB/sOC-48 / STS-482488.32Mbit/s311.04MB/sUsed in USAOC-96 / STS-96*4976.64Mbit/s622.08MB/sOC-192 / STS-1929953.28Mbit/s1.24GB/sInternet BackboneOC-256 / STS-25613.271Gbit/s1.659GB/sInternet BackboneOC-768 / STS-76839.813Gbit/s4.977GB/sInternet BackboneOC-3072 / STS-3072159.252Mbit/s19.44GB/sNOTE: * Several of the listed links are not standardized and/or commonly used including 9, 18, 24 and 36.