Programming Bad Performance

Image courtesy of Wikipedia

Last week an interesting problem surfaced at work. An application engineer received reports of slow performance on a particular website and needed some help from my group to track down the source of the problem. This engineer had done some fantastic research on the problem and was able to answer almost every question we threw at him in regards to details surrounding the issue. I am going to try and run through the problem itself and the questions we asked which led us to the possible culprit. The solution to the problem was discovered a few days later and ended up surprising us as it was not even something we had considered could be the cause. Although the application engineer collected a lot of data in the form of trace logs and packet captures, my group didn’t examine any of this data. The problem was solved before we actually had to get in and look at the data ourselves. With a white board and some direct questions, we were able to point the engineer in the right direction. He did all the work.

Problem:

A URL that was Internet facing was performing very sluggish compared to others.

When did the problem start? Unknown.

Possible causes to consider:

1. Remote end of the connection
2. Internet connectivity
3. Firewall
4. Intrusion prevention sensor/Content filter/Other security hardware
5. Router/Switch/Load balancer problem on the internal network hosting the site
6. Server hosting the site
7. Web server software on server hosting the site(ie IIS,Apache)
8. Web site code (ie HTML,ASP,JScript,CSS,XML)

Troubleshooting: For the purposes of isolating the problem, we started with the remote connectivity and worked our way inward. From here on out, I am going to refer to the application engineer as Bob. That’s not his real name, but it’s a lot easier to type than “application engineer” or his actual name.

Had Bob checked into the remote side as being the source of the problem? Yes, he had. In fact he ran the same checks from other ISP’s and experienced the same result. That rules out item 1 on the list of possible causes.

Bob had a lot of additional information to add regarding this problem. First, this particular website was really a specfic URL that was problematic. Over a dozen URL’s using the same exact hostname were fine. It was just this one particular URL that was having a problem. That rules out item 2 as being the issue. Second, Bob stated that the problem was occurring on the internal network as well. That rules out items 3 and 4 from the list of possible causes. Now we’re getting somewhere. At this point, we know that we aren’t dealing with a problem isolated to the Internet. That’s actually a good thing because it’s never easy when you have to explain to people that you have no control over traffic once it leaves your network. It just comes off like you are passing the buck to non-network savvy people.

Bob added an additional piece that would vindicate the network hardware from being the culprit. He stated that the average MTU on all of the URL’s that were working great was somewhere over 1000 bytes. However, for the URL that was operating sluggish, the average MTU size was a little over 200bytes. Now the discussion goes on for a few minutes about how MTU size will affect performance and that 200byte average sizes are not good when compared to the other URL’s and their greater than 1000 MTU averages.

At this point, we know there is an MTU problem and that problem occurs on the external and internal network. Now I know that every switch this traffic is traversing on the internal network is going to allow an MTU of 1500, so I don’t think there is a piece of networking gear causing the problem. This seems like it is going to be something with the system itself. It turns out that this particular server hosting all of these URLs is one of several servers hiding behind a load balancer. I know my load balancer isn’t messing with the MTU, so I feel comfortable in ruling out item 5 as being the source of the problem.

Has Bob checked the server hosting these URLs? Bob indicates that there are 4 different servers behind the load balancer hosting these same URLs and they are all having the same problems. He tested the URL on each individual server and experienced latency. It is possible that we are dealing with a problem on all 4 servers, However, the odds of that being a sever hardware probem are very low. Considering the fact that these same servers host over a dozen more URL’s that are running with no problems, I am convinced that we can rule out item 6 as a possible culprit.

Now we are looking at the web server software or the site code itself as being the culprit. While I am by no means an expert when it comes to IIS, Apache, or other web server software, I am willing to bet that the issue is not with the web server software. My reasoning is that only 1 URL is experiencing the problem and over a dozen other URL’s are not. They are all using the same hostname, so one would expect any sort of MTU setting in the web server software, if there is one, to be the same across every URL.

At this point in the troubleshooting process, we figured it must be something in the code. Our recommendation to Bob was that he go back to the developers and have them check their code.

Bob came back several days later. He found the problem. Actually, there was no problem. The way the developers had coded this particular URL was what caused the problem. In this case, they had a bunch of really small CSS files that were used in conjunction with the URL that was problematic. The client would make the request and then it would have to grab tons of really small CSS files. Due to the small size of these files, the MTU itself was small. I suppose that small file sizes wouldn’t be too much of a problem, except in this case, there were too many files that had to be transferred. That is what was causing the latency.

In this particular case, there was nothing wrong with any infrastructure or server equipment. Everything was working as designed. If nothing else, it was a reminder that developers don’t always consider application performance over the network when designing software. They routinely get beat up for having poor security. I guess you can add poor network performance to the list as well. I think it is a generally accepted belief that programs are usually designed for low latency LAN environments, and very rarely are designed with WAN performance in mind. I shouldn’t be surprised to find a case like this in which the code wasn’t designed with network performance in mind at all.

I feel that it is also important to point out that it is fairly difficult to write code that takes all factors into consideration(ie Security, Network). Maybe the best solution is to involve the various entities during the testing of code to ensure it will perform properly. I can see how this issue would have been overlooked since it was a simple URL that was affected. Had it been an entire program that was affected, it might have been caught during testing.

Posted in efficiency, troubleshooting | Tagged , , | 11 Comments

Dealing With Knowledge Gaps

Image courtesy of www.freedigitalphotos.net

Inevitably, we are all going to come across things in our jobs that we are deficient in. Maybe we know a little about a certain topic, but we need to know more. Maybe we know absolutely nothing and need a basic introduction to the topic. Regardless, there will come a time in which we need to increase our knowledge and understanding of something in this ever growing world of networking or just IT in general.

The problem as I see it, is how I go about filling in those gaps. When you just start out in the IT world, you may not have a good methodology in which to learn about IT things. If you have been in the industry for a long time, you may already have a good system that works for you. No matter which category you fall into, the fact that you will constantly have to learn is unavoidable. There are NO exceptions to this rule. If you wish to be at the top of your game in IT from a technical standpoint, you must make a habit of constantly learning new things. Failure to do so means that your knowledge will become dated and you will drift off into obscurity working as some corporate slave in a dark and dreary cubicle. This may or may not involve working for the government. ūüôā

Now that we have established that static knowledge is a dead end, let’s look at how to ensure we are always at the top of our game. I offer you the 5 step plan. Others have 12 step programs. Maybe some have less. I only have 5. I am all about efficiency…..and my program doesn’t cost you a dime.

1. Examine your current level of knowledge. – How much do you already know about the subject in question? The answer to that question is going to dictate the kind of resources you use. Let’s use BGP for example. If you need to learn about the basics of it, there are a few good books that can handle that. There are also plenty of websites with white papers and blog posts that give a generic overview of BGP. There are some classes out there that will accomplish the same thing. However, there are quite a few books and white papers that will completely blow your mind if you don’t already have a decent understanding of BGP. The service provider side of BGP comes to mind. Enterprises and service providers use BGP in VERY different ways.

2. Find out where the information is. – For starters, you need to identify what kind of learner you are. Some of us are visual learners. Some of us are audible learners. Some of us learn by doing. Perhaps you are a mix of several different methods. Only you know what works best for you. If you need a lot of pictures and the topic is relatively mainstream, maybe a visual CBT(computer based training) course is what you need. If that is the case, I highly recommend you check out CBT Nuggets. If what you are looking for is somewhat more obscure, then I would recommend asking other people who do what you do. There are a variety of resources in which you can ask these questions like LinkedIn, forums, or Twitter. I prefer Twitter because it is a lot quicker. The only possible problem would be having enough people see the request. If you are new to Twitter, or very rarely use it, you may not have many followers who would see your message. Feel free to engage others in a substantive manner and over time your followers will grow. If all you do is tell everyone what you ate for lunch or what the weather is like in your part of the world, you probably aren’t going to get anywhere. If you absolutely refuse to use something like Twitter, then consider posting on Cisco’s forums if your issue is of a Cisco nature or networking-forum.com. There are other forums out there as well as mailing lists(NANOG comes to mind). All of the major vendors have support forums as well. Keep in mind that you may have to sift through tons of information before you finally find the information you are looking for. There is not always going to be a technical paper or book that explains exactly what you are looking for. Sometimes you have to piece it together from multiple sources. Actually, I would recommend that you use multiple sources unless it is some vendor specific thing that you can only get in one place. I have found out that you cannot trust a single source for 100% accuracy. Not that all sources are wrong, but imperfect human beings write books, white papers, and blog posts. Other imperfect human beings double check these same sources. When the content is of a technical nature, things get missed. This is especially true for the deeper technical things.

3. Execute. – You have all of the appropriate resources identified. Now you just need to get that information into your head. There are no shortcuts. While I wish I could learn kung-fu like Neo did in The Matrix, it isn’t going to happen. You have to put in the time required to absorb all of that information. Sometimes it can be done in a matter of minutes. Sometimes it takes weeks.

4. Ignore any distractions. – In the course of your learning, you are bound to come across something else that is interesting or neat. Resist the temptation to get sidetracked and stay focused on the main thing you are trying to learn. If you want to go back at another time and research the other items that pop up, then make a note of them. By focusing on the main thing you are trying to learn, you have a better chance of retaining information then if you start going in 100 different directions with every new thing that appears.

5. Allow the information to digest. – Sometimes it helps to simply think about things. Just go over it in your head. I tend to do this in conjunction with step 3. If I need to absorb a large amount of information, I like to take it in pieces and digest it little by little. By stopping to sort things out in your head, you can really come to terms with what makes sense and what doesn’t. I am very thankful my current employer allows me the freedom to do this. While it may look like I am spacing out on any given day in my cubicle, lots of times I am just thinking about something I just read or watched. It’s my way of performing a “write memory” on my brain. One of the other things I will do is drive to and from work in complete silence. That really helps because all I have to focus on is not crashing the car, which is relatively simple.

**Note – When asking others about a certain technology or product, do yourself a favor and research it first. Try and figure some things out on your own. This isn’t so much a problem with people who have been in the industry for a number of years as it is with those who have only been in IT for a few years or less. It’s not that people don’t want to answer the question. There will always be someone who will just blurt out an answer. The issue with asking without having done any research on your own is that you miss out on a great opportunity to develop your own research methods. There’s a reason that lmgtfy.com was created and is often quoted on Twitter. It has been my experience that those who last in IT are the ones that only need a nudge in the right direction. They don’t want their hand held. They just want a sanity check every now and then. The people who never want to put in the time or effort to figure something out and habitually want you to solve their problems are the ones that won’t make it in the long run. Well, they might have a job, but they won’t be anywhere near what they could be if they put forth some effort.

I am not going to make the bold claim that the 5 steps I laid out will work for everyone. They work for me when I follow them, and I don’t always follow them. I find the instances in which I have tried to cram something new into my head without following these steps ends badly. I forget something and have to start all over again. When I take the time to really dig into something and not rush it, it tends to stay with me at least from a conceptual point of view.

Posted in career, efficiency, learning, training | Tagged , , , , | 2 Comments

Chasing the “Ah-ha!” Moments

Image courtesy of Wikipedia

Whenever I talk to people who are just getting started in networking, there’s a part of me that wishes I was in their shoes. I say that because I know several of the things they are going to learn or figure out in the next couple of years and I remember having to go through the same process. Before I understood variable length subnet masks(VLSM), the numbers in the subnet mask field of a workstation’s TCP/IP settings didn’t really mean a whole lot to me. If someone used slash notation(ie /24, /16, /27), I had no idea what that meant. Like a lot of people, I relied on someone to tell me what the subnet mask was. However, once I learned about VLSM, it was as if a whole new world opened up. That was one of my absolute favorite “Ah-ha!” moments. You’ve had those yourself haven’t you? It is the point in time in which a certain technical concept just clicks in your head. You go from not really understanding it, to comprehending it. In fact, it’s almost as if that concept is only represented in binary inside your head. You go from a 0 to a 1 with no in between.

As you progress along in networking, more and more of these “Ah-ha” moments come. Unfortunately, over time they become fewer and fewer. That’s not to say that they go away completely. They don’t. They are just harder to come by. I’ve found that I am able to keep a steady stream of these “Ah-ha” moments coming as long as I look at technology without taking anything for granted. What I mean by that is that I don’t assume anything when it comes to trying to understand a protocol or technology. What I “think” I know might actually be wrong. My understanding might only be partial. I have to continually ask “why/what/how/when/where” when dealing with technology.

Let me give you a personal example. I have known for many years that a T-1 is 1.544Mbps in terms of bandwidth. It is comprised of 24 64kb channels. The only problem is that 24×64,000 is 1536000 and not 1544000. Oops. Where did the other 8k go? To further drive this home, a “show interface” on a serial link that is configured as a full T-1 shows the interface bandwidth to be 1536kbps. Why the discrepancy? I could have just moved on and ignored the reason behind the discrepancy. However, by researching the issue and figuring out what the issue with this discrepancy was, I learned a whole lot more about T-1’s. I learned how alarms over the circuit get propagated. I learned what the extended super frame(ESF) actually was. In other words, had I not been curious as to why the math didn’t add up when it came to T-1 bandwidth, I would be far more deficient in the inner workings of the T-1.

In the spirit of chasing the “Ah-ha” moments, take a look at the 4 questions below. Go find the answers if you don’t already know them.

1. Why is MPLS faster than conventional IP based routing?
2. What are the differences between a multi-layer switch and a router?
3. Why do you need different antennas for wireless access points and where would you use each antenna type? Sure, this is rather open ended, but what I am getting at is the radiation pattern of each antenna.
4. How does traceroute really work? Not just the TTL mechanics, but look at the various ICMP type codes as well.

Can you remember the last “Ah-ha” moment you had? If not, why? If so, does it make you want to go out and find more of those moments?

Posted in career, learning, training | Tagged , , , | 9 Comments

The Myths of IT – Part 2

Unicorn - Stolen/Borrowed from http://katemckinnon.files.wordpress.com/2008/07/unicorn.jpg

See my previous post here for the first 3 “Myths of IT”.

4. More bandwidth will solve all your problems. – Bandwidth isn’t always the cure for traffic problems. You can add bandwidth all day long to a circuit, but if the problem is latency, you are wasting your money. One of those pesky laws of physics is that light can only travel so fast. Sometimes a better solution is to take care of the latency. That involves really understanding what the problem is before implementing solutions. An ever growing amount of traffic on the wires these days is of the real-time nature(ie voice and video) so it is going to be especially critical in the coming years to understand the latency needs of your customers. Many larger corporations and content providers deal with this problem by using multiple data centers across the country or world.

You can extend this myth to the wireless side as well. Simply adding more AP’s to a network will not necessarily make things better. In some cases, it will make things worse! As with anything, before you can provide a working solution, you have to fully understand the problem.

5. Everyone in IT must have business skills. – No. No. No. No. No. I can’t say no enough. I am paid to perform a highly technical function. In the course of providing that function to my employer, I have to deal with associated costs. I do have to understand what the capital expenses and operational expenses are, but other than that, I don’t care about the financial end. MY job is to provide solutions. I am not an accountant. I do not look at spreadsheets all day and try to figure out how to align my technology solutions with the vision of the various MBA’s and marketing people running the company. IT exists to provide solutions. Those solutions are based on requirements given from the business side. Tell me what you want to do and I will design a solution to implement that. It is hard enough to stay current with all of the various technology vendors out there. The last thing I need is to worry about how it affects the bottom line. I am reminded of an old joke when it comes to technology.

Good. Fast. Cheap. Pick 2. You can’t have all 3.

Are there people within the average IT department that NEED business skills? Yes. They’re called managers. Does someone providing front line support on a help desk need business skills? Nope. How about that engineer doing wireless installs? Nope. Maybe that systems architect designing your virtual infrastructure? Nope. Again, tell me what you want to do from the business perspective. I will provide you a solution. You may not like the solution because it costs a lot of money, but then again, I am not designing a dress(No offense fashion fans!). In the case of the network side of things, I am designing a conduit to move information around. That doesn’t come cheap. I think the problem behind the whole “IT people need business skills” mantra is that the business side of the house doesn’t always articulate what it is they are trying to accomplish. Additionally, the IT side of the house doesn’t ask all the appropriate questions to extract the information they need to design a proper solution. I keep reading about these IT executives who claim they only want to hire IT people with business skills. Good luck with that. People with business skills continually run companies into the ground. Are you sure you want your IT department to contribute to that? It’s better to let them take care of the nerdy things.

6. All IT people can fix your computer. – There was a time when I was somewhat in tune with computers. I knew a little about graphics cards, sound cards, memory, etc. That was back when I was running DOS and really just wanted to play computer games. Those days are long gone. I know next to nothing about PC’s and laptops. That goes for the operating systems as well. I work on all the gear that facilitates communication between the PC’s(ie routers, switches). I don’t have enough capacity in my brain to memorize how many cores the CPU should have and which graphics chipset will give you the best performance for World of Warcraft. I am not alone. Many of my fellow IT professionals are in the same boat. We take phone calls from family and friends on a regular basis. They need help with this or that thing on their home computer. Maybe it is infected with spyware or a virus. Maybe they need to upgrade their 10 year old computer and need our help figuring out which new one to buy. Guess what? I’m winging it. I’m taking a semi-educated guess on what computer you need to buy or how to fix your existing computer that runs like an old 286. Odds are I am using Google to figure out what to do.

7. IT people change jobs all the time because all they care about is making more money. – As with any career field, there is a certain percentage of people that will constantly change jobs to make more money. However, IT people tend to change jobs for different reasons. First, you have to understand that IT is a profession in which career development is generally up to the individual. The more they learn and the more they get exposed to, the better the options. Quite a few jobs in the IT world can be grown out of. After a year or two in certain positions, there is nothing more to learn. Nothing new to experience. Your job is simply to serve as a caretaker of the network and look at logs all day long. Most people can’t deal with that. They need something new and exciting. They need opportunities to grow their technical skillset. This happens a lot faster than other career fields. At some point in your career, you learn enough and have enough experience to get that coveted position at company XYZ that you have been looking for. Generally, it takes several years. We all have to pay our dues and work up the technical ladder. Unfortunately, there’s not a lot that companies can do to stop this transition from occurring. It’s just part of IT. You can’t hold on to your engineers forever.

The second reason, and perhaps the most common one, is that IT people tend to get burned out at companies after a year or two. Let me paint a picture of corporate IT for those of you who aren’t familiar with it. Your typical company is understaffed when it comes to the IT department. The people that they do employ tend to work a lot to make up for this fact. Chances are, you are fast asleep when they are doing their real work. You see, the business can’t afford to have any part of their network down during business hours, so any maintenance will be done real late at night. If you happen to have any sort of Internet presence, the maintenance windows will get even tighter. Of course, anyone who gets into IT expecting to work a 40 hour week from 8am to 5pm is not living in the real world. You work the odd maintenance hours but you still have to respond to issues that come up when everyone shows up for work the next day. Unlike many other departments within a company, you can’t sit on any hot issues. The network is what the business runs on. Failure to get it fixed means the business loses money. When people do call you, it is generally because they have a problem. Nobody ever calls you with good news. When you propose designs for projects, everybody second guesses you. Even if they have no idea what equipment it is that you have included in the design. If a project is over budget, IT usually gets trimmed down. Nevermind the fact that your new facility is buying some hideous sculpture that costs more than your house. Forget the fact that the company is buying everyone a $500 chair. The solution to the money problems are to cut out that pair of core switches you needed and go with the collapsed core model. Then, when the network sucks because you don’t have enough capacity, it’s your fault. Well, at least you have a pretty sculpture to look at as you make your way to your $500 chair at your desk. Issues like that happen time and time again and IT people get fed up with it. They move to another company that feels new and different. At least for a year or two. Then, the cycle repeats. Now I don’t want to be completely negative and say that all companies are like this. They aren’t.

To add on to the reasons behind burnout, most companies simply pay lip service to training. They want their engineers to do a million different things, but have no interest in providing training or even aiding them in their technical development. They want all the benefits of a highly skilled engineer, but don’t want to invest anything in making that happen. Loyalty is a two way street. Don’t expect any sort of loyalty from your IT staff if you don’t show any to them. Demand for skilled IT practitioners is high. Even in down economies, there is still a large demand. When people have options, their tolerance for corporate nonsense is a lot lower than someone who is just grateful to have a job and doesn’t want to rock the boat. They WILL leave.

That’s all I got folks. A total of 7 IT myths. Maybe you agree with some, none, or all of them. I’ve taken these from my experiences and from conversations from many of my peers. As always, if you disagree or have something else to add, leave me a comment below.

Posted in career | Tagged | 22 Comments

The Myths of IT – Part 1

Unicorn - Stolen/Borrowed from http://katemckinnon.files.wordpress.com/2008/07/unicorn.jpg
About 15 years ago, I started my journey through the wonderful world of IT. I realize I am not as old as this guy, but I think I have been around long enough to form my own opinions in regards to some of the myths of IT. Well, at least my perspective is that they are myths. They may be truths for you.

1. Big enterprise experience trumps all. – I’ve been in big environments, small environments, and places in between. To me, there’s not a whole lot of difference. Okay, so the tools might be more expensive in larger environments. The equipment might be a bit beefier in some regards. However, the fundamentals are still the same. In a larger company, chances are you have less responsibilities than someone at a medium or small company. You’re just working on an exponentially larger number of devices than your small/medium company counterparts.

Why is it that people seem to think large network experience equates to more professionalism or better knowledge around whatever it is you do? Here’s a dirty little secret about large networks. They typically employ a lot more people. Guess what that means? It is easier to hide out. It is easier to be mediocre and not have anyone notice. After all, you’re an engineer in a Fortune 1000/500/100/50/whatever company. They wouldn’t have hired you if you were a bum would they? The truth is most IT managers, and certainly most HR managers don’t really understand what it is that you do. That is NOT true for all IT managers. If my manager happens to read this, you are the best. Remember that when it’s annual bonus time. Some IT managers have technical backgrounds and are more than capable of determining technical skill sets. Let us be honest though. Managers are supposed to be good at managing people and expenses. You don’t hire managers to manage routers and switches. You hire engineers and administrators to do that. My last job had a network that was easily 20 times the size of the network I am on now, and many people would consider my current network to be a decent size. Does that give me credibility when it comes to a lot of IT managers? Sure. Should it? Probably not. Sadly, I have had to witness really good engineers get turned down for their dream jobs because they didn’t have “large enterprise” experience.

2. Certification equals ability. – Let me just throw in the one caveat that just came to mind. Yeah. That’s right. I read your mind. The expert level certifications from Cisco(CCIE,CCDE) and Juniper(JNCIE) are usually indicative of ability within an engineer. If you’re a systems person, feel free to add whatever certs you have within your respective discipline that give you instant street cred. Let me go out on a limb even further and mention that if you got your CCIE or something similar 10 years ago and let it lapse, I still consider you an able bodied practitioner of technology even if you can’t put those acronyms on your business card anymore.

Within the IT field, we all know about the “paper” MCSE(or whatever it is called today) and CCNP/CCNA. Yes, those are the ones who crammed for a test and passed it with a little luck and a plethora of TestKing, HotCert, Pass4Sure, or even legitimate study materials. You see them on various forums asking for things as simple as “How do I configure EIGRP on a Cisco router?” and their user profile shows they are a CCNP. You’ve probably even worked with them. They have managed to get by simply by faking it. They learn a few repetitive things over the course of several years and are able to do the most basic things. When something hard comes up they get to pass it off to someone else. Eventually, they land another job making even more money somewhere else because someone was impressed with the acronym soup that came after their name in an e-mail signature. Since there was no big technical interview for this new job, they were able to astound the IT manager with the depth of their knowledge. Maybe they have some “big enterprise” experience under their belt. Years ago, I held a lot of these certifications in high regard. Due to the large number of people with certifications they clearly did not take seriously, I no longer get excited by those little acronyms. I’m almost biased towards the people with no little letters after their name. Yet, job advertisement after job advertisement lists these certs as pre-requisites. Manager after manager wants those little letters. If you are a reseller, I understand your requirements. You need discounts. You need references. I get that. It’s the corporate IT types that I am referring to.

3. Gartner knows all. – Are you a Magic Quadrant(Am I supposed to put that little “TM” thing after that?) fan? Do you read with all fear and reverence the reports that are issued that say so much and yet say so little? If so, I hate to burst your bubble, but there’s a general disdain amongst many of my technical peers when it comes to Gartner reports. Oh sure, they make the reports pretty wordy. They use lots of buzzwords and phrases like “visionary” and “ability to execute”. It’s enough to make an IT marketing professional weep tears of joy!

“Yay! Another report from Gartner on the market leaders in the “Layer 5-application-accelerator-firewall-router” device thingy space. Oh look. Flim-flam networks is in the top right square and Jim-jam networks is in the bottom right square. I guess we better go with Flim-flam instead. Cut the purchase order to Flim-flam and let’s buy today. In fact, buy 2 of whatever they sell! What do they sell? I don’t know. It’s a layer 5 thingy and I think we only go up to layer 4 in our switches. We’re losing ground to the competition without that layer 5 thingy that Gartner tells us we need. We’re visionaries! We need to own the hardware that backs up that claim!”

If you have never read an entire Gartner report, let me suggest that you do. Try this one or this one. Now, after reading either of those, find me something of substance that a technical person would be able to relate to. Is the lact of specifics done on purpose? I wonder. In the interest of fairness, I should point out that Gartner is not the only group I have a problem with. I also tend to ignore reports from “independent” test labs in which a vendor funds the tests. I know. I know. You gave the loser the chance to refute your findings. Here’s perhaps the funniest example of when these “independent” tests go wrong. You have to read all of the comments to get the full effect of it.

Well, that’s it for part 1. Just 3 little myths. Something to ponder over. If you disagree, let me know. I have an open mind, so I am always willing to change it given an effective argument. Just don’t give me a Gartner report telling me why I am wrong. ūüėČ Part 2 will have a few more of these IT myths. I’ll post it sometime in the near future unless the IPocalypse happens!

Posted in career | Tagged , | 9 Comments

The Most Interesting Man In The World

If you live in the United States(and probably Canada) and happen to watch even a small amount¬†of television, you have probably seen those fantastic beer commercials from Dos Equis(XX) about the most interesting man in the world. They say pretty¬†funny things like: “When he goes to the museum he is allowed to touch the art.” Essentially,¬†he is the most interesting man in the world. Everyone wants to be him and they listen to what¬†he says. At least that’s the impression you are left with.

In our Western society, we tend to¬†elevate our doctors to that level. I can’t speak for other cultures, but I am willing to bet¬†it is similar. Doctors are viewed as the most successful people due to the sheer amount of¬†stuff they have to know to do their jobs. They go to school for many years and then spend¬†several years after that prowling the floors of hospitals gaining on the job experience.¬†Then, they can go out on their own and make a bajillion dollars in private practice.¬†Ever been to a party or gathering and ran into a doctor? They become the “Most Interesting¬†Man(or Woman)” in the room. Heaven forbid they are an emergency room doctor. A real life¬†episode of ER, Chicago Hope, Trapper John MD, Scrubs, House, or whatever else medical show¬†you watch on TV emerges.

My brother is a fire fighter in San Antonio, and recently came off¬†an extended tour as a paramedic in that fine city. He has all sorts of stories that can gross¬†you out or make you glad you chose the cubicle for the career path. Medicine isn’t for¬†everyone. The other night my 4 year old daughter knocked her head on something pointy and¬†started bleeding on her scalp. Like any over protective suburban dweller, the wife and I were¬†in full panic mode and so I called my brother up. In a scene that many in the networking¬†industry are familiar with, the “troubleshooting” ensued. “How big is the cut?” Is it still¬†bleeding? Are there any other symptoms like dizziness, vomiting, or general disorientation¬†associated with this injury?” I sent him a picture via my smart phone(Yay cellular network!)¬†and he was able to ascertain that she was fine. Put some Neosporin on it and secure that with¬†a band-aid and problem solved. At that moment, my brother was the most interesting man in the¬†room, or at least within the PSTN. ūüôā

As I write this, I am sitting on a plane flying home from another network upgrade. I didn’t¬†see my comfy hotel room much, and I had very little time to see the sights of a city I¬†love(San Antonio, Texas). However, a routine network upgrade was anything but routine. I had¬†to bang away on CLI all night to get it done. Think about some of the problems you have¬†tackled. Think about the troubleshooting that just flows forth from your mouth like the voice¬†at the end of those car commercials that talks really, really fast. Some things are just¬†natural because you have done them so many times. Go ask a brand new route/switch¬†CCIE(someone with a number) how to configure multi-area OSPF and they should spew forth a¬†barrage of CLI kung-fu. Why? They’ve done it a bunch of times. It just comes natural.

When I try and explain to people what I do for a living, I sometimes use the emergency room analogy. Like a doctor analyzing a traumatized patient, I have to assess network problems and fix them before the people who sign my pay check figure out I am faking this whole IT thing.

In a previous post, I tried to enumerate many of the things your standard enterprise network¬†engineer has to know. If you are a consultant, that list might be even longer based on how¬†many vendor product sets you push….I mean offer. You have to know the differences in all¬†the products to design the proper solution. It doesn’t just encompass triage and treatment¬†like an emergency room doctor. There’s the consult in which you go over options. There’s the¬†post-trauma care in which you look at ways to prevent the problem from occurring again.¬†There’s the favorite part of the vendor which is the dispensation of new hardware/software(ie¬†drugs) that goes hand in hand with the post-trauma care.

The medical field has their specialists. So do network people. You’re an oncologist? Congratulations. We have wireless engineers. If you think tumor removal is hard, try planning¬†an enterprise wide wireless roll out using multiple controllers, location services, voice¬†capabilities, logically isolated guest access, 802.1x, and a wide array of antennas from¬†patch, to omni-directional, to Yagi, to parabolic. Now you may take issue with my comparing a¬†cancer doctor with a network engineer. Is that because you don’t think a network has any¬†power over life or death? Maybe 20 years ago I would agree, but not today. Having worked at a¬†company that owned a ton of hospitals I could tell you stories that would say otherwise, but¬†that’s for another time and another blog post. ūüôā

In a world that uses career choices to dictate who the most interesting people are in the¬†room, it’s time for networking professionals to rise up and claim their place as the center¬†of the conversation. We shouldn’t rest until people are coming up to us and begging us to¬†tell them about our latest network escapades. Let them come and sit in front of us like a¬†cult member would to their cult leader and be fully attentive to our tales of lore. May we¬†spew forth tales of nights spent troubleshooting broadcast storms and routing loops. Let them¬†focus on us intently as we tell them why using static routing is from the devil and why¬†Robert Metcalfe gave us a gift greater than the Polio vaccine(Sorry Jonas Salk. It’s tongue¬†and cheek.). May they run off to their spouses, co-workers, significant others and re-tell¬†the amazing things they have heard.

In all seriousness, I can’t think of many other career fields besides networking/IT in which the learning is never done and the¬†education never ends. Well, not for those who want to be successful in this crazy world of¬†the Internet, clouds, and other over used marketing terms.

The next time you are hanging out with a doctor and they start yammering about how they performed surgery on someone and saved their life with a scalpel and a pack of gauze, ask them if they know the difference between a type 5 and a type 3 LSA. If not, call them a loser and declare yourself the most interesting man/woman in the room! Stay thirsty my friends.

Posted in career | Tagged | 2 Comments

The Many Hats of the Network Engineer

Remember when the network field wasn’t so complicated? Think back to the early 1990’s. Wireless for enterprise users was in its infancy. Firewalls seemed to be a bit easier to administer. Virtualization was limited to the mainframe community. A T-1/E-1 cost a billion dollars a month and could provide Internet connectivity for thousands of users. Voice was still confined to its own cable plant and the PBX was humming along using TDM. RIPv1 was still pretty popular. Hubs made packet captures easy to obtain, but broadcast storms constantly took down segments of the network. Storage involved connecting an external disk array to a server via a SCSI cable. ISDN was what the rich people used at home for Internet access. You know. The good old days.

Well it seems that a lot has changed since then. While I have no desire to go back to those days, I do miss the simplicity. Or at least what seems simplistic compared to today. Let’s take a look at what your typical enterprise network person has on their plate. Keep in mind that in some environments, these people also have systems related duties such as Active Directory administration, Linux/Unix administration, e-mail, database, etc.

Routing – Static, OSPF, EIGRP, and BGP

Switching – STP and its variants(RST, MST, PVST), Link aggregation(port channels/etherchannels)

Wireless – AP’s(antenna types), controllers, extras(location services, management), 802.11a/b/g/n

Circuits/WAN – T-1’s, DS-3’s/T-3’s, OC-3/12/48(SONET), Metro Ethernet, ISDN(Yes, it’s still out there), FrameRelay(Yep. That one too.), MPLS

Voice – call routing, phone(station) administration, voice mail, conferencing(audio and video), PRI’s, DID’s, signaling, codecs, voice gateways

Other Services – Multicast, load balancing, firewall, IPS, VPN, WAN optimization, content filters(web,e-mail), network management platforms, QoS, packet capture analysis(ie Wireshark,tcpdump), storage networking

Does that about sum it up? Yes, some of those things were being done back in the 90’s and in some cases, even earlier. However, a lot of them are relatively new things. Maybe you don’t have to touch all of those things. Maybe you do. For some of the service provider type things (MPLS, SONET), you may not ever have to administer that end, but if you’re buying those services, you better be familiar with them. Perhaps your organization is large enough to break out the security side of things or the voice side of things. Maybe you have a dedicated storage group that handles the storage network side. If you are lucky, you may even have a dedicated wireless engineer or two depending on the size of your wireless deployment.

It is a monumental task to become proficient in all of those areas, but wait; there’s more. For many people in the network space, they also have to become data center/facility engineers focusing on the following things:

Monitoring – temperature, humidity, water leak, smoke, power load levels

Cooling – BTU calculations, hot/cold aisle design, airflow on hardware

Power – Circuit requirements, UPS requirements, generator requirements

Cabling – Sub-floor, above the rack, CAT-5/6/7 differences, patch panel choices/locations, SM and MM fiber differences

Space Requirements – Rack deployments, 2 post, 4 post, full height, half height

Think that’s all? Well, the past few years have added some additional requirements, and more are coming. Things such as:

Virtualization – It has been around for at least 5 years now in enterprise environments. It’s not going away and without using newer hardware/software from networking vendors, you can’t see what’s going on inside the server farm.

The Return to Layer 2 in the DC – TRILL and every vendor’s particular flavor of it aim to resolve the ineffiencies of Spanning Tree and turn your network switches into an intelligent fabric. This will be similar to what storage networks have today via Fiber Channel.

Consolidation of Storage and Data/Voice Traffic – It happened to voice about 10 years ago. Now it is happening to storage. Everything will be on 1 wire in a matter of years.

Traditional Endpoint Death – No longer will the phone, desktop, and laptop rule the network. Cellular phones, tablets, and other similar compact devices will show up on the wireless networks in even greater numbers than they are today. Congratulations corporate wireless person. You just become a Google, Apple, Microsoft, Blackberry, HP, Cisco, and Avaya engineer for their mobile product set.

IPv6 – And you thought planning IPv4 deployments were interesting? The migrations to IPv6 are going to be interesting. Using NAT and 6to4/4to6 tunnels will become commonplace until the IPv4 is gone. I realize this is already happening/happened in many other parts of the world. However, in the US, there’s still a LOT of work to be done.

Now I realize that nobody is going to be an expert in all of these areas. I also know that many employers are not going to require you to even be familiar with all of these things. With things like hosted data centers, you may not ever have to deal with data center build out. Power and cooling may never be an issue for you. I also know that there are plenty of good consultants out there that specialize in one or more of these areas. Of course, nobody stays at the same company forever, so what you do at company X today doesn’t mean you won’t do a bunch of other things at company Z tomorrow. I guess the point I am trying to make is that our jobs are only going to become more complex in the years to come. The amount of hardware we use may decrease, but the functions within that hardware will increase. I can see a day in which something like WAN optimization is built into the router itself, and I don’t mean via a service module. I mean built into the processors or ASIC’s themselves. Of course, that’s assuming we’re still using TCP at that time. I don’t even want to contemplate what wireless will be like after 802.11n because it makes my head hurt just trying to understand how 802.11n works today with multiple antennas.

Start looking at the blueprint for something like a Cisco CCIE Route/Switch(Insert any other track as well) or Juniper JNCIE exam and you’ll find that it only covers a portion of what you need to know in this day and age. Anyone who has been involved in that process from start to finish knows how much information you have to know to pass. For those who don’t know, it is a TON. Yikes! Still want the job? Maybe becoming a specialist isn’t such a bad idea after all.

Posted in career, learning | Tagged , , , , | 9 Comments

Competing With Cisco

***Please note that these are my own thoughts and not those of my employer.

First and foremost, I have to credit Jimmy Ray Purser from Cisco for putting the idea for this post in my head. Back in late April of this year, he wrote an article for Network World entitled “The ABC’s of Anybody but Cisco.”

Like a lot of people, I work on networks that have a lot of Cisco gear. I’m very comfortable with their stuff. I’ve used CatOS, IOS, and NX-OS. Switches, routers, phones, firewalls, load balancers, access points, voice gateways, ACS, etc. It’s all familiar. Old hat. A lot like the old comfy t-shirt that Tom Hollingsworth wears. I’ve invested a lot of my time and energy into learning their product set. One could say my familiarity with their gear has been responsible for a significant portion of my earnings over the years. I don’t want to give the impression that I am a paid shill for Cisco. After all, I don’t work for a Cisco partner. I’m one of those corporate types who occupies a cubicle and acts as the caretaker for a decent sized network. Not a massive network, but big enough to have some direct support from our local Cisco office when we need it. We don’t use Cisco for every single thing on the network side, but if the various vendors on our network had voting rights, let’s just say that Cisco would be able to influence any election in their favor.

Although I use a lot of Cisco gear, I also try and keep an eye on the other vendors out there. On a given day, I probably get at least a dozen e-mails from other vendors and networking publications. Additionally, I have an ever growing RSS feed list comprised of dozens of vendor blogs and blogs from the many networking professionals I follow on Twitter or have found via word of mouth. In other words, I spend at least 10% of my day consuming information from Cisco, other vendors, and networking professionals who may or may not share my viewpoint. I feel that gives me the potential to have a very well rounded view of the networking industry. Whether or not I can process all of that information to form worthwhile opinions is another matter. Suffice to say, I am trying to put in the hours to ensure I can make the best decisions for my employer.

The challenge to vendors other than Cisco is figuring out how to pitch their product and have it SERIOUSLY considered by people who manage networks where Cisco dominates. This can be done, but it takes marketing and sales people who understand who their likely customers are. I’ve seen and read plenty of non-Cisco pitches. Some are very good and offer compelling reasons to consider their product. A lot offer nothing other than “We’re not Cisco!”.

A few years ago, I read a very interesting book about Wal-Mart called “The Wal-Mart Effect”. I thought it was a mostly balanced look at the company and how they do business. One of the most interesting parts of that book was where a retail executive was giving advice on how to beat Wal-Mart. He said that you would NEVER beat them on price. They are too big and have too much influence over their suppliers. Wal-Mart will do whatever it takes to get the lowest price possible from the supplier. You beat Wal-Mart by creating a better experience for the customer. Give them better quality. Give them better product selection. Give them nicer facilities to shop in. That’s how you beat them. If you try and take them on in a price war, you WILL lose. I think about what that person said and try and apply that to Cisco’s competitors. How do you compete with the overall networking market leader? In my mind, that requires some different thinking.

1. Don’t spend all day bashing Cisco. – If I am willingly talking to another vendor as an alternative to Cisco, it’s probably because I realize there are other options out there. You are not helping your cause any if the main thing you have to offer is that Cisco sucks and you are better. Up until July of this year, the Riverbed blog was full of posts slamming Cisco. I’m not going to say that Cisco didn’t deserve some of those. I’m not even going to make the argument that WAAS is the same or better as Riverbed when it comes to WAN optimization. Clearly Riverbed is doing some great things in WAN optimization. They’re very easy to setup and administer. Their products work very well. Sell me on those points. Sell me on the fact that it works and that you’re offering me things that the other WAN optimization vendors are not. When I have to deal with corporate arrogance be it from marketing content or sales people, I get turned off on the product real fast. Why? Well, arrogance breeds complacency. It doesn’t allow an organization to see clearly and eventually someone is going to catch up and turn your double digit market share into single digits. When competing with a company like Cisco, you are competing with a very well oiled marketing machine. Don’t ever forget that. In fairness to Riverbed, I haven’t seen that mentality lately. As a result of that, my feelings toward them have improved greatly.

2. Don’t preach to me about standards. – If there’s one thing I hear the most, it’s that vendor XYZ is a completely “standards” based company. If you have been around the Cisco community for a few years, you know how instrumental they are in driving standards. VRRP, MPLS, PoE, LLDP, 802.1q, and CAPWAP among others are a direct result of Cisco and their influence. Additionally, some of these companies that want you to use their hardware because it is standards based are producing their own variants of certain standards. I can think of a couple of vendors right off the top of my head that have their own enhancements to VRRP. Most of the Cisco gear I have been around support these “standards”. Remember, they drove the creation of quite a few of them in the first place. Yes, Cisco does make modifcations/enhancements to things like OSPF and other protocols, but so do most large vendors. My point is that everyone supports standards, or they wouldn’t be standards. What vendors are really trying to tell me when they are preaching the fact that they are standards based is that by using proprietary protocols, I am locking myself into Cisco. That, in their minds, is a bad thing. I can assure you that I am aware of the risks of running EIGRP, HSRP, or whatever proprietary protocol that Cisco puts out. It’s also not a good idea to assume that I am running any proprietary protocols just because I have Cisco gear.

3. Your presentation needs to be polished. – Please, please, please spend some time on your documentation and presentation material. Understand that people who are buying and deploying Cisco have access to very polished design guides, configuration guides, product data sheets, etc. While I am not asking you to replicate the entire Cisco documentation/support ecosystem, it would help if you ran your documentation through a spelling and grammar check before displaying it on your website. For examples of good documentation outside of Cisco, see Juniper. They have their act together.

4. Innovate – Sometimes a radical approach to the way we are used to seeing things is needed. Aerohive is doing some very interesting things in the wireless space. They’re not using controllers to manage their AP’s. The AP’s manage each other. Very different and very cool. I’m not a wireless expert. I don’t use their products. I don’t have any plans to use their products in the near future. However, I AM keeping an eye on them and if an opportunity comes up in which they would be a good fit, I won’t hesitate to reach out to them.

Juniper has completely sold me on their SA line of SSL VPN appliances. Everyone I talk to about them has the same feeling. Very feature rich!

Riverbed is the leader in WAN optimization. I use their product and am very satisified with it. It just works.

Arista has created a compelling offering in the data center switching environment. The fact that I can run numerous third party applications on their switch due to their granting me shell access is VERY interesting. Oh, and I should also mention that nobody else can give me the same amount(384) of wire speed 10Gbps ports in a chassis(7500) of their size(11RU).

Brocade and Force10 both offer an interesting alternative to the standard top of rack copper switch or Nexus 2k FEX line. They have line cards that support the MRJ21 connector system developed by Tyco Electronics. Essentially, your top of rack switch is reduced to a patch panel that connects back to a Brocade or Force10 chassis. However, this is not a 1 to 1 patch panel. Rather, a single MRJ21 cable(about the width of a pencil) connects the patch panel and chassis. Each cable supports 6 10/100/1000 connections. The patch panel is not the ordinary punch down block you are used to seeing. It’s a modular cassette or fixed 24 or 48 port 1U patch panel. None of these cassettes or patch panels require power. All management is done from the central chassis. If you are familiar with Nexus 2k administration from the Nexus 5k, this is similar. The benefits to this technology are three fold. First, there is no power consumption required in the top of rack. Second, all administration is done from the central chassis which can be located at the end of the row, middle of the row, or on the other side of the data center. Third, with the modular capabilities, you can split up the 24 or 48 ports your typical top of rack switch has. 1 logical 48 port switch being managed at the central chassis can be spread out over 2 or more racks.

————————————————————————————-

That’s it. Four simple things from my perspective. There may be more, but to me, these are the big ones. I didn’t mention pricing. I don’t usually see this as being an issue. Other vendors know what the pricing is going to be from Cisco. They can figure out based on the size of the company what the discount is probably going to be. They may not know the exact amount, but they can make a pretty good guess. Your product needs to be cheaper. That’s a great selling point for a lot of vendors. They know you are paying for the Cisco name, so they can use that as leverage. If it is not cheaper, it better have some sort of compelling reason to be chosen. There needs to be a “wow” factor and not vaporware. It needs to be legitimate.

You’ll also notice that I did not address the problems with Cisco itself. That would take another long post and there are plenty of other capable bloggers out there hammering away at them on a regular basis. My goal in this post is to focus on how to compete with Cisco.

Posted in cisco, vendors | Tagged , | 9 Comments

What Exactly Do You Do For A Living?

Once upon a time I wanted to grow up and be an astronaut, police officer, pilot, and cartographer. Well, as you can probably guess, I didn’t end up anywhere near those professions. Here I am neck deep in the world of information technology. Although my primary focus is on the routing and switching areas of networking, I find myself routinely pulled into other areas of networking as well. If I am at one of my employer’s remote sites, I might get asked about power and cooling needs, or even how we are going to replace a PBX with a VOIP solution. Just a few days ago, myself and several co-workers pulled an all-nighter at the data center in order to add more redundancy. That involved security as well as other services such as DNS. Although our wireless networks typically work well, if there is a problem, my group is responsible for fixing it. Basically, I get to run the full range of networking.

If you are reading this, odds are you understood everything I just wrote in the opening paragraph. You understand it because you are probably involved in the industry to some degree. The average non-IT person out there does not. That’s not to say that they are completely in the dark about IT or networking in general. It’s just that they don’t deal with this stuff every day and thus, it is a foreign world to them. One of the questions I get asked quite often is what I do for a living. I can tell people I am a network engineer/architect/janitor/technician, but I usually end up telling them I am in IT. Some people know what IT is, and for others I have to define it as “Information Technology”. I usually just mention the word computer and people understand. One of things I am quick to point out is that I don’t actually work on the computers themselves. I simply make sure they can all connect to each other and pass data back and forth. Perhaps the most frequent comparison I use is that of a road network. If you think of the various homes and buildings around your part of the world, those are the computers. The roads are the network. I take care of the roads. Whether it means building a new one, filling in potholes, or widening an overcrowded stretch of road, that’s my job. Thanks to the wonderful marketing power of Cisco, I can usually ask people if they are familiar with the “Cisco” brand and most times they are. Then it’s easy enough for me to say that about 90% of what I do revolves around that brand name. Surprisingly, nobody has asked me to help fix their Flip device yet!

Why am I mentioning all of this you ask? Well, it is simply a reminder that for those of us who are networking professionals, our job is relatively unknown outside of IT circles. Perhaps the one big exception might be those of you who focus on voice. The phone is a device everyone is familiar with. It is a tangible product that we surround ourselves with. If you tell someone you take care of the phone system, they understand that. Router, switch, firewall, access point, and load balancer are all foreign terms to most people. They hear the word computer and more often than not, assume you can remove that virus that is preventing them from getting to YouTube. That couldn’t be further from the truth.

As you grow in your networking or IT career, you’ll find that you need to constantly improve your skills and knowledge. What you know today is only a fraction of what you’ll need to know a few years from now. In the course of all this, you’ll have designs that fail and implementations that go bad. You’ll also have a lot of scars to prove it. There will be plenty of war stories that you acquire over the years. That information is of great value to others in this field. You’ll also want to hear the war stories from others. Since the average person out on the street is not involved in what you do for a living, you will have to find other people who do what you do. You need to share what you know with others. Likewise, they need to do the same. There’s a reason I chose the “therapy” theme for this blog. I can’t go home at night and talk to my wife about what I do for a living. My son and daughter are both under the age of 10. If my job doesn’t have anything to do with Barbie or Star Wars, it’s not very interesting to them. With very few exceptions, I can’t go and talk to people at church about the struggles and successes about life in corporate IT. Thus, I need an outlet. We ALL need an outlet.

Here’s a few suggestions:

1. Find a local user group. This would be a group of people who do what you do for a living and get together once a month or once a quarter and talk about various technologies, hardware, or software.

If you happen to be a predominantly Cisco shop, there are quite a few of these groups around the United States. You can find one here.

For all other types of IT related groups, this site can help.

2. Get involved online. It goes without saying that if you are reading this, you are already involved to a certain extent. However, there’s more out there than just blogs.

LinkedIn is a great way to connect with other IT professionals. There are tons of groups on LinkedIn that pertain to a certain local area and technology. Via the group discussions, you can interact with thousands and thousands of IT professionals all over the world.

Twitter is one of my favorite sites to use. It provides almost instant feedback on virtually anything you can think of. Plus, there are tons of really GOOD technical people on there. People who write books you have read. People who work on networks you dream about. With a worldwide audience, there’s always somebody online watching their Twitter feed who can help out with an issue or just listen to you rant about why you should have gone to law school instead. Not only will you learn a lot, you will get to know people you will probably never meet in person.

Forums are a great resource to ask questions or help out with other people’s problems. Cisco has a fairly large technical forum on the main cisco.com website. Another decent site to use is networking-forum.com.

There’s more things like IRC, mailing lists(NANOG comes to mind), and even sites like Facebook, but I think you get the idea.

As technology becomes a larger part of our everyday lives, people will figure out more about our networking jobs. Wireless is becoming the norm out there, so a lot of people are learning more about wireless hardware and terms. With the dangers of the Internet, the term firewall is being thrown around a lot more. While I doubt everyone out there is going to learn about the OSI model, I do think our jobs will become less esoteric and more mainstream. Until then, we have to find a way to help each other out.

Posted in career | Tagged | 4 Comments

An NDA Can Keep Bad Decisions Away

Over the past year, I have seen some interesting presentations from vendors showing me some things that they have on their future roadmap. Some of these things have already been released to the public. I’m still waiting on the rest. All of this was a result of having non-disclosure agreements or NDA’s in place. The vendors agree to show us some of their stuff that is coming to market soon on the assumption that we will not release this information to anyone else. While I do enjoy knowing about things before they hit the market, I sometimes feel bad for companies that don’t have access to this information. Not only that, I often wish I had access to all vendor product roadmaps. Let’s face it. From the network hardware/software standpoint, we generally do business with only a handful of vendors. I say that as someone who works in a corporate environment. If you are a consultant, that doesn’t necessarily hold true as you may sell a wide variety of hardware and software.

If your dealings with companies are limited to a select few, those companies have a vested interest in making sure you stay with them. One of the ways to do that is to give you a better view into their product cycle so that you know what is coming. Look at the switch market for example. The number of vendors offering products in that space is growing and growing. I recently spent a LOT of time comparing 10Gig aggregation switches between 6 vendors. What if the vendor I use today had a platform that was average or below average in terms of 10Gig capabilities? If I had a hard requirement for a certain number of 10Gig ports and it had to be contained to 1 chassis, my choices are really going to be driven by 10Gig port density. It could be some other factor like power consumption or even chassis size. It doesn’t really matter. As long as my usual switch vendor cannot meet that requirement, I am going to go outside and look for another vendor. If I am dead set on staying with that vendor, I am going to change my requirements, To me this does not seem like a viable option unless it would cause unbelievable pain and suffering to introduce another vendor into the network. If that is the case, you probably need to re-think the whole single vendor strategy. Then again, if that single vendor works for you, then go for it. It’s your network and we each have to make the decisions that serve our company and customers best.

Back to the fictional switch problem. What if I am the incumbent vendor and I know you have a need that I cannot fill today, but will be able to fill that need in a couple of months, or even a year from now? Should I tell you even if nobody else knows about it? This is where the NDA comes into play. If you have done a bunch of research and are looking at alternatives to the incumbent, your mind might be changed if you happen to know something better is coming. Maybe it is far better than every other vendor’s current products you have been looking at. Maybe it is on par with the replacement vendor you have been looking at. Can you wait that long?

After seeing the new shiny thing that is coming out soon, you may decide to stick with your existing vendor. However, who is to say that the other vendors won’t be coming out with even better hardware/software around the same time or a month or two after? This is the point in which I find myself wishing I had access to all the vendor’s product road maps. I know some vendors will do an NDA on the notion that it will get them a sale, but I don’t know that I am going to be able to spend a bunch of time with every vendor to the point in which an NDA can be put in place. Perhaps it is best to bring in the consultants/integrators that sell products from a number of different companies. I would suspect they have some sort of idea when it comes to the future direction of certain product lines. Or maybe not. They might be in the same boat as I am.

The last thing you want to do is buy a product and have an even better solution appear a week later. I do think that most vendors will let you know that a better product is coming rather than lose the sale as long as the dollar amount is high enough. I don’t think company X is going to reveal a whole lot about their future road map if the net gain is a couple thousand dollars. Then again, if the salesperson has had a REALLY bad quarter all bets are off, but at that point you can smell the desperation in their sales pitch and I tend to be put off by that. That leads me to the thought that you really have to consider a wide range of factors when dealing with vendors. To me, the product has to meet the technical requirements above all. After that, cost is important. Right along with cost is the experience the vendor will give you. What are the hardware/software support capabilities of that vendor? What is the direction of the company? How long has the company been in business? If it is a recent startup(ie less than 2 or 3 years), who are the people running the R&D for these products? Are they known in the sector they are doing business in? In other words, if they sell security solutions, are they using experienced security professionals to develop the products or is this strictly an “academic” operation in which someone had a decent idea and got some venture capital funding? Granted, you can’t always figure all of that stuff out, but if you can, it sure helps when the decision making time comes.

To sum it all up, I think the smart vendors are going to tell their customers what is coming and when they can expect to see it for sale. It helps people like me plan for things down the road. I’m more interested in a vendor that is constantly updating their technology as opposed to one who releases new products with lesser frequency than leap years occur. When it comes to NDA’s, I don’t think the size of the customer should matter either. Small companies can get big relatively quick in this age of acquisitions and mergers. IT professionals DO talk to each other and tend to trust each other’s opinions MORE than a vendor paid “performance test” by an “independent lab”. If you are a vendor and want to show me your road map, I promise not to tell anyone outside of my company. ūüôā I don’t even want you to buy me lunch. I might just put pictures of your hardware up in my cubicle and drool over it all day until I my manager lets me buy it.

Posted in vendors | Tagged , | 2 Comments