What I Am Looking For At Interop

I’m returning to Las Vegas for Interop. I was fortunate enough to attend last year on behalf of HP and Ivy Worldwide. I am returning again as a guest of HP and Ivy Worldwide. I should point out that HP has never asked me for anything other than an opinion. They sent me to the Las Vegas and New York Interop shows last year, but the New York show is much smaller than the Las Vegas one.

A stark difference in my Interop Las Vegas experience last year and my Interop experience this year is my relationship to these vendors. Last July, I left my corporate IT job and went to work for a reseller. I LOVE talking to vendors, and being in the reseller space allows me a different kind of relationship with vendors. It is more of a collaborative type relationship and less of a buyer/seller relationship that you see in the corporate IT side.

For the bulk of the conference, I will be on the expo floor. That’s the place I want to be as much as possible. While the pens and t-shirts are very plentiful, I am not really there for that. However, if they are giving away electronics in a LEGITIMATE drawing, I will not turn that down. I just have to figure out which ones are legitimate and which ones are steered towards potential customers(ie rigged drawings). I’m there to absorb as much of the vendor’s solutions and company secrets as possible. Okay, so I don’t get a whole lot of company secrets, but I keep hoping someone in a vendor booth will slip up and tell me something they shouldn’t. 😉

Here’s some random thoughts around some of the vendors I want to talk to. I’ll indicate if my company is a partner with them as we partner with a lot of companies.

Aerohive (Partner) – They don’t have a booth, but they are here. Their Bonjour gateway seems to have attracted a fair amount of attention and they just released the BR200 device for remote offices/mobile employees. I’m curious to see what might be coming from them in the near future.

Alcatel-Lucent – I don’t ever run into them in the enterprise space. I am curious as to how they are doing in that market. If I recall correctly, they tried to sell their enterprise line and focus on the SP market, but that fell through. They re-brand their wireless from Aruba, so I probably am not going to ask much about that.

Arista – I love this company. They do one thing and do it well. I am curious to see if they have any announcements around 40 and 100gig. I know they will be supporting it, but not sure to what extent and when. Listening to various Arista people talk during the Getstalt IT Network Field Day 3, I get the feeling they will continue to innovate and dominate the low latency switching market.

Avaya – I had a nice long chat with one of their engineers at their booth at last year’s show in Las Vegas. I am interested to see how well their switching line is doing. They also have a wireless line that I didn’t get to look at last year. That intrigues me. Finally, they came out with their own UC tablet like the Cisco Cius. I am sure they will have their Flare tablet on display.

Barracuda Networks (Partner) – As a Barracuda partner, I have a different view of them now. I heard a presentation on their Next Gen firewall last year, but the presentation didn’t sell the features of the NG firewall that I was able to see during my partner dealings with Barracuda. It really is a neat firewall that came via an acquisition and was not developed in house. I think that is where people have an issue. They think it is a feature-lite firewall like some of their other products when compared to their much more expensive competitors. They also dabble in the UC and video surveillance market. Odd considering they got their start doing anti-spam work. They also have a storage replication product that I am interested in seeing demonstrated.

BlueCoat (Partner) – I really just want to see if/when full combination of content filtering/WAN optimization will happen. Maybe it shouldn’t happen. Maybe it should. I used Packeteer products in a previous job and liked them.  Most of my questions to BlueCoat are really around differentiating themselves from Riverbed.

Brocade (Partner) – It turns out that Brocade does not have a booth on the expo floor, but they will be sharing part of the NEC booth. We are doing a fair amount with Brocade, so I am just interested in a general conversation with them. I’ve put forth my theory that they will buy a firewall vendor in the coming years if they don’t get acquired and taken private. I’ve discussed this with multiple Brocade employees, but that’s just complete speculation on my part and cannot be validated by anyone I have talked to. I also wonder about their relationship with Motorola since they re-brand their wireless gear. Would Brocade ever buy that part of Motorola? At the risk of sounding too “conspiracy theory”, I will end my comments on Brocade here. 😉

Cisco (Partner) – I have tons of questions for Cisco. I want to see the 3600 AP up close and personal. I would also like to know if they have the module that is rumored to be coming. I’ve heard about it on the No Strings Attached podcast and pictures of the AP indicate that future capability. As a partner, I might know a little more than I am letting on, but that wouldn’t be very professional if I blabbed about every little thing now would it? 😉 There are some other wireless things I would like to chat about. Some of the other things I would like to see and talk about are WAAS, Nexus 3000 series, ASA CX, UC ver 9 enhancements, ACE 30 update and potential for a Nexus 7000 module.

Citrix (Partner) – My company has someone covering the Citrix Summit / Citrix Synergy conference in San Francisco, and that person is far more versed in Citrix than I am. My interest in Citrix begins and ends with the NetScaler as it relates to load balancing and SSL VPN connectivity.

Dell (Partner) – Obviously, the big thing for me with Dell is the Force10 gear. I’m interested to see how they are rolling the Force10 gear into their lineup and what that means for the other vendors(ie Brocade) that they re-branded. I’d also like to talk to them about Aruba wireless that they re-brand as PowerConnect W-Series. I wonder if Dell would consider buying Aruba. They seem to be buying everyone else lately. SonicWall will have a booth at Interop as well, so naturally I am interested to see how that integration with Dell is going.

ExtraHop Networks (Partner) – What’s not to like about this NPM/APM vendor started by ex-F5 people? They are entirely agentless and can be up and running in a matter of minutes. They also just released a new Citrix module, so I would like to talk with them about that. Other solutions require software agents for the type of information they are pulling off the network with a simple packet capture.

Extreme Networks – I remember last year’s Las Vegas show where they showed the Black Diamond X8 switch in a smoke filled booth. 192 ports of 40GbE at line rate. A monster switch. I am curious how they are selling and who is using that kind of  throughput.

Ekahau – They will be located with the MetaGeek booth on the expo floor. I had the privilege of seeing this Finnish company present at the last Gestalt IT Wireless Field Day event back in January. I’ve been using their Site Survey tool lately, and have also been able to use their Android-based Mobile Survey utility.

F5 Networks (Partner) – I haven’t seen the Viprion line in person yet. I am hoping they have some at their booth. They are always good to talk to since they seem to be expanding beyond the load-balancing function they are so well known for.

Gigamon – This orange colored company is hard to miss. I have talked to them before, but I am interested about what life would be like as a Gigamon partner. With all the monitoring that is required in data centers these days, their name comes up more and more in discussions amongst other engineers.

HP (Partner) – Since HP is bringing me to Interop, I get a fair amount of access to their people. I have several questions around their firewall solutions, the collapse of switching lines on the ProCurve side, as well as the future of their voice platform they inherited from 3Com. Then, there’s the wireless solution, which I assume will be simplified in the coming years just like their switching lines. I also want to see how their publishing venture has been received. Finally, I would like to see if there are going to be any changes to the AllianceOne program, which I happen to think is a good idea for them.

Huawei – I talked to this newcomer to the US market at Interop NYC last October. They have a full line of everything from switches to telepresence video conferencing suites. I am interested to see how their growth in the US market is going.

Juniper Networks (Partner) – Of course, QFabric will be on display. It’s hard to miss that impressive hardware. I’ll take better pictures this year than last. I’m also keen on seeing anything new they have coming up. I believe a fair amount of their focus has been on the security market lately.

MetaGeek – I have had the pleasure of seeing MetaGeek at the Gestalt IT Wireless Field Days for two years in a row. Their new EyePA product is simply amazing and I look forward to seeing the looks on people’s faces when they see it demonstrated for them.

NEC – I need someone to sell me on OpenFlow in the Enterprise. I see the applications for it in the SP market. I just don’t see it in the Enterprise. If anyone can show me where it fits, it will probably be NEC.

ShoreTel – I am not a voice person, but I do like to have a basic understanding of who the key players are. I am always looking for a solution that could possibly rival Cisco and Avaya as they seem to dominate the voice deployments I come across in enterprise networks. I spoke with ShoreTel last year and have seen them around for a number of years.

SonicWALL (Partner – Sort of.) – Dell recently acquired SonicWALL, so I would expect to see a lot more of them in the near future. As a Dell partner, my interest is really centered around how to position them in the enterprise space. I used to work for a SonicWALL partner and we sold quite a few of their boxes in the SMB arena. Although my SonicWALL cert is long expired, I am curious to see how much progress they have made since I dealt with them last.

Vyatta – This company has been around for a number of years running routing software on commodity hardware. I have never used their software and have not come across many companies that do. Having said that, I do know there are a fair amount of people that are big fans of Vyatta. I’d be interested to find out from them about large real world deployments.

Xirrus – There’s really one big question I have for Xirrus, and it has to do with 802.11n MIMO. They have directional antennas in their arrays, so I want to understand how they can associate multiple antennas with a single client. I have no doubt they have thought about this, so I am sure they can answer that question.

There’s a lot more vendors at Interop. See the full list here:


If you happen to read this before Thursday, May 10th 2012, let me know if there are any questions you would like me to ask a vendor or vendors. I’m more than willing to do that. Just drop me a comment at the end of this post and I will do my best to get it answered while I am here.

Posted in vendors | Tagged | Comments Off on What I Am Looking For At Interop

How Bad Do You Want It?

A friend of mine posted this on Facebook, and I felt compelled to share it. Although the primary focus is on physical training, it has massive parallels to IT and “brain training”. Watch the whole thing. It takes 5 minutes. If you have trouble with professional development, it should give you a much needed boost.



Posted in career | Tagged | 1 Comment

Just Another Cisco Nexus Install

The most recent Cisco Nexus implementation I was involved in had some challenges. Since you can’t really have an in depth discussion on Twitter due to the 140 character limit, I figured I would throw this post together and share in the fun that is Cisco Nexus switch deployments. Maybe it will help out with someone’s Cisco Nexus implementation. As always, comments are always welcome. Maybe you will see something I missed, or have a question regarding why something was done a certain way.

I have removed any identifiers to the actual customer this network belongs to and received permission from the customer to post this.

Initial Build

This was going to be a standard Nexus 7010, dual core install with a handful of FEX 2232/2248’s for TOR server connectivity. Ordinarily this is a pretty straightforward install. The Nexus 7000 series switch does not support dual-homing the 2200 series FEX like the Nexus 5000 series does. However, there is a way to connect a FEX to a pair of Nexus 7000’s and have it fail over if one of the 7000’s fails. You just have to wait about 90 seconds or so for the FEX to come online with the second Nexus 7000. This particular configuration is not supported by TAC(that I know of), so getting it to work is done without the assistance of Cisco.

Here’s what the network would look like at this particular data center after the Nexus 7010’s were put in place:










Know what those non-Cisco switches connected into each Nexus 7010 mean from a Spanning Tree perspective? You guessed it. MST. That changes things a little bit, but not too much.

I should also point out that these Nexus 7010’s were purchased with a single N7K-M132XP/12 line card in each chassis. That means the only way into the box was via a 10Gbps connector. We had a 2Gbps circuit connecting this data center to another as the primary link, and a backup 1Gbps circuit connecting to that same remote data center. The 2Gbps circuit came via a 10Gbps handoff, so connecting it to the primary Nexus 7010 was not a problem. The other circuit came via a 1Gbps handoff, so that ruled out using the secondary Nexus 7010 as there were no 1Gbps interfaces. There was the option to buy a module for the Nexus 7010 that supported 1Gbps fiber connections, but since that particular card lists at $27,000(It appears you CAN buy anything on Amazon!), even with a decent discount, that’s a lot of money to spend just for a single 1Gbps connection. That also still doesn’t fix the inability to have redundant connections to a FEX with fast failover capabilities.

Design Change

Due to issues around redundancy and flexibility of 1/10Gig connectivity, the design was changed to include Nexus 5548UP switches. A pair of 5548UP’s were purchased, and the design changed to the following:










*Note – Although the orange oval is listed as FEX 101 in the legend, it was actually several FEX’s numbered 101, 102, 103, etc. I realized the image looked confusing, but I am too lazy to go back and re-do the Visio drawing. Just know that it represents multiple FEX’s.

Implementation went fine. Everything appeared to be functioning normal. vPC was up on the 7k’s as well as the 5k’s. I tried to make the drawing above reflect what the configuration would look like. I didn’t want to drop lines and lines of config for 4 devices as it would really inflate the size of this post. Each Nexus 5548UP connected to both Nexus 7010’s via vPC. Each FEX connected to both Nexus 5548UP’s via vPC as well.

During testing right after the implementation, one problem did surface. An iSCSI storage array connected to one of the Nexus 2248’s was unable to communicate with another iSCSI storage array in a different data center. Basic ICMP(ie ping) communication was possible, but it was erratic in dropping some of the traffic. It looked like an asymmetrical flow issue where one path was getting black-holed or dumped somewhere along the way. There was also no direct ratio of pings that responded to pings that did not respond. We also tried to restart the replication process as ping is not a foolproof test. However, all ping tests were THROUGH the Nexus gear and not TO the Nexus gear. In the case of the Nexus 7000, there is a built in control plane policing policy that will start dropping excessive ICMP traffic sent TO itself. I’ve personally witnessed an issue where someone thought the Nexus 7000’s we just installed for them were messed up because they were throwing massive amounts of pings at the Nexus 7000’s, and they were not getting 100% of ping requests answered. Just because there is no response doesn’t mean there is a problem. Sometimes it is by design.

Due to the replication failure, we started to unplug some things. The FEX 2248 in question was unplugged from the second Nexus 5548UP it was connected to. That secondary Nexus 5548UP still had connectivity to the primary Nexus 5548UP and both Nexus 7010’s. The problem still happened.

Next, we reconnected the FEX 2248 to the secondary Nexus 5548UP. Then, we disconnected the secondary Nexus 7010 from the secondary Nexus 5548UP. The problem still occurred.

We ended up disconnecting all FEX 2232/2248’s and both Nexus 7010’s from the secondary Nexus 5548UP. The only thing the secondary Nexus 5548UP was connected to was the primary Nexus 5548UP. Everything stabilized after we isolated the secondary Nexus 5548UP.

Lab It Up!

The following week, I was able to recreate the same environment we experienced the failures in except for the iSCSI arrays. I had another pair of Nexus 7010’s and Nexus 5548UP’s for another data center this customer was going to upgrade in the near future. I also had several FEX’s to use for testing along with a 3750 switch that was going to serve as the connection point for the management interfaces on each Nexus 7010/5548UP. To simulate the remote data center, I had a Cisco 3560-X switch with a 10Gbps interface module to connect to the primary Nexus 7010.

I plugged a box into the Nexus 2248 FEX that would be the remote target. I plugged my laptop into the 3560-X using a completely separate VLAN than the link between the 3560-X and Nexus 7010. This ensured I was crossing multiple layer 3 boundaries between my system and the simulated remote target host. Testing showed no problems whatsoever. No packet loss. Nothing.

If At First You Don’t Succeed……

The next week, we went back to the data center and reconnected everything back up to the second Nexus 5548UP. The configuration of the Nexus 7010’s and Nexus 5548UP’s were modified to where the configuration reflected this:










The iSCSI arrays were now able to connect to each other across the WAN. However, another problem surfaced. We weren’t able to connect to the management side of the Nexus 5548UP’s consistently.

Remote Management Woes

While administration of the Nexus 7010’s was in-band via IP addresses assigned to SVI’s, management access to the Nexus 5548UP’s is only possible via the management interface itself. The problem is that this particular client doesn’t have an out of band dedicated management network. These Nexus 7010 switches and Nexus 5548UP switches are going to be the only switches left in place at this data center once everything gets migrated off of the other vendor switches and onto the FEX 2248’s and FEX 2232. Due to that, the gateway for the Nexus 5548UP’s resides on the Nexus 7010’s. Normally, this wouldn’t be that big of a problem. Just plug the management interfaces into a free port on the Nexus 7010’s and be done with it. There’s just one small problem with that. These 7010’s only have 10Gig interfaces on them. Checking Amazon again, this module for 48x1Gbps copper connections is listed at $27,000. Factor in your discount, but even at 50% off, that’s still over 10k per Nexus 7010, and we would need 2 of them.

In order to get the management interfaces connected to the Nexus 7010’s for layer 3 reachability for remote management, they had to connect to a third party switch. We used a Cisco 3560G switch since it was on hand, and had copper and fiber capabilities. The management interfaces from the Nexus 5548UP’s connected into the copper Gig ports on the 3560G. The 3560G connected to the Nexus 5548UP’s via the 1Gig SFP slots. Then, the management traffic would ride the normal 10Gbps uplinks from the 5548UP’s to the 7010’s. Simple right? 😉 The black lines below represent regular links between the 7010’s and the 5548UP’s. They also represent the fiber connections from the 3560G to the 5548UP’s. The red lines are the management interface connections from the 5548UP’s to the 3560G switch. The green lines are the management interface connections from the 7010’s to the 3560G switch.


Back to the problem at hand. We could reach the Nexus 7010’s from across the WAN with no problems. We weren’t able to reach the Nexus 5548UP’s from across the WAN or from the local Nexus 7010’s. Additionally, remote access to the 3560 was pretty much non-existent. From the console port of the 3560, I could connect to both 5548UP’s. From the 5548UP’s and the 3560, Spanning-Tree looked fine. This was an MST environment, so spanning-tree was pretty simple to troubleshoot. We found that if we disconnected one of the uplinks from the 3560G SFP port to a 5548UP, the problem went away. Reconnect the second uplink and the problem reappeared. The inter-switch links connecting the 3560G with the 5548UP’s were setup as access ports. The only VLAN flowing across those links was the management VLAN that the 3560G and the 5548UP’s were using for remote device administration. Also, in the case of the 5548UP’s, the management interfaces were also where the vPC keepalive link was setup. For the 7010’s, I just needed 2 IP’s that could talk to each other through the 3560G switch. I followed Chris Marget’s suggestion and used 169.254.1.X addresses for the 7010 management IP’s. These IP’s were only used for the vPC keepalive link.

Testing went on for a bit and then the client made a suggestion that ultimately fixed it. He asked if we could setup both uplinks from the 3560 as a vPC link. Of course! The 5548UP’s were reconfigured for vPC on the interfaces connecting to the 3560. The 3560’s SFP interfaces were configured as a standard port-channel as well. The problem went away after that. The final management connectivity looked like the following:



A few questions regarding the Nexus line the perhaps someone can answer:

  1. Is it assumed that organizations deploying Cisco Nexus switches are going to have a dedicated management network?
  2. Will the Nexus 7000’s ever support dual connections to Nexus 2000 FEX’s? Were that feature available, no 5548UP’s would have been purchased for this particular implementation.
  3. Is there ever going to be a comprehensive Nexus 7000/5000/2000 design guide? Or, is it buried somewhere deep within the Cisco website and I just missed it?
  4. On a lighter note(no pun intended), why in the world are the Nexus 2000 FEX’s so heavy if they don’t even do local switching?



Posted in cisco, data center, nexus, switching | Tagged , , , | 10 Comments

Inside Wireless Hardware – Part 2

In part one of this series, I looked at wireless access points from Meraki and Ruckus. I also looked at a controller from Ruckus and an old Linksys wireless card. In part two, I am going to look at AP’s from Aerohive, Aruba, and Cisco. This post has a lot more photos in it! As always, you can click on the photo to get a full size image.

Aerohive 120 Access  Point

I didn’t take a bunch of photos of this particular AP because it is their lower end model. What I did want to show was the antenna position. It is a dual radio AP capable of running 2.4GHz and 5GHz simultaneously.








The larger antennas connected with the red and gray wires are for the 2.4GHz radios. The smaller antennas connected with the black and blue wires are for the 5GHz wires.

























Aerohive 330 Access Point

Here’s what the 330 looks like before I opened it up. This is a 3×3:3 indoor AP.






You take off the cover and you see the antenna array. Look familiar? It is the same pattern as the Meraki MR24.








Here’s a closer shot of the antenna array with the manufacturer listed.















Here’s a shot of the TPM chip.








When you see me holding the AP in pictures, it is because the other side of the circuit board is holding the antennas and I can’t set it down without bending those delicate metal pieces.

Flip the AP over and you can see the radios on the bottom right and the CPU on the bottom left.. They are covered, but we’ll take care of that in a second. 🙂






Here is the processor.








Here is one of the radios. It has an Atheros chipset(which is at the bottom of the photo), but this particular AP has separate PA’s and LNA’s. Since it is a 3×3:3, there are 3 of each component.








Here’s a close up of the radios. The 5GHz is on the left and the 2.4GHz is on the right.






A zoomed out view of the radios. You can see the Ethernet, console, and power ports in the middle.






Here’s a better view of the ports.








As best I can tell, this is flash memory.








And this appears to be DRAM.








Finally, a few shots of random components.
















Aruba Instant IAP-135

This is a 3×3:3 AP that can function as a virtual controller and manage up to 16 AP’s. It can also be converted to function as a normal lightweight AP.








One thing you notice right away is that this AP is not light. When you take the plastic covering off you see why. It reminds me of a giant heatsink.








Here’s some closer shots of the antennas. Remember that the smaller antennas are for 5GHz and the larger ones are for 2.4GHz.























Here’s what the other side of the AP looks like:








Let’s pull off that metal cover and see what is underneath! It looks like we found the processor. It is a Marvell 88F6560 according to the numbers on the chip. The processor is the silver looking chip in the middle. To the left of it is another Marvell chip. My guess is that it is a Gigabit Ethernet transceiver. Similar model numbers are found in the Marvell Alaska line here.








The 2 small black chips appear to be Samsung SDRAM. Not sure how much in each one though.








Here are some more chips on the same side, just outside the metal enclosure area. The MXIC chip is some sort of flash memory chip. Not sure what the Atmel chip is next to it.








One last pic for this side of the AP. Here is a close up of the chips next to the Ethernet and console interfaces.








Flip the AP over and you can see where the radios themselves are.








Unscrew the containing bracket and you can then pop out the individual radios. Here’s what they look like by themselves. This is the Atheros AR9390 chip.








With both radios removed, not much is left underneath.








There is a single chip from Texas Instruments, but I am not sure what it does. Searching by part number was inconclusive.








The Aruba 135 was by far my favorite AP to take apart! Lots of interesting things. I especially liked the pop out radios.

 Cisco 1142 Access Point

If there was one word I would use to describe this AP, it would be “sturdy”. It was a little heavy, but that’s because the back side of the AP is metal. The front side surrounding the center piece with the LED and Cisco logo is plastic, but not cheap plastic. It felt like a quality piece of hardware and I didn’t feel like I was going to break it while disassembling it. I can’t say the same for some of the other AP’s.








Let’s open her up and see what is inside!

With the top off, you can see the six antennas. The three on the left are the 2.4GHz antennas, and the three on the right are the 5GHz antennas.






Here are some closer shots of the antennas.

























The part covering the antennas is plastic. That would make sense as the rest of the AP is metal. 🙂






Look closer at who made this plastic component. Does that name ring a bell?






Flip the AP over and you can see the wires connected to the antenna. I disconnected them from the radios in order to take this picture. Otherwise, you wouldn’t be able to see much of the guts of the AP.








The insides of the AP is basically comprised of two separate boards. One houses the radios and the other has the processor, memory, etc. Here’s a picture of both of them. The board housing the radios is in the back and is wider than the main circuit board.






Here’s the main circuit board disconnected from the radio board. You can see the CPU in the middle.








A closer shot of the CPU. You can find the specifications on the CPU here.








A closeup of the Broadcom gigabit transceiver.








In case you were wondering, Tyco(now TE Connectivity) makes the Ethernet port.








My guess is these 2 Samsung chips are SDRAM or some other sort of memory.








I think this is some type of Intel memory.








Some additional random circuit board shots.















Here is the radio board. Unfortunately, the metal covers over the chips were sealed. I could not pull them off without breaking things and I didn’t want to destroy a working AP. Perhaps when this 1142 is dead, I will tear into it, but for now, it must remain a mystery what is underneath those metal covers.






Here it is with the antennas disconnected.






Finally, here is the other side of the radio board.






Closing Thoughts

A lot of what I saw when I took these devices apart was somewhat of a mystery to me. It led me down many rabbit holes on chip distributor websites. Most of these sites had some Chinese language displayed, which should give one an idea of where all the components are being manufactured. Admittedly, I have a LOT to learn about the how and why when it comes to AP hardware. However, being a visual person, simply taking these things apart gave me a better understanding how things work from a layer 1 perspective. I have some additional hardware to take apart and look at, but none of it will be wireless, unless you count the Cisco 871W router I have. The wireless piece in that platform is less than exciting.

Disclaimer: As a delegate for Gestalt IT’s Wireless Field Day 2, I received the Aerohive AP 120, and the Aruba AP 135 for my own personal use. I also happen to be employed by an Aerohive and Cisco partner, so taking apart their gear will only help me in my future engineering endeavors supporting/installing/selling their gear.

Posted in hardware, wireless | Tagged , | 1 Comment

Inside Wireless Hardware – Part 1

This post is about me taking wireless hardware apart and taking pictures of it. I’m not going to bore you with stories of me as a child taking things apart, because to be quite honest, I wasn’t that kid. I was too busy playing with GI Joe and Star Wars toys to be bothered with the mechanics of how the radio or television worked.

These days, I am fascinated with how software and hardware work. I may not fully understand everything in these pictures shown below, but the more I take things apart, the closer I get to understanding the “how” and the “why” behind each particular product. I should also point out that I will only go so far with my hardware. Every access point I took apart was in working order. I didn’t want to risk damaging any of them so some hardware isn’t going to be as detailed with the pictures as others.

For part 1, I took apart the following:

1. An old Linksys 802.11b PCMCIA wireless card.

2. A Ruckus ZoneFlex 7962 access point.

3. A Ruckus ZoneDirector 1100 wireless LAN controller.

4. A Meraki MR16 access point.

5. As a bonus, I have some pics from a Meraki MR24 access point courtesy of my visit to their headquarters in January of this year.

In part 2, I have a few more access points from Aerohive, Aruba, and Cisco.

You can click on any of the images below to get a larger image displayed.

Linksys 802.11b PCMCIA Card

I bought this card back in 2004 and used it for a few years.






Once you remove the outer casing, you can see the few chips that are connected to the circuit board. The antenna is also visible on the end of the card.








Here’s a closeup shot of the antenna.








Ruckus ZoneFlex 7962 Access Point

You have to hold this thing in your hands to truly appreciate how big and heavy this AP is. From the top looking down, you can see the antenna array that Ruckus has built. This particular model is a 3×3:2 configuration.






A closer shot of the array.






Here is a picture of the connections going back to the radios themselves.








Here is some ribbon cable that connects the antenna array to the AP main circuit board.






The Chief Wireless Architect and co-founder of Ruckus, Victor Shtrom, has a motto of “Git-Er-Done” according to his bio page on the Ruckus website. Interestingly enough, that logo appears on the AP itself.








Flip it over, and you can see the massive heat sink on the bottom.








While there were more components underneath the array itself, I didn’t want to disconnect any of the components out of fear of not being able to reassemble it properly.

Ruckus ZoneDirector 1100 WLAN Controller

I did manage to completely take apart the controller that manages the AP. The interesting thing about the controller was how little components that were actually inside the box. I expected a lot more stuff to be present on the main circuit board. This particular controller can support up to 50 access points.

Here’s what it looks like assembled.






With the top removed.








I can only assume that this is flash based storage since it looks like a regular USB interface. The memory portion is glued on to the interface, so I didn’t want to remove it.








A closeup of the Atheros chip.








This looks to be a battery.








More chips.















Meraki MR-16 Access Point

What struck me the most about the Meraki AP was how light it was. Compared to Aerohive, Aruba, Cisco, HP, Meru, and Ruckus AP’s I have held, it was by far the lightest.

Here’s what it looks like assembled.








The overall view with the cover off. The long silver metallic piece on the right covers additional chips and/or the actual radios. This was fairly standard in most of the AP’s I took apart. Since this AP is in working condition, I didn’t want to pry off the cover and risk damaging anything. I am not sure why they do this. There might be a legitimate reason behind it, but I haven’t done enough research or talked to enough RF gurus to figure out why.








What looks like a missing corner is actually where the power and Ethernet connection are located. It is most likely done to keep the footprint of the AP to a minimum. Here is what it looks like on the underside of the AP.






Here is a closeup of the Atheros chip. This seems to be a commonality among a lot of these AP’s, no matter the vendor. Much like Broadcom on the wired side.








More chips from the MR-16 AP. These particular ones from Etron Technology appear to be SDRAM chips according to this.















Here’s the top down view of the 5GHz antenna on the MR-16 AP.








Here is a side view of the 2.4GHz antenna.








Remember those internal antennas. You’ll see very similar ones on the Aerohive and Aruba AP’s in the next segment.

Meraki MR24 Access Point

Although I didn’t take the next AP apart, I managed to snap a few pictures of the Meraki MR-24 AP when I was at their headquarters for the Gestalt IT Wireless Field Day 2 event back in January.








This is Meraki’s top of the line indoor AP featuring dual-radio 3×3:3 capabilities. The pic on the left is one I took. The one on the right showing the antenna layout came from Meraki’s website. That circular pattern of 6 internal antennas is the same one in use by the Aerohive 330AP, and I would guess other similar AP’s.








That’s all for part 1. Part 2 will have pics from 3 other vendors(Aerohive, Aruba, Cisco). Even if you can’t make sense of all the components(I certainly can’t…….yet.), it is still neat to see what is on the inside of things we use on a regular basis and probably take for granted. If you like these kinds of pictures(and who wouldn’t?), check out George Stefanick’s post from 2010 on Cisco wireless products at his site here.

Disclaimer: As a delegate for Gestalt IT’s Wireless Field Day 2, I received the Ruckus ZoneFlex 7962 AP, ZoneDirector 1100 controller, and the Meraki MR16 AP for my own personal use.

Posted in hardware, wireless | Tagged , | 5 Comments

Will The Future Include Feature Licenses?

One of the challenges(I mean that in a good way.) of working at a VAR that sells for so many vendors is keeping track of the various solutions. If it were as simple as being familiar with the products from a technical perspective only, that would be great. However, anyone who has ever built a bill of materials knows that knowing the products alone isn’t enough. You have to know how they are licensed as well. Multiply that times a dozen or so and you can see my dilemma.

As my company continues to partner with vendors, I am always interested in a few things. First, I want to know how I can get access to their product without my company having to fork over a pile of money for “lab/demo” gear. Second, I want to know what differentiates them from their competitors. Third, I want to know how they license their product.

Although licensing is the least exciting of the three things I just mentioned, it can make a difference in winning deals with customers.

Are Feature Licenses A Big Deal?

Yes and no. For some companies or individuals, they will always pick a certain vendor no matter the cost. You are not going to change their mind and you are not going to be able to reason with them. They have their reasons whether you agree with them or not. For a lot of other organizations/people, cost and simplicity are important.

I could go on and on about why I think cost should NEVER be the main reason for a technology purchase, but the reality of this business is that for some, that will be the main driver. You can argue things on principle, but when it comes down to it, someone else will get the sale if you don’t give the customer what they want.


Let’s make one thing clear. Building decent technology isn’t cheap. It costs lots of money from a development standpoint. Even if you buy your technology through acquisitions, it still costs lots of money. Then you have to market it and support it. Those are additional costs and they aren’t cheap either.

Here I sit on the VAR side pitching a solution to a customer. They need the product to do X, Y, and Z. There’s quite a few vendors that can do X, Y, and Z, but each of those things require a different license. That means more line items on the quote. Now the customer has to make some tough choices. They can either scale back some of the features, or pick another vendor. Scaling back features means the solution now does less but hopefully meets their price point. Picking another vendor means the original vendor of choice just lost out on a sale.

Let’s say the customer reduces the features on the first solution. Maybe they wait a year and add the missing feature when the next budget allocation rolls around. Alternatively, maybe they resent the vendor for charging more for a feature that another vendor is giving away.

What if the customer decides to go with another vendor that offers similar functionality but for significantly less cost? Maybe they dismissed that vendor initially due to them not being familiar with them. Perhaps the first choice vendor is better at marketing. It is also possible that the first choice vendor might even have better technology. Regardless, lots of purchasing decisions happen due to cost, or the appearance of additional cost. Sometimes bundles are a good thing. People like to think they are getting a deal and bundles usually convey that image, whether the savings is true or not.

A Cost Hypothetical

How many more Nexus 7000’s would Cisco sell if OTV was free? Instead, there is a Transport Services license you have to buy for each 7000 you own to turn OTV on. Add on the Advanced Services license for each of your Nexus 7000’s since you’ll want to create another VDC for OTV. Watch the cost go up. You already bought it with the Enterprise Services license since the Base license doesn’t do much. What’s two more licenses in the grand scheme of things? Yes, I realize there are bundles you can buy, but it still costs more than a plain ole’ Nexus 7000. The OTV license give away idea isn’t mine. I stole it from @ccie22126!

When considering the complete cost of OTV, plenty of customers probably looked at other vendors, even though nobody else can offer what OTV does(that I know of). Yes, I realize it is proprietary, but I don’t think you can really compare it to something like running EIGRP instead of OSPF as your IGP of choice.

Other vendors are giving away all, or almost all, of the features of their platforms as a way to compete with the companies that nail you with the license tax for every additional feature you want. At some point, they will gain more traction with the cost conscious buyers. If all I need is a switch with a decent amount of 10Gbps ports and OSPF/BGP capability, how hard will it be to find a vendor who will do it cheaper with less licensing costs? Off the top of my head, I can name a handful.


Let’s pretend that cost doesn’t matter. There’s something to be said for simplicity. If a solution from one vendor requires a ton of feature licenses and the solution from another vendor requires none, how will those quotes stack up? How many of us have spent hours upon hours going over quotes making sure every license was accounted for? The last thing you want to do is sell a solution and then have to go back to the customer for more money because you forgot about some obscure license you forgot to include in the quote. The last thing you want as a customer is to have to pay more money for a solution you thought you already owned. The last thing you want as a vendor is for the customer to hate your guts and count down the days until your product is depreciated 100% off the books and they can get rid of it.

It’s an amazing thing to be able to quote something and have it take up a whopping 2 or 3 lines versus a solution from another vendor that requires a full page. I think customers appreciate that because they can understand it a lot easier than a billion different line items that may or may not contain every thing they need. I also realize that the network infrastructure side has it easy compared to some of the licensing issues you have to deal with on the systems side. I’ve seen Microsoft employees get confused over their own licensing requirements. That’s when you know you have a problem.

With most companies being understaffed(I have no hard numbers to support this. Just a lot of complaining from people on Twitter and my own personal observations out in the world.) when it comes to IT, the easier you can make it on them, the better. That doesn’t necessarily mean it is cheaper. You can include a fair amount of features in a product and still charge a premium for it. There are companies out there who are willing to pay for a particular product, provided it does what it says it can. Well, even if it can’t do what it says, some companies will still buy it.

Closing Thoughts

Some vendors are changing their licensing for the better. HP is giving away pretty much everything on the networking side. Brocade has some pretty straightforward licensing on the Ethernet side as well. Cisco brought about a welcomed change to licensing on their second generation ISR line with IOS 15 by greatly simplifying the feature set choices.

Other vendors just give it away “whole hog” as we would say here in Tennessee….or at least I do. A10 Networks gives you pretty much all functionality with their load balancers and includes global capabilities along with local(For all you F5 fans, that’s essentially GTM and LTM.). Barracuda gives you everything but AV updates and VPN/NAC with their NextGen firewall. Aerohive gives you everything with their AP’s. Meraki does the same with their AP’s and switches, but not with their MX firewalls. There are others, but I think you get the drift.

My gut tells me that feature licenses are going to be scaled down significantly in the next decade or so. It is a problem that I have seen voiced to vendors on numerous occasions. Whether or not the vendors feel the pain in their market share numbers or revenue will dictate whether or not they change. If everyone is buying and market share is growing or holding steady, I just wasted an hour or so of my night writing this. 🙂 I’d be curious to hear your thoughts on this whether they be from the vendor, VAR, or end user perspective.

Posted in vendors | Tagged | 4 Comments


I spend a lot of time pouring over vendor solutions. A lot. Unfortunately, I don’t specialize in one area like wireless, security, storage, voice, etc. I am more of a network generalist. Because of that, I tend to focus on the big companies or those who fill a certain niche. There are just too many vendors to keep track of. Because of that, I seem to have missed Meraki. Thank goodness for Wireless Field Day 2! For an in depth recap of the WFD2 visit to Meraki, read Tom Hollingsworth’s post here.

It’s not that I was completely ignorant of them. I’d seen some of their advertisements. I knew that they sold wireless gear. I knew that they had some switches and firewalls. Everything was managed by the “cloud”. I took all that into consideration and thought: “Yay. Another vendor shilling for the “cloud”. As if we didn’t have enough already.” That was my thoughts on Meraki coming into Wireless Field Day 2. I work for a VAR that sells Cisco, HP, Aerohive, Brocade, Juniper, and a ton of other vendors. We have a variety of wireless choices at our disposal. I can sell a product for almost any environment. In short, I can work with all sorts of “it depends” scenarios. That should help explain some of my skepticism.

That doesn’t mean I was opposed to hearing Meraki’s pitch. I LOVE talking to vendors no matter the size of the company or the particular area they focus on. I firmly believe that every vendor has something to offer. They ALL employ smart people.

So that’s where I was at on Thursday, January26th when we pulled up to the Meraki headquarters in San Francisco. I was a skeptical non-believer. We walked through their offices and it felt 100% like a tech startup. Lots of young people typing away on large screen Macs. Plenty of flair attached to desks of varying heights. A few dogs wandering around. Snacks galore in their break area(which also included a few kegs of beer). All the things that scream: “You want to come to work.” Nothing like the usual corporate environments where cubical walls run high, dress codes are enforced, and the only semblance of a corporate perk is free coffee or a meager selection of carbonated beverages. Still, I was skeptical. I’ve seen this before. It is replayed “ad nauseum” across the entire Silicon Valley landscape.

The more I listened to the people from Meraki talk, the more I began to change my mind about them. We all asked a fair amount of questions. They answered them with a fair amount of candor. They also did a demonstration of their management platform. During this demonstration, I couldn’t help but think how polished the management interface looked. It was full of features, yet didn’t seem cluttered. It wasn’t a canned demo either. Below is a video of Pablo Estrada from Meraki walking us through the wireless product. He also took a little time showing us the switch and firewall management piece as well.

Live demonstrations of Meraki Wi-Fi gear with Pablo Estrada from Stephen Foskett on Vimeo.

In addition to the extensive demonstration that Pablo did, one of the delegates from WFD2, Daniel Cybulskie, put together an article and video on Meraki’s Device Manager product. You can access it here.






Their wireless product set is very easy to understand. A few AP models for indoor and a few for outside. They have all the major features other vendors have. WIPS, RTLS, spectrum analysis, RRM, 802.1x, device fingerprinting, etc. Everything is managed through the MCC(Meraki Cloud Controller). After our initial discussions at the Meraki headquarters, it seemed as if their controller wasn’t very controller-like at all. If you lost your connection to the MCC(ie the Internet), you could still authenticate new users provided your RADIUS box was local. Your captive portal would also continue working for guest access provided you hosted that locally. With a lot of the other solutions out there, this wasn’t the case. If you lost your link to the controller, you could still function, but no new client connections could happen.

Could this be another controller-less solution? After all, the MCC seemed to be more management plane oriented and less control plane oriented. Then, the mystery was solved. RRM didn’t work 100% without the connection to the MCC. In effect, layer 3 roaming was dead without the MCC link. However, we were informed that Meraki is working on making that functionality available locally(read the “Comments” section in this link). At some point in the future, you can lose the MCC link and still be able to roam via layer 3. I suspect this isn’t a big deal in some of the areas Meraki is prevalent in(ie retail), but can be an issue in the larger networks like educational institutions, hospitality, etc. Once the layer 3 roaming piece is worked out, Meraki will be a lot closer to the controller-less solution that Aerohive has been alone in for the past couple of years. Ideally, most decent sized networks will have redundant Internet connections. You could even buy a cheap DSL or cable modem circuit and only use it for backup connectivity to the MCC should your primary circuit fail. DSL and cable are cheap compared to a full blown redundant Internet circuit for a medium to large network. I am not entirely sure of the bandwidth requirements, but I can’t imagine they are huge given the fact that all data plane traffic stays local and quite a few of the control plane functions can stay local as well.






But Wait……..There’s More!

If one were to simply look at the wireless portion of Meraki, you would miss out. As of today, they are selling their own wired switches and firewalls. This is where I think they start to differentiate themselves from other companies even more. These devices are also managed over the Internet. The switches are basically 24 or 48 x 1Gbps interfaces. All wire speed/non-blocking. You can get them with or without PoE and they support 802.3af and 802.3at for your devices requiring a bit more power. Oh, and all ports can provide power at once if need be, but the limitation on the switch(24 and 48 port) is 380W in total.

There are some additional limitations on the switching side. They do provide 10Gbps uplinks from the switches, but it appears they are only doing so with short-haul multi-mode optics. I suspect this is because most of their customers are going to use short haul optics if any optics at all. In my opinion, these switches are not going to be dropped into a network to run as a core or distribution switch unless it is a rather small environment.






Yes. There’s Still More!

Not to be content with basic wired/wireless products, Meraki also has a decent firewall/UTM/router/call-it-what-you-will offering. Ranging from 100Mbps to 2Gbps stateful firewall throughput, they have a box that can meet most organization’s needs at least from a throughput perspective. As with any other vendor, when you spin up VPN connectivity, throughput drops like a rock. I suspect other features cause a performance hit as well, but those numbers aren’t displayed. That isn’t unique to Meraki. No other vendor that I know of will openly tell you how poorly their box runs when you turn all the features on. Why would they? Marketing isn’t in the business of highlighting the negatives. 🙂

A firewall is a firewall right? Not these days. If you want to differentiate yourself from the others, you need to have some serious application intelligence. A big part of what makes PaloAlto Networks a good firewall vendor is their ability to understand a ton of applications and affect throughput based on that understanding instead of the usual IP address and port access rules. Barracuda Networks can do this too with their Next Generation firewall as does Check Point with their latest and greatest offering. The Meraki firewall is able to do substantial application recognition and when paired with device fingerprinting, you can give users an experience that they will either love you for, or hate you for. In short, you have some serious options. Allow iPads to access Facebook, but prohibit laptops. Those kinds of things.

Apparently, that wasn’t enough for Meraki, so they took their firewalls to the next level. They included content filtering, routing, WAN optimization, link bonding, and automatic VPN creation.

The VPN portion looks to be pretty straightforward. They can create IPsec tunnels between your remote sites with a single click. I’ve built massive numbers of VPN connections over the years. However, I have never built one with a single click. Obviously, since all these devices are managed over the Internet, it can send down the appropriate phase1/2 parameters that you normally enter manually on each end of the connection. Very quick and efficient provided you are using Meraki gear on both sides.

The link bonding is really just a way of aggregating dissimilar connections so that you can perform basic load balancing and also have failover without having to employ a large amount of devices to support it all. Barracuda Networks has a similar product. This isn’t a service that a large enterprise customer would necessarily employ as they are going to use redundant hardware and other mechanisms to control the flow of traffic. I can see the benefit of this for the medium to smaller customers. Especially considering it doesn’t cost you any more money. In fact, the only increased licensing charge appears to be for content filtering, client VPN connectivity, and anti-virus.

I saw a bit of the content filtering during the product demo at the Meraki headquarters. It appears to be fairly in depth like most decent content filters are. You can filter by category, end user, etc. Nothing too exciting there, but a nice feature to have considering other vendors have separate appliances you have to install for this type of service.

The one thing out of all of these features that caught my eye was the WAN optimization piece. The low end MX60 has 100MB of cache. The other models all have 1TB of cache on a SATA drive with the exception of the large MX600. It has 4TB in a RAID configuration. That’s a lot of space for WAN optimization cache. If you look at Riverbed Steelheads, you have to get the 5050H to even get 800GB of cache storage. To get 4TB or more with Riverbed, you have to buy the largest box they sell, the 7050M. I can assure you that box costs 4 times or more of the price of the MX600. I would be willing to bet the Riverbed Steelheads will outperform the Meraki MX boxes from a WAN optimization perspective, but considering you get the WAN optimization for no additional fee, it would be worthwhile to use it even if the gain was minimal. Meraki states you can get up to 99% intersite bandwidth reduction using their WAN optimization and that you can accelerate CIFS, FTP, HTTP, and TCP traffic by up to 209 times the non-accelerated rate. Of course, that all depends on a variety of factors. I wouldn’t count on seeing those kinds of performance numbers outside of pristine lab scenarios, but again, it’s WAN optimization without the price tag of the larger vendors.

That covers the 3 main Meraki lines. Wireless, wired, and security. A product set that is easy to remember. Now how about pricing? What does a Meraki network cost? Thankfully, you can find that out before you even pick up the phone or send out an e-mail to their sales staff. They actually have a fairly decent pricing tool right on their web site. The cost for each AP, switch, and MX firewall is shown along with licensing, maintenance, etc. This is something I wish more vendors would do as it shows the customer right up front what their costs are going to be. If they get it cheaper, then even better, but at least they know how much they should budget for initially. What makes the pricing tool even more interesting is that they compare it to the cost of comparable Cisco products. They actually give the corresponding Cisco part number and the associated cost.








The prices are all list, but you can add the appropriate discount into the calculation to get more realistic numbers, because who pays list? 😉

Closing Thoughts

Like a lot of companies out in the Silicon Valley and Bay Area, Meraki employs a fair amount of smart and talented people. Using commodity hardware and some decent software, they appear to have built a nice product set with a management model that is appealing to your small to medium environments. The questions that I find myself asking are where they fit and don’t fit. What environments would they thrive in? Certainly the heavily distributed environments that survive off a single Internet connection are applicable. Retail immediately comes to mind. I don’t doubt their wireless can scale, but until they can handle layer 3 roaming without a link to the MCC, I will remain cautious. This isn’t a controller/controller-less argument on my part. It’s just that any time you rely on something off site to serve a critical function, you better make sure you have more than one way to connect to it.  As for the wired switches, I think they would be fine on the edge. Will they build more capable switching platforms in the future? I am guessing that they will. Hard-wired switches don’t have the same dependency on the dedicated link to a management platform like the wireless solution does. As for the security solutions, they appear to be able to scale up to the large network size. Perhaps some in depth testing of the MX platforms would paint a different picture.

As of today, I have a much different opinion of Meraki compared to just a month ago. I like what they are doing with the concept of “cloud” based management of their hardware. It makes it easier to sell to certain organizations that want to run lean IT shops. The Meraki product is easy to install and easy to use, based on my own testing of one of their wireless AP’s and the corresponding management console. Their products are not for everyone. Certain organizations are not going to want to go the route of “cloud” managed hardware. They also don’t have a ton of different hardware options that some organizations will want. That’s the risk you take when running with a model such as theirs. I do think that we’ll see more and more interest in companies like Meraki as IT resources become scarce in the years to come. By resources, I mean people. I look forward to seeing what they will come up with in the future.

Disclaimer: As a delegate for Wireless Field Day 2, my travel, lodging, and meals were paid for by a number of vendors including Meraki. I also received some items from Meraki with their logo on it along with an MR16 access point to keep for testing or personal use. None of this was done with the expectation that I would write or say anything about them.


Posted in cloud, security, switching, vendors, wireless | Tagged , , , | 8 Comments

Where Packets Never Die

I talked with a fairly large service provider yesterday regarding quality of service(QoS) possibilities for a particular network. Essentially, we were trying to figure out what was available from a queues perspective so that we could make sure we were able to classify traffic as effectively as possible.

I had been told that the existing QoS policy was defined as a 75/24 ratio with 3 queues. I didn’t think that could be right. Who runs 75% EF, 24% AF, and 1% BE on their network? Every other enterprise class network I had worked on had somewhere between 4 and 6 queues with 10% at the most dedicated to EF.

A phone call with this provider verified that they were in fact dedicating 75% to EF traffic. Once my brain processed that, I asked what other queue models were available. To my surprise, the following models were available:


Class(DSCP) Policy 1 Policy 2 Policy 3 Policy 4 Policy 5
EF(46) 75 75 90 50 25
AF41(34) 24 20 9 25 40
AF21(18) N/A N/A N/A 24 30
BE(0) 1 5 1 1 5


I didn’t see anything that made me jump for joy, but at least there were a couple of models that gave me the 4 queues I wanted. EF, per RFC 3246, is designed for the following:

“EF is intended to provide a building block for low delay, low jitter and low loss services by ensuring that the EF aggregate is served at a certain configured rate.”

That sounds a lot like voice traffic, but not too much like all the other types of traffic that need to traverse the WAN. You could consider putting video in the EF queue, but it has been my experience that you leave EF just for voice. Video tends to go in one of the AF buckets.

For the rest of the afternoon, I was trying to understand why a service provider would do something like this. I have to be honest and state that I have no idea. I’ve read a few things that mentioned service providers in other countries only having 3 queues for high, medium, and low traffic, so service providers in the US who have global networks try to match the lowest common denominator. In this case, I don’t know that we can make that assumption. I wasn’t aware of any service from this provider outside of the US.

Is this a case of “just because you can doesn’t mean you should”? Or, am I missing something here and not considering the bigger picture? I see an EF queue value of 90% and I think this must be the network sprinkled with unicorn dust where packets never die. Tail drop is non-existent and all packets fly first class. Nerdvana indeed.

Posted in data center, qos | Tagged , | 6 Comments

You Never Mentioned Me To The Client

I work for a company that partners with numerous vendors. That’s a big part of why I took the job with them. I wanted to work with a bunch of different platforms. It was important that I wasn’t just an extension of one or two vendors as can often happen in the VAR world.

Having been in the reseller or VAR space for half a year now, I am still struggling to understand the relationship that vendors have with their resellers. If a particular reseller only deals with one particular vendor, then what I am about to say might not be particularly valid. Since I work with a variety of different vendors with my current employer, I typically have 2 or more options available when a client needs new hardware or software.

Once we have established that a client actually NEEDS new hardware or software(the answer isn’t always to sell them something), we have several different avenues of approach. Here’s a short list of the vendors we resell for:

Blue Coat
Palo Alto

There are others, but I think the point is made. I usually have a lot of choices from a solution standpoint. The big question now becomes which vendor to use.

Bear with me for a second while I sidetrack…….

One of my friends from church has been debating on whether or not he should buy a MacBook Pro for business use. He is in sales and works out of his home. He spends a fair amount of time at various locations throughout the local region so he is always on the go. He needs a reliable laptop and just wants it to work with minimal administration on his part. He’s had an iPad for some time and enjoys that. He also switched recently from an Android based phone to an iPhone and loves the simplicity of the iOS based phone.

I kept telling him to just go out and get the MacBook, but he had a working Windows 7 laptop from Gateway and didn’t really want to spend the extra money for the MacBook Pro. Last week his Gateway laptop crashed. He immediately went out and bought the 15″ MacBook Pro. Since he was a lifetime Windows user, he called me up and asked me if I had some time to walk him through some of the basics. I told him to come by my house and we spent a few hours just going over some things. He LOVES his MacBook. Loves it. He can’t stop talking about how much he enjoys the user experience of it from the display to the little hardware extras. He considers it to be a quality product and is glad he made the switch.

I say all of that not to promote the Apple product line. We each make our choices based on a number of factors, so I realize the Apple cult isn’t for everyone. However, I told that story to illustrate the point that sometimes it takes some hands on experience with a product before someone becomes fully converted. Until he sat down and spent a few hours with the product, he wasn’t completely sold. Once he did, he was like a kid in a candy store. He will go out and evangelize for Apple and never see a dime from Apple for that. He’ll do it because he loves the product.

Comfort Selling

I consider myself to be a fairly independent person when it comes to IT vendors. I like products from a  variety of different vendors. As independent as I try to be, I have to be honest. I like certain products more than others because I am intimately familiar with them. I can answer all sorts of questions about certain products because I have used them time and time again. I might even speak passionately about a particular vendor based on positive experiences I have had with one or more of their products. If you don’t think that affects the solutions I am pitching to clients, then you’re crazy. I’m not saying it isn’t possible to sell something with just an “academic” understanding of what the product does. I’m just saying that it makes it that much harder if you haven’t used the product more than a handful of times.

The Problem As I See It

Tell me if this makes sense. Vendor X wants me to sell their product instead of their competitor’s product which I am already familiar with. However, I have no access to vendor X’s product until I sell it to a client and install it in their network. I have a few technical documents available to read instead. Sound familiar for any of you in the VAR space? 🙂 Isn’t that like giving me the driver’s manual for a car but not letting me drive it before making a purchasing decision?

Wait a second, the vendor will say. You can buy your own gear for lab use from us at a deeply discounted rate. Hmmmm….. I need to spend hundreds or thousands of dollars in order to get your product so that I can get familiar with it? In all fairness, there are some vendors out there(Cisco, Juniper) who offer remote lab use of their equipment for partners. Cisco will even go so far as to let me come over to the local office and run proof of concepts on various hardware that I may not normally carry in my company lab(i.e. Nexus 7000 switches).

I also know that some vendors can temporarily provide you select equipment for demo/lab/proof of concept use. However, that isn’t always an easy process. Distributors sometimes have remote labs as well. Ease of gaining access will vary with the distributor.

And then there’s the communication……

Some vendors are very good about keeping you in the loop on the new products or any noteworthy news in general. Others, not so much.

Perhaps you are asking yourself “Why don’t you as the VAR take the initiative and equip yourself with all the tools needed to sell more of vendor X’s product?” My answer would be that it takes time and money. In my company’s case, we would have to do that 10 times over based on the number of vendors we are partnered with. The easy answer is to only sell for one or two vendors. The problem with that is if I really want to provide a valuable service for a client, I need to give them options. I can’t exactly do that with one or two vendors. Of course, that’s just my opinion.

There are other issues, but those are the big ones.

The Solution

This is a tough one because I struggle with how much of the problem should be solved by the reseller and how much of it should be solved by the vendor. Let me try and approach this from the easiest to the hardest. Also keep in mind that some companies don’t have these problems. Or, if they happen to be a larger vendor, they may only have these issues with a few of their products. I don’t want to sound like I am indicting every IT vendor, because I am not.

Communication – This goes beyond sending out a dozen e-mails every month to various partners. It involves face to face communication with your various partners from the vendor’s channel sales and engineering staff. Even if it is just an hour here or there to strategize and figure out how to better equip the partner to sell more of your product. Sometimes you just have to sit down and talk. That doesn’t mean someone has to buy the other lunch or spend any money at all. Perhaps just swinging by each other’s office on a semi-regular basis to have a chat will suffice. Phone calls work too.

Product Information – More is always better when it comes to product/technology information. Don’t tell me you don’t have this information. Any vendor worth their weight has loads of documentation on their products. There’s always proprietary stuff that the partner will never see. I understand that. I realize there is a need to protect that information. However, there’s a lot of other information that could be provided that will help in the pre-sales capacity.

Competitive Information – This one can be tricky. Some people think the competitive information is a bunch of marketing junk. I tend to look at it as another tool I can use to better understand competing products. There is always something in the competitor information that I overlooked. When vendors are forthright about their weaknesses and their strengths when comparing themselves to their competition, it makes the sale that much easier. Unless of course your product really is inferior junk.

You wouldn’t sell junk though would you? 😉 Of course not. Let’s take the competitive information to the next level. Show me a real tangible application of this. If you say your product is ten times easier to deploy, let me see it. If your product has a bunch of different features that the competition doesn’t have, then show me. Don’t leave it up to a slide deck or “battle card” to show me how much better you are. Demonstrate it to me. That doesn’t mean I need to always have your physical product in front of me to make the comparison, although that DOES help. You can do what other vendors have done and produce some decent quality videos walking through whatever features or differences you want to illustrate. The videos need to have substance though. Sometimes a whiteboard isn’t enough.

Hands On Time – Finally, and most importantly, engineers need to get hands on your equipment if you expect them to pitch it to customers. I am not sure it is realistic to expect vendors to ship their entire product line to resellers so they can build their own on site lab. I also don’t think it is realistic to expect resellers to always buy “discounted” products from vendors either. You want me to sell your product right? Make me a believer. I can get fairly passionate about products I have not used yet as long as the marketing information is good and detailed. I tend to get even more passionate about those products once I have touched them and seen them work as intended. Why do car dealerships want you to take a test drive? Same concept here.

Perhaps the best answer might be to offer remote access to equipment in a lab environment. Cisco and Juniper already do this. Other vendors do this as well. Even some of the distributors can give you remote lab access. Additionally, some vendors will let you come over to their local office and spend time in their on site lab testing things out. This is especially helpful when the device you need access to costs as much or more than a house.

Closing Thoughts

My job as an engineer working for a VAR is to provide solutions for clients. When those solutions require new hardware or software, I am going to suggest the solutions that my company believes in.  It’s easy to sell for market leaders. For the challengers, it takes a little more on the preparation side. If you are going to leverage the partner network to move your products, what lengths are you willing to go to in order to ensure they can make the sale? For some vendors, it doesn’t appear they are willing to do much. For others, they are willing to do a lot.

How bad do you want people to sell your product? If you put all of the load on the partner or distributor, with minimal contribution from the vendor side, don’t expect to get mentioned to clients. That’s not a partnership. It’s a pyramid scheme.

This is the part where you tell me I am wrong, naive, etc. I fully admit that I see things from my  engineer level and don’t always get the benefit of the big picture. Feel free to educate me in the comments below.

Posted in vendors | Tagged | 5 Comments

Why I Don’t Love Your Company Or Your Products.

I just returned home from Walt Disney World. Let me just state up front that I love Walt Disney World. I may not necessarily like all the moving parts of the greater Disney corporation, but from a vacation perspective, there is no finer place to take one’s family.

Was it expensive? Of course. Was it worth it? Absolutely. Let me tell you why.

Take a look at this picture:

Yep. That’s my little girl getting an autograph from the white rabbit and a hug from Alice. For those of you who aren’t familiar with them, they are two of the more notable characters from “Alice in Wonderland”. Notice the expression on “Alice’s” face. Eyes closed. Warm smile. Arms embracing a little girl she’s never even met. I should also point out that when she was speaking to my little girl, she never broke character. In fact, over 6 days in various Disney parks, none of the people playing various princesses and other roles EVER broke character. Believe me. I stood in plenty of lines. Endured several costly meals. Took plenty of pictures. I’d remember if one of them slipped up. 🙂 Ask yourself this. Is that a quality Disney employee in that picture? Could I pay a lot less at some other amusement park and get the same experience? Possibly, but more likely the answer is no.

Between the 11 of us that were there together, I have over 2000 pictures and videos culled from multiple smart phones and several hand-held cameras. That’s from 6 days in Disney parks. Of all those photos, this one struck me as the most representative of the Disney experience.

I could go on and on and bore you to death with story after story about the Disney employees and the quality experience they provide. I think you get my point though, so I will stop with the sole example shown above.

The first day we walked into the Magic Kingdom, I told my 8yr old son: “You are about to see what true customer service looks like. Don’t expect to see this many other places in the world.” What I didn’t tell him was how much money it took to experience that kind of customer service. 🙂

There are brands out there similar to Disney and not very many of them are inexpensive. Do any come to mind?

Here are some brands that I have used and I prefer from a quality product/experience(Feel free to disagree, but remember that sometimes the experience plays a large role in addition to the product.):

Barnes and Noble
Mellow Mushroom Pizza
SouthWest Airlines
Craftsman Tools
LL Bean

There’s a reason I like each of the brands mentioned above. There are also reasons I don’t particularly care for their competitors or just happen to be indifferent towards them. I have no problems spending money on any company’s products listed above. That doesn’t mean I like all of their products. You just won’t have to twist my arm as much as you would for a competitor’s product.

Finally. Something Technical…..Sort Of.

Here’s the part you care about….I hope. Think about IT companies out there. Any particular brands you love? I bet there are far more that you don’t love. Why don’t you love them? Why don’t you care about all the advertisements, marketing e-mails, webcast invites, etc?

For me, it is because those companies or products don’t give me a reason to care about them. There is no differentiation from everyone else. There is nothing special about them. Their marketing stinks. Their ability to execute stinks. All they can do is jump up and down like a spoiled little child demanding that you notice them, but when you do take time to notice them, there is absolutely nothing remarkable about them. Not a thing.

Whenever I get a chance to talk to a vendor, I always like to mention a competitor or two of theirs and then ask what makes them different. In my personal experience, the good companies will tell you straight up what makes them different. The bad ones will typically respond with: “You can’t really compare us to company XYZ because of blah, blah, blah.” They then proceed to trash their competition without ever really answering why they are different.

Maybe it just comes down to hiring good people. People that are “evangelistic” in their love of the company product will yield far greater results than people just trying to sell one more widget to make a quota. Can’t find any people like that?

Here’s a few tips:

  1. Pay your people more. – This means you charge more for your product. I get that. Build quality and people will pay more. Problem solved. Profit margins will still be intact.
  2. Develop your people. – I realize that goes against the modern corporate mantra, but if you actually treat people like humans instead of a number, they might surprise you. There are people who will actually go the extra mile for you if they know you are investing in them and their long-term career prospects. I’m not really referring to the executive that you send to MBA school or wherever else they go to learn the latest ways to inflate the stock price. I’m talking about the people that work underneath those executives. The ones that have to park their cars far away from the building.
  3. Hire weirdos. – Normal people should only be found in the accounting department. Every other department should be full of weirdos. Not social deviants. Weirdos. There’s a difference. If your employees WANT to come to work each day just to observe the freak show, you are doing it right. Bonus points if they don’t figure out that they ARE the freak show.

Keep in mind that number 3 only applies to IT companies. Weirdos don’t fare well in sectors like banking, healthcare, and transportation.

Closing Thoughts

Somewhere in this disjointed post I had a point. I believe it was simply that I don’t love a lot of IT companies or their products because they just don’t give me a reason to. I’d love to hear your reasons for why you do love certain products or vendors.

Posted in vendors | Tagged | 19 Comments