By Alastair Hartrup, CEO of Network Critical  

Network speeds continue to increase as business and life rely more and more heavily on digital components to function. Applications are controlling HR, sales, back office functions, supply chain management and many other business-critical elements. Social media and online video advertising are indispensable marketing tools generating increased traffic. Cloud and hybrid-cloud architectures are often leveraged to improve business scalability and agility. As a result, network bandwidth must expand rapidly in order to keep up with increasing demands.

According to a September 2018 Forbes report, enterprise interconnection bandwidth with network providers is expected to grow at a 42% CAGR between 2017 and 2021. Furthermore, the interconnection of enterprise with cloud and IT providers is expected to grow at a 98% CAGR over the same period. All of this growth is making it more and more challenging for IT to properly monitor and secure the network.

There are many specialized network monitoring, performance and security tools on the market that are embedded in enterprise and network provider networks around the globe. These monitoring tools are traditionally designed to operate efficiently at a fixed capacity. Therein lies the problem. Once that capacity is reached, new tools must be introduced to manage additional traffic. Simply adding tools is acceptable as long as the link speed does not exceed the tool capacity. When adding links to the network at 1 Gbps, simply connect a new 1 Gbps tool to monitor the link. But, what happens when new links are 25 Gbps, 40 Gbps or 100 Gbps? This creates a major capacity problem! And, according to a Crehan Research, Inc. report, 25 Gbps and 100 Gbps links will comprise over half of Ethernet switch shipments by 2021. 

Unfortunately, many monitoring tools struggle to keep up with 10 Gbps rates. These tools must capture, open, inspect and report on link traffic. If the tool is overwhelmed with traffic, packets are dropped, and performance and monitoring capabilities suffer. There are two ways to solve the tool overload problem. One is to send less data to the tool (i.e. with filtering) and other is to find a way to use multiple tools on the same link (load balancing).

Let's explore how advanced packet broker technologies can help you optimize network monitoring using these two approaches:

Filtering - Depending on what information you need, monitoring tools don't always need to see all the packets all the time. Packet brokers allow you to write and implement filter rules that determine what data actually is passed on to a monitoring tool. For example, if the tool is only monitoring http traffic (because your organization is interested in performance between browser and network, for example), it is not necessary to send other traffic (such as UDP, FTP or email protocols). You can set packet broker rules that ensure they only pass http traffic to the tool, thereby filtering out all other traffic so that amount of traffic actually sent to the monitor from a high-speed link may well be within the processing capacity of the monitoring tool.

Due to the fact that you can modify these filters with relative ease, you can make the necessary adjustments to ensure that your monitoring tools accurately provide reports within their processing capabilities. One shortcoming of the filtering approach, however, is that traffic monitoring tools sometimes need to process all the data on a link. In this situation, filtering is not an option.

Load Balancing - Based on a sophisticated algorithm, load balancing functions evenly distribute traffic from an incoming link to multiple outgoing links. Simply put, a high-speed link at 40 Gbps can be sending traffic from the network into a packet broker, which will then evenly distribute that traffic to a pre-determined number of monitoring tools to which it is also connected. Assuming that there's an NPMD solution, and several other connected monitoring tools, each with a maximum processing capacity of 8Gbps, the aggregate capacity of the five tools equals 40 Gbps. 

The unfortunate problem packet brokers will experience from time to time is knowing how much data to send to each tool and when to move on to the next port. To that end, there are different methods you can use to manage how and when packets are sent to your tools.  For example, it might be important for an entire conversation to be sent to each tool rather than random bits of data equally distributed among all tools. This allows the monitoring tool to accurately monitor the entire packet - which includes the source, destination and the payload in between. Another option may use a "round robin" method, sending data to one port until it is full, then moving to the next port. There are many different approaches you can take to manage packet distribution, but the most important thing is to start by fully understanding what your monitoring needs are in the first place. Load balancing can help you more effectively utilize network monitoring tools that might struggle to keep up with increasing network speeds, but it's also a very useful tool for maintaining budget discipline in a dynamic and growing network environment. 

Optimize Your Network Monitoring Tools Today

Today's networks are growing and changing at a rapid clip. As such, it's crucial for network teams to be able to utilize existing tools as long as possible, and to keep those tools functioning at a high level. By tackling the ever-present network bandwidth overload issue with advanced packet broker techniques like filtering and load balancing, you can extend the life of legacy lower-speed monitoring tools to ensure top network performance and save valuable capital expenditures along the way.

##

Alastair Hartrup is the CEO and founder of Network Critical, a company that provides industry-leading network TAPs and Packet Brokers, which help organizations increase visibility across dynamic and complex networks. He founded Network Critical in 1997, and today more than 5,000 companies worldwide rely on its technology to help power the network and security monitoring tools needed to control changing infrastructure.