Opportunity
I see a growing demand for Wi-Fi performance analytics (location, content, and experience analytics are a separate discussion piece) and assurance to form more robust and accountable service level agreements with wireless service providers. Customers want to know if their network is performing properly, and the wireless network is no exception.
While wireless engineers can perform surveys to check coverage and interference, and perform packet analysis in various locations to assess operations and the user experience, and monitor network management system logs and reports; it is expensive to send engineers onsite at the hint of any wireless issue (and as we know, more often than not it can be an issue with the client device as opposed to the wireless network; or an issue further down the line in the wired network infrastructure), and it is difficult to replicate issues and correlate them to certain points in time.
I also see a strong use-case for post-implementation performance baseline establishment. Performing post-implementation wireless surveys, end-user acceptance testing, throughput and packet analysis, and application tests are all necessary components of verifying if a wireless network is operating as per design. However it would be great if there was a way to perform longer-term testing of the network to establish normal and expected operating conditions over say a two week period, perhaps pre and post-go live.
Progress
More and more WLAN vendors are incorporating analytics and testing capabilities into their access points and WLAN infrastructure. Some examples are: Meraki with in-built packet captures in their APs, throughput tests, as well as utilisation and application visbility dashboards; Mist and Mojo access points can perform ping, DNS, HTTP and application tests from their infrastructure, utilising an additional radio to perform tests imitating a wireless client by associating to a nearby AP.
These are fantastic features and I look forward to seeing how these vendors grow and develop the analytics and reporting capabilities. It is definitely a great start in the right direction for wireless application and performance monitoring and assurance.
However there is a fundamental problem with these testing models: the tests are being performed by the wireless LAN infrastructure access points – high spec radios with typically greater transmit powers and receive sensitivities than the wireless clients our users operate on; and these tests are being performed on radios located where the APs are – not where our clients are!
Ideally these tests would be performed at locations where our wireless clients are located, and using similar wireless hardware that you would find in a laptop or USB wireless adapter. It would also be great if the same tests could be performed on the wired network to help determine if application performance issues are strictly being experienced on the wireless network only; and correlate these results and historical monitoring into a single dashboard.
Enter NetBeez.
Solution
You may be familiar with Ordoid and Raspberry Pi setups for networking monitoring, as presented at WLPC Phoenix 2017: https://www.wlanpros.com/turn-odroid-raspberry-pi-remote-wi-fi-sensors-panos-vouzis-wlpc-phoenix-2017/ , and even the maker session by Jerry & Ferney for a throughput test computer: https://www.wlanpros.com/maker-session-odroid-throughput-test-computer-build-configure-testing-jerry-olla-ferney-munoz-wlpc-phoenix-2017/
Brilliant stuff!
The WLPC session was presented by Panos Vouzis, one of the co-founders of NetBeez; so this had me interested in finding out what NetBeez was and what it could do.
Essentially the NetBeez solution is a deployment of wired and/or wireless sensors (called agents, or Beez) that can be setup to run various scheduled tests (such as ping, DNS resolution, traceroute, HTTP load time, Speedtest, VoIP MOS, iPerf v2&3) to targets (either other Beez, application and/or webservers) and present the historical data in a dashboard along with alerting and threshold monitoring.
So I decided to purchase a single wireless agent ($US180 + shipping) and try out the Free plan of their NetBeez dashboard for monitoring.
The Agent
Setting up the agent is simple enough. It is a branded Raspberry Pi with power adapter, an ethernet interface (for initial setup and management) and a wireless interface (the testing interface).
In this case the wireless interface is an Asus dual-band USB-AC56 adapter as shown below.
Upon connecting the wired interface to your network it should receive an IP address from DHCP (as long as you have that configured!) and the agent will establish a connection to the NetBeez cloud.
The Dashboard
Once your NetBeez account and dashboard have been setup, you’ll be able to sign in and begin to configure any agents that have been assigned to your account. I did some basic setup steps including configuring a wireless profile, setting up facebook.com as a target for some scheduled tests, and setting up a scheduled SpeedTest throughput test.
With the help of NetBeez’s excellent support I was also able to use the interactive console to the agent via the dashboard to configure the wireless interface to periodically disconnect and re-associate every hour. Apparently authentication latency tests will be coming in 2018, very cool and very useful for WPA2-Enterprise networks!
Buzz
This is the overview dashboard which gives a summary of how your agent(s) and target(s) are performing. I only have one agent and one target so everything looks pretty good right now.
Agents
Next you can view your agent(s) dashboard, see some statistics about availability and interface details, see any incidents and assigned tests with green/amber/red indicators, and configure the agent via the settings button.
I first went into the main NetBeez settings to set up a wireless connection profile for my WPA2-PSK network. The agent can also connect to Open, WEP and WPA2-Enterprise networks.
Then you can assign the wireless profile to the wireless interface on your agent(s):
Once the interface is connected you can view connection details and also view connection quality, signal strength, bit rate (and also throughput once SpeedTest and/or iPerf tests are setup).
Targets
The Targets page is where you set up remote targets (such as a website or application server) to perform periodic tests against and configure alerts.
Here you configure the test types to be performed by the target.
For each test type you can configure specific options, and then assign alerts to agents for each of these tests. You can also choose for alerts to be emailed to groups or individual email addresses.
Tests
Here is where you monitor your scheduled tests as well as configure additional SpeedTest, iPerf, and VoIP quality tests.
You can view detailed historical data about each test, as well as export the data to CSV. You can also set up multiple iPerf, SpeedTest and VoIP tests with various settings. VoIP tests are performed between agents (Beez) and can be between wireless and wired agents; and you can choose the audio codec to be tested with (such as g711, g729, 9722 e.t.c.).
Ad-hoc Testing
You also have the ability to perform ad-hoc tests on specific agents, which is pretty cool.
The image below shows a 10Mbps UDP iPerf3 test.
Reports
The NetBeez dashboard has the ability to generate reports for network status, your agents, your targets, and for specific scheduled tests. These can then be exported to PDF and emailed. Great for management viewing and SLA reporting!
Alerts
Lastly you can view the log of events such as wireless association/disassociations, test failures, and other system status events. These can be searched by agent name or IP address, SSID, target name, test name, severity e.t.c.
Summary
I have thoroughly enjoyed testing my NetBeez agent, and I have barely even scratched the surface. I would love to see a solution such as this implemented by wireless integrators and service providers as a means to provide client-level monitoring and assurance of their wireless networks. How great would it be to have the 100% assured information to respond to a ‘wireless incident’ with “The wireless network is operating fine, here’s my proof!”.
In future I plan to play around with remote packet capturing as demonstrated in the maker session and this NetBeez blog post.
Please let me know if you have any questions about this product, or let me know your ideas for wireless monitoring and assurance!
DISCLAIMER: All products, hardware and software outlined above have been purchased at full cost by myself. I have received no promotional consideration or sponsorship from any vendors.
Wi-Fi Coops 