Why cant Web Application Firewalls (WAFs) detect and block sophisticated bots?

WAFs are effective and useful tools as part of any secure web based system, however they are solving a different problem to that posed by non-human traffic or bot attacks.WAFs are designed to look for patterns within requests that are targeted at exploiting security weaknesses within systems. Non-human traffic is usually legitimate requests but just made in a different pattern to standard human users. To effectively identify bot traffic, therefore, it is necessary to look at the nature and patterns of requests that are being made and compare those to that being made by human users.WAFs may help stop basic bot traffic but will not identify any more sophisticated traffic sources which are becoming ever more common.

One way of dealing with bot traffic is by simply creating a blacklist of IP addresses on a firewall, CDN or other device - traditionally this was a fairly common way of handling this problem. However it is a very limited solution and suffers from several key issues.Firstly, it is always reactive, typically adding in the IP addresses of bot attacks retrospectively or in the best case scenario by making a config update after the attack is identified. Therefore it only contains details of the IP addresses of past attacks whereas typically automated traffic will regularly rotate IP addresses.Secondly, it needs constant maintenance to ensure that new threats are added to the list as they are discovered. It also needs a method of validating if the added IP addresses are still valid.Thirdly, it usually only allows for hard blocking, that is dropping the connections of any requests coming from matched IP addresses. This allows little scope for giving friendly messages back to any false positive users or any of the other ways of dealing with non-human traffic offered by effective bot management tools.Any good bot defence tool should address all of these limitations to provide a valid amount of insight and control into the nature of non-human traffic on your system.

The primary purpose of Netacea Virtual Waiting Room is to maximise your revenue by ensuring your website stays online and performing at its best when under high load.Web traffic is unpredictable. As well as being influenced by your own marketing efforts, such as TV advertising or email campaigns, it can also spike when you get unexpected buzz from social media or other uncontrolled external factors. In some cases your IT team might not even be aware of your marketing team's next major campaign.If your site were to slow down or crash at these times, there would be a direct loss of sales revenue. Any marketing activities would suffer from low ROI, the brand would be damaged and customers would be less likely to return after suffering from a poor experience. This whole effect is amplified due to the increased attention to the website during these times. Those points aside, think about how long it would take you to fix the problem. How much resource would be used up? What would be the operational cost to the business?Netacea Virtual Waiting Room solves these problems by guaranteeing that an optimal amount of customers will continue to flow through your site even if the volume of traffic is greater than your capacity. You'll be fully prepared to serve more customers in a faster time with ease and certainty.

Netacea Virtual Waiting Room is ideal for any website or system that receives large spikes in traffic, putting its availability or performance at risk. This covers many industries, including retail and eCommerce, ticketing, media and broadcasting, education and government, to name a few.

There are many ways of engineering websites to be more scalable and robust against high volumes of traffic, and we're not suggesting that you don't do those things. However, Netacea Virtual Waiting Room is that last line of defence against a site-crushing wave of traffic - think of it as an insurance policy for when all else fails or you get a spike that you just couldn't have coped with otherwise. Engineering highly scalable websites is a costly and time-consuming task that may not be feasible or cost effective.Web traffic is unpredictable. Even when looking at traffic trends and analytics, the timing and size of peaks can change dramatically based on both expected and unexpected factors. It's not always possible to know how much traffic to expect or when exactly it might hit your website. Because of this, it's impractical and expensive to pay for the infrastructure needed to comfortably cope with your biggest "predicted" peaks all year round or to know how much traffic to load test for.Peak Management allows you to maintain a right-sized infrastructure for your budget whilst giving you the ability to cope with the larger peaks in traffic associated with sales events and seasonal activity.

You wouldn’t. In real life no shop owner would ever want to put people in a waiting room, however waiting is a regular and accepted part of the physical world and an essential way of ensuring an efficient flow of customers.Virtual Waiting Room carries out the same role in the virtual world. Queueing customers when there are too many to be able to manage effectively and efficiently allows for the optimal overall throughput of customers. A better way of thinking of the problem would be to think about why you wouldn’t put your users in a queue in a situation where the alternative would be complete website failure and therefore no customers getting the outcome and experience they want and that you want to provide.We see Virtual Waiting Room as an insurance policy you don’t necessarily want to use, but should always have.Virtual Waiting Room's functionality is a last resort. We would advise any business to do all they can to ensure their website is prepared well ahead of time to gracefully handle the amount of traffic they are expecting to receive at any given time. However, even with a plan in place, there is always a limit to any website’s capacity and therefore the chance that it will receive more traffic than it can physically handle.Virtual Waiting Room is there as the ultimate insurance policy so that even if all else fails and a website simply can’t cope with the amount of traffic hitting it, it will always be online and performing well for as many visitors as possible.The alternative to queuing (aside from allowing the site to go offline entirely) would be to just stop new visitors from entering the site and asking them to “come back later”. However, studies have shown that people are much happier to wait if they know how long their wait is going to be. Virtual Waiting Room's gives this transparency to those waiting to enter a site and confidence that they will get onto the site if they stay in the queue.You can also use the waiting room to reinforce your branding though images and videos.

For many older systems this is not necessarily an option, systems have to be built and configured to be able to scale automatically.Likewise, while it’s true that cloud-based solutions allow you to scale up your infrastructure on demand, this is not an instant process. It takes several minutes to spin up the additional capacity and “warm up” load balancers in order for them to be effective. In the case of “cliff face” spikes, such as traffic from a TV advert where we often see 40-50 times usual traffic within seconds, autoscaling just can’t react quickly enough to help.Netacea Virtual Waiting Room reacts instantly when a sudden spike in traffic hits, keeping the site online at all times for those already browsing. This actually compliments any autoscaling you might have in place.

Our Adaptive architecture automatically pre-empts potential bad traffic, and kicks in-line only when critical conversion or login paths are under threat, or abnormal behavioural activity is detected. This behavioural mode ensures that there is no speed reduction or latency whatsoever in our architecture for legitimate customers

Instead of supplying a ‘black box’ Netacea works with our enterprise customers to provide the best possible integration of our behavioural threat layer via set of APIs. We complement existing controls such as WAF rulesets, rate limiting and threat databases, to provide deep analysis of all website visitors., through a practical use of A.I. to understand human and bot behaviours and adjust their website journey in real-time. We have no customer premises equipment, and everything is cloud-based, to push threat alerts to your service of choice.

Our adaptive data model and micro-services API approach gives huge power and flexibility to ensure that even the most complex of visitor requirements can be elegantly and reliable handled at volume, using the existing infrastructure that enterprise customers already maintain and own. Using our rich set of API, you can send the threat alerts to your WAF, CDN provider, or firewall of choice.

Netacea has been designed with accessibility in mind, and we regard it as a duty to support all users who have accessibility issues and use readers. Our core behavioural learning does not change across platform types. Where we specific bot mitigations - for example when we display a Captcha, we do have a range of accessibility options for visually or audio impaired users, or those users who lack the fine motor skills necessary to complete some difficult Captchas.All our Capcha’s have a text alternative, allowing them to be read by a reader, and audio alternatives for those with vision impairment. In addition, our fingerprinting does not require the use of JavaScript. Although providing a text based Captcha does provide bot writers with a potential exploit to bypass the Captcha, we monitor the accessibility options very carefully to ensure that the small percentage of traffic accessing the text based Captcha is legitimate and under normal thresholds.Summary of Accessibility options • Does not require Javascript so all assistive technology will work • Provides and Audio Alternative so login can be navigated with a screen reader • Settings, permissions • No complex or repetitive navigation links – so each page is placed in its natural order so the flow is easy to use.