Understand User Agents

Introduction

You might be asking yourself, "Okay, why do I care about the user agent?". Well, the user agent can be important and can help you out in lots of situations. Let's say iTunes keeps prompting users to enter in their credentials, or a windows media player stream simply will not play. You might need to bypass these programs from certain rules to get them functioning the way they should be. You can easily do that with 1 header, and that is the user agent header.

What is a User Agent?

A User Agent is a header in an HTTP request that identifies the client software originating the request.

Why is this header important?

This is important because it identifies what piece of software may be making a request so you can properly identify where it is coming from. This could be anything from the browser, to windows media player. They all have a user agent to identify what software it is.

Why would I need to create a rule to match on the user agent?

There are times when you need to match the user agent criteria for a rule. These times usually include bypassing a specific rule set within Secure Web Gateway that may be causing issues with the user agent. The beauty about Secure Web Gateway 7 is that you don’t have to bypass all of the rules. Say you are having an issue where a specific user agent is not working with authentication (which is one of the most common times) so you only want to bypass authentication. You can easily do that. There will be some examples of this and other common scenarios at the end.

How do I find out what the user agent is for a request?

There are a few ways to find out this sort of information. Below I will list and explain some common ways to find this out:

http.user_agent matches "Mozilla"

Create a rule based on the User Agent header in Secure Web Gateway

There are several reasons for why you might want to create a rule to match a user agent. You might want to either allowlist or block a specific user agent. For example, if you didn’t want users to use the Internet Explorer 6 browser, you could block the user agent. Another reason is that some user agents do not know how to respond to proxy authentication. Some common ones we have encountered in the past that have trouble with authentication are: Silverlight,iTunes, Windows Media Player, Real Player, Adobe Updater, Adobe Flash. These user agents typically do not know how to respond to a 407 proxy authentication request and simply do not answer when the Secure Web Gateway asks for it, so they need to be bypassed.

The Secure Web Gateway does not have a specific property for the user agent so it needs to be entered manually.

• First you would select the header.request.get property from the list
• Then, you would press the “parameters” button

• Now, here is where you have to manually type in: User Agent
• Next, you have a choice to either have it match a single User Agent, or have it match in a list of user agents. The choice is yours of which you want to do, but lists are always recommended because then if you have new entries you want to add later, all you have to do is add it to the already existing list. Here is an example of a list of user agents it can match.

• The action you want to choose is completely up to you. You can either block it, stop rule set, or stop cycle. It all depends on what you are trying to accomplish with the rule.

Conclusion

Hopefully, this has given you a better understanding of what the user agent header is exactly, and what purpose it has. Also, hopefully, this gets you on the right path to creating a rule based on the user agent property to use within your Secure Web Gateway appliance.

Rule Set Examples

TCP Dump Troubleshooting Examples

Here is an example of what you might see in a TCP Dump that shows when a user agent doesn't respond to proxy NTLM authentication requests, and also how to find the user agent in a GET request.