Customer Maintained Subscribed Lists and External Lists

Simple Lists 

a) Text 

Here is the format of a Simple Text File: 

type=list type 

"data item 1" "data item 1 comment" 

"data item 2" "data item 2 comment" 

When creating your simple list, you'll need to substitute in the appropriate list type. Thebold text below indicates the list type you'll need to specify: 

• Application Name = applcontrol 
• Dimension = dimension 
• String = string 
• Category = category 
• IP = ip 
• IPRange = iprange 
• MediaType = mediatype 
• Number = number 
• Wildcard Expression = regex 

EXAMPLES:

type=category 

com.scur.category.115 

com.scur.category.166 

NOTE: Trusted Source category codes can be downloaded here: McAfee Trusted Source Web Database Reference Guide. 

type=applcontrol 

"Google News" 

"Ebay.com" "Ray asked me to add this application" 

NOTE: For a listing of Applications, you can view the System List in the Secure Web Gateway UI:(Policy > Lists > Systems Lists > Application Name) 

type=regex 

"*.espn.com/nfl*" "HD-0002 - VP wants to check NFL scores" 

"*.ebay.com/auctions*" 

type=string 

"www.google.com" "HD-0003 - Jon needs access" 

"www.startribune.com" "HD-0004 - Tim needs to view" 

type=string 

"Hello\"Michael\" \"Michael!\"" "" 

"*empty comment\"\"\" "" 

NOTE: The escape character \ is used to allow a double quote as part of the data or comments as seen above. 

b) XML 

Here is the format of a Simple XML File:

<content type="list type"> 

<listentry> 

<entry> data item 1 </entry> 

<description> data item 1 comment</description> 

</listentry> 

<listentry> 

<entry> data item 2 </entry> 

<description> data item 2 comment</description> 

</listentry> 

</content> 

<content type="regex"> 

<listEntry> 

<entry>*.windowsupdate.com</entry> 

<description>HD Ticket: 1234</description> 

</listEntry> 

<listEntry> 

<entry>*.microsoft.com</entry> 

<description>HD Ticket: 1235</description> 

</listEntry> 

</content> 

 

Complex Lists 

Complex lists can only use the .xml file format. The first line of any complex list will begin with a line that looks like this: 

<content type="complex.nexthopproxy">

The content type, in this example nexthopproxy, relates to the type of complex list you're creating. Below is a list of content types you can use. 

NOTE: The bold text below indicates the content type syntax you will need to use. 

• Certificate Authority = complex.ca 
• Extended List = complex.extendedlist 
• Element = complex.element 
• HostAndCertificate = complex.hostandcertificate 
• ICAP Server = complex.icapserver 
• NextHopProxy = complex.nexthopproxy 
• MapType = complex.maptype 

Due to the complexity of the format, it is recommended that you use the following steps to create your complex list: 

1. Create a sample complex List: Use the Secure Web Gateway to create a complex list of the 'type' you want. In the Secure Web Gateway UI, go to: Policy> Lists > Add > give the list a name and choose a content type. Press the OK button. Lastly, add content to the list and Save changes. 
2. Export the complex List: In the Secure Web Gateway UI, go to: Policy > Lists- Select the list and then click the Export button. Save the list to your local machine. 
3. Modify the list: The exported list adds extra lines that we will need to remove. Edit the exported list in a text editor. 
4. Delete all lines in the file that precede the opening <content>tag and follow the closing </content>tag. 
5. Modify the opening <content>tag to read <content type="complex.content type">, for example <content type="complex.nexthopproxy"> 

Here is what an exported and modified next hop proxy complex list example would look like: 

<content type="complex.nexthopproxy"> 

<listEntry> 

<complexEntry defaultRights="2"> 

<configurationProperties>

<configurationProperty key="name" type="com.scur.type.string" value="Default NextHop Proxy" /> 

<configurationProperty key="host" type="com.scur.type.string" value="10.10.79.50" /> 

<configurationProperty key="port" type="com.scur.type.string" value="9090" /> 

<configurationProperty key="user" type="com.scur.type.string" value="" /> 

<configurationProperty key="password" type="com.scur.type.string" value="" /> 

<configurationProperty key="retries" type="com.scur.type.string" value="1" /> 

<configurationProperty key="waittime" type="com.scur.type.string" value="10" /> 

<configurationProperty key="persistent" type="com.scur.type.boolean" value="true" /> 

</configurationProperties> 

</complexEntry> 

<description /> 

</listEntry> 

</content> 

 

Here is a map type complex subscribed list example: 

<content type="complex.maptype"> 

<listEntry> 

<complexEntry defaultRights="2"> 

<configurationProperties> 

<configurationProperty key="key" type="com.scur.type.string" value="bob" /> 

<configurationProperty key="value" type="com.scur.type.string" value="pas$w0rD"/> 

</configurationProperties> 

</complexEntry> 

<description /> 

</listEntry>

</content> 

Option 1

1. Navigate to Policy > Lists> Add List (green circle icon) 
2. Enter a name for the list and select list type. Note: the type selected here must match the actual list you created. Otherwise you will get an error when trying to save changes. 
3. Select "List Content is managed remotely" 
4. Select "Customer Maintained List" 
5. Click "Setup" 
6. Setup window opens: 

Setup options: 

• URL to download list from: HTTP, HTTPS, or FTP supported 
• Use this "Certificate Authority Chain": select if HTTPS used and import certificate in.PEM format from local workstation 
• Ignore certificate errors: select for HTTP or FTP, or if you wish to ignore certificate errors on HTTPS 
• URL Authentication: can leave blank if none required to access URL for download 
• Proxy: Only configure if there is an upstream proxy 
• List content update: Select how often you want the Secure Web Gateway to update the list 
• When done configuring your list, click "OK".
• If successful, you will see your list appended with the label "Customer maintained" and list contents will be displayed. 

 

Option 2

1. Navigate to Policy > Rule Sets 
2. Edit an existing rule or create a new one 
3. For "Rule Criteria", edit or add criteria that matches your list type, for example,URL.Categories. 
4. Click "Add List..." 
5. Then fill out customer-maintained list according to the setup procedures shown in option 1. 

Example of using a customer-maintained subscribed list in a rule. 

Web Service 

A web service will be accessed under HTTP, HTTPS, or FTP protocol. The file format will need to be in XML, JSON, or Text format. 

a) XML 

The format of this file simply needs to follow typical XML standards. Secure Web Gateway can use a standard called 'XPath' that will allow it to navigate through elements of the XML file it is retrieving and filter for specific data you want populated in the external list. Additional information about creating an Xpath can be found at http://www.w3schools.com/xpath/. 

XML file example: 

<user-info> 

<user> 

<name lang="eng">tom</name> 

<level>1</level> 

</user> 

<user> 

<name lang="eng">bob</name> 

<level>5</level>

</user> 

</user-info> 

Using the xml example above, we will show you how to use xPath to retrieve certain elements of the XML file. 

- Settings under Policy> Settings> Engines> External Lists: 

XPath Expression = /user-info/user/name/text() 

This XPath expression will give the name of the user in the XML file. Essentially, this expression will search down the XML file looking for each tag you've specified in the path above and then use the 'text()' function to only pull out the text value of the <name> tag. 

XPath Expression = /user-info/user/level/text() 

This XPath expression will filter out the text value of the <level> tag for use in the external list. 

 

b) JSON 

The data source will return data as JSON object. 

- JSON examples can be found starting on page 152 in the product guide. 

- Use of parameters (placeholders) in the rules is explained starting on page 139 in theproduct guide. 

- Example using external JSON file list in rules: 

 

c) TXT 

The format of this file need to simply be a text file. For text files a regular expression is used to locate the data for our list. 

Text file example: 

Member Group name 

------ ---------- 

group1 admin 

group1 sudoadmin 

group2 report 

group3 users 

group4 guest 

Added sudo admin group. Updated on 9/28/2017 

- The regular expression below uses a grouping operator so only the "group name"column will be retrieved for our list. 

- Settings under Policy> Settings> Engines> External Lists: 

Data Type = Plain Text 

Regular Expression = regex(^group. (.*)) 

- Example using external text file list in rules:

File on disk 

File on disk refers to a file within your local file system (on the Secure Web Gateway appliance itself). This option is good for testing your external list when another external source is not yet available. Accepted list formats are the same as for web service data sources (XML, JSON, and TEXT). 

LDAP or LDAPS 

Another data source for external lists is LDAP. Once connected to an LDAP server, you can specify an LDAP Search Filter and Attribute to fetch specific data for the list. The search filter is an LDAP standard and the attribute will depend on your LDAP configuration. Below is an example that searches for the user bob and returns the 'Title' attribute for that user. 

- Settings under Policy> Settings> Engines> External Lists: 

Data Type = LDAP 

Search Filter = (sAMAccountName=bob) 

Attribute = Title 

Database 

Another data source for external lists is Database. You can choose between retrieving data from a PostgreSQL or SQLite3. Once connected to the database, you can specify a SQLquery to fetch specific data for the list. Below is an example SQL query used with External Lists. 

- Settings under Policy> Settings> Engines> External Lists: 

Data Type = Database 

SQL Query = SELECT user from userTable;