Troubleshooting NTLM
Category:
Authentication
Environment:
Secure Web Gateway
Problem:
Users are being prompted to authenticate.
Cause:
-
Rule misconfiguration
-
SWG to DC communication issues
-
Client to SWG communication issues
-
Time Skew
-
Performance issues
-
Known bugs and errors
Log Collection:
-
Client-side packet capture (-s 0 -i any host <client IP>)
-
DC-side packet capture (-s 0 -i any port 445 or port 53)
-
Rule trace
-
Feedback file (Enable authentication debugging logs)
Note: collect both packet captures at once by using -s 0 -i any host <client IP> or port 445 or port 53, keep in mind this will collect more traffic and might cause the packet capture to reach the UI limit of 200k frames.
Log Analysis:
Rule Misconfiguration
NTLM authentication can be implemented on SWGs deployed in either “explicit” or “transparent” mode. The ruleset that will be used will depend on which deployment method is being used. Below I have included screenshots of common rules available within the Rule Set Library.
Note: In explicit mode, ALL HTTP/HTTPS requests are authenticated, while in transparent mode it is common for authentication to occur on a time-base, once every hour, two, depending on the configuration.
Explicit Proxy
Transparent Proxy
For more information on NTLM authentication, review Choose the Right Authentication Method for your Deployment
SWG to DC Communication Issues
Ensure the web gateway is able to resolve the DC’s FQDN IP address and is able to communicate to the DC over port 445.
GUI: -s 0 -i any port 445 or port 53 CLI: tcpdump -s 0 -i any port 445 or port 53 -w SWG_DC.pcap |
Packet capture over port 445
Client to SWG Communication Issues
Review the communication between the client and SWG. The packet capture between the client and SWG should show the following interaction:
-
Client sends an initial request
-
SWG responds with 407 AuthRequired
-
Client sends NTLM Negotiate
-
SWG responds with NTLM Challenge
-
Client sends NTLM Auth (contains credentials)
-
SWG responds with 200 OK
Explicit Proxy packet capture:
Note: The communication should all be within a single TCP stream. Load balancers in between the client and SWG are known to break this communication into two or more TCP streams, resulting in issues.
Time Skew
In the context of NTLM authentication, time skew refers to the allowable time difference between the clocks of the client and the server involved in the authentication process. In ideal circumstances, the time difference between the Domain Controllers and the SWG should be null. Beneficial checks:
- service ntpd status ( Verify if the service is running )
- timedatectl status ( Verify if NTP is active)
- service ntpd stop
- ntpd -gq > sends an immediately queries the NTP servers, makes necessary time adjustments, synchronizes the system clock, and then exits
- ntpq -pn > outputs the NTP server system is synching with
- service ntpd start
(These 3 commands will stop the ntp autosync, manually sync the time, then restarts the ntp autosync)
Performance Issues
Performance issues can range from SWG hardware issues to network latency, and anything in between. Reviewing the SWG counters can help determine whether the SWG is overloading and/or there is network latency experienced within the customer’s environment.
ResolveHostViaDNS - Average Time to Resolve Host via DNS (ms)
AuthNTLMRequests - NTLM Requests (number)
NTLMRequestProcTime - NTLM Request Average Processing Time (ms)
AuthNTLMCacheRequests - NTLM Requests from Cache (number)
Known bugs and Errors
Latest Secure Web Gateway (On-prem) Releases
Reference |
Description |
Resolved in |
WP-5419 |
When the authentication process on Secure Web Gateway uses the basic NTLM authentication method, adding the default domain of the NTLM authentication server to the settings, no longer leads to a failure of the process. |
11.2.12 |
WP-4245 |
An admin user can again log onto Web Gateway using NTLM authentication successfully |
11.2 |