Reporting the URL of HTTP web requests

If you have purchased the Web and Cloud product, you may set up the Collector to send the URLs of those HTTP web requests that the end-users address to a selected group of domain names. By default, for every web request, the Collector only reports the domain name inside the request to the Engine (and not the full URL) to keep the amount of generated network traffic low and avoid flooding the Engine with lots of URLs. Nevertheless, when the Collector is allowed to report the URLs of just a few web requests, the generated traffic still remains reasonably low, while you may benefit from this additional information to define services based on particular URL paths or investigations that include conditions on URLs of web requests.

Learn in this chapter how to specify the list of domain names for which the Collector must report the URLs of the HTTP requests that are addressed to them from the devices of the end-users.

Accepted syntax for the list of domains

Independently of the method chosen to configure the Collector, the accepted syntax for specifying domains is the same. The allowed characters to write domain names are a subset of the ASCII character set that comprises:

• The range of letters from a to z and from A to Z.

• The digits from 0 to 9.

• The symbols . (dot) and - (hyphen).

• The symbols : (colon) and / (slash).

• The symbol * (star) to substitute zero or more characters.

Let us see some examples of domain names and how are they interpreted by the Collector:

www.example.com	Matches all HTTP requests addressed to www.example.com
http://www.example.com	Same as above: matches HTTP requests to www.example.com
www.example.com	Matches all HTTP requests to www.example.com
http://example.com/index.html	Matches the same as www.example.com (the URL path after the host name is ignored)
*.example.com	Matches any prefix before the first dot (e.g. www.example.com and www.example.com, but not www.example.com)
*example.com	Matches any prefix (e.g. www.example.com, ftp.example.com, www.example.com, another-example.com)
***example.com	Same as above (multiple consecutive stars count as one)
ftp.example.com	Matches all HTTP requests addressed to ftp.example.com (Note that the protocol is HTTP and not FTP)
ftp://ftp.example.com	Error: only HTTP scheme is allowed
https://example.com	Error: only HTTP scheme is allowed
-example.com	Error: domain names cannot begin or end with a hyphen
*	Error: the match all star pattern is not allowed alone

Configuring the list of domains in the Collector

Specify the list of the domains for which the Collector reports the URLs of web requests either before or after deploying the Collector:

• Before deploying the Collector:

  • Passing parameters to the MSI.

  • Using the Nexthink Collector Installer.

• After deploying the Collector:

  • Using the Nexthink Collector Configuration Tool.

  • Changing the value of a registry key.

Beware that if you use the Updater to deploy the Collector, many parameters of the MSI, and the list of domains in particular, cannot be set at installation time and are not saved between updates. For every automatic update of the Collector, you must reapply the settings after deployment.

Passing parameters to the MSI

Specify the list of domain names by setting the value of the parameter DRV_WEB_AND_CLOUD_HOSTS when you install the Collector using its MSI file. The value supplied must be a comma separated list of the domains with the syntax defined in the previous section.

This option requires the parameter DRV_WEB_AND_CLOUD_DATA to be set to 1 (its default value) for the Collector to gather web related information.

Using the Nexthink Collector Installer

If you use the Nexthink Collector Installer to deploy the Collector, specify the list of domains for which you want to get the full URLs in the Web And Cloud Settings dialog that appears when you click the Settings button:

In the case that you are updating the Collector, the new settings replace any previously configured list of domains.

Using the Nexthink Collector Configuration Tool

If you have already deployed the Collector, use the Nexthink Collector Configuration Tool to modify the list of domains for which to report full URLs accessed from a particular device. This requires the presence of the Nexthink Collector Configuration Tool in the device; which is installed along with the Collector by default, unless you set the MSI option CFG_INSTALL to 0.

Execute the tool with administrator privileges and specify the list of domains as a parameter in the command line with domains separated by commas:

C:\Windows\System32\nxtcfg.exe /s wm_domains="csv_list_of_domains"

Setting the value of a registry key

The list of domains for which to report full URLs is saved in the registry under: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\nxtrdrv\params\hosts

If you change the value of this variable, the Collector detects its modification and applies the changes accordingly. If an error is detected in the syntax of a domain, the error is logged but the service just skips to the next domain in the list. Under high load, the Collector can miss the modification of the environment variable and you must reboot to force the change. For this reason, this method is recommended only for testing in pre-production environments.

For debugging purposes, it is allowed in this case to use the match all star pattern: *. This is the only exception to the rule and it may help you detect connectivity problems in a particular device.

Technical and security limits

By using any of the described methods, you can specify up to a maximum of 20 domains. The Collector limits the length of a URL to a maximum of 1024 characters. In the rare case of processing a URL longer than 1024 characters, the Collector truncates it to the first 1024 characters.

Note that the feature is only available for HTTP and not for HTTPS web requests. Due to TLS encryption, it is not possible to get the URLs of HTTPS requests. Moreover, reporting the exact URL of an HTTPS request might incur in a security or privacy breach.

In the same sense, the Collector never reports the query string part of a URL, that is, the optional list of parameters used by web applications that is placed at the end of the URL after a question mark. Query strings often carry sensitive information such as login names and passwords.

---

RELATED TASKS

• Creating a service
• Specifying URL paths of web-based services
• Installing the Collector

RELATED REFERENCES

• Collector MSI parameters reference table
• Nexthink Collector Configuration tool