Establishing a data retention policy in the Engine
Last updated
Last updated
The Engine stores the real-time data that it receives from the the Collectors in the form of events. Events are very numerous and they usually take most of the memory reserved to the Engine. The types of events that occupy most of the space in memory are executions, connections, and web requests. When two or more of these events are very similar to each other and they occur in sequence, the Engine may consider that they are actually the same event. In that case, the Engine combines the data of the events and stores only one event in its database. We say then that the Engine aggregates the information of several events into one; thus saving memory space and resulting in a larger history for the Engine.
When you have the web monitoring feature fully enabled, you usually collect a huge number of web domains. In the same spirit of event aggregation, when two or more domain names share their highest level domains, the Engine may group them into one generalized domain by obeying specific rules. This process is known as domain compaction or domain compression and it replaces one or more of the lower level domains in the domain name by the wildcard character *. For instance, the Engine might compact the domains one.example.com and two.example.com into *.example.com. Note however that those domains declared as internal or included in the definition of a web-based service are considered of special interest to you and, therefore, they are never compacted.
Learn here how to set the maximum number of events and establish the policies for both the aggregation of events and the compaction of domains in the Engine.
To set the maximum number of events that the Engine can store:
Log in to the Web Console as admin.
Click the Engine tab at the top of the window.
Select the General section from the left-hand side menu.
Under Parameters, choose a number from the Max stored events drop-down list.
The amount of RAM available in the Engine limits the possible choices for the maximum number of events. To be able to select a high number of events, ensure that the Engine complies with the hardware requirements regarding the available memory.
Click SAVE CHANGES. Note that the Engine is restarted after saving the changes.
Choose among four strategies of aggregation for an optimal trade-off between detailed event information and history length. The more aggressive the policy, the fewer individual (non aggregated) events are visible from the Finder.
Log in to the Web Console as admin.
Click the Engine tab at the top of the window.
Select the General section from the left-hand side menu.
Under Parameters, choose one of the following aggregation policies from the list labeled Aggregation policy:
very low - normal history, for the traditional minimal aggregation.
low - up to 10% more history, for increasing the history 10% approx. while keeping most of the individual events.
medium - up to 80% more history, for a more aggressive aggregation policy to increase history in the Engine up to 80%.
high - up to 100% more history (default), for the most aggressive aggregation policy to practically double the history traditionally available in the Engine. Starting from V6.19, this is the default setting for new installations and for upgraded appliances that never changed their default value.
Click SAVE CHANGES. Note that the Engine is restarted after saving the changes.
To further increase history, especially if you installed the Collector on servers, configure your Engines to apply the aggresive reduction of destinations to Collector traffic. An increase of up to 30% in history is expected on setups with servers (or other types of devices) that communicate in bursts with multiple destinations, at the cost of losing the individual information of every connection.
Log in to the Web Console as admin.
Click the Engine tab at the top of the window.
Select the General section from the left-hand side menu.
Under Parameters, choose one of the following domain compression policies from the list labeled Domain compression:
medium (recommended), the default compression policy for domains with more than five levels or with repetitive (or randomly generated) subdomains. This is the recommended setting.
high, to apply a compression method to all the stored domain names according to a public list of domain suffixes.
Click SAVE CHANGES. Note that the Engine is restarted after saving the changes.
For a detailed explanation of compaction policies, see the section about compaction in the definition of domain.
If you have purchased Nexthink Act and your Engines support more than 8000 devices each, allocate an extra half gigabyte of memory for your Engines to deal with the additional outputs from remote actions.
This extra amount of memory should be present on every Engine if you followed the hardware requirements indications.
Log in to the Web Console as admin.
Click the Engine tab at the top of the window.
Select the General section from the left-hand side menu.
Tick the option under Allocate Extra Memory:
Click SAVE.
RELATED CONCEPTS
RELATED REFERENCES
