Continuously measuring the satisfaction of employees

Overview

To estimate the degree of satisfaction of your employees with their IT environment on a regular basis, Nexthink Engage offers you a powerful technique called continuous satisfaction measurement (CSM) which combines a particular type of campaign with a specific method to aggregate the responses of the employees.

Continuous satisfaction measurement lets you know how your employees feel about different subjects and how this feeling evolves over time, while minimizing work disruption.

Suitability of continuous satisfaction measurement

Continuous satisfaction measurement is appropriate for asking your employees about subject matters that have a long-lasting effect on them. For example:

✔ General sentiment about the digital equipment provided.

✔ Perception on the quality of support.

✔ Opinion about long-term IT initiatives.

Campaigns designed for CSM are not suitable though for either punctual occurrences or recurring matters that must be announced to all concerned employees at once. For instance:

✘ Notifying all users urgently about a service outage.

✘ Reacting to an application crash.

Working principle

To evaluate the satisfaction of employees with the minimum disruption possible, continuous satisfaction measurement makes use of campaigns that, once published, send notifications to a randomly selected small group of the targeted employees each day. After a few days of collecting answers (the window of observation), the opinion of the employees who already participated in the campaign statistically reflects the opinion of the whole group of employees plus or minus a particular range (the credibility interval) and with a certain degree of confidence (the confidence index).

For instance, suppose a CSM campaign with an opinion scale question whose choices are valued between 1 and 10 that is targeted to a total of 2000 employees. After collecting 150 answers, let us imagine that the average opinion of the employees is 7.5. The statistical calculations of CSM tells us then that the opinion of the whole group of 2000 employees is

• 7.5 ± 0.5 (credibility interval) with a probability of 93% (confidence index)

With 300 answers, CSM raises the confidence index to 99%, keeping the same credibility interval.

Confidence index vs. asking employees balance

Therefore, there is a compromise between achieving a high confidence index and asking more employees. To limit the burden on employees, CSM campaigns avoid asking the same employee more than once within a time interval, called the quiet period. To respect the quiet period, a CSM campaign cannot send notifications each day to groups of employees that count more members than the total number of targeted employees divided by the quiet period in days.

The campaign computes the actual number of notified employees per day using the following empirical formula:

Where

• M is the maximum number of employees that can be targeted each day to respect the quiet period

• T is the total population (number of employees targeted by the investigation).

• Q is the quiet period (in days).

• k is an empirically determined coefficient.

• N is the final number of users that will be targeted each day (capped at 110 users per day).

Not all employees who receive a notification will answer a campaign though. The number of employees per day who actually answer the campaign determines the window of observation required to get a specific number of answers. Configure the window of observation in the definition of a Nexthink metric that aggregates the responses of the employees, as described in the procedure below. Note that the window of observation in CSM is a moving window, as we are interested in how the opinion of employees evolve with time.

Parameter tuning

By tuning the quiet period of the CSM campaign and the window of observation in the associated metric, achieve the desired results in terms of employee disruption and confidence in the reported satisfaction level. In the tables below, get the confidence index for different windows of observation, given a quiet period. The tables assume a credibility interval of ±0.5, a uniform distribution of values between 1 and 10 for the choices in the campaign, and a response rate of 50% (which can be much higher in practice). For the complete set of tables, refer to the full CSM methodology document, referenced from this article in Community.

Quiet period is one month

Targeted employees	Notified employees per day	Window 7 days	Window 14 days
Answers required	Confidence index	Answers required	Confidence index
200	5	16	50%	32	66%
2000	43	149	93%	299	99%
20000	110	385	100%	770	100%

Quiet period is two months

Targeted employees	Notified employees per day	Window 7 days	Window 14 days
Answers required	Confidence index	Answers required	Confidence index
200	2	8	40%	16	50%
2000	23	81	81%	163	94%
20000	110	385	99%	770	100%

Quiet period is six months

Targeted employees	Notified employees per day	Window 7 days	Window 14 days
Answers required	Confidence index	Answers required	Confidence index
200	1	3	32%	7	38%
2000	8	27	58%	54	73%
20000	64	223	97%	446	100%

Procedure to set up CSM

To set up the continuous measurement of the employee satisfaction regarding a particular subject, follow these steps:

• Create a special type of campaign for CSM about the subject of interest.

• Create a couple of associated metrics:

  1. One metric to aggregate the results of the campaign along the window of observation.

  2. One metric that counts the actual number of answers during the window of observation.

• Add appropriate widgets for the metrics to a dashboard in the Portal to analyze the results.

Designing a campaign for continuous satisfaction measurement

A campaign must fulfill the following conditions to be suitable for continuous satisfaction measurement:

• Target a group of employees with an investigation.For better results, the targeted group of employees should be fairly stable along the total duration of the campaign (e.g. employees belonging to a department, employees with a laptop, etc).

• Target a different subset of employees every day.This setting differentiates a CSM campaign from both one-off and recurring campaigns.

• Define an appropriate quiet period.Look in the tables for the quiet period to maximize confidence while minimizing burden on employees.

• Include one or more opinion-scale questions.

Set the above options when creating a CSM campaign.

Analyzing the results of a CSM campaign

To gather results from a CSM campaign, create a quantity metric that aggregates the opinion of employees (its numerical value):

1. Create a metric on users.
2. In the COMPUTE DAILY section, choose to compute the opinion scale question of the campaign and Aggregate by average value per user.

3. In the MATCHING section, set the window of observation (W) in the form of a condition on users:

User | - Number of days since last action | is less or equal to | days

To know the actual number of employees who answered the campaign, create a count metric:

1. Create a metric on users.
2. In the COMPUTE DAILY section, select the total number of | all users.

3. In the MATCHING section, select the employees (users) who fully answered the campaign within the window of observation with these two conditions:

User | - Status | is | fully answered

User | - Number of days since last action | is less or equal to | days

Add a line chart widget to your dashboard for the first metric and a KPI widget for the second in the Portal.

---

RELATED TASKS

• Creating a campaign
• Creating a metric
• Continuously measure end-user satisfaction (Community)

RELATED REFERENCE

• Response rates for employee surveys (Nexthink blog)