>> Welcome everyone. On behalf of the National Institute for Occupational Safety and Health and the Total Worker Health program, thank you for joining us for introducing the NIOSH worker well-being questionnaire. We're so pleased to have you with us today. For those new to the series, a quick reminder that we'll hold all questions until the end for a moderated question and answer discussion. And now I'd like to pass it over to our moderator for today's webinar, Chia-Chia Chang, to introduce our featured speakers and the topic for today's webinar. Thank you.

>> Thanks a lot for joining us today. We're extremely happy to offer this to you. I am the moderator, I'm Chia-Chia Chang in the Office for Total Worker Health. Today, we will have two fantastic speakers about this very relevant topic. Dr. Ramya Chari with the RAND Corporation. And Dr. Gwenith Fisher with Colorado State University. NIOSH started this initiative to examine worker well-being back in 2014. Very early on, we were able to bring on RAND to join us in this long-term effort. And we really appreciate their expertise and their leadership in doing this. So it's great to have Dr. Chari join us today. And then, toward the end we were able to bring on Dr. Fisher for her tremendous expertise in psychometric analysis. As well as occupational health psychology. So it is a pleasure to have these two available to share our work with you. I am moving the slide. I hope it moves. This is the outline of what we have planned for you today. I'm starting off with an introduction about why this is such an important topic. Then Dr. Chari will discuss the development of the framework and the instrument. Then Dr. Fisher will discuss how to use the NIOSH WellBQ, the instrument, and how we can apply that to really advance our knowledge about worker well-being. We will end with plenty of time for questions and answers. And we look forward to hearing from you. I'll start it off with brief look at this initiative and worker well-being and why this is important. You may have heard about the Gross National Happiness Index. It was created by the Country of Bhutan. The idea is that instead of the merely focusing on something like GDP, Gross Domestic Product, or economic measures, the country also looks at non-economic aspect of well-being. I'm sorry. Just to make sure that you can hear me okay. If I turned off my video, hopefully the sound will be good. Bhutan looks at non-economic aspects of how the country is doing, including well-being in their national happiness index. They look at measures such as logical well-being, time used, cultural diversity and resilience, good governance, community vitality, and living standards. And internationally, the OECD, the Organization for Economic Cooperation and Development, also developed a well-being framework. They have reports regularly about how countries are doing in dimensions such as in common wealth, job quality of work, health knowledge and skills, subjective well-being, safety, work-life balance, social connections, civic engagement. On the broad level, those are you know, country approaches and within the United States, we have many initiatives, also. The National Academies of Medicine launched a few years ago the action collaborative for clinician well-being and resilience, that is specifically focused on healthcare providers, but as an example of the many different things that are going on. Other ways of looking at well-being have been developed by The National Wellness Institute and Gallup. Those are about general well-being. What about what will be specific to the workplace. There certainly is widespread recognition of the importance of well-being of workers and how it relates to organizations. Virgin Pulse Well Steps, he wrote, there is the widespread recognition by corporate leaders of the importance of worker well-being. In light of that NIOSH has included well-being as is one of our many priorities. Within NIOSH advancing well-being is one of the strategic goals for the Institute as a whole. Many of you probably know about NIOSH, NORA National Occupational Research, which has ten sectors and seven cross sectors. And the Healthy Work Design and Well-Being cross sector is new. So we have well-being, as one of the cross sectors. And of course the goal is to advance well-being across all the different industries and occupations. And then within our beloved Office for Total Worker Health. Where our goal is also to advance work well-being. So we talk a lot about well-being, but what is it? What does it really mean? And how do we know when we have achieved it? That is what prompted this project. And I am very pleased that NIOSH has invested in this. I'll hand it over to Ramya to discuss the initiative and how we got started in looking at well-being, defining well-being and then moving on to measuring well-being.

>> Thank you, Chia-Chia, for your introduction and for orienting everyone to the importance of a worker well-being. When we began this work, we had a deceptively simple question. What is worker well-being? And, as we quickly found out, the answer to that question really is anything but simple. So in my part of the presentation, I will take you through the development of a worker well-being framework and the NIOSH WellBQ, which is the survey instrument that measures this concept of worker well-being. So to fully understand worker well-being, we posed three questions. First, how should worker well-being be defined? Second, how should worker well-being be measured? And then finally, as Gwen will present in her section, how can worker well-being measures improve workplace programs, policies and practices? So to answer the three questions posed, our project first developed a definition and a framework for worker well-being. Following that, we developed a measurement tool, the NIOSH worker well-being questionnaire, or the NIOSH WellBQ. And then finally, we looked at some practical applications of the NIOSH WellBQ. So I'll go into more detail of the definition and framework for worker well-being. So we carried out a comprehensive [inaudible] reviewed in the gray literature. Gray literature meaning, white papers or government documents or anything that really is not peer-reviewed. And then reviewed the definition and measurement of well-being. So we focused on multiple disciplines, including the social sciences, the biomedical sciences, economics, [inaudible], so a lot of different disciplines. We looked at review articles, meta-analyses, conceptual/theoretical, and methods papers. And our search terms included the word well-being, as well as similar constructs, such as happiness or flourishing. General well-being and work-related well-being in the literature. So overall, we identified a little bit over a thousand articles and books, and this was a lot so to scope further, we implied inclusion and exclusion criteria and really focused on review or conceptual or theoretical articles and books on well-being. And we also selected a sample of empirical studies just to confirm some of the findings and the conclusions that were being offered in the larger review articles. So altogether, we ended up reviewing 141 sources that really informed the development of the worker well-being framework and also a definition. So what we found was in the literature, there are lot of different terms to capture this idea of well-being as related to work. And you can see a sample of these terms in the word cloud that's on the slide. And it really runs the gamut, from psychosocial to physical to emotional to functional. You can see words here like happiness or enjoyment, which is indicative of well-being, as this pleasant emotional state. But then there also words like accomplishments or mastery, which is indicative of well-being, as having meaning or purpose in life. And so there's a lot of different ways to think about and conceptualize well-being. And the common elements that we found is that there's no one commonly accepted definition of well-being. There are many models of well-being. And there is no consensus as to the components that make up well-being. So as you can imagine this was a great place to start this entire project. In addition, we note that well-being can be context free. As in just overall well-being not tied to a specific setting or a population, or it can be context dependent as in well-being a workplace setting or state of well-being of healthcare workers in particular. And so are review covered both context free and context dependent. So where we ended up with conceptualizing worker well-being as a collection of different indicators that each give you a really different piece of information about worker well-being as a whole. And so we ended up thinking of well-being in the same way that we might think about a concept like the weather. And that's what we're showing you on the slide. The weather, of course, can be conceptualized in multiple ways as well. So if I ask someone, you know, hey, how's the weather today? One person might say the weather is great. It's really warm out today. But another person might answer. The weather is really great because it's really sunny. And still another person, you know, might talk about how the weather is wonderful because the humidity is low, and so on and so forth. There might even be a person who says the weather is awful because it's sunny and, you know, and they love the rain . So there are a lot of different ways to define weather and really your total picture of the weather is looking across multiple indicators. But importantly, individuals can also choose what aspects of weather to focus on according to what is important to them. And so we felt that this was a really nice way to also conceptualize this concept of well-being. So using the literature, we ended up pulling out any domain or indicator that researchers had used in their own definitions of their own measurements of well-being and we put them together and then grouped them into these categories, or different domains. And so our framework really presents a snapshot of worker well-being across several different indicators. And we realize that the indicators may be causally related, just like they could be with the weather. But we're treating them a separate indicators of the same overall phenomenon. So on this slide, I present the worker well-being framework along with our worker well-being definition. And we ended up defining five domains. Workplace physical Environment and safety climates. Workplace policies and culture. Health status. Work evaluation and experience. And then home, community, and society. And in the later slides, I'll define these domains further. And on the slide is also our definition of worker well-being. And this definition is as follows. Worker well-being is an integrative concept that characterizes quality of life with respect to an individual's health and work-related environmental, organizational, and psychosocial factors. It is the presence of positive perceptions and the experience of positive perceptions and the presence of constructive conditions at work and in other areas of life that enables workers to thrive and achieve their full potential. And so I want to just unpack this a little bit more. If you look at the different elements in this definition, you can see how the literature review informed it. We talk about worker well-being is an integrative concept that encompasses multiple domains and multiple indicators of well-being. Therefore, it's multidimensional. We also talked about well-being as both the experience of positive perceptions and the presence of constructive conditions. So this definition marries these different concepts of well-being that we found in the literature: that well-being is this subjective concept that includes one's emotions or their subjective judgments or evaluations about their life or their work. But it's also an objective concept that well-being is about having the conditions at work or in other areas of life that allow you to succeed or the opportunities that allow you to thrive and to achieve. And so I also want to make a note about our use of the phrase worker well-being. So in a review of the literature, we found many terms that describe this concept of well-being in work and these are terms such as, work-related well-being, workplace well-being, well-being and work, well-being at work, occupational well-being, or employee well-being. And we ultimately decided that the term 'worker well-being' really captured the ideas that were trying to convey when we talk about and when we think about total worker health. So we thought that this term worker well-being, really conveyed a more holistic concept. For example, the term employee tends to tie an individual to an organization, whereas worker may convey a greater sense of individual agency but still within this work in context. So if you think about the example of someone who is self-employed or say an author or a writer, the term employee, doesn't really seem to fit. And the same issues arose with terms that tied individuals to a setting, such as workplace or worksite well-being, where today, more than ever before, people might work in more flexible ways and in different locations or remotely. So it wouldn't make sense to tie things to a specific place. And so finally just this term worker as opposed to some of the other terms we saw like workplace, or occupational work-related well-being, it's the only one that calls out an individual identity, in a sense. And for many people we spend a lot of time at work. We spend a lot of time doing work and who we are as a worker can make up a large part of our overall identity as a full person. And do this was another reason we decided to go with the term worker. Now I basically addressed that first question of how do we define worker well-being. Next. I'll discuss how we went about determining how to measure worker well-being. So to answer the second question, our project posed and to develop a measurement tool for worker well-being. We carried out a number of tasks. We obtained input from an expert panel. We conducted cognitive interviews and pretesting of a pilot survey instrument. We field-tested the pilot survey. And finally, we analyzed data, we refined the instrument and we developed scoring guidance. I'll briefly go over each task in the next few slides. So for our first task, we identified a number of existing well-being instruments, and we pulled out the measurement items from those instruments. So we ended up with a database of about 500 to 600 items to consider for framework. We then convened an expert panel consisting of 12 scholars or practitioners with expertise in different areas of well-being in work. And these areas included general workplace or worker well-being, including perspectives from positive psychology. They included expertise in occupational safety and health. The panel had expertise in workplace policies and benefits and effects on health and well-being. The panel also had expertise in occupational psychology, as well as the working community, which was important for addressing that external context domain. Home, community and society. So with the database of items we went through a prioritization process with the expert panel to select the subdomains and subdomain constructs for each of the five domains. And so what I mean by that is that within those five domains, we had to figure out what are the topics that we wanted to cover within each of those. So to help with prioritizing those kinds of topics that would comprise each of the domains, we applied criteria. Such as whether a concept had existing measures already, the importance of a concept as detailed in the literature, whether a concept overlapped with others. And of course the measurability of the concepts. And so following this prioritization process with the panel, we ended up selecting 135 items that were drawn from 27 existing instruments or sources. And then we obtained permission if it was needed for their use for noncommercial applications. We also created 52 new items for a total of 187 items in the initial pilot worker well-being questionnaire. So then following development of the pilot questionnaire, we first conducted cognitive testing to assess the content and clarity of the instrument. We used a convenient sample of nine people from RAND who worked in a variety of RAND office related occupations. Such as administration, mail services, conference services, plant engineering, custodial operations. And the cognitive testing resulted in just minor revisions or clarifications of wording of the items in the response scales. Secondly, we did a pretest of the pilot questionnaire. So we use GFK's knowledge panel. And this is a probability-based online panel that's designed to be representative of the civilian non institutionalized population of the US. And a sample 25 people were drawn from knowledge panel to assess things like whether the mechanics of the online survey were functioning properly. So were the skit patterns working? We also want to look at the burden on respondents in terms of survey length and so the pretest went well and it didn't result in any changes to the questionnaire. Thirdly, we performed in a field test where the survey was sent to about 1900 panelists in the knowledge panel. To be eligible, panelists had to be 18 or older and employed full or part-time. A total of 975 participants completed the questionnaire. We had a response rate of about 52% and respondents were split evenly by male-female, arranged age from 18 to 84 years and they represented a range of occupational industries. And fourthly, after we received the data from the survey responses, we performed psychometric analyses to evaluate candidate items or measures in the survey. These methods included exploratory factor analysis, confirmatory factor analysis, examination of item characteristic data using item response theory analyses. Examination of correlations among items and scales, and then also assessments of internal consistency and reliability for the scales. Fifth, using a combination of these analysis results and expert judgment, we finalized the questionnaire, paying particular attention to the flow of the survey and its validity through an examination of expected relationships among items and scales in the questionnaire. And then finally we developed scoring procedures and instructions. So the final instrument is comprised of 126 items. Ultimately, we removed 61 items from the pilot version based on the interpretation of results from psychometric analyses. And the final survey takes about 15 minutes to complete. So now that you've heard about how the NIOSH WellBQ was developed, I will briefly present those topics, or those subdomains and subdomain constructs that are measured within each of the domains of the WellBQ. So the first domain is workplace physical Environment and safety climate. This domain consists of factors that relate to the physical aspects of workplaces in both the physical and psychological safety features of the work environment. So you can see here the types of topics that are measured in NIOSH WellBQ under this domain. There are items that capture overall workplace safety. So this is a global assessment of workplace safety. Workplace safety climate. So these are more specific assessments of safety such as asking about different ways in which management does support safety in the workplace. We have items that look at physical work environment satisfaction and so it looks at satisfaction with environmental conditions in the workplace. The physical surroundings, the pleasantness of the environment and also the presence of accommodations for disability or other conditions. We have items that look at job discrimination for age, race or ethnicity, and gender. And then finally have items that look at work-related sexual harassment, work-related physical violence and work-related bullying. Just trying to go back. I think I advanced that one. Perfect. Thank you. So next, we have the domain of workplace policies and culture. And this domain refers to organizational policies, programs and practices that have the potential to influence worker well-being. And so this domain has items that capture topics such a supportive work culture. And so, you know, items that look at whether workers are treated with respect, or feeling valued, organizational support and recognition. This domain also looks at health culture at work. And so looks at perceptions of organizational commitment to health and supports in place for encouraging healthy behaviors. We have items that look at availability and also types of job benefits. The availability and types of health programs at work, such as wellness programs like on-site fitness centers, access to healthy foods, that sort of thing. There are items in this domain that look at trust in management. Items that look at work to non-work conflict and also non-work to work conflict. So these are the work like integration or work life balance measures. And then finally under this domain, we have items that look at workplace or schedule flexibility. Such as the freedom to vary your work schedule, or freedom to work wherever is best. The next domain is health status and these are aspects of individuals lives relating to their physical and mental health, and there are many topics under this domain of the NIOSH WellBQ covers including overall health, days of poor physical health. The presence of a set of chronic health conditions, presence of insomnia, the days of poor mental health, overall stress across four areas; health, finances, relationships and work, poor mental health. There's assessment questions about depression and anxiety. And then we have a series of questions about health behaviors like degree of physical activity, tobacco use, alcohol consumption, risky drinking, healthy diet,s sleep hours, sleepiness at work. And then there items that look at cognitive functioning limitations and work limitations in terms of functionality, work productivity and then finally work-related injury and injury consequence or severity. Next is the work evaluation and experience. And this domain refers to individual's assessment of the quality of their work life, including satisfaction with aspects of the job, meaningfulness of work, job engagement and emotional state at work. And here the NIOSH WellBQ has items that capture topics such as assessments of supervisor and coworker support, assessment of job security, work-related, positive and negative affect or the emotions felt while working. There are items on the overall satisfaction with the job, or the wages, benefits and advancement opportunities. Job autonomy, given the freedom to decide how do you work. There's also items on meaningful work, looking at whether the work a person does is meaningful to them or serves a greater purpose. There are items in job engagement. Is the job, is it inspiring? Like how much a person is immersed in work or how eager they are to get to work. There are items in time paucity and work overload, assessment of whether there is time to do everything. And finally an item on work-related fatigue. And finally, we come to the final domain. Home, community, and society. And this domain refers to the external context or aspects of individual's lives that are situated outside of work. But may still influence their well-being at work. And so items in this domain capture topics like life satisfaction, financial insecurity, support outside of work in terms of social and emotional support. And the extent to which people are involved in activities outside of work. And so, I know that that was a lot of dry material, but I wanted to make sure you came away with a good understanding of what was included in this in the survey. And so the final slide for this part of the presentation really summarizes the domains and then the subdomains that are part of the NIOSH WellBQ. And so these are kind of the higher level categories that all those topics I just went through fall under. So to sum up, I just want to revisit the definition of well-being that I provided earlier. So you can see how we went from constructing a definition to then operationalizing it in this survey instrument. So the beginning of that definition is worker well-being is an integrative concepts that characterizes quality of life with respect to an individual's health and work-related, environmental, organizational, and psychosocial factors. And so these are the five domains that the NIOSH WellBQ covers. The second part of that definition for worker well-being is the experience of positive perceptions and the presence of constructive conditions at work and in other areas of life that enables workers to thrive and achieve their full potential. So as you can see on the slide the WellBQ really encompasses the positive perceptions piece in subdomains like satisfaction or meaning in the work evaluation and experience domain. The WellBQ also encompasses aspects related to constructive conditions at work in other areas of life in subdomains like you know, safety climate in the workplace physical environment domain or health measures in the health status domain. And then of course the subdomains in the home, community, and society domain. So we really tried to take that definition, that framework we developed and operationalize it through this survey. So next, Gwen will present on how the NIOSH WellBQ can be used in its practical applications. Thank you.

>> Thank you so much, Ramya, for your excellent presentation and I've already seen that there are number of questions in the Q&A portion. So I want to answer, while I'll be presenting the slides that I have here. I will do my best to address some of the relevant questions in this section and then leave as much time as we can so that we can answer the remaining questions that you may have. The focus of my presentation is on practical applications of the NIOSH WellBQ. So whereas, Dr. Chari has shared how we developed the WellBQ and covered the broad list of many, many dimensions that are included. I will talk about how we use it. So first and foremost, importantly, this is where you can go ahead to access the NIOSH WellBQ. I'm hoping that we can retain your attention and that you don't all immediately turn to the Internet, if you haven't already, to go look at the details of the instrument. But it is there. And most importantly, it is available freely for public use and can be shared, copied, and distributed for your general use, which I will describe. One of the extreme benefits of the NIOSH WellBQ is that it was designed to be applicable and relevant to workers across all industries and occupations. And I want to underscore and reinforce what Ramya already said about the choice of the word worker instead of employee. Because importantly, we recognize that work arrangements take many forms. And if we refer to employee there may be people who feel excluded on account of being a contract worker or freelance worker, but rather broadly. Additionally, the vast range and breadth of the constructs and topics in the NIOSH WellBQ should be applicable across all industries and occupations, although some more so than others. Importantly, and I would say one of the most frequent questions that we've received since developing this instrument is whether it's appropriate and possible to just use a subset of the items. We strongly recommend and encourage using all of the items in the instrument rather than cherry picking and just focusing on a few constructs. This is for a number of reasons. First and foremost, based on a NIOSH total worker health definition of what constitutes well-being, we think all of these topics are important when thinking about well-being. And it becomes really difficult to say that some aspects of well-being are more important than others. And therefore, by assessing all of these things, and using all of the items there's an opportunity to capture the breadth and benefit of looking at well-being in this really wide context. Particularly taking into account the multidimensional approach across the five domains. Additionally, and this is really important as well is that although many of the items that are in the current NIOSH WellBQ came from other existing scales, we sought permission to make sure that it was appropriate to use all of these items in the NIOSH WellBQ and that they could be distributed. However, there are some items that, when we obtained permission, only gave us permission to do so when using the entire NIOSH WellBQ. In other words, if anyone were to use just a subset of those items and ones that might be copyrighted by individual authors elsewhere, that our permissions for distributing this instrument do not cover the use of just single specific items. And also importantly is the notion that the instrument, although free for public use, that there are some items that are indicated that they, that we cannot charge fees, or others cannot charge fees, when using those items. And so that's just something to keep in mind as follows, is that it's free for use, but at the same time, it should not be used in a manner where others are profiting from or collecting fees for the use of this instrument. In other words, that wasn't really why it was developed. So importantly, a very common question in very practical survey settings is how long does it take to complete the instrument. And the answer is approximately 15 minutes. When we develop the initial version and validated it, the initial version that constituted many more items longer, but we were able to pare it down to the most essential items and it takes roughly 15 minutes per administration. Which in the grand scheme of survey time is fairly short when taking into account how many different aspects of well-being are included in the instrument. There was already a question about demographic characteristics and specifically why industry and occupation were not included in the WellBQ. There are ten optional items available that assess participant demographic characteristics. Many of these items were taken based on large national surveys, like the National Health Interview Survey and others that are broadly used. So In other words, we didn't reword those items, but borrowed from existing surveys and how this is commonly done. However, we want to emphasize that the demographic characteristics are optional. Because really, which demographics are gathered and how they're used really are going to depend on the specific purpose for which the survey is being conducted. And rather than giving a blanket list of industry and occupation, there may be certain situations when, for example, the survey might be used within a single organization. And then asking about industry really isn't relevant to the extent that that one organization should all fall under a single industry umbrella. And likewise there may not be occupations that are relevant in all settings. And so ultimately the selection and choice of participant demographic characteristics should be up to the user to decide and use as is seen fit, although there are ten items that we have compiled and that are included in the code book just as examples of what can be done. And so we just encourage those using the NIOSH WellBQ to add, exclude or edit demographic items based on your needs and what will serve your purpose. So another question that you may have is why might somebody use the NIOSH WellBQ, how can it be used or what purpose can it serve? One is that you could use it for benchmarking within an organization. It can be done as a worker climate assessment to assess well-being and get a sense of overall how are workers doing across these multiple domains of well-being, based on this total worker health definition. Or it might be done to look, you know, at the workforce more broadly, in the sense of a surveillance survey. This can be done in both HR or applied settings, as well as in academic research. The second one is to look at changes over time. So for example, if an organization were to implement a new HR policy or practice or develop some kind of intervention, and then look to assess the effectiveness of that intervention on aspects of well-being, it would be an opportunity to use this tool in order to examine changes over time and evaluate the effectiveness of those tools. Third, is that you might be able to compare results across groups. So within an organization or within a facility, you might be able to compare how workers in some jobs or in some departments might be doing in terms of well-being compared to others. There was a question of the chat by Dr. Penny asking about how the instrument was developed with regard to certain occupations or settings in which there may be some workers in higher risk situations, such as those who may be more prone to physical violence or to certain physical hazards in the workplace. And so one benefit of the NIOSH WellBQ is that the questions are broad so they should be applicable over those situations if somebody experiences workplace violence, they would, based on, I don't recall the exact response scale for those items right off the top of my head. But there'd be an opportunity for workers who experience violence to respond based on what their experiences are. And workers who do not experience it or experience less of it would just likely rate on that worker type response scale a response that is a little bit lower and so across different types of jobs, different types of hazards risk, there would be opportunities to look at where, how workers are faring with regard to those issues that we would expect that those that are in jobs that might be considered more hazardous or where there may be more acute exposure to particular hazards that workers would [inaudible] in a manner where we would see perhaps lower levels of well-being with regard to those specific issues. So there are, again, are opportunities for comparing groups and so forth. What we do not want to do, and the NIOSH WellBQ was not designed in order to form clinical judgments of worker well-being. So, for example, although we ask about physical health, the purpose of the instrument is not to diagnose workers with particular chronic diseases or medical conditions. And this also goes for example, for mental health with regard to depression, anxiety, post-traumatic stress disorder, or any number of other psychological or psychiatric syndromes or symptoms. We're not looking to have this be a diagnostic tool for forming clinical judgments, but rather to gather broad information about worker well-being. And similarly, we are not recommending that it be used with regard to firm thresholds to signal clear actions, but rather as a more general measure. I'm happy to talk more about that after the presentation if there are further questions. I'm pleased to say that the translation team at the Center for Health Work and Environment, which is a NIOSH Center of Excellence in Total Worker Health, has developed a Spanish version of the entire NIOSH WellBQ. And to give an update on where this project stands, there's already been the initial translation into Spanish, as well as, for those of you not familiar with a survey or scale development translation process, we've also done what is considered standard practice to fact translate the instrument from Spanish back to English to evaluate the quality of the translation and then to review that in detail. And make adjustments as necessary. The next step in that project will be to conduct an empirical study to validate the Spanish version of the instrument, and the Center for Health Work and Environment will move forward with conducted network. While also of course, partnering with and keeping the NIOSH Total Worker Health Team informed of how that goes. Oops. One too many. So with regard to scoring you may be wondering okay there is this instrument the NIOSH WellBQ, we know the items. How do we score the instrument? The quickest answer to that is that there is a detailed codebook that is available on the NIOSH Total Worker Health website that provides scoring instructions for each of the items in the WellBQ. And there are individual items that could be scored as basically single item measures, as well as multiple item scales. Where we would then recommend a composite across the items that are in those scales. What has not been done at great length at this point, is a composite scoring algorithm with regard to combining items and scales across domains. How one might do that and the results you might find with likely depend on the population that you are studying and the purpose of using the instrument. And so there is a contingency approach to exactly how it would be scored for combining items may vary depending on the results and depending on the purpose of using the instrument. But basic scoring is already available in the codebook. And there is a very detailed guide that describes the NIOSH WellBQ, and the development and much more about the instrument that Chia-Chia Chang, Dr. Steven Sauter and others have put together that really goes into quite a bit of detail to specific items. So the next issue that I would like to talk about is the ethical issues associated with this. So one question you might have is about worker privacy when conducting or using the NIOSH WellBQ. Another issue that has come up quite a bit is perhaps a sense that it is NIOSH that is collecting the data and storing the data from the use of the NIOSH WellBQ. And that is not the way it is intended to. Nor is it what is happening. In other words, this is an instrument that has been developed and validated and is available to all of you and others in the community for broad use for your own purposes in organizations to accomplish work that I've described. And as such, we really encourage you to follow the best practices in human subject approach. And to follow the IRB guidelines and rules for your institution or organization. Obtain permission from an institutional review board. So for example, most universities and some other nonprofit organizations will have their own IRB. And so it's important to go through the IRB at your organization, if there is one that governs work of instruments and uses of space like this. And follow those guidelines. It's important to follow best practices in human subjects research with regard to informed consent. So letting people know what it is that you're doing. Why they are doing it. Describing how the data will be used and how privacy and confidentiality will be maintained. And then doing the absolute best to protect worker privacy in gathering this information about their well-being. And protecting the confidentiality of workers. Related to that, goes back to what I'd already indicated about ways to ask about demographic characteristics, but being sensitive and careful in choosing demographic characteristics so that individual workers cannot or won't be identified on the basis of answering questions about who they are. And finally to communicate results to relevant constituents in a timely manner. When possible, there are strong benefits to sharing results back to workers themselves who have participated in research. The research methodology literature talks about the benefits of sharing results so that people understand how their data have been used. And they have an opportunity to see what is going to happen as a result. One ethical practice that I follow in my work and really try to instill upon with my students is the notion of not asking questions of workers unless there is a preparation for being able to act upon the results. In other words, when we ask people about their well-being and how they're doing, if there isn't a plan in place to address those answers and do something about the results, it begs the question of what the purpose of conducting the assessment is to begin with. And recognizing that workers may develop an expectation of something happening as a result of having been asked about their well-being. And so it's important to organizations that are implementing this, particularly in HR practice or in wellness initiatives are prepared to handle the results and act upon the results as needed. Again, I'm happy to talk about that in more detail. Nearing the end of my presentation, I just want to encourage those using the NIOSH WellBQ to follow and use best practices in conducting surveys. There are many ways to go about this. There are resources available. For example, the Society for Human Resource Management has a wonderful kit available. There's a link here to that toolkit. And it's important to clarify what the purpose is for doing the survey. And to provide resources to support the survey and those using the results. It's also really helpful to obtain support from senior management or leadership in organizations. Again that goes back making sure that there is the plan and resources available to support what the results may be. And importantly, as I indicated before, to protect participants' privacy as well as their confidentiality. We are asking questions about well-being when using the NIOSH WellBQ. And it's important that this be treated as sensitive information. We are asking about health and wellness. And we want to make sure that we respect that information and it not be shared publicly or inappropriately with others. And lastly again, the importance of sharing and communicating the results. NIOSH has developed an excellent tool guide with regard to providing information and resources for action. And this toolkit or this resource has information about various aspects. For example supervisors for coworker support. And what can be done as a result of the surveys. So if you are planning to use and implement the NIOSH WellBQ, I encourage you to also look to these resources for action when developing a plan for how to not only use the instrument, but for making a plan for how to act upon the results. So I want to thank you for your time and attention and listening to the presentation. I just want to ask Sarah is we have the poll questions ready. I know there were some questions in the Q&A that we wanted to make sure that we get to. But we also may have a poll for you. Here it is. You should have a poll that came up on your screen. So please just go ahead and choose the option that is most appropriate for you. Okay here are the results from the poll. So, 52% indicate occupational safety and health as a primary profession. And the second most common is health promotion, wellness and health education. Followed by other. And a handful of people in human resources. Risk management, or organizational or employee development. So thank you for that. It's really helpful to know a bit about you and the work that you're doing. Which can hopefully guide us in our discussion. So one final poll question is how do you anticipate being most likely to use the NIOSH WellBQ? You should be able to check multiple answers here if the answer is more than one. Or if you're unlikely to use it you can indicate that as well. So now the results of the poll should be showing, with almost half of you indicating that you would administer the instrument as part of a research project. Followed closely by administering the instrument within your organization or recommending the instrument to be used by clients. And about 14% saying that you would use the instrument for public policy. And a small number, 6%, saying that you are unlikely to use it. Thank you so much for your input. At this point we'll move ahead with the Q&A portion of our session. Chia-Chia, I'll perhaps turn it over to you for moderating the Q&A.

>> Yes, thank you very much. And thanks a lot for your engagement for all on the call in responding to the poll and for the questions that you have provided. There have been many questions. I will start with one that I will direct to you Ramya. There was a question about one of the, it's a very specific question. Apparently, many of you have already looked at the questionnaire, which is good. One was [inaudible] why is the financial question phrased in the negative, versus the positive framing of that? Ramya, can you address that?

>> Yes, I think this is referring to the financial insecurity measure in the home, community, and society domain. And it's a good question. For the most part when we took these items or gathered these items from their original sources. We tried to stay as true as possible to those original sources and not change the wording when we didn't need to. And so I think this is just one of those examples where the original source was framed in that way. And so we decided to keep it framed in the negative versus changing into the positive.

>> Thank you. And there were many questions actually going through them. There was also a question about, do we cover burnout at work.

>> I don't believe that we specifically cover burnout at work. But we cover lots of factors that are related to burnout, both as antecedents or stressors that are likely to lead to burnout. As well as outcomes very commonly associated with burnout. Such as fatigue. Such as exhaustion. In other words, be we capture burnout, but in more specific ways of looking at fatigue and exhaustion. And even some of the mental and physical health outcomes associated with burnout rather than just asking, for example, do you feel burned out. Ramya if you have anything else on that you'd like to add please go ahead.

>> No, I mean not specifically. I will say there might be a number of questions about you know why certain concepts or certain, you know, other kinds of issues aren't in our survey. And you know, I did talk about that prioritization setting process that we went through with the expert panel. Like there's a lot that could have gone into this. And at the end of the day we needed to have this balance of not having an instrument that was you know, too long and too unwieldy. And so it was hard to make certain decisions at times about what would go in and what would go out. But at the end of the day, you know I think it's fair to have a discussion about some of the things that might've been left out. You know, of this iteration of the survey. But I think for the most part, given our process, you know, hewing closely to the literature. Working with the expert panel that we ended up with you know, an overall comprehensive instrument that covered most of the areas that we felt were most important. And I think there's always good discussions to be had about, you know, specific aspects that might've been left out. Or specific topics that might have been left out.

>> Thank you. I'll ask one more sort of about these very specific things. Someone had also asked whether we at all, whether there's anything about, sorry, safety related behaviors outside of work that could potentially be, impact health and well-being performance in the workplace.

>> No, so for, you know that domain of home, community, and society. That was one where we felt there wasn't as much research out there in terms of the factors that happen outside of work that directly impact worker well-being. You know, we believe that this is an area that is ripe for more research in order to assess those sorts of relationship. And identify those factors. And the ones that did make it into this domain were the ones that we could find evidence for in the literature. And so it's not to say that you know, issues like safety or other topics that happen outside work are not important to work. But we just couldn't support it with the evidence that was out there.

>> Thank you.

>> I'd also just like to add in response to that as well as some of the other questions about the decisions to not include questions about specific topics that there are certainly other aspects of worker well-being where there are existing scales of the literature that go into much more depth. And there is a focus on depth, rather than breath. And one of our goals in developing and validating this instrument was something that would be broadly applicable. And also keeping in mind the overall length of the survey and trying to be sensitive to the fact that many organizations or other uses of the instrument, it wouldn't work well if we had an incredibly long instrument. And so we, in some cases, were faced with just some difficult decisions about making sure to be broad and not to be, to have too many questions that would be too specific. And take away from that broad approach of not being too long.

>> Thank you. Now I think that covers many of the questions about what we did or did not ask. Ramya and Gwen addressed that very well. Certainly keeping the instrument to, minimizing the burden of burden on the public was very important you know a 15-minute estimate is definitely accurate. And already people are thinking it's very long. So I'm not sure if we had even more items. We'll address some of the other questions. One is how do you control for anonymity for small organizations, specifically regarding health conditions?

>> So one recommendation I would have if using the NIOSH WellBQ in a small organization would be to limit the number of demographic questions that are answered. And to be careful about not sharing results based on too few responses. In my own work, I always try to aggregate a minimum of at least five or six responses to reduce the likelihood that any one individual person could be identified. So for example, if you're conducting this in a small organization and you're interested in knowing about job level or department, but there are only a few workers in particular job level or department, then perhaps not ask questions that will result in individually identifying people or sharing results in a manner in which it's possible to identify individual responses or views on the particular questions. Thank you. And then sort of related to that, given that the demographic questions and the industry occupational questions are voluntary, would it actually be possible to use this to compare one business with another, or comparing one facility with another? It seems like you would need to control for these issues. Would it be possible to use the instrument to make awards, or recognition, more punitively? What are some considerations for that given that demographic and industry questions are voluntary.

>> That's a really great question. And it's definitely one of the challenges associated with having the demographic questions be voluntary. I think it would be important if making comparisons between any two groups that there would be a metric by which to be there statistically controlled for, or otherwise make sure that you're comparing similar groups. In other words, apples and apples versus apples and oranges in making those comparisons. So in that regard, if collecting data to be clear and to document what information about the population or the sample. And to document which demographic items were used and what those distributions look like, but again, in a way that individual people can't be identified, but in a way that it would allow for recognizing similarities with or differences with other responses from other organizations, other departments, and so forth. Definitely an excellent question.

>> And then, so a little more on usage of this. What are your thoughts on using this instrument to assess the well-being of the impact of COVID-19 on frontline workers?

>> I'm sorry I had trouble hearing the last part of your question, Chia-Chia, could you repeat that please?

>> Oh, sure apply the instrument to, like the impact of COVID on frontline workers. Or maybe any workers.

>> I think this instrument could certainly be used to assess aspects of some of the issues that organizations and workers have faced during COVID. With regard to that, I think it's important to determine whether you are collecting data specifically due to COVID, or if it's just collecting data during the time of COVID. And being clear about what the purpose is. And how the survey is being used. And what the overall research question or policy issue is that you're looking to assess. But many of the items and domains within the NIOSH WellBQ seem to be relevant to issues regarding COVID. And the extent to which that's the case may depend on the population of worker, or the specific occupation in which people are working. So for example if a focus is on remote work and tele-work or ways that COVID has impacted work design and well-being, there may be other tools for looking at that in more detail beyond the WellBQ. But for a broad approach looking at multiple domains of well-being. This could be an excellent tool for looking at that.

>> Thank you. And on a broader level, there's a question of can human resources, or other organizational management professionals use this to evaluate their organization?

>> Yes. I think that is an excellent way in which this tool can be applied and used within an organization, is to get a sense of how are workers doing. And overall what does well-being look like? And are there certain domains of well-being that are better or worse than others, relatively speaking, within the organization. But it could serve an excellent purpose of being used as a broad well-being tool within HR or looking at wellness.

>> Okay, and there was a question about electronic implementation of the instrument. How could it be used as something like REDCap or Qualtrics?

>> Sure, so the items in the NIOSH WellBQ are publicly available on the NIOSH website. And then really it would just be a matter of setting up the survey within REDCap or within Qualtrics, Survey Monkey, or whichever survey tool or assessment method is used by an organization. But it can certainly be administered online in that manner. And I imagine that is a very common way. There still are many ways in which paper and pencil surveys or more traditional way, I shouldn't say traditional, but without the use of technology ways in which surveys are conducted. But I would expect that in many cases, setting it up online would be common and most likely to generate responses and have workers participate.

>> Thank you. And just to be clear, can you explain again why the demographic, or at least the industry and occupation questions were omitted. They are voluntary and they are included. There are options included, but why were they not actually part of the instrument itself?

>> The industry and occupation questions and other demographics were not part of the instrument itself simply because the instrument can be used for multiple purposes. And the demographic items that are most relevant are likely to depend on the specific use or purpose for using the NIOSH WellBQ. And so our recommendation is to add, exclude, or edit demographic items as needed to fit the purpose for the use of the instrument.

>> All right. and do you have guidance on how to best combine results from a group survey, or target or prioritize topics for a response [inaudible]?

>> My short answer to that, I don't mean to dodge the question, but just based on having conducted organizational surveys for 25 years. How best to combine and use it I think to some degree will depend on the purpose and the audience. And why the survey is being done and for who. And so, although the instrument is broad, and therefore can serve multiple purposes and multiple populations, the manner in which the data are analyzed and how they are combined may vary depending on the purpose.

>> That's a good point. Yeah, I guess it really depends on the usage and the specific environment in which it will be used. But I will say, following up on what you had said previously about follow up action, certainly we would recommend that if you're going to administer the survey that there is some plan to take action afterward. There was a question about is NIOSH working with ESG that's environmental sustainability and governance. Sustainability spheres to have this included as a recommended standard measure. We have not done that yet. But we certainly would be pleased to do that and we would do everything we can to reach out to those professions and networks to spread word about that. And if you have recommendations on how to do that, we welcome that. Please do reach out. I think that would be a great idea. One question was also, are there any thoughts of having a voluntary listing of employers who have utilized the questionnaire, be available as a user group to share experiences or lessons learned. That's a great idea. We have not currently established that. But we are definitely looking for ways to build on this in the future. So that there is benchmarking data and there is data from research projects that can be shared. So that going forward you could know maybe how you've done compared to others. So that is something that we would love to do. It would be in a long term plan to do that. But certainly in the short-term, even something such as a user group is something that we would be happy to explore. And I want to point out for what it's worth, NIOSH does have a LinkedIn group for Total Worker Health. And that could be something that we could start discussions about. That's a great idea. So thanks for suggesting it and we will explore that. There are so many questions. It's tough to keep track. Here's one sort of going back to the specific questions. There's a question about the timeframes of retrospective questions being different. You know some ask about over the past year, some ask about over the past 30 days. Some are have you ever. Could this be problematic? I mean I do know that we pull the items from existing instruments. And then the edits that we've made were often made to make those things more consistent. But what are your thoughts, Ramya and Gwen about whether this would be a problem?

>> Gwen may have more to add to this, but I'll take a stab. Chia-Chis as you said, we did draw from existing instruments and tried to stay as close to those instruments as possible. But you know the NIOSH WellBQ is as Gwen said it's about breadth. So it covers a lot of different topics. And we drew in many instances from instruments that were meant to examine a particular topic in depth. And those particular topics, you know, there may be best practices for how you know the timeframes for how you might measure something. And because we collected from a lot of different sources and were putting together a lot of these different well-being indicators. You know, it isn't always the case that it would be the right thing to do to make a standard timeframe for everything. For instance, some if I recall the questions on positive or negative affect or emotional state. You know, they have a shorter timeframe for which we ask people to recall those. Because those are motions. And those are fleeting. So we wouldn't want to you know have a longer timeframe for that. And so, I understand having these different timeframes might seem confusing, but in terms of what's best for answering that specific question for that topic that you're looking at. You know, what we tried to do was firstly make sure that this timeframe makes sense for what we were measuring. Gwen, do you have more to add on that?

>> I completely agree with everything that you said, Ramya, that makes perfect sense and I think your example with positive and negative affect is an excellent example. One that came to mind first for me is something like workplace violence. For example maybe a physical injury at work. And first of all, we hope that people don't experience those. But yet we also know that they do happen and that they are threats to worker well-being. However, hopefully they aren't situations that happen very frequently. And therefore response scales for some of those items take into account that it may be over a much longer period of time. And so again, if we were to harmonize and use the same timeframe across all of the items we would end up, one not going with standard practice in the measurement of specific [inaudible] or if it's related to worker well-being. And secondly, not necessarily measuring things in a manner that would make the data to be most useful. So if we were to ask just about violence or injuries in the past week, we would likely not see very much. So if we really want to capture the extent to which these things are happening, we would need a longer timeframe for doing it.

>> Thank you. There was a question that I think we touched upon before. But just to hone it in some more. Can organizations use this to compare themselves with the national data on certain questions? I think so, to the extent that such data exists. One of the things was you know, like we talked about, having some kind of some sort of clearinghouse, some sort of data sharing so that people would have those data available. Otherwise I think to the extent that we pull the items from existing questionnaires. It certainly could be used. What do you all think of that? Ramya and Gwen?

>> I agree with that, Chia-Chia. I don't have anything else to add.

>> Okay. And there was a question about scoring and whether it could be used as part of our administrative [inaudible]. I think Gwen had talked about that during her presentation. We strongly recommend administering the questionnaire as a whole. Because that was the propose to look at worker well-being, comprehensively and holistically. And you know see in our documentation that we list the scores for questionnaires from which we drew the items. So if there's a specific topic that you want to dive more into, you could certainly go into those source questions and dive further into it. In that case you would be you know, looking at specific topics. Not just, not comprehensively worker well-being, but certainly something that you could do. And that's why we provide those source instruments. And there was one question which I will try to address. Is it safe to assume that NIOSH [inaudible] panels will simply look for the use of the WellBQ if something, if a researcher is sort of pitching a total workup approach. I'm not going to comment on the NIOSH intramural process. So I can't comment on that. But I'm speaking for us on the team. We certainly think this is a great instrument and we think it will be very useful for organizations, and researchers, and policymakers. So if you have any questions about why you should not use this instrument or good circumstances to use the instrument, please for free to reach out to us. And I think we've addressed all the questions. I will ask, just to be clear, we are planning on publishing the results of the question, of a lot of the data analysis that we've talked. You'll see in the slides, that the slides reference a 2018 paper Ramya was the lead author and discussed the development of the worker well-being framework. So that is out there. Please, you're welcome to review that. And we, the three of us, and many colleagues are developing another paper to put in writing, many of the things that have been discussed here about the development of the instrument itself. So definitely look for that if you want to see it in the scientific literature. And there was one question that was asked. I want to make sure that we were clear on that. The question was how does the instrument adjust for the different types of conditions? Such as higher risk jobs across each measure. Or physical hazards and workplace violence. Gwen, I think you touched upon that in your presentation, but can you go through that a little bit again? Thanks.

>> Sure, I can just try to clarify that and answer that in a little bit more detail and say that the way the questions are asked, often, rather than being yes/no questions, they're typically on worker type or frequency response scales. And so individuals who are in jobs that include higher levels of risk, we would expect that worker who have experienced those situations more often would then respond to the items in a way that indicates experiencing them more often. Or to the extent that they are experiencing negative consequences as a result of certain work-related situations or work experiences. That they would indicate that in the survey. And so one of the things we really aim to do in the development of the NIOSH WellBQ is to not only identify items that are broadly applicable but also to have response scales that would potentially capture the full range of possible responses. In other words, workers who are in jobs where there are fewer physical hazards, or they're not exposed, or fortunately for them do not experience workplace violence. Then, there would still be an opportunity for them to answer the question. And to just answer in a manner that is consistent with their experience. There was a related question that I thought was excellent and important. Which is in comparing results across different organizations or different populations of workers, I think it would be important in making those comparisons, to know who those workers are. And for example, if we're talking about workers who might be in a high risk occupation or industry, that whoever would be doing the work would want to make note of who those workers are in order to make appropriate conclusions when comparing them to other groups who may or may not be in jobs with the same kinds of risk hazards. But in terms of, and we would expect that responses to the WellBQ would correspond with what those experiences are. In other words, I would expect workers who are in more physically hazardous work or work where they're exposed to more violence would respond to items indicating that. And those who aren't, would say no. And then you would have that comparison there. So if that's not clear please let me know. I'll try to explain it a little better, but that's it.

>> Okay, well with that. I just want to thank all of the presenters for taking the time to present today. This was a really great discussion. And some really interesting questions. So I hope that everyone who joined, enjoyed the presentation and learned a lot. Feel free to connect with us. Please visit our website. We have a group on LinkedIn. The URL is included on this slide. We also have a newsletter. Please subscribe, if you haven't already, because we send out quarterly updates about all things Total Worker Health. And have the Twitter channel as well where we provide updates, daily almost. So please connect with us there. And just once again, thank you all for your time. And have a great rest of your week and weekend. Thank you.