Implementation of a population mental health and wellness text-message service: a mixed-methods study

Abstract

Background

Despite the growing adoption of digital health tools as a means to support mental health, many individuals remain unaware of the variety of mental health resources available to them through this format. To address this knowledge gap, this study advanced the design, development, and implementation of a text-based service called SaskWell to raise awareness of evidence based mental health resources and create more immediate connections to these tools. The two primary objectives of the study were to assess and evaluate the adoption of SaskWell by focusing on user acceptance, satisfaction, and perceived benefit, and to identify factors which contributed to user engagement with the SaskWell text-based service. Both quantitative and qualitative data contributed to the final study results.

Results

This study utilized a co-designed text-messaging service to provide residents of Saskatchewan an important connection to digital mental health and wellness resources during the height of the COVID-19 pandemic. Using the RE-AIM framework as an implementation guide, four distinct cycles of SaskWell were delivered with modifications to the service in each subsequent cycle based on user engagement, feedback, and the direction of a community advisory group. Quantitative data was collected through user engagement text message response data, along with enrollment and exit surveys, while semi-structured interviews served as the primary means of qualitative data collection. In addition to the quantitative user data, these user interviews resulted in themes exploring motivation to sign-up for the service, perceptions of texting as a mechanism to deliver mental health resources, the impact of SaskWell on mental health and well-being, and beliefs on the future potential of text-based mental health supports.

Conclusions

Both the user engagement survey and the qualitative data supported the worth of ongoing efforts to refine and extend the use of text messaging as a means to engage citizens around the awareness and use of digital mental health and wellness resources. As the pandemic has receded into the background in many peoples’ daily lives, for healthcare providers, and others who continue to be impacted more heavily by the persistent challenges of this global event, this type of service may continue to be timely.

Background

The COVID-19 pandemic revealed urgent needs in relation to population mental health supports on a global scale [1,2,3,4]. According to the Canadian Mental Health Association, in any given year, 20% of Canadians experience a mental health problem or illness [5]. However, during the COVID-19 pandemic, a longitudinal survey found that between May 2020 – January 2022, around 25% of Canadians experienced moderate to severe anxiety and 22% of Canadians felt depressed [6]. The uncertainty, distress, and isolation caused by the pandemic has resulted in an echo pandemic of declining population mental health [7, 8]. Furthermore, the under resourced and overburdened Canadian mental healthcare system, continues to struggle to meet the demands and needs of individuals presenting with mental health problems or illnesses. Given these challenges, it is worthwhile considering that some mental health needs existing at the population level could be appropriately addressed through the use of existing digital health resources (e.g., apps to support mental wellness).

Given the need for social distancing during the initial onset of pandemic, and the limited capacity of service providers to support individuals, digital health tools and resources were rapidly adopted to support or augment care provision [9, 10]. For instance, early on during the pandemic, mental health care providers shifted from in-person care delivery to virtual care services delivered via telephone or videoconferencing when appropriate [11,12,13]. In a similar vein, those seeking mental health care supports, sometimes turned to consumer digital health tools such as web-based courses or platforms, mobile health apps, chatbots, and phone or text-based supports [14]. According to a 2021 pan Canadian survey, approximately 70% of respondents reported using a digital health tool to support their mental health in the past year [15]. Moreover, approximately one third of respondents were satisfied with the digital mental health tools available to them [15]. Unsurprisingly, other studies have found that those with increased rates of depressive and anxiety symptoms are more likely to use digital mental health tools [16,17,18].

Despite the growing adoption of digital health tools as a means to support one’s mental health, many individuals remain unaware of what mental health resources are available to them, be it digital or in-person [10, 19]. For instance, awareness of digital mental health tools was cited as a top barrier to using these resources in a nationwide Canadian survey where 32.6% percent of respondents indicated they were unaware of digital mental health supports [15]. Furthermore, for those who are aware of the available tools to aid their mental health, distinguishing which tools are evidence based and trustworthy is yet another challenge for those seeking to use these types of resources [15, 20]. This creates a clear need to provide more information to those seeking to improve their mental health and wellness, both about available digital mental health resources as well as which of these are trustworthy and evidence based.

In an effort to provide this support for improved population mental health during and beyond the pandemic, this study advanced the design, development, and implementation of a text-based service to raise awareness of and bridge connections to evidence based mental health resources and tools [21]. Specifically, the purpose of this mixed-methods study was to describe and assess the adoption of a text-based mental health and wellness support focusing on acceptability, satisfaction, and potential benefits for users. Saskatchewan was selected as the province of focus for the text-based service given the high number of residents reporting feelings of anxiety and depression during the pandemic and the limited number of mental health resources available to support the population [21].

The use of text messaging was identified as the most ubiquitous means of connection to the provincial target population. Saskatchewan residents are represented among the 91.3% of Canadian households that have access to at least one cellphone [22] and the 84% of Canadians with access to a smartphone device [23]. Consideration of data availability and general cellular connectivity was also taken into consideration in the study design. These factors were important as part of efforts to maximize connectivity and bridge existing digital divides that emerged early in the pandemic transition to virtual service delivery. Text-messaging services have increasingly been adopted and used as a means to support public and population health, ranging from smoking cessation to sexual health to perinatal health [24,25,26]. Moreover, text messaging is a low-cost, low tech and established technology commonly used by a diverse range of populations [27]. Recent initiatives have established text-messaging supports or services to deliver timely mental health information and resources to underserved populations [28, 29]. Given the previously detailed need to advance the connection between individuals and mental health resources and tools, a text-based intervention, SaskWell, was co-designed to support population mental health in the province of Saskatchewan, Canada.

Research objectives

In support of the research purpose, this study was guided by two main objectives, each with subsequent research questions:

• 1. To assess and evaluate the adoption of SaskWell by focusing on user acceptance, satisfaction, and perceived benefit.

This objective was further addressed through the following research questions:

• a. What is the average message response rate among users of the service?

• b. What are users’ experiences with SaskWell, particularly as it relates to satisfaction and perceived benefits?

• 2. To identify factors which contribute to user engagement with the SaskWell text-based service.

This objective was further addressed through the following research questions:

• a. What are the characteristics of participants who enrolled in the service?

• b. What demographic factors are associated with an ‘engaged’ user?

• c. What are user preferences for tailoring the tools they receive based on select characteristics?

Methods

Theoretical framework

The Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework [30,31,32] was used as a theoretical foundation to guide the ongoing evaluation and evolution of SaskWell during several intervention cycles. RE-AIM is a well-established framework for evaluating the impact of population health programs. Moreover, the scope and depth of RE-AIM can be tailored pragmatically [33]. For this study, the researchers leveraged three domains of the RE-AIM framework in particular: (1) Reach; (2) Effectiveness/Efficacy; and (3) Adoption, with more preliminary exploration of how scaled implementation and maintenance of the project might be possible beyond the scope of the study.

In RE-AIM, Reach pertains to the proportion and characteristics of individuals willing to participate in the intervention [32]. For this project, Reach supported an improved understanding of who opted to use SaskWell and highlighted population groups and regions where recruitment efforts should be focused. Effectiveness or Efficacy describes the positive and negative consequences of a new intervention [32]. We evaluated the Effectiveness of SaskWell by examining user experiences as it relates satisfaction and perceived benefits. Adoption pertains to the proportion and representativeness of settings in which the intervention implementation takes place [32]. To measure Adoption, we examined user demographics, usage data, and experiences to better understand engagement with, and feasibility of, this text-based service.

Study design

This study utilized a mixed methods sequential explanatory design. We purposefully selected a sequential explanatory design, beginning with a quantitative phase, followed by a qualitative. In this design, information was first generated through the quantitative focused data collection and analysis, driven primarily by usage and participant survey data. A qualitative phase using semi-structured participant interviews followed to allow a more a complete and contextualized understanding of user adoption and engagement with SaskWell. This study was conducted between September 2020 to August 2022. The Good Reporting of a Mixed Methods Study (GRAMMS) [34] and the National Institutes of Health’s Best Practices for Mixed Methods Research in the Health Sciences [35] were used to guide reporting of the study findings.

Study setting

As noted, this study took place in Saskatchewan, Canada at a population health level. As of February 2021, approximately 60 and 47% of Saskatchewan residents reported experiencing symptoms of anxiety or depression, respectively [36]. Furthermore, 12% of Saskatchewan residents also identified a need for mental health support but were not able to access mental health services during the pandemic [36].

Text-based intervention

SaskWell is a co-designed text-based service which was developed to support population mental health in Saskatchewan during the COVID-19 pandemic. A community advisory committee of six Saskatchewan residents was established to support the design, development, and implementation of SaskWell. The service delivered weekly text messages to participants over a course of 10 weeks. Those who enrolled were paired with an evidence based digital mental health tool (i.e., online course, mobile app, etc.) that corresponded to their reported internet and general technology access, and if preferred, their age or race. In addition to the digital mental health tool, users received two wellness tips (i.e., general self-care information, motivational messages, breathing techniques, etc.) and one polling question each week. Additional user feedback was captured throughout the service to determine whether specific messages were found to be helpful or relevant to users. Examples of the wellness text messages delivered throughout the SaskWell service are included in additional file 1.

SaskWell was launched in March 2021 and ended in April 2022. During this time, four distinct 10-week cycles of SaskWell were deployed. Each intervention cycle was further refined based on the feedback and lessons learned from the previous cycle. Additional details about the design, development, and delivery of SaskWell are reported elsewhere [21, 37].

Participant recruitment and enrollment

In partnership with the community advisory committee and other community and provincial organizations, SaskWell recruitment information was distributed across the province. Traditional recruitment methods were utilized (i.e., printed posters and flyers, word of mouth) in addition to more innovative recruitment methods (i.e., social media posts, the use of organizational email list serves for large scale distribution, webinar presentations, online discussion forums, and media news articles and interviews).

Those who were interested in signing up for SaskWell could do so by accessing the enrollment weblink, texting ‘JOIN’ to 759,355 or by calling a toll-free phone number. Eligible participants included Saskatchewan residents who were 16 years of age or older and able to comprehend English. To enroll in the service, users were required to complete a web-based enrollment survey that consisted of three main sections: (1) Demographic Characteristics (age, race, gender, place of residence) (2) Technology and Internet Access (smartphone access, internet access), and (3) Mental Health Self-Check questionnaire. A validated 14-item measure was used as the Mental Health Self-Check enrollment questionnaire aligning participant mental health status into one of three categories: Flourishing, Moderate, or Languishing [38]. In the first iteration of SaskWell, individuals who received a Languishing mental health score were excluded from the service. However, in subsequent iterations, owing largely to participant and community advisory feedback, the eligibility was expanded to include individuals who received a languishing result. The appropriate disclaimers and precautions (I.e., SaskWell is not a replacement for professional mental healthcare or treatment, connections to provincial crisis support lines, etc.) were expressed and provided to all individuals who enrolled. The full expansion of the service to all self-reported levels of mental health was well supported by the participants.

Quantitative phase

Data collection

A variety of quantitative methods were used to gather participant data in this study. First, the SaskWell enrollment survey provided detailed participant characteristic data. Second, back-end analytics and usage data from the SaskWell service was captured as a proxy to user engagement. Specifically, user response rates, unique and cumulative link clicks, and drop-out rates were tracked. Finally, at the endpoint of the service, an additional user experience survey was sent to all participants. The survey contained a series of Likert scale questions that further explored participant perspectives and experiences with the SaskWell service. This user experience survey was not mandatory to complete; but many users of the service completed the survey on a voluntary basis. There were several draws made during the course of the survey and participants selected offered a 10-dollar gift card of their choice from one of three vendors.

Data analysis

Participant characteristics and user experience survey data were summarized using descriptive statistics (e.g., frequencies). To understand user engagement with SaskWell, a response ratio was created related to how frequently users responded to the text messages delivered to their devices, with the recognition that not all users engaged in this way, as was further revealed through qualitative interviewing. The response ratio was calculated by dividing the total number of user responses to all message types (polling, wellness, feedback) by the total number of base messages sent by the SaskWell service. In iteration one, there were 21 base messages, and in iterations two, three and four, there were 31 base messages. Participants were dichotomized to “more engaged” and “less engaged” with a data-driven approach using a median-cut of the response ratio. ‘Engaged’ was dichotomously defined based on the median value (0.26) of the response ratio score. When the response ratio score was < 0.26, users were classified as ‘less engaged,’ and when the response ratio score was > = 0.26, users were classified as ‘more engaged.’ Moreover, univariate logistic regression models were used to investigate the association of factors with participants’ engagement with SaskWell. A chi-square test was conducted to compare users’ preferences for tailoring the tools they received through SaskWell to their specified ethnicity or age. Statistical analyses were performed using R version 3.5.0. Two-sided p-values < 0.05 were considered statistically significant. Given the exploratory nature of this study, no formal adjustment for multiple hypothesis testing was performed.

Integration: connecting the quantitative and qualitative data

The findings from the quantitative data and analysis were used to inform the subsequent qualitative data collection. Initial demographic and engagement data captured from the SaskWell service was used in the design of the semi-structured interview guide. Additionally, the initial results from the quantitative analysis aided in finalizing the objectives of the qualitative portion. This sequencing and flexibility maximized the value of the qualitative data in providing greater depth and breadth to the quantitative findings.

Qualitative phase

Sampling and recruitment

SaskWell users were invited to participate in virtual semi-structured interviews through an additional question provided at the completion of the aforementioned user experience survey. All those who completed the survey were provided a prompt to consider volunteering to participate in a brief interview. Those interested were then asked to submit their name and email address through a separate question embedded in the conclusion of the user experience survey and were later contacted via email by a member of the study team (IK). Users were also invited to participate in interviews through social media callouts and virtual bulletin board posts with use of SaskWell confirmed prior to interviewing.

Data collection

Semi-structured virtual interviews were done between December 2021 – April 2022. The interviews were conducted by two members of the research team (TR, IK) and were approximately 30-minutes each in length. Drawing from the RE-AIM framework, and questions raised from quantitative data findings, 28 participants were asked a series of prompts to further explore their experience in using SaskWell. The complete interview guide can be found in additional file 2. Interview participants provided informed consent and were compensated for their time with a 10-dollar gift card from one of three vendors of their choice. All interviews were audio-recorded, transcribed verbatim, de-identified and anonymized.

Data analysis

The interview transcripts were analyzed thematically by two members of the research team (IK, HDS) following the six steps outlined by Braun and Clarke [39]. First, IK and HDS read all transcripts independently to become familiar with the data. Second, after this initial reading, IK and HDS generated the first codes together. Third, IK and HDS each re-read and coded two transcripts using this initial codebook. Within this stage, IK and HDS searched for repeated patterns of meaning to continue developing initial themes. Fourth, IK and HDS reviewed each other’s coded transcripts and refined the codebook. Finally, IK and HDS defined and named themes, and synthesized the qualitative findings. The qualitative analysis was iterative in nature, whereby HDS and IK met frequently to assess and resolve any interpretation discrepancies and to refine the coding structure and themes. Both HDS and IK have extensive experience in qualitative research methods and have backgrounds in nursing, mental health, and health informatics.

Results

RQ1A: what are the characteristics of participants who enrolled in the service?

In total, 710 unique participants enrolled in the SaskWell service with increasing enrollment in subsequent iterations as community engagement and recruitment across multiple platforms and through media sources continued. Most participants identified as female (538/710, 75.8%), white (516/710, 72.7%), and resided in an urban region (557/710, 78.5%). Moreover, 66.7% (475/708) of participants were between 25 and 64 years in age, where the median age was 32. Finally, over half of participants received a mental health score of Moderate (370/708, 52.1%). Further details can be found in Table 1.

Table 1 Participant Characteristics (n = 710)

RQ1B: what is the average response rate for the messages sent to users?

Table 2 describes participants’ average response ratios. Throughout the duration of SaskWell, a total of 21 messages were delivered in iteration one. This first trial had a shorter period and some differing frequency of messaging as the service was stood up, and the messaging included on average 1 initial wellness tool, followed by at least 1 wellness message, and 1 polling question per week). In iterations two, three, and four 31 messages were delivered on average 1 initial wellness tool, with 2 wellness messages, and 1 polling question per week following over a span of 10 weeks. The wellness tools and messages are reported collectively as wellness resources in the table that follows. A higher response ratio indicates greater engagement with SaskWell. Overall, it was found that the average response ratio across all iterations was 0.43.

Table 2 User Text Message Response Ratio

RQ2A: what demographic factors are associated with an ‘engaged’ user?

In univariate models, gender (p < 0.001), age (p < 0.001), and ethnicity (p = 0.02) were factors significantly associated with being an ‘engaged’ user. Specifically, SaskWell participants who identified as female (OR 2.21, 95% CI 1.46–3.32), between the ages of 25–64 (OR 1.85, 95% CI 1.29–2.66), or 65 and older (OR 3.46, 95% CI 1.50–8.51), were more likely to be engaged with the SaskWell service. Participants who identified their ethnicity as racialized (OR 0.52, 95% CI 0.33–0.80), were significantly less likely to be engaged with SaskWell. Further details are presented in Table 3.

Table 3 Demographic factors with an ‘engaged user’

RQ2B: what are user preferences for tailoring the tools they receive based on select characteristics?

Overall, user preference for tailoring the formal digital mental health tools received to their ethnicity, offered as an option to all users on enrollment, was significant among youth, adult, and senior age groups (chi-squared test p = 0.004).

RQ1C: what are users experiences with SaskWell, particularly as it relates to satisfaction and perceived benefits?

In total, 115 participants (115/710, 16.2%) completed the user experience survey. A majority of respondents (86%) stated it was very easy to sign-up for SaskWell, where no respondents required assistance in enrolling. Moreover, 42.6% of respondents found the text messages delivered through SaskWell to be helpful and supportive. Similarly, 42.6% of respondents found the resources and tips delivered through the text messages easy to implement into their daily routines. Finally, 80% of respondents stated they would recommend the SaskWell service to their friends, families and/or colleagues. Detailed findings from the user experience survey are described in Table 4.

Table 4 SaskWell User Experience Survey Responses (n = 115)

In addition to the quantitative data gathered during the delivery of this texting service, a total of 28 SaskWell users participated in a follow-up interview. These participants shared details related to a number of themes such as their motivation to sign-up for SaskWell, their perception of texting as a mechanism to deliver mental health resources, the impact of SaskWell on their mental health and well-being, and their beliefs on the future potential for text-based mental health supports. Narrative summaries and illustrative quotes related to these qualitative themes follow.

Motivation to sign-up for SaskWell

Lack of awareness of existing mental health resources

A common theme expressed by participants was a general lack of awareness of available mental health supports or resources during the pandemic. Although some participants had used digital mental health supports or accessed in-person mental health services in the past, others were not aware of local or provincial mental health resources available to them or the range of digital mental health resources. As such, they opted to sign up for SaskWell to see what was available.

“I think I was definitely interested in trying to see if there were local resources. Some were local resources, supports that I was not aware of. Though research and the university, sometimes there’s a lot of things going on, so I wanted to kind of educate myself on what some were being handed out, and how it was going to be delivered.” (P11).

“I was really hoping to see new reading materials that I hadn’t really been through before from different sources. I know the Saskatchewan Health Authority has lots of resources, but sometimes they feel outdated or less relevant. I was just hoping to see what was new in the space. And so, a lot of those links that I clicked did bring up some interesting articles that were new to me.” (P9).

COVID-19 impacting population mental health

While participants had varying interests in trying out or using SaskWell, a majority of participants had opted to sign-up for SaskWell due to the impact of the COVID-19 pandemic on their mental health. Some participants had expressed challenges they had experienced during the pandemic, especially as it related to social distancing and other public health precautionary measures enacted to prevent viral spread. Specifically, participants had expressed feelings of loneliness, isolation, and feeling distant from friends and families. Other participants spoke about feelings of burnout, exhaustion, and anxiety as a result of the pandemic measures. “.

Over the last year and a half, probably close to 2 years now, since COVID began, I would say I’ve really struggled through my mental health. I burnt-out workwise as well and came back to work probably too quickly.” (P1).

“I’ve noticed that through COVID, I’ve become rather anxious about going out into the world.” (P7).

“I had a lot of friends at the time that were having difficulties psychologically with all the restrictions and issues with COVID. There was not a lot of assistance available for mental health unless you happened to work for an employer that has a wellness program.” (P28).

Altruism and personal motivators driving adoption of SaskWell

For some participants, motivation to sign up for SaskWell was driven by their personal goals in maintaining or supporting their mental health. For instance, some participants had prior experience in accessing mental health services or supporting their mental health condition and were looking for other tools to incorporate into their practice. Other participants had a general curiosity about the SaskWell service and wanted to trial out texting as a means to support their mental health.

“I need more balance in my life and I need to focus more on wellness and taking care of myself, physically, mentally, emotionally, and all of that. So, I’m kind of always open to more tips and even just reminders, the texts were very helpful actually.” (P5).

“I’ve dealt with mental illness stuff, forever. So, I am always looking for new resources because there aren’t a lot in Saskatoon, honestly, that are free. So, yeah, that’s why I was interested. Just wanted to see what it was.” (P14).

There were a few participants who opted to try out SaskWell prior to recommending the service to their patients, employees, or friends/family. These participants exhibited altruistic views in using SaskWell, whereby they not only hoped to better their own mental health but also wanted to identify a credible resource that could be used to support the mental health of others.

“Well in my role here I am always kind of on the outlook or look out for different resources that, I mean I can use for myself but also for our employees and just other resources especially during these times that might be more accessible.” (P22).

“The reason why I subscribed to the service was because both of my children, who are both in high school, are struggling with their own mental health challenges. I thought it might be a good way to have conversations with them, to connect to materials that we might not have thought of yet.” (P18).

Usability of texting as a mode of mental health resource delivery

Texting as an easy, convenient, and low commitment intervention

Commonly echoed throughout the participant interviews, was the usability of text messaging. Texting was often described as an intuitive, convenient, and easy to use technological medium. Participants spoke about how routine texting is in today’s technology landscape, where smartphones and cellular devices are increasingly ubiquitous. Text messaging was also described as being a daily communication tool for participants; and as such, receiving messages from SaskWell often did not invoke the same sense of ‘alert fatigue’ created by other mobile apps.

“I like how it’s over text, you can kind of look at it whenever you want, you can go back to it so you don’t need to download any special app or anything, so quite accessible.” (P5).

Moreover, participants frequently described receiving the mental health messages and resources through text message as a subtle push or reminder. For example, some participants professed they would receive a message, but not engage with the message immediately. Rather, in some cases, they would return to the text message and associated content later as an ongoing reminder to reflect on their mental health and well-being. This repeated description of message use provided additional insight into meaningful participant engagement with this content outside of response or usage data.

“SaskWell becomes part of the routine and part of the background of what you do. So, to me it was particularly nice and not being sort of a unique and special effort. It was a push, it was really easy to pick up on the way by, and it was just simple.” (P18).

“The other thing that I did find is that there were times when I was busy and I didn’t answer them, but I could answer them later. So, the fact that they were still there for me to look at was beneficial.” (P27).

Promoting digital equity and minimizing the divide

The use of text messaging was also described as a way to bridge known digital divides that were exacerbated during much virtual service delivery during the pandemic. A few participants spoke about the usability of text messages for a variety of populations such as those living in rural and remote communities and those with poor technology literacy or access. SaskWell, specifically, was described as a low barrier and anonymous method to support one’s mental health. For example, one participant described how an anonymous text-based service, like SaskWell, can serve as just such intervention for someone who is new to navigating the mental healthcare space. Furthermore, the service being free of cost positively influenced participants’ willingness to adopt and engage with SaskWell.

“I’m just thinking of my own personal networks, like, the people who text versus some of the other mechanisms, that span kind of all age groups. I’ve got a nephew who’s eight, who texts me. I’ve got an elder in the community, a knowledge keeper in Saskatoon, who texts me, like, eighty-five-plus. And it’s [texting] across socioeconomic, ethnic, age – like, in terms of – how ingrained it is in our society right now.” (P1).

“I think it’s [texting] probably the most affordable way, not the most affordable way, but it’s very – it opens up the door to just about anyone who has a phone no matter what their social economic status is. Because you don’t need the internet, you just need your basic cellular connection.” (P15).

Impact of SaskWell mental health and well-being

Invoking a sense of connection

Interview participants reported a sense of connection when receiving regular text messages SaskWell. The messaging content, style, and tone, in some cases, made users feel as if someone was checking in on them. A few participants brough forth the notion of SaskWell being a friendly connection, especially during times of isolation, uncertainty, and stress.

“Yes, I think it was how it’s delivered, right. It was simple, whether I had time to dive into it or not. And so, it was simple and friendly, which is great in these times, right. Everything just seems so thick and boggy, you know, and then you get this friendly, hey, that’s my wellness message.” (P11).

SaskWell also seemed to invoke a sense of togetherness. Knowing that this service is available and used by others helped some users feel less alone in how they were feeling during the pandemic. The service helped to remind participants that there were other Saskatchewan residents grappling with the negative effects of the COVID-19 pandemic.

“I don’t know how to explain it really, but I didn’t feel I was alone because I felt like there’s this service for a reason; I’m not the only one doing this. So that stood out for me and made me kind of feel a little bit more comfortable.” (P16).

Relevancy of content delivered through SaskWell

Overall, participants found the content and information delivered within SaskWell to be helpful and useful. Some wellness resources, such as grounding exercises, were described as easy to integrate and sustain in one’s daily routine. Other wellness tips, while already known by participants, were described as a welcomed reminder or suggestion.

“I felt they were really very nice suggestions that may seem obvious to us, but in times of stress, those are the things we lose first. Like, even taking a walk, or looking outside right now, just taking that time. It was nice to have those reminders.” (P8).

For some participants, there were a few wellness resources that they did not find particularly relevant or helpful for themselves. In these instances, participants reported they would either ignore the message, or sometimes pass along the resources to a friend or loved one. Participants recognized that not all messages and resources sent through SaskWell would be relevant to them, acknowledging that strategies to support positive mental wellness are rarely a one-size fits all approach.

“I think that it’s been useful to give basic pieces of information that you could then delve further into. So, I think that that’s useful. And it also gives you such a variety of topics, that even if one’s not applicable to you the next one probably is.” (P12).

Future potential of texting as a feasible way to support population mental health

User suggestions for improving SaskWell

Several suggestions were voiced on ways to improve SaskWell. As many users were motivated to help their loved ones, participants suggested a built-in option to save and send resources directly to their friends or families within the texting program. If users did not save the resource at the time of receiving the message, users tended to forget or they had to scroll up in the chat to find the resource.

“But having that option of like, “Would you like to send this to someone else”, yeah, I would’ve used that on a few occasions. So having a place that I could store things to come back to them. And I do admit that once or twice I actually accidentally deleted the SaskWell texts that I had received and so there were some links on there I hadn’t looked at yet, that I never got to because I accidentally deleted them.” (P28).

“I find a lot of the time with the messages; things get lost because either I’m not able to deal with it in the moment and then I’m not getting reminded or I can’t remember what it was so I can’t figure out how to get back there or it’s gotten lost in my bookmarks with somewhere. So, if there was a place that I could clearly identify and didn’t require a password, that I could just go and be like, “OK, here’s my pages you sent me recently,“ that would be a great.” (P17).

Participants also suggested the service could be improved by connecting users to crisis supports more proactively. Although participants were aware that SaskWell was not a crisis line or a replacement for formal mental health care, they suggested embedding crisis supports directly into the service as a way to prevent adverse events.

“The one thing that I pondered openly was, there was one text that came through and it was catching some of my triggers in the process and I wonder if there isn’t an opportunity for SaskWell to have almost that panic button process to dial in to the provincial mental health system.” (P18).

Future potential for texting messaging supports

The personalization, customization, and utility of SaskWell positively influenced participants’ desire to have the service continue beyond the COVID-19 pandemic. Participants valued the personalized and customized nature of SaskWell, primarily as it related to the messaging content and tailored digital mental health tools delivered.

“I think it was also really helpful to have a Saskatchewan-based tool, right. Reopening of the province would be specific to Saskatchewan residents, and so to have something that’s responsive to our own local context, even though it’s part of a global context, I think was really relevant too.” (P1).

SaskWell users suggested ways to enhance the customization of the service to meet diverse mental health needs. Knowing that these needs differ from one individual to the next, participants voiced building more complexity into the service by offering a variety of resources based on where an individual is at in their mental health journey.

“I don’t know if there would be like an option saying like what kind of things would you be interested in and then just give like a few options and then a person can select like A, B or C and then it can kind of gravitate towards those interests, if that makes sense? Then it kind of does a similar thing with asking and giving tips provided on that topic”. (P2).

Integration of quantitative and qualitative findings

These qualitative findings confirmed and expanded our understanding of the quantitative results. For example, in the quantitative phase of this study, more than half of the users indicated that weekly wellness texts were either helpful or very helpful, along with 40% of users who indicated somewhat helpful. In the qualitative exploration, this is further explained by additional detail regarding the personal motivations of users. Some individuals signed up for the program with altruistic views in using SaskWell, hoping to identify a credible mental health resource that could be used to support others. Therefore, not all resources were as immediately relevant to these users as revealed in the interview findings. Additionally, some users reported they did not mind receiving what they may have considered irrelevant resources to them individually, recognizing that mental health resourcing is not a one-size-fits-all approach, and/or choosing to pass those messages along to others.

In the quantitative survey results, almost all participants found SaskWell to be easy or very easy to use. No participants reported needing assistance when joining the program, and 95% of users found the frequency of text messages suitable. These results were confirmed by the qualitative findings that users found texting not only to be convenient and easy, but also feasible, with respect to delivering mental health information and resources. The user interviews provided additional insights into data, such as user response rate from the quantitative data collection and analysis. While the quantitative results revealed active user engagement in the service was low, this metric (or the way we quantified engagement) can be contextualized through interviews that highlighted specific user engagement behaviour with SaskWell. For instance, some participants described simply reading the text messages as opposed to interacting or clicking into the text message resource. While it was clear that users enjoyed receiving text messages and were happy with the frequency of messages, comparing user satisfaction to user engagement metrics may not be appropriate to understand user adoption.

Discussion

Principal findings

This study advanced a co-designed text-messaging service to provide residents of Saskatchewan a critical connection to digital mental health and wellness resources during the height of the COVID-19 pandemic. Using the RE-AIM framework as an implementation guide, four distinct cycles of SaskWell were delivered with modifications to the service in each subsequent cycle based on user engagement, feedback, and the direction of a community advisory group. SaskWell users were predominantly white (75.8%) and identified as women (75.8%), very similar to another recent Canadian mental health texting program [24]. The most common age of SaskWell users was 25–64 (66.9%) with youth users 16–25 (29.0%) the next largest participant group. Only 3.8% of users were 65 years or older, a result worth additional exploration and consideration in future text-based interventions in order to address potential digital divide barriers for this group. Most users also resided in urban settings (78.5%) another area for future consideration in terms of digital inclusion. Lastly, the results from a service enrollment questionnaire found the most common self-reported mental health rating among participants to be Moderate (52.1%).

The two primary objectives of the study were to assess and evaluate the adoption of SaskWell by focusing on user acceptance, satisfaction, and perceived benefit, and to identify factors which contributed to user engagement with the SaskWell text-based service. Both quantitative and qualitative data contributed to the final study results.

The quantitative data provided descriptive insights from user enrollment and experience surveys about the participants who engaged with the service as well as the perceived usability and usefulness. The benefits of a collaborative approach to the user interface and ongoing co-design efforts around customization and messaging content of the service are reflected in the survey results. The analysis of the back-end user data supported an exploration of engagement, including reflection on how this is captured and reported in texting interventions. In this study, a dichotomous approach established more and less engaged categories of participants with demographic factors revealing females across a wide age range to be the most engaged and those who identified their ethnicity as racialized less. These results identify key areas for ongoing study and targeted participant collaborations. Even within programs using a population wide approach, additional attention to the equitable engagement of participants across all gender and ethnicity groups will be critical in order to truly achieve the successful uptake and sustained use of these types of digital solutions. These factors should also continue to be part of the broader discussion of user engagement, how that is captured in these types of interventions and whether or not usage analytics alone address the nuances of this concept. Certainly, in this study the qualitative and additional survey data provided additional insights into participant experiences and those elements that may also have influenced engagement and other metrics.

The user experience survey demonstrated the ease with which participants were able to navigate the SaskWell sign-up process, and of these respondents none reported needing assistance with enrollment. An overwhelming majority of these users were happy with the frequency of the text messages that they received during the week (83.3%) and most found the weekly wellness messages to be helpful and supportive (56.5%). This is similar to data from Shalaby et al. [24] who ran a Text4Support program with recently discharged patients from acute mental health care in Alberta. Many of the SaskWell participants who completed the user experience survey also indicated they would recommend the service to a friend, family member, or colleague (80.0%). While many studies on the use of texting to support health choices or behaviours report strong user satisfaction, a scoping review by MacDougall et al. [40] on the use of text message interventions for adolescent mental health concluded more intensive and rigourous study designs are required in order to truly determine which features and approaches are the most successful, and more importantly how satisfaction is related to or influences engagement.

From the qualitative data this substantial referral intent was contextualized by several participants who indicated they had partially been motivated to sign-up for SaskWell as the means to assess the service and support its use among family members or through occupational roles which included providing health supports for staff or students. The interviews also revealed the ability of SaskWell to not only provide information and direction to digital mental health resources, but to provide a needed sense of connectivity to others or the wider community. These findings are echoed in a large Australian study that provided wellness supports to breast cancer survivors during the pandemic [41]. These researchers reported the importance of this connectivity first among a list of identified engagement factors which included: “(1) feeling supported and less alone, (2) motivation and reassurance for health self-management, (3) the variety of information, (4) weblinks to information and resources, and (5) the option to save the SMS text messages” [41]. SaskWell interview participants highlighted many of these same benefits as part of the qualitative themes reported in this study.

The use of text messaging as a means to influence health behaviour through both education and ongoing prompts or support is increasingly being explored [42, 43]. In a community based RCT, Runsen et al. [42] found that text messaging can surpass traditional health education information delivery methods such as pamphlets or other teaching methods. In this study, in the user interviews in particular, the effectiveness of SaskWell in creating awareness and use of new digital mental health tools was shared. As in similar studies the curation of text messages on the users’ phones was seen as a very low impact way to maintain access to preferred links or message content. Participants did suggest that being able to save or interact further with content on a personalized web location, for example, could further enhance the utility of the service, and this desire for increased integrated customizability was also shared by participants in the work of Shalaby et al. [24, 29]. The use of text messaging in the service of health education and promotion is emerging as opportunity to support improved health literacy, although some study has found challenges persist among those with the lowest literacy levels [42].

Careful consideration must also be given to ongoing disparity in terms of access to mobile phone technology and connectivity, especially as this divide is often further compounded by differences in other social determinants of health such as income and education [44]. Access to SaskWell could be obtained through a text message, online website, or a toll-free telephone number that was included in all study promotion. Although the phone number access was not heavily used, its purpose was to provide a known point of contact that could potentially reach across digital divide or other disparities. This is also why the study enrollment featured several questions about device ownership, use, data plans, and the reliability of cellular connectivity. If challenges were identified in these areas, less technology heavy resources were delivered through the texting service. Significant efforts were also made to engage residents from rural or northern Saskatchewan locations. As noted, the majority of SaskWell users reported urban residence, with only 21.6% of participants reporting rural (19.3%) or northern (2.3%) residence upon enrollment. In their review on the suitability of delivering psychological therapeutic services to rural and remote communities Dwyergra et al. [45] noted there is an urgent need for additional research into how this technology can reach and support clients in rural locations, but also across other underserved populations. This is especially needed they noted as “converging evidence shows that text-based counseling services and interventions are effective in treating a variety of mental health conditions” [45].

Strengths and limitations

The refinement of the SaskWell service through four distinct cycles, with directive input from the community advisory panel as well as research team members, including our technology delivery partner is a strength of this study. The use of the RE-AIM was also an effective foundational approach to the development, delivery, and evaluation of the service, and is seen in other similar studies on the use of text messaging for health support. The value of the community advisory group was seen not only in the ongoing co-design of the features of the texting service, but in its content. Several users spoke to the sense of connectivity or friendliness of the SaskWell, and this was likely influenced by the ongoing participation of our community members in designing and advising the research team on the tone and content of the weekly wellness messaging.

The study did face ongoing recruitment challenges, especially as the pandemic intensified. Certainly, future research in this area would benefit from a larger sample size. There were some good lessons learned about community engagement with such a texting service during this study. These included the benefits of partnering with a large community hub, such as a local university, the use of multiple social media channels, which a majority of users reported learning about the service from in the experience survey and creating community connections with local libraries and other locations to reach messaging across a wider geographic area.

There may have been selection bias at play, not only in who elected to enroll in SaskWell, but from those participants who elected to participate in further survey or interview opportunities. Despite efforts to engage diverse populations, similarly to several other texting studies, ultimately the demographics of users of this service were quite homogenous.

Conclusion

This mixed methods exploration of a population texting service to improve awareness of and use of digital mental health and wellness resources revealed promising opportunities for the uptake of this information via text messaging. While the quantitative data revealed mixed engagement, user interviews revealed core text message content was being read even when links were not being clicked. This information is useful in advancing future message construct and in ongoing reflections of how user engagement should be evaluated in such study.

Both the user engagement survey and the qualitative data supported the worth of ongoing efforts to refine and extend the use of text messaging as a means to engage citizens around the awareness and use of digital mental health and wellness resources. As the pandemic has receded into the background for many, for healthcare providers, and others who continue to be impacted more heavily by the persistent challenges of this event, this type of service may be especially timely. There is also uncertainty about how much disruption the pandemic will create as its evolution continues, and the mental health of the global citizenry continues to face additional challenges beyond this unpredictability with increasing living costs and multiple impacts of climate change as just two examples. Lastly, as the use of artificial intelligence continues to be advanced, especially in terms of abilities around linguistic analytics and prediction, the expanded use of text messaging in mental health support could be a valuable avenue for risk identification, ultimately supporting the prioritization of resource allocation where demand continues to outpace access.