Supporting mental health of medical students: Needs and demands concerning an e-mental health application within medical education

Catharina Grüneberg ¹
Alexander Bäuerle ^1,2
Sophia Karanukaran ¹
Dogus Darici ³
Christoph Jansen ^1,2
Nora Dörrie ^1,2
Sven Benson ⁴
Martin Teufel ^1,2
 Anita Robitzsch <sup>1,2

1</sup> University of Duisburg-Essen, LVR-University Hospital, Clinic for Psychosomatic Medicine and Psychotherapy, Essen, Germany
² University of Duisburg-Essen, Center for Translational Neuro- and Behavioral Sciences (C-TNBS), Essen, Germany
³ University of Münster, Institute of Anatomy and Neurobiology, Münster, Germany
⁴ University of Duisburg-Essen, University Hospital Essen, Institute for Medical Education, Essen, Germany

Abstract

Background: Digital technologies are increasingly integrated into medical education and healthcare services. Studies have demonstrated the efficacy of e-mental health approaches, offering cost-effective, user-friendly, anonymous, and geographically flexible support.

Objective: This study aims to investigate the needs and demands of medical students regarding the design of an e-mental health application within the context of medical education to foster stress management and personal skills.

Methods: A cross-sectional study was conducted from November 2022 to July 2023 at the University of Duisburg-Essen, Germany. N=229 students were incorporated in the final data evaluation. The survey comprised standardized and established self-generated inquiries. Needs and demands were analyzed descriptively. A cluster analysis was conducted to explore hypothetical subgroups. Differences and similarities of the clusters were compared.

Results: Most students expressed a preference for smartphone or tablet accessibility, with sessions lasting between 10 to 20 minutes during stressful situations and on an as-needed basis. Participants indicated a desire for information and practical exercises related to self- and time-management, self-esteem and confidence, coping with helplessness, learning methodologies, and self-care/resilience. Video, downloadable, and audio content along with access to expert guidance, were deemed valuable. K-medoids clustering revealed a low and high burden cluster.

Conclusions: By aligning with specific needs and demands of target populations, e-mental health apps with enhanced usability and a more user-focused approach can be developed to establish a blueprint for an e-mental health app tailored to their requirements.

---

---

1. Introduction

1.1. Background

Digital technologies are increasingly integrated into medical education and healthcare services, offering cost-effective, user-friendly, anonymous, and geographically flexible support [1], [2], [3]. Participatory designs prioritizing the future users’ needs and demands during the development of digital health apps are paramount for optimizing usability and acceptance [4], [5].

The necessity for preventive initiatives targeting university students, with a specific emphasis on stress management, was highlighted in a 2023 health report from a German health insurance provider describing that two-thirds of German students are burdened by stress-related health issues [6]. Medical students are susceptible to heightened stress levels and associated conditions, including anxiety, depression, burnout, and suicidal ideation [7], [8], [9], [10], [11], [12]. The challenge of fostering self-care practices and mental health awareness due to low rates of help-seeking behavior and barriers such as mental health stigma among medical students and physicians is well-documented [13], [14], [15], [16], [17], [18], [19], [20], [21]. Consequently, supportive programs that enhance mental well-being among medical students are needed [22].

Participatory design approaches present promising avenues for developing accessible solutions tailored to the preferences of the target demographic, with the potential to mitigate barriers and enhance user satisfaction [4], [23], [24]. By focusing on a user-centered design approach, the quality and usage of e-health apps (electronic health applications) may be improved, and high-drop-out rates reduced [25], [26]. The imperative for implementing interventions targeting university students, particularly medical students, is evident [16], [27], [28]. A study in 2021 described positive views among medical students regarding evidence-based Internet- and mobile-based interventions (IMIs) and indicated IMIs as promising tools for stress prevention [29]. Thus, there is still a lack of systematical research focusing on the future user’s needs and demands. More evidence is needed to develop and implement tailored e-mental health approaches (electronic mental health) [30], [31] and investigate the impact and effectiveness of participatory research [32] within medical students and education [33].

This study wants to transmit results of patient-centered studies into medical education research to fill the gap of research focusing on medical students’ preferences within the development of tailored e-health approaches.

1.2. Objectives

The objective of this study was to investigate the needs and demands for a tailored e-mental health app among medical students to foster stress management and promotion of personal skills within medical education. Previous research focusing on the acceptance and predictors of acceptance among medical students regarding the implementation and potential use of tailored e-mental health apps [34] laid the foundation of this secondary analysis and extension. A cluster analysis was conducted to investigate the presence of subgroups to get detailed insights regarding individual needs [35], [36], [37].

Key question addressed:

• What are the needs and demands of medical students regarding the design, content and integration of an e-mental health app within medical education in relation to possible cluster regarding mental health condition?

2. Methods

2.1. Study design and participants

A cross-sectional study was conducted at University of Duisburg-Essen, North-Rhine-Westphalia, Germany from November 2022 to July 2023. N=305 (100%) students attending the course of psychosomatic medicine within the 5^th clinical semester were invited to participate. All students were aged 18 or above. Of N=245 (80.3%) students participating, N=16 (6.5%) individuals could not be included in the final data analysis due to missing content. In total, N=229 (93.5%) students were included in the final data analysis.

2.2. Ethical considerations and reporting guideline

The survey was conducted according to the Declaration of Helsinki and approved by the Ethics Committee of the Medical Faculty of the University of Duisburg-Essen (21-10196-BO). The results were reported in compliance with the Strengthening the Reporting of Observational Studies in Epidemiology guidelines [38]. For participation no financial or any other compensation was offered. The survey was anonymous and voluntary. Written informed consent was obtained.

2.3. Assessment instruments

The cross-sectional study encompassed a paper-pencil assessment including established self-designed questionnaires to assess sociodemographic data, use of digital technologies, prior experience with and attitudes towards e-health apps, needs and demands regarding the development of a tailored e-mental health app. Mental health data such as symptoms of anxiety and depression were assessed using validated measures, PHQ-2 (Patient Health Questionnaire-2) and GAD-2 (Generalized Anxiety Disorder Scale-2).

2.3.1. Sociodemographic data

Firstly, sociodemographic data regarding gender, age and marital status were assessed.

2.3.2. Mental health data

The Patient Health Questionnaire-4 (PHQ-4), a validated and well-established four-item instrument was applied to measure the medical students’ anxiety and depression level [39]. Answers were given on a four-point Likert-Scale from 0 (“never”) to 3 (“nearly every day”). The PHQ-4 was divided into the two 2-item measures: PHQ-2 and GAD-2. The PHQ-2 screens for symptoms of depression and the GAD-2 screens for symptoms of general anxiety over a period of two weeks. Both set a cutoff score of ≥3 as an indicator for depression/anxiety [40], [41]. The distress among medical students was analyzed on a scale from 0 (“no distress”) to 10 (“extreme distress”) to cover the students’ distress in the past week. Additionally, self-generated questions contained information on prior contact to internal or external psychosocial support system.

2.3.3. Needs and demands

Needs and demands were assessed by tailored items with different answer options (dichotomous, single- or multiple-choice, and Likert-Scales) and were designed and adapted by the research group based on prior studies [35], [36], [42], [43], [44]. The suitability of several formats were measured on a five-point Likert scale (1=“very unsuitable” to 5=“very suitable”). Main characteristics regarding the content and construction were analyzed. The students rated the importance of nine predetermined subjects on a five-point Likert scale (1=“unimportant” to 5=“very important”). Suitable contents and subjects were identified based on current literature and through expert interviews from the clinic of psychosomatic medicine and working-group of digital health at the LVR-University Hospital in Essen [42], [45], [46]. Further topics of interest or notifications could be provided using open-ended questions.

2.3.4. Prior experience and usage of digital media

Prior experience with online-based programs for health promotion and stress management, was assessed using adapted five-point Likert scales, along with information on the average usage of digital media. The Likert scales ranged from 1 (“do not agree”) to 5 (“totally agree”) and were developed based on current literature [42], [43].

2.3.5. Statistical analysis

Statistical analysis was performed using SPSS Statistic version 26 (IBM, New York, NY, USA), RStudio version 4.0.2 (RStudio PBC, Boston, MA, USA) and Microsoft Excel version 16.86 (Microsoft). Prior to any statistical test, relevant assumptions and prerequisites were tested. The level of significance was set at alpha=0.05. P-values were adjusted using Bonferroni correction for multiple tests. Descriptive statistics were performed. Cut-off scores for PHQ-2 and GAD-2 were computed. Mental health data (PHQ-2, GAD-2, distress score) was used to perform a cluster analysis to investigate hypothetical subgroups within the sample. K-medoids method, a conservative measure, was chosen because of its robustness against outliers. Hopkins’ H statistic was used to assess the clustering tendency of the data and the suitability for clustering. The parameter package was utilized to assess the optimal number of clusters [47]. A two-cluster analysis was conducted. The cohort was divided into a low- and high-burden cluster. The model’s overall performance was assessed using R² statistics and the classifiably was analyzed by linear discrimination analysis. Wilcoxon rank sum test was utilized to perform group comparisons of the clusters. Differences and similarities regarding the needs and demands of those affected by mental loads were analyzed descriptively.

3. Results

3.1. Study population

N=229 medical students could be included in the final analysis with a mean age of 25.05 years (SD=2.83; MIN=20, MAX=37 years). The majority was female (68.6%, n=157). 39% were living in a relationship or married.

3.2. Mental health data

12.2% of the medical students reported elevated depression levels with PHQ-2 sum scores ≥3 and 22.3% elevated anxiety levels with GAD-2 sum scores ≥3. A detailed description regarding the mental health data is represented in table 1 [Tab. 1].

60.70% (N=139) acknowledged awareness of the support services for help-seeking to foster mental health available at the medical faculty, yet had not availed themselves of these resources. 13.97% (N=32) reported previous engagement with psychological consultation services. 9.61% (N=22) of respondents indicated a lack of prior knowledge regarding the availability of assistance at the medical faculty but expressed interest in utilizing such services. In contrast to that, 12.22% (N=28) of participants displayed disinterest in seeking help at the medical faculty and were unaware of the support options available. A minority, comprising 3.06% (N=7) of respondents, expressed a preference for seeking external assistance.

3.3. Prior experience and usage of digital media

Medical students reported spending an average of 1.42 hours per day (SD=1.82) utilizing digital media in occupational contexts outside of medical school, 3.81 hours per day (SD=2.06) for university-related purposes, and 3.23 hours per day (SD=1.76) for personal activities. Regarding e-health utilization and prior experiences with health-related information, 84.72% (N=194) of the students indicated frequent engagement with digital health solutions. Specifically, 40.35% (N=92) reported frequent use for physical activity and fitness, 26.64% (N=66) for self- and time-management, 8.73% (N=20) for stress-management, and 8.30% (N=19) for sleep tracking.

3.4. Needs and demands

Medical students rated various needs and demands regarding the implementation and design of an e-health app.

3.4.1. Availability and (un)suitable formats

The vast majority, 97.3% (N=222) of the students rated availability via smartphone (M=4.82, SD=0.52) and 91.2% (N=209) via tablet (M=4.55, SD=0.72) as most suitable. Video material (85.59%), download material (72.37%), audio material such as podcasts (70.31%) and contact to experts (70.31%) were considered suitable or very suitable for promotion of e-mental health within medical education by most of the students. Chatbot (36.40%) and games (31.44%) were seen as less suitable to unsuitable. A detailed overview of the response frequencies are represented in table 2 [Tab. 2].

3.4.2. Usage, usability and construction

Most of the students preferred using the app throughout their whole studies especially as required or on demand, e.g. in incriminating situations and stressful times. New content should be available one to three times per week (51.98%, N=118). Most students favored usability on demand and as required (79.91%, N=183) followed by a guided modular design (43.23%, N=99). Each session should be around 10 to 20 minutes (40.17%, N=92). A detailed overview is represented in table 3 [Tab. 3].

3.4.3. Content and subjects

The following issues were rated as most important and should be addressed: self-/time-management (66.81%) as well as self-esteem/-confidence and dealing with helplessness (63.16%). A detailed overview of responses is presented in figure 1 [Fig. 1].

3.4.4. Cluster Analysis

To conduct a cluster analysis mental health data (i.e., PHQ-2, GAD-2, Distress score) were chosen for clustering. Using the Hopkins’ H-statistic, the data showed a clustering tendency (Hopkins’ H=0.41). The choice of two clusters was supported by 12 (41.38%) methods out of 29. The k-medoids method was used for performing a two-cluster analysis. The two clusters were interpreted as low burden cluster (n=111) and high burden cluster (n=118). The model performance was R²=0.342. The overall accuracy of classification was 93.01%.

The cluster analysis is visualized in figure 2 [Fig. 2].

Significant differences between the low burden cluster and the high burden cluster were identified. Among the high burden cluster, the mean PHQ-2, GAD-2, and Distress sum scores were significantly higher than in the low burden cluster (p_adj<.001). An overview of the group characteristics is represented in table 4 [Tab. 4].

3.4.5. Comparison of the clusters

Characteristics of the low and high burden cluster were compared descriptively with focus on preferred devices, formats and content. 50% of the students within the high burden cluster rated interaction with other students as rather suitable to very suitable compared to 44% of the students within the low burden cluster. Within the high burden cluster learning methods, risk assessment/analysis of stressors, self-care and resilience as well as relaxation/attentiveness/meditation/yoga were rated as most important. Students within the low burden group valued self-care and resilience, learning methods and physical activity and fitness as highly important. An overview of the preferred devices and formats is represented in figure 3 [Fig. 3]. The importance of contents between both clusters is compared in figure 4 [Fig. 4].

Students within the high burden cluster preferred a duration of 1 to 10 minutes of each session, compared to the low burden cluster with a preference of 10 to 20 minutes. The favored usability and construction are shown in table 3 [Tab. 3].

4. Discussion

The primary analysis of the current data found that most of the students reported high acceptance of a tailored e-mental health app within medical education [34]. Accordingly, this study examined the needs and demands regarding content and design of an e-mental health app among medical students in consideration of possible clusters regarding mental health condition.

Elevated levels of anxiety and depression are well-known among medical students. Our findings underline the importance of tailored mental health support due to elevated anxiety, depression and distress levels. Even though, nearly 10% of the students reported to be interested in seeking help at our medical faculty, 15% of the students would not seek help at their own faculty. These findings align with previous research [48] and should be considered within the design of additional supportive programs. A low-threshold possibility can be a valuable tool to support and extend in-person consultation on-campus to dismantle barriers and optimize help-seeking among medical students.

Relating to the needs and demands, smartphones or tablets were mostly preferred devices aligning with previous patient-centered research [35], [49]. Within our study, most of the participants were comparatively young. Digital media was frequently used for university and personal activities. Due to this, it might be conceivable that the students assign a high value to apps on smartphones and tablets offering optimal accessibility and low-threshold opportunities [29]. Most of the students preferred using the app throughout their whole study, especially as required or on demand, e.g. in incriminating situations and stressful times. The students favored weekly updates with new content one to three times a week. One session should take 10 to 20 minutes. This might indicate the wish to implement the app continuously within their routines. A modular design and guidance was preferred by nearly four out of ten whereas most students wanted to use the content on demand. This wish for self-guided usage might underline a pursuit of autonomy indicating that the structure and consent should be presented succinctly. To optimize usability and suitability, the mostly preferred formats (video and audio materials) need to be developed accordingly. Additionally, contact to experts was considered suitable for promoting e-mental health and should be offered correspondingly. Various topics were rated as most important by the students and should be addressed such as self-/time-management, self-esteem/-confidence and dealing with helplessness, learning methodologies and self-care/resilience. Due to individual characteristics of the target groups different topics need to be considered and investigated when developing digital apps.

Explorative analysis revealed two clusters, a low and a high burden cluster. Within the high burden cluster distress, anxiety and depression scores were significantly higher compared to the low burden cluster. Formats such as video material, downloadable material, contact to experts, interactive exercises and audio material were valued as useful especially within the high burden cluster. Connectivity and networking with other students were highly valued by students with more distress or anxiety aligning with the study by Kappner et al in 2022 [50]. These findings underline the importance of social support and should be investigated further. Comparing both clusters regarding content and subjects, the high burden cluster valued learning methods, risk assessment/analysis of stressors, self-care and resilience as most important in contrast to students within the low burden cluster who esteemed self-care and resilience, learning methods as well as physical activity/fitness. To encounter individual characteristics qualitative interviews or assessments designed as educative workshops could be considered for future studies. A study by Dederichs et al described educative workshops as suitable tools within medical education research on participatory design approaches [51].

The findings of the study establish an introductory framework for further studies and the design of e-mental health apps within medical education. Future research should focus on the impact of user-centered approaches within the design of e-mental health interventions among target groups [27], [52], [53] to supplement and extend existing support, e.g. on-campus consultation, and focus on individual needs. Even though tailored e-mental health apps represent promising approaches, the imperative of the learning environment and adaptation of the requirements within medical education should be focused.

4.1. Limitations

When interpreting our analysis, the following limitations should be considered. All data was collected within medical students attending psychosomatic seminars during the 5^th clinical semester, selection bias needs to be considered. The needs and demands at other medical faculties as well as within other semesters should be investigated further. This data based on self-reporting. The development of the survey based on the expertise of our working unit and from patient-centered studies. The needs and characteristics of male and divers/non-binary are highly essential and should be addressed within future studies. Despite the limitations and biases of our study the realization as paper-pencil-survey needs to be noted. Despite biases may limit generalizability and representativeness, our study provides a practical and easy-to-use approach to evaluate the needs and characteristics of medical students within the development and designing process of a tailored e-mental health app.

4.2. Implications

Medical students’ (in general: the future user’s) preferences should be linked to actual usage and adherence to mobile apps in future research and need to be explored further. On basis of our finding, a tailored e-mental health app for medical students can be developed and implemented. Additionally, designers, and other stakeholders should be directly involved within the designing process. More research is needed to investigate the effectiveness of tailored design approaches.

5. Conclusions

By developing e-mental health apps through a more user-centered design approach, highly beneficial approaches can be designed with optimized usability to support medical students’ mental health. This information may give important and useful advice to create a tailored app and should be considered within future research as well as directly within the designing process to optimize usability, adherence and help-seeking behavior. Our results will be used to design, develop, implement and evaluate an e-mental health app within medical education.

Abbreviations

• e-mental health: electronic mental health
• GAD-2: Generalized Anxiety Disorder Scale-2
• PHQ-2: Patient Health Questionnaire-2
• PHQ-4: Patient Health Questionnaire-4

Notes

Data availability

The datasets used and/or analyzed during the current study are available on reasonable request. Please contact the corresponding author Dr. med. Anita Robitzsch (anita.robitzsch@lvr.de).

Funding

The study was conducted without external funding. We thank the Open Access Fund of the University of Duisburg-Essen for their support during the publication of our article.

Authors’ contributions

AR, MT, and AB initiated and conceptualized the study. Project administration was conducted by AR, AB, and SB. CG and SK performed the statistical analyses and interpretation of the data. CG wrote the original draft of the manuscript. Data acquisition was performed by AR, AB, and CG. AR, MT, AB, and SB supervised the project and contributed to the study design and data collection. SB, DD, CJ and ND gave important input regarding the critical revision of the manuscript. All authors contributed to the further writing of the manuscript and approved its final version.

Authors’ ORCIDs

• Catharina Grüneberg: [0009-0002-0918-5905]
• Alexander Bäuerle: [0000-0003-1488-8592]
• Sophia Karunakaran: [0009-0000-1202-2950]
• Dogus Darici: [0000-0002-2375-8792]
• Nora Dörrie: [0000-0002-9760-9167]
• Sven Benson: [0000-0002-4487-4258]
• Martin Teufel: [0000-0003-2120-1840]
• Anita Robitzsch: [0009-0005-2941-1123]

Acknowledgements

We thank Prof. Dr. med. Joachim Fandrey, for his support and input regarding the critical revision of the manuscript. We thank Lisa Jahre and Anna-Lena Frewer for their support during the statistical analysis.

We acknowledge support by the Open Access Publication Fund of the University of Duisburg-Essen.

Competing interests

The authors declare that they have no competing interests.

---

References

[1] Ebert DD, Lehr D, Heber E, Riper H, Cuijpers P, Berking M. Internet- and mobile-based stress management for employees with adherence-focused guidance: efficacy and mechanism of change. Scand J Work Environ Health. 2016;42(5):382-394. DOI: 10.5271/sjweh.3573
[2] Hedman E, Andersson E, Ljotsson B, Andersson G, Ruck C, Lindefors N. Cost-effectiveness of Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: results from a randomized controlled trial. Behav Res Ther. 2011;49(11):729-736. DOI: 10.1016/j.brat.2011.07.009
[3] Ebert DD, Heber E, Berking M, Riper H, Cuijpers P, Funk B, Lehr D. Self-guided internet-based and mobile-based stress management for employees: results of a randomised controlled trial. Occup Environ Med. 2016;73(5):315-323. DOI: 10.1136/oemed-2015-103269
[4] Clemensen J, Rothmann MJ, Smith AC, Caffery LJ, Danbjorg DB. Participatory design methods in telemedicine research. J Telemed Telecare. 2017;23(9):780-785. DOI: 10.1177/1357633X16686747
[5] Reiners F, Sturm J, Bouw LJW, Wouters EJ. Sociodemographic Factors Influencing the Use of eHealth in People with Chronic Diseases. Int J Environ Res Public Health. 2019;16(4):645. DOI: 10.3390/ijerph16040645
[6] Techniker Krankenkasse (TK). Gesundheitsreport - Wie geht’s Deutschlands Studierenden? Hamburg: Techniker Krankenkasse; 2023. Zugänglich unter/available from: https://www.tk.de/resource/blob/2149886/e5bb2564c786aedb3979588fe64a8f39/2023-tk-gesundheitsreport-data.pdf
[7] Thapa B, Sapkota S, Khanal A, Aryal BK, Hu Y. Predictors of Depression and Anxiety among Medical Students. J Nepal Health Res Counc. 2023;21(1):63-70. DOI: 10.33314/jnhrc.v21i1.4514
[8] Wege N, Muth T, Li J, Angerer P. Mental health among currently enrolled medical students in Germany. Public Health. 2016;132:92-100. DOI: 10.1016/j.puhe.2015.12.014
[9] Nair M, Moss N, Bashir A, Garate D, Thomas D, Fu S, Phu D, Pham C. Mental health trends among medical students. Proc (Bayl Univ Med Cent). 2023;36(3):408-410. DOI: 10.1080/08998280.2023.2187207
[10] Almutairi H, Alsubaiei A, Abduljawad S, Alshatti A, Fekih-Romdhane F, Husni M, Jahrami H. Prevalence of burnout in medical students: A systematic review and meta-analysis. Int J Soc Psychiatry. 2022;68(6):1157-1170. DOI: 10.1177/00207640221106691
[11] Seo C, Di Carlo C, Dong SX, Fournier K, Haykal KA. Risk factors for suicidal ideation and suicide attempt among medical students: A meta-analysis. PLoS One. 2021;16(12):e0261785. DOI: 10.1371/journal.pone.0261785
[12] Kamski L, Frank E, Wenzel V. Suizidalität von Medizinstudierenden: Fallserie [Suicide in medical students: case series]. Anaesthesist. 2012;61(11):984-988. DOI: 10.1007/s00101-012-2094-1
[13] Kaiser H, Grice T, Walker B, Kaiser J. Barriers to help-seeking in medical students with anxiety at the University of South Carolina School of Medicine Greenville. BMC Med Educ. 2023;23(1):463. DOI: 10.1186/s12909-023-04460-5
[14] Lynch L, Long M, Moorhead A. Young Men, Help-Seeking, and Mental Health Services: Exploring Barriers and Solutions. Am J Mens Health. 2018;12(1):138-149. DOI: 10.1177/1557988315619469
[15] Winter RI, Patel R, Norman RI. A Qualitative Exploration of the Help-Seeking Behaviors of Students Who Experience Psychological Distress Around Assessment at Medical School. Acad Psychiatry. 2017;41(4):477-485. DOI: 10.1007/s40596-017-0701-9
[16] Gold JA, Johnson B, Leydon G, Rohrbaugh RM, Wilkins KM. Mental health self-care in medical students: a comprehensive look at help-seeking. Acad Psychiatry. 2015;39(1):37-46. DOI: 10.1007/s40596-014-0202-z
[17] Mitchell KM, Holtz BE, McCarroll AM. Assessing College Students' Perceptions of and Intentions to Use a Mobile App for Mental Health. Telemed J E Health. 2022;28(4):566-574. DOI: 10.1089/tmj.2021.0106
[18] Tong G. World Mental Health Day 2018, but how aware are medical students? Adv Med Educ Pract. 2019;10:217-219. DOI: 10.2147/AMEP.S190187
[19] Morgan AJ, Ross A, Reavley NJ. Systematic review and meta-analysis of Mental Health First Aid training: Effects on knowledge, stigma, and helping behaviour. PLoS One. 2018;13(5):e0197102. DOI: 10.1371/journal.pone.0197102
[20] Pinto IC, Bernardo M, Sousa S, Curral R. Evaluation of mental health stigma on medical education: an observational study with Portuguese medical students. Porto Biomed J. 2020;5(4):e074. DOI: 10.1097/j.pbj.0000000000000074
[21] Braun J, Jocham D. Arztgesundheit: Was macht uns krank – Was hält uns gesund? [Physician health: What makes us ill? What keeps us healthy?]. Aktuelle Urol. 2019;50(2):195-202. DOI: 10.1055/a-0854-5049
[22] de Sá E Camargo ML, Torres RV, Cotta KCG, Ezequiel ODS, Lucchetti G, Lucchetti ALG. Mental health throughout the medical career: A comparison of depression, anxiety, and stress levels among medical students, residents, and physicians. Int J Soc Psychiatry. 2023;69(5):1260-1267. DOI: 10.1177/00207640231157258
[23] Saparamadu AA, Fernando P, Zeng P, Teo H, Goh A, Lee JM, Lam CW. User-Centered Design Process of an mHealth App for Health Professionals: Case Study. JMIR Mhealth Uhealth. 2021;9(3):e18079. DOI: 10.2196/18079
[24] Narvaez S, Tobar AM, Lopez DM, Blobel B. Human-Centered Design of an mHealth App for the Prevention of Burnout Syndrome. Stud Health Technol Inform. 2016;228:215-219. DOI: 10.3233/978-1-61499-678-1-215
[25] van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, Seydel ER. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011;13(4):e111. DOI: 10.2196/jmir.1672
[26] Eysenbach G. The law of attrition. J Med Internet Res. 2005;7(1):e11. DOI: 10.2196/jmir.7.1.e11
[27] Apolinario-Hagen J, Harrer M, Salewski C, Lehr D, Ebert DD. Acceptance and use of e-mental health services among university students Secondary analysis of an experiment. Pravent Gesundheit. 2023;18(2):196-203. DOI: 10.1007/s11553-022-00945-1
[28] Peng P, Hao Y, Liu Y, Chen S, Wang Y, Yang Q, Wang X, Li M, Wang Y, He L, Wang Q, Ma Y, He H, Zhou Y, Wu Q, Liu T. The prevalence and risk factors of mental problems in medical students during COVID-19 pandemic: A systematic review and meta-analysis. J Affect Disord. 2023;321:167-181. DOI: 10.1016/j.jad.2022.10.040
[29] Dederichs M, Weber J, Pischke CR, Angerer P, Apolinario-Hagen J. Exploring medical students' views on digital mental health interventions: A qualitative study. Internet Interv. 2021;25:100398. DOI: 10.1016/j.invent.2021.100398
[30] Enam A, Torres-Bonilla J, Eriksson H. Evidence-Based Evaluation of eHealth Interventions: Systematic Literature Review. J Med Internet Res. 2018;20(11):e10971. DOI: 10.2196/10971
[31] Rigby M, Ammenwerth E. The Need for Evidence in Health Informatics. Stud Health Technol Inform. 2016;222:3-13. DOI: 10.3233/978-1-61499-635-4-3
[32] Orlowski SK, Lawn S, Venning A, Winsall M, Jones GM, Wyld K, Damarell RA, Antezana G, Schrader G, Smith D, Collin P, Bidargaddi N. Participatory Research as One Piece of the Puzzle: A Systematic Review of Consumer Involvement in Design of Technology-Based Youth Mental Health and Well-Being Interventions. JMIR Hum Factors. 2015;2(2):e12. DOI: 10.2196/humanfactors.4361
[33] Ungar P, Schindler AK, Polujanski S, Rotthoff T. Online programs to strengthen the mental health of medical students: A systematic review of the literature. Med Educ Online. 2022;27(1):2082909. DOI: 10.1080/10872981.2022.2082909
[34] Grüneberg C, Bäuerle A, Karunakaran S, Darici D, Dörrie N, Teufel M, Benson S, Robitzschg A. Medical Students’ Acceptance of Tailored E-Mental Health Apps to Foster Their Mental Health: A cross-sectional study. JMIR Med Educ. 2025;11:e58183. DOI: 10.2196/58183
[35] Jahre LM, Lortz J, Rassaf T, Rammos C, Mallien C, Skoda EM, Teufel M, Bäuerle A. Needs and demands for mHealth cardiac health promotion among individuals with cardiac diseases: a patient-centred design approach. Eur Heart J Digit Health. 2023;4(5):393-401. DOI: 10.1093/ehjdh/ztad038
[36] Jansen C, Bäuerle A, Jahre LM, Kocol D, Krakowczyk J, Geiger S, Dörrie N, Skoda EM, Teufel M. Needs and Demands for e-Health Interventions in Patients Attending a Psychosomatic Outpatient Clinic: A Baseline Assessment for a User-Centered Design Approach. Telemed J E Health. 2024;30(6):e1747-e1756. DOI: 10.1089/tmj.2023.0360
[37] Otto N, Böckers A, Shiozawa T, Brunk I, Schumann S, Kugelmann D, Missler M, Darici D. Profiling learning strategies of medical students: A person-centered approach. Med Educ. 2024;58(11):1304-1314. DOI: 10.1111/medu.15388
[38] von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61(4):344-349. DOI: 10.1016/j.jclinepi.2007.11.008
[39] Kroenke K, Spitzer RL, Williams JBW, Lowe B. An Ultra-Brief Screening Scale for Anxiety and Depression: The PHQ-4. Psychosomatics. 2009;50(6):613-621. DOI: 10.1176/appi.psy.50.6.613
[40] Kroenke K, Spitzer RL, Williams JB. The Patient Health Questionnaire-2 - Validity of a two-item depression screener. Med Care. 2003;41(11):1284-1292. DOI: 10.1097/01.Mlr.0000093487.78664.3c
[41] Lowe B, Wahl I, Rose M, Spitzer C, Glaesmer H, Wingenfeld K, Schneider A, Brähler E. A 4-item measure of depression and anxiety: validation and standardization of the Patient Health Questionnaire-4 (PHQ-4) in the general population. J Affect Disord. 2010;122(1-2):86-95. DOI: 10.1016/j.jad.2009.06.019
[42] Geiger S, Steinbach J, Skoda EM, Jahre L, Rentrop V, Kocol D, Jansen C, Schüren L, Niedergethmann M, Teufel M, Bäuerle A. Needs and Demands for e-Mental Health Interventions in Individuals with Overweight and Obesity: User-Centred Design Approach. Obes Facts. 2023;16(2):173-183. DOI: 10.1159/000527914
[43] Stoppok P, Frewer AL, Schweda A, Geiger S, Skoda EM, Müßgens D, Bingel U, Teufel M, Bäuerle A. Needs and Demands for eHealth Pain Management Interventions in Chronic Pain Patients. J Pers Med. 2023;13(4):675. DOI: 10.3390/jpm13040675
[44] Schröder J, Dinse H, Maria Jahre L, Skoda EM, Stettner M, Kleinschnitz C, Teufel M, Bäuerle A. Needs and Demands for e-Health Symptom Management Interventions in Patients with Post-COVID-19 Condition: A User-Centered Design Approach. Telemed J E Health. 2024;30(8):2194-2202. DOI: 10.1089/tmj.2024.0088
[45] Bäuerle A, Graf J, Jansen C, Musche V, Schweda A, Hetkamp M, Weismüller B, Dörrie N, Junne F, Teufel M, Skoda EM. E-mental health mindfulness-based and skills-based 'CoPE It' intervention to reduce psychological distress in times of COVID-19: study protocol for a bicentre longitudinal study. BMJ Open. 2020;10(8):e039646. DOI: 10.1136/bmjopen-2020-039646
[46] Bäuerle A, Jahre L, Teufel M, Jansen C, Musche V, Schweda A, Fink M, Dinse H, Weismüller B, Dörrrie N, Junne F, Graf J, Skoda EM. Evaluation of the E-Mental Health Mindfulness-Based and Skills-Based "CoPE It" Intervention to Reduce Psychological Distress in Times of COVID-19: Results of a Bicentre Longitudinal Study. Front Psychiatry. 2021;12:768132. DOI: 10.3389/fpsyt.2021.768132
[47] Lüdecke D, Ben Shachar M, Patil I, Makowski D. Extracting, Computing and Exploring the Parameters of Statistical Models using R. J Open Source Software. 2020;5:2445. DOI: 10.21105/joss.02445
[48] Berliant M, Rahman N, Mattice C, Bhatt C, Haykal KA. Barriers faced by medical students in seeking mental healthcare: A scoping review. MedEdPublish (2016). 2022;12:70. DOI: 10.12688/mep.19115.1
[49] Lortz J, Simanovski J, Kuether T, Kreitschmann-Andermahr I, Ullrich G, Steinmetz M, Rammos C, Jánosi RA, Moebus S, Rassaf T, Paldán K. Needs and Requirements in the Designing of Mobile Interventions for Patients With Peripheral Arterial Disease: Questionnaire Study. JMIR Form Res. 2020;4(8):e15669. DOI: 10.2196/15669
[50] Kappner N, Lang J, Berthold A, Gaum PM. The moderating effect of social resources on the link between study-related stressors and depressive symptoms among medical students in North Rhine-Westphalia, Germany - a cross-sectional study. BMC Psychiatry. 2022;22(1):524. DOI: 10.1186/s12888-022-04170-0
[51] Dederichs M, Nitsch FJ, Apolinario-Hagen J. Piloting an Innovative Concept of e-Mental Health and mHealth Workshops With Medical Students Using a Participatory Co-design Approach and App Prototyping: Case Study. JMIR Med Educ. 2022;8(1):e32017. DOI: 10.2196/32017
[52] Oti O, Pitt I. Online mental health interventions designed for students in higher education: A user-centered perspective. Internet Interv. 2021;26:100468. DOI: 10.1016/j.invent.2021.100468
[53] Mak WW, Ng SM, Leung FH. A Web-Based Stratified Stepped Care Platform for Mental Well-being (TourHeart+): User-Centered Research and Design. JMIR Form Res. 2023;7:e38504. DOI: 10.2196/38504