mibe000225
10.3205/mibe000225
urn:nbn:de:0183-mibe0002253
Research Article
Improving health IT monitoring via an electronic system for the exchange between science and practice
Verbesserung des IT-Monitorings im Gesundheitswesen durch ein elektronisches System zum Austausch zwischen Wissenschaft und Praxis
Weiß
Weiß
Jan-Patrick
JP
Department of Business Management and Social Sciences, Health Informatics Research Group, University of Applied Sciences Osnabrück, Germany
author
Hübner
Hübner
Ursula
U
University of Applied Sciences Osnabrück, Faculty of Business Management and Social Sciences, PO 1940, 49009 Osnabrück, Germany, Phone: +49 541–969 2012Department of Business Management and Social Sciences, Health Informatics Research Group, University of Applied Sciences Osnabrück, Germany
u.huebner@hs-osnabrueck.de
author
Rauch
Rauch
Jens
J
Department of Business Management and Social Sciences, Health Informatics Research Group, University of Applied Sciences Osnabrück, Germany
author
Thye
Thye
Johannes
J
Department of Business Management and Social Sciences, Health Informatics Research Group, University of Applied Sciences Osnabrück, Germany
author
Teuteberg
Teuteberg
Frank
F
School of Business Administration and Economics, Faculty of Corporate Accounting and Information Systems, University Osnabrück, Germany
author
Van der Linde
Van der Linde
Jan
J
Department of Engineering and Computer Science, University of Applied Sciences Osnabrück, Germany
author
Thiesing
Thiesing
Frank
F
Department of Engineering and Computer Science, University of Applied Sciences Osnabrück, Germany
author
Liebe
Liebe
Jan-David
JD
Department of Business Management and Social Sciences, Health Informatics Research Group, University of Applied Sciences Osnabrück, Germany
Institute of Medical Informatics, University for Health Sciences, Medical Informatics and Technology (UMIT), Hall in Tyrol, Austria
author
German Medical Science GMS Publishing House
Düsseldorf
610
health IT monitoring
benchmarking
health information technology
information system
knowledge translation
Gesundheits-IT-Monitoring
Benchmarking
Gesundheitsinformationstechnologie
Informationssysteme
Wissenstransfer
20210712
engl
This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License.
Dieser Artikel ist ein Open-Access-Artikel und steht unter den Lizenzbedingungen der Creative Commons Attribution 4.0 License (Namensnennung).
1860-9171
17
3
GMS Medizinische Informatik, Biometrie und Epidemiologie
GMS Med Inform Biom Epidemiol
11
Niedersächsisches Ministeriums für Wissenschaft und Kultur
Die Gesundheits-Informationstechnologie (IT) ist für Krankenhäuser ein entscheidender Faktor bei der Optimierung der Bereitstellung von Gesundheitsdienstleistungen. Viele Länder sind daran interessiert, die Mechanismen der digitalen Transformation auf nationaler Ebene zu verstehen, zu überwachen und zu benchmarken. Teil dieser Bemühungen ist die Nutzung von wissenschaftlich fundierten Reifegradmodellen, um den Digitalisierungsgrad der Gesundheitseinrichtungen zu messen. Die Ergebnisse der Reifegradmessung werden schließlich durch Konzepte des Wissenschaft-Praxis-Dialogs in die Breite getragen. Ziel dieser Studie sind die Entwicklung und Evaluation einer technischen Plattform zur Unterstützung dieses Wissenschaft-Praxis-Dialogs im Rahmen eines bundesweiten Gesundheits-IT-Monitorings in Krankenhäusern. Dabei stellen die Nutzer, die Chief Information Officer (CIO) in den Krankenhäusern, Daten zum Status der Gesundheits-IT in ihren Einrichtungen durch eine strukturierte Selbsteinschätzung auf der Plattform zur Verfügung. Die Daten werden mit wissenschaftlichen Instrumenten, das heißt (1) nach wissenschaftlich etablierten Verfahren der quantitativen Methodik und (2) im Kontext aktueller wissenschaftlicher Erkenntnisse und Fragestellungen, analysiert und verrechnet und den Anwendern in einem Dashboard individualisiert präsentiert. Die Plattform besteht aus den sechs Komponenten Datenerfassung, Indikatoren und Scores, Algorithmen, Speicherung, Datenzusammenstellungspipeline und Dashboard. Sie wurde nach den Prinzipien von Design Science entwickelt. Eine Evaluation der Plattform zeigt, dass sie Chief Information Officer bei der strategischen Steuerung ihrer IT unterstützt. Usability-Tests weisen darauf hin, dass die Plattform die Zufriedenheit bei den Nutzern steigert. In dieser Studie konnten wir zeigen, wie ein Wissenschaft-Praxis-Dialog zur Generierung praktischer Implikationen im Rahmen eines landesweiten Gesundheits-IT-Monitorings technisch umgesetzt werden kann.
Health information technology (IT) is a decisive factor for hospitals in optimizing the provision of healthcare services and many countries are interested in understanding, monitoring, and benchmarking the mechanisms of digital transformation on the national level. Part of this effort is the use of science-based maturity models to measure the level of digitization of healthcare institutions. The results of the maturity measurement are finally disseminated through science-practice dialogue concepts. The aim of this study is to develop and evaluate a technical platform to support this science-practice dialogue in the context of nationwide health IT monitoring in hospitals. Here users, the Chief Information Officer (CIO) in hospitals, provide data on the status of health IT in their institutions through a structured self-assessment on the platform. The data is analyzed and processed with scientific instruments, i.e. (1) according to scientifically established procedures of quantitative methodology and (2) in the context of current scientific findings and research questions. The individual findings are presented to the user on a dashboard. The platform comprises the six components data capturing, indicators and scores, algorithms, storage, data-compiling pipeline and dashboard. All of them were developed pursuing the design science methodology. The evaluation of the platform revealed that it supports CIOs in IT strategic management. Usability tests showed that the platform increases user satisfaction. Within this study we could show how a science-practice dialogue to generate practical implications in the context of a nationwide health IT monitoring can be technically implemented.
IntroductionHealth IT is a decisive factor for hospitals in optimizing the provision of healthcare services regarding effectiveness, quality and costs and is thus part of the discourse of strategic management . Therefore, many countries are interested in understanding, monitoring, and benchmarking the mechanisms of digital transformation on the national level to face the increasing pressures for quality and cost in health care. Prominent examples are the Health Information Technology for Economic and Clinical Health (HITECH) Act of the USA , the Digital Maturity Assessment (DMA) of England and the collaboration of the countries Denmark, Finland, Iceland, Norway, Sweden, the Faroe Islands, Greenland, and Åland in the Nordic eHealth benchmarking . With the pass of the Hospital Future Act in September 2020, hospitals in Germany will receive a total of up to EUR 4.3 billion in funding by the government for modernization . The funding program will be accompanied by the determination of the digital maturity level of the hospitals, which will be used to measure the transparency and comparability of the digitization progress.The HITECH Act can also be seen as an example of how important the system for measuring digitization can be for a funding program, because here the model initially underestimated IT adoption rates and accordingly in the end of the program overestimated the impact of the program . Furthermore, it is important that the collected data are used and presented optimally regarding the science-practice dialogue. The theoretical concept of the science-practice dialogue can be put into practice in three different ways. One approach for the science-practice dialogue is knowledge push, in which the pursuit of knowledge itself drives the scientific process, without the necessity to directly apply this new knowledge to solve problems within this process . Knowledge-pull strives to solve a practical problem. However, this approach does not necessarily result in sustainable knowledge implementation due to the different cultural background of the two realms of science and practice . The preferable science-practice dialogue to repeatedly monitoring health IT is the third approach which combines the models of knowledge-pull and push into a co-production of knowledge . Information can be co-created by transforming data to information, i.e., pooling data from practice and processing them according to generally accepted rules and with scientific instruments. This co-production cycle is finally highlighted presenting the results to the practitioners so that they can incorporate the information into their body of knowledge and can act accordingly. If their new experience and knowledge is again shared with the scientists, the loop is closed. In the context of health IT, practice is mainly represented by the CIOs (and other IT decision makers) in hospitals, who want to use the data for the purpose of information management and decision support in their own organization and by the political decision makers, who need the data for governing digitalisation in healthcare on a macrolevel. Science is represented by researchers who are interested to examine the antecedents and consequences of IT on the health care system with quantitative and longitudinal studies.While there are different concepts to encourage the science-practice dialogue , , , , , , practical cooperation often lacks an appropriate exchange system that combines valid and reliable instruments with a technical platform to allow for information storage, processing and output in a secure environment. Whereas such systems do exist for clinical research using patient data , , only a few proprietary platforms for monitoring health IT are available e.g. HIMSS platform , Meisterworks platform , however, without aiming to improve research in understanding health IT. It is therefore the overall goal of this study to go beyond a mere technical platform and to provide and evaluate a combined science-practice exchange system that supports monitoring, modelling, and predicting health IT <![CDATA[adoption</PlainText></TextGroup>, use, and maturity. Its task is to streamline the process from data gathering to information presentation and decision making based on knowledge, thereby shortening the data-to-knowledge cycle in which the knowledge is applied in practice and the resulting new data is in turn collected and used to generate new knowledge <TextLink reference="19"></TextLink>. The technical solution chosen must be open to any kind of analysis including the longitudinal perspective using advanced statistical methods.</Pgraph><Pgraph>The overall aim of the study is broken down into the following research questions (RQ):</Pgraph><Pgraph><UnorderedList><ListItem level="1">RQ 1. How can an exchange system be designed and technically realised to support the science-practice dialogue for monitoring health IT?</ListItem><ListItem level="1">RQ 2. How do practitioners – here, CIOs of hospitals – evaluate the potentially added values of the exchange system in particular its dashboard interface?</ListItem></UnorderedList></Pgraph><Pgraph>Based on these findings this study intends to propose a model for a science-practice dialogue cycle.</Pgraph></TextBlock>
<TextBlock linked="yes" name="Methods">
<MainHeadline>Methods</MainHeadline><SubHeadline>Components</SubHeadline><Pgraph>The exchange system is meant to support the communication between science and practice for performing repeated scientific benchmarks of healthcare organisations and their digital performance. The requirements <TextLink reference="20"></TextLink>, <TextLink reference="21"></TextLink> for such exchange systems resulted from an incremental evolutionary process that started with a benchmark carried out in 2011 (Figure 1 <ImgLink imgNo="1" imgType="figure"/>) to coordinate the information demands of IT managers of German hospitals <TextLink reference="22"></TextLink>. The benchmark was repeated in 2013 and modified according to the feedback gained in 2011, which gave rise to the development of a hierarchically structured scoring system for measuring the degree of digitisation of clinical processes, i.e. the Workflow Composite Score System (WCS) <TextLink reference="23"></TextLink>, <TextLink reference="24"></TextLink>. Finally, the evaluation of the 2013 system led to the specification of the exchange system of 2018 and its finalisation in 2019 including six components (C). The following requirements can be summarized based on the experience of the two previous iterations in 2011 and 2013 and in the preparations for the 2018 benchmark:</Pgraph><Pgraph><UnorderedList><ListItem level="1">The calculation of the scores for each participant must be automated.</ListItem><ListItem level="1">The individual score of each user must be shown together with his/her reference classes in a single diagram.</ListItem><ListItem level="1">The dashboard user must be able to select between processes and reference groups on every score level.</ListItem><ListItem level="1">The dashboard user must be able to select all indicators assigned to a single process-descriptor pair in a user-friendly manner from one screen.</ListItem><ListItem level="1">The system must be modularised, so it is independent from the survey tools in use.</ListItem><ListItem level="1">The collected data must be persistent and independent of the statistical analysis to avoid causalities or directional relationships.</ListItem><ListItem level="1">The platform must allow for historisation (resp. longitudinal analyses) and consolidation of the data.</ListItem><ListItem level="1">All software, frameworks and libraries must be open source.</ListItem></UnorderedList></Pgraph><Pgraph>Figure 1 <ImgLink imgNo="1" imgType="figure"/> shows the evolution of the components from a primarily manual procedure with only selected electronic tools (2011, 2013) to the targeted digitally integrated system of 2018 with the new components and their integration marked in grey. All these components should be embedded in a technical platform to provide a coherent environment for establishing and maintaining the communication through this system. In this sense, the exchange system should be more than a technical space but should exploit scientific knowledge and methods for practical use (Table 1 <ImgLink imgNo="1" imgType="table"/>). RQ 1 addresses the interaction of all the components via the exchange system. Therefore, this paper focuses on the development of a technical platform to orchestrate these components and thereby makes use of previous work particularly regarding the components 1, 2 and 3 as well as component 4. The technical platform itself must realise the components 5 and 6 and should provide mechanisms for interfacing all components. Research question 2 specifically targets the dashboard, i.e., component 6, and its evaluation by CIOs and IT management related experts. Research questio<TextGroup><PlainText>n 2</PlainText></TextGroup> also addresses the potential added value gained when using the exchange system.</Pgraph><SubHeadline>Data from surveys</SubHeadline><Pgraph>The existing data of the IT Report Health Care surveys from 2011 and 2013 <TextLink reference="25"></TextLink>, <TextLink reference="26"></TextLink> were uploaded together with new data captured by the electronic questionnaire <TextLink reference="27"></TextLink> which made use of the validated scores <TextLink reference="23"></TextLink> (<TextGroup><PlainText>Table 2 </PlainText></TextGroup><ImgLink imgNo="2" imgType="table"/>). The integration of data from 2011 and 2013 should demonstrate the feasibility of the historicisation and consolidation of various research items. The new data stem from two studies. The first study surveyed chief information officers and IT management experts of German hospitals. The link to the electronic questionnaire tool was sent to 1,224 persons responsible for 1,950 institutions of which 224 hospitals (represented by the CIOs and the experts) took part and 197 participated in the Health IT Benchmarking 2018. In addition, a second study surveying clinical directors was conducted to measure the satisfaction of users and the perceived usefulness of the systems. The survey targeted medical and nursing directors of 1,951 German hospitals of which 355 hospitals (represented by the clinical directors) took part and 145 participated in the Health IT Benchmarking 2018. Furthermore, the data of quality reports from German hospitals for the years 2011 to 2016 and the demographic data of the German hospital register were pre-processed and uploaded. This procedure was meant to demonstrate the feasibility of consolidating primary data with secondary data from various sources.</Pgraph><SubHeadline>Evaluation</SubHeadline><Pgraph>In order to answer the second research question, a summative evaluation of the exchange system as such and its interactive dashboard interface was performed. The first aim was to obtain general feedback on the benchmarking itself and the scores used in the benchmark. This general feedback should include information about the extent to which the benchmarking results were used to support the communication between the IT managers and the board of executives. The second aim was to find out if the interaction with the dashboard was acceptable and the dashboard itself was usable. Usability was evaluated pursuant to the two-factor model of Zhang and Dran <TextLink reference="28"></TextLink> based on Herzberg’s two-factor theory (hygiene factors and motivators) <TextLink reference="29"></TextLink>, <TextLink reference="30"></TextLink>, <TextLink reference="31"></TextLink>. According to this theory, hygiene factors also known as “dissa<TextGroup><PlainText>tis</PlainText></TextGroup>fiers”, are essential basic functions that are implicitly expected by the user, so that dissatisfaction arises if they are not fulfilled, but no satisfaction occurs if they are fulfilled <TextLink reference="28"></TextLink>, <TextLink reference="30"></TextLink>. Motivators, on the other hand, have a positive influence on the user's perception if they are present, but their absence has no generally negative <TextGroup><PlainText>influence</PlainText></TextGroup> and, therefore, they are called “satisfiers” <TextLink reference="28"></TextLink>, <TextLink reference="30"></TextLink>. An instrument was developed that operationa<TextGroup><PlainText>lised e</PlainText></TextGroup>leven of the twelve categories adapting them to <TextGroup><PlainText>the d</PlainText></TextGroup>ashboard interface. The instrument embraced <TextGroup><PlainText>18 s</PlainText></TextGroup>tatements for hygiene factors and eight statements for motivator factors to be assessed on a five-point Likert scale (Table 3 <ImgLink imgNo="3" imgType="table"/>). No statements were developed for the “enjoyment” category expressing “fun” and “humour” according to Zhang and Dran <TextLink reference="28"></TextLink> because it was not deemed suitable for this application. </Pgraph></TextBlock>
<TextBlock linked="yes" name="Results">
<MainHeadline>Results</MainHeadline><SubHeadline>Exchange system</SubHeadline><Pgraph>In order to answer research question 1 a technical platform was developed realising and integrating the six components to form the exchange system as a whole. The technical platform comprises four tiers (Figure 2 <ImgLink imgNo="2" imgType="figure"/>):</Pgraph><Pgraph><OrderedList><ListItem level="1" levelPosition="1" numString="1.">data source tier, i.e., integrating component 1,</ListItem><ListItem level="1" levelPosition="2" numString="2.">data tier and</ListItem><ListItem level="1" levelPosition="3" numString="3.">logic tier within the data warehouse system, i.e., implementing component 5, integrating component 4 and component 3 pursuant to component 2 and</ListItem><ListItem level="1" levelPosition="4" numString="4.">presentation tier, i.e., implementing component 6.</ListItem></OrderedList></Pgraph><Pgraph>The source systems were decoupled from the data warehouse system so that any data sources could be connected (Figure 3 <ImgLink imgNo="3" imgType="figure"/>). A data warehouse was set up to persist and consolidate the data. The data model combined the EAV model and the DV approach. It was thus independent of the analysis and, therefore, did not imply causalities or directional relationships for the variables that are based on the items from the survey data and used to calculate the scores for the benchmark. In the source layer, all data is transferred as raw data to the relational database of the data warehouse and archived for the future. The programming language R <TextLink reference="32"></TextLink> was used mainly for advanced statistical analyses. The dashboard served as the interactive user interface of the exchange system. The electronic questionnaires were realised with LimeSurvey 2.54.3 <TextLink reference="33"></TextLink> and Unipark <TextLink reference="34"></TextLink>. PostgreSQL 9.6 <TextLink reference="35"></TextLink> was used as database technology on Ubuntu Server 16.04 <TextLink reference="36"></TextLink>. Extract, transform and load (ETL) processes were built with Pentaho Data Integratio<TextGroup><PlainText>n 7</PlainText></TextGroup> <TextLink reference="37"></TextLink> to load the data sources into the different layers of the data warehouse, transform the data to the data model and for partial calculations for the scores.</Pgraph><Pgraph>Additional processing of the data for the dashboard diagrams was realised through a connector written in Clojure <TextLink reference="38"></TextLink>. These computations embraced e.g. calculating the various boundaries for the references groups is done and all this data is combined with the calculated scores from the data warehouse in a JavaScript Object Notation (JSON) format and made available under a RESTful (Representational state transfer) application programming interface (API) <TextLink reference="39"></TextLink>. The RESTful API is used by the dashboard to request the calculated scores and other data e.g., indicator values, reference classes and demographic data, which are needed for the benchmarking diagrams, in a single hierarchical format via the two parameters User ID and year of participation (Figure 4 <ImgLink imgNo="4" imgType="figure"/>). The benchmarking diagrams were modelled with the library D3.js <TextLink reference="40"></TextLink> and the TypeScript-based web framework Angular 6 <TextLink reference="41"></TextLink> was used to reuse these new diagrams on different levels of the web interface. The access to the dashboard was secured by implementing the open source software Keycloak <TextLink reference="42"></TextLink> for identity access management. </Pgraph><Pgraph>Figure 5 <ImgLink imgNo="5" imgType="figure"/>, Figure 6 <ImgLink imgNo="6" imgType="figure"/> and Figure 7 <ImgLink imgNo="7" imgType="figure"/> show screens of the dashboard, which is divided into a navigation bar at the left and a canvas for the diagrams and additional information. When choosing the synopsis view, the Workflow Composite Score (WCS) as the main indicator for the degree of digital support for patient care processes is displayed (Figure 5 <ImgLink imgNo="5" imgType="figure"/>). The hospital's value is marked as a red bar within the distribution of all WCS values of one of the selected benchmarking reference groups (ownership or bed size). This distribution is divided into the innovation classes according to Rogers’ Diffusion of Innovation Theory <TextLink reference="43"></TextLink>, which are colour-coded in the diagram. The width of each of these classes indicates the absolute number of hospitals in this class. The next segment displays the individual process scores, which are visualised as a red line in a stacked bar chart with the two halves of the distribution marked with different grey shades. The type of reference group (ownership or size) can be selected in the blue box below the headline. In addition, IT satisfaction data from the clinical users of this organisation is displayed as a boxplot below.</Pgraph><Pgraph>More details can be retrieved when selecting one of the processes, e.g., admission (Figure 6 <ImgLink imgNo="6" imgType="figure"/>). The top diagram again shows the admission score depending on the reference group. The admission score is further broken down into the four descriptor sub-scores and is finally amended by a boxplot that depicts the user satisfaction of electronic information logistics for the admission process.</Pgraph><Pgraph>The descriptor sub-scores can be further detailed selecting a corresponding indicator and its diagram (Figure 7 <ImgLink imgNo="7" imgType="figure"/>). The top diagram again shows the selected descriptor sub-score depending on the reference group. For each indicator, user satisfaction with the functions that support the care process is displayed in a boxplot.</Pgraph><SubHeadline>Evaluation</SubHeadline><Pgraph>Out of all the hospitals that participated in the Health IT Benchmarking 2018, 35 IT managers, i.e., the practitioners, took part in the evaluation, which corresponded to a response rate of 17.8% (13.2% completed the full survey). Table 4 <ImgLink imgNo="4" imgType="table"/> shows the percentage of IT managers who rated the comprehensibility of the diagrams and scores as “comprehensible” or “very comprehensible”.</Pgraph><Pgraph>Positive ratings in Table 4 <ImgLink imgNo="4" imgType="table"/> exceeded the 80% limit for all types of diagrams. Answering the questions of how the benchmarking results were used (Table 5 <ImgLink imgNo="5" imgType="table"/>) the participants reported that it served as an information source to be shared when communicating with colleagues (75.8%) and with the hospital management (66.7%) [n=33]. The benchmarking results were mostly used for the comparison of the status quo with other hospitals (85.7%), as a starting point for IT development (80%) and as an assessment of IT maturity (77.1%) [n=35]. When asked about new features, the participants reported that they would like to see specific experience reports (69.2%), get in contact with very well performing hospitals (57.7%) as well as obtain a link to national (61.5%) and international studies (42.3%) [n=26]. </Pgraph><Pgraph>The following statements were made by the participants in the free text field, for example: “<Mark2>the currently approved project ‘Introduction of Digital Dictation’ was especially justified by the fact that we stood out (lagging behind) on this point in the study” or “sensitisation to the topic of ’digitisation’ in the corporate strategy. [...] A scientific and independent study with concrete comparisons of the competitors is a good opportunity to get ’attention’ and to bring the topic ’to mind’ to the top management once more.</Mark2>”. Furthermore, the majority of the participants (57.7%), wished to see their results within three months after submitting their data.</Pgraph><Pgraph>The median value for all hygiene factors was 4 (“agree”) and for the motivators 3.5 (between “neutral” and “agree”, Figure 8 <ImgLink imgNo="8" imgType="figure"/>). Both the hygiene factors and the motivators were rated high and above the neutral line of 3. The motivators were not rated similarly but a bit more towards the neutral line. Table 6 <ImgLink imgNo="6" imgType="table"/> provides the Cronbach’s alpha values of both scales, which were well above 0.70 and thus spoke in favour of high internal consistency.</Pgraph><Pgraph>The hygiene factors’ median of 4 indicated that the basic needs for the web-based dashboard have largely been met but there was still room for improvements. The motivators achieved a median of 3.5, so that the current state of implementation significantly increased the user satisfaction with the web-based dashboard. Again, there was a need for further improvements. </Pgraph></TextBlock>
<TextBlock linked="yes" name="Discussion">
<MainHeadline>Discussion</MainHeadline><SubHeadline>Summary</SubHeadline><Pgraph>The present study shows the technical integration and orchestration of the components of a technical platform, which is meant to enable an exchange system between science and practice to support CIOs in the strategic management of their IT. This exchange system was practically used for benchmarking the digital performance and the user satisfaction of 197 German hospitals. It thus provided feedback to the hospitals that had given IT related information via their CIOs and clinical directors in a first step and closed a first loop of science-practice interaction.</Pgraph><Pgraph>Addressing RQ 1 the components needed were transferred from a predominantly manual system into an integrated digital system by an iterative process. The data is exchanged via the questionnaire interface (C 1) and in turn aggregated information can be extracted via the dashboard (C 6). Both make use of a reliable and validated score system (C 2) so that data processing, i.e., the computation of the score system (C 3), could be fully automated. Comprehensive data management enables the consolidation and historicisation of different data sources (C 4). The different components were integrated with each other (C 5) so that the feedback cycle from collecting the data to providing the results (C 6) could be closed faster than before. The evaluation showed that most of the participants had expressed the wish to obtain feedback within three months after submitting their data. Therefore, compared to previous experience in the years 2011 and 2013, the technical platform leverages a faster process from data collection to information computation and visualisation. </Pgraph><Pgraph>Addressing RQ 2 the benchmarking information generated and presented by the exchange system was mainly used for the strategic communication and planning especially with the hospital’s top management as well as with IT staff and colleagues. Usability testing of the dashboard showed that hygiene factors and motivators were both indicating that there were no major issues dissatisfying the users and many indicators satisfying them. Hygiene factors and motivators were rated nearly similarly. </Pgraph><Pgraph>The study shows for the first time how a science-practice dialogue can be technically implemented within the framework of a nationwide health IT monitoring. It could be shown how the science-practice dialogue can generate scientifically founded strategic information for the management of hospital IT. A special focus was placed on the user-oriented visualization of the strategic information by means of a dynamic dashboard. This enabled extensive insights into the information needs of CIOs and IT-decision makers in healthcare organizations.</Pgraph><SubHeadline>Limitations</SubHeadline><Pgraph>Although this platform has proved its use and usefulness it is still work in process towards a full exchange system. We will expand on this in the outlook. Furthermore, there are some technical improvements necessary that were revealed through the evaluation for example the integration of additional information like specific experience reports and national or international studies. Also, the automation processes of the platform are specific for the construction of the WCS system. If the WCS system changes, the automation tasks must also be adapted. Despite its capabilities to make use of longitudinal analyses such service has not been integrated into the dashboard. Another limitation of this study is the low response rate to the evaluation of the dashboard. This means that it can only be interpreted as a first indication. If, however, we combine these results with the previous findings based on the comments of 33 IT managers and CIOs <TextLink reference="44"></TextLink>, the picture is corroborated that substantial insights can be gained when using the exchange system. Even though hygiene factors and motivators were rated nearly similarly, further improvements of the visual design (access, availability, presentation and navigation) of the dashboard seem advisable.</Pgraph><SubHeadline>Implications and outlook</SubHeadline><Pgraph>Figure 9 <ImgLink imgNo="9" imgType="figure"/> shows the ideal exchange cycle with its different activities of science and practice when exchanging and accumulating data, information, and knowledge <TextLink reference="11"></TextLink>, <TextLink reference="12"></TextLink>, <TextLink reference="13"></TextLink>, <TextLink reference="45"></TextLink>. Informed by <TextLink reference="11"></TextLink> it signifies a new model for a science-practice-dialogue. This ideal cycle (<TextGroup><PlainText>Figure 9 </PlainText></TextGroup><ImgLink imgNo="9" imgType="figure"/>), which goes beyond typical benchmarking, inspired the development of the exchange system, which in the current version, however, only partly realizes this cycle. The data to information exchange cycle is fully implemented while the transformation to knowledge, in particular the co-production of knowledge, needs further development. </Pgraph><Pgraph>CIOs use the information internally for their own purposes, i.e., mainly to make them aware of their status quo, for communication with others and decision making involving the top management. Communication problems between IT experts and executives can arise due to IT knowledge <TextLink reference="46"></TextLink>. The exchange system can provide indicators and scores comprehensibly prepared for the management. This can help the CIOs to develop their line of argument when communicating with the top management. Hereby, diagrams can support verbal presentations and underpin the main message. As the free text fields in the evaluation showed the information extracted from the exchange system finally resulted in practical decisions, e.g., for the implementation of a dedicated application system, or in strategic decisions, e.g., to align the corporate strategy better with IT.</Pgraph><Pgraph>The conceptual taxonomy of knowledge transfer activities by Lomas <TextLink reference="13"></TextLink> that distinguishes between “diffusion”, “dissemination” and “implementation” will guide the next steps of further developing the exchange system. Pursuant to this taxonomy the current stage of the exchange system would be best classified as “dissemination” where the research results are tailored to a particular audience <TextLink reference="47"></TextLink>. While the evaluation results already proved that management changed its behaviour due to the information provided, this was no active process supported virtually by the technical platform but rather triggered by physical meetings between CIOs with other decision makers in their organization. Therefore, the next development steps will further steer the exchange system and more specifically the technical platform towards the category “implementation”, which is defined <TextLink reference="13"></TextLink> by putting research results into practice in an active process. Allowing communities of practice <TextLink reference="8"></TextLink> to share experience and knowledge in a collaborative approach together with scientists is part of this goal to foster the co-production of knowledge. For this purpose, existing knowledge, i.e., scientific publication and best practice reports, is to be displayed directly at the level of the key performance indicator.</Pgraph><Pgraph>In conclusion, this technical platform implements scientific research instruments and integrates authentic data from practice, so that the information provided is relevant both from a scientific and a practical point of view. The digitalisation of this science-practice dialogue through this exchange system ensures a more systematic and scalable process of collecting, processing and transforming data into new findings. This data-driven procedure is meant to bind science and practice in a steady and sustainable manner and thereby overcome the problems of exchanging and communicating between the two different realms of science and practice. Altogether, the study provides a comprehensive basis for the technical realisation of a national health IT monitoring platform.</Pgraph></TextBlock>
<TextBlock linked="yes" name="Notes">
<MainHeadline>Notes</MainHeadline><SubHeadline>Competing interests</SubHeadline><Pgraph>The authors declare that they have no competing interests.</Pgraph><SubHeadline>Funding</SubHeadline><Pgraph>This study was funded by the Ministry of Science and Culture of the State of Lower Saxony, Germany (grant no. ZN 3062).</Pgraph><SubHeadline>Acknowledgements</SubHeadline><Pgraph>We would like to thank Moritz Esdar for conducting the IT Report Health Care survey and defining the computation of the WCS. We would also like to thank Michael Tissen for contributing to the developments of the web-based dashboard. We wish to thank all IT managers/CIOs for participating in this study.</Pgraph></TextBlock>
<References linked="yes">
<Reference refNo="1">
<RefAuthor>Gholami R</RefAuthor>
<RefAuthor>Añón Higón D</RefAuthor>
<RefAuthor>Emrouznejad A</RefAuthor>
<RefTitle>Hospital performance: Efficiency or quality? Can we have both with IT?</RefTitle>
<RefYear>2015</RefYear>
<RefJournal>Expert Syst Appl</RefJournal>
<RefPage>5390-400</RefPage>
<RefTotal>Gholami R, Añón Higón D, Emrouznejad A. Hospital performance: Efficiency or quality? Can we have both with IT? Expert Syst Appl. 2015;42(12):5390-400. DOI: 10.1016/j.eswa.2014.12.019</RefTotal>
<RefLink>https://doi.org/10.1016/j.eswa.2014.12.019</RefLink>
</Reference>
<Reference refNo="2">
<RefAuthor>Adler-Milstein J</RefAuthor>
<RefAuthor>Jha AK</RefAuthor>
<RefTitle>HITECH Act Drove Large Gains In Hospital Electronic Health Record Adoption</RefTitle>
<RefYear>2017</RefYear>
<RefJournal>Health Aff (Millwood)</RefJournal>
<RefPage>1416-22</RefPage>
<RefTotal>Adler-Milstein J, Jha AK. HITECH Act Drove Large Gains In Hospital Electronic Health Record Adoption. Health Aff (Millwood). 2017 Aug;36(8):1416-22. DOI: 10.1377/hlthaff.2016.1651</RefTotal>
<RefLink>https://doi.org/10.1377/hlthaff.2016.1651</RefLink>
</Reference>
<Reference refNo="3">
<RefAuthor>Johnston DS</RefAuthor>
<RefTitle>Digital maturity: are we ready to use technology in the NHS?</RefTitle>
<RefYear>2017</RefYear>
<RefJournal>Future Healthc J</RefJournal>
<RefPage>189-92</RefPage>
<RefTotal>Johnston DS. Digital maturity: are we ready to use technology in the NHS? Future Healthc J. 2017 Oct;4(3):189-92. DOI: 10.7861/futurehosp.4-3-189</RefTotal>
<RefLink>https://doi.org/10.7861/futurehosp.4-3-189</RefLink>
</Reference>
<Reference refNo="4">
<RefAuthor>Nøhr C</RefAuthor>
<RefAuthor>Koch S</RefAuthor>
<RefAuthor>Vimarlund V</RefAuthor>
<RefAuthor>Gilstad H</RefAuthor>
<RefAuthor>Faxvaag A</RefAuthor>
<RefAuthor>Hardardottir GA</RefAuthor>
<RefAuthor>Andreassen HK</RefAuthor>
<RefAuthor>Kangas M</RefAuthor>
<RefAuthor>Reponen J</RefAuthor>
<RefAuthor>Bertelsen P</RefAuthor>
<RefAuthor>Villumsen S</RefAuthor>
<RefAuthor>Hyppönen H</RefAuthor>
<RefTitle>Monitoring and Benchmarking eHealth in the Nordic Countries</RefTitle>
<RefYear>2018</RefYear>
<RefJournal>Stud Health Technol Inform</RefJournal>
<RefPage>86-90</RefPage>
<RefTotal>Nøhr C, Koch S, Vimarlund V, Gilstad H, Faxvaag A, Hardardottir GA, Andreassen HK, Kangas M, Reponen J, Bertelsen P, Villumsen S, Hyppönen H. Monitoring and Benchmarking eHealth in the Nordic Countries. Stud Health Technol Inform. 2018;247:86-90.</RefTotal>
</Reference>
<Reference refNo="5">
<RefAuthor>Kucera M</RefAuthor>
<RefTitle>Krankenhauszukunftsgesetz: Milliarden für die Zukunft</RefTitle>
<RefYear>2020</RefYear>
<RefJournal>kma – Klinik Management aktuell</RefJournal>
<RefPage>34-5</RefPage>
<RefTotal>Kucera M. Krankenhauszukunftsgesetz: Milliarden für die Zukunft. kma – Klinik Management aktuell. 2020;25(10):34-5. DOI: 10.1055/s-0040-1718815</RefTotal>
<RefLink>https://doi.org/10.1055/s-0040-1718815</RefLink>
</Reference>
<Reference refNo="6">
<RefAuthor>Everson J</RefAuthor>
<RefAuthor>Rubin JC</RefAuthor>
<RefAuthor>Friedman CP</RefAuthor>
<RefTitle>Reconsidering hospital EHR adoption at the dawn of HITECH: implications of the reported 9% adoption of a “basic” EHR</RefTitle>
<RefYear>2020</RefYear>
<RefJournal>J Am Med Inform Assoc</RefJournal>
<RefPage>1198-205</RefPage>
<RefTotal>Everson J, Rubin JC, Friedman CP. Reconsidering hospital EHR adoption at the dawn of HITECH: implications of the reported 9% adoption of a “basic” EHR. J Am Med Inform Assoc. 2020 Aug;27(8):1198-205. DOI: 10.1093/jamia/ocaa090</RefTotal>
<RefLink>https://doi.org/10.1093/jamia/ocaa090</RefLink>
</Reference>
<Reference refNo="7">
<RefAuthor>Dilling L</RefAuthor>
<RefAuthor>Lemos MC</RefAuthor>
<RefTitle>Creating usable science: Opportunities and constraints for climate knowledge use and their implications for science policy</RefTitle>
<RefYear>2011</RefYear>
<RefJournal>Global Environ Chang</RefJournal>
<RefPage>680-9</RefPage>
<RefTotal>Dilling L, Lemos MC. Creating usable science: Opportunities and constraints for climate knowledge use and their implications for science policy. Global Environ Chang. 2011;21(2):680-9. DOI: 10.1016/j.gloenvcha.2010.11.006</RefTotal>
<RefLink>https://doi.org/10.1016/j.gloenvcha.2010.11.006</RefLink>
</Reference>
<Reference refNo="8">
<RefAuthor>Roux DJ</RefAuthor>
<RefAuthor>Rogers KH</RefAuthor>
<RefAuthor>Biggs HC</RefAuthor>
<RefAuthor>Ashton PJ</RefAuthor>
<RefAuthor>Sergeant A</RefAuthor>
<RefTitle>Bridging the science-management divide: Moving from unidirectional knowledge transfer to knowledge interfacing and sharing</RefTitle>
<RefYear>2006</RefYear>
<RefJournal>Ecol Soc</RefJournal>
<RefPage>4</RefPage>
<RefTotal>Roux DJ, Rogers KH, Biggs HC, Ashton PJ, Sergeant A. Bridging the science-management divide: Moving from unidirectional knowledge transfer to knowledge interfacing and sharing. Ecol Soc. 2006;11(1):4. Available from:
http://www.ecologyandsociety.org/vol11/iss1/art4/</RefTotal>
<RefLink>http://www.ecologyandsociety.org/vol11/iss1/art4/</RefLink>
</Reference>
<Reference refNo="9">
<RefAuthor>Thye J</RefAuthor>
<RefAuthor>Esdar M</RefAuthor>
<RefAuthor>Liebe JD</RefAuthor>
<RefAuthor>Jahn F</RefAuthor>
<RefAuthor>Winter A</RefAuthor>
<RefAuthor>Hübner U</RefAuthor>
<RefTitle>Professionalism of Information Management in Health Care: Development and Validation of the Construct and Its Measurement</RefTitle>
<RefYear>2020</RefYear>
<RefJournal>Methods Inf Med</RefJournal>
<RefPage>e1-e12</RefPage>
<RefTotal>Thye J, Esdar M, Liebe JD, Jahn F, Winter A, Hübner U. Professionalism of Information Management in Health Care: Development and Validation of the Construct and Its Measurement. Methods Inf Med. 2020 Jun;59(S01):e1-e12. DOI: 10.1055/s-0040-1712465</RefTotal>
<RefLink>https://doi.org/10.1055/s-0040-1712465</RefLink>
</Reference>
<Reference refNo="10">
<RefAuthor>Carlile PR</RefAuthor>
<RefAuthor>Rebentisch ES</RefAuthor>
<RefTitle>Into the Black Box: The Knowledge Transformation Cycle</RefTitle>
<RefYear>2003</RefYear>
<RefJournal>Manag Sci</RefJournal>
<RefPage>1180-95</RefPage>
<RefTotal>Carlile PR, Rebentisch ES. Into the Black Box: The Knowledge Transformation Cycle. Manag Sci. 2003;49(9):1180-95. DOI: 10.1287/mnsc.49.9.1180.16564</RefTotal>
<RefLink>https://doi.org/10.1287/mnsc.49.9.1180.16564</RefLink>
</Reference>
<Reference refNo="11">
<RefAuthor>Liew A</RefAuthor>
<RefTitle>Understanding data, information, knowledge and their inter-relationships</RefTitle>
<RefYear>2007</RefYear>
<RefJournal>J Knowl Manag Pract</RefJournal>
<RefPage>1-16</RefPage>
<RefTotal>Liew A. Understanding data, information, knowledge and their inter-relationships. J Knowl Manag Pract. 2007;8(2):1-16.</RefTotal>
</Reference>
<Reference refNo="12">
<RefAuthor>Liyanage C</RefAuthor>
<RefAuthor>Elhag T</RefAuthor>
<RefAuthor>Ballal T</RefAuthor>
<RefAuthor>Li Q</RefAuthor>
<RefTitle>Knowledge communication and translation – a knowledge transfer model</RefTitle>
<RefYear>2009</RefYear>
<RefJournal>J Knowl Manag</RefJournal>
<RefPage>118-31</RefPage>
<RefTotal>Liyanage C, Elhag T, Ballal T, Li Q. Knowledge communication and translation – a knowledge transfer model. J Knowl Manag. 2009;13(3):118-31. DOI: 10.1108/13673270910962914</RefTotal>
<RefLink>https://doi.org/10.1108/13673270910962914</RefLink>
</Reference>
<Reference refNo="13">
<RefAuthor>Lomas J</RefAuthor>
<RefTitle>Diffusion, dissemination, and implementation: who should do what?</RefTitle>
<RefYear>1993</RefYear>
<RefJournal>Ann N Y Acad Sci</RefJournal>
<RefPage>226-35; discussion 235-7</RefPage>
<RefTotal>Lomas J. Diffusion, dissemination, and implementation: who should do what? Ann N Y Acad Sci. 1993 Dec;703:226-35; discussion 235-7. DOI: 10.1111/j.1749-6632.1993.tb26351.x</RefTotal>
<RefLink>https://doi.org/10.1111/j.1749-6632.1993.tb26351.x</RefLink>
</Reference>
<Reference refNo="14">
<RefAuthor>Wehrens R</RefAuthor>
<RefTitle>Beyond two communities – from research utilization and knowledge translation to co-production?</RefTitle>
<RefYear>2014</RefYear>
<RefJournal>Public Health</RefJournal>
<RefPage>545-51</RefPage>
<RefTotal>Wehrens R. Beyond two communities – from research utilization and knowledge translation to co-production? Public Health. 2014 Jun;128(6):545-51. DOI: 10.1016/j.puhe.2014.02.004</RefTotal>
<RefLink>https://doi.org/10.1016/j.puhe.2014.02.004</RefLink>
</Reference>
<Reference refNo="15">
<RefAuthor>Athey BD</RefAuthor>
<RefAuthor>Braxenthaler M</RefAuthor>
<RefAuthor>Haas M</RefAuthor>
<RefAuthor>Guo Y</RefAuthor>
<RefTitle>tranSMART: An Open Source and Community-Driven Informatics and Data Sharing Platform for Clinical and Translational Research</RefTitle>
<RefYear>2013</RefYear>
<RefJournal>AMIA Jt Summits Transl Sci Proc</RefJournal>
<RefPage>6-8</RefPage>
<RefTotal>Athey BD, Braxenthaler M, Haas M, Guo Y. tranSMART: An Open Source and Community-Driven Informatics and Data Sharing Platform for Clinical and Translational Research. AMIA Jt Summits Transl Sci Proc. 2013 Mar 18;2013:6-8.</RefTotal>
</Reference>
<Reference refNo="16">
<RefAuthor>Murphy SN</RefAuthor>
<RefAuthor>Weber G</RefAuthor>
<RefAuthor>Mendis M</RefAuthor>
<RefAuthor>Gainer V</RefAuthor>
<RefAuthor>Chueh HC</RefAuthor>
<RefAuthor>Churchill S</RefAuthor>
<RefAuthor>Kohane I</RefAuthor>
<RefTitle>Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2)</RefTitle>
<RefYear>2010</RefYear>
<RefJournal>J Am Med Inform Assoc</RefJournal>
<RefPage>124-30</RefPage>
<RefTotal>Murphy SN, Weber G, Mendis M, Gainer V, Chueh HC, Churchill S, Kohane I. Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2). J Am Med Inform Assoc. 2010 Mar-Apr;17(2):124-30. DOI: 10.1136/jamia.2009.000893</RefTotal>
<RefLink>https://doi.org/10.1136/jamia.2009.000893</RefLink>
</Reference>
<Reference refNo="17">
<RefAuthor>HIMSS – Healthcare Information and Management Systems Society</RefAuthor>
<RefTitle></RefTitle>
<RefYear>2017</RefYear>
<RefBookTitle>EMRAM model: A strategic roadmap for effective EMR adoption and usage</RefBookTitle>
<RefPage></RefPage>
<RefTotal>HIMSS – Healthcare Information and Management Systems Society. EMRAM model: A strategic roadmap for effective EMR adoption and usage. 2017 [cited 2019 Jul 24]. Available from: http://www.himss.eu/healthcare-providers/emram</RefTotal>
<RefLink>http://www.himss.eu/healthcare-providers/emram</RefLink>
</Reference>
<Reference refNo="18">
<RefAuthor>Moeller H</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>Meisterworks platform</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Moeller H. Meisterworks platform. [cited 2020 Nov 3]. Available from: https://m.works/</RefTotal>
<RefLink>https://m.works/</RefLink>
</Reference>
<Reference refNo="19">
<RefAuthor>Friedman CP</RefAuthor>
<RefAuthor>Rubin JC</RefAuthor>
<RefAuthor>Sullivan KJ</RefAuthor>
<RefTitle>Toward an Information Infrastructure for Global Health Improvement</RefTitle>
<RefYear>2017</RefYear>
<RefJournal>Yearb Med Inform</RefJournal>
<RefPage>16-23</RefPage>
<RefTotal>Friedman CP, Rubin JC, Sullivan KJ. Toward an Information Infrastructure for Global Health Improvement. Yearb Med Inform. 2017 Aug;26(1):16-23. DOI: 10.15265/IY-2017-004</RefTotal>
<RefLink>https://doi.org/10.15265/IY-2017-004</RefLink>
</Reference>
<Reference refNo="20">
<RefAuthor>Weiß JP</RefAuthor>
<RefAuthor>Hübner U</RefAuthor>
<RefAuthor>Rauch J</RefAuthor>
<RefAuthor>Hüsers J</RefAuthor>
<RefAuthor>Teuteberg F</RefAuthor>
<RefAuthor>Esdar M</RefAuthor>
<RefAuthor>Liebe JD</RefAuthor>
<RefTitle>Implementing a Data Management Platform for Longitudinal Health Research</RefTitle>
<RefYear>2017</RefYear>
<RefJournal>Stud Health Technol Inform</RefJournal>
<RefPage>85-9</RefPage>
<RefTotal>Weiß JP, Hübner U, Rauch J, Hüsers J, Teuteberg F, Esdar M, Liebe JD. Implementing a Data Management Platform for Longitudinal Health Research. Stud Health Technol Inform. 2017;243:85-9.</RefTotal>
</Reference>
<Reference refNo="21">
<RefAuthor>Rauch J</RefAuthor>
<RefAuthor>Weiss JP</RefAuthor>
<RefAuthor>Teuteberg F</RefAuthor>
<RefAuthor>Hübner U</RefAuthor>
<RefTitle>Konsolidierte Datenmodellierung von Versorgungsdaten mit dem Entity-Attribute-Value-Modell und Data Vault</RefTitle>
<RefYear>2017</RefYear>
<RefJournal>GMS Med Inform Biom Epidemiol</RefJournal>
<RefPage>Doc03</RefPage>
<RefTotal>Rauch J, Weiss JP, Teuteberg F, Hübner U. Konsolidierte Datenmodellierung von Versorgungsdaten mit dem Entity-Attribute-Value-Modell und Data Vault [Consolidated data modeling of health services research data with the entity-attribute-value model and data vault]. GMS Med Inform Biom Epidemiol. 2017;13(1):Doc03. DOI: 10.3205/mibe000170</RefTotal>
<RefLink>https://doi.org/10.3205/mibe000170</RefLink>
</Reference>
<Reference refNo="22">
<RefAuthor>Liebe JD</RefAuthor>
<RefAuthor>Egbert N</RefAuthor>
<RefAuthor>Frey A</RefAuthor>
<RefAuthor>Hübner U</RefAuthor>
<RefTitle>Characteristics of german hospitals Adopting health IT systems – Results from an empirical study</RefTitle>
<RefYear>2011</RefYear>
<RefBookTitle>User centred networked health care: Proceedings of MIE 2011 (23rd International Conference of the European Federation for Medical Informatics); 2011 Aug 28-31; Oslo, Norway</RefBookTitle>
<RefPage>335-8</RefPage>
<RefTotal>Liebe JD, Egbert N, Frey A, Hübner U. Characteristics of german hospitals Adopting health IT systems – Results from an empirical study. In: Moen A, editor. User centred networked health care: Proceedings of MIE 2011 (23rd International Conference of the European Federation for Medical Informatics); 2011 Aug 28-31; Oslo, Norway. Amsterdam: IOS Press; 2011. (Studies in health technology and informatics; 169). p. 335-8.</RefTotal>
</Reference>
<Reference refNo="23">
<RefAuthor>Liebe JD</RefAuthor>
<RefAuthor>Hübner U</RefAuthor>
<RefAuthor>Straede MC</RefAuthor>
<RefAuthor>Thye J</RefAuthor>
<RefTitle>Developing a Workflow Composite Score to Measure Clinical Information Logistics. A Top-down Approach</RefTitle>
<RefYear>2015</RefYear>
<RefJournal>Methods Inf Med</RefJournal>
<RefPage>424-33</RefPage>
<RefTotal>Liebe JD, Hübner U, Straede MC, Thye J. Developing a Workflow Composite Score to Measure Clinical Information Logistics. A Top-down Approach. Methods Inf Med. 2015;54(5):424-33. DOI: 10.3414/ME14-02-0025</RefTotal>
<RefLink>https://doi.org/10.3414/ME14-02-0025</RefLink>
</Reference>
<Reference refNo="24">
<RefAuthor>Liebe JD</RefAuthor>
<RefAuthor>Hübner U</RefAuthor>
<RefTitle>Developing and trialling an independent, scalable and repeatable IT-benchmarking procedure for healthcare organisations</RefTitle>
<RefYear>2013</RefYear>
<RefJournal>Methods Inf Med</RefJournal>
<RefPage>360-9</RefPage>
<RefTotal>Liebe JD, Hübner U. Developing and trialling an independent, scalable and repeatable IT-benchmarking procedure for healthcare organisations. Methods Inf Med. 2013;52(4):360-9. DOI: 10.3414/ME12-02-0016</RefTotal>
<RefLink>https://doi.org/10.3414/ME12-02-0016</RefLink>
</Reference>
<Reference refNo="48">
<RefAuthor>MacKenzie SB</RefAuthor>
<RefAuthor>Podsakoff PM</RefAuthor>
<RefAuthor>Podsakoff NP</RefAuthor>
<RefTitle>Construct measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques</RefTitle>
<RefYear>2011</RefYear>
<RefJournal>MIS Q</RefJournal>
<RefPage>293-334</RefPage>
<RefTotal>MacKenzie SB, Podsakoff PM, Podsakoff NP. Construct measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques. MIS Q. 2011;35(2):293-334. DOI: 10.2307/23044045.</RefTotal>
<RefLink>https://doi.org/10.2307/23044045.</RefLink>
</Reference>
<Reference refNo="49">
<RefAuthor>Bucher T</RefAuthor>
<RefAuthor>Dinter B</RefAuthor>
<RefTitle>Process Orientation of Information Logistics An Empirical Analysis to Assess Benefits, Design Factors, and Realization Approaches</RefTitle>
<RefYear></RefYear>
<RefBookTitle>Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008); 2008 Jan 07-10; Waikoloa, Hawaii</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Bucher T, Dinter B. Process Orientation of Information Logistics An Empirical Analysis to Assess Benefits, Design Factors, and Realization Approaches. In: IEEE, editor. Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008); 2008 Jan 07-10; Waikoloa, Hawaii. DOI: 10.1109/HICSS.2008.361</RefTotal>
<RefLink>https://doi.org/10.1109/HICSS.2008.361</RefLink>
</Reference>
<Reference refNo="50">
<RefAuthor>Delone WH</RefAuthor>
<RefAuthor>McLean ER</RefAuthor>
<RefTitle>The DeLone and McLean Model of Information Systems Success: A Ten-Year Update</RefTitle>
<RefYear>2003</RefYear>
<RefJournal>J Manag Inform Syst</RefJournal>
<RefPage>9-30</RefPage>
<RefTotal>Delone WH, McLean ER. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. J Manag Inform Syst. 2003;19(4):9-30. DOI: 10.1080/07421222.2003.11045748</RefTotal>
<RefLink>https://doi.org/10.1080/07421222.2003.11045748</RefLink>
</Reference>
<Reference refNo="51">
<RefAuthor>Saaty TL</RefAuthor>
<RefTitle>What is the analytic hierarchy process?</RefTitle>
<RefYear>1988</RefYear>
<RefBookTitle>Mathematical models for decision support</RefBookTitle>
<RefPage>109-21</RefPage>
<RefTotal>Saaty TL. What is the analytic hierarchy process? In: Mitra G, Greenberg HJ, Lootsma FA, Rijkaert MJ, Zimmermann HJ, editors. Mathematical models for decision support. Berlin: Springer Verlag; 1988. p. 109-21.</RefTotal>
</Reference>
<Reference refNo="52">
<RefAuthor>Saaty TL</RefAuthor>
<RefTitle>Decision making with the analytic hierarchy process</RefTitle>
<RefYear>2008</RefYear>
<RefJournal>Int J Serv Sci</RefJournal>
<RefPage>83-98</RefPage>
<RefTotal>Saaty TL. Decision making with the analytic hierarchy process. Int J Serv Sci. 2008;1(1):83-98. DOI: 10.1504/IJSSCI.2008.017590</RefTotal>
<RefLink>https://doi.org/10.1504/IJSSCI.2008.017590</RefLink>
</Reference>
<Reference refNo="43">
<RefAuthor>Rogers EM</RefAuthor>
<RefTitle></RefTitle>
<RefYear>2010</RefYear>
<RefBookTitle>Diffusion of Innovations</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Rogers EM. Diffusion of Innovations. 5th ed. New York: Free Press; 2010.</RefTotal>
</Reference>
<Reference refNo="53">
<RefAuthor>Löper D</RefAuthor>
<RefAuthor>Klettke M</RefAuthor>
<RefAuthor>Bruder I</RefAuthor>
<RefAuthor>Heuer A</RefAuthor>
<RefTitle>Integrating healthcare-related information using the entity-attribute-value storage model</RefTitle>
<RefYear></RefYear>
<RefBookTitle>Health information science. Proceedings First International Conference HIS 2012; 2012 Apr 8-10; Beijing, China</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Löper D, Klettke M, Bruder I, Heuer A. Integrating healthcare-related information using the entity-attribute-value storage model. In: He J, Liu X, Krupinski EA, Xu G, editors. Health information science. Proceedings First International Conference HIS 2012; 2012 Apr 8-10; Beijing, China. Berlin, Heidelberg: Springer; 2012. (Lecture Notes in Computer Science; 7231). p. 13-24. DOI: 10.1007/978-3-642-29361-0_4</RefTotal>
<RefLink>https://doi.org/10.1007/978-3-642-29361-0_4</RefLink>
</Reference>
<Reference refNo="54">
<RefAuthor>Linstedt D</RefAuthor>
<RefAuthor>Olschimke M</RefAuthor>
<RefTitle></RefTitle>
<RefYear>2016</RefYear>
<RefBookTitle>Building a scalable data warehouse with Data Vault 2.0</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Linstedt D, Olschimke M. Building a scalable data warehouse with Data Vault 2.0. Waltham, MA: Morgan Kaufmann; 2016.</RefTotal>
</Reference>
<Reference refNo="55">
<RefAuthor>Dresch A</RefAuthor>
<RefAuthor>Lacerda DP</RefAuthor>
<RefAuthor>Antunes Jr JAV</RefAuthor>
<RefTitle></RefTitle>
<RefYear>2015</RefYear>
<RefBookTitle>Design Science Research</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Dresch A, Lacerda DP, Antunes Jr JAV. Design Science Research. Cham: Springer International Publishing; 2015. DOI: 10.1007/978-3-319-07374-3</RefTotal>
<RefLink>https://doi.org/10.1007/978-3-319-07374-3</RefLink>
</Reference>
<Reference refNo="25">
<RefAuthor>Hübner U</RefAuthor>
<RefAuthor>Liebe JD</RefAuthor>
<RefAuthor>Egbert N</RefAuthor>
<RefAuthor>Frey A</RefAuthor>
<RefTitle></RefTitle>
<RefYear>2012</RefYear>
<RefBookTitle>IT-Report Gesundheitswesen. Schwerpunkt IT im Krankenhaus</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Hübner U, Liebe JD, Egbert N, Frey A. IT-Report Gesundheitswesen. Schwerpunkt IT im Krankenhaus. 2012 [cited 2019 Jan 22]. Available from: http://www.it-report-gesundheitswesen.de/</RefTotal>
<RefLink>http://www.it-report-gesundheitswesen.de/</RefLink>
</Reference>
<Reference refNo="26">
<RefAuthor>Hübner U</RefAuthor>
<RefAuthor>Liebe JD</RefAuthor>
<RefAuthor>Straede MC</RefAuthor>
<RefAuthor>Thye J</RefAuthor>
<RefTitle></RefTitle>
<RefYear>2014</RefYear>
<RefBookTitle>IT-Report Gesundheitswesen. Schwerpunkt IT-Unterstützung klinischer Prozesse</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Hübner U, Liebe JD, Straede MC, Thye J. IT-Report Gesundheitswesen. Schwerpunkt IT-Unterstützung klinischer Prozesse. 2014 [cited 2019 Jan 22]. Available from:
http://www.it-report-gesundheitswesen.de/</RefTotal>
<RefLink>http://www.it-report-gesundheitswesen.de/</RefLink>
</Reference>
<Reference refNo="27">
<RefAuthor>Hübner U</RefAuthor>
<RefAuthor>Liebe JD</RefAuthor>
<RefAuthor>Egbert N</RefAuthor>
<RefAuthor>Thye J</RefAuthor>
<RefAuthor>Esdar M</RefAuthor>
<RefAuthor>Hüsers J</RefAuthor>
<RefAuthor>Rauch J</RefAuthor>
<RefAuthor>Weiß JP</RefAuthor>
<RefTitle></RefTitle>
<RefYear>2018</RefYear>
<RefBookTitle>IT-Report Gesundheitswesen: Schwerpunkt: Wie reif ist die IT in deutschen Krankenhäusern?</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Hübner U, Liebe JD, Egbert N, Thye J, Esdar M, Hüsers J, Rauch J, Weiß JP. IT-Report Gesundheitswesen: Schwerpunkt: Wie reif ist die IT in deutschen Krankenhäusern? 2018 [cited 2019 Jan 22]. Available from: http://www.it-report-gesundheitswesen.de/</RefTotal>
<RefLink>http://www.it-report-gesundheitswesen.de/</RefLink>
</Reference>
<Reference refNo="28">
<RefAuthor>Zhang P</RefAuthor>
<RefAuthor>Dran GM von</RefAuthor>
<RefTitle>Satisfiers and dissatisfiers: A two-factor model for website design and evaluation</RefTitle>
<RefYear>2000</RefYear>
<RefJournal>J Am Soc Inform Sci</RefJournal>
<RefPage>1253-68</RefPage>
<RefTotal>Zhang P, Dran GM von. Satisfiers and dissatisfiers: A two-factor model for website design and evaluation. J Am Soc Inform Sci. 2000;51(14):1253-68. DOI: 10.1002/1097-4571(2000)9999:9999<:AID-ASI1039>3.0.CO;2-O</RefTotal>
<RefLink>https://doi.org/10.1002/1097-4571(2000)9999:9999<:AID-ASI1039>3.0.CO;2-O</RefLink>
</Reference>
<Reference refNo="29">
<RefAuthor>Herzberg FI</RefAuthor>
<RefTitle></RefTitle>
<RefYear>1966</RefYear>
<RefBookTitle>Work and the nature of man</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Herzberg FI. Work and the nature of man. Cleveland: World Publishing; 1966.</RefTotal>
</Reference>
<Reference refNo="30">
<RefAuthor>Herzberg F</RefAuthor>
<RefTitle>One more time: How do you motivate employees?</RefTitle>
<RefYear>1968</RefYear>
<RefJournal>Harvard Bus Rev</RefJournal>
<RefPage>53-62</RefPage>
<RefTotal>Herzberg F. One more time: How do you motivate employees? Harvard Bus Rev. 1968 Jan-Feb:53-62.</RefTotal>
</Reference>
<Reference refNo="31">
<RefAuthor>Herzberg F</RefAuthor>
<RefTitle>One more time: How do you motivate employees?</RefTitle>
<RefYear>1987</RefYear>
<RefJournal>Harvard Bus Rev</RefJournal>
<RefPage>109-20</RefPage>
<RefTotal>Herzberg F. One more time: How do you motivate employees? Harvard Bus Rev. 1987 Sep-Oct:109-20.</RefTotal>
</Reference>
<Reference refNo="32">
<RefAuthor>The R Foundation</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>The R Project for Statistical Computing</RefBookTitle>
<RefPage></RefPage>
<RefTotal>The R Foundation. The R Project for Statistical Computing. [cited 2021 Jan 23]. Available from: https://www.r-project.org/</RefTotal>
<RefLink>https://www.r-project.org/</RefLink>
</Reference>
<Reference refNo="33">
<RefAuthor>LimeSurvey GmbH</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>LimeSurvey</RefBookTitle>
<RefPage></RefPage>
<RefTotal>LimeSurvey GmbH. LimeSurvey. [cited 2021 Jan 23]. Available from: https://www.limesurvey.org/</RefTotal>
<RefLink>https://www.limesurvey.org/</RefLink>
</Reference>
<Reference refNo="34">
<RefAuthor>Tivian</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>UNIPARK</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Tivian. UNIPARK. [cited 2021 Jan 23]. Available from: https://www.unipark.com/</RefTotal>
<RefLink>https://www.unipark.com/</RefLink>
</Reference>
<Reference refNo="35">
<RefAuthor>The PostgreSQL Global Development Group</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>PostgreSQL</RefBookTitle>
<RefPage></RefPage>
<RefTotal>The PostgreSQL Global Development Group. PostgreSQL. [cited 2021 Jan 23]. Available from: https://www.postgresql.org/</RefTotal>
<RefLink>https://www.postgresql.org/</RefLink>
</Reference>
<Reference refNo="36">
<RefAuthor>Canonical Ltd.</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>Ubuntu</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Canonical Ltd. Ubuntu. [cited 2021 Jan 23]. Available from: https://ubuntu.com/</RefTotal>
<RefLink>https://ubuntu.com/</RefLink>
</Reference>
<Reference refNo="37">
<RefAuthor>Hitachi Vantara Corporation</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>Pentaho Data Integration</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Hitachi Vantara Corporation. Pentaho Data Integration. [cited 2021 Jan 23]. Available from: https://www.hitachivantara.com/de-de/products/data-management-analytics/pentaho-platform/pentaho-data-integration.html</RefTotal>
<RefLink>https://www.hitachivantara.com/de-de/products/data-management-analytics/pentaho-platform/pentaho-data-integration.html</RefLink>
</Reference>
<Reference refNo="38">
<RefAuthor>Hickey R</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>Clojure</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Hickey R. Clojure. [cited 2021 Jan 23]. Available from: https://clojure.org/</RefTotal>
<RefLink>https://clojure.org/</RefLink>
</Reference>
<Reference refNo="39">
<RefAuthor>VMware</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>Spring Framework</RefBookTitle>
<RefPage></RefPage>
<RefTotal>VMware. Spring Framework. [cited 2021 Jan 23]. Available from: https://spring.io/projects/spring-framework</RefTotal>
<RefLink>https://spring.io/projects/spring-framework</RefLink>
</Reference>
<Reference refNo="40">
<RefAuthor>Bostock M</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>D3.js</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Bostock M. D3.js. [cited 2021 Jan 23]. Available from: https://d3js.org/</RefTotal>
<RefLink>https://d3js.org/</RefLink>
</Reference>
<Reference refNo="41">
<RefAuthor>Google LLC</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>Angular</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Google LLC. Angular. [cited 2021 Jan 23]. Available from: https://angular.io/</RefTotal>
<RefLink>https://angular.io/</RefLink>
</Reference>
<Reference refNo="42">
<RefAuthor>Anonym</RefAuthor>
<RefTitle></RefTitle>
<RefYear></RefYear>
<RefBookTitle>Keycloak</RefBookTitle>
<RefPage></RefPage>
<RefTotal>Keycloak. [cited 2021 Jan 23]. Available from: https://www.keycloak.org/</RefTotal>
<RefLink>https://www.keycloak.org/</RefLink>
</Reference>
<Reference refNo="44">
<RefAuthor>Weiß JP</RefAuthor>
<RefAuthor>Thye J</RefAuthor>
<RefAuthor>Rauch J</RefAuthor>
<RefAuthor>Tissen M</RefAuthor>
<RefAuthor>Esdar M</RefAuthor>
<RefAuthor>Teuteberg F</RefAuthor>
<RefAuthor>Hübner UH</RefAuthor>
<RefTitle>IT-Benchmarking als Zusammenspiel von Wissenschaft und Praxis – ein Web-Portal zur Dissemination individueller Ergebnisse für Krankenhäuser</RefTitle>
<RefYear></RefYear>
<RefBookTitle>Data driven X-Turning Data into Value. Multikonferenz der Wirtschaftsinformatik (MKWI 2018); 2018 Mar 6-9; Lüneburg, Germany</RefBookTitle>
<RefPage>659-70</RefPage>
<RefTotal>Weiß JP, Thye J, Rauch J, Tissen M, Esdar M, Teuteberg F, Hübner UH. IT-Benchmarking als Zusammenspiel von Wissenschaft und Praxis – ein Web-Portal zur Dissemination individueller Ergebnisse für Krankenhäuser. In: Drews P, Funk B, Niemeyer P, Xie L, editors. Data driven X-Turning Data into Value. Multikonferenz der Wirtschaftsinformatik (MKWI 2018); 2018 Mar 6-9; Lüneburg, Germany. p. 659-70.</RefTotal>
</Reference>
<Reference refNo="45">
<RefAuthor>Gagnon ML</RefAuthor>
<RefTitle>Moving knowledge to action through dissemination and exchange</RefTitle>
<RefYear>2011</RefYear>
<RefJournal>J Clin Epidemiol</RefJournal>
<RefPage>25-31</RefPage>
<RefTotal>Gagnon ML. Moving knowledge to action through dissemination and exchange. J Clin Epidemiol. 2011 Jan;64(1):25-31. DOI: 10.1016/j.jclinepi.2009.08.013</RefTotal>
<RefLink>https://doi.org/10.1016/j.jclinepi.2009.08.013</RefLink>
</Reference>
<Reference refNo="46">
<RefAuthor>Thye J</RefAuthor>
<RefAuthor>Hübner U</RefAuthor>
<RefAuthor>Hüsers J</RefAuthor>
<RefAuthor>Babitsch B</RefAuthor>
<RefTitle>IT Decision Making in German Hospitals - Do CEOs Open the Black Box?</RefTitle>
<RefYear>2017</RefYear>
<RefJournal>Stud Health Technol Inform</RefJournal>
<RefPage>112-16</RefPage>
<RefTotal>Thye J, Hübner U, Hüsers J, Babitsch B. IT Decision Making in German Hospitals - Do CEOs Open the Black Box? Stud Health Technol Inform. 2017;243:112-16.</RefTotal>
</Reference>
<Reference refNo="47">
<RefAuthor>Graham ID</RefAuthor>
<RefAuthor>Logan J</RefAuthor>
<RefAuthor>Harrison MB</RefAuthor>
<RefAuthor>Straus SE</RefAuthor>
<RefAuthor>Tetroe J</RefAuthor>
<RefAuthor>Caswell W</RefAuthor>
<RefAuthor>Robinson N</RefAuthor>
<RefTitle>Lost in knowledge translation: time for a map?</RefTitle>
<RefYear>2006</RefYear>
<RefJournal>J Contin Educ Health Prof</RefJournal>
<RefPage>13-24</RefPage>
<RefTotal>Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13-24. DOI: 10.1002/chp.47</RefTotal>
<RefLink>https://doi.org/10.1002/chp.47</RefLink>
</Reference>
</References>
<Media>
<Tables>
<Table format="png">
<MediaNo>1</MediaNo>
<MediaID>1</MediaID>
<Caption><Pgraph><Mark1>Table 1: Components of the exchange system with the focus of this study marked in grey</Mark1></Pgraph></Caption>
</Table>
<Table format="png">
<MediaNo>2</MediaNo>
<MediaID>2</MediaID>
<Caption><Pgraph><Mark1>Table 2: Data uploaded onto the technical platform</Mark1></Pgraph></Caption>
</Table>
<Table format="png">
<MediaNo>3</MediaNo>
<MediaID>3</MediaID>
<Caption><Pgraph><Mark1>Table 3: Categories and associated questions</Mark1></Pgraph></Caption>
</Table>
<Table format="png">
<MediaNo>4</MediaNo>
<MediaID>4</MediaID>
<Caption><Pgraph><Mark1>Table 4: Reported comprehensibility of the diagrams and scores displayed on the web-based dashboard</Mark1></Pgraph></Caption>
</Table>
<Table format="png">
<MediaNo>5</MediaNo>
<MediaID>5</MediaID>
<Caption><Pgraph><Mark1>Table 5: Reported use of the benchmark</Mark1></Pgraph></Caption>
</Table>
<Table format="png">
<MediaNo>6</MediaNo>
<MediaID>6</MediaID>
<Caption><Pgraph><Mark1>Table 6: Cronbach’s alpha for hygiene factors and motivators and median and interquartile range for each category</Mark1></Pgraph></Caption>
</Table>
<NoOfTables>6</NoOfTables>
</Tables>
<Figures>
<Figure format="png" height="626" width="1042">
<MediaNo>1</MediaNo>
<MediaID>1</MediaID>
<Caption><Pgraph><Mark1>Figure 1: Evolution of the exchange system</Mark1></Pgraph></Caption>
</Figure>
<Figure format="png" height="891" width="1084">
<MediaNo>5</MediaNo>
<MediaID>5</MediaID>
<Caption><Pgraph><Mark1>Figure 5: Visualisation of the Workflow Composite Score, process scores and user satisfaction score</Mark1></Pgraph></Caption>
</Figure>
<Figure format="png" height="934" width="1078">
<MediaNo>6</MediaNo>
<MediaID>6</MediaID>
<Caption><Pgraph><Mark1>Figure 6: Process sub-score, descriptor sub-scores and user satisfaction in admission</Mark1></Pgraph></Caption>
</Figure>
<Figure format="png" height="903" width="1081">
<MediaNo>7</MediaNo>
<MediaID>7</MediaID>
<Caption><Pgraph><Mark1>Figure 7: Descriptor score and indicator visualisation</Mark1></Pgraph></Caption>
</Figure>
<Figure format="png" height="331" width="791">
<MediaNo>2</MediaNo>
<MediaID>2</MediaID>
<Caption><Pgraph><Mark1>Figure 2: Mapping of the six components to the four-tier architecture</Mark1></Pgraph></Caption>
</Figure>
<Figure format="png" height="536" width="916">
<MediaNo>3</MediaNo>
<MediaID>3</MediaID>
<Caption><Pgraph><Mark1>Figure 3: High level overview of the various used software, frameworks, libraries, and file formats in each tier</Mark1></Pgraph></Caption>
</Figure>
<Figure format="png" height="662" width="1230">
<MediaNo>4</MediaNo>
<MediaID>4</MediaID>
<Caption><Pgraph><Mark1>Figure 4: Sequence diagram of the dashboard requesting the data and displaying the results</Mark1></Pgraph></Caption>
</Figure>
<Figure format="png" height="261" width="486">
<MediaNo>8</MediaNo>
<MediaID>8</MediaID>
<Caption><Pgraph><Mark1>Figure 8: Boxplots with median separated by hygiene factors and motivators of the web-based dashboard (Five-point Likert scale with 1 = “completely disagree” and 5 = “completely agree”)</Mark1></Pgraph></Caption>
</Figure>
<Figure format="png" height="572" width="807">
<MediaNo>9</MediaNo>
<MediaID>9</MediaID>
<Caption><Pgraph><Mark1>Figure 9: Continuous science-practice dialogue: A – provision of data by practice, B – aggregation of data by science/transformation into information, C – enrichment of information by scientific knowledge and field experience to develop new knowledge</Mark1></Pgraph></Caption>
</Figure>
<NoOfPictures>9</NoOfPictures>
</Figures>
<InlineFigures>
<NoOfPictures>0</NoOfPictures>
</InlineFigures>
<Attachments>
<NoOfAttachments>0</NoOfAttachments>
</Attachments>
</Media>
</OrigData>
</GmsArticle>]]></plaintext></textgroup></pgraph></textblock></origdata></gmsarticle>