Comparison of Key Evaluation Models in Health Informatics Create graphic representations of four evaluation models. Use MS Word to create your graphic. You

Click here to Order a Custom answer to this Question from our writers. It’s fast and plagiarism-free.

Create graphic representations of four evaluation models. Use MS Word to create your graphic. Your graphic should include:

Descriptions of each overall model
Key components of each overall model
An important figure or figures in the development of each model
Significant ways each model has been used
Potential uses of each model in health informatics

These representations will be for your use in your upcoming course project, so the greater the detail, the more useful these representations will be to you.

Potential formats could include but are not limited to, tables, mind maps, Venn diagrams, or concept maps.

References attached.

HANDf~OOk:[ OF EVALUATION PII~TPlODS

11 Overview of Assessment
Methods

In principle all aspects o f a system are candidates for assessment in all phases o f
the system’s development. In practice, some aspects are more prominent in some
o f the phases than in others. During its life cycle, the assessment m a y change in
nature from being prognostic (during planning), to screening and diagnosing
(prior to switching over to daily operation), to treating (in the handling o f known
error situations or shortcomings). Be aware, therefore, that even if a method is not
listed under a specific phase, an information need may arise that requires
inspiration from the methods listed under other phases.

Note that few o f the references given include a discussion o f the weaknesses,
perils, and pitfalls o f the method described.

6.1 Overview of Assessment
Methods” Explorative Phase

The methods included in this section are particularly relevant to the
assessment o f issues raised during the establishment o f a User Re-
quirements Specification, such as objectives, requirements, and expectations.

Method

Analysis of Work
Procedures

Assessment of
Bids

Balanced
Scorecard

BIKVA

Areas of application

Elucidation of how things are actually carried out
within an organization.

Comparative assessment of a number of offers from
one or more bidders/vendors.

, . .

Ongoing optimization of the outcome of a
development project by balancing focus areas by
means of a set of indicators for a set of strategic
objectives.

. .

Critical, subjective assessment of an existing practice.

Page no

Brender, McNair, Jytte, and Jytte Brender. Handbook of Evaluation Methods for Health Informatics, Elsevier Science &
Technology, 2006. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/waldenu/detail.action?docID=306691.
Created from waldenu on 2022-03-04 00:20:40.

C
o
p
yr

ig
h
t
©

2
0
0
6
.
E

ls
e
vi

e
r

S
ci

e
n
ce

&
T

e
ch

n
o
lo

g
y.

A
ll

ri
g
h
ts

r
e

se
rv

e
d
.

IdANDBOOI< OF: EVALUATION M{]TIdODS

Delphi

FieM Study

Focus Group
Interview

Future Workshop

Grounded Theory

Heuristic
Evaluation

Interview
(no ns tandardiz ed)

KUBI

Logical
Framework
Approach

Organizational
Readiness

�9 (Qualitative) assessment o f an e f f e c t – for
instance, where the solution space is otherwise
too big to handle

�9 Exploration o f development trends
�9 Elucidation o f a problem a r e a – for instance,

prior to strategic planning.

Observation o f an organization to identify its practice
and to clarify mechanisms controlling change.

This is in principle used for the same purposes as
other interview methods. In practice, the method is
most relevant during the early Explorative P h a s e – for
instance, where attitudes or problems o f social groups
need elucidation or when a model solution is being
established.

Evaluation and analysis o f an (existing) situation in
order to identify and focus on areas for change – that
is, aiming at designing future practices.

Supportive analytical method for data acquisition
methods that generate textual data, such as some open
questionnaire methods and interviews (individual and
group interviews).

This is used when no other realizable possibilities
e x i s t – for instance, when:

�9 The organization does not have the necessary
time or expertise

�9 There are no formalized methods
�9 There is nothing tangible to assess yet.

This is particularly suited for elucidation o f
individuals’ opinions, attitudes, and perceptions
regarding phenomena and observations.

Optimization of the outcome o f a long-term
development project, based on a set o f user or
customer/client-defined value norms and objectives.

Situation analysis to support the choice o f focus for a
development but at the same time a simple technique
for incorporation o f risk handling within project
planning.

Assessment o f the readiness o f a healthcare
organization for a clinical information system.

106

111

116

125

128

132

142

147

149

154

C
o
p
yr

ig
h
t
©

2
0
0
6
.
E

ls
e
vi

e
r

S
ci

e
n
ce

&
T

e
ch

n
o
lo

g
y.

A
ll

ri
g
h
ts

r
e

se
rv

e
d
.

NANDi~OOK O~ EVALUATION I”‘]gTBtODS

Pardizipp

Questionnaire
(no ns tandardiz ed)

Requirements
Assessment

Risk Assessment

Social Network
Analysis

Stakeholder
Analysis

SWOT

Usability

Videorecording

WHO: Framework
for Assessment of
Strategies

Preparation of future scenarios.

Questionnaires are used to answer a wide range of
questions, but its main area of application is
(qualitative) investigations of subjective aspects
requiting a high level of accuracy.

Within the European culture the User Requirements
Specification is the basis for purchasing an IT-based
solution or engaging in a development project.
Consequently, the User Requirements Specification is
a highly significant legal document that needs
thorough assessment.

Identification and subsequent monitoring of risk
factors, making it possible to take preemptive action.

Assessment of relations between elements within an
organization (such as individuals, professions,
departments or other organizations), which influence
the acceptance and use of an IT-based solution.

Assessment of stakeholder features and their inner
dynamics, aiming to identify participants for the
completion of a given task, problem-solving activity,
or project.

Situation analysis: establishment of a holistic view of
a situation or a model solution.

Assessment of user friendliness in terms of ergonomic
and cognitive aspects of the interaction (dialogue)
between an IT system and its users. In this phase the
concern is a planning or purchasing situation.

Monitoring and documenting as a means of analysis
of what/how the work procedures or the users’
activities are actually carried out or for investigation
of complex patterns of interaction.

Assessment of different (development) strategies
either individually or as a comparative analysis.

156

163

180

185

190

192

196

207

219

222

C
o
p
yr

ig
h
t
©

2
0
0
6
.
E

ls
e
vi

e
r

S
ci

e
n
ce

&
T

e
ch

n
o
lo

g
y.

A
ll

ri
g
h
ts

r
e

se
rv

e
d
.

ldANDSOOI< O5 EVALUATION M~TIdODS

6.2 Overview of Assessment Methods”
Technical Development Phase

The m e t h o d s listed in this section are particularly suited to user activities during
the d e v e l o p m e n t and installation o f an IT-based solution and m a y be used to
provide feed-back for the technical development.

A s s e s s m e n t in this phase is typically carried out under experimental
conditions and not during real operation. The phase is usually
completed with a technical verification to make certain that all
necessary functions and features are present and work properly in
compliance with the established agreement.

Method

Balanced Scorecard

Clinical~Diagnostic
Performance

Cognitive
Assessment

Cognitive
Walkthrough

Areas of application

Ongoing optimization of the outcome of a
development project by balancing focus areas by
means of a set of indicators for a set of strategic
objectives.

Measurement of diagnostic ‘correctness’ (for instance,
measures of accuracy and precision) of IT-based
expert systems and decision-support systems. ‘

Assessment of cognitive aspects of the interaction
between an IT system and its u s e r s – for instance:

�9 Identification of where and why operational
errors occur

�9 Identification of areas to be focused on for
improvement in user friendliness.

Assessment of user ‘friendliness’ on the basis of
system design, from specifications, muck-ups, or
prototypes, aimed at judging how well the system
complies with the users’ way of thinking for
instance:

�9 Identification of where and why operational
errors occur

�9 Identification of causes behind problems with
respect to user friendliness and consequently
identification of areas for improvement.

Page no

102

C
o
p
yr

ig
h
t
©

2
0
0
6
.
E

ls
e
vi

e
r

S
ci

e
n
ce

&
T

e
ch

n
o
lo

g
y.

A
ll

ri
g
h
ts

r
e

se
rv

e
d
.

I-IANID~OOI< O~ ~VALUATION IVIETIdODS

Heuristic
Evaluation

Risk Assessment

SWOT

Technical
Verification

Think Aloud

Usability

This is used when no other realizable possibilities
e x i s t – for instance, when:

�9 The organization does not have the necessary
time or expertise

�9 There are no formalized methods
�9 There is not something tangible to assess yet.

Identification and subsequent monitoring of risk
factors, making it possible to take preemptive action.

Situation analysis: establishment of a holistic view of
a situation or a model solution.

Verification that the agreed functions are present, and
work correctly and in compliance with the agreement.
This may take place, for instance, in connection with
delivery of an IT system or prior to daily operations
and at any subsequent change of the IT system
(releases, versions, and patches).

An instrument for gaining insight into the cognitive
processes as feed-back to the implementation and
adaptation of IT-based systems.

Assessment of user friendliness in terms of ergonomic
and cognitive aspects of the interaction (dialogue)
between an IT system and its users.

132

185

196

199

204

207

6.3 Overview of Assessment Methods”
Adaptation Phase

In this phase, evaluation has the purpose o f
providing support for the modification or
refinement o f the IT-based solution, work
procedures, and functions implemented within the
IT system to make them work optimally as a
whole during daily operations. This phase should
be fairly short, provided that the implemented
solution is functioning well from the beginning.

C
o
p
yr

ig
h
t
©

2
0
0
6
.
E

ls
e
vi

e
r

S
ci

e
n
ce

&
T

e
ch

n
o
lo

g
y.

A
ll

ri
g
h
ts

r
e

se
rv

e
d
.

ldANDBOOI( OlZ EVALUATION IvlI~TI-IODS

N o w that real operational a s s e s s m e n t can take place, e r g o n o m i c , cognitive, and
functionality a s s e s s m e n t will gain m u c h m o r e focus, as potential i n a d e q u a c i e s or
s h o r t c o m i n g s will s h o w t h e m s e l v e s as operational errors, misuse, or the like.

Method

Analysis of Work
Procedures

BIKVA

Clinical~Diagnostic
Performance

Cognitive
Assessment

Cognitive
Walkthrough

Equity
Implementation
Model

Field Study

Focus Group
Interview

Areas of appfication

Elucidation of how things are actually carried out, in
comparison with the expected. This includes the
actual use of the IT system in relation to its
anticipated use.

Critical, subjective assessment of an existing practice.

Measurement of diagnostic ‘ correctness’ (for instance,
measures of accuracy and precision) in IT-based
expert systems and decision-support systems.

Assessment of cognitive aspects of the interaction
between an IT system and its u s e r s – for instance:

�9 Identification of where and why operational
errors occur

�9 Identification of areas to be focused on for
improvement in user friendliness.

�9 Identification of where and why operational
errors occur

�9 Identification of causes behind problems with
respect to user friendliness and consequently
identification of areas for improvement.

Examine users’ reaction to the implementation of a
new system, focusing on the impact of the changes
such a system brings about for the users.

Observation of an organization to identify its practices
and to expose mechanisms that control change.

This is in principle used for the same purposes as
other interview methods. In practice, the method is
most relevant during the early Explorative P h a s e – for
instance, where the attitudes or problems of social
groups need elucidation or when a model solution is
being established.

Page no

102

109

111

116

C
o
p
yr

ig
h
t
©

2
0
0
6
.
E

ls
e
vi

e
r

S
ci

e
n
ce

&
T

e
ch

n
o
lo

g
y.

A
ll

ri
g
h
ts

r
e

se
rv

e
d
.

I-IANDBOOI< OF– ~VALUATION METHODS

Functionality
Assessment

Grounded Theory

Heuristic
Evaluation

Interview
(nonstandardized)

Prospective Time
Series

Questionnaire
(nonstandardized)

RCT, Randomized
Controlled Trial

Risk Assessment

Root Causes
Analysis

1. Validation of fulfillment of objectives (realization
of objectives) – that is, the degree of compliance
between the desired effect and the actual solution

2. Impact Assessment (also called effect assessment)
3. Identification of problems in the relationship

between work procedures and the IT system’s
functional solution

The method will expose severe ergonomic and
cognitive problems, but it is not dedicated to capture
details of this type.

Supportive analytical method for data acquisition
methods that generate textual data, such as some open
questionnaire methods and interviews (individual and
group interviews).

This is used when no other realizable possibilities
e x i s t – for instance, when:

�9 The organization does not have the necessary
time or expertise

�9 There are no formalized methods
�9 There is not something tangible to assess yet.

Is in particular suited for the elucidation of individual
opinions, attitudes, and perceptions regarding
phenomena and observations.

Measurement of development trends, including the
effect of an intervention.

Questionnaires are used to answer a wide range of
questions, but its main area of application is
(qualitative) investigations of subjective aspects
requiting a high level of accuracy.

Verification of efficacy – that is, that the IT system –
under ideal conditions – makes a difference to patient
care. Particularly used in studies of decision-support
systems and expert systems.

Identification and subsequent monitoring of risk
factors, making it possible to take preemptive action.

Exploration of what, how, and why a given incident
occurred to identify the root cause of undesirable
events.

120

128

132

142

159

163

172

185

188

C
o
p
yr

ig
h
t
©

2
0
0
6
.
E

ls
e
vi

e
r

S
ci

e
n
ce

&
T

e
ch

n
o
lo

g
y.

A
ll

ri
g
h
ts

r
e

se
rv

e
d
.

HANDBOOI( OiZ EVALUATION METHODS

Social Network
Analysis

SWOT

Technical
Verification

Think Aloud

Usability

User Acceptance
and Satisfaction

Videorecording

Assessment of relations between elements within an
organization (such as individuals, professions,
departments, or other organizations), which influence
the acceptance and use of an IT-based solution.

Situation analysis: establishing a holistic view of a
situation or a model solution.

An instrument for gaining insight into the cognitive
processes as feed-back to the implementation and
adaptation of IT-based systems.

Assessment of user friendliness in terms of ergonomic
and cognitive aspects of the interaction (dialogue)
between an IT system and its users.

Assessment of user opinion, attitudes, and perception
of an IT system during daily operation.

Monitoring and documenting as a means of analyzing
how work procedures and user activities, respectively,
are actually carried out or for investigation of complex
patterns of interaction.

190

196

199

204

207

215

219

6.4 Overview of Assessment
Methods” Evolution
Phase

The starting point in time o f this phase is usually
considered to be when the entire IT-based solution
has reached a state o f sufficient stability with respect
to bugs and corrections and w h e n evolutionary
activities are started. Consequently, the shi~
be twe e n this and the previous phase m a y be fluid.

C
o
p
yr

ig
h
t
©

2
0
0
6
.
E

ls
e
vi

e
r

S
ci

e
n
ce

&
T

e
ch

n
o
lo

g
y.

A
ll

ri
g
h
ts

r
e

se
rv

e
d
.

I-IANDE~OOI< (3[= EVALUATION METHODS

Method

Analysis of Work
Procedures

Balanced Scorecard

BIKVA

Clin&al/Diagnostic
Performance

Cognitive
Assessment

Cognitive
Walkthrough

Delphi

Equity
Implementation
Model

FieM Study

Areas of application

Elucidation of how things are actually carried out, in
comparison with the expected. This includes its use in
relation to measures of effect.

Ongoing optimization of the outcome of a
development project by balancing focus areas by
means of a set of indicators for a set of strategic
objectives.

Critical, subjective assessment of an existing practice.

Measurement of diagnostic ‘ correctness’ (for instance,
measures of accuracy and precision) of IT-based
expert systems and decision-support systems.

Assessment of the cognitive aspects of the interaction
between an IT system and its u s e r s – for instance:

�9 Identification of where and why operational
errors occur

�9 Identification of areas to be focused on for
improvement in user friendliness.

Assessment of the user ‘friendliness’ on the basis of
system design, from specifications, muck-ups, or
prototypes of the system, aimed at judging how well
the system complies with the users’ way of t h i n k i n g –
for instance:

�9 Identification of where and why operational
errors occur

�9 Identification of causes behind problems with
respect to user friendliness and consequently
identification of areas for improvement.

1. (Qualitative) assessment of an e f f e c t – for
instance, where the solution space is otherwise
too big to handle

2. Exploration of development trends
3. Elucidation of a problem a r e a – for instance, prior

to strategic planning.

Examine users’ reaction to the implementation of a
new system, focusing on the impact of the changes
such a system brings about for the users.

Observation of an organization to identify its practices
and to expose mechanisms that control change.

Page no

102

106

109

111

C
o
p
yr

ig
h
t
©

2
0
0
6
.
E

ls
e
vi

e
r

S
ci

e
n
ce

&
T

e
ch

n
o
lo

g
y.

A
ll

ri
g
h
ts

r
e

se
rv

e
d
.

HANDE~OOI< O.l::Z EVALUATION I”~I~TIdODS

Focus Group
Interview

Functionality
Assessment

Grounded Theory

Heuristic
Evaluation

Impact Assessment

Interview
(nonstandardized)

KUBI

Prospective Time
Series

It is in principle used for the same purposes as other
interview methods. In practice, the method is most
relevant during the early analysis s t a g e – for instance,
where attitudes or problems o f social groups need
clarification or elucidation or when a model solution
is being established.

1. Validation o f fulfillment o f objectives (realization
o f objectives) – that is, the degree o f compliance
between the desired effect and the actual solution

2. Impact Assessment (also called effect assessment)
3. Identification o f problems in the relationship

between work procedures and the IT system’s
functional solution

The method will expose severe ergonomic and
cognitive problems, but it is not dedicated to capture
details o f this type.

Supportive analytical method for data acquisition
methods that generate textual data, such as some open
questionnaire methods and interviews (individual and
group interviews).

This is used when no other realizable possibilities
e x i s t – for instance, when:

�9 The organization does not have the necessary
time or expertise

�9 There are no formalized methods
�9 There is not something tangible to assess yet.

Measurement o f the e f f e c t – that is, the consequence
or impact in its broadest sense – o f an IT-based
solution, with or without the original objective as a
frame o f reference.

This is in particular suited for elucidation o f
individual opinions, attitudes, and perceptions
regarding phenomena and observations.

Optimization o f the outcome o f a long-term
development project, based on a set o f user or
customer/client defined value norms and objectives.

Measurement o f development trends, including the
effect o f an intervention.

116

120

128

132

135

142

147

159

C
o
p
yr

ig
h
t
©

2
0
0
6
.
E

ls
e
vi

e
r

S
ci

e
n
ce

&
T

e
ch

n
o
lo

g
y.

A
ll

ri
g
h
ts

r
e

se
rv

e
d
.

HANDE~OOk( O4Z ~VALUATION NI~TI4ODS

Questionnaire
(nonstandardized)

RCT, Randomized
Controlled Trial

Risk Assessment

Root Causes
Analysis

Social Network
Analysis

Stakeholder
Analysis

SWOT

Technical
Verification

Think Aloud

Usability

User Acceptance
and Satisfaction

Questionnaires are used to answer a wide range of
questions, but its main area of application is
(qualitative) investigations of subjective aspects
requiting a high level of accuracy.

Verification of efficacy – that is, that the IT s y s t e m –
under ideal conditions – makes a difference to patient
care. In particular used for studies of decision-support
systems and expert systems.

Identification and subsequent monitoring of risk
factors, making it possible to take preemptive action.

Exploration of what, how, and why a given incident
occurred to identify the root cause of undesirable
events.

Assessment of stakeholder features and their inner
dynamics, aiming to identify participants for the
completion of a given task, problem-solving activity,
or project.

Situation analysis: establishment of a holistic view of
a situation or a model solution.

Verification that the agreed functions are present,
work correctly, and are in compliance with the
agreement. This may take place, for instance, in
connection with delivery of an IT system or prior to
daily operations, and at any subsequent change of the
IT system (releases, versions, and patches).

An instrument for gaining insight into the cognitive
processes as feed-back to the implementation and
adaptation of IT-based systems.

Assessment of user friendliness in terms of ergonomic
and cognitive aspects of the interaction (dialogue)
between an IT system and its users.

Assessment of users’ opinion, attitudes, and
perception of an IT system at daily operation.

163

172

185

188

190

192

196

199

204