Tài liệu bài tập môn Thương mại điện tử | Trường Đại học Tài nguyên và Môi trường Hà Nội
Tài liệu bài tập môn Thương mại điện tử | Trường Đại học Tài nguyên và Môi trường Hà Nội được sưu tầm và soạn thảo dưới dạng file PDF để gửi tới các bạn sinh viên cùng tham khảo, ôn tập đầy đủ kiến thức, chuẩn bị cho các buổi học thật tốt. Mời bạn đọc đón xem!
Môn: Thương mại điện tử (TMĐT1)
Trường: Đại học Tài nguyên và Môi trường Hà Nội
Thông tin:
Tác giả:
Tài liệu liên quan:
Preview text:
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/235298489
Factors influencing the effectiveness of performance measurement systems
Article in International Journal of Operations & Production Management · November 2011 DOI: 10.1108/01443571111187457 CITATIONS READS 132 4,140 3 authors, including: Kevin Baird Macquarie University
68 PUBLICATIONS 2,071 CITATIONS SEE PROFILE
All content following this page was uploaded by Kevin Baird on 29 March 2021.
The user has requested enhancement of the downloaded file.
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/0144-3577.htm Factors influencing the The effectiveness of PMSs effectiveness of performance measurement systems
Amy Tung, Kevin Baird and Herbert P. Schoch 1287
Department of Accounting and Corporate Governance, Received June 2010
Macquarie University, Sydney, Australia Revised November 2010 Accepted April 2011 Abstract
Purpose – The purpose of this paper is to examine the association between the use of
multidimensional performance measures and four organizational factors with the effectiveness of
performance measurement systems (PMSs).
Design/methodology/approach – Data were collected by mail survey questionnaire from a random
sample of 455 senior financial officers in Australian manufacturing organizations.
Findings – The results reveal that the use of multidimensional performance measures is associated
with two dimensions of the effectiveness of PMSs (performance and staff related outcomes). The
results also reveal that organizational factors were associated with the effectiveness of PMSs.
Specifically, top management support was found to be associated with the effectiveness of PMSs in
respect to the performance related outcomes, and training was associated with the staff related outcomes.
Practical implications – The findings provide managers with an insight into the desirable PMS
characteristics and the specific organizational factors that they can focus on in order to enhance the
effectiveness of their performance measurement system.
Originality/value – This study contributes to the limited empirical research examining the
effectiveness of PMSs regarding the extent to which organizational processes are achieved. In
addition, the study provides an empirical analysis of the association between the five perspective
(financial, customer, internal business process, learning and growth, and sustainability) BSC model
and four organizational factors with the effectiveness of PMSs.
Keywords Australia, Manufacturing industries, Performance measures,
Performance measurement system, Multidimensional performance measures, Top management
support, Training, Employee participation, Link of performance to rewards Paper type Research paper 1. Introduction
To survive in today’s rapidly changing environment, organizations must identify their
existing positions, clarify their goals, and operate more effectively and efficiently.
Performance measurement systems (PMSs) assist organizations in achieving such
objectives. Neely et al. (1995, p. 81) defines a PMS as “a set of metrics used to
quantify both the efficiency and effectiveness of actions”. An effective PMS enables
an organization to assess whether goals are being achieved, and facilitates the
improvement of the organization as a whole (Lebas, 1995) by identifying their
position, clarifying goals, highlighting areas requiring improvement, and facilitating
International Journal of Operations
reliable forecasts (Neely et al., 1996). Hence, an effective PMS enables an & Production Management
organization to measure and control its performance in line with the defined strategy. Vol. 31 No. 12, 2011 pp. 1287-1310
While the recent PMS literature has focused on the shift from traditional PMSs, which q Emerald Group Publishing Limited 0144-3577
focus on financial measures, to multidimensional PMSs such as the performance DOI 10.1108/01443571111187457 IJOPM
pyramid (Lynch and Cross, 1991), the balanced scorecard (BSC) (Kaplan and Norton, 1992), 31,12
and the performance prism system (Neely and Adams, 2000), there is limited empirical
evidence examining the effectiveness of such PMSs. Furthermore, the majority of these
studies assess PMS effectiveness in relation to overall organizational performance (Crabtree
and DeBusk, 2008; Braam and Nijssen, 2004; Davis and Albright, 2004; Ittner et al., 2003;
Hoque and James, 2000), thereby assuming a direct association between the PMS and 1288
performance. This approach is inconsistent with Hamilton and Chervany’s (1981) claim that
the impact of the PMS on performance is indirectly influenced by the effect on improvements
in organizational processes. In other words, organizational objectives such as sales revenue,
profit contribution and customer satisfaction will not be realized unless specific organizational
objectives (e.g. motivating performance, developing individual’s skills and knowledge,
providing useful feedback to employees, and providing an accurate assessment of business
unit performance) are achieved. Accordingly, the first objective of this study is to contribute to
the limited empirical research (Malina and Selto, 2001; Whorter, 2003) examining the
effectiveness of PMSs based on the extent to which organizational processes are achieved.
The measurement of performance is an on-going task, hence, in order to achieve system
effectiveness, organizations need to devote time and effort to managing the system (Neely et
al., 2000). Hence, in an attempt to provide practitioners with an insight into how to achieve
and maintain effectiveness, the second objective of the study is to contribute to the
contingency literature by examining the factors associated with the effectiveness of PMSs. The
first factor examined, the use of multidimensional performance measures, has been advocated
by both academics and practitioners in order to complement the limitations of traditional
financial PMSs and to increase the effectiveness of PMSs (Van der Stede et al., 2006; Kaplan
and Norton, 2001, 1996, 1992). While many multidimensional frameworks have been
advocated, and the benefits of using multidimensional performance measures have received
wide publicity in the literature (Van der Stede et al., 2006; Bryant et al., 2004), there is
considerable variation in the adoption rates reported for the most common multidimensional
approach, the BSC (Rigby and Bilodeau, 2009 (53 percent); Chung et al., 2006 (31 percent);
Ittner et al., 2003 (20 percent); Speckbacher et al., 2003 (26 percent)). The variation in the
adoption of multidimensional performance measures raises concerns regarding the
contribution of such measures towards the effectiveness of PMSs. Accordingly, this study aims
to contribute to the literature by examining the association between the use of
multidimensional performance measures and the effectiveness of PMSs.
The study also aims to provide an empirical analysis of the association between
specific organizational factors (top management support, training, employee participation
and the link of performance to rewards) with the effectiveness of PMSs.
While these organizational factors do not represent a comprehensive list of all relevant
factors, they were chosen for two reasons. First, they have been widely cited as factors
contributing to the success of various management accounting practices such as activity-based
costing (ABC) (Baird et al., 2007; Shields, 1995), enterprise resource planning (Motwani et
al., 2002; Rao, 2000), and management information system (MIS) (Raghunathan et al., 1999;
Doll, 1985; Schultz and Ginzberg, 1984). Second, while they have been identified in previous
studies as the main contingency factors associated with the effectiveness of PMSs (Burney et
al., 2009; Hoque and Adams, 2008; Cheng et al., 2007; Kleingeld et al., 2004; Chan, 2004),
this was in isolation, and no study has analysed
all four factors together. Hence, this study is motivated to fill this gap in the literature The effectiveness
by examining the link between all four organizational factors and the effectiveness of of PMSs
PMSs within Australian manufacturing organizations.
In addition, given the majority of previous studies examining the influence of
organizational factors on PMS effectiveness have used the case study approach
(Kleingeld et al., 2004; Bourne et al., 2002; Emerson, 2002; Kennerley and Neely,
2002; Kaplan, 2001), there is a gap in the literature empirically examining this 1289
association. Hence, the current study is motivated to fill this gap by using the survey
method in an attempt to enhance the generalizability of the findings.
The remainder of this paper is structured as follows. Section 2 provides the
literature review and develops the relevant hypotheses. Sections 3 and 4 then discuss
the method and results. Finally, Section 5 provides the conclusion, limitations, and
future directions for research. 2. Literature review
2.1 Performance measurement systems
PMSs have become a field of interest over the last two decades with many studies
discussing various aspects of performance measurement such as: the purpose and
usage (Marchand and Raymond, 2008; Horngren et al., 2005; Simons, 2000), design
(Bhasin, 2008; Kennerley and Neely, 2002; Neely and Adams, 2000; Kaplan and
Norton, 1996, 1992; Lynch and Cross, 1991), and implementation (Ratnasingam,
2009; Othman, 2008; Speckbacher et al., 2003; Kaplan, 2001).
An effective PMS, which is defined as the achievement of the objectives set for a task
(Clinquini and Mitchell, 2005), is important for a number of reasons. First, it can
encourage goal congruence. For example, an appropriate PMS can be used to
communicate the strategy and goals of an organization and align employees’ goals with
organizational goals. Second, an effective PMS can provide accurate information to enable
managers to track their own performance and evaluate employees’ performance in an
effective and efficient manner. Finally, an effective PMS can provide organizations with an
indication of their current market position and assist them in developing future strategies
and operations (Langfield-Smith et al., 2009). This study operationalises an effective
PMS as the extent to which 16 desired PMS outcomes are achieved.
Traditionally, PMSs have focused mainly on financial measures such as profit, cash
flow and return on investment to evaluate the performance of employees (Chan, 2004).
This focus has a number of shortcomings. First, these outcome-oriented measures do not
allow managers to assess how well employees perform across the full range of
strategically important areas, such as quality and service delivery. Second, traditional
financial measures describe consequences rather than causes, hence they are not
actionable. Such measures provide limited guidance for future actions since they do not
tell managers what needs to be fixed (Langfield-Smith et al., 2009). Third, the focus on
aggregate financial outcomes may encourage managers to engage in “gaming” behavior to
maximize short-term results at the expense of long-term effectiveness (Chow and Van der
Stede, 2006). Finally, traditional financial measures can conflict with strategy and they are
not externally focused (Chow and Van der Stede, 2006; Kaplan and Norton, 1996).
The limitations of traditional PMSs, together with intense competitive pressures
and changing external demands, have led to the increased advocacy of non-financial
measures (Neely, 1999). Such contemporary PMSs have been espoused by both IJOPM
academics and practitioners in order to address the limitations of traditional financial 31,12
performance measures and to assist organizations to build competitive advantage
under changing economic conditions (Kaplan and Norton, 2006, 2004, 2001, 1996, 1992).
The common characteristics of contemporary systems include the linking of strategies,
objectives and measures, and the incorporation of both financial and non-financial
measures that cover a range of perspectives (Langfield-Smith et al., 2009). Since the 1290
BSC is the most recognized and utilized contemporary PMS (Rigby and Bilodeau, 2009;
Chang et al., 2008; Jusoh et al., 2008; Bedford et al., 2006; Pike and Roos, 2004;
Atkinson et al., 1997), it is used in this study to exemplify the use of multidimensional performance measures. 2.2 The BSC
The first-generation BSC was mainly a PMS which proposed a specific structure to
measure tangibles and intangibles (Speckbacher et al., 2003; Kaplan and Norton,
1992). The framework complemented the financial perspective measures with non-
financial operational measures emphasizing three other perspectives: customer
satisfaction, internal processes and learning and growth. It provided a more balanced
view of organizational performance by capturing both leading (e.g. customer
satisfaction, on-time delivery, employee training, etc.) and lagging (e.g. sales revenue,
ROI, cash flows, etc.) performance measures (Kaplan and Norton, 1996, 1992).
In 1996, Kaplan and Norton advocated the causal links between the perspectives
included within the BSC. The refined model communicated the organization’s desired
outcomes and hypothesized the means by which the desired outcomes could be achieved.
For instance, if organizations trained their employees well, then the quality of service
would be improved as well as customer satisfaction; if customer satisfaction improved,
then customers would purchase more, thereby improving the overall profitability of the
organization. Hence, the second-generation BSC was proposed as a multidimensional
PMS which describes strategy through cause and effect relationships (Speckbacher et al.,
2003; Kaplan and Norton, 1996). It enabled organizational units and employees to
understand the strategy and identify how they can contribute to its achievement by
becoming aligned with the strategy. Consequently, today’s BSC has become a strategic
management system that implements strategy through communication, action plans and
incentives (Speckbacher et al., 2003; Kaplan and Norton, 2001).
As a further development, the BSC included additional perspectives (Kaplan and
Wisner, 2009; Kaplan and Norton, 2006, 2004, 2001). With sustainability becoming a
major concern for various stakeholders (e.g. customers, investors, and the
government) and affecting the organizational “bottom line”, a sustainability BSC was
subsequently advocated (Langfield-Smith et al., 2009; Epstein, 2008; Figge et al.,
2002). Epstein (2008) suggested that the inclusion of the sustainability perspective is
appropriate where sustainability is considered a part of the business core strategy and
important to creating competitive advantage. To provide a more comprehensive
account of the use of multidimensional performance measures, this study adopts the
five perspective (financial, customer, internal business process, learning and growth, and sustainability) BSC model.
2.2.1 Adoption and use of the BSC. Silk (1998) estimated that 60 percent of the
Fortune 1000 companies in the USA have had experience with a BSC. In the UK, 57
percent of businesses were reported to use a BSC and 53 percent of non-users
were discussing possible implementation. In contrast, Speckbacher et al. (2003) reported The effectiveness
that more than 60 percent of the companies in their study had not considered the BSC. of PMSs
Similarly, Ittner et al. (2003) indicated that only 20 percent of the firms in their study used
a BSC, while 50 percent of the firms had not even considered implementing it.
Use of the BSC however does not guarantee satisfaction with De Geuser et al. (2009)
referring to the literature highlighting the gap between the use of the BSC and evidence of
its effectiveness (Davis and Albright, 2004; Norreklit, 2003; Speckbacher et al., 2003; 1291
Otley, 1999). Thus, while the Management Tools and Trends Survey (Rigby and Bilodeau,
2009) showed that in 2008, 53 percent of organizations globally used the BSC and by the
end of 2009, the usage rate was expected to reach 69 percent, it was found that 51 percent
of user organizations were not satisfied with their BSC. Similarly, Ittner et al. (2003)
revealed that organizations were only moderately satisfied with the measurement system
with 37.2 percent of respondents rating it as not meeting expectations. Bedford et al.
(2006) also concluded that while respondents agreed that the BSC had helped in achieving
some objectives, the extent to which the proclaimed benefits of the BSC were achieved
was still fairly low. Given the mixed findings with respect to the success of the BSC, this
study investigates the association between the use of multidimensional performance
measures and the effectiveness of PMSs.
2.3 The association between the use of multidimensional performance
measures and the effectiveness of PMSs
Multidimensional PMSs assist organizations by enhancing the likelihood that all
relevant performance dimensions are considered (Ittner et al., 2003). Furthermore,
such systems allow managers to focus on the “means to the end”, while also enabling
them to demonstrate strong performance in a variety of areas (Baird, 2010). Hoque
and Adams (2008) suggest that multidimensional PMSs are capable of providing
signals and motivating improvement in crucial activities. Similarly, Van der Stede et
al. (2006) found that regardless of strategy, organizations with more extensive PMSs,
especially those that included objective and subjective non-financial measures, have
better overall performance. Van der Stede et al. (2006) also demonstrated that non-
financial performance measures are better than financial measures in helping
organizations implement and manage new initiatives.
A growing stream of literature provides evidence that the use of multidimensional
performance measures contributes to the effectiveness of PMSs (Crabtree and
DeBusk, 2008; Braam and Nijssen, 2004; Davis and Albright, 2004; Ittner et al.,
2003; Whorter, 2003; Malina and Selto, 2001; Hoque and James, 2000). Most of these
studies examined the effectiveness of PMSs from the perspective of their contribution
to the company’s financial performance. For example, Davis and Albright (2004)
applied a quasi-experimental study in a US banking organization to investigate the
relationship between BSC implementation and the financial performance of bank
branches. The study supports the theory that the BSC can be used to improve financial
performance, with bank branches that implemented the BSC outperforming other
branches on key financial measures. Similarly, Braam and Nijssen (2004) suggest that
BSC usage, which is aligned to company strategy, positively influences overall company performance.
Ittner et al. (2003) found that while BSC usage was associated with higher
measurement system satisfaction, there was no evidence that BSC usage was related IJOPM
to stock returns. However, Crabtree and DeBusk (2008) extended this study to 31,12
investigate the contribution of the BSC to shareholder returns in different public sector
companies, and found that BSC usage was associated with higher stock returns.
Malina and Selto (2001) and Whorter (2003) assessed the effectiveness of PMSs based
on organizational processes (e.g. communicating strategic objectives, creating strategic
alignment, motivating employees and serving as a management control device) 1292
as opposed to financial performance. Malina and Selto (2001) found that the BSC was an
effective device for evaluating corporate strategy. Their results also show evidence of
casual relations between motivation, strategic alignment and effective management
control with the BSC. Similarly, Whorter (2003) showed that BSC users consistently
reported higher agreement about having the information needed for making the best
work-related decisions. Whorter (2003) also concluded that the BSC not only
provides useful performance feedback to employees but is also an aid in the accurate
assessment of employee performance:
H1. The extent of use of multidimensional performance measures is associated
with the effectiveness of the PMS.
2.4 The association between organizational factors and the effectiveness of
PMSs Prior studies have identified top management support (Hoque and Adams,
2008; Johanson et al., 2006; Bourne, 2005; Chan, 2004; Bourne et al., 2002;
Kennerley and Neely, 2002; Kaplan, 2001), training (Chan, 2004; Emerson, 2002),
employee participation (Hoque and Adams, 2008; Kleingeld et al., 2004), and the
link of performance to rewards (Burney et al., 2009; Chan, 2004) as key
organizational factors associated with the effectiveness of PMSs.
2.4.1 Top management support. Top management support has been highlighted as an
important contingency factor in supporting various management accounting practices such as
ABC (Baird et al., 2007; Shields, 1995) and MISs (Doll, 1985). The impact of top management
support on PMS effectiveness has been referred to in a number of studies (Bourne, 2005; Chan,
2004; Bourne et al., 2002; Emerson, 2002; Kennerley and Neely, 2002). For example, Bourne
et al. (2002) investigated the success of the redesign of PMSs. They found that top
management support was influential in the successful implementation and on-going usage of
the new PMS. This study also indicated that the continuous involvement by top management
was invaluable in resolving problems when crises and conflicts arose. Chan (2004) and
Emerson (2002) also reported that top management commitment and leadership buy in are key
factors in enhancing PMS effectiveness. Similarly, Kennerley and Neely (2002) found that top-
level management support was critical for PMS design and implementation, while the
availability of management time to reflect on measures was a major contributor to the effectiveness of PMSs:
H2. The extent of top management support is associated with the effectiveness of the PMS.
2.4.2 Training. Training is defined as “a planned effort by an organization to facilitate the
learning of job-related behavior” (Wexley, 1984, p. 13). The importance of training in
relation to the development and implementation of a successful PMS is highlighted in a
number of studies. Cavaluzzo and Ittner (2004, p. 249), for example, found that
performance measurement development and outcomes are positively associated with the
extent of related training provided to the manager. The provision of training
resources indicates that an organization is willing to provide sufficient resources to The effectiveness
support the development and implementation of PMSs. of PMSs
Chan (2004) cites training as a crucial factor for PMSs to be effective. All
performance measures need to have a clearly communicated purpose and be perceived
as both relevant and reliable so that managers can access useful information for
decision making. Without training, managers may perceive the PMS measures as less
useful and ignore them when making decisions. Similarly, Emerson (2002) concluded 1293
that training is the key to maintaining the usefulness and the effectiveness of PMSs.
Training not only allows users to understand performance measurement concepts and
principles, but also provides both employees and managers with an opportunity to
operate the system. Hence, the better that users understand the purposes of the system
and how to operationalise it, the more likely they will commit to it, thereby enhancing
the likelihood that the desired results will be achieved:
H3. The extent of PMS-related training provided is associated with the effectiveness of the PMS.
2.4.3 Employee participation. Many studies have referred to the benefits of employee
empowerment (Morrell and Wilkinson, 2002; Koberg et al., 1999; Chiles and Zorn, 1995)
and employee involvement and participation (Cox et al., 2007, 2006; Pun et al., 2001;
Wimalasiri and Kouzmin, 2000). These studies tend to operationalise these concepts in
terms of employees’ involvement in decision making. Similarly, employee participation
refers to the “involvement of managers and their subordinates in information processing,
decision making, or problem solving endeavors” (Wagner, 1994, p. 312). This study
operationalises employee participation in terms of the extent to which lower level
employees participate in designing the PMS.
The association between employee participation and the effectiveness of PMSs has
support from prior studies (Chan, 2004; Kleingeld et al., 2004; Kaplan and Norton,
2001). These studies report that a higher level of employee participation contributed
to the effectiveness of PMSs. For instance, Kleingeld et al. (2004) found that on
average the improvement in performance was significantly greater for those
employees in a high participation situation as opposed to those in a low participation
situation. This performance improvement was attributed to both cognitive
mechanisms (including increased communication, better utilization of knowledge,
increased understanding of the job) and motivational mechanisms (less resistance to
change, commitment to the system, acceptance of feedback and goals).
Similarly, Kaplan and Norton (2001) maintained that in order to achieve an
effective BSC, employees at lower levels in the organizational hierarchy should be
involved in the establishment of performance measures. This bottom-up participation
approach allows employees to take the initiative in defining their responsibilities as
well as the associated performance indicators. Therefore, employees will commit to
the system and desired outcomes can be achieved to a greater extent:
H4. The extent of employee participation in designing the PMS is associated with the effectiveness of the PMS.
2.4.4 The link of performance to rewards. The link of performance to rewards is a
vital contingency factor in motivating employees (Rynes et al., 2005; McShane and
Travaglione, 2003; Bonner and Sprinkle, 2002; PA Consulting Group, 1998). IJOPM
A survey of 500 companies reported that companies that link performance to pay 31,12
showed twice the shareholder returns as those who did not (PA Consulting Group, 1998).
McShane and Travaglione (2003) suggested that companies need to align rewards with
performance that is within the employee’s control. Hence, the more employees see a
“line of sight” between their daily actions and the reward, the more motivated they will be to improve performance. 1294
Linking performance to rewards has also been identified as a crucial factor
influencing the effectiveness of PMSs (Burney et al., 2009; Johanson et al., 2006;
Chan, 2004). For instance, in Chan’s (2004) study of municipal governments in the
USA and Canada, it was found that the linkage of the PMS to compensation was
uncommon, and “the lack of linkage of the BSC to rewards” was considered to be a
barrier to the systems’ effectiveness.
While there is a lack of empirical evidence examining the link of performance to rewards on
the effectiveness of PMSs, given the importance of the link of performance to rewards and the
increasing number of large businesses rewarding both employees and managers based on BSC
performance (Epstein and Manzoni, 1998), H5 is stated as follows:
H5. The extent of the link of performance to rewards is associated with the effectiveness of the PMS. 3. Method
A survey questionnaire was mailed to the senior financial officer of a random sample of
445[1] Australian manufacturing business units identified from the Kompass Australia
(2009) directory[2]. The manufacturing industry was selected as a number of prior studies
on PMSs suggest that manufacturing organizations are more likely to have a mature and
comprehensive PMS in place (Malina and Selto, 2001; Simons, 2000; Kaplan and Norton,
1996, 1992). Business units were chosen as the unit of analysis because PMS
characteristics may differ across business units within an organization. Senior financial
officers were chosen as they were expected to have a sound understanding of their
business unit’s PMS. The Dillman (2007) tailored design method was employed to
administer the survey[3]. In total, 141 responses were received for a response rate of 30.9
percent. In total, 23 of the questionnaires were incomplete, hence 118 questionnaires were
used for the data analysis. As was the case in Robert (1999), non-response bias was
assessed by comparing the independent and dependent variable values across early and
late respondents. No significant differences were detected. 3.1 Variable measurement
3.1.1 The effectiveness of the PMS. The effectiveness of PMSs is measured by
assessing the extent to which 16 desired outcomes of PMSs have been achieved. The
16 measures (the Appendix) were developed based on a review of the literature
relating to the effectiveness of PMS (Lawler, 2003) with minor modifications made to
fit the context of the study. Respondents were required to indicate the extent to which
their PMS had achieved each of the 16 perceived outcomes using a five-point Likert
scale with anchors of 1 “not at all” and 5 “to a great extent”.
Factor analysis (principal components with varimax rotation) using a cutoff point of
0.60 revealed that the 16 outcomes loaded onto two dimensions, with the factor structure
consistent with Baird (2010). The first dimension included nine items which all refer to the
achievement of organizational goals and objectives, hence, this dimension
was labeled “performance-related outcomes”. The second dimension included The
seven items which are more concerned with employees, hence this dimension was effectiveness of
labeled “staff-related outcomes”. These two dimensions were subsequently scored PMSs
as the average score of the items loading on to each dimension with higher (lower)
scores representing a more (less) effective PMS.
3.1.2 The usage of multidimensional performance measures. The extent to
which respondents were using multidimensional performance measures was measured
using two approaches. The first approach required respondents to simply indicate if they 1295
were using a BSC (“yes” or “no”). Since this approach is reliant on respondents
understanding of the nature of a BSC, a more comprehensive approach which focuses
on the performance measures employed within organizations, was also adopted. This
approach required respondents to indicate the extent to which they were using 26
different performance measures (the Appendix) to assess their business units’
performance, on a five-point Likert scale with anchors of 1 “not at all” to 5 “to a great
extent”. These measures were derived primarily from the BSC literature and were
mainly designed for manufacturing organizations (Epstein, 2008; Jusoh et al., 2008;
Van der Stede et al., 2006; Bryant et al., 2004; Ittner et al., 2003; Kaplan and Norton, 2001, 1996).
Factor analysis (principal components with varimax rotation) using a cutoff
point of 0.6 revealed that the 26 items loaded onto six specific dimensions covering
the following perspectives: financial, customer, internal business, learning, growth
and sustainability. These findings are in line with Figge et al. (2002), except that
the learning and growth perspectives were separated. These two perspectives were
subsequently combined in accordance with the five perspectives BSC model.
Each of the five perspectives were scored as the sum of the items loading onto
each perspective with higher (lower) scores indicating the PMS focused on each
perspective to a greater (lesser) extent. Since a different number of items loaded
onto each of the perspectives, average scores were calculated with the use of
multidimensional performance measure scored as the sum of the averages across
the five perspectives with higher (lower) scores indicating that multidimensional
performance measures were used to a greater (less) extent.
3.1.3 Organizational factors. Each of the four organizational factors was
measured using a summated five-point Likert scale with anchors of 1 “strongly
disagree” and 5 “strongly agree”.
Top management support was measured using a three-item summated scale (the
Appendix) with respondents required to indicate the extent to which top
management provided adequate resources (Krumwiede, 1998), communicated
effectively (Grover, 1993) and exercised its authority in support of the PMS. Top
management support was measured as the average score for the three items, with
higher (lower) scores indicating a higher (lower) level of top management support.
The level of related training was measured using three items (the Appendix) drawn
from Baird et al. (2007), with minor adjustments made to fit the context of the current
study. Specifically, respondents were required to indicate if adequate training had been
provided to develop, to implement and to ensure employees understood the PMS.
Training was measured as the average score for the three items, with higher (lower)
scores indicating a higher (lower) level of related training provided by the organization.
In the absence of specific measures in the literature on employee participation in a
PMS context, two self-developed items (the Appendix) were adopted following a review IJOPM
of the employee participation/involvement literature (Sinclair et al., 2005; Harel and 31,12
Tzafrir, 1999; Huselid, 1995; Wagner, 1994). Specifically, respondents were required to
indicate the extent to which lower level employees participated in designing the PMS and
were involved in selecting performance measures. The perceived level of employee
participation was subsequently scored as the average score for the two items with higher
(lower) scores indicating a higher (lower) level of employee participation. 1296
The link of performance to rewards was assessed using two items (the Appendix)
based on the literature on performance and rewards (Rynes et al., 2005; Lawler,
2003; Huselid, 1995). Respondents were required to indicate the extent to which
performance is linked to financial rewards such as pay or bonus, and non-financial
rewards such as recognition or service awards in their organization. The analysis
revealed that the two questions were measuring different factors: the extent to which
performance is linked to financial rewards and to non-financial rewards. These
measures are analyzed as separate independent variables, with higher (lower) scores
indicating a stronger (weaker) link of performance to rewards. 4. Results
Table I shows summary statistics for the dependent and independent variables. For the
multi-item scales, the actual range was comparable with the theoretical range, and the
Cronbach’s a coefficients met or exceeded the 0.70 threshold generally considered
acceptable in regard to scale reliability (Nunnally, 1978, p. 245).
The mean scores of the effectiveness of PMSs for both the performance-related
outcomes (3.50) and the staff-related outcomes (3.26) are slightly higher than the mid-
point of the range, indicating that on average the respondents assessed their PMS to
be moderately effective. The performance-related outcomes were achieved to a
greater extent, with the mean scores of all nine items equal to or greater than the
seven staff-related outcomes. The performance-related outcomes that were achieved n a Minimum Maximum Variables Mean SD (theoretical) (theoretical) Cronbach’s a Independent variables Use of multidimensional performance measures 118 2.94 0.70 1.17 (1) 4.67 (5) Top management support 117 3.51 1.02 1 (1) 5 (5) 0.915 Training 117 3.11 1.07 1 (1) 5 (5) 0.963 Employee participation 117 2.41 1.02 1 (1) 5 (5) 0.761
Link of performance to financial rewards 117 3.50 1.16 1.00 (1) 5.00 (5) Link of performance to non- financial rewards 117 2.93 1.13 1.00 (1) 5.00 (5) Dependent variables Effectiveness of PMS (performance-related outcomes) 117 3.50 0.81 1 (1) 5 (5) 0.932
Effectiveness of PMS (staff-related outcomes) 117 3.26 0.93 1 (1) 5 (5) 0.924 Table I.
Note: aThe number of responses (n) varies due to the fact that not all survey items were completed Descriptive statistics by respondents
to the greatest extent included: assisting in achieving the goals (mean score of 3.68); The effectiveness
providing useful performance feedback to employees (mean score of 3.64); of PMSs
developing a performance-oriented culture (mean score of 3.59); and providing an
accurate assessment of business unit performance (mean score of 3.59). The staff-
related outcomes that were achieved to the greatest extent included: developing
individual’s skill and knowledge (mean score of 3.38), identifying talented employees
(mean score of 3.36), and rewarding talented employees (mean score of 3.31). 1297
In regard to the four organizational factors, while the mean score of most of the
factors lie on the higher end of the scale, the mean value of the link of performance to
non-financial rewards (2.93) was slightly below the mid-point of the range indicating
a relative weak link between performance and non-financial rewards.
As discussed in the method section, two approaches were used to assess the use of
multidimensional performance measures. Table II reveals that 39 respondents (33.1
percent) indicated that they were using a BSC in their business unit. The more
comprehensive approach to measuring the use of multidimensional performance
measures focused on the extent to which business units were employing 26
performance measures covering the five perspectives of the BSC. Table I reveals that
the mean score for the use of multidimensional performance measures (2.94) was
slightly lower than the mid-point of the range, indicating a moderate use of
multidimensional performance measures in Australian manufacturing organizations.
Table III provides a more detailed analysis of the extent to which measures relating
to each of the five perspectives were employed. The greatest emphasis was placed on
the financial perspective (3.59) followed by the customer (3.43), learning and growth
(3.11), and internal business process (3.06) perspectives. The mean score of the
sustainability perspective (2.19) was below the mid-point of the range indicating a
relatively low level of usage of this perspective.
4.1 Analysis of the association between the use of multidimensional performance
measures and organizational factors with the effectiveness of PMSs
Table IV presents the results of the one-way analysis of variance (ANOVA) used to
examine the difference in the level of PMS effectiveness based on whether respondents
were using a BSC. Respondents using a BSC reported a significantly higher level of PMS
effectiveness with respect to both performance- and staff-related outcomes. BSC usage Frequency Adjusted percentage Yes 39 33.1 Table II. No 79 66.9 BSC usage BSC perspectives n Minimum Maximum Mean Rank Financial 118 1.00 (1) 5.00 (5) 3.59 1 Customer 118 1.00 (1) 5.00 (5) 3.43 2 Internal business process 118 1.00 (1) 5.00 (5) 3.06 4 Table III. Learning and growth 118 1.17 (1) 5.00 (5) 3.11 3 Use of multidimensional Sustainability 118 1.00 (1) 5.00 (5) 2.19 5 performance measures IJOPM
These results provide preliminary evidence that the use of multidimensional 31,12
performance measures is associated with the effectiveness of PMSs, thereby providing support for H1.
The association between the use of multidimensional performance measures and
PMS effectiveness was also analyzed using a more comprehensive approach based on
the extent of use of multidimensional performance measures. Stepwise regression was 1298
used to examine the association between both the use of multidimensional performance
measures and organizational factors with PMS effectiveness, with the results presented
in Table V. For the effect on performance-related outcomes, the model was
statistically significant (F ¼ 63.812, p ¼ 0.000) with an R 2 of 0.530 indicating that
53 percent of the variance in the achievement of performance-related outcomes can be
explained by the explanatory factors. The model reveals that the use of
multidimensional performance measures ( p ¼ 0.000) was significantly associated
with the effectiveness of PMSs. In addition, top management support ( p ¼ 0.000)
was significantly associated with the performance-related outcomes.
Table V also provides the findings for staff-related outcomes, with the model found
to be statistically significant (F ¼ 38.535, p ¼ 0.000) with an R 2 of 0.405 indicating
that 40.5 percent of the variance in the achievement of staff-related outcomes can be
explained by the explanatory factors. The model reveals that the use of
multidimensional performance measures was found to be significantly associated with
the achievement of staff-related outcomes ( p ¼ 0.000). The level of training ( p ¼
0.000) was also significantly associated with PMS effectiveness.
The findings provide further support for H1 and partially support H2 and H3. The
importance of the use of multidimensional performance measures in explaining the
level of PMS effectiveness prompted further exploratory analysis to investigate the
association between each of the five perspectives of the BSC with the effectiveness of
PMSs. These findings are presented in Section 4.2. Table IV. Performance-related outcomes Staff-related outcomes Results of the one-way BSC usage n Mean F-statistic Significance Mean F-statistic Significance ANOVA comparing the BSC user 39 3.88 14.297 0.000 3.71 15.869 0.000 level of PMS effectiveness based on BSC usage Non-BSC user 78 3.31 3.03 Performance-related outcomes Staff-related outcomes Table V. Variables Coefficient t-statistics
Significance Coefficient t-statistics Significance Results of stepwise Multidimensional PMS 0.343 4.512 0.000 0.374 4.465 0.000 regression analysis Top management of the association support 0.487 6.411 0.000 between the use Training 0.362 4.325 0.000 of the multidimensional F-value 63.812 38.535 performance measures p-value 0.000 0.000 and organizational R 2 0.530 0.405 factors with the Adjusted R 2 0.522 0.395 effectiveness of PMSs n 115 115
4.2 Analysis of the association between the five perspectives of the BSC The effectiveness with the effectiveness of PMSs of PMSs
Table VI reveals the stepwise regression analysis findings. The performance-related
outcomes model was statistically significant (F ¼ 63.847, p ¼ 0.000) with an R 2 of 0.528
indicating that 52.8 percent of the variance in the achievement of the performance-related
outcomes can be explained by the two perspectives of the BSC found to be significantly
associated with the performance-related outcomes: the internal business process ( p ¼ 1299
0.000) and learning and growth ( p ¼ 0.000) perspectives.
The staff-related outcomes model was also statistically significant (F ¼ 56.768, p
¼ 0.000) with an R 2 value of 0.499 indicating that 49.9 percent of the variance in the
achievement of the staff-related outcomes can be explained by the two perspectives of
the BSC found to be significantly associated with the staff-related outcomes: the
learning and growth ( ¼ 0.000) and sustainability ( p p ¼ 0.000) perspectives. 5. Conclusion 5.1 Discussion
The first objective of the study was to examine the effectiveness of PMSs in respect to
their impact on organizational processes. The study evaluated the effectiveness of
PMSs based on the extent to which 16 desired outcomes were achieved. By focusing
on the outcomes achieved, the study contributes to the empirical body of knowledge
on PMSs since the majority of previous studies have only assessed PMS effectiveness
based on overall organizational performance. This approach provides managers with a
more detailed insight into the ability of the PMS to assist their organization in
achieving specified desired outcomes. Factor analysis revealed that these items
reflected two dimensions of PMS effectiveness: performance- and staff-related
outcomes. The results revealed that the mean score for the effectiveness of PMSs for
both dimensions was slightly above the mid-point of the range, indicating that the
PMSs of Australian manufacturing organizations were only moderately effective. This
finding highlights the significance of the study’s investigation of the contingency
factors associated with the effectiveness of PMSs.
The results also showed that organizations were more successful in achieving the
performance-related outcomes than the staff-related outcomes. This suggests that PMSs
have mainly been used as a managerial tool to assist the organization in motivating
performance, implementing the organizational strategy and achieving goals. Performance-related outcomes Staff-related outcomes Variables Coefficient t-statistics
Significance Coefficient t-statistics Significance Internal business process 0.277 3.830 0.000 Table VI. Learning and growth 0.558 7.730 0.000 0.539 7.445 0.000 Results of stepwise Sustainability 0.289 4.001 0.000 regression analysis F-value 63.847 56.768 of the association p-value 0.000 0.000 between each of the five R 2 0.528 0.499 perspectives of the BSC Adjusted R 2 0.520 0.490 with the effectiveness n 116 116 of PMSs IJOPM
Less emphasis is being placed on achieving staff-related outcomes such as addressing the 31,12
concerns of staff, ensuring staff time is used efficiently, and managing poorly performing staff.
The latter finding is of concern given that survival in today’s rapidly changing world is
dependent on the achievement of both staff- and performance-related outcomes. Harel and
Tzafrir (1999, p. 185) highlighted the importance of focusing on employees, suggesting that an 1300
organization’s staff are its strategic assets which “form a system of resources and rare abilities
that cannot easily be copied, and provide the company with its competitive edge”. Hence,
organizations which view staff as potential partners and important assets enhance the
likelihood of achieving better organizational performance.
There is also evidence that the achievement of staff-related outcomes can assist in the
achievement of performance-related outcomes. If organizations adequately address the
concerns of their employees, they are more likely to be emotionally attached to a
particular organization, and hence more willing to assist in the achievement of
organizational goals (Myer and Allen, 1991). Accordingly, we suggest that managers
place greater emphasis on the achievement of staff-related outcomes. This should be
embodied in the design of the PMS so as to incorporate both contributions from
employees as well as reflecting their personal needs.
The second objective of the study was to examine the association between the use of
multidimensional performance measures and four organizational factors with the effectiveness
of the PMS. The initial analysis focused on ascertaining the extent to which organizations
were using multidimensional performance measures. Results revealed that only 33.1 percent
of organizations were using the BSC, which is consistent with previous findings (Crabtree and
DeBusk, 2008 (35 percent); Chung et al., 2006 (31 percent); Speckbacher et al., 2003 (26
percent); Whorter, 2003 (35 percent)).
A more comprehensive analysis of the use of multidimensional performance measures
revealed that Australian manufacturing organizations placed the greatest emphasis on
measures relating to the financial perspective of the BSC, followed by the customer, learning
and growth internal business process, and sustainability perspectives. This finding is
consistent with the majority of the BSC literature which suggests that financial measures are
still used to the greatest extent (Crabtree and DeBusk, 2008; Hoque and Adams, 2008; Davis
and Albright, 2004; Braam and Nijssen, 2004; Ittner et al., 2003; Hoque and James, 2000;
Lipe and Salterio, 2000; Ittner and Larcker, 1998). The findings indicate that while
organizations may be enticed to use a BSC, and even claim to use the BSC, the reality is that
the greatest emphasis is still placed on the traditional financial-based perspective. Therefore, if
organizations are to reap the benefits of using multidimensional PMSs such as the BSC, it is
crucial that they do not just pay lip service to the inclusion of measures covering the other
perspectives. Rather they need to acknowledge the importance of the other perspectives of the
BSC and place increasing emphasis on using measures relating to each of the perspectives.
Analysis of the association between the use of multidimensional performance
measures and organizational factors with the effectiveness of PMSs revealed that the use
of multidimensional performance measures, as operationalized by the BSC, and two
organizational factors (top management support, and training) exhibited a significant
association with the effectiveness of PMSs.
The use of multidimensional performance measures was positively associated with
both the performance- and staff-related outcomes. This finding is in line with previous
studies (Chow and Van der Stede, 2006; Van der Stede et al., 2006; Bryant et al., The effectiveness
2004) which have advocated that organizations should incorporate both financial and of PMSs
non-financial measures in the PMS. Similarly, the findings reinforce the literature
advocating the benefits of the BSC (Langfield-Smith et al., 2009; Epstein, 2008;
Kaplan and Norton, 2006; Speckbacher et al., 2003). The findings highlight the need
for managers to evaluate the inherent characteristics of their PMS and their impact on
the achievement of such outcomes. In particular managers need to focus on the extent 1301
to which diversified performance measures reflecting the five perspectives of the BSC are incorporated in their PMS.
Additional exploratory analysis revealed that the internal business and learning and
growth perspectives were associated with the effectiveness of PMSs regarding the
performance-related outcomes, while the learning and growth and sustainability
perspectives were significantly associated with the staff-related outcomes. While this
finding highlights the importance of adopting a BSC, it also provides managers with
insight into the specific BSC dimensions which warrant their attention in order to
enhance PMS effectiveness. Managers are therefore encouraged to ensure that their
PMS emphasises the use of performance measures in relation to the internal business
process (e.g. productivity, usage of resources, cycle time and number of product
returns), learning and growth (e.g. hours of training provided, improvements made to
employee facility, number of new product produced and percentage of revenue from
new applications) and sustainability (e.g. investment in environmental management,
promotion of environmental causes and investment in community services)
perspectives in order to enhance the effectiveness of their PMS.
Analysis of the association between the organizational factors and the effectiveness of
the PMS provides an insight into the prevailing organizational conditions that could
enhance/prohibit PMS effectiveness. Top management support was found to be associated
with the performance-related outcomes, and the level of training was associated with the
staff-related outcomes. While top management support has been found to be a critical
success factor for PMS implementation (Bourne, 2005; Chan, 2004; Bourne et al., 2002;
Emerson, 2002; Kennerley and Neely, 2002), the findings highlight the importance of the
continued involvement and support from top management. Hence, in order to achieve the
desired performance-related outcomes, a concentrated effort by top management aimed at
continuous improvement, open communication and consistent support is required
(Kaynak, 2003). Top management is therefore encouraged to personally commit to the
PMS and ensure that enough time and resources are dedicated on an on-going basis to
properly develop and manage the existing PMS. In addition, organizations which provide
more related training to their staff are able to achieve the desired staff-related outcomes.
This supports Harel and Tzafrir’s (1999) suggestion that moving knowledge information
and power to lower levels of the organization is a way to sustain competitive advantage.
Organizations could therefore employ appropriate training with respect to the use of PMSs
across different business levels to enhance the knowledge and skill of employees in
developing and implementing the systems.
The study contributes to the literature by examining PMS effectiveness in terms of the
effect on organizational processes. The two dimensions of PMS effectiveness,
performance- and staff-related outcomes, serve to make management more aware of the
need to focus on different aspects of PMS effectiveness as well as providing researchers
with a new measure which can be used to evaluate its effectiveness. In addition, IJOPM
the association between the use of multidimensional performance measures 31,12
and organizational factors with the effectiveness of the PMS provides managers of
organizations with an insight into the desirable characteristics of an effective PMS and
the prevailing organizational conditions which can support the PMS. Hence, managers
need to focus on using multidimensional performance measures, and increase the level of
top management support and related training in relation to their PMS. 1302
5.2 Limitations and future research
The study is subject to the usual limitations of the survey method. While the survey
method is useful in ascertaining associations rather than causal relationships between
variables (Singleton and Straits, 2005), this approach generates potential threats as
respondents may answer questions in accordance with social desirability bias. Future
studies may incorporate face-to-face interviews in order to provide richer descriptions
into the hypothesised associations. Future studies could also collect data from
multiple respondents across different management levels. This may assist in
overcoming the common method bias associated with the single respondent approach.
The study also used a number of self-developed measures. For instance, the
measures of PMS effectiveness, the usage of multidimensional performance
measures, and two organisational factors, employee participation and the link of
performance to rewards, were self-developed. The face validity of these measures was
enhanced through a pilot study of ten academics with relevant expertise, and their
content validity was enhanced by developing the measures based on an extensive
review of the relevant literature. However, while factor analysis provides evidence of
the construct validity of the first three measures, the validity of these measures still
needs to be confirmed in future studies, especially given the sample size is considered
small for performing factor analysis. In addition while the Cronbach’s a scores
confirm the reliability of the first three of these measures, the two items used to
measure link of performance to rewards were found to be measuring separate
constructs. Hence, there is concern as to the reliability of this measure and future
studies may explore alternative ways of measuring this factor.
In addition, the current study only provides empirical evidence in relation to the
association between four organizational factors (top management support, training,
employee participation and link of performance to rewards) and the effectiveness of PMS.
Future studies may consider the association between other organizational factors such as
organizational structure, and management style, with PMS effectiveness. To enhance the
generalizability of the findings, future studies could be conducted using similar parameters
in other industries such as service and the non-profit sector. Notes
1. Cohen’s (1988) formulae which considers the number of independent variables, statistical
significance and power, and effect size, was used to determine the required number of valid
responses (91). Assuming a conservative response rate of 20 percent, a sample size of 445 was determined.
2. The Kompass Australia business directory provides details of manufacturing businesses in
Australia. It is assumed that a random sample taken from this directory is representative of
the Australian manufacturing industry.
3. The Dillman (2007) Tailored Design Method provides guidelines in respect to the The
format and style of questions, personalisation, and distribution procedures. There is effectiveness
evidence that this approach leads to improved response rates. of PMSs References
Atkinson, A., Balakrishnan, A.R., Booth, P., Cote, J.M., Groot, T., Malmi, T., Roberts, H.,
Uliana, E. and Wu, A. (1997), “New directions in management accounting research”,
Journal of Management Accounting Research, Vol. 9, pp. 79-108. 1303
Baird, K. (2010), “The effectiveness of strategic performance measurement systems”,
unpublished working paper, Macquarie University, Sydney.
Baird, K., Harrison, G. and Reeve, R. (2007), “Success of activity management practices: the influence
of organizational and cultural factors”, Accounting and Finance, Vol. 47 No. 1, pp. 47-67.
Bedford, D.S., Brown, D.A. and Malmi, T. (2006), “Balanced scorecard content, use, and
performance impacts: some Australian evidence”, paper presented at the European
Accounting Association, 29th Annual Conference, Dublin.
Bhasin, S. (2008), “Lean and performance measurement”, Journal of Manufacturing
Technology Management, Vol. 19 No. 5, pp. 670-84.
Bonner, S.E. and Sprinkle, G.B. (2002), “The effects of monetary incentives on effort and
task performance: theories, evidence, and a framework for research”, Accounting,
Organizations and Society, Vol. 27 Nos 4/5, pp. 303-45.
Bourne, M.C. (2005), “Researching performance measurement system implementation: the
dynamics of success and failure”, Production Planning and Control, Vol. 16 No. 2, pp. 101-13.
Bourne, M.C., Neely, A.D., Platts, K.W. and Mills, J.F. (2002), “The success and failure of
performance measurement initiatives: the perceptions of participating managers”,
International Journal of Operations and Production Management, Vol. 22 No. 11, pp. 1288-310.
Braam, G.J.M. and Nijssen, E.J. (2004), “Performance effects of using the balanced scorecard:
a note on the Dutch experience”, Long Range Planning, Vol. 37 No. 4, pp. 335-49.
Bryant, L., Jones, D.A. and Widener, S.K. (2004), “Managing value creation within the
firm: an examination of multiple performance measures”, Journal of Management
Accounting Research, Vol. 16 No. 1, pp. 107-31.
Burney, L.L., Henle, C.A. and Widener, S.K. (2009), “A path model examining the
relations among strategic performance measurement system characteristics,
organizational justice, and extra- and in-role performance”, Accounting,
Organizations and Society, Vol. 34 Nos 3/4, pp. 305-21.
Cavaluzzo, S.K. and Ittner, C.D. (2004), “Implementing performance measurement
innovations: evidence from government”, Accounting, Organisations and
Society, Vol. 29 No. 1, pp. 243-67.
Chan, Y.C.L. (2004), “Performance measurement and adoption of balanced scorecards: a
survey of municipal governments in the USA and Canada”, The International
Journal of Public Sector Management, Vol. 17 No. 3, pp. 204-21.
Chang, C.J., Lee, H.I. and Chen, W.C. (2008), “A fuzzy AHP and BSC approach for
evaluating performance of IT department in the manufacturing industry in Taiwan”,
Expert Systems with Applications, Vol. 34 No. 1, pp. 96-107.
Cheng, M.I., Dainty, A. and Moore, D. (2007), “Implementing a new performance
management system within a project-based organisation: a case study”,
International Journal of Productivity and Performance Management, Vol. 56 No. 1, pp. 60-75. IJOPM
Chiles, A.M. and Zorn, T.E. (1995), “Empowerment in organizations: employees’ perceptions 31,12
of the influences on empowerment”, Journal of Applied Communication Research, Vol. 23 No. 1, pp. 1-25.
Chow, C.W. and Van der Stede, W.A. (2006), “The use and usefulness of nonfinancial
performance measures”, Management Accounting Quarterly, Vol. 7 No. 3, pp. 1-8.
Chung, L.H., Gibbons, P.T. and Schoch, H.P. (2006), “The management of information and 1304
managers in subsidiaries of multinational corporations”, British Journal of
Management, Vol. 17 No. 2, pp. 153-65.
Clinquini, L. and Mitchell, F. (2005), “Success in management accounting: lessons from the
activity-based costing/management experience”, Journal of Accounting and
Organizational Change, Vol. 1 No. 1, pp. 63-78.
Cohen, J. (1988), Statistical Power Analysis for the Behavioral Sciences, 2nd ed., Lawrence-Erlbaum, Mahwah, NJ.
Cox, A., Marchington, M. and Sutter, J. (2007), Embedding the Provision of Information
and Consultation in the Workplace: A Longitudinal Analysis of Employee
Outcomes in 1998 and 2004, DTI Employee Relations Research Series, No. 72,
Department of Trade and Industry, London.
Cox, A., Zagelmeyer, S. and Marchington, M. (2006), “Embedding employee involvement and
participation at work”, Human Resource Management Journal, Vol. 16 No. 3, pp. 250-67.
Crabtree, A.D. and DeBusk, G.K. (2008), “The effects of adopting the balanced scorecard on
shareholder returns”, Advances in Accounting, Vol. 24 No. 1, pp. 8-15.
Davis, S. and Albright, T. (2004), “An investigation of the effect of balanced scorecard
implementation on financial performance”, Management Accounting Research, Vol. 15 No. 2, pp. 135-53.
De Geuser, F., Mooraj, S. and Oyon, D. (2009), “Does the balanced scorecard add value? Empirical
evidence on its effect on performance”, European Accounting Review, Vol. 18 No. 1, pp. 93-122.
Dillman, D.A. (2007), Mail and Internet Surveys: The Tailored Design Method, 2nd ed., Wiley, New York, NY.
Doll, W.J. (1985), “Avenues for top management involvement in successful MIS development”,
MIS Quarterly, Vol. 9 No. 1, pp. 17-35.
Emerson, B. (2002), “Training for performance measurement success: an effective training
program can help get performance measurement off the ground and sustain the system as
it matures into a catalyst for government accountability and improvement”, Government
Finance Review, April, available at: www.thefreelibrary.com/Trainingþ
forþperformanceþmeasurementþsuccess%3aþAnþeffectiveþtraining-a085048611 (accessed 1 May 2009).
Epstein, M. (2008), Making Sustainability Work: Best Practices in Managing and
Measuring Social and Environmental Impacts, Greenleaf, Sheffield.
Epstein, M. and Manzoni, J.F. (1998), “Implementing corporate strategy: from tableaux de bord
to balanced scorecard”, European Management Journal, Vol. 16 No. 2, pp. 190-203.
Figge, F., Hahn, T., Schaltegger, S. and Wagner, M. (2002), “The sustainability balanced
scorecard – linking sustainability management to business strategy”, Business Strategy
and the Environment, Vol. 11 No. 5, pp. 269-84.
Grover, V. (1993), “An empirical derived model for the adoption of customer-based inter-
organizational systems”, Decision Sciences, Vol. 24 No. 3, pp. 603-39.
Hamilton, S. and Chervany, N.L. (1981), “Evaluating information system effectiveness – part I:
comparing evaluation approaches”, MIS Quarterly, Vol. 5 No. 3, pp. 55-69.
Harel, H.G. and Tzafrir, S.S. (1999), “The effect of human resource management practices The
on the perceptions of organizational and market performance of the firm”, Human Resource Management effectiveness , Vol. 38 No. 3, pp. 185-200. of PMSs
Hoque, Z. and Adams, C. (2008), Measuring Public Sector Performance: A Study of
Government Departments in Australia, CPA Australia, Melbourne.
Hoque, Z. and James, W. (2000), “Linking balanced scorecard measures to size and market
factors: impact on organizational performance”, Journal of Management
Accounting Research, Vol. 12 No. 1, pp. 1-17. 1305
Horngren, C.T., Bhimani, A., Datar, S.M. and Foster, G. (2005), Management and Cost
Accounting, 3rd ed., Pearson Education, Harlow.
Huselid, M.A. (1995), “The impact of human resource management practices on turnover,
productivity, and corporate financial performance”, Academy of Management
Journal, Vol. 38 No. 3, pp. 535-672.
Ittner, C.D. and Larcker, D.F. (1998), “Innovations in performance measurement: trends and
research implications”, Journal of Management Accounting Research, Vol. 10, pp. 205-38.
Ittner, C.D., Larcker, D.F. and Randall, T. (2003), “Performance implications of strategic
performance measurement in financial services firms”, Accounting, Organizations
and Society, Vol. 28 Nos 7/8, pp. 715-41.
Johanson, U., Skoog, M., Backlund, A. and Almqvist, R. (2006), “Balancing dilemmas of the balanced
scorecard”, Accounting, Auditing & Accountability Journal, Vol. 19 No. 6, pp. 842-57.
Jusoh, R., Ibrahim, D.N. and Zainuddin, Y. (2008), “The performance consequence of multiple
performance measures usage: evidence from the Malaysian manufacturers”, International
Journal of Productivity and Performance Management, Vol. 57 No. 2, pp. 119-36.
Kaplan, R.S. (2001), “Strategic performance measurement and management in nonprofit
organizations”, Nonprofit Management and Leadership, Vol. 11 No. 3, pp. 353- 70.
Kaplan, R.S. and Norton, D.P. (1992), “The balanced scorecard-measures that drive
performance”, Harvard Business Review, Vol. 70, January/February, pp. 71-9.
Kaplan, R.S. and Norton, D.P. (1996), “Linking the balanced scorecard to strategy”,
California Management Review, Vol. 39 No. 1, pp. 53-79.
Kaplan, R.S. and Norton, D.P. (2001), The Strategy-focused Organization: How
Balanced Scorecard Companies Thrive in the New Business Environment,
Harvard Business School Press, Boston, MA.
Kaplan, R.S. and Norton, D.P. (2004), “Measuring the strategic readiness of intangible
assets”, Harvard Business Review, February, pp. 52-63.
Kaplan, R.S. and Norton, D.P. (2006), Alignment: Using the Balanced Scorecard to
Create Corporate Synergies, Harvard Business School Press, Boston, MA.
Kaplan, S.E. and Wisner, P.S. (2009), “The judgmental effects of management
communications and fifth balanced scorecard category on performance evaluation”,
Behavioral Research in Accounting, Vol. 21 No. 2, pp. 37-56.
Kaynak, H. (2003), “The relationship between total quality management practices and their
effects on firm performance”, Journal of Operations Management, Vol. 21 No. 4, pp. 405-35.
Kennerley, M. and Neely, A. (2002), “A framework of the factors affecting the evolution of
performance measurement systems”, International Journal of Operations and
Production Management, Vol. 22 No. 11, pp. 1222-45.
Kleingeld, A.D., Tuijl, H.V. and Algera, J. (2004), “A participation in the design of
performance management systems: a quasi-experimental field study”, Journal of
Organizational Behavior, Vol. 25 No. 7, pp. 831-51. IJOPM
Koberg, C.S., Boss, R.W., Senjem, J.C. and Goodman, E.A. (1999), “Antecedents and 31,12
outcomes of empowerment: empirical evidence from the health care industry”, Group
and Organsiation Management, Vol. 24 No. 1, pp. 71-91.
Kompass Australia (2009), Kompass Australia, Peter Isaacson Publications, Elsternwick.
Krumwiede, K.R. (1998), “The implementation stages of activity-based costing and the impact
of contextual and organizational factors”, Journal of Management Accounting 1306 Research, Vol. 10, pp. 239-77.
Langfield-Smith, K., Thorne, H. and Hilton, R. (2009), Management Accounting: An
Australian Perspective, 6th ed., McGraw-Hill, Sydney.
Lawler, E.E. (2003), “Reward practices and performance management system effectiveness”,
Organizational Dynamics, Vol. 32 No. 4, pp. 396-404.
Lebas, M.J. (1995), “Performance measurement and performance management”, International
Journal of Production Economics, Vol. 41 Nos 1-3, pp. 23-35.
Lipe, M.G. and Salterio, S.E. (2000), “The balanced scorecard: judgmental effects of common
and unique performance measures”, Accounting Review, Vol. 75 No. 3, pp. 283-9.
Lynch, R.L. and Cross, K.F. (1991), Measure Up! Yardsticks for Continuous Improvement, Blackwell, Cambridge.
McShane, S. and Travaglione, T. (2003), Organizational Behaviour on the Pacific Rim, McGraw-Hill, Sydney.
Malina, M.A. and Selto, F.H. (2001), “Communicating and controlling strategy: an empirical
study of the effectiveness of the balanced scorecard”, Journal of Management
Accounting Research, Vol. 13, pp. 47-90.
Marchand, M. and Raymond, L. (2008), “Researching performance measurement systems: a
information systems perspective”, International Journal of Operations and
Production Management, Vol. 28 No. 7, pp. 663-86.
Morrell, K. and Wilkinson, A. (2002), “Empowerment: though the smoke and past the
mirrors?”, Human Resource Development International, Vol. 5 No. 1, pp. 119-30.
Motwani, J., Mirchandani, D., Madan, M. and Gunasekaran, A. (2002), “Successful
implementation of ERP projects: evidence from two case studies”, International Journal
of Production Economics, Vol. 75 Nos 1/2, pp. 83-96.
Myer, J.P. and Allen, N.J. (1991), “A three-component conceptualization of organizational
commitment”, Human Resource Management Review, Vol. 1, pp. 61-89.
Neely, A. (1999), “The performance measurement revolution: why now and what next?”, International
Journal of Operations and Production Management, Vol. 19 No. 2, pp. 205-28.
Neely, A. and Adams, C. (2000), Perspectives on Performance: The Performance Prism, Gee Publishing, London.
Neely, A., Gregory, M. and Platts, K. (1995), “Performance measurement system design: a
literature review and research agenda”, International Journal of Operations and
Production Management, Vol. 25 No. 12, pp. 1228-63.
Neely, A., Mills, J., Platts, K., Gregory, M. and Richards, H. (1996), “Performance
measurement system design: should process based approaches be adopted?”,
International Journal of Production Economics, Vol. 46, pp. 423-31.
Neely, A., Mills, J., Platts, K., Richards, H., Gregory, M., Bourne, M. and Kennerley, M.
(2000), “Performance measurement system design: developing and testing a process-
based approach”, International Journal of Operations and Production Management, Vol. 20 No. 10, pp. 1119-45. Singleton, R.A. and
Norreklit, H. (2003), “The balanced scorecard: what is the score? A rhetorical analysis of the Straits, B.C.
balanced scorecard”, Accounting, Organizations and Society, Vol. 28 No. 6, pp. 591- (2005), 619. Approaches to Social
Nunnally, J.C. (1978), Psychometric Theory, McGraw-Hill, New York, NY. Research,
Othman, R. (2008), “Reflective practice: enhancing the effectiveness of the balanced Oxford
scorecard with scenario planning”, International Journal of Productivity and University
Performance Management, Vol. 57 No. 3, pp. 259-66. Press, New York, NY.
Otley, D. (1999), “Performance management: a framework for management control
systems research”, Management Accounting Research, Vol. 10 No. 4, pp. 282- 362.
PA Consulting Group (1998), “An incentive scheme that works: pay purview, how one
knowledge-intensive company has overhauled its incentives”, The Economist,
August, available at: www.paconsulting.com/introducing-pas-media-site/archive/an-
incentive-scheme-that-works-pay-purview-how-one-knowledge-intensive-company-
has-overhauled-its-incentives/ (accessed 10 April 2009).
Pike, S. and Roos, G. (2004), “Mathematics and modern business management”, Journal
of Intellectual Capital, Vol. 5 No. 2, pp. 243-56.
Pun, K., Chin, K. and Gill, R. (2001), “Determinants of employee involvement practices in
manufacturing enterprises”, Total Quality Management, Vol. 12 No. 1, pp. 95-109.
Raghunathan, B., Raghunathan, T. and Tu, Q. (1999), “Dimensionality of the strategic grid
framework: the construct and its measurement”, Information Systems Research, Vol. 10 No. 4, pp. 343-55.
Rao, S.S. (2000), “Enterprise resource planning: business needs and technologies”,
Industrial Management and Data Systems, Vol. 100 No. 1, pp. 81-8.
Ratnasingam, P. (2009), “Service quality management applying the balanced scorecard: and
exploratory study”, International Journal of Commerce and Management, Vol. 19 No. 2, pp. 127-36.
Rigby, D. and Bilodeau, B. (2009), Management Tools and Trends 2009, Bain & Company, Boston, MA.
Robert, E.S. (1999), “In defence of the survey method: an illustration from a study of user
information satisfaction”, Accounting and Finance, Vol. 39 No. 1, pp. 53-77.
Rynes, S.L., Gerhart, B. and Parks, L. (2005), “Personnel psychology: performance
evaluation and pay for performance”, Annual Review of Psychology, Vol. 56, pp. 571-600.
Schultz, R. and Ginzberg, M.J. (1984), “Implementation research-third generation”,
Applications of Management Science, Vol. 4, pp. 1-83.
Shields, M.D. (1995), “An empirical analysis of firms’ implementation experiences with
activity-based costing”, Journal of Management Accounting Research, Vol. 7, pp. 148-66.
Silk, S. (1998), “Automating the balanced scorecard”, Management Accounting, Vol. 79 No. 11, pp. 38-44.
Simons, R. (2000), Performance Measurement and Control Systems for
Implementing Strategy, Prentice-Hall, Englewood Cliffs, NJ.
Sinclair, R.R., Leo, M.C. and Wright, C. (2005), “Benefit system effects on employees’
benefit knowledge, use, and organizational commitment”, Journal of Business and
Psychology, Vol. 20 No. 1, pp. 3-29. The effectiveness of PMSs 1307 IJOPM
Speckbacher, G., Bischof, J. and Pfeiffer, T. (2003), “A descriptive analysis on the 31,12
implementation of balanced scorecard in German speaking countries”, Management
Accounting Research, Vol. 14 No. 4, pp. 361-88.
Van der Stede, W.A., Chow, C.W. and Lin, T.W. (2006), “Strategy, choice of performance measures,
and performance”, Behavioral Research in Accounting, Vol. 18, pp. 185-205.
Wagner, A.J. (1994), “Participation’s effects on performance and satisfaction: a reconsideration 1308
of research evidence”, Academy of Management Review, Vol. 19, pp. 312-30.
Wexley, K.N. (1984), “Personnel training”, Annual Reviews, Vol. 35, pp. 519-51.
Whorter, L.B.M. (2003), “Does the balanced scorecard reduce information overload?”,
Management Accounting Quarterly, Vol. 4 No. 4, pp. 23-7.
Wimalasiri, J. and Kouzmin, A. (2000), “A comparative study of employee involvement
initiatives in Hong Kong and the USA”, International Journal of Manpower, Vol. 21 No. 8, pp. 614-34. Appendix. Variable measurement
Effectiveness of PMS (adapted from Lawler (2003), Ittner et al. (2003) and Kaplan and Norton (2001, 1996))
Please indicate the extent to which your business unit’s PMS assists your business unit in achieving each of these. Performance-related outcomes: Motivating performance.
Assisting in the achievement of goals.
Developing a performance-oriented culture. Supporting change efforts.
Providing useful performance feedback to employees.
Implementing the organizational strategy.
Providing an accurate assessment of business.
Ensuring staff commitment to organizational objectives.
Linking individual performance to business unit performance. Staff-related outcomes:
Developing individual’s skill and knowledge.
Addressing the concerns of staff.
Ensuring the concerns of staff.
Identifying talented employees. Rewarding talented employees.
Identifying poor performing staff.
Managing poor performing staff.
The use of multidimensional performance measures Financial perspective: Sales revenue. Return on investment.
Improvement in net assets/liabilities. Customer perspective: The effectiveness On-time product delivery. of PMSs Number of new customers. Quality of products. Number of product returns.
Internal business process perspective: 1309 Usage/wastage of resources. Productivity. Cycle time.
Expenditure on warranty claims.
Learning and growth perspective: Hours of training provided.
Improvements made to employee facilities.
Number of employee suggestions implemented.
Number of new products produced.
Time to market for new products.
Percentage of revenue from new products/new applications. Sustainability:
Investment in environmental management.
Promotion of environmental causes.
Investment in community services.
Community connectedness services. Promotion of community causes. Organizational factors Top management support:
Top management has provided adequate resources to support the PMS.
Top management has effectively communicated its support for the PMS.
Top management exercises its authority in support of the PMS. Training:
Adequate training has been provided to ensure employees understand the PMS.
Adequate training has been provided to develop the PMS.
Adequate training has been provided to implement the PMS. Employee participation:
Lower level employees participated in designing the PMS.
Lower level employees were involved in selecting performance measures.
Link of performance to rewards:
Performance is linked to financial rewards (pay, bonuses, etc.) in your business unit.
Performance is linked to non-financial rewards (recognition, service awards, etc.) in your business unit. IJOPM About the authors 31,12
Amy Tung has taught both undergraduate and postgraduate subjects in the management
accounting area. Her research interests include performance measurement systems,
environmental management and employee organizational commitment. She is undertaking her
PhD in sustainability, with focus on environmental management systems and environmental
performance. Amy Tung is the corresponding author and can be contacted at: manamy. tung@mq.edu.au 1310
Kevin Baird has taught both undergraduate and postgraduate subjects in the management
accounting area for 16 years. He has also supervised Honours and PhD students across many
different topic areas within the management accounting discipline including activity-based
management practices, total quality management, performance measurement systems,
management control systems, outsourcing, employee organizational commitment and employee empowerment.
Herbert P. Schoch has taught both undergraduate and post-graduate courses, primarily in
Management Accounting and he has supervised PhD and Honours students. He has also taught
Financial Accounting, Business Strategy and Entrepreneurship and Entrepreneurial
Management. He has taught in Australia, Singapore, Hong Kong, Canada and the USA. His
research interests include management control systems, management accounting, outsourcing,
accounting education and entrepreneurship. He has published numerous journal articles, book
chapters and monographs. He also has experience in working in manufacturing, public
accounting and has managed and operated his own business.
To purchase reprints of this article please e-mail: reprints@emeraldinsight.com
Or visit our web site for further details: www.emeraldinsight.com/reprints View publication stats