The CPA Examination Pass Rate as an Assessment


By: Raymond B. Isleib, Barry R. Marks and Douglas B. Rusth

This paper investigates the performance of candidates taking the CPA examination for the first time. Factors influencing performance are dependent on the candidate working in accounting prior to taking the examination. Satisfaction with quality of instruction, an indirect measure of assessment, has no explanatory power in any of the models. This paper discusses the implications of these findings on using the CPA examination pass rate as an assessment measure in accounting programs.

Key Words: Assessment, Assurance of learning, CPA examination


Introduction

T he National Association of State Boards of Public Accountancy ( NASBA, 2003 ) publishes an annual report on candidate performance on the CPA examination . This report shows , by university, the percentages of first-time candidates earning passing scores on each part of the examination . T he pass rate for first-time candidates on certification exam ination s is potentially an important assessment measure of the effectiveness of accounting programs (Kimmel et al., 1998) . Many universities have a 150-hour accounting program that includes a graduate admission standard. This standard provides some assurance that students in the program have similar educational backgrounds and capabilities. These programs typically consist of full-time students who are enrolled in a structured academic program that is focused on the successful completion of the CPA examination. At this type of university, a somewhat homogeneous group of students take the CPA examination each semester. On the other hand, many universities have undergraduate accounting programs with a combination of part and full-time students, and some even have open admission standards. The characteristics of the first-time candidates for the CPA examination from these universities may vary significantly from one examination period to the next. Hence, for these universities, a change in CPA examination pass rate from one period to the next may be caused by a change in the characteristics of their first-time candidates rather than any change in quality of their programs. Although the NASBA gathers some information on the characteristics of the candidates taking the CPA examination, NASBA does not release that information by university.

The primary purpose of this study i s to identify factors that may a ffect first-time candidates ability to pass one or more parts of the CPA examination. This study assesses the comprehensive learning outcomes of the accounting program of a regional public urban university with an open admissions policy for undergraduate students. The accounting program has both part and full-time stude nts. This paper reports the results from the analysis of data obtained from a survey of alumni. The factors that predict the pass rate depend upon whether they had worked as accountants prior to initially taking the CPA examination. Satisfaction with the quality of classroom instruction, an indirect measure of assessment, is not related to performance on the CPA examination in any of the models. The conclusion describes the implications of these results for utilizing the CPA pass rate as an assessment measure, which should be of interest to faculty in other areas of business. Many other business disciplines, such as finance, human resources, and computer information systems, have professional certifications that require the completion of an examination. Because of the increasing demand for assessment, the faculty in these areas may consider first-time pass rates on these professional examinations as assessment measures in the future and similar issues may arise.

Review of Literature

The impetus for accounting programs to move into assessment began in the early 1980's. During this period, grade point averages were climbing and, concurrently, average scores on various college entrance examinations, such as the Scholastic Aptitude Test (SAT), were decreasing. At the same time, employers were voicing concerns that recent accounting graduates did not have the knowledge and skills necessary for success in a rapidly changing work environment (Kimmel et al., 1998). This concern resulted in a number of studies and reports, such as The Bedford Committee Report (American Accounting Association, 1986), The White Paper (Kullberg et al., 1989), and the Accounting Education Change Commission Report (AECC, 1990). Each had a major impact on assessment in accounting. The AECC report concluded that accounting education was too narrowly focused and graduates were lacking in communication, critical-thinking, problem-solving, and active-learning skills. At the same time, however, the state boards of public accountancy have been highly concerned about the CPA pass rate of students in their states. For example, the Texas State Board of Public Accountancy has historically sent out information about the CPA pass rate to the presidents of the universities in the state. In addition, state legislatures have been interested in whether the dollars allocated to higher education were used efficiently and effectively.

Although assessment and the concept of continuous improvement in education had an altruistic ring, one of the primary factors instrumental in implementation was that accreditation agencies began requiring universities, schools, and programs to establish a process that measured achievement. Both AACSB International–The Association to Advance Collegiate Schools of Business (AACSB) and regional accreditation agencies like the Southern Association of Colleges and Schools (SACS) required evidence that schools had embarked on formalized processes of continuous improvement and assessment of learning.

These events had a major impact on the rapidity with which accounting programs embraced and adopted various assessment measures. For example, the results of a 1996 survey of 92 accounting programs indicated that only 42 percent had a comprehensive outcomes assessment program in place (Kimmel, et al., 1998). In 1999, the results of a survey of the chairs of 167 accounting programs indicated that over 70 percent had established formal assessment programs. The primary reason for implementation, as indicated by 72.6 percent of the respondents was to meet accreditation agency requirements (Hindi and Miller, 2000).

The term assessment as used by AACSB relates to the measurement of the achievement of the learning goals established by the school or program. The assessment process is the means by which this is done and the objective is to demonstrate that learning has occurred for each learning goal. AACSB (2004) has established three reasons to assess learning accomplishments:

• To provide feedback and guidance for individual students

• To assist the school and faculty members to improve programs and courses

• To assure external constituents such as potential students, trustees, public officials, supporters, and accrediting agencies, that the organization meets its goals

Assessment from the viewpoint of AACSB has changed dramatically in recent years, motivating accounting programs and schools of business to implement comprehensive learning assessment programs. In the past, the accreditation agencies considered assessment as a process of resource evaluation, e.g., inputs such as number of faculty with terminal degrees and number of volumes in the library. The previous 1991 accreditation standards by AACSB emphasize the accomplishment of the mission of the institution and the establishment of a functioning continuous improvement process but did not directly address the topic of outcomes assessment (Kimmell, et al., 1998).

The accreditation standards adopted by AACSB in 2003 specify that the university must demonstrate that learning has occurred for each learning goal. Three approaches relating to measuring the accomplishment of learning goals are provided:

• Selection, as a measurement process, relates to the qualifications of the students admitted into the program.

• Course-embedded measurement relates to assessments within the required courses that effectively measure whether the learning goals have been achieved.

• Stand-alone testing or performance refers to demonstration of learning via a test or some form of performance (AACSB, 2004, pp 60-62).

The current AACSB standards consider questionnaires on the perception of the quality of a program as indirect measures of assessment and cannot replace the three previously described direct methods (AACSB, 2004, p. 64). Historically, accounting programs frequently used such questionnaires for assessment purposes (Apostolou, 1999). One reason for downplaying the significance of questionnaires is that they measure satisfaction rather than accomplishments. A satisfaction measure is included in this study to ascertain if it is related to the first-time pass rate on the CPA examination.

One of the several learning goals in consideration by the accounting program at the university surveyed and at various universities is preparation for the CPA examination. This goal meets one of the requirements for measuring the assessment of learning goals by AACSB. For example, a learning goal in the accounting program at Marquette University was that students should acquire the knowledge to pass the CPA examination. The results of a survey of Marquette alumni in 1992 rated this as an essential learning goal of the accounting program, ranking it 4.35 on a 5-point scale. The alumni and faculty in a different survey, ranked CPA examination preparation as the fifth most important outcome of the accounting program (Akers et al., 1997).

In many states, the state board of accountancy requires students to complete at least 30 credit hours of accounting beyond principles of accounting and 150 hours of university credit hours to sit for the CPA examination. Therefore, the completion of an undergraduate accounting degree at a university in such a state does not provide the accounting major with the minimum requirements to sit for the CPA examination. Many undergraduate accounting students meet the requirements by taking additional undergraduate courses. Some undergraduates go to work immediately after graduation and accumulate the necessary requirements to sit for the CPA examination several years after graduation. Other undergraduates put off full-time work until after they complete all the requirements to sit for the CPA examination and take the CPA examination prior to working in the profession.

Whether the CPA examination is an accurate measure of student learning is the subject of much debate. Schick (1986) feels that the CPA examination is a good measure of learning achievement. He argues that students should be required to take the CPA examination as a condition of entering the accounting profession because the examination is standardized and designed to measure accounting knowledge and skills, most of which are attained from the learning experiences provided in an accounting program. Thus, the pass rate of first-time candidates provides an indication of the value-added by the learning experience in the accounting program.

In a rebuttal to this paper, Ponemon (1998) argues that the CPA examination is not an adequate indicator of learning achievement in accounting programs. He states that the CPA examination is not a “complete measure of accounting knowledge, aptitude or mastery,” but rather is a “test of certain core competencies in professional accounting and auditing” (p. 421). He points out that many accounting majors do not work in the public accounting profession. Because of this emphasis on passing the CPA examination, Ponemon is concerned that these accounting students may not receive the type of education necessary for success in their chosen field. A recent survey of the managing partners of public accounting firms indicates that accounting graduates are still weak in the areas of creative problem solving skills, oral communication skills, and written communication skills (Ulrich et al., 2003).

Data and Model

During the summer of 2003, questionnaires were mailed to alumni who received a Bachelor of Science Degree during the time period from 1990 to 2003. The questionnaire asked respondents about their first-time performance on the CPA examination and characteristics impacting their performance. Of the 1,285 questionnaires mailed, 189 were returned. Of those returned, 110 of the respondents had taken the CPA examination. Due to incomplete responses, 101 of the 110 questionnaires were analyzed.

The empirical model created for this study contains variables included in a previous study by Brahmasrene and Whitten (2001) as well as variables unique to the characteristics of the analyzed accounting program. For example, the program has open enrollment, so SAT and ACT scores are not required for admission and not included as variables. The logit model can be stated as the following:

+ + + - -
Probability (PART) = f(GPA, FFR, PUB, BACK, SATIS)

The function f represents the logistic function (Greene, 2003), and the sign above the variable name in the equation is the anticipated sign of that variable's coefficient. The variables in the model are defined as follows:

PART

=

1 if a respondent initially passed part of the CPA examination and otherwise 0;

GPA

= =

1 if a respondent's GPA is between 3.5 and 4.0 and otherwise 0;

FFR

=

1 if a respondent took a formal, face-to-face review course prior to initially taking the CPA examination and otherwise 0;

PUB

=

1 if a respondent was employed in public accounting prior to initially taking the CPA examination and otherwise 0;

BACK

=

Respondent's perception that the accounting courses taken provided the necessary background to sit for the CPA examination (Strongly Agree = 1, Agree = 2, Neutral = 3, Disagree = 4, or Strongly Disagree = 5);

SATIS

=

Respondent's satisfaction with the quality of instruction in the accounting courses (Strongly Agree = 1, Agree = 2, Neutral = 3, Disagree = 4, or Strongly Disagree = 5).

 

The variable BACK is included in the analysis to capture the difference in educational background of the respondents to the questionnaire. This specific undergraduate accounting degree requires 24 credit hours of accounting courses beyond principles of accounting and 129 credit hours to graduate. Of the 24 credit hours of accounting courses required, the student is allowed to take one 3 credit hour accounting elective. So the accounting department has limited control over what accounting courses are chosen by our graduates.

The variable SATIS was included in the analysis for two reasons. First, business accreditation standards define answers to similar questions as indirect measures of the assurance of learning. The analysis could therefore examine if this indirect measure is related to alumni performance on the CPA examination. Second, this variable serves to separate the perceived quality of instruction in the accounting program from the respondents' perception of their educational background.

The dependent variable in the logit model is PART, while the other variables are the independent variables. The left column contains the distribution of values for the independent variables when PART = 0, while the right column contains the distribution of values for the independent variables when PART = 1.

Table 1. Descriptive Data

 

Full Sample (N = 101)

 

PART = 0 (N = 48)

PART = 1 (N = 53)

Value

0
1
0
1

 

GPA
34

(71%)

14
(29%)

19
(36%)

34
(64%)

FFR
23

(48%)

25
(52%)

17
(32%)

36
(68%)

PUB
31

(65%)

17
(35%)

34
(64%)

19
(36%)


Value

1

2

3

4

5

1

2

3

4

5

BACK
6

13%

13

27%

17

35%

8

17%

4

8%

11

21%

31

58%

3

6%

8

15%

0

0%

SATIS
14

29%

26

54%

5

10%

2

4%

1

2%

25

47%

25

47%

3

6%

0

0%

0

0%

Table 1 contains descriptive information for the sample variables. The table splits the data into two categories: individuals who initially did not pass any part of the CPA examination and those who passed at least one part. The descriptive data indicate that GPA is strongly related to initial success on the CPA examination. Twenty-nine percent of the unsuccessful candidates have a GPA between 3.5 and 4.0, while 64 percent of the successful candidates have a GPA between 3.5 and 4.0.

he majority of candidates have taken a formal, face-to-face review course prior to initially taking the CPA examination and have pre-examination public accounting experience. As regards the variable BACK, only 40 percent of the unsuccessful candidates think the accounting courses they have

taken provide the necessary background to sit for the CPA examination. On the other hand, 79 percent of the successful candidates consider the accounting courses to be adequate preparation (BACK = 1 or 2). Responses regarding satisfaction with the quality of instruction indicate that 83 percent and 94 percent of the unsuccessful and successful candidates, respectively, are satisfied with the quality of instruction (SATIS = 1 or 2).

Empirical Results

The logit model parameters were estimated using the maximum likelihood procedure. Because of the small size of one of the subsamples, t ratios are reported in Table 2 (Aldrich and Nelson, 1984). The results from the full sample of 101 observations appear in Table 2 in the column on the left, labeled Full Sample (N = 101). Two variables, GPA and BACK, are statistically significant at the 0.05 level, indicating that a high GPA and the proper accounting course background increased the probability of initially passing one or more parts of the CPA examination. The other three variables, FFR, PUB, and SATIS, are not statistically significantly at the 0.10 level.

The logit model yields the probability that a respondent initially passed part of the CPA examination. This table presents the logit model parameter estimates for three different samples. The left column has the parameter estimates for the full sample. The middle column has the parameter estimates for the sample of respondents who worked in an accounting related position prior to initially taking the CPA examination. The third column has the parameter estimates for the sample of respondents who had not worked in an accounting related position prior to initially taking the CPA examination.

The middle column of Table 2 contains the results from analyzing only those respondents who had either worked in public accounting or an accounting-related position prior to the first time that they took

Table 2. Logit Model Parameter Estimates

 

Full Sample

(N = 101)

Worked in Accounting Prior to Initially Taking the CPA Examination

(N = 75)

Did Not Work in Accounting Prior to Initially Taking the CPA Examination

(N = 26)

Variables

Parameter Estimate

(t ratio)

Parameter Estimate

(t ratio)

Parameter Estimate

(t ratio)

Constant

0.922

(1.18)

1.126

(1.19)

2.840

(1.27)

GPA

1.425

(3.11)**

1.171

(2.14)**

3.924

(2.54)**

FFR

0.453

(0.98)

0.970

(1.80)*

-2.605

(-1.51)

PUB

0.163

(0.35)

0.121

(0.23)

NA

(NA)

BACK

-0.524

(-2.07)**

-0.714

(-2.72)**

-0.661

(-1.08)

SATIS

-0.298

(-0.84)

-0.235

(-0.52)

-0.723

(-1.04)

**Significant at the .05 level
* Significant at the .10 level
NA Not Applicable

the CPA examination. In this case, GPA and BACK are statistically significant at the 0.05 level as in the full sample. But now FFR is statistically significant at the 0.10 level, indicating that for subjects with pre-examination accounting work experience, taking a formal, face-to-face review course increases the probability of initially passing one or more parts of the CPA examination.

The third column of Table 2 contains the results from analyzing only alumni who had no accounting related-work experience prior to initially taking the CPA examination. Only the variable GPA is statistically significant at the 0.05 level. The logit model results are consistent with examination of the raw data for this group. This group was equally divided between those alumni who had a GPA = 1 and those who had a GPA = 0. Seventy-seven percent of the alumni who had a GPA = 1 passed at least one part of the CPA examination. Twenty-three percent of the alumni with a GPA = 0 passed at least one part of the CPA examination. Only one of the seven students with a GPA = 0 who took a formal face-to-face review course passed at least one part of the CPA examination. This sample appears to have different characteristics than that of the respondents with pre-examination, accounting-related work experience.

Another estimation procedure for a model with a 0 or 1 dependent variable is the probit model. The function f in the probit model is the cumulative normal distribution. The same variables are statistically significant in the probit models so the results are not presented in this paper.

Table 3 contains the probabilities of initially passing part of the CPA examination given a respondent's characteristics and the marginal impact of taking a formal face-to-face review. This table is constructed from the results in Table 2 for the sample consisting of respondents who worked as accountants prior to initially taking the CPA examination.

Table 3. Probabilities

In order to gain a better understanding of the effect of the statistically significant variables on the probability of passing part of the CPA examination for the respondents who had pre-examination accounting-related work experience, the logit model was re-estimated. The two statistically insignificant variables, PUB and SATIS, and the statistically insignificant constant term from the model were excluded from the analysis.

The marginal impact of an independent variable on the probability in a logit model is dependent upon the values of the other independent variables (Greene, 2003). Table 3 shows the probability of initially passing part of the CPA examination for various combinations of the variables GPA, FFR, and BACK.

The first analysis looks at the impact that taking a formal, face-to-face review course has on the probability of initially passing part of the CPA examination depending upon a respondent's GPA. As shown in Table 3, if GPA = 1, FFR =1, and BACK = 1, the probability of a respondent initially passing one or more parts of the CPA examination is 0.883. If a respondent had a GPA = 1, FFR = 0, and BACK = 1, the probability of initially passing part of the CPA examination is 0.717. The marginal impact of taking the formal, face-to-review course is the difference in probabilities (0.883 – 0.717), which is 0.166.

Using another scenario from Table 3, if a subject has GPA = 0, FFR = 1, and BACK = 1, the probability of passing one or more parts of the CPA examination on the first sitting is 0.638. That probability decreases to 0.373 when the values are GPA = 0, FFR = 0, and BACK = 1. From these two examples, the marginal impact of taking the formal, face-to-face review is 0.638 – 0.373, which is 0.265. In this scenario, the marginal impact is higher than the 0.166 observed in the first scenario. Thus, for this sample, the importance of a candidate taking a formal, face-to-face review course increases as the variable GPA decreases.

Next, the influence of the formal, face-to-face review course on the probability of initially passing one or more parts of the CPA examination depending upon a candidate's educational background is examined. Table 3 shows a respondent whose GPA = 1, FFR = 1, and BACK = 2 has a probability of initially passing part of the CPA examination of 0.818. A respondent with variable values of GPA = 1, FFR = 0, and BACK = 2 has a probability of passing part of the CPA examination of 0.602. The marginal impact of taking the formal, face-to-face review course is 0.818 – 0.602, which is 0.216 and higher than the 0.166 observed in the case where GPA =1 and BACK = 1. Thus, in this case, the marginal impact of taking the formal, face-to-face review course increases as the candidate's perception that the accounting courses taken provided the necessary background to sit for the CPA examination decreases.

Conclusion

The majority of survey respondents have either taken the CPA examination or plan to take it in the future.

For the full sample of those respondents who had taken the CPA examination, the results of the empirical analysis indicate that a high GPA (3.5 to 4.0) and a strong perception that the background provided by the accounting courses is appropriate preparation increases the probability of initially passing one or more parts of the CPA examination. For the subsample consisting of those respondents who had worked in an accounting-related position prior to taking the CPA examination, the variable of taking a formal, face-to-face review course becomes significant. For this subsample, additional analysis shows that the marginal importance of taking a formal, face-to-face review course depends upon GPA. The marginal importance of a formal, face-to-face review course also depends upon the respondent's perception that the accounting courses taken provide the necessary background to sit for the CPA examination. For the subsample consisting of respondents who had no accounting-related work experience prior to initially taking the CPA examination, only a high GPA explains a respondent's performance on the CPA examination. Consistent with the emphasis by AACSB on direct measures of assessment, the indirect assessment measure of satisfaction with quality of instruction has no explanatory power in any of the models. Satisfaction with quality of instruction cannot be utilized as a proxy for the performance of alumni on the CPA examination.

Universities with a mixture of part and full-time students or with open admissions do not have a homogeneous student body. If the characteristics of individuals taking the CPA examination vary from examination period to examination period from those universities, the performance of their alumni on the CPA examination may vary from one examination period to the next examination period due to conditions beyond the control of the accounting department. Because of the interaction between the various factors, the information profile for each individual from that school is necessary to understand the performance of its alumni on the examination. NASBA could help accounting programs that want to utilize the pass rate for assessment by gathering more information about first-time candidates and releasing that information to schools of candidates.

Since many undergraduate alumni of the university surveyed have chosen to meet the 30 credit hours of accounting courses requirement to sit for the CPA examination by taking additional undergraduate courses, it is important that a full range of undergraduate accounting courses are available. The results from this analysis have been utilized in justifying the change in number and mix of undergraduate accounting electives being offered by the accounting program.

When universities do not require students to take professional examinations prior to graduation, students may score low on the exams because they wait several years after graduation to take the exams and may find that they have not taken the appropriate courses to prepare for the exams. For this reason, professional examinations that may be taken prior to graduation are preferable to professional examinations that must be taken after graduation. If the professional examination must be taken after graduation, the program needs to discover what information about the characteristics of their alumni who are taking the professional examination will be released by the professional organization.

*Raymond Isleib, Ph. D., is a Lecturer in Accounting at the University of Houston-Clear Lake , where he teaches financial accounting. **Barry Marks, Ph.D., is a Professor and Chair of Accounting at the University of Houston-Clear Lake , where he teaches government and not-for-profit accounting and individual income tax. ***Douglas Rusth, Ph. D., is an Associate Professor of Accounting at the University of Houston-Clear Lake , where he teaches business ethics and financial accounting.

References

Accounting Education Change Commission (AECC). (1990). Objectives of education for accountants: position statement no. one. Issues in Accounting Education , 5(2), 307-312.

Akers, M. D., D. E. Giacomino and J. P. Trebby. (1997). Designing and implementing an accounting assessment program. Issues in Accounting Education , 12(2), 259-280.

Aldrich, J. H. and F. D. Nelson. (1984). Linear probability, logit, and probit models . Newbury Park , CA : Sage Publications, Inc.

American Accounting Association Committee on the Future Structure, Content, and Scope of Accounting Education. (1986). Future accounting education: preparing for the expanded profession (The Bedford Committee). Issues in Accounting Education, 1(1), 168-193.

AACSB International – The Association to Advance Collegiate Schools of Business (AACSB). (2004). Eligibility procedures and standards for business accreditation . St. Louis , MO : AACSB.

Apostolou, B.A. (1999). Outcomes assessment. Issues in Accounting Education , 14 (1), 177-194.

Brahmasrene, T. and D. Whitten. (2001). Assessing success on the uniform CPA exam: a logit approach. Journal of Education for Business , 77 (1), 45-50.

Greene, W. H. (2003). Econometric analysis . (5th ed.) Upper Saddle River , NJ : Prentice-Hall, Inc.

Hindi, N. and D. Miller. (2000). A survey of assessment practices in accounting departments of colleges and universities. Journal of Education for Business , 75(5), 286-296.

Kimmell, S. L., R. P. Marquette, and D. H. Olsen. (1998). Outcomes assessment programs: historical perspective and state of the art. Issues in Accounting Education , 13(4), 851-868.

Kullberg, D. R., W. L. Gladstone, P. R. Scanlon, J. M. Cook, R. J. Groves, L. D. Horner, S. F. O'Malley and E. A. Kangas. (1989). Perspectives on education: capabilities for success in the accounting profession (The White Paper). New York: Arthur Andersen & Co., Arthur Young, Coopers & Lybrand, Deloitte Haskins & Sells, Ernst & Whinney, Peat Marwick Main & Co., Price Waterhouse, and Touche Ross.

National Association of State Boards of Public Accountancy (NASBA). (2003). CPA candidate performance on the uniform CPA examination . New York , NY : NASBA.

Ponemon, L. A. (1998). Arguments against the CPA exam to gauge accounting program success. Issues in Accounting Education , 13(2), 421-424.

Schick, A. G. (1998). Should undergraduate education in accounting be evaluated, in part, based on graduates' performance on the CPA examination? Issues in Accounting Education , 13(2), 417-420.

Ulrich, T. A., A. R. Michenzi, and W. E. Blouch. (2003) CPAs assess the development of professional skills of recent accounting graduates. Journal of the Academy of Business Education , 4(Spring), 126-137.