December 1994

The goal of an agency audit is to insure compliance with the client's work standards, evaluate performance and maximize profits. Obviously, no matter how competent the auditor or how sophisticated the collection software, reviewing each account is a physical impossibility. Even if 100 percent of the information could be tested, the cost of testing would likely exceed the expected benefits (the assurance that accompanies examining 100 percent of the total) to be derived. What is required is a sampling of the accounts.

To accomplish this, the auditor needs to examine a representative sample or cross-section of the various type of accounts (e.g., legal, good telephone, skip, payment arrangements, settled, closed) as well a review of the remittance history.

How the sample should be selected and how large the sample should be are critical issues for researchers as well as auditors.

According to researchers M. Hanson and P. Hauser, in their article "Principles of Sample Design," "The science of sampling design involves: (1) looking at the resources available, the restrictions under which one must work, the mathematical and statistical tools available, the accumulated knowledge of certain characteristics of the populations to be sampled; and (2) putting these together to arrive at the optimum design for the purpose at hand."

Hanson and Hauser point out that the overall criterion that should be applied in choosing a sampling design is to design the sample so that it will yield the desired information with the reliability required at a minimum cost; or conversely, that "at a fixed cost it will yield estimates of the statistics desired with the maximum reliability possible."

Simply stated, a sampling plan is nonstatistical when it fails to meet at least one of the criteria required of a statistical sampling plan. Auditors should know the requirements of statistical plans, because, by definition, any deviation constitutes a nonstatistical approach.

The difference between the two types of sampling is that the sampling risk of a statistical plan can be measured and controlled, while even a perfectly designed nonstatistical plan cannot provide for the measurement of sampling risk.

The basic similarity between the two types is that both sampling approaches require the exercise of auditor judgment during the planning, implementation and evaluation of the sampling plan. In other words, the use of statistical methods does not eliminate the need to exercise judgment.

In addition, the actual audit procedures performed on the items in the sample will be the same, whether a statistical or nonstatistical approach is used. The employment of a statistical plan does not mean the auditor can alter the procedures designed to collect evidence to draw an audit conclusion.

It is up to the auditor to evaluate the individual and situational costs and benefits associated with each sampling approach before making a determination.

In some circumstances, statistical sampling is more appropriate than judgment sampling. Before deciding whether to use statistical or judgmental sampling, the auditor must determine the audit objectives; identify the population characteristics of interest; and state the degree of risk that is acceptable. After making those determinations, it may be advisable to use statistical sampling if the auditor has a well-defined population and can easily access the necessary documentation.

Obviously, if the audit methodology and parameters limit the on-site portion of an agency audit to one or two days, the sample design and size must be a realistic reflection of this time constraint.

It is a fallacy that the "statistical rule of thumb" is to sample 10% of the accounts. There is no such magic number. If the entire population is 10, a 10% sample equals one account -- not very representative. Therefore, it is the absolute numbers, not the percentage, that is important.

Accounts to be reviewed during an audit are normally selected through one of the probability sampling methods -- random, systematic or stratified. Probability sampling provides an objective method of determining sample size and selecting the items to be examined. Unlike nonstatistical sampling, it also provides a means of quantitatively assessing precision (how closely the sample represents the population) and reliability (confidence level, the percentage of times the sample will reflect the population).

In auditing, this method uses sampling without replacement; that is, once an item has been selected for testing it is removed from the population and is not subject to re-selection. An auditor can implement simple random sampling in one of two ways: computer programs or random number tables.

This method provides for the selection of sample items in such a way that there is a uniform interval between each sample item. Under this method of sampling, every "Nth" item is selected with a random start.

This method provides for the selection of sample items by breaking the population down into stratas, or clusters. Each strata is then treated separately. For this plan to be effective, dispersion within clusters should be greater than dispersion among clusters. An example of cluster sampling is the inclusion in the sample of all remittances or cash disbursements for a particular month. If blocks of homogeneous samples are selected, the sample will be biased.

Remember, an essential feature of probability sampling methods is that each element of the population being sampled has an equal chance of being included in the sample and, moreover, that the chance of probability is known. Only in this way, is a probability sample representative of a population.

Some selection methods can be used only with nonstatistical sampling plans.

In this method, the auditor selects the sample items without intentional bias to include or exclude certain items in the population. It represents the auditor's best estimate of a representative sample -- and may, in fact, be representative. Defined probability concepts are not employed. As a result, such a sample may not be used for statistical inferences. Haphazard selection is permitted for nonstatistical samples when the auditor believes it produces a fairly representative sample.

Block selection is performed by applying audit procedures to items, such as accounts, all of which occurred in the same "block" of time or sequence of accounts. For example, all remittances in the month of November. Alternatively, remittances 300-350 may be examined in their entirety. Block selection should be used with caution because valid references cannot be made beyond the period or block examined. If block sampling is used, many blocks should be selected to help minimize sampling risk.

Judgment sample selection is based on the auditor's sound and seasoned judgment. Three basic issues determine which items are selected:

1. Value of items. A sufficient number of extensively worked or older accounts should be included to provide adequate audit coverage.

2. Relative risk. Items prone to error due to their nature or age should be given special attention.

3. Representativeness. Besides value and risk considerations, the auditor should be satisfied that the sample provides breadth and coverage over all types of items in the population.

An agency audit need not be based on a statistical sample to be considered valid. In fact, to concentrate on a statistical selection method is to miss the point of the agency audit. It is more important to be able to identify areas in need of improvement than to identify the standard deviation of the population mean. It is more valid to address issues of concern than calculate the confidence level of the sampling statistic.

An experienced auditor with good judgment and a well-defined audit goal need only review a random cross-section to know if the agency is in compliance and what steps must be taken to improve performance and maximize profits.

Remember the goals of an agency audit:

1. To insure compliance with the client's work standards

2. To evaluate current agency performance; and

3. To maximize profits for both client and agency.

With these goals clearly in view, experienced auditors balance the resource available, the restrictions of each audit, the mathematical and statistical tools available, and his or her accumulated knowledge of the characteristics of the population being sampled and arrive at the optimum audit design for the purpose at hand. Just as Hanson and Hauser recommend.