«Glossary of Sampling Terms 100% Review Stratum. A stratum of sample items that is selected based on auditor judgment rather than by random means. The ...»
Focused Assessment Program Exhibit 6A
Glossary of Sampling Terms
100% Review Stratum. A stratum of sample items that is selected based on auditor
judgment rather than by random means. The purpose of this stratum is to ensure
adequate coverage of high dollar and/or sensitive items. Unlike random strata, this
stratum is not a subset of a portion of the frame and the audit results for this stratum are
Attribute Sampling. A type of statistical sampling used for compliance testing whereby sample items are evaluated for compliance or attributes. Items either are or are not (yes or no) in compliance. This type of sampling reaches a conclusion on the frequency of occurrence of a particular attribute in a universe.
Attribute Discovery Sampling. A special case of attribute sampling in which the occurrence of a single error constitutes a failure of the universe. This feature, which produces a sample size that is minimal in general, is achieved by ignoring any risk of erroneously rejecting an acceptable universe. This type of statistical sampling provides an objective method of indicating the risk or probability of locating at least one irregularity or characteristic in question.
Block Test. A nonstatistical method of selecting sample items (usually a judgmental or non-statistical sample) in which specific blocks of units are selected. The blocks may be periods of time or consecutive groupings, such as all expense vouchers in June or all invoices with vendor names beginning with the letters M through P.
Clerical Error. Human processing errors (e.g., transpositions, typo’s, etc.). Internal controls should be designed to minimize and catch these (through training, supervision, monitoring, checking, etc.). Isolated clerical errors that slip through despite adequate internal controls designed to prevent and catch them would be nonsystemic, nonrecurring errors. Repetitive clerical errors would be considered to be recurring errors and may be indicative of internal control weaknesses (lack of controls or controls not being followed); in which case they would also be systemic errors.
Clusters. Sample items or units that are made up of clusters or groups of smaller items or units. For example, an ACS (Automated Commercial System) tariff line that is made up several invoice lines, or an invoice line that is made up of several part numbers.
Coefficient of Variation (CV). A measure of dollar dispersion or variability in a frame. It is standard deviation expressed as a percentage (i.e., standard deviation divided by the frame mean multiplied by 100). The higher the CV, the more variation in the frame.
General rules of thumb: a CV 50% indicates low variation and a CV ≥ 50% indicates moderate to high variation.
Confidence Interval (Precision Interval). The range within which the actual error/value in the frame should fall at a given confidence level or assurance. It is also known as tolerance.
Confidence Level. The probability that the true or actual value will be within the corresponding confidence interval. It is sometimes called reliability, assurance, or probability.
Convenience Test. A nonstatistical method of selecting sample items in which convenience is the prime consideration. The most readily available items are selected, without reason or randomness, simply because it is expedient. Records that are in storage, in the bottom of file drawers, not filed or at another location are excluded when this type of testing is used. This method rarely reflects good auditor judgment, can be manipulated by the auditee, and is not recommended.
Critical Error Rate. The maximum universe error rate considered acceptable by the auditor.
Cross-Section Test. A method of selecting sample items in which the auditor attempts to choose items from all parts of the area being tested. It is common under this type of testing to designate a fixed percentage, such as 5%, of items to be selected. Many times the selection is made using a fixed or uniform interval, such as every 10th item, for selection. If this method were used with a random start, the sample generally would meet the selection requirements of a statistical sample. However, it is not uncommon for the auditor, using the cross-section approach, to go through the records and haphazardly select items until the desired quantity is obtained.
Desired Precision (Desired Sampling Error). The amount of sampling error that can be tolerated and still permit the results to be useful.
Dollar Unit Sampling. A type of variable sampling in which the sampling unit is defined as an individual dollar, with each dollar given an equal chance of selection. The selected dollars are then tied to physical units (items or transactions) that are examined.
Error. A sample item in noncompliance with applicable testing criteria (i.e., laws and regulations).
EZ-Quant. A computer program containing statistical analysis audit tools with modules for statistical sampling, regression, and improvement curves. Auditors may use DOSbased Version 3.10 (which combines all modules) or Windows-based Version 1.0.1 (which separates the modules). The two versions do the same analyses, but have different user interfaces and menus for the same procedures.
October 31, 2004 Focused Assessment Program Exhibit 6A Appendix VI EZ-Quant ATTDISC Attribute Discovery Sample Size Procedure. A computer procedure that determines sample sizes for attribute discovery samples. In EZ-Quant DOS Version 3.10, it is call ATTDISC. In EZ-Quant Windows Version 1.0.1, the procedure is selected by choosing Discovery Acceptance in the Attribute Sample Size Development window.
EZ-Quant ATTEVAL1 Attribute Discovery Acceptance Sample Evaluation Procedure. A computer procedure that evaluates the results of an attribute discovery sample by estimating the total error rate in the universe. In EZ-Quant DOS Version 3.10, it is called ATTEVAL1. In EZ-Quant Windows Version 1.0.1, the procedure is selected by choosing Discovery Acceptance, One Step Acceptance, or Rate Estimation in the Attribute Sample Evaluation window.
EZ-Quant DUSAM Dollar Unit Sample Evaluation Procedure. A computer procedure that evaluates the results of a dollar unit sample (i.e., projects the sample results to the frame and provides reliability measures for evaluating that projection). In EZ-Quant DOS Version 3.10, the procedure is called DUSAM. In EZ-Quant Windows Version 1.0.1, the procedure is selected by choosing Variable Sampling and Dollar Unit Sample Evaluation in the initial EZ-Quant window.
EZ-Quant DUSSEL Dollar Unit Sample Selection. A computer procedure that statistically selects dollar unit samples. In EZ-Quant DOS Version 3.10, the procedure is call DUSSEL. In EZ-Quant Windows Version 1.0.1, the procedure is selected by choosing Variable Sampling and Dollar Unit Sample Selection in the initial EZ-Quant window.
EZ-Quant RANUM Random Numbers Generator. A computer procedure that generates random numbers that can then be used to randomly select sample items. In EZ-Quant DOS Version 3.10, the procedure is called RANUM. In EZ-Quant Windows Version 1.0.1, the procedure is selected by choosing Variable Sampling and Generate Random Number/Sets in the initial EZ-Quant window.
EZ-Quant RASEQ Random Number Sets Generator. A computer that generates sets of random numbers that can then be used to randomly select sample items. In EZ-Quant DOS Version 3.10, the procedure is called RASEQ. In EZ-Quant Windows Version 1.0.1, the procedure is selected by choosing Variable Sampling and Generate Random Number/Sets in the initial EZ-Quant window.
EZ-Quant SAMPL Physical Unit Sample Evaluation Procedure. A computer procedure that evaluates the results of a physical unit sample (i.e., projects the sample results to the frame and provides reliability measures for evaluating that projection). In EZ-Quant DOS Version 3.10, the procedure is called SAMPL. In EZ-Quant Windows Version 1.0.1, the procedure is selected by choosing Variable Sampling and Physical Unit Sample Evaluation in the initial EZ-Quant window.
October 31, 2004 Focused Assessment Program Exhibit 6A Appendix VI EZ-Quant STRAT Physical Unit Sample Selection Procedure. A computer procedure that statistically selects physical unit samples and can automatically stratify a frame into equal dollar strata (the number of strata is specified by the auditor). In EZ-Quant DOS Version 3.10, the procedure is called STRAT. In EZ-Quant Windows Version 1.0.1, the procedure is selected by choosing Variable Sampling and Physical Unit Sample Selection in the initial EZ-Quant window.
Frame (Sampling Frame). A physical or electronic representation of the universe from which a sample will be taken. The sampling frame excludes sample items that are separated or stratified for 100% examination.
Frame Validation. The process of verifying that the chosen sampling frame is an adequate representation of that universe it is intended to represent. This typically involves reconciling the frame to the universe, analyzing any differences, and correcting, adjusting, or accepting those differences.
Frame Variability (Homogeneity). Refers to the degree of differences or similarities of items in a frame in terms of dollar amounts and characteristics. Dollar variability can be measured with indices of dispersion (e.g., standard deviation and coefficient of variation). The degree of variability in the frame will directly impact the sample size and need for stratification. The higher the variability, the larger the sample size should be and the greater the need for stratification.
Government Risk (Risk). The tolerable level of risk of accepting a faulty universe (a universe with an actual error rate exceeding the critical error rate). The government bears this risk of a failure to detect flawed conditions. Risk is the complement of confidence level (probability or assurance).
Horizontal Stratification. Stratifying or separating a frame into subgroups according to dollar values or amounts. The idea is that similar size items will have similar size errors.
Horizontal stratification improves sample results (i.e. precision).
Judgmental (Non-statistical) Sampling. See Nonstatistical (Judgmental) Sampling.
Large Dollar Test. A nonstatistical method of selecting sample items in which the largest dollar items are selected. Emphasis is placed on the materiality of the items selected. No examination is made of lesser dollar value items. Conclusions based on the review of the high dollar items may not be applicable to the lesser dollar items.
Also, a breakdown of internal controls is generally more pronounced in the lower dollar items.
Macro Analysis. Any high level analysis not involving the review of individual items or transactions (not sampling). Typically this could include analysis of totals, trends, file comparisons, etc. Macro analysis is a key part of assessing risk exposure but may also be used anytime it will satisfy the audit objectives. It is often more efficient and may be more precise than sampling (micro testing) and therefore should be considered first.
Manual Systematic Interval. The manual application of a statistical sample selection procedure using a random start and a fixed interval to select every nth item.
Micro Testing. Review of individual items or transactions (sampling), usually in order to make conclusions about the population from which they are drawn.
Multistage Sampling. A sampling process involving several stages, in which units at each subsequent stage are subsampled from previously selected larger units. For example: in the first stage, 100 ACS tariff lines are selected, and in the second stage, up to 5 invoice lines are selected for each ACS tariff line. This type of sampling is considerably more complex (in selection and evaluation) than simple or single stage sampling and therefore, is recommended only as a last resort.
Nonrecurring Error. An error that would not be expected to recur in the frame from which the sample was taken. Typically these are nonsystemic, isolated clerical or human errors that occurred despite adequate internal controls (monitoring, checking, training, supervision, etc.). They may also be errors found outside the sampling frame.
The designation of recurring or nonrecurring is required for revenue projection. Only recurring errors are projected. Nonrecurring errors are not projected. However, nonrecurring errors should be added to the projected revenue loss when calculating total revenue loss.
Nonstatistical Projection. A nonstatistical extrapolation of the sample results to the universe, which cannot be evaluated statistically. Evaluating a sample for the purpose of reaching a conclusion about the universe without using the laws of probability.
Nonstatistical (Judgmental) Sampling. Any sampling process in which the sample items are selected subjectively rather than by a random process.
Nonsystemic Error. An error that is not caused by any apparent weakness in internal controls. Typically these are occasional clerical or human errors that happen despite adequate internal controls (monitoring, checking, training, supervision, etc.). Repetitive clerical errors may be indicative of some sort of weakness in the internal controls, such as incompetent personnel, inadequate training, lack of supervision or monitoring, etc.
The designation of systemic or nonsystemic is required for the determination of compliance. Only systemic errors are included in the computation of compliance rates.
Nonsystemic errors are not used when calculating compliance rates.
Physical Unit Sampling. A type of variable sampling in which the sampling unit is defined as a physical unit (item or transaction), with each physical unit having an equal chance of selection (or determinable nonzero chance in the case of stratification).
Point Estimate. A single, specific estimate for a universe characteristic or value.
Population (Universe). See Universe (Population).
Post Audit Stratification. Stratifying the sample and frame after the review is complete and projecting “like to like” in order to produce more accurate projections.