Psicothema, 1995. Vol. Vol. 7 (nº 3). 667-675

José Manuel Reales y Soledad Ballesteros

The SDT-SP is a computer program which allows to compute parameters from the more used detection theory designs in different experimental settings. A new extension of the program allows to estimate sensitivity as well as bias parameters corresponding to "same/different" designs. Among the ex-perimental designs that can be analyzed with this program are: (a) one interval experiments, (b) rating experiments, and (c) mAFC experiments. Group’s parameters can also be obtained. SDT-SP is also useful in hypothesis testing. The program, written in Pascal, runs in IBMs compatible computers. It is accurate, fast, and easy to use. An User’s Manual and the corresponding software is being published by the Universidad Nacional de Educación a Distancia.

El SDT-SP es un programa que permite calcular un amplio número de parametros a partir de los diseños más utilizados en la investigación sobre detección de señales en diversos ambientes experimentales. Una reciente extensión del programa incluye la posi-bilidad de analizar los resultados provenientes de diseños del tipo "igual-diferente". Además, el programa calcula con rapidez y precisión una serie de parámetros correspondientes a los diseños más utilizados en este campo como son: (a) experimentos de un intervalo; (b) experimentos de valoración; y (c) experimentos de elección forzada entre varias altenativas. STD-SP proporciona también estimaciones de datos grupales y realiza cálculos sobre prueba de hipótesis. El programa está escrito en Pascal y funciona en ordenadores IBMs y compatibles. La Universidad Nacional de Educación a Distancia va a publicar un Manual del usuario junto al correspondiente software.

Since David Green and John Swets (1966) published their classic book entitled Signal detection theory and psychophysics, psychologists have made a wide use of this approach as a way to explain human performance on detection tasks conducted in a noise background. The number, however, of problems to which the theory have been applied is in these days much larger. At the beginning, the common user of signal detection (SDT) was a psychologist studying sensory processes. Today, psychologists interested in perception, memory and cognition, in general, as well as other social and medical scientists are also potential users of the SDT analysis.

The type of problems to which SDT can be applied are numerous, a few of which are:

a) Determining the ability of a person to discriminate whether a familiar object or a photograph has been previously seen or corresponds to something new.

b) Assessing the capacity of a medical student in distinguishing X-rays showing a normal organ from an organ with a tumour.

c) Measuring the ability of a blind person to discriminate between a closed and an open shape haptically explored.

These are only just a few examples to which detection theory can be applied. Detection theory is today a widely used approach to study human performance in a large number of experimental as well as clinical settings.

The signal detection approach has many important advantages over classical psychophysics methods in which sensitivity is confounded with response bias (see Jáñez, 1992; Muñiz, 1991). The main advantage of SDT and Choice Theory (Luce, 1963) is that both provide sensitivity indexes that are unaffected by bias toward a particular response (see MacMillan & Creelman, 1991).

SDT and Luce theory are useful tools in situations in which performance is not perfect and accordingly errors appear. In other situations in which "noise" does not affect performance and a near perfect correspondence between stimuli and response is obtained, these approaches are not useful. However, in situations in which errors arise, SDT as well as Luce’ Choice Theory, are useful approaches to compute and interpret subject’s sensitivity, as well as bias from different experimental designs.

Computations, however, are sometimes rather complex and very time consuming. For this reason, psychologists working in several areas, medical as well as social scientists using detection theory in their work may find useful a comprehensive software that speeds up calculations.

Overview

A first version of the SDT-SP was presented at the 23rd Annual Meeting of the Society for Computers in Psychology that gathered at Washington, DC, United States, in the fall of 1993. A paper has just appeared in the journal Behaviour Research Methods, Instruments and Computers (Reales & Ballesteros, 1994).

In this paper we will present very briefly the main features of the program. We will also show its accuracy comparing SDT-SP results to outputs from a small software program written by Boice and Gardner (1988) and to results from a well-known textbook on detection theory. Finally, we will present the most recent extension of the program prepared to calculate the parameters corresponding to "same-different" designs.

SDT-SP main features

SDT-SP is a computer program written in Pascal 6.0 which runs on IBM-compatible personal computers. The main features of this program are: accuracy, quickness, easiness of use, and comprehensiveness. The program is menu-driven and easy to use. SDT-SP computes descriptive statistics from different experimental designs. It also computes inferential statistical tests that make possible to know about the goodness of an estimate and whether the parameter values differ significantly from 0 or from some other value.

Furthermore, the program allows to plot receiver operating characteristic (ROC) curves for individual as well as for group’s d’.

SDT-SP was prepared for practical purposes related to our ongoing research program on haptic and visual perception and implicit and explicit memory for objects (Ballesteros, 1993; Ballesteros, Manga, & Reales, 1994a,b,c; Ballesteros & Reales, 1992). In these studies we need to com-pute fast and accurately a number of descriptive as well as inferential SDT statistics. The starting point of SDT_SP was the Appendix 6 of MacMillan and Creelman’s (1991) manual for computing d’, c, and ß. From here, a comprehensive program developed in an intent to provide the user with the necessary statistical tools to compute parameters from a large number of experimental designs.

SDT-SP accuracy

A question that naturally arises in relation to any new statistical package is how accurate the computations performed by the program are?

SDT-SP is very accurate in its calculations. Table 1 (top) presents results comparing performance from an"one interval experiment" on haptic perception of symmetry using 3-D unfamiliar wooden objects conducted in our laboratory (Ballesteros et al. 1994b). The leftmost column presents the results obtained with the SDT-SP. The central column contains the results obtained with an independent statistical program (Boice & Gardner, 1988). These data correspond to performance of one of the subjects who participate in our experiment (subject number 2). The second part of Table 1 (bottom) presents the parameters from a "same-different" experiment design computed with SDT-SP (see below). These results are compared to examples provided by MacMillan and Creelman’s (1991, p. 142) textbook, Signal detection theory: A user’s guide (rightmost column). As can be observed from these comparisons, the program’s computations are very accurate. A larger number of comparisons are provided by Reales and Ballesteros (1994. p. 152) including results from significance tests for two as well as for more signal detection parameters (Marascuilo, 1970). All these tests have shown that the program’s accuracy is very high.

How to use the program

Figure 1 (top) shows a printout of the SDT-SP principal menu, showing the available options. The program requires to enter the option number. The user may choose among the available options according to the experimental design and the theoretical assumptions. The available options are: (1) Yes/No Experiments (or one interval discriminations); (2) Nonparametric Analysis; (3) The Rating experiments; (4) Forced-Choice experiments; and (5) "Same-Different" experiments; (6) Setup; and (7) Exit. Fi-gure 1 (middle) shows a printout of the menu with the available options. This menu appears at the screen when option number 1 corresponding to "Yes/No experiments" is selected.

Option number 1 of STD-SP, allows the computation of parameters from one interval experiments. Consider, for example, a situation in which unfamiliar objects are presented under blind conditions and subjects have to judge the objects as "symmetric" or "asymmetric". Any of the four joint events presented in Table 2 can occur in each trial.

The number responses under any event is presented in the corresponding box of the stimulus-matrix (in parenthesis). The correct recognition of a symmetric object is called a hit. The failure to recognize it is called a miss. Recognizing an asymmetric object as "symmetric" is called a false-alarm while correctly responding "asymmetric" to an asymmetric object is called a correct rejection.

The best way to determine the observers sensitivity is to compute a measure of the discrepancy between the hit and the false alarm rate. The following step is to introduce the program the number of hits, false-alarms, misses and correct rejections from the stimulus-response matrix for each subject. Then, the program faster and accurately generates the indexes presented in Table 3:

a) The most important SDT sensitivity index is d’. This index is the distance between the noise and the signal+noise distributions defined in terms of z scores.

b) d’ standard deviation and its confidence interval.

c) α y logα from the choice theory.

d) c, or criterion, is the basic SDT bias index. This is the best index of subject bias because it is statistically independent of d’ (see MacMillan & Creelman, 1991). The program also provides c’s standard deviation and the likelihood ratio, another bias measure.

e) SDT-SP provides the corresponding bias indexes for the choice theory.

Reales and Ballesteros (1994) provided the formulae used by the program to calculate these indexes. SDT-SP allows three options at this point: The first option creates a file in which store this information, the second option presents a plotting of the ROC curve in the computer screen, finally, the third option allows to exit from this menu.

Figure 2 presents the ROC curve corresponding to the detection of symmetry data showed in Table 2. This graph represents the functional relationship between the proportion of hits (in our example, to say "symmetric" when the object is symmetric) and false-alarms (to say "symmetric" when the object is asymmetric). If a print of the graph is wanted, before to enter the program a graphic manager, for example, the GRAPHYCS.COM from the MS-DOS, should be loaded. The type of printer to be used must also be specified. The program allows to obtain a graph in z-coordinates as shown in Figure 3.

Figures 2 and 3 show that the subject’s sensitivity in detecting objects’ symmetry or asymmetry, is quite accurate (d’= 2.1).

The third option of the menu allows to escape the program.

Detection Theory computation for group data is also available from option number 3. SDT-SP used the formula provided by MacMillan and Kaplan (1985, Appendix A) which allows to com-pare the variances of two statistics than estimate d’ (sensitivity) or c (the bias index).

SDT-SP allows also to compute the significance test for hypothesis testing from option 4 (see Reales & Ballesteros, 1994).

The rating experiments

To illustrate the rating experiments we are using an example from the Handbook of perception and human performance (Vol. 1) provided by Falmagne (1986) in the chapter entitled "Psychophysical measurement and theory" (p. 1-40). See Table 4.

Rating experiments are those in which subjects are presented with two stimuli or events and have to indicate their level of confidence that one or the other event is present using a rating scale with several levels. When this option is selected the program asks for the number of possible responses. In our example we enter 6 as this number. The data must be entered from more certainty to less certainty, first the signal trials and then the noise trials. The program, then produced the parameters shown in Table 5.

Pressing just a computer key a receiver operating characteristic (ROC) curve can be presented at the computer screen and a printout can also be obtained in linear coordinates or in z-coordinates. See Figures 4 and 5.

To estimate the parameters of line fitting in rating experiments we have followed the procedure proposed by Churchhause (1981) in the Handbook of applicable mathematics (Vol III). Instead of minimizing the vertical distance between the points and the estimated curve, the distance of the points was minimized from the nearest point of the curve. The reason is that both, the z(F)i and z(H)i are subjected to experimental error. The main advantage of using this procedure is that it takes into account that experimental errors can be obtained in both coordinates.

The "same-different" design

The most recent development in the SDT-SP includes the possibility to analyze data from "same-different" designs.

In a "same-different" experiment the observer is presented in each trial with two stimuli <S1S2>; <S1S1>; <S2S1>; or <S2S2> which may be "same" or "different" and he/she has to classify the pair as "same" or "different". The results can be presented in a 2 x 2 table as shown in Table 6. These data correspond to an example provided by MacMillan and Creelman (1991, p. 142, see Table 1, bottom). Suppose we want to compare the ability of children to discriminate syllables such a /ba/ and /pa/. We create artificial syllables in the speech perception laboratory varying gradually "voice onset time" (the point in which the sound starts to vibrate) from, for example, 0 msec (/ba/) to 60 msec (/pa/) in 20 msec steps (i.e., 0, 20, 40, 60). A way to investigate the listener’s sensitivity to discriminate between pairs of sounds is using a "same/different" design.

If the problem is to discriminate between two sounds (i.e., 0 and 20), the experimenter creates the four different combinations of the sounds (<S1S1>, <S2S2>, <S1S2>, and <S2S1>). The perceiver task is to decide whether each pair of sounds is "same" or "different".

This experimental procedure can be very useful as it does not require an extensive amount of training and it requires very simple judgments. This main characteristics make the task very adequate to be used with young children. As in other options, the program asks for data input. Following the program’s prompts, a Table is obtained (see Table 7). This table shows results for fixed as well as for roving expe-rimental designs. In a fixed experiment only two stimuli are presented in the whole block of trials. In this example, 0 and 20 msec "voice onset time". In a roving experiment, however, the stimuli vary within the block in a continuum scale. In the example on the /da/, /ba/ speech discrimination experiment, four different stimuli must be artificially constructed (S1, S2, S3, and S4), corresponding to 0, 20, 40, and 60 msec "voice onset asynchrony". We are now interested in the assessment of subjects’ sensitivity and bias corresponding to any adjacent pair of sounds. A roving design can use an only block of trials, instead of the three different blocks required by a fixed experimental design.

SDT-SP provides the basic sensitivity and bias indexes corresponding to fixed and roving data (see Table 7). The user can also obtain a printout of the "same/different" ROC curve in linear coordinates following the same procedure described in the "yes-no" experiments section as shown in Figure 6.

Other experimental designs

Other experimental designs that can be analyzed using this statistical program are the 2AFC and the mAFC designs. The mAFC designs are general cases of 2AFC experiments in which two stimulus classes are presented in each trial but the number of response alternatives is m instead of just 2.

Summary and Conclusion

The SDT-SP User’s Manual and the corresponding software is being published by the Universidad Nacional de Educación a Distancia.

SDT-SP is a program fast and accurate, written in Pascal 6.0, that runs in IBM-compatible computers. The program calculates indexes from SDT and Luce’s Choice Theory and allows to plot ROC curves in linear as well as in z-coordinates. It is menu driven, accurate, easy to use and fast.

Ballesteros, S. (May, 1993). Haptic perception and types of memory for raised patterns and three-dimensional objects. Paper presented at the International Conference on Object Representation in Visual and Haptic Systems. Madrid, Spain.

Ballesteros, S., & Reales, J. M. (November, 1992). The perception of symmetric and asymmetric patterns by touch and vision. Paper presented at the 33rd Annual Meeting of the Psychonomic Society, St. Louis, MO.

Ballesteros, S., Manga, D., & Reales, J. M. (1994a submitted). Haptic discrimination of bilateral symmetry in raised unfamiliar displays.

Ballesteros, S., Manga, D., & Reales, J. M. (1994b, submitted). Haptic perception of symmetry in unfamiliar three-dimensional objects: Effects of hand movements and exploration time.

Ballesteros, S., Manga, D., & Reales, J. M. (July, 1994c). Implicit and explicit memory for familiar objects presented haptically. Paper presented at the 23rd International Congress of Applied Psychology, Madrid, Spain.

Boice, R., & Gardner, R. M. (1988). A computer program to generate parametric and nonparametric signal-detection parameters. Bulletin of the Psychonomic Society, 26, 365-367.

Curchhouse, (1991). Handbook of applicable mathematics. Vol. III. New York: Wiley.

Falmagne, J. C. (1986). "Psychophysical measurement and theory". In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.). Handbook of perception and human performance, Vol. 1. New York: Wiley.

Green, D. M., & Swets, J. A. (1966). Signal detection theory and Psychophysics. New York: Wiley. Reprinted 1974 by Krieger, Huntington, NY.

Jáñez, L. (1992). Psicofísica. En J. L. Fernández Tresplacios and P. Tudela (Coor.), Atención y percepción (pp. 1-44). Madrid: Alhambra Universidad.

Luce, R. D. (1963). Detection and recognition. In R. D.

Luce, R. R. Bush and E. Galanter (Eds.), Handbook of mathematical psychology (vol. 1, pp. 103-189). New York: Wiley.

MacMillan, N. A., & Creelman, C. D. (1991). Detection theory: A user’s guide. New York: Cambridge University Press.

MacMillan, N. A., & Kaplan, H. L. (1985). Detection theory analysis of group data: Estimating sensitivity from average hit and false-alarm rates. Psychological Bulletin, 98, 185-199.

Marascuilo, L. A. (1970). Extensions of the significance test for one-parameter signal detection hypothesis. Psichometrika, 35, 237-243.

Muñiz, J. (1991). Introducción a los métodos psicofísicos. Barcelona: PPU.

Reales, J. M., & Ballesteros, S. (1994). SDT_SP, a program in Pascal for computing parameters and significance tests from several detection theory designs. Behavior Research Methods, Instruments, & Computers, 26, 151-155.

Agradecimientos

The research reported in this paper was supported by an UNED predoctoral fellowship to J. M. Reales and by grant PB90-0003 from the DGICYT to S. Balles-teros. This support is gratefully acknowledged.