Our visual perception arises from the interpretation of light information, which varies in space, wavelength, and time. It is the latter of these attributes that is explored in this chapter. Subjectively, the world appears to be stable despite continuous changes in the visual scene. How does the visual system respond to and interpret light variations that occur as a function of time?
The duration of a light affects both the ease with which we can see it, as well as its subjective appearance. This chapter emphasizes the first of these issues, that is, how sensitive we are to temporal variations in light and which factors influence our sensitivity. Temporal sensitivity cannot be studied in isolation because other stimulus attributes, such as spatial properties, chromaticity, background features and surround characteristics, all influence our ability to detect temporal variations. In the natural world, most temporal variation occurs through image motion. This may arise from motion of the observer, the eyes, or the object itself. Motion is a special form of temporal variation in which the change with time is associated with a change in spatial position.
This chapter summarizes a number of basic phenomena that describe the sensitivity of our visual system to temporal information. The application of these phenomena to the clinical study of abnormalities of visual processing, such as in disease, is also discussed.
Temporal summation and the critical duration
To detect the presence of something in the visual world, it must be present for a finite period of time. Although a single quantum of light may be sufficient to generate a neural response, multiple quanta are generally required during a short period before the light is reliably seen, a property known as temporal summation. In the human visual system, temporal summation occurs for durations of approximately 40 to 100 milliseconds, depending on the spatial and temporal properties of the object and its background, the adaptation level, and the eccentricity of the stimulus. The maximum time over which temporal summation can occur is the critical duration.
Let’s say we wish to determine how long a light needs to be presented on a dark background to be visible. In general terms, a more intense light does not need to be presented for as long as a less intense one to reach threshold visibility. The relationship between the luminance of the light and the duration of its exposure to reach visibility is linear over a limited range. Provided that the light pulse is shorter than the critical duration, it will be at threshold when the product of its duration and its intensity equals a constant. The formula that describes this time–intensity reciprocity is Bloch’s law :
B t = K
Bloch’s law is shown schematically in Figure 37.1A . When stimulus intensity and duration are plotted on log-log coordinates, as in Figure 37.1A , Bloch’s law describes a line with a slope of −1. When the critical duration is reached, the threshold intensity versus duration function is described by a horizontal line; that is, a constant intensity is required to reach threshold. Bloch’s law has been shown to be generally valid for a wide range of stimulus and background conditions, including both foveal and peripheral viewing. Once the duration of the stimulus exceeds the critical duration, the luminance required for it to reach visibility is classically considered to be constant.
The preceding discussion assumes that whenever the observer’s threshold is exceeded, he or she will respond accurately to the stimulus. This predicts an abrupt and idealized transition between the two curves, as depicted in Figure 37.1A . In the real world, both visual stimuli and the physiologic mechanisms that we use to detect them are subject to random fluctuations in response. We may consider the length of the stimulus presentation to be divided into a number of discrete time intervals. The signal is detected when the response exceeds threshold in at least one interval and the probability of detection in each interval is considered independent. This description of the probabilistic nature of visual detection is known as probability summation over time . The concept of probability summation is included in many models of temporal visual processing and is thought to be at least partly responsible for the less-than-abrupt transition between the region of temporal summation and constant intensity that occurs under some experimental conditions. One experimental situation in which this arises is when threshold contrast is measured as a function of signal duration for sinusoidal grating stimuli. This is illustrated in Figure 37.1B . In Figure 37.1B the upper curve shows threshold duration data for a 0.8 cycle per degree grating, and the lower curve shows those for an 8 cycle per degree grating. The upper curve conforms well to the schematic diagram that is illustrated in Figure 37.1A . For the lower curve depicting results for the 8 cycle per degree grating, however, a gradual transition is observed. In this latter case the actual critical duration (traditionally the point of intercept of the two slopes in Fig. 37.1A ) is somewhat difficult to define. Data similar to those shown in Figure 37.1B , have been interpreted to indicate that the critical duration increases with increasing spatial frequency. Gorea & Tyler use an alternative form of analysis that includes the effects of probability summation; they conclude that the critical duration is minimally affected by spatial frequency.
Factors affecting the critical duration
The time interval defined by the critical duration depends on properties of both the stimulus and the background. The critical duration has been shown to vary with light adaptation level; that is, with brighter backgrounds, the critical duration decreases. Conversely, with dark adaptation, the critical duration increases. Unless dark adapted, the size of the stimulus also affects the critical duration, with larger stimuli having a decreased critical duration. Retinal eccentricity also influences the critical duration, as does the visual task. Temporal summation is also affected by the spectral composition (wavelength or color) of the light stimuli, with isolated chromatic stimuli having longer temporal integration than achromatic (luminance) stimuli. For colored lights, the critical duration decreases with increased chromatic saturation of the background, similar to the decrease in critical duration with increased luminance for achromatic stimuli. Figure 37.2 demonstrates how chromaticity and retinal eccentricity markedly alter the sensitivity to temporal pulses and the critical duration.
Temporal sensitivity to periodic stimuli
The preceding section has considered how the human visual system responds to aperiodic stimuli (for example, a single pulse of light). We will now consider how the visual system responds to periodic stimuli (repeatedly flickering stimuli). Most research in this area has been directed to explore the following questions: (1) what is the fastest flicker rate that can be detected by the human visual system (the critical flicker fusion frequency); and (2) what factors influence sensitivity to flicker slower than this critical rate?
Critical flicker fusion frequency
When a light is turned on and off repeatedly in rapid succession, the light appears to flicker, provided the on and off intervals are greater than some finite time interval. If the lights are flickered fast enough, we perceive the flashes as a single fused light rather than a series of flashes. In simple terms, when the perception of fusion occurs, we have reached the limit of the temporal-resolving ability of our visual system. The transition from the perception of flicker to that of fusion occurs over a range of temporal frequencies; the boundary between the two is called the critical flicker fusion (CFF) frequency. The value of the CFF varies, depending on a large number of both stimulus and observer characteristics. Some of the important factors that influence the CFF ( Box 37.1 ) are discussed in the following sections.
- •
The critical flicker frequency (CFF) describes the fastest rate that a stimulus can flicker and just be perceived as a flickering rather than stable.
- •
While the CFF is dependent on the temporal resolution of visual neurons, it is also considered to be a measure of conscious visual awareness because, at CFF threshold, an identical flickering stimulus varies in percept from flickering to stable. Functional magnetic resonance imaging demonstrates involvement of the frontal and parietal cortex in the conscious perception of flicker.
- •
CFF has been utilized as a measure of conscious visual awareness in a wide range of pharmacological and psychological research studies.
- •
Perimetric tests of CFF have also been developed with most research directed towards visual field assessment in glaucoma.
Effect of stimulus luminance on CFF
In general, the CFF increases as the luminance of the flashing stimulus increases. This relationship is known as the Ferry–Porter law, which states that CFF increases as a linear function of log luminance. The Ferry–Porter law is valid for a wide range of stimulus conditions and is illustrated in Figure 37.3 . The lower curves ( solid lines and symbols ) show data collected in the fovea, and the upper curves show data collected at 35 degrees eccentricity. For both locations the upper curves are for smaller targets (0.05 degree foveally and 0.5 degree eccentrically), and the lower curves are for larger targets (0.5 degree foveally and 5.7 degrees eccentrically). Figure 37.3 demonstrates several interesting observations about the relationship between CFF and luminance. First, the Ferry–Porter law is upheld despite changes in stimulus size. Second, the linear relationship between log luminance and CFF is present for both central and peripheral viewing, although the slope of this relationship increases in the periphery, implying faster processing. The Ferry–Porter law holds not only for spot targets but also for grating stimuli. For scotopic luminance levels, at which rods mediate detection, CFF decreases substantially to approximately 20 Hz and no longer obeys the Ferry–Porter law.
Effect of stimulus chromaticity on CFF
The linear relationship between CFF and log luminance, as described by the Ferry–Porter law, is also valid for purely chromatic stimuli. However, the slope of the relationship has been shown to vary with stimulus wavelength. This relationship is demonstrated in Figure 37.4 , which shows CFF versus illuminance functions derived from four separate studies. In all four studies the foveal CFF illuminance functions are well fit by Ferry–Porter lines, and in all cases the functions for green (middle wavelength) lights were found to be steeper than those for red (long wavelength) lights. The steeper slope for green stimuli has been interpreted as evidence supporting the green cone pathways being inherently faster than the red cone pathways for the transmission of information near the CFF. The CFF is lowest for blue stimuli detected by the short-wavelength pathways.
Effect of eccentricity on CFF
The CFF varies as a function of eccentricity in the visual field. If the stimulus size and luminance are kept constant, the CFF increases with eccentricity over the central 50 degrees or so of the visual field and then decreases with further increases in eccentricity. This is illustrated in Figure 37.5 , which plots the CFF as a function of eccentricity in the temporal visual field.
Effect of stimulus size on CFF: the Granit–Harper law
As shown in Figure 37.3 , the CFF increases with stimulus size. For a wide range of luminances, CFF increases linearly with the logarithm of the stimulus area. This relationship is known as the Granit–Harper law, named after the investigators who first reported it. The Granit–Harper law holds for a wide range of luminances, retinal eccentricities out to 10 degrees, and stimuli as large as almost 50 degrees. However, subsequent investigators have determined that it is not the overall area of the stimulus that is critical, but rather the local retinal area with the best temporal resolution. This was demonstrated by Roehrig, who measured the same value for the CFF for a complete 49.6-degree field in comparison to an annulus of the same diameter, with its central 66 percent not illuminated. Because the midperiphery has better temporal resolution than central vision, an eccentric annulus produced the same CFF as the full 49.6-degree stimulus. The Granit–Harper law does not hold under dim light conditions in which rods mediate performance.
Beyond the fovea, a revised form of the Granit–Harper law is required to fit the changes in CFF with stimulus area. Rovamo & Raninen have shown that the Granit–Harper law can be generalized across the visual field by replacing the retinal stimulus area with the number of ganglion cells stimulated. In this more general case, the CFF increases linearly with the logarithm of the number of ganglion cells stimulated. This is illustrated in Figure 37.6 . Figure 37.6A plots CFF against eccentricity for three different stimulus areas. The CFF decreases with increasing eccentricity irrespective of the stimulus area. Figure 37.6B plots the same CFF data as a function of the number of retinal ganglion cells stimulated at each eccentricity and results in a linear relationship between these two parameters.
Temporal contrast sensitivity
The CFF defines an upper limit for temporal sensitivity ( Box 37.2 ), beyond which we can no longer detect that a light is flickering. How sensitive are we to flicker below the CFF, and how does the visual system respond to more complex temporal variations of light than simple flashes or trains of flashes? Classical work on human temporal contrast sensitivity was performed by De Lange in the 1950s. De Lange evaluated temporal sensitivity to flicker using mathematical analysis of temporal waveforms and linear filter theory.
There are several versions of clinical perimetry that measure variants of temporal contrast sensitivity. These include:
- •
flicker on a pedestal temporal modulation perimetry, such as in the Medmont perimeter
- •
mean modulated contrast sensitivity perimetry
- •
both of these display small spot targets that vary in contrast either about the mean luminance of the perimeter background or above and below a luminance pedestal
- •
- •
spatio-temporal contrast sensitivity is measured in Frequency Doubling Technology Perimetry.
Figure 37.7 shows the results from flicker-sensitivity experiments by Kelly. The vertical axis plots flicker sensitivity, which is the reciprocal of the flicker threshold, measured as a “threshold modulation ratio,” which is the extent that the sinusoidally modulated light deviates from its average direct current (DC) component. The modulation ratio is calculated as a fraction of its DC value because the amplitude cannot be greater than the DC component (negative values of luminance are not physically possible). The modulation ratio expresses the amplitude as a percentage deviation of the stimulus from its average value.
The left panel of Figure 37.7 plots the modulation ratio as a function of flicker frequency. A series of curves are shown, each obtained at a different average retinal illumination or adaptation level. The curves define the flicker detection boundary for the particular level of adaptation. For a given level of adaptation, any combination of frequency and modulation amplitude below the curve is seen as flickering, whereas any combination above the curve is perceived as a steady light. The point at which the curve intersects the abscissa (x-axis) corresponds to the CFF. Note that at low adaptation levels, the shape of the curve is low-pass, meaning that modulation sensitivity is similar for low temporal frequencies and then falls off systematically for higher temporal frequencies. At high adaptation levels, the shape of the curve is band-pass, meaning that modulation sensitivity is greatest for a band of intermediate temporal frequencies and systematically falls off for lower and higher temporal frequencies.
With increasing retinal illuminance, more conditions are seen as flickering, consistent with the simpler Ferry–Porter law. At low frequencies the curves are similar, indicating that low-frequency flicker reaches threshold at a similar value of modulation ratio for all levels of photopic adaptation. This is not the case for high frequencies, at which the threshold is determined by both adaptation level and frequency. At higher luminance levels the sensitivity for flicker detection peaks at frequencies of approximately 15 to 20 Hz. This is similar to the Brücke brightness enhancement effect, which is discussed later.
The right panel of Figure 37.7 replots the data of Figure 37.7A as a function of the absolute amplitude of the high-frequency flicker. This has the effect of reversing the curves so that the amplitude sensitivity is greatest at the lowest adaptation level. It can be seen that the curves of Figure 37.7B approach a common asymptote at high frequencies. This convergence of curves measured at many different luminance levels implies that at high temporal frequencies, sensitivity is predicted by the absolute amplitude of the signal, independent of adaptation level. This behavior is not apparent at the low temporal frequency range.
Chromatic temporal sensitivity
Chromatic temporal contrast sensitivity measured with sinusoidal grating stimuli also varies with retinal illuminance, however there are two key differences when compared to performance measured with achromatic stimuli. Firstly, chromatic functions are generally low-pass, that is, sensitivity is similar for a range of low temporal frequencies before falling off at higher temporal frequencies. Secondly, the high-frequency cut-off is at lower temporal frequencies, that is, a lower CFF is found for chromatic in comparison to achromatic stimuli for a given retinal illuminance.
Spatial effects on temporal sensitivity
The work of De Lange and Kelly describes our temporal contrast sensitivity characteristics, that is, how sensitive we are to temporal sinusoidal variations in stimulus contrast. This does not consider the effect that the spatial characteristics of the light source have on sensitivity. Human contrast sensitivity depends on both the spatial and temporal properties of the stimulus. Figure 37.8 shows a surface plot of the human spatiotemporal contrast sensitivity function, as derived by Kelly, from a large number of psychophysical measurements. One axis of the graph shows the temporal frequency of the stimulus; the other shows the spatial frequency. The height represents the observer’s contrast sensitivity for the particular spatial and temporal conditions. Paths running through the curve parallel to the temporal frequency axis represent the temporal contrast sensitivity functions, and those running parallel to the spatial frequency axis represent the spatial contrast sensitivity functions. The temporal contrast sensitivity function is band-pass at low spatial frequencies, becoming low-pass at high spatial frequencies.
Mechanisms underlying temporal sensitivity
The shape of the human spatial contrast sensitivity function has been explained in terms of a multiple-channel model in which the total curve is the window of a number of channels, each tuned to a different peak spatial frequency. Similar to spatial processing, there is evidence for a discrete number of channels for temporal processing, each tuned to a different peak temporal frequency. There appear to be fewer independent temporal frequency filters than there are filters conveying spatial frequency information. The number of temporal mechanisms identified has been shown to be dependent on the spatial frequency of the stimulus, in addition to the retinal location.
How can we determine whether different mechanisms are governing performance? One method relies on the assumption that different mechanisms result in different subjective representations of the stimulus; that is, when separate mechanisms are governing detection, the stimulus looks different. Kelly and Kulikowski report that flickering gratings at threshold produce one of three percepts depending on the spatiotemporal parameters of the grating: slow flicker, apparent motion, or frequency doubling (i.e. the perceived spatial frequency is twice the actual spatial frequency). This is consistent with three mechanisms underlying temporal processing. Watson & Robson measured temporal discrimination, that is, the ability to determine that stimuli are flickering at different rates. For grating patches of 0.25 cycle per degree, there were three temporal frequencies that were uniquely discriminable, which Watson & Robson interpreted as evidence for three unique filters mediating temporal processing. Mandler & Makous modeled temporal discrimination performance and also found evidence for three mechanisms underlying temporal processing.
An alternative method for identifying mechanisms underlying visual processing is that of selective adaptation. If a stimulus that activates one mechanism to a greater extent than another mechanism is presented, the sensitivity of the activated mechanism will be selectively depressed if the stimulus is viewed for a prolonged period. This should allow other, less sensitive mechanisms to detect the stimulus and hence allow their profile to be explored. Selective adaptation has been widely used to isolate the mechanisms underlying color vision processing. Hess & Snowden used selective adaptation with temporal stimuli to uncover temporal mechanisms; an example of their data is displayed in Figure 37.9 . The first stage of these experiments (not shown in the figure) involved measuring foveal contrast detection thresholds for a wide range of spatial and temporal conditions. These measures were used to set the contrast of the probe stimulus, which was set to be just detectable (4 dB above threshold). The detectability of this probe stimulus was then measured in the presence of masking stimuli of the same spatial frequency but different temporal frequency. The contrast of the masking stimulus was varied until the probe was just visible.
Figure 37.9A presents data for three probes presented foveally with no spatial content (0 cycles per degree) but flickering at either 1, 8, or 32 Hz. The figure shows the contrast sensitivity of the mask plotted against the mask temporal frequency. Three mechanisms are revealed: one low-pass ( red circles ) and two band-pass ( blue circles, green triangles ). Figure 37.9B shows data for a probe of 3 cycles per degree, and only two mechanisms are revealed. The band-pass mechanism, centered on higher temporal frequencies in Figure 37.9A disappears when the spatial frequency content of the stimulus increases. Using a similar method, Snowden & Hess have shown that for retinal eccentricities of 10 degrees, only two mechanisms are found, reducing to a single mechanism at eccentricities of greater than 30 degrees.
More recent evidence suggests that these channels are not as independent as initially thought. Using a masking paradigm, Cass & Alais demonstrated two channels underlying human temporal vision. High-frequency masks were able to suppress low-frequency targets suggesting interaction between temporal channels. The high-frequency channel was orientation invariant, which suggests a precortical origin (as significant orientation selectively of visual neurons is not apparent until V1). In contrast, the low-frequency filter demonstrated orientation dependence, suggesting a cortical origin.
So what is the neural basis of these psychophysically measured temporal channels? Visual information from the retina is carried to the visual cortex, via the lateral geniculate nucleus, by several major neural pathways (magnocellular, parvocellular, and koniocellular). These pathways have been shown to carry largely independent, but sometimes overlapping, visual information. Magnocellular neurons are capable of processing achromatic fast flicker. Parvocellular retinal ganglion cells are well established to be the physiological substrate for red-green chromatic modulation. It is important to note, however, that retinal ganglion cell responses persist at considerably higher temporal frequencies than the human psychophysical CFF. This is particularly the case for chromatic modulation where the human psychophysical function limit is approximately 10–15 Hz, yet parvocellular retinal ganglion cells will respond up to 30–40 Hz. These differences are explained by models where responses from multiple single retinal cells converge on cortical detection mechanisms. The exact manner in which this convergence occurs and how the site of convergence is represented in the visual cortex is an active area of current research.
Surround effects on temporal sensitivity
Our sensitivity to temporal variations depends not only on the properties of the flickering light but also on those of the background surrounding the light. In the dark (scotopic conditions), detection of light increments is mediated by rods, and in the light (photopic conditions), detection is mediated by cones. For flicker sensitivity, it has been shown that when photopic flickering lights are presented on dark backgrounds, interactions between rods and cones (rod–cone interactions) act to decrease sensitivity. The effects of rod–cone interactions on flicker sensitivity are most significant at high temporal frequencies ( Fig. 37.10 ) and in the retinal periphery. Suppressive effects between cone mechanisms of flicker thresholds (e.g. long-wavelength-sensitive cones and medium-wavelength-sensitive cones) have also been demonstrated.
