Our visual perception arises from the interpretation of light information, which varies in space, wavelength, and time. It is the latter of these attributes that is explored in this chapter. The amount of light falling on a particular retinal location may alter over time because a target changes luminance (e.g., a light is abruptly turned on [an aperiodic stimulus], or a light is repeatedly flashing [a periodic stimulus]) or because of image motion (e.g., a steadily illuminated light sweeps over the retinal location). Retinal image motion is extremely common—even for nominally stationary objects—because people’s eyes and heads are commonly moving relative to their surroundings. How does the visual system respond to and interpret light variations that occur as a function of time?
This chapter summarizes several basic phenomena that describe the sensitivity of our visual system to temporal information, including how some of these are influenced by spatial properties, chromaticity, background features, and surround characteristics. The application of these phenomena to the clinical study of abnormalities of visual processing, such as in disease, is also discussed.
Temporal summation and the critical duration for single pulses of light (aperiodic stimuli)
To detect the presence of something in the visual world, it must be present for a finite period. Although a single quantum of light may be sufficient to generate a neural response, multiple quanta are generally required within a particular period before the light is reliably seen, a property known as temporal summation . In the human visual system, temporal summation occurs for durations of approximately 40 to 100 milliseconds, depending on the spatial and temporal properties of the object and its background, the adaptation level, and the eccentricity of the stimulus. The maximum time over which temporal summation occurs is the critical duration .
Let’s say we wish to determine how long a light needs to be presented on a dark background to be visible. In general terms, a more intense light does not need to be presented for as long as a less intense one to reach threshold visibility. Provided that the light pulse is shorter than the critical duration, it will be at threshold when the product of its duration and its intensity equals a constant. The formula that describes this time-intensity reciprocity is Bloch’s law :
Bt = K
Bloch’s law is shown schematically in Fig. 37.1A . When stimulus intensity and duration are plotted on log-log coordinates, as in Fig. 37.1A , Bloch’s law describes a line with a slope of −1. When the critical duration is exceeded, the threshold intensity versus duration function is described by a horizontal line; that is, a constant intensity is now required to reach threshold, irrespective of the stimulus duration. Bloch’s law has been shown to be generally valid for a wide range of stimulus and background conditions, including both foveal and peripheral viewing.
The preceding discussion assumes that whenever the observer’s threshold is exceeded, they will respond accurately to the stimulus. This predicts an abrupt and idealized transition between the two curves, as depicted in Fig. 37.1A . In reality, both visual stimuli and the physiologic mechanisms that we use to detect them are subject to random fluctuations. We may consider the length of the stimulus presentation to be divided into a number of discrete time intervals. The signal is detected when the response exceeds threshold in at least one interval and the probability of detection in each interval is considered independent. This description of the probabilistic nature of visual detection is known as probability summation over time . The concept of probability summation is included in many models of temporal visual processing and is thought to be at least partly responsible for the less-than-abrupt transition between the region of temporal summation and constant intensity that occurs under some experimental conditions. This can be seen in Fig. 37.1B . Here we see the amount of contrast required to detect a grating as initially decreasing with increased stimulus duration, as we would expect from Bloch’s law. However, there is not an abrupt transition to where threshold contrast is independent of stimulus duration, particularly for higher spatial frequency gratings (lower curves). However, Gorea and Tyler concluded that critical duration is minimally affected by spatial frequency once the effects of probability summation were factored in.
Factors affecting the critical duration
The critical duration depends on properties of both the stimulus and the background. At least in part, differences in critical duration reflect differences in neural response latencies for the relevant neural pathways. The latency of a neural response likely sets a limit to critical duration as it is this duration over which temporal summation for a single neuronal response occurs. The critical duration varies with light adaptation level; that is, with brighter backgrounds, the critical duration decreases. Conversely, with dark adaptation, the critical duration increases. Unless dark adapted, the size of the stimulus also affects the critical duration, with larger stimuli having a decreased critical duration. Retinal eccentricity also influences the critical duration, as does the visual task. Temporal summation is also affected by the spectral composition (wavelength or color) of the light stimuli, with purely chromatic stimuli having longer temporal integration than achromatic (luminance) stimuli. For colored lights, the critical duration decreases with increased chromatic saturation of the background, similar to the decrease in critical duration with increased luminance for achromatic stimuli. Fig. 37.2 demonstrates how increased retinal eccentricity markedly reduces the critical duration, although by similar amounts for both the “red-green” and S-cone chromatic pathways. Note Fig. 37.2 plots the inverse of the threshold (i.e., sensitivity).
Temporal sensitivity to periodic stimuli
In contrast to the aperiodic stimuli considered in the previous section (e.g., single pulses of light), periodic stimuli have luminance changes over time that repeat at a given frequency. Most research examining periodic stimuli has been directed to explore the following questions: (1) What is the fastest flicker rate that can be detected by the human visual system (the critical flicker fusion frequency [CFF]); and (2) What factors influence sensitivity to flicker slower than this critical rate?
Critical flicker fusion frequency
When a light is turned on and off repeatedly in rapid succession, the light appears to flicker, provided the on and off intervals are greater than some finite time interval. If the lights are flickered fast enough, we perceive the flashes as a single fused light rather than a series of flashes. In simple terms, when the perception of fusion occurs, we have reached the limit of the temporal-resolving ability of our visual system. The transition from the perception of flicker to that of fusion occurs at the critical flicker fusion (CFF) frequency . The value of the CFF varies, depending on characteristics of the stimulus and the observer. Some of the important factors that influence the CFF are discussed in the following sections.
Effect of stimulus luminance on critical flicker fusion
In general, the CFF increases as the luminance of the flashing stimulus increases. This relationship is known as the Ferry-Porter law , which states that CFF increases as a linear function of log luminance. The Ferry-Porter law is valid for a wide range of stimulus conditions and is illustrated in Fig. 37.3 . The lower curves ( solid lines and symbols ) show data collected in the fovea, and the upper curves show data collected at 35 degrees eccentricity. For both locations the upper curves are for smaller targets (0.05 degrees foveally and 0.5 degree eccentrically), and the lower curves are for larger targets (0.5 degrees foveally and 5.7 degrees eccentrically). Fig. 37.3 demonstrates several interesting observations about the relationship between CFF and luminance. First, the Ferry-Porter law holds despite changes in stimulus size. Second, the linear relationship between log luminance and CFF predicted by the Ferry-Porter law is present for both central and peripheral viewing, although the slope of this relationship increases in the periphery, implying faster processing. The Ferry-Porter law holds not only for spot targets but also for grating stimuli. For scotopic luminance levels, at which rods mediate detection, CFF decreases substantially to approximately 20 Hz and no longer obeys the Ferry-Porter law.
Effect of stimulus chromaticity on critical flicker fusion
The linear relationship between CFF and log luminance (the Ferry-Porter law) is also valid for purely chromatic stimuli. However, the slope of the relationship has been shown to vary with stimulus wavelength. This relationship is demonstrated in Fig. 37.4 , which shows foveal CFF versus illuminance functions from four separate studies. In all four studies the CFF illuminance functions were well fit by Ferry-Porter lines, and in all cases the functions for green (middle-wavelength) lights were found to be steeper than those for red (long-wavelength) lights. The steeper slope for green stimuli has been interpreted as evidence supporting the green cone pathways being inherently faster than the red cone pathways for the transmission of information near the CFF. The CFF is lowest for blue stimuli detected by the short-wavelength pathways.
Effect of eccentricity on critical flicker fusion
The CFF varies as a function of eccentricity in the visual field. If the stimulus size and luminance are kept constant, the CFF increases with eccentricity over the central 50 degrees or so of the visual field and then decreases with further increases in eccentricity. This is illustrated in Fig. 37.5 , which plots the CFF as a function of eccentricity in the temporal visual field.
Effect of stimulus size on critical flicker fusion: the Granit-Harper law
As shown in Fig. 37.3 , the CFF increases with stimulus size. For a wide range of luminances, CFF increases linearly with the logarithm of the stimulus area. This relationship is known as the Granit-Harper law , named after the investigators who first reported it. The Granit-Harper law holds for a wide range of luminances, retinal eccentricities out to 10 degrees, and stimuli as large as almost 50 degrees diameter. However, subsequent investigators have determined that it is not the overall area of the stimulus that is critical, but rather the local retinal area with the best temporal resolution. This was demonstrated by Roehrig, who measured the same value for the CFF for a complete 49.6-degree field in comparison to an annulus of the same diameter, with its central 66% not illuminated. Because the midperiphery has better temporal resolution than central vision, an eccentric annulus produced the same CFF as the full 49.6-degree stimulus. The Granit-Harper law does not hold under dim light conditions in which rods mediate performance.
Outside 10 degrees eccentricity, a revised form of the Granit-Harper law is required to fit the changes in CFF with stimulus area. Rovamo and Raninen have shown that the Granit-Harper law can be generalized across the visual field by replacing the retinal stimulus area with the number of ganglion cells stimulated. In this more general case, the CFF increases linearly with the logarithm of the number of ganglion cells stimulated. This is illustrated in Fig. 37.6 . Fig. 37.6A plots CFF against eccentricity for three different stimulus areas. The CFF decreases with increasing eccentricity irrespective of the stimulus area. Fig. 37.6B plots the same CFF data as a function of the number of retinal ganglion cells stimulated at each eccentricity and results in a linear relationship between these two parameters.
Temporal contrast sensitivity
The CFF defines an upper limit for temporal sensitivity, beyond which we can no longer detect that a light is flickering. How sensitive are we to flicker below the CFF, and how does the visual system respond to more complex temporal variations of light than simple flashes or trains of flashes? Classical work on human temporal contrast sensitivity was performed by De Lange in the 1950s, who mathematically analyzed temporal waveforms and employed linear filter theory.
Fig. 37.7 shows the results from flicker-sensitivity experiments by Kelly, who continued the analytical approach pioneered by De Lange. The stimulus was a very wide (68 degree diameter), uniform field, with indistinct edges. The vertical axis of the left panel indicates the “threshold modulation ratio,” which is the extent that the sinusoidally modulated light deviates from its average (the direct current [DC] component). A modulation ratio of one means the stimulus modulates over time between zero luminance and two times the average, which is the maximum modulation possible. The axis is reversed so that higher points on the axis indicate higher sensitivity to flicker (i.e., only a very small modulation ratio is required for the flicker to be detected).
A series of curves are shown, each obtained at a different average retinal illumination or adaptation level. The curves define the flicker detection boundary for the particular level of adaptation, with the flicker rate specified by its temporal frequency in cycles per second (cps), or hertz (Hz). For a given level of adaptation, any combination of frequency and modulation amplitude below the curve is seen as flickering, whereas any combination above the curve is perceived as a steady light. The point at which the curve intersects the abscissa (x-axis) corresponds to the CFF. Note that at low adaptation levels, the shape of the curve is low pass, meaning that modulation sensitivity is similar for low temporal frequencies and then falls off systematically for higher temporal frequencies. At high adaptation levels, the shape of the curve is bandpass, meaning that modulation sensitivity is greatest for a band of intermediate temporal frequencies and systematically falls off for lower and higher temporal frequencies.
With increasing retinal illuminance, more conditions are seen as flickering, consistent with the simpler Ferry-Porter law. At low frequencies the curves are similar, indicating that low-frequency flicker reaches threshold at a similar value of modulation ratio for all levels of photopic adaptation. This is not the case for high frequencies, at which the threshold is determined by both adaptation level and frequency. At higher luminance levels the sensitivity for flicker detection peaks at frequencies of approximately 15 to 20 Hz. This is similar to the Brücke brightness enhancement effect, which is discussed later.
The right panel of Fig. 37.7 replots the data of Fig. 37.7A as a function of the absolute amplitude of the high-frequency flicker (rather than this amplitude expressed as a fraction of the average luminance, as given on the left ). This has the effect of reversing the curves so that the amplitude sensitivity is greatest at the lowest adaptation level. The curves of Fig. 37.7B approach a common asymptote at high frequencies. This implies that, at high temporal frequencies, sensitivity is predicted by the absolute amplitude of the signal, independent of adaptation level. This behavior is not apparent at the low temporal frequency range.
Chromatic temporal sensitivity
Chromatic temporal contrast sensitivity measured with sinusoidally modulating stimuli also varies with retinal illuminance, but there are two key differences when compared with performance measured with achromatic stimuli. First, chromatic functions are generally low pass, that is, sensitivity is similar for a range of low temporal frequencies before falling off at higher temporal frequencies. Second, the high-frequency cutoff is at lower temporal frequencies, that is, a lower CFF is found for chromatic in comparison to achromatic stimuli for a given retinal illuminance.
Spatial effects on temporal sensitivity
The work of De Lange and Kelly describes our sensitivity to temporal sinusoidal variations in stimulus contrast but does not consider the effect that the spatial characteristics of the light source have on sensitivity. This can be investigated by using grating stimuli—rather than spatially uniform spots—the contrast of which can be modulated over time. When this is done, contrast sensitivity is found to depend on both the spatial and temporal properties of the stimulus. Fig. 37.8 shows a surface plot of the human spatiotemporal contrast sensitivity function, as derived by Kelly, from a large number of psychophysical measurements. One axis of the graph shows the temporal frequency of the stimulus; the other shows the spatial frequency. The height represents the observer’s contrast sensitivity for the particular spatial and temporal conditions. Paths running through the curve parallel to the temporal frequency axis represent the temporal contrast sensitivity functions for gratings of a particular spatial frequency, and those running parallel to the spatial frequency axis represent the spatial contrast sensitivity functions for gratings of a particular temporal frequency. The temporal contrast sensitivity function is bandpass at low spatial frequencies, becoming low pass at high spatial frequencies.
Mechanisms underlying temporal sensitivity
The shape of the human spatial contrast sensitivity function has been explained in terms of a multiple-channel model in which the overall function actually represents the summed sensitivity of a number of separate channels, each sensitive to a comparatively narrower range of temporal frequencies and tuned to a different peak spatial frequency. The possible physiologic basis for these channels is considered at the end of this section. Similar to spatial processing, there is experimental evidence for a discrete number of channels for temporal processing, each tuned to a different peak temporal frequency. However, there appear to be fewer independent temporal frequency filters than there are filters conveying spatial frequency information. The number of temporal mechanisms identified has been shown to be dependent on the spatial frequency of the stimulus, in addition to the retinal location.
How can we determine whether different mechanisms are governing performance? One method relies on the assumption that different mechanisms result in distinct subjective representations of the stimulus; that is, when different mechanisms are responding to the stimulus, the stimulus looks different. Kelly and Kulikowski report that flickering gratings at threshold produce one of three percepts depending on the spatiotemporal parameters of the grating: slow flicker, apparent motion, or frequency doubling (i.e., the perceived spatial frequency is twice the actual spatial frequency). This is consistent with three mechanisms underlying temporal processing. Further support for multiple mechanisms comes from studies of temporal discrimination , or the ability to determine that stimuli are flickering at different rates. For grating patches of 0.25 cycles per degree, Watson and Robson found there were three temporal frequencies that were uniquely discriminable, which they interpreted as evidence for three unique filters mediating temporal processing. Mandler and Makous modeled temporal discrimination performance and also found evidence for three mechanisms underlying temporal processing.
An alternative method for identifying mechanisms underlying visual processing is that of selective adaptation . Adaptation is where sensitivity is depressed when a mechanism is exposed to a stimulus for a protracted period, with selective adaptation highlighting that mechanisms will have their sensitivities reduced in proportion to their individual sensitivities to the adapting—also called the masking —stimulus. Selective adaptation has been widely used to isolate the mechanisms underlying color vision processing. Hess and Snowden used a selective adaptation approach to examine temporal processing with temporal stimuli to uncover temporal mechanisms; an example of their data is displayed in Fig. 37.9 . The first stage of these experiments (not shown in the figure) involved measuring foveal contrast detection thresholds for a wide range of spatial and temporal conditions. These measures were used to set the contrast of the probe stimulus, which was set to be just detectable (4 dB above threshold). The detectability of this probe stimulus was then measured in the presence of masking stimuli of the same spatial frequency but different temporal frequency. The contrast of the masking stimulus was varied until the probe was just visible.
Fig. 37.9A shows the contrast sensitivity (the reciprocal of contrast) of the mask plotted against the mask temporal frequency, for foveally presented stimuli of no spatial content (i.e., 0 cycles per degree). Three mechanisms are revealed by the probe stimuli: one low pass ( solid circles ) and two bandpass ( open circles, solid triangles ). Fig. 37.9B shows data for a probe of 3 cycles per degree, and only two mechanisms are revealed. The bandpass mechanism, centered on higher temporal frequencies in Fig. 37.9A , disappears when the spatial frequency content of the stimulus increases. Using a similar method, Snowden and Hess have shown that for retinal eccentricities of 10 degrees, only two mechanisms are found, reducing to a single mechanism at eccentricities of greater than 30 degrees.
More recent evidence suggests that these channels are not as independent as initially thought. Using a masking paradigm, Cass and Alais demonstrated two channels underlying human temporal vision. High-frequency masks were able to suppress low-frequency targets, suggesting an interaction between temporal channels. The high-frequency channel was orientation invariant which suggests a precortical origin, given that significant orientation selectively of visual neurons is not apparent until area V1 in the brain. In contrast, the low-frequency filter demonstrated orientation dependence, suggesting a cortical origin.
So, what is the neural basis of these psychophysically measured temporal channels? Visual information from the retina is carried to the visual cortex, via the lateral geniculate nucleus, by several major neural pathways (magnocellular, parvocellular, and koniocellular). These pathways have been shown to carry largely independent, but sometimes overlapping, visual information to the cortex, although significant convergence arises at the cortical level. Magnocellular neurons are capable of processing achromatic fast flicker, with neuronal temporal contrast sensitivity data resembling the form of that of human observers. Parvocellular retinal ganglion cells are well established to be the physiologic substrate for red-green chromatic modulation. It is important to note, however, that retinal ganglion cell responses persist at considerably higher temporal frequencies than the human psychophysical CFF. This is particularly the case for chromatic modulation in which the human psychophysical function limit is approximately 10 to 15 Hz, yet parvocellular retinal ganglion cells will respond up to 30 to 40 Hz. These differences are explained by models in which responses from multiple single retinal cells converge on cortical detection mechanisms. The exact manner in which this convergence occurs and how the site of convergence is represented in the visual cortex is an ongoing area of debate.
Surround effects on temporal sensitivity
Our sensitivity to temporal variations depends not only on the properties of the flickering light but also on those of the background surrounding the light. In the dark (scotopic conditions), detection of light increments is mediated by rods, and in the light (photopic conditions), detection is mediated by cones. For flicker sensitivity, it has been shown that when photopic flickering lights are presented on dark backgrounds, the dark-adapted rods surrounding the stimulus act to decrease cone-mediated flicker sensitivity via suppressive rod-cone interactions. These suppressive rod-cone interactions are most significant at high temporal frequencies (see Fig. 37.10 ) and in the retinal periphery. Suppressive effects between cone mechanisms of flicker thresholds (e.g., long wavelength–sensitive cones and medium wavelength–sensitive cones) have also been demonstrated.
