Shading and the Landmarks of Relief

Shading is a visual artist’s tool. It enables the indication of “landmarks” interior of the outline of shapes. Shading triggers behavioral responses in organisms throughout the animal kingdom and even aﬀects the habitus of plants. Radiometry might be expected to account for the phenomenology. We derive the formal structures of shading that are expected to play a dominant role in perception. That they fail to do so suggests that shading is more of an interface template than a “cue”. This ﬁts the artistic use as a “releaser” very well. Pre-modern artists hardly acknowledge causal relations between various photometric variables. Their works show an eﬀective use of various elements on their own right, without attempts at causal congruity. Modern art often deﬁes physics on purpose. We identify manifest templates and relate these to conventional techniques in the visual arts.


Ovoid shading
The paradigmatic case of ovoid shading (figures S1 and S2) involves the sphere. In Cartesian coordinates {x, y, z} one has: x = cos ϕ sin ϑ, (1) y = sin ϕ sin ϑ, were denotes the radius. Since spheres of all sizes are similarly shaded, we may as well set = 1. We let ϑ = 0 denote the direction of the incident beam. Lambert showed that the surface illumination is proportional to the scalar product of the light direction and the surface normal, that is cos ϑ = z. Since Figure S1: Two examples from treatises on drawing by George Bridgman (left 1962[1932], right 1961[1924]). At left an application of ovoid drawing, at right ribbon drawing (called "mouldings" by Bridgman). Both are conventional methods in common use. They are of a complementary nature, although often used in a single drawing.
for a white Lambertian surface the intensity of the scattered beam incident on the pupil does not depend upon the viewing direction (this is "Lambert's Law" proper), that is all the analysis that is required.
The loci of equal illumination are thus simply z = constant, implying that there is one maximum at ϑ = 0 and one minimum at ϑ = π. The minimum will not be visible, since the hemisphere z < 0 is in body shadow, for the illuminance cannot be negative. In computer graphics it is usual to add an "ambient term", thus using 1 2 (1 + cos ϑ) instead of cos ϑ for the illumination. Here the "1" is the ambient term, it neutralizes the negativity of the shading. But of course this is nonsense from a physical perspective, since adding something positive does not change the negativity of the original contribution cos ϑ for ϑ > π 2 at all. But notice that Thus the sum can be rewritten as a product. Although one summand may become Figure S2: At left a white sphere illuminated by direct sunlight from above. The point where the beam hits the sphere head on receives the highest illumination, whereas the "southern hemisphere" is wrapped in body shadow, because the beam cannot reach it. At right two views of a white sphere illuminated by a very extended ("diffuse") source, such as an overcast sky. With such illumination all points of the sphere receive some light and there is both a lightest and a darkest point. Here the "posterization" renders the zones of equal tone more readily visible.
negative, the factors in the product are both positive. Indeed, the product makes physical sense, whereas the sum does not. A possible physical interpretation of the product is that one factor is due to the vignetting of a diffuse source, the other the Lambert cosine factor for the effective local direction of incidence (see figure S3). Although the factors are formally identical, their physical meanings are quite distinct. In the equality (4) the two sides are only formally equal, whereas they are physically very different. Physical identity is not mathematical equivalence. This interpretation is exactly the solution for an extended source like the overcast sky. It shows that one does not have a case of true "shading" there, most SFS algorithms don't apply.
Here the {x, y} are Cartesian coordinates-somewhere in the plane, whereas εz(x, y) denotes the articulation, that is the orthogonal deviation from the reference plane z(x, y) = 0. We included the factor ε for convenience, it is an overall measure of the degree of articulation. For the moment one should perhaps simply assume that ε = 1 and ignore it. By taking the factor ε as infinitesimal we can conveniently study various degrees of approximation. The Euclidean surface (inner) normals are Figure S3: At left the schematic geometry of the case of a white Lambertian sphere viewed against an infinite luminous halfspace (e.g., like an overcast sky). The point P is viewed against the light and might be expected to be in body shadow. However, it is "illuminated from behind": the average light vector of the beam that illuminates P indeed comes from behind. Notice that the point P is only illuminated by part of the extended source because the sphere itself occludes the major part, an instance of vignetting. At right an impression of what is seen in this case. Since the sphere is viewed against the light it appears darkish. However, light "creeps around the form" and illuminates the outer parts from behind. In photographs the contours are often "lost".
where z x = ∂z(x, y)/∂x, z y = ∂z(x, y)/∂y. Let the direction of the radiant beam be T (ϑ) = {cos ϑ, 0, − sin ϑ}, thus a direction in the XZ-plane, the "plane of incidence" (figure S4). We defined ϑ such that a "striking" irradiation corresponds to ϑ = 0. Then the irradiance caused by a beam of unit intensity is Treating both ϑ and ε as infinitesimal, the gray level G(x, y) distribution becomes Notice that there are no second order terms. The simplest approximation is linear in both ϑ and ε, the next higher order terms being 3 rd order.
The "ribbon" interpretation is obtained by fixing the Y-coordinate (to y 0 say). The dependence on z x (x, y 0 ) and invariance with respect to z y (x, y 0 ) implies that the shading reveals a "bending" of the ribbon, but fails to reveal any "twist" (figure S5). In examples shown in treatises on painting technique such twists are indeed generally ignored, that is to say, the artist intuitively approximates the surface to be shaded as a twist-free ribbon, that is a general cylinder, much as a Bridgman "moulding". This yields the all important rule of thumb: The gray tone along the ribbon is proportional with the obliquess. Figure S4: The geometry of "ribbon shading". The "plane of incidence" contains the direction of the beam as well as all surface normals along the ribbon, that defines the notion of a "ribbon along the direction of illumination". The shading then is taken proportional to the obliquity of the surface, that is the complement of the angle subtended by the direction of the beam and the local surface normal. This is essentially what Bridgman illustrates in figure S1 right. Figure S5: At left a surface. At center it has been cut into ribbons. Notice that these ribbons are both twisted and a bended. At right the twist has been taken out. These strips are purely bended. Notice that they fail to "mesh" to a smooth surface. Shading due to illumination in the strip direction only reveals the bending, it fails to reveal any twist.
There is a base gray level ϑ, which, of course, is just a constant level that could be anything (except zero!) in radiant units. It is irrelevant to our analysis. We assume that |z x | ≤ ϑ/ε, which can always be obtained by adjusting ϑ appropriately. Thus no part of the surface will be in (body) shadow.
In this approximation the gray tone is simply proportional to z x , which-in the lowest approximation-is again simply the obliqueness of the local surface in the X-direction, which was taken to be in the plane of incidence.
Notice that there will not be an extremum of the type found in the case of ovoid shading, because "head on" illumination simply does not occur.
We obtain so called "ribbon shading" by simply ignoring the Y-dependence of I(x, y), say we set y = y 0 in equation (8). The biologically important "cue" will perhaps be the gradient relative to the base level ϑ, that is (Notice that ε/ϑ has a finite values, even when both ε and ϑ are both infinitesimal.) This yields another important rule of thumb: The tonal contrast along the ribbon is proportional with the curvature.
This is what Bridgman's "moulding method" attempts to impress on the art student. It is indeed a very intuitive rule that can be used on the gut level. Notice that the tonal contrast depends only on the ratio ε/ϑ, this is the "basrelief ambiguity" (Belhumeur et al., 1999).
The categorical difference with ovoid shading should be clear.

Depth and its ambiguities
The rule of thumb derived above yields a cue that only yields the surface obliquity in a chosen direction up to an arbitrary factor and an arbitrary additive constant. Thus, an inference z(x, y) is just as valid as an inference z (x, y) defined as where ζ denotes an unknown overall offset, {σ x , σ y } an unknown spatial attitude of a reference plane and µ an unknown measure of articulation. This is an ambiguity that is well documented in phenomenology of pictorial space. In the exact formalism of machine vision one meets with an ambiguity of essentially (up to order two) the same form, the so called "bas-relief ambiguity" (Belhumeur et al., 1999).
The linear expression given here is very likely a useful approximation to the kind of transformations applied by the human psychogenesis of visual awareness.
3 The basic shape classes 3.1 The 0 th order In the 0 th order the ambiguity is simply where ζ is arbitrary. Thus there is no way to use the shading cue to estimate the absolute distance of a relief to the eye. Visual awareness will decide on some distance that makes sense in some overall sense.

The 1 st order
In the 1 st order the ambiguity is simply where {σ x , σ y } and µ are arbitrary. Thus there is no way to use the shading cue to estimate the spatial attitude of a relief with respect to the viewing direction.
Nor is there any way to estimate the "depth of relief". Visual awareness often comes up with a default frontoparallel impression. The depth of relief often stays undecided. There is no need to decide, unless one requires the observer to expressly estimate it. This leads to very fluctuating results.

The 2 nd order
Neither the 0 th nor the 1 st order give rise to contrast. In a very real sense they are not even registered by biological optical sensors.
The 2 nd order gives rise to contrast though, a uniform gradient of tone. Here things start to be interesting.
The elementary 2 nd order case is that of the general quadric surface z(x, y) = 1 2! a xx x 2 + 2a xy xy + a yy y 2 , which leads the gray tone distribution (equation 8) G(x, y) = ϑ + ε (a xx x + a xy y) .
The first thing to notice is that the coefficient a yy does not occur, thus it is in no way "specified" by the shading cue. It needs to be suggested by a different cue, or has to be hallucinated. This is of major importance in mainstream research, which is singularly focussed on the second order relief approximation. This important observation is illustrated in figures S6 through S9. These are exact numerical shadings, but they show precisely the behavior predicted by our coarse approximation. Apparently the approximation is "good enough". Moreover is has the advantage of simplicity which makes it easy to implement in a biological system. Figure S6: The case of the "cap". Left, the height contour plot of the surface. (red is more, blue is less.) Center, a contour plot of the gray tone. Right, habitus with shading for a particular elevation.
Since ϑ is typically unknown, the actual shading cue is the gradient of the gray tone pattern, modulo an arbitrary factor (the factor is absorbed in the contrast), that is to say: ∇G(x, y) ∝ {a xx , a xy },  Notice that this is-with respect to shading-equivalent to the cup. Figure S9: Another case of a "saddle". Here the direction of incidence is along an "asymptotic line", an equal height direction. The ribbon along this direction is purely twisted. Left, the height contour plot of the surface. Center, a contour plot of the gray tone. Right, habitus with shading for a particular elevation. Notice that a ribbon at right angles to the direction of incidence is also purely twisted and that this leads to an illumination gradient.
that is essentially the direction of the gradient with respect to the plane of incidence, the angle between them being arctan(a xy /a xx ). Thus, given the plane of incidence (that is the XZ-plane) one may infer the ratio a xy /a xx , whereas a yy remains fully undetermined. If the direction of incidence is unknown, a linear gradient is fully compatible with any variety of quadric, convexity, concavity, or saddle. For each of these shapes one can find a direction of incidence for which it occurs.
Thus the shape cue is rather uninformative in this case, some finite "curvedness" being the formal inference. Yet-and this should really be astonishing-this case is the most common in main stream research.

The 3 rd order
The 3 rd order describes the generic instances of surface inflections. It gives rise to singular "landmarks" of the tonal pattern due to shading.
Here one expects the shading cue to reveal surface specific features. Thus affordance provided by the 3 rd order might possibly explain the importance of the shading cue. In this case we assume a relief that is composed of both 2 nd and 3 rd order terms-the 0 th and 1 st order terms are irrelevant anyway (the reader may want to verify that they fall out from the equation at the very first steps). Thus we set z(x, y) = 1 2! a xx x 2 + 2a xy xy + a yy y 2 + (16) 1 3! a xxx x 3 + 3a xxy x 2 y + 3a xyy xy 2 + a yyy y 3 + (17) leading to the gray tone distribution (apply equation (8)) G(x, y) = ϑ + ε a xx x + a xy y + 1 2! a xxx x 2 + 2a xxy xy + a xyy y 2 .
We notice right away that the coefficients a yy and a yyy fail to occur, so they remain completely ambiguous. (Please remember that -as everywhere in these derivations -the direction of incidence is in the XZ-plane, thus-in the projectionalong the X-axis.) The condition for a singular point is (G x ≡ 0) ∧ (G y ≡ 0), thus we have the simultaneous constraints a xx + a xxx x + a xxy y ≡ 0 (20) a xy + a xxy x + a xyy y ≡ 0, which are equivalent to x = a xy a xxy − a xx a xyy a xxx a xyy − a 2 xxy (22) y = a xx a xxy − a xy a xxx a xxx a xyy − a 2 xxy , This implies, as anticipated, that we need to set a xx = a xy = 0 in order to place the singularity at the origin. This is a convenience, since it greatly simplifies all formal expressions. It is assumed understood from here. Notice that we assume a xxx a xyy − a 2 xxy = 0, but this is required anyway, as will be shown below. This again allows us to simplify the expression for the surface relief and leaves us with the relevant part z(x, y) = 1 3! a xxx x 3 + 3a xxy x 2 y + 3a xyy xy 2 , which leads to the gray level gradient ∇G(x, y) ∝ {a xxx x 2 + a xxy y, a xxy x + a xyy y}.
From this we obtain the Hessian of the gray tone pattern as the Jacobian of the gradient, that is H ∝ a xxx a xyy − a 2 xxy .
If the Hessian is positive the singular point will be an extremum, otherwise a saddle. We may forget about the case H = 0 as non-generic. Notice that the cubic has a factor x. Dividing it out we obtain the quadric form z(x, y) ∝ a xxx x 2 + 3a xxy xy + 3a xyy y 2 , which has the discriminant D ∝ 3a 2 xxy − 4a xxx a xyy . When this discriminant is positive the cubic has three real roots and is a "monkey saddle", if it is negative it has only single real root and is a "shoe surface" (see below).
It is immediate to verify that the surface is a monkey saddle in the case of a saddle and a shoe surface in the case of an extremum.
A typical case of a monkey saddle (figure S10 right) is and of a shoe surface (figure S10 left) is z(x, y) ∝ 1 6 (x 3 + 3xy 2 ).
The monkey saddle and the shoe surface fully characterize the generic cases. Of course, we have ignored numerous special non-generic cases, but these will not be of relevance in realistic situations, since they do not survive even infinitesimal perturbations. Figure S10: At left the shoe surface and at right the monkey saddle, in both cases with some additional curvature (irrelevant, but it affects the habitus) at right angles to the direction of illumination. The drawn line indicates the plane of incidence. At left one sees a shading extremum (dark blotch), at right a shading saddle. These are exact numerical simulations. Our coarse approximation evidently suffices fully to predict the shading topology of the shading patterns. (Of course, a formal analysis shows that the approximation captures all that is necessary to capture the qualitative structure.)

A gray tone saddle, the monkey saddle surface
In the case of the monkey saddle the gray tone pattern is given by which is indeed a saddle since there are two directions of zero equi-illumination, namely x = ±y (figure S11). For all other directions the illumination increases quadratically, that is very slowly, with the distance to the singular point. Figure S11: The case of the "monkey saddle" (monkeys need a saddle that not only accommodated the legs, but also a tail). Left, the height contour plot of the surface. Center, a contour plot of the gray tone. Right, habitus with shading for a particular elevation.

A gray tone saddle, the shoe surface
In the case of the shoe surface z(x, y) ∝ 1 6 (x 3 + 3xy 2 ), the gray tone pattern is given by which is indeed an extremum (figure S12). The loci of equal gray tone are circles concentric with the singular point. The illumination increases quadratically, that is very slowly, with the distance to the singular point. Figure S12: The case of the "shoe surface" (adding the irrelevant a yy coefficient leads to something that looks like the instep of a shoe). Left, the height contour plot of the surface. Center, a contour plot of the gray tone. Right, habitus with shading for a particular elevation.

How good are the rules of thumb?
The predictions of our coarse approximation are fully borne out, as they should, of course, in exact numerical simulations. The approximation serves fine in all cases of daily interest. It requires no computations at all and allows flexible, intuitive thinking. Of course, this requires a formal understanding (as supplied here) as a base. The mathematical expressions can be forgotten once the quantitative relations are clear though. That is because the rule of thumb reduces the physics to simple intuitions.

The experiment
The formal sheets used in the experiments are shown in figures S13 through S15. The only difference is that in the actual sheets the language was Italian.
now draw your own cross sections! Figure S14: Sheet 2.
Here are four pictures, describe how they appear to you (some people say: "a flat blotch", "no idea!", "sphere", "a hole", "view of a curved surface", … ).
But anything goes, it is up to YOU! Remember: YOU are always right, because YOU are the only person who knows what YOU see! Do use at least one, but not more than ten words.