Continuous Functions and Parallel Algorithms on Concrete Data Structures

We report progress in two closely related lines of research: the semantic study of sequentiality and parallelism, and the development of a theory of intensional semantics. We generalize Kahn and Plotkin's concrete data structures to obtain a cartesian closed category of generalized concrete data structures and continuous functions. The generalized framework continues to support a defi nition of sequential functions. Using this ccc as an extensional framework, we define an intensional framework — a ccc of generalized concrete data structures and parallel algorithms. This construc tion is an instance of a more general and more widely applicable category-theoretic approach to intensional semantics, encapsulating a notion of intensional behavior as a computational comonad, and employing the co-Kleisli category as an intensional framework. We discuss the relationship between parallel algorithms and continuous functions, and supply some operational intuition for the parallel algorithms. We show that our parallel algorithms may be seen as a generalization of Berry and Curien\s sequential algorithms.


Introduction
In this paper we present progress in two closely related themes of research in programming language semantics. The first concerns the semantic study of sequentiality and parallelism, and the second is the development of a general theory of intensional semantics in which one may give a variety of semantics to a language, at differing levels of intensional detail, and establish natural relationships between the meanings of terms at each level.
There has been much work on the search for a semantic characterization of sequential computation. Since the conventional continuous functions semantic model contains inherently parallel functions, such as parallel-or, a suitable definition of sequential functions is a necessary pre-requisite in the search for a natural (i.e., syntax-and language-independent) fully abstract semantic model for sequential programming languages such as PCF [Plo77,Mil77,BCL85,Sto88].
A general definition of sequential functions has been given by Kahn and Plotkin in the restricted setting of concrete data structures [KP78]. Berry and Curien have shown, however, that concrete data structures are not closed under any of the continuous function space, stable function space or sequential function space; as a consequence, concrete data structures do not form a ccc when the morphisms are taken to be any of the continuous functions, stable functions or sequential functions [BC82]. To date, no sequential extensional model has been found.
Our first contribution is the definition of a new class of generalized concrete data structures, introduced in section 2. Essentially, the generalization consists in adding a poset structure to the cells of a concrete data structure; the original Kahn-Plotkin concrete data structures correspond to cases where the cell poset is discrete. We show that generalized concrete data structures are closed under the continuous function space, and form a ccc with continuous functions as morphisms. The states of a generalized concrete data structure, ordered by set inclusion, form what we call a generalized concrete domain. Every generalized concrete domain is also a Scott domain, but the converse is false. We define distributive generalized concrete data structures, a generalization of the deterministic (or stable) concrete data structures, and we show that they form a full sub-ccc of the category of generalized concrete data structures. We also sketch the construction of a ccc of distributive gCDSs and stable functions, obtained by varying the notion of a state.
The generalized concrete data structures continue to support a definition of sequentiality, so that we have significantly expanded the setting where sequential functions may be identified. We believe that the category of generalized concrete data structures and continuous functions is the first non-trivial ccc in which one may identify the sequential functions between any two objects. The identity function on a generalized concrete data structure is sequential, and the sequential functions between generalized concrete data structures are closed under composition. We do not know yet if the set of sequential functions between two generalized concrete data structures itself forms a generalized concrete data structure, so we do not claim (yet) to have produced a satisfactory sequential extensional model.
The failure of concrete data structures to support an extensional semantic model has led Berry and Curien to define an intensional semantic model: a cartesian closed category of deterministic concrete data structures and sequential algorithms [BC82,Cur86]. A sequential algorithm may be seen as a sequential function paired with a sequential computation strategy.
The appeal of intensional semantics lies in making it possible to use semantic methods to reason about a broader range of properties of programs. Traditionally, the denotational semantics approach focuses on the extensional aspects of programs, and abstracts away all intensional details; other tools must be used to reason about intensional properties. By employing a different level of abstraction that retains intensional information about programs (at a level appropriate to the task at hand), one should be able to use an intensional denotational semantics to reason about the intensional aspects of programs, such as laziness and complexity (see for instance [Col89] for a potential application).
One of our initial goals in this study has been the definition of a richer intensional semantic model by generalizing Berry and Curien's sequential algorithms between concrete data structures to parallel algorithms. Our thesis is that, by analogy with the characterization of sequential algorithms, a parallel algorithm should correspond to a continuous function paired with a parallel computation strategy. A previous attempt was our "query model" of parallel algorithms [BG90]; although this work has generated some useful insights, it was only partly successful in providing the desired generalization of sequential algorithms, since we were unable to equip this model with a satisfactory categorical structure. Our continued efforts to generalize sequential algorithms have led to the progress reported herein.
We have been able to formalize the construction of an intensional semantic framework, given an extensional semantic framework and a notion of intensional behavior [BG91]. In accordance with this approach, we use here the terms "extensional" and "intensional" as relative terms -they serve to identify different levels of abstraction. Category-theoretically speaking, the extensional framework is a ccc C, the intensional behavior is defined by a computational comonad T over C, and the derived intensional framework is the co-Kleisli category Cj of C and T. We remark that if C is a ccc and T preserves products then CT is also a ccc [See89]. This construction is quite general, and completely divorced of the concrete data structures setting where we first observed its applicability. We believe that this approach can serve as the basis for the development of a rather general theory of intensional semantics.
However, since there is no suitable extensional ccc with concrete data structures as objects (and some class of functions as morphisms), the desired parallel generalization of sequential algorithms cannot be obtained by a direct application of the co-Kleisli construction. Instead, we move to the setting of generalized concrete data structures, which does not suffer from these limitations. Using the ccc of generalized concrete data structures and continuous functions as an extensional framework, we define in section 3 a simple and intuitive notion of intensional behavior based on the idea that a computation is a sequence of incremental evaluation steps. We encapsulate this notion in the definition of a comonad of paths, and we use the co-Kleisli construction to obtain a ccc of generalized concrete data structures and parallel algorithms. We discuss the relationship between the intensional and extensional categories, by showing that every algorithm determines a continuous input-output junction and that every continuous function is the input-output function of some algorithm. This shows that a parallel algorithm may indeed be viewed as a continuous function paired with a computation strategy. Although we do not give a formal definition of the notion of computation strategy, we do supply some operational intuition.
In section 4 we show how our parallel algorithms on generalized concrete data structures generalize Berry and Curien's sequential algorithms on concrete data structures. We define an embedding function that takes each Berry-Curien algorithm to its analogue in our model, which may be thought of as a degenerate parallel algorithm that operates sequentially.
We conclude by outlining a number of directions for further work.
In this paper we do not present the details behind the co-Kleisli construction and the related category-theoretic development. Instead we focus directly on the specific case at hand. For an exposition in more general terms, with full explanations of the relevant category-theoretic definitions and results, we refer the reader to [BG91], which also contains a detailed exploration of the relationships between extensional and intensional semantic models that may be defined within the frameworks described here. • A countable set V of values.
The set of events must be upwards-closed with respect to the cell ordering: if (c,v) G E and c < c' then (c', v) G E.
• An enabling relation h between finite sets of events and cells.
The enabling relation must be upwards-closed with respect to the cell ordering: if y h c and c < c l then y h c'.
The enabling relation defines a precedence relation <C over cells: c <C c' iff y U {(c, i;)} h c' for some v and y. We require that the precedence relation be well-founded.
Let M, M', ete., denote gCDSs in the following discussion. • We say that a cell c is filled in a set y of events iff (c, v) G 2/ f°r some v; we write F(y) for the cells filled in y. If y 1 h c we say that y' is an enabling of c. A cell c is enabled in y iff there exists an enabling y f C y of c. We write this as y' h y c, and we let E(y) be the set of cells enabled in y. A cell is accessible from y iff it is enabled in y but not filled; we let A(y) = E(y) \ F(y). A cell is initial iff it is enabled by the empty set of events.
Definition 2.2 A state of M is a set x C E with the following three properties: • Functional: if (c, vi),(c, V2) G x then v\ = V2.
• Safe: every cell filled in x has an enabling in x.
• Upwards-closed with respect to the cell ordering: if (c,v) G x and c < c' then (c',v) G ^.
Equivalently, this property may be stated as the requirement that x = up(V), where up is the upwards-closure operation over sets of events: We write V(M) for the poset of states of M, ordered by set inclusion. We say that this is the domain generated by M. We refer to the domains generated by generalized concrete data structures as generalized concrete domains. •

Example 2.3
The gCDS Null has no cells, values, events or enablings. It has a single state, the empty set.
The gCDS Two has a single cell *, which is initial and may be filled with the value *. It generates (a domain isomorphic to) the two point domain, with states 0 and T = {(*,*)}.
The gCDS Bool has a single cell b, which is initial and may be filled with either of the values tt or ff. It generates (a domain isomorphic to) the usual boolean domain, with states 0, {(b,tt)} and {(b,ff)}.

3
The gCDS Vnat has the natural numbers as cells, ordered discretely. Each cell may be filled with the value *. The cell 0 is initial, and for every k, {(&,*)} h H 1. The domain X>(Vnat) is isomorphic to the vertical ordering of the natural numbers (i.e., n < n + 1), with a limit point added at infinity: an integer n corresponds to the state {(&,*) | k < n}, and u corresponds to the state {(&,*) | k G IN}. We may use the integers and u to denote the states of Vnat.

Generalized Concrete Domains
We now give a partial domain-theoretic characterization of the generalized concrete domains. Not all Scott domains are generalized concrete domains. This is because all generalized concrete domains have property (Qj), the uniqueness part of property (Q) enjoyed by CDSs [KP78].
For x and y elements of a domain D, we say that y covers x iff x < y and there is no z such that x < z < y. We say that a domain D has property (Q») iff: It follows that V(M) has property (Qj). •

The Continuous Functions Category
We define the category gCDScont with gCDSs as objects and continuous functions between V(M) and V(M') as the morphisms between M and A/'. Composition is taken to be function composition, and the identity morphisms are just the identity functions. An equivalent category is the category of generalized concrete domains and continuous functions, a full sub-category of the category of Scott domains and continuous functions. We now show that gCDScont is cartesian closed. The gCDS Null is a terminal object in gCDScont.
The product construction is a straightforward generalization of the product for concrete data structures [Cur86]. We write c.i for the pair (c,i), where c is a cell and i is a tag -we use 1 and 2 as tags for the product. For a set of cells C and a set of events y, we write C.i and y.i for {c.i | c G C} and {(c.i,v) \ (c,v) G y}, respectively. We build the product of two gCDSs by taking a disjoint union of the two posets of cells, of the two sets of events, and of the two enabling relations.
Definition 2.6 The product of two gCDSs A/ a and A/ 2 is defined by:

Proposition 2.12 The domain V(M -M') is isomorphic to the continuous function space between V(M) and V(M'), ordered pointwise.
The isomorphism is given, for a 6 V(M M f ) and f : Given the isomorphism, it is clear that the morphisms from M to M' may equivalently be taken to be the states of M -* A/'. Since application is continuous and currying is well behaved, it is clear that M -? M' is in fact an exponentiation object for M and M' in the category gCDScont.

2.3
Distributive gCDSs Definition 2. 15 We extend the cell ordering to events as follows: We then extend the ordering to finite sets of events by: y < y' iff there exists a bijection y\ h x c and c then there exists y such that y t -< y, for / = 1,2.
• This property is a generalization of the stability and determinism properties of concrete data structures [Cur86], and similar results follow. We recall that Berry and Curien's sequential algorithms model was limited to deterministic CDSs.

Proposition 2.17 If M is distributive then the gib in V(M) of any two consistent states is their intersection.
Proof: Let M be a distributive gCDS, and let x\ and X2 a consistent pair of states of M. Clearly x\ D x 2 is their gib as sets of events, so we only need to show that it is a state. The intersection clearly preserves functionality and upwards-closure. To show that it preserves safety, let c € F(xi fl x 2 ). For i = 1,2, c G F(x,-), so that there exists yi \RXI c, and therefore yi HRIUAR2 c * Now, by distributivity there exists y such that, for i = 1,2, yi < y, and by upwards closure y h Xt c, and, finally, y \-XLNX2 C -It follows that x\ fl 2*2 is a state. • Definition 2.18 A consistently-complete poset is distributive iff for all x and all consistent pairs

Proposition 2.19 If M is distributive then V{M) is distributive.
Proof: An immediate corollary of 2.17. •

Proposition 2.20 The category of distributive gCDSs and continuous functions is a full subcategory of gCDScont, and it is cartesian closed.
Proof: Null is distributive and product and exponentiation preserve distributivity.
To see that the exponentiation preserves distributivity, let a G V(M -± M') and assume that, is the value on x of the continuous function corresponding to a (by the isomorphism of proposition 2.12). By distributivity of M' there must be an enabling

{(CJJVJ)
I 1 < j < 1} HA(X) c', where / = l\ = l 2 , and for j < /, v'j = v\ -and c\ -< c'--without loss of generality assume that the bijections are identities. But now {(xc'-^v 1 -) \ 1 < j < 1} serves as an upper-bound in the extended cell ordering of the two enablings of xc f in a, so that we may conclude that M -A/' is distributive. •

Relationship to Original Definition
Proposition 2. 21 Kahn and Plotkiris original definition of concrete data structures and their generated domains [KP78,Cur86] can be obtained by considering gCDSs with a discrete cell ordering (i.e., c < c' iff c -c').
Proof: Under the discreteness assumption all upwards-closure requirements are vacuously satisfied, and our definition collapses to the original definition. • Note that the gCDS product preserves discreteness: if the cells of M\ and M2 are ordered discretely, then so are the cells of their product. As a corollary, the gCDS product is a conservative extension of CDS product.
Importantly, the exponentiation does not preserve discreteness; even if M and M' are discrete, Pfi n (M) will not be discrete in general, so that the cells of the exponentiation will not be ordered discretely. This is of course necessary for our purposes, since (discrete generalized) concrete data structures are not closed under continuous function space.
The intuition behind the introduction of an ordering on cells may perhaps be explained thus: in the concrete data structures setting, a cell corresponds to a flat domain -a choice between a number of (mutually inconsistent) ways to increase information. An appropriate domain may be "decomposed" into such atomic choices. The notion of cell may itself be seen as a generalization of an argument position, the notion used by early approaches to defining sequentiality [Vui73,Mil77]. Once we introduce an ordering on cells, it is possible to talk not only of a discrete choice between alternatives for a given cell, but also of the extent to which the choice must be pursued. This seems to be essential if higher-order domains are to be represented using this approach to decomposition.
The concrete domains are the domains generated by concrete data structures (or, equivalently, discrete generalized concrete data structures). Kahn and Plotkin's representation theorem characterizes concrete domains as Scott domains satisfying a number of axioms. In particular, concrete domains satisfy axiom (I): (I) Every finite element dominates finitely many elements.
But the continuous function space does not, in general, preserve property (I), and this is the key to Berry and Curien's proof that concrete domains and continuous functions do not form a cartesian closed category [BC82]. Our generalization of concrete domains must not, in general, satisfy axiom (I). See example 2.14 for a continuous function space and a gCDS that violate (I).

Stable Functions on gCDSs
We have concentrated so far on continuous functions, and defined a ccc gCDScont of gCDSs and continuous functions. Other classes of functions may be considered, by varying the definitions of a state and the domain generated by a gCDS. We will now introduce the category gCDSstab of distributive gCDSs and stable functions, a full sub-ccc of dl-domains and stable functions. First, a few definitions are needed. For stable functions /, g from D to E, f is below g in the stable ordering iff / is pointwise below g and, for each d 6 D and e < /(d), M(/,d,e) = M(</,d,e).
• A dl-domain is a distributive Scott domain that has property (I). It is well known that the category of dl-domains and stable functions is a. ccc. See [Ber78] for a fuller treatment, as well as alternative (but equivalent) definitions of stability and the stable ordering.
We qualify the states introduced so far as being ct-states, and use V ct (M) for the domain of ctstates of M, ordered by set inclusion -we call this the ct-domain generated by M. (In particular, our partial domain-theoretic characterization of generalized concrete domains only applies to the ct-domains). We now define the "stable states" of a gCDS.

Definition 2.23
A st-state of M is a set of events x C EM with the following three properties: • Safe: every cell filled in x has an enabling in x.
• Stable: if c\ and c 2 are filled in x and C\ and c 2 have an upper bound in the cell ordering, be the domain of st-states of Af, ordered by set inclusion -we say that this is the st-domain generated by M. • The difference between ct-states and st-states amounts to the replacement of the upwardsclosure requirement of ct-states by a "stability" condition.

Proposition 2.24 For a distributive M, V st (M)
is a dl-domain. The empty set is the least element, and the lub of an upper-bounded or directed set of states is given by its union. The finite elements (i.e., isolated elements) are states that are finite sets of events.
We now emulate the development carried out above for gCDScont.

Definition 2.25
The category gCDSstab has distributive gCDSs as objects, and the stable func- Seen from a different angle, inclusion on st-states corresponds to the stable ordering on stable functions, while inclusion on ct-states corresponds to the pointwise ordering on continuous functions.
Note that for discrete gCDSs the stability requirement is vacuously satisfied, as is the upwardsclosure requirement, so that the ct-domain V ct (M) and the st-domain T> st (Af) coincide when M is a discrete gCDS. The two notions diverge, however, on gCDSs with non-trivial cell ordering (such as exponentiations). Moreover, the classes of (distributive) ct-domains and st-domains are incomparable -we have shown that property (I) holds for st-domains, but is violated by ct-domains; on the other hand, property (Qt) holds for ct-domains, but not for st-domains. We will not delve deeper here into the category gCDSstab, and we will consider exclusively gCDScont and ct-domains in the remainder of the development. However, we point out that most of the ensuing development may be carried out with gCDSstab replacing gCDScont as the extensional framework. Many more properties remain to be checked. We are currently investigating whether the set of sequential functions between two gCDSs itself forms (the states of) a gCDS. If so, we might finally obtain a ccc of gCDSs and sequential functions. Even if this fails with the above definition of sequentiality, we may be able to generalize the definition to take more explicit account of the cell ordering (while collapsing onto the original definition when the cell ordering is discrete). We are also trying to discover whether, following the general approach exemplified by the construction of gCDScont and gCDSstab, one may define a notion of "sequential state" and use this to generate a third kind of domain from a gCDS, ideally to yield a class of domains closed under the sequential function space.

Parallel Algorithms on Generalized Concrete Data Structures
In this section we present the category gCDSalg of gCDSs and parallel algorithms, using paths as a notion of intensional behavior with respect to gCDScont. We do not present the construction in its full generality -this may be found in [BG91], where a similar construction is carried out over the category of Scott domains and continuous functions. The path domain over M is isomorphic to the continuous function space from Vnat to M, ordered pointwise. Yet another equivalent way of viewing paths is as infinite non-decreasing sequences of states of M, ordered componentwise. We work freely with the different representations of paths, omitting explicit mention of the isomorphisms. We write, e.g., ti for the application'of (the function corresponding to) the path t to (the state of Vnat corresponding to) the integer leaving the various isomorphisms implicit.
We will use paths over M to represent computations over M. Events are regarded as quanta of information produced by the computation, so that ti is the information known about the computed value by time point i + 1, starting with no information at all at time point 0. The ordering of paths may be viewed as comparing paths by their eagerness: t C iff for every z, ti C t'i, i.e., for every (c, v) G ti we also have (c,v) £ t'i. Informally, / C /' if the computation represented by t' computes everything that / computes, and each event in / occurs no later than it does in It is easy to verify that map / and map itself are continuous.

t)i,(T 2 t)i).
In other words, there is a uniform way of converting back and forth between a pair of computations and a computation of a pair.
Definition 3.5 For each A/, define the following: Intuitively, val^// is the value computed by t\ pre M t is the computation built from the prefixes of t; and pathA/x is the constant path to x, regarded as a canonical "degenerate" computation of x. The first three identities assert that (P,val,pre) is a comonad over gCDScont. The remaining two assert that (P, val, pre, path) is a computational comonad (in the sense of [BG91]). An additional inequality that stems from the choice of canonical computations is idpM < pathMovalM.

The Category of Algorithms
Definition 3.7 The category gCDSalg has gCDSs as objects, and continuous functions from PM to A/' as the morphisms from M to M'. We define an algorithm to be a morphism in this category, and we The identity algorithm from M to M is val^/.
• In words, the algorithm composition of a and a' applies a' to the computation produced by mapping a over the prefixes of the argument computation. The identity algorithm disregards everything except the value computed by its argument, and it returns this value. Algorithm composition is a continuous function on algorithms. It is straightforward to verify, using the algebraic identities given earlier, that gCDSalg is indeed a category: with these definitions composition is associative and the identity algorithm is a unit for composition. In fact, gCDSalg is just the co-Kleisli category of gCDScont and the comonad (P, val.pre) [ML71,BG91]. For clarity and ease of comparison with the underlying category, we use for morphisms in gCDSalg, and we retain -* for morphisms in gCDScont.
Since gCDScont has finite products, it is easy to show that: Intuitively, algorithm application disregards the computation of the algorithm being applied, and is only concerned with the actual algorithm and the computation of its argument. Currying and uncurrying are simple adaptations of the standard currying and uncurrying operations on functions to take account of the structure of paths. The fact that P preserves product is used crucially here.
Putting these results together, we have: Proposition 3.10 gCDSalg is cartesian closed.
See [BG91] for a more detailed category-theoretic treatment from which these results follow.
Example 3.11 Figure 4 presents a partial Hasse diagram of the algorithm space from Two x Two to Two. We present each algorithm by its action on the paths of Two x Two in figure 3, a shaded circle for a result of T (with actions on other paths determined by monotonicity). We present below the operational intuition behind these algorithms. •

Relating the Categories
We define the input-output function fun(a) of an algorithm a, and the canonical algorithm alg(/) for a continuous function /. These turn out to be the morphism parts of a pair of functors between the two categories.  Thus, alg / has / as its input-output function, and every continuous function between gCDSs is the input-output function of some algorithm between gCDSs. M', WE SA Y that a\ input-output approximates a 2 , written ai < 10 a 2 iff FUN a! pointwise approximates FUNA 2 , i.e., FUN a\ < FUNA 2 -We say that a\ and a 2 are input-output equivalent, written a\ -l0 a 2 , iff FUN(AI) = FUN(a2). • In words, two algorithms are input-output equivalent iff they have the same input-output function; this is the equivalence relation induced by the input-output approximation pre-order.

Proposition 3,16
If fx < f 2 then alg/i < alg/2. If a\ < a 2 then ci\ < l° a 2 , but the converse is not generally true.
This indicates that the pointwise ordering on algorithms (as continuous functions) takes into account intensional aspects of algorithms that are disregarded by the input-output approximation ordering.

Proposition 3.17 The quotient of the domain V(M => M') by input-output equivalence is isomorphic to the domain V(M -• M'), with the isomorphism induced by fun and alg:
Consider now the input-output equivalence class of algorithms that share an input-output function /. Since idpjv/ < pathA/ovalA/ we have idv/^AP < algofun. That is, the canonical algorithm alg / is maximal among the algorithms with input-output function /. Intuitively this means that the canonical algorithm is the "laziest" algorithm with input-output function /; it provides a result based solely on the input value, independent of the way in which the value is computed. This may be contrasted with the behavior of the algorithm minalg/, defined by .

f{tO).
It is easy to see that fun o minalg = idA/__A//, but minalgofun < ida/=>a/', and therefore minalg/ is the least algorithm with input-output function /. Intuitively, this is the "most eager" algorithm with input-output function /, since it specifies that the computation of the input value must be completed in one time step. (Note, however, that minalg does not define a functor.)

Remarks on Canonicity
Note that the identity algorithm valA/ is canonical: valA/ = algid^/-The projection algorithms are also canonical, 5?, -= alg7T;. The application algorithm app, however, is not canonical. Let app be the input-output function for app. We have: That is, the application algorithm is uniquely determined as the uncurrying of the identity algorithm, and in general it is not maximal in its input-output equivalence class (nor is it minimal). This reflects the fact that app ignores the computation of the algorithm to be applied, but does pay attention to the computation of the argument of the application, while the algorithm application function ignores the computation of both the applied algorithm and its argument. T poll Figure 5: X>(Two X Two -Two) Example 3.19 In table 1 we list input-output functions for some of algorithms of TwoxTwo Two, shown in figure 4. The corresponding function space, Two x Two -* Two, is shown in figure 5. We use identical names for some of the algorithms and functions, such as poll and T, but it should be clear from the context whether we refer to the algorithm or to the function.
We take this opportunity to give an operational intuition, in lieu of a formalization of what constitutes a computation strategy, or a detailed discussion of an operational semantics for algorithms.
We take a coroutine-like view of the computation, much as in Berry and Curien's operational semantics for sequential algorithms [Cur86]. Computation is demand driven: a request for the value of a cell in the result may lead the algorithm to issue sub-computations until enough information  has been gathered about the input value for an output value to be determined. In the sequential case, only one sub-computation may be active at any point in time, and hence the coroutine flavor. In the parallel case, sub-computations may be issued in parallel, and several sub-computations (at differing levels) may be active simultaneously. Note that we assume some discrete global clock, with respect to which all computations are synchronized.
In the above example, one may ask for the value filling the cell * in the application of one of these algorithms to an argument. A sub-computation of the left argument to an algorithm corresponds to a computation of the cell *.l of the argument. Returning to the table, under the Termination heading we give the least value on which the algorithm will produce a result T, i.e., fill the cell *; this is of course determined by the input-output function. Under Step 1 and Step 2 we list the sub-computations that must be performed by the (end of the) first or second step of the computation, respectively, if the algorithm is to fill *. In this specific case, only termination of the sub-computations matters, since there is only one way to fill any cell; in general, the value with which a computation terminates will also be important.
For instance, the algorithms that compute the function b (standing for both, the doubly strict function) can be characterized as follows: • The algorithm pb specifies that both the left component and the right component of the argument must be computed by the first time point. This is the most eager algorithm for b.
• The algorithm lb specifies that the left component must be computed by the first time point, and that the right component must be computed by the second time point.
• The algorithm db specifies that either the left or the right component must be computed by the first time point, and both must be computed by the second time point.

Intensional and Extensional Aspects of Algorithms
Properties of functions, such as stability and sequentiality, apply to algorithms in several ways. We say that properties of an algorithm's input-output function are (input-output) extensional properties of the algorithm, and properties of the algorithm itself, regarded as a function on paths, are intensional properties of the algorithm.
As an example, of the algorithms in figure 4, poll is neither extensionally nor intensionally stable and neither extensionally nor intensionally sequential; db, pi and pr are extensionally stable (and sequential) and are not intensionally stable (or sequential); and all other algorithms are stable (and sequential), both intensionally and extensionally. It is not by chance that some possible combinations of properties are not represented: Proposition 3.20 The input-output function of a stable (respectively, sequential) algorithm is stable (respectively, sequential). A function is stable (respectively, sequential) iff it the input-output function of some stable (respectively, sequential) algorithm.
Proof: If a is stable then fun(a) is stable, because any counter-example to the stability of fun(a) generates a counter-example to the stability of a. Similarly if / is stable then minalg(/) is stable. The proofs for sequentiality are analogous. • Therefore there can be no algorithm that is extensionally, but not intensionally stable (or sequential). The function poll, for instance, has no sequential or stable algorithm.

Relationship to Berry and Curien's Sequential Algorithms
We discuss now the strong connections between the work presented here and Berry and Curien's sequential algorithms [Cur86]. Although we will not discuss them in detail, similar relationships can be established between our earlier attempts to define parallel algorithms [BG90] and the current work. We believe that these connections show how our view of intensionality as a computational comonad is a natural outcome of the earlier lines of research.

Berry-Curien Sequential Algorithms on CDSs
We first present some relevant definitions concerning Berry-Curien sequential algorithms. The reader is referred to [Cur86] for a complete exposition. The following results are shown in [Cur86]: the category of (discrete generalized) deterministic CDSs and sequential algorithms is a ccc; the product is the product of gCDSs given above and Null is a terminal object; the exponentiation object of M and M 9 is M =>* M'.

Definition 4.2
The input-output function fun*6 of an algorithm b is given (as in [Cur86]) by: There is no analogue in the Berry-Curien model to our definition of the alg map on parallel algorithms, since there is no uniform way to pick a canonical sequential algorithm for a sequential function with more than one sequentiality index (such as the doubly-strict-or function and the function b). Nevertheless, Berry and Curien have shown that each element of the sequential function space is the input-output function of some sequential algorithm. Moreover, the quotient of V(M =>t M') by input-output equivalence (having the same input-output function) is isomorphic to the sequential function space from V{M) to V(M'), ordered by the stable ordering. Example 4.3 Figure 6 presents X>(Two x Two Two), and table 2 presents the operational intuition for those algorithms. Note that, of the continuous functions in figure 5, only poll is not sequential. Also, the sequential functions of this type yield a flat domain when ordered by the stable order.
The operational behavior of sequential algorithms is more straightforward than for the parallel algorithms. Computation is again demand-driven, based on a coroutine-like but sequential operational semantics. For instance, when the algorithm lb^ is applied to some input and a request is  made for the value of the result cell *, the algorithm specifies that a sub-computation of the left input argument (the cell *.l) must be issued first; if this cell is filled with the value *, the right argument is similarly computed; if this too is filled, the algorithm specifies that result cell * can then be filled, with the value *. • Our example is limited in that only one value may fill each of the cells. In the presence of several values, branching may take place, based on the value, and the enabling structure of the algorithm takes on a tree shape (a linear list in this example). A computation simply determines a path in this tree -a sequence of (strictly increasing) states, serving as (finite) approximations to the input. This is the basic intuition which may be carried over to our formulation of parallel algorithms.

Embedding Berry-Curien Algorithms into Parallel Algorithms
We sketch how the Berry-Curien sequential algorithms space V(M M') may be embedded into the parallel algorithm space V(M A/') in a way that preserves the input-output function and the computation strategy. The embedding also respects the ordering (set inclusion) of the Berry-Curien model. Intuitively, hist(6,xc / ) is a finite sequence of states of M, that may be seen as a computation undertaken when 6 is applied to an argument approximated by x, trying to fill the cell c' in the result. We use juxtaposition for concatenation of finite sequences. The lub V S of a set S of finite sequences is obtained by a componentwise lub (i.e., componentwise union) of the sequences, with last components of repeated as necessary to obtain sequences as long as the longest one 1 . The lub of the empty set of sequences is the empty sequence, and the empty sequence is extended by repeating 0 as often as necessary. Here we implicitly extend the finite sequences hist(6,o:c / ) to infinity by repeating their last component, with the convention that the empty sequence represents 0". • Embedding preserves the input-output function: Example 4.7 The embedding of figure 6 into figure 4 is straightforward, shown in the embed(i) column in table 2. It is easy to see that the (informally given) computation strategy is preserved. The image of D(Two x Two =>t Two) under embedding is a proper subset of the intensionally sequential algorithms in D(Two x Two -Two). While pb is intensionally sequential, it does not impose a linear order of evaluation on the two sequentiality indices and therefore does not correspond to a Berry-Curien algorithm. • Although we cannot give a rigorous proof without first formalizing the notion of computation strategy, it should be intuitively clear that the embedding function always preserves the computation strategy of its argument.
It is easy to show that embedding preserves order, in the following sense:

Directions for Further Research
In this paper we have introduced a generalization of concrete data structures and concrete domains, and parallel algorithms between generalized concrete data structures. We would like to demonstrate the utility of our new structures in supporting the development of a theory of sequentiality and parallelism and in the development of a framework for intensional semantics. The results of this paper constitute a foundation on which to build, but there are many topics for further investigation and several directions for us to follow. We have presented a cartesian closed category of gCDSs and continuous functions, and we discussed briefly a ccc of distributive gCDSs and stable functions. These two categories employ a common underlying concrete representation -the gCDS -but use different notions of states to obtain different notions of generated domains. We would like to give a domain-theoretic characterization to both families of domains. We have made a start in this direction, with the partial characterization of the domains generated by the continuous notion of state.
It seems likely that some other natural classes of functions may yield to an analogous development. In particular, we would like to try to use the same approach to define a category of gCDSs and sequential functions, centered on a suitable notion of a sequential state. This task is harder, since we cannot rely on the desired category being a full sub-category of some already known ccc, such as Scott domains and continuous functions or dl-domains and stable functions. Nevertheless, our initial investigations in this direction are encouraging. We have pointed out that a definition of sequentiality may be formulated in the gCDS setting. We intend to study the implications of such a definition, and whether it proves useful in obtaining sequential semantic models. We have already made some remarks concerning these issues in section 2.6.
We have presented an intensional semantic model -the category of gCDSs and parallel algorithms, obtained by the co-Kleisli category construction from the ccc of gCDSs and continuous functions and the comonad of paths. Since the paths comonad preserves product the obtained intensional category is a ccc, so that it may be used in a standard way to provide an (intensional) model for the simply-typed lambda calculus. This choice of comonad is not the only reasonable one. Indeed, for some purposes the paths comonad may be regarded as too detailed. For instance, if one is not interested in the number of steps between successive increments in a computation, but only in the relative order in which the increments occur, it would seem appropriate to use the comonad of strictly increasing (rather than non-decreasing) paths 2 . Another possible choice of comonad might be obtained by regarding events not as linearly ordered but merely as partially ordered or even pre-ordered, so that we may dispense with any assumption of a global clock.
However, comonads based on strictly increasing paths or on pre-or partial orders on events do not preserve finite products. This means that the intensional category for these notions of intensional behavior will not normally be a ccc, even if the underlying extensional category is cartesian closed. Nevertheless, we believe that algorithm categories built with such comonads may still provide sensible intensional models for the A-calculus, and we will report on this separately.
As an aside, the reasoning here helps to explain the shortcomings of our earlier "query model" [BG90]; in our current terminology, we were attempting there to obtain a ccc (with currying builtin), using (the analogue of) the non-product preserving comonad of strictly increasing paths. We now realize that this combination does not yield a ccc.
We have exhibited a generalization of Berry and Curien's sequential algorithms into parallel algorithms, together with an embedding of the former into the latter. Essentially the same embedding should also work when we consider comonads based on strictly increasing paths, or on partial orders, or on pre-orders over events. A "tighter" embedding could be obtained into the stable algorithms, which we may construct as the co-Kleisli category of gCDSstab, using any of the above variations on the paths comonad. We would now like to understand better the Berry-Curien construction itself. On the face of it, the Berry-Curien category of deterministic concrete data structures and sequential algorithms is not attainable as an application of the co-Kleisli construction, since there is no underlying extensional category of concrete data structures. We conjecture, however, that there are strong connections between the Berry-Curien category and an intensional category of gCDSs employing a suitable notion of sequential algorithms. In order to establish this conjecture we must first, of course, exhibit an appropriate extensional category of gCDSs and sequential functions, one of the goals listed above.
Once we have a sufficiently established theory of intensional semantics, we would like to turn to its application to reasoning about intensional properties of programs, such as efficiency.