Gerry's Home Page Preliminary Materials Chapter 1 Chapter 2 Chapter 3 Chapter 4 Chapter 5 Chapter 6 Chapter 7 Chapter 8 Chapter 9 Chapter 10 Chapter 11 Bibliography Appendix

Sec 7.3

7.3.    Languages for Human Problem-Domain Communication

As discussed in Section 7.1, a central thrust of design environments like Janus is to eliminate the need for designers to “master the many details inherent in general purpose programming languages” (Fischer, et al., 1989, p.6) by supporting “human problem-domain communication.” For example, Janus provides a palette of icons that represent key objects in the problem domain of kitchen design. These icons can be manipulated tacitly with a mouse, with no need for the designer to express decisions in an explicit programming language. Similarly, the designer views discussions in the issue-base and messages from the critics in natural language statements that are formulated in the language of kitchen design, not that of computational operations.

However, it has also been noted that this kind of human problem-domain communication is insufficiently expressive for supporting interpretation in innovative, cooperative design. This section will discuss three arguments for supplementing human problem-domain communication with some form of programming language: (1) Recent versions of Janus have come up against the limit to expressivity in various ways that call for an end-user programming language. (2) Phidias and other systems have successfully incorporated end-user programming languages to different degrees. (3) Discussions of knowledge representation languages provide strong arguments for the utility of general purpose programming languages for communicating with the computer more flexibly. Once more, the point is to find the right mix of tacit and explicit support. It may be necessary to include an end-user programming language within design environments for designers to use in extending the current vocabulary of human problem-domain communication. However, the need to use that language should be minimized to where it is truly necessary to express things explicitly. Furthermore, the structure of that language itself should be designed to promote tacit usage as much as possible, in order to minimize the number of details of the language that must be mastered.

Increasing expressivity. The need for increased expressivity of communication in design environments is greatest when it comes to modifying existing knowledge representations. As long as one can express one’s ideas adequately with the given representations (for instance, the defined palette items, domain distinctions, critic rules), there may be no need to go beyond them. This follows from the analysis of interpretation: explication is triggered by breakdowns in one’s understanding. Only when the nexus of signification of one’s current tacit preunderstanding is inadequate for the understanding of something is there a need to make one’s understanding more explicit. However, when one’s understanding does have to be extended innovatively, then one needs linguistic resources to make things explicit. The degree to which things must be made explicit and the length of time for which they must remain explicit before being resubmerged into a revised tacit understanding depends on the particular circumstances. It is the same with interpretation within a design environment (which is, after all, a model of the designer’s understanding). When the current representation is inadequate, there must be a way of analyzing that representation more explicitly, modifying its structure or significance, and then re-submerging the new representation in a form that can once more be used tacitly.

The end-user modification component of Janus (Girgensohn, 1992) failed to provide a smooth transition to the explication of palette items and critic rules that needed to be modified. It provided extensive interface support for creating new representations in the form of examples, context-sensitive help, checklists of required steps, and even critics of the modification process. But it provided no means for explicating existing tacit representations short of the Lisp code of their implementation. For non-programmers, Lisp does not offer a graceful transition from human problem-domain communication. Moreover, although this version of Janus was completely rewritten to support end-user modification, it’s structure severely limits the scope of possible innovation, even using the power of Lisp. This is because it is not designed to make use of an end-user programming language to, for instance, define critic rules that are significantly different from the existing seeded rules. As Girgensohn (1992) writes:

The representation of critic rules in Janus proved to be difficult to understand for many of the subjects in the user studies. Especially the applicability condition that relates a critic rule to descriptions of design units such as (Cooks Self Food) was a source of problems. The Lisp-like format of descriptions and the use of keywords such as Self was a part of these problems. A representation has to be found that is more familiar to users such as natural language and at the same time constrained so that the system can reason with it. Another source of confusion was the mechanism for specifying how many combinations of design units had to be tested. For example, a stove has to be near to only one refrigerator, but it should be away from all windows. . . . A related problem [is] that critic rules [to] check for the absence of design units cannot be formulated in the current representation. For example, it is impossible to check whether a kitchen has a door or whether the window area is at least 10 percent of the floor space. Stahl (1992) proposes to tackle problems of this kind by formulating critic rules with a natural-language-like query language in which the user can formulate queries. (pp. 79f)

This statement lists some of the kinds of issues that an adequate end-user language would need to be able to express: applicability conditions, self-reference, combinatorics, absence of units.

There is no reason why these issues could not be stated in a format more natural to designers than abstract Lisp syntax—particularly since Lisp itself is designed to build up more application-specific languages. As suggested in this quotation, what is needed is a language that can represent the explicit relationships that are implicit in the tacitly used critic rules and design units in a way that makes sense both for the designer and for the computer. The Hermes language (discussed in Chapter 10) tries to do just this. It builds up an end-user language for defining critic rules and other design knowledge in formulations which are as close as possible to domain terminology. Through the generality of its syntax, the Hermes language permits designers to define a much less constrained set of critics than those in Janus.

A similar limit to expressivity is found in X-Network (Shipman, 1993). This design environment employs agents to “search for information with certain attributes within the system and perform operations based on what, if any, objects they locate. Agents consist of a trigger, a query, and an action” (p.32). By basing the agents on queries of the current database, the system allows all actions and displays to be dynamic. The power of the agents is largely determined by the expressivity of the queries, which determine what kind of information the agents can respond to. In particular, if a designer wishes to modify the way a given agent is operating, the designer is dependent upon the expressivity of the query expressions for extending or innovating the agent behavior.

Currently, the queries in X-Network (Shipman, 1993) are limited to the specifying of a conjunction of attributes:

The query defines the information that must be located before the agent will execute its action. The query definition area within the agent editor is similar to the property sheet used to attach attributes to objects in the information space. This interface limits the expressiveness of the query to the location of objects matching attribute patterns but allows for the transfer of skills acquired in using the property sheets. Use of a more powerful query language based around a hypermedia model, such as that found in Hermes (Stahl, this dissertation), would allow greater expressiveness but with the added cost of the users being required to learn the syntax and semantics of the formal language. Beyond such traditional query mechanisms, query definitions should also be allowed to use built-in primitives that do complex analysis (p.34).

An end-user language such as that in Hermes could express more than conjunctions of attributes without requiring a forbidding syntax and semantics. It could also include primitives for the complex analyses mentioned. As noted in this passage, the concern is with the trade-off between increased expressivity and mounting cognitive overhead. The ideal would be to partially end-run this trade-off by minimizing the cognitive overhead that comes from making things explicit by keeping even the use of the language as tacit as possible.

Incorporating end-user programming languages. Phidias (McCall, et al., 1990a) incorporates a query language for its issue-base. That is, the issue-base consists of a hypertext network in which each issue, answer, or argument is a distinct node connected with labeled links. In order to display part of the issue-base, Phidias evaluates a query. Thus, if issue-234 is “What should be the location of the refrigerator?”, then the issue-base discussion of the proper location of the refrigerator would be generated with the query:

issue-234 with answers with arguments

The Phidias query language began in an earlier version named Mikroplis (McCall, 1989). This was primarily a system for constructing and using issue-bases for designing. It was based on McCall's (1987; 1991) variant of Rittel's Ibis: Procedurally-Hierarchical-Issues (Phi). A Phi issue-base consists primarily of issues, answers, arguments, and resolutions connected by links based on the "serves" relation. Phiql, the Phidias Query Language was developed to meet the needs of someone using a Phi issue-base. Its primary use was to display subtrees of the issue-base hierarchy, such as:

answers of subissues to issue-105

answers of issues with arguments

issues containing "doorway"

The original Phiql language was based on a number of observed regularities in the formulation of queries in natural language. It was hoped that by incorporating these patterns of natural language in the computer language it would seem more natural and "English-like" than existing programming languages. In particular, it was noted that the procedure of following links of type L from node N was expressed by the phrase L of N in English. N was considered to be the subject of this expression and L to express a relationship applied to that subject. Successive relationships (i.e., link traversals) could be applied by adding additional phrases to the front of the query: L3 of L2 of L1 of N. Various conditions, such as containing a given substring, could follow the subject (McCall, 1989). By means such as this, a query language was defined with a simple syntax that could be easily parsed and that appeared English-like. The Mikroplis User Manual (McCall, et al., 1983) noted:

The Mikroplis command code is similar to natural English. It is, however, a code, and as such must be learned and followed. The intention in imitating natural language—for instance, the fact that a variety of prepositions and articles is allowed, or that the syntax generally follows the subject/predicate conventions of English—was to minimize learning effort and to essentially eliminate the kind of opaque codes often found in other command sets (n. p.).

A number of limitations were imposed to maintain the workability of this approach: issue, topic, and document nodes had to have standardized names like issue-123; other nodes could not be direct subjects of a query; and the syntax was kept simple.

Query input was done through a prompted command line interface, so users simply typed in the query like a sentence. Incidental words like articles and prepositions were allowed, but ignored by the parser. Users could think about the information they wanted in the same terms that Phidias accepted as a query. This made the query language easy to learn and to use. In proposing to use Phiql for defining queries in virtual structures, McCall (1990/91) continued to emphasize the English-like character of the language:

Phiql is a highly English-like language which has been in use by end users for more than six years. This experience has consistently shown Phiql to be learnable within a single day—often within an hour. . . . Though it appears purely declarative to end users, Phiql is in fact an applicative (functional) language . . . [yet] statements in Phiql almost always appear to be ordinary (declarative) English expressions. (p.6)

McCall (1990/91) proposed extending the original Phiql to help make Phidias a viable alternative to expert systems. This proposal focused on adding the functionality of virtual structures, that is expressions in the query language embedded in hypertext nodes. When a node that has been defined as a virtual structure is evaluated, it returns the results of the embedded query in place of its textual contents. This mechanism was seen as a way of embedding computational power in the very nodes of the hypertext database. (See Appendix B for a discussion of how this idea has been developed in the Hermes language.)

To give significant inferencing power to virtual structures, the Phidias query language needed to be extended to include several new operations. The technical section of the proposal (McCall, 1990/91) detailed the planned modifications as follows:

1. Addition of comparison operators: Phiql now only uses substring matching. It will need other comparison operators, including >, <, and =.

2. Addition of existential and cardinal (numerical counter) quantifiers: These will allow queries which ask for such things as nodes having no children.

3. Addition of negation operator (“not”).

4. Addition of a true "if" statement, so that conditional queries can be stated more naturally.

5. Addition of capability for conjunction of conditions.

6. Addition of capability to compare the contents of a pair or more of retrieved nodes. This will allow the comparison of user-input nodes, whose contents cannot be known in advance by the system (p.7).

These extensions were implemented in an initial version of the Hermes language. They were tested by developing an application in the text-based domain of academic advising. This domain was chosen because it took good advantage of the inferencing capability. It also provided a basis for comparison with a hypertext system lacking inference (Peper, et al., 1990) and with expert system approaches. This version of the language is reported on by Stahl, et al. (1992). Appendix A reports on a programming walkthrough of this language and Appendix B discusses the academic advising application.

To support a wide range of inferencing, the language had to be extensively expanded to include true/false conditionals, numerical calculations, comparison operations, and nesting of phrases. A typical request in the new language—taken from the test domain of academic advising—might look like the following:

courses of sandra that have studio_types and that also have less than 3 prerequisites, with their prerequisites.

To evaluate this statement, the system would navigate from the student node, sandra, across all its courses links; check which nodes arrived at had at least one studio_types link and also had less than three prerequisites links; and output a list of the course nodes that satisfied these conditions, along with a sublisting of their prerequisites. Here is the output:

***COURSES:

1. ENVD 2110 Architectural Studio

  *** PREREQUISITES:

  1. ENVD 1000 Environmental Design Studio

  2. ENVD 1014 Intro to Environmental Design

2. ENVD 2120 Planning Studio

  *** PREREQUISITES:

  1. ENVD 2110 Architectural Studio

The structure of statements in this language and their method of evaluation are based on the structure of hypermedia. The queries investigate the node and link structure, rather than the content of a relational database, and their evaluation proceeds by navigation across the links from initial nodes. In this sense, the research represents an effort within the hypermedia paradigm. The thrust of the effort is to exploit hypermedia mechanisms to achieve certain functionality of artificial intelligence and information retrieval technologies. Thus, the goal was to expand hypermedia to include:

*    Some of the inferencing capability of Prolog, but without the comprehension difficulties of predicate calculus and explicit variables;

*    Some of the querying ability of SQL, but applied to an object-oriented, hypermedia database;

*    Some of the advantages of semantic databases, but allowing semantic relationships to be defined between instances as well as types by labeled links; and

*    Some of the utility of semantic networks, but without restriction to a pre-defined set of link types or to semantic relationships.

The Phidias query language illustrates a number of important principles: (1) It shows one approach that has proven successful for defining an end-user programming language that minimizes the cognitive overhead by modeling its syntax and semantics on that of natural language. (2) While the syntactic structure of queries follows a standard subject/predicate order, the vocabulary of terms is taken from the problem-domain (e.g., the node names and link types for issue-base queries are user-defined). (3) Moreover, the vocabulary is easily extensible by the users, so they can develop terminology for expressing their innovative interpretations of the structure of the issue-base. (4) The advanced version of the language includes logical and computational operations for specifying complex conditions. (5) Additional computational primitives can be included that would be useful in a design environment for a specific domain. (6) The language is based on the hypermedia structure of the database that it queries and expressions in the language can be incorporated in the nodes of the hypermedia as virtual structures.

The Hermes language is based on the Phidias query language. It significantly extends the computational power and flexibility in order to support interpretation in design. It retains the idea of a constrained syntax, that made the Phidias language easy to use, and it provides additional interface supports for its much larger syntax. However, the Hermes language moves away from the “English-like” emphasis in Phidias as a result of issues that arose in programming walkthrough evaluations of intermediate forms of the language. Rather, the Hermes language normally hides much of the computational details to support tacit usage while allowing users to make the structure of expressions more explicit as needed for modification and interpretation.

The Phiql language suggests the possibility of including an end-user programming language in a design environment. Such languages have proven useful in other computer applications. Many commercial products like CAD systems, spreadsheets, word processors, database management systems, Mathematica, and HyperCard include macro languages, scripting languages, or end-user programming languages. While these languages are not always easy for non-programmers to use (Nielsen, et. al., 1991), they often provide a middle ground between canned applications and programming environments in which non-programmers can gradually learn to customize operations and local developers or local-site super-users can help people go beyond the limitations of standard applications (Nardi & Miller, 1990).

Another advantage of a language for design environments is to integrate the knowledge representations of various components. Even if some effort is involved in learning to use the language, if the same language applies uniformly to many or all of the system’s components then once it is learned it provides the power to modify and extend all knowledge in the system. The language can impose a unity on a complex system. For instance, the Kid version of Janus is the most multi-faceted implementation of that system, with many components and linking subsystems. The developer of Kid has remarked that “now that all these components have been prototyped with different knowledge representations it is time to integrate them with a unified substrate, and the Hermes language would be a great way to do that.” (K. Nakakoji, personal communication, June 7, 1993.)

Of course, the inclusion of a language does not change a design environment’s knowledge representations, but the decision to use a language across components imposes a design constraint that favors an integrated system built on a consistent knowledge representation. Part III discusses the role of the language in the Hermes integrated substrate.

Communicating more flexibly with the computer. For a design environment that is centrally built around an end-user language, the design of the language and the design of the knowledge representation are intimately linked. The developers of the design rationale language, Drl (Lee & Lai, 1992), for instance, point out that “a large body of research in the last two decades or so points to the importance of choosing the right representation for a given task (e.g., Brachman & Levesque, 1985; Winston, 1984). The task of using and reusing design rationales is no different.” They argue that the common perception that the capturing of design rationale is not worth the effort may be based on a representation problem. It may be that a different representation system or language would allow people to represent more easily what they want to represent in a way that can provide significant benefit.

Many research programs in AI have concentrated on designing knowledge representations and languages that had maximal computational power based on formal schemes. Only afterward did they attempt to add a user-friendly interface facade. However, it may make more sense to start out from an effort to design a language oriented toward tacit understanding and then to gradually extend its computational power, always keeping in mind its usability. Systems like Krl, Pie, and Drl demonstrate the importance of sophisticated programming languages in knowledge-based systems. However, they require a high level of explicit, abstract analysis to use. The Phidias Query Language, in contrast, proved to be rather natural to use, but it had limited power.

The Ace project took the alternative approach to combining computational power with usability by starting with a consideration of how to empower non-programmers and then worrying about formal computational issues. The Ace developers studied the programming language needs of end-users and local developers, and attempted to develop end-user languages for them. This resulted in Ace, an Application Construction Environment (Johnson, et al., 1993). Ace provides a set of frameworks for defining application-specific programming languages. These languages allow end-users and local developers to extend the functionality of applications by adding behavior to their systems without their having access to source code and without recompilation (p.47). Ace adopts the model of spreadsheets, by focusing on a well-thought-out set of primitives and avoiding complex explicit control structures, such as “for” loops and recursion. Thus, it goes beyond the superficially user-friendly style of HyperTalk, which requires iterations and conditionals to be explicitly expressed rather than being implicit in application-specific operators. Ace is intended to “foster a methodology more like that fostered by spreadsheets. It puts users at the center of the development process” (p.53). That is, users are seen as the primary implementors of applications; they assemble the main components of their application and get it working. To support this, users are supplied with application componentry in the form of reusable, extensible software libraries, and components are self-describing so that new ones can be easily added to the system.

The Hermes language adopts much of the Ace approach, starting with a consideration of the user’s (interpretive) needs. The Hermes language provides a computationally active medium in which designers can build up their own behaviors and supports. Systems of defined language expressions can evolve, be organized in perspectives, and be shared by different designers of different skill levels.

Programmable design environments. The idea of combining design environments with the approach of programmable applications (Eisenberg, 1992) has been explicitly proposed by Eisenberg and Fischer (1992). They motivate this combination on pragmatic grounds, hoping to integrate the best features of each approach and overcome their respective weaknesses. While they do not address many of the issues that arise in actually implementing a programmable design environment, they recognize the advantages of extending a direct manipulation system with an appropriately designed end-user language. Hermes can be seen as a first instantiation of the idea of a programmable design environment, moreover one that is motivated by principles of human understanding.

The system suggested by Eisenberg and Fischer (1992) consists of two independently conceived halves (as mirrored in the structure of the paper, which discusses the notions of programmable applications and design environments separately, and then worries about their interactions). However, the indications are already present in their discussion that the attempt to implement such a system would lead to an integrated approach, as it did in Hermes. For instance, they suggest that, “the programming environment ‘half’ of the application should be constructed around a domain-enriched language (which might be a newly-constructed language or an application-specific ‘dialect’ of some existing general-purpose language)” (p.81). This distinction between a new language and a dialect of an existing one underestimates the extent to which the language must be integrated with the structure of the design environment’s knowledge representations. Any language useful for extending the expressibility of the design environment will be severely constrained by (i) the need to incorporate primitives that refer to the elements of the design environment, (ii) the need to incorporate functionality that corresponds to the structure and tasks of the design environment (e.g., navigating hypermedia links and filtering by specific kinds of criteria), and (iii) the goal of making the language accessible to designers. Whether or not an end-user language that meets these constraints is built on top of an existing language like Pascal, Fp, Lisp, or Scheme it will look like a new language. On the other hand, any flexible end-user language would want to incorporate much of the power of programming languages and would do well to build upon or model itself after a successful language at the computational level. So, in the end the proposed alternatives come down to roughly the same thing and the important point is the integration of the language with the structure and goals of the design environment. Chapter 10 will show how the Hermes language was designed specifically to satisfy these constraints.

Design rationale is a particular concern of Eisenberg and Fischer (1992). They want the user to be able to reuse design examples from the catalog and to copy and modify the associated rationale. They recognize that this puts a general requirement on the system to support incessant reuse and plasticity: “The upshot of all this is that our systems must support users in creating, retrieving, browsing, modifying, storing, and reusing information structures that capture design-related decisions—and the systems must moreover support this type of activity interactively, in the broader context of creating new artifacts” (p.86). Unfortunately, they never suggest mechanisms to achieve such system-wide plasticity of representation or to organize the proliferation of different versions of rationale and other kinds of knowledge. The substrate in Hermes (see Chapter 8) is designed to maximize the ability of users to modify all forms of knowledge and the perspectives mechanism (Chapter 9) is available for users to organize their versions of knowledge and reuse design examples with their associated rationale through virtual copying.

The Hermes perspectives also solve another problem raised by Eisenberg and Fischer (1992), that of maintaining “historical” information. Versions of a design at a given state can be saved in their own perspectives. The issue of whether something was created by direct manipulation or by programming does not arise in Hermes, because the Hermes language does not change state; it only displays, analyzes and critiques the current state—which is created via direct manipulation and dialog boxes. Although the SchemePaint (Eisenberg, 1992) programmable application showed that a programming language could be useful in creating precise, complex graphics, there is little sense in Eisenberg and Fischer (1992) how the language would be used in the creation of designs in other domains.

The example given for use of the language in a programmable design environment for graphs does not create new graphs, but queries the hypermedia database. It is to “write a procedure to find all catalog entries of a particular graphical type within two links of a particular entry” (p.87). This example is not worked out, but could be accomplished in the Hermes language. Say the particular catalog entry was named mike’s graph and the particular graphical type of interest was pie graph. Then the following expression would return the desired results:

all associations of all associations of mike’s graph that are of kind pie graph

If the only kind of link type of interest is that of similar example links between catalog entries, then one could define list of pies as:

similar examples of similar examples of mike’s graph that are of kind pie graph

and then evaluate the expression,

deliberation of list of pies

to display the design rationale attached to all the catalog entries in the list.

Note that in order to carry out this task, all the relevant knowledge must be in an integrated knowledge representation, linked together by hypermedia links. That is, the catalog items (graphical representations), the specification elements, and the design rationale (text and pictures) must be compatible nodes that can be linked together in a way that the language can navigate. Both by the nature of the tasks that are proposed as desirable by Eisenberg and Fischer (1992) and by the necessity of having a single language refer to and analyze all knowledge in the system, an argument is implicitly made for the use of an integrated substrate to define a hypermedia structure and its accessibility via the language and perspectives. As Part III will show, this is precisely the role of the Hermes substrate as a basis for programmable design environments.

There are good reasons to incorporate programming languages in design environments in order to go beyond the limited expressivity of human problem-domain communication as suggested by the notion of a programmable design environment. Systems like Krl, Pie, and Drl have shown the utility of languages to define sophisticated knowledge representations and design rationale capture systems that can be supported with powerful computational means. Each of these systems has, however, run into the need to make the systems more usable by supporting tacit understanding. The Phidias query language has suggested an approach to syntax and semantics that promises to reduce the explicit cognitive overload of formal programming languages for non-programmers. Ace suggests additional techniques for keeping computations tacit and for designing languages that help promote plasticity of knowledge representation. Hermes tries to incorporate ideas like these to achieve an adequate mix of mechanisms requiring tacit and explicit understanding on the part of designers using an end-user language in a design environment. Thereby, it hopes to go far toward overcoming the problem of tacit and explicit understanding in the computer support of interpretation in design.

Go to top of this page

Return to Gerry Stahl's Home Page

Send email to Gerry.Stahl@drexel.edu

This page last modified on January 05, 2004