Lecturer's Précis - Minsky (1977)

"Frame-System Theory"

Copyright Notice: This material was written and published in Wales by Derek J. Smith (Chartered Engineer). It forms part of a multifile e-learning resource, and subject only to acknowledging Derek J. Smith's rights under international copyright law to be identified as author may be freely downloaded and printed off in single complete copies solely for the purposes of private study and/or review. Commercial exploitation rights are reserved. The remote hyperlinks have been selected for the academic appropriacy of their contents; they were free of offensive and litigious content when selected, and will be periodically checked to have remained so. Copyright © 2004-2018, Derek J. Smith.


First published online 07:40 BST 20th April 2004, Copyright Derek J. Smith (Chartered Engineer). This version [2.0 - copyright] 09:00 BST 9th July 2018.

Readers unfamiliar with the nature of research into "artificial intelligence" may care to note the various sub-areas of enquiry reviewed in Sections 1 and 3 of our e-paper on "Short-Term Memory Subtypes in Computing and Artificial Intelligence (Part 5)" before proceeding.

1 - The Theory

Minsky begins by stating "the essence" of frame theory as follows .....

"When one encounters a new situation (or makes a substantial change in one's view of a problem), one selects from memory a structure called a frame. This is a remembered framework to be adapted to fit reality by changing details as necessary. A frame is a data-structure for representing a stereotyped situation like being in a certain kind of living room or going to a child's birthday party. Attached to each frame are several kinds of information. Some of this information is about how to use the frame. Some is about what one can expect to happen next. Some is about what to do if these expectations are not confirmed. We can think of a frame as a network of nodes and relations." (Minsky, 1977, p355; italics original; bold emphasis added.)

The frame is thus an attempt to apply one of the longest-standing ideas in memory theory, namely David Hartley's 18th century proposal that the mind is basically a vast network of interrelated fragmentary memories, a doctrine known formally as "Associationism" [glossary]. As a long-term memory for a stereotyped situation, the frame is the first stage in bringing fragments of knowledge together, and it does this by selecting the relevant "nodes and relations" from the broader network. The frame thus makes momentary higher-order sense of the world. Here is Minsky's explanation of how the permanent frame is temporarily associated with its equally permanent attachments .....

"The 'top levels' of a frame are fixed, and represent things that are always true about the supposed situation. The lower levels have many terminals - 'slots' that must be filled by specific instances or data. Each terminal can specify conditions its assignments must meet. [.....] Simple conditions are specified by markers that might require a terminal assignment to be a person, an object of sufficient value, or a pointer to a subframe of a certain type. More complex conditions can specify relations among the things assigned to several terminals. Collections of related frames are linked together into frame-systems [and] the effects of important actions are mirrored by transformations between the frames of a system." (p355; italics original.)

ASIDE: Note the similarity between the above concepts and Shattuck-Hufnagel's (1987) "slot-and-filler" approach to sentence construction, for details of which see Section 1.1 of our e-paper on "Speech Errors, Speech Production Models, and Speech Pathology". Note also the term "supposed situation" in the above quotation, and remember that one of the main purposes of the higher cognitive system is to maintain a "mental model" of the outside world. Note also that frame theory is an example of a schema-based approach to memory. The term "schema" became popular in the early 20th century (eg. Head, 1926; Bartlett, 1932). Schema theories emphasise the role of superordinate cognitive structures in understanding. They do not deny the existence of individual concepts as such, nor the importance of associations between them, but they look up a level at how subsets of concepts are habitually organised by "an active organisation of past reactions, or of past experiences" (Bartlett, 1932, p201). Schemas predict what will happen to things in the future from how those things (or similar things) have behaved in the past.

It follows that frame theory has considerable explanatory appeal to students of mis-cognition. Indeed, the literature on disastrous decision making contains many instances where the supposed situation did NOT in the event match the real world, and where people died unnecessarily as a result.  This is how Minsky explains such expectancy effects .....

"A frame's terminals are normally already filled with 'default' assignments. Thus, a frame may contain a great many details whose supposition is not specifically warranted by the situation. These have many uses in representing general information [and making] useful generalisations. The default assignments are attached loosely to their terminals, so that they can be easily displaced by new items that fit better the current situation." (p356; italics original.)

2 - Frames and Visual Perception

Minsky then devotes several pages to explaining how frame theory can be applied to another long-standing philosophical problem, namely that of visual perception. The issue he addresses is how an accurate three-dimensional perception can be obtained from what is typically a series of incomplete two-dimensional retinal images. With a person on a chair at a table, for example, we can usually see little of the chair, yet we rapidly activate what we already know about chairs. We "know" (or, more properly, "confidently suspect") that a chair is there, even when we cannot actually see it.

We can also cope, usually effortlessly, with objects or persons seen from a variety of angles. The trick here is to have stored away several frames for the object at hand, each representing a different perspective, say, and to resort to an overriding "frame-system" to integrate them all. As to what happens next ..... 

"It is not proposed that this kind of complicated structure is recreated every time one examines an object. It is imagined instead that a great collection of frame-systems is stored in permanent memory, and one of them is evoked when evidence and expectation make it plausible that the scene in view will fit it. [.....] Each frame has terminals for attaching pointers to substructures. Different frames can share the same terminal, which can thus correspond to the same physical feature as seen in different views. This permits us to represent, in a simple place, view-independent information gathered at different times and places." (p358.)

An added layer of complexity arises when complex "scenes" (that is to say, those containing more than one object) need to be processed ..... 

"Visual experience seems continuous. One reason is that we move continuously. A deeper explanation is that our 'expectations' usually interact smoothly with our perceptions. Suppose you were to leave a room, close the door, turn to reopen it, and find an entirely different room. You would be shocked. [.....] A naive theory of phenomenological continuity is that we see so quickly that our image changes as fast as does the scene. There is an alternative theory: the changes in one's frame-structure representation proceed at their own pace; the system prefers to make small changes whenever possible; and the illusion of continuity is due to the persistence of  assignments to terminals common to the different view-frames. Thus continuity depends on the confirmation of expectations which in turn depends on rapid access to remembered knowledge about the visual world. Just before you enter a room, you usually know enough to 'expect' a room rather than, say, a landscape. You can usually tell just by the character of the door. And you can often select in advance a frame for a new room. Very often, one expects a certain particular room. Then many assignments are already filled in. [.....] One has to assign to the frame's terminals the things that are seen. If the room is familiar, some are already assigned. If no expectations are recorded already, the first priority might be locating the principal geometric landmarks." (pp360-361; italics original.)

3 - Frames and Language

Minsky also sees a role for frame theory in explaining various linguistic phenomena. He borrows the specimen nonsense sentence <COLOURLESS GREEN IDEAS SLEEP FURIOUSLY> from Chomsky (1957), and points out that although this sentence has no obvious propositional meaning, its grammatical structure is relatively normal, and it can come some way towards making sense if pushed. Specifically .....

"[This sentence] can certainly generate an image. The dominant frame is perhaps of someone sleeping; the default system assigns a particular bed, and in it lies a mummy-like shape-frame with a translucent green colour property. In this frame there is a terminal for the character of the sleep - restless, perhaps - and 'furiously' seems somewhat inappropriate at that terminal [.....]. 'Idea' is even more disturbing, because one expects a person, or at least something inanimate. One senses frustrated procedures trying to resolve these tensions and conflicts more properly ....." (p365.)

The point is that by analysing sentences of varying grammaticality in this way, one can start to unravel the frame-subframe structure of grammatical processing. Minsky thus regards language comprehension as the search for sentence fragments - words, phrases, or clauses - which "satisfy subframes well enough" to produce "an image adequate for certain kinds of comprehension" (p365). Similar arguments may then be applied at paragraph level and above when trying to establish the deeper points of a communication, where frame-systems for "events" start making sense of frame-systems for scenes. Thus .....

"Almost any event, action, change, flow of material, or even flow of information can be represented to a first approximation by a two-framed generalised event. The frame-system can have slots for agents, tools, side-effects, preconditions, generalised trajectories, just as in 'case grammar' [citation], but we have the additional flexibility of representing changes explicitly. To see if one has understood an event or action, one can try to build an appropriate instantiated frame-pair. [.....] We condense and conventionalise in language and thought, complex situations and sequences into compact words and symbols." (p367.)

In fact, by the time the full complexity of human discourse has been catered for, no less than four levels of superordinate framing can be identified, as follows .....

(1) Syntactic Frames: These are frames to specify major syntactical relationships such as word order and the functions of prepositions.

(2) Semantic Frames: These are frames to specify the "action-centred meanings of words" (p369). 

(3) Thematic Frames: These are frames to specify what is going on at scenario level. 

(4) Narrative Frames: These are frames to specify the logical progression of a story from one scene to the next. [Thorndike (1977) offers an alternative analysis of story structures, if interested.]

4 - Frames, Memory, and Problem Solving

Minsky closes with a discussion of framing during problem solving. He begins by suggesting that a problem is only a problem in the first place because it creates conflicting cognitions. Frame theory helps us to analyse such conflicts, by identifying two potentially contradictory but nonetheless complementary pressures. Firstly, perception and/or prior recall will have made a number of knowledge fragments available to the task, but, as yet, not successfully assigned to explanatory frames. And secondly, perception and prior reasoning will have partly activated frames. We thus have fragments seeking explanations and explanations seeking fragments, and it is when these do not reconcile one with another that we have our problem. Minsky then identifies the following options .....

Matching: This is when alternative frames are experimented with in the hope of finding a better fit. This matching process involves finding as-yet-untried frames which share terminals with the original.

Excuse Making: This is where a minor residual frame-assignment mismatch is explained away [or "excused"] by simply accepting it. (Minsky's examples here include a chair which matches all required characteristics except size, and which could therefore be excused away if framed as a toy rather than as the real thing.)

Advice Taking: This is where a mismatch can be resolved by referring hierarchically upwards within the knowledge network.

When considering advice taking, Minsky returns to his example of moving about a familiar house .....

"When we move through door D, in room X, we expect to enter room Y (assuming D is not the exit). We could represent this as an action transformation of the simplest kind, consisting of pointers between pairs of room frames of a particular house system." (p371.)

Things are different, however, when the house is not a familiar one, whereupon the frames need to be a level higher, dealing with non-specifics and generalisations. He refers to these categorical frames as "classes", and invokes Winston's (1977) notion that mental "pointers" can make an effective retrieval system for classes. Thus .....

"Winston proposes pointers from each description in memory to other descriptions, with each pointer labelled by a difference marker. Complaints about mismatch are matched to the difference pointers leaving the frame and thus may propose a better candidate frame. Winston calls the resulting structure a similarity network." (p372; italics original.)

Minsky then gives a detailed example of how all the frames work together in supporting analogic problem solving ..... 

"Suppose your car battery runs down. You believe that there is an electricity shortage and blame the generator. The generator can be represented as a mechanical system [so it is relevant to ask] is the belt tight enough? Is it even there? The output, seen mechanically, is a cable to the battery or whatever. Is it intact? Are the bolts tight? Are the brushes pressing on the commutator? Seen electrically, the generator is described differently [detail]. The differences between the two frames are substantial. The entire mechanical chassis of the car plays the simple role, in the electrical frame, of one of the battery connections. The diagnostician has to use both representations. A failure of current to flow often means that an intended conductor is not acting like one [and] any conduction disparity revealed by electrical measurement should make us look for a corresponding disparity in the mechanical frame. In fact, since 'repair' [is] synonymous with 'mechanical repair', the diagnosis must end in the mechanical frame. Eventually, we might locate a defective mechanical junction and discover a loose connection, corrosion, wear, or whatever. One cannot expect to have a frame exactly right for any problem or expect always to be able to invent one. But we do have a good deal to work with, and it is important to remember the contribution of one's culture in assessing the complexity of problems people seem to solve. The experienced mechanic need not routinely invent; he already has engine representations in terms of ignition, lubrication, cooling, timing, fuel mixing, transmission, compression, and so forth. Cooling, for example, is already subdivided into fluid circulation, air flow, thermostasis, etc. Most 'ordinary' problems are presumably solved by systematic use of analogies provided by the transformations between pairs of these structures. The huge network of knowledge, acquired from school, books, apprenticeship, or whatever is interlinked by difference and relevancy pointers." (p.374; italics original.)

Finally, Minsky deals with the nature of the mental "problem space". This is a convenient term to describe the transient allocation of short-term memory resources to whatever subprocesses are involved in problem solving. Thus ..... 

"The primary purpose in problem solving should be better to understand the problem space, to find representations within which the problems are easier to solve. The purpose of search is to get information for this reformulation, not - as is usually assumed - to find solutions; once the space is adequately understood, solutions to problems will more easily be found." (p375.)

Frames, in other words, help both structure and continually restructure the problem space. 

5 - Evaluation

Frame theory provides a reasonably complete and coherent set of explanatory propositions for considering perception and problem solving. Nevertheless, it is only a specific instance of the schema approach to explanation, and for historical reasons it is probably less well known today than the alternative script-story theory put forward by workers such as Schank and Abelson. Nevertheless, frame theory remains particularly relevant to students on the programming side of artificial intelligence research because the AI tradition at MIT has half a century of practical development under its belt. It is also relevant to students of disastrous decision making, thanks to its ability to explain inappropriate defaults or otherwise mis-allocated terminal assignments [see Section 1].

Here are some additional readings on the subject .....

6 - References

See the Master References List