Action!

In discussing this post of mine with Derik, I realized that I should post on the technique I used of substituting a whole panel for an "action star," like this:



This usage is somewhat similar to what I talk about in this older article on metonymy, and the same phenomena creeps up overtly in McCloud's famous "Closure" example from Understanding Comics:



In both cases, we never see the action, because it's replaced by a panel that implies action took place, but replaces the image with some neutral information. In the case of the action star though, we associate that sign with events, so it further indicates the presence of an action, whereas in McCloud's example the text does most of this work, since the cityscape is entirely neutral.

I've recently been exploring this phenomena a lot more, especially since I keep seeing it in Peanuts strips, first shown (I believe) in this one:



So, we now have this phenomenon where we know we can substitute certain types of panels for others to get an entailment of the actions. For storytelling, this is pretty cool, since it forces the reader to draw an inference about the actions (a result interesting enough that McCloud extended this out to all interactions between panels).

However, are there also restrictions on which types of panels we can replace? Since the action star essentially just means "events occurring!" but doesn't show them, it can be considered as a kind of "visual pronoun." Because of this, it can also be used as a diagnostic for determining certain categories of panels versus others. This "pro-form" replacement is a common technique in linguistics for determining grammatical categories: we can replace Noun Phrases with the pronoun "it", and Prepositional Phrases with "there":

1. Martin pushed the really huge boulder up a massive hill.
2. Martin pushed [it] up a massive hill.
3. Martin pushed the really huge boulder [there].
4. * Martin pushed the really huge boulder [it].
5. *Martin pushed [there] up a massive hill.

In 2 and 3, we can see that this substitution works fine, but when we reverse which ones we're substituting for, in 4 and 5, it sounds awful (indicated by the asterisks).

So... can we do this using an "action star" as a kind of visual "pronoun"? Check out these Peanuts strips, where the action star replaces either the second or third panels:

1. *
2. *
3.
4.

When the second panel is replaced, it does not seem to make much sense (nor would it make much sense in the first or final positions here either). However, replacing it for the third panels does work — hinting that those panels belong to a certain class of words where a culmination of an action occurs (even when that action isn't an "impact").

Note also that an approach using linear "transitions" between panels would be unable to express this: what would the action star be a transition of — a "Non-sequitur"? That wouldn't be able to capture the understanding of the event occurring in that panel. Rather, this hints that transitions (based on the relations between panels) are not the way sequences are understood, and the need of some sort of global narrative structure (with categories for actual panels) underlying the sequence.

Continuity across panels

Derik posts a quote from this article on Narration in Comics that discusses cognitive schema and comics. I'd read the article awhile ago, but seized on this part of the quote :

"An extrinsic norm crucial to comics is the interpretation of a figure reappearing in several panels as one and the same figure shown at different moments in time (usually in chronological order)… Usually it is assumed that the event represented in the second panel happens after the event represented in the first one…"

This constraint is no doubt what led Saraceni to posit a principle of sequential images that weighs "new" versus "given" information across a sequence. It's also a type of constraint placed by Gestalt organization: Continuity.

However, what struck me on this reading of that quote, is that this schema is exactly the sort of thing that people who lack knowledge of the visual grammar (or who have a competing grammar) have trouble with.

For instance, kids below four years old seem to have no ability to make coherent sense of connecting juxtaposed panels — they can recognize the meaningful content of the things in each panel, but they can't seem to connect them as part of a narrative sequence. (They also seem unable to recognize any representations in the images that are predicated on understanding the causation between panels).

A comparable thing happens with the native Australians who use sand narratives. They draw their narratives unfolding in the same space over time, and when presented with juxtaposed panels, think that each panel is a new scene. For them, their own system inhibits this recognition of continuity across panels.

What's also striking is that there are tons of examples where this constraint is not upheld immediately — many (most?) sequences don't feature the same characters over and over in panels. This is one of the reasons that a linear approach to sequential images (like "panel transitions") just can't work.

For example, let's say panel 1 shows person A, then panels 2 through 4 show other things, then you're back to person A at panel 5. You can't just integrate 4 and 5, because you would have had to lose track of person A through 3 panels. Rather, you have to keep them and their actions in mind somehow. Transitions can't capture this relationship.

There has to be a way of upholding this constraint of continuity across longer distances — which thus requires a bigger system than linear sequence alone provides.

The Development of Language

This related to both sequential images and linguistics, so how could I not post it? Via LanguageLog:

For example...

I've been working very hard lately on a few projects and papers that have been occupying a lot of my time and energy. One of them is a write-up of my model of visual language grammar that I've been developing over the past several years. This one is particularly important to me because it will frame a lot of the issues for future studies, especially for all the psychological experiments I'm now planning.

To drive home the points in this article, I've been trying to use a great deal of example sequences from various comic books, which shows the diverse structures involved in examples that have not just been created (or altered) by me. However, the amusing thing about this is that comics themselves feature a lot of wild and diverse topics, making my examples often dealing with wild themes.

Often linguistics papers have reasonably cut and dry example sentences. Not mine: my papers are filled with guys fighting zombies, samurai chopping each other in half, people psychically blasting each other, names being carved on the moon, and sex toys being cut off people's hands.

Whatever might be said about my theories, my papers at least have crazy examples!

Camera angles and meaning

Kraft, Robert N. 1987. The influence of camera angle on comprehension and retention of pictorial events. Memory and Cognition 15 (4):291-307.

Kraft explores the semantic associations made to different camera angles (high, eye-level, low) in a four frame photo story. Subjects were asked to rate the story along a 7-point scale, use a recall test for remembering the order, and then a recognition task. Each story used two characters, which were contrasted in each frame position with differing angles in different story sequence types.

Overall, results dramatically supported that angle does correlate with semantic meaning when comparing how characters were discerned. Low angles support senses of shortness, weakness, afraid, timid, and passive, while high angles were thought of as tall, strong, unafraid, bold, and agressive. A lesser correlation was found to value judgements like good/bad. Eye level angles did not contrast between characters. These results seemed to be sustained across several experimental tasks.

In recall tasks, analysis did show that camera angles influenced a connotative meaning for how characters were remembered.

Explanations for this correlation claim it comes from of our experience with the visual world, such as how looking upward at taller people gives them a sense of power (like children to adults). An alternative view says that the different angles allow the viewer to see different things in the images, from which they draw the semantic implications.

If these results extend to drawings, it would be interesting to do further study on the semantic correspondences. I find it dubious to fully believe both of the reasonings above, at least in a universal sense. There is no semantics attached to the aerial view in Australian sand narratives, nor do fixed high angles in Japanese children's representations or ancient Asian graphics have any semantic correlation that such a theory would require.

Rather, this may simply be a case of learned conventions. We've built up these meanings by continually viewing them, particularly in movies. Or... the "visual world" explanation could be valid, but only in systems that allow for flexibility in viewpoints, not those that have fixed perspective. To be honest, I had always had doubts about claims that camera angles had semantic meanings, so I'm glad there's actually work that backs it up.

Review: Drawing Words & Writing Pictures

Drawing Words and Writing Pictures by Matt Madden and Jessica Abel

Before I started reading DW&WP, Matt Madden warned me that it was a book for praxis, not theory. As amusing as I find it that such a disclaimer needs to be given to me, the book draws upon aspects of theory throughout in an informed and well-measured way, and I am lead to thinking further about the relationship between theory and praxis.

First off, this book is a great resource. It’s put together well, lays out all the essential issues from hand lettering to stretching to avoid tendonitis (I know from personal experience: important and overlooked!). It even has homework and lesson plans, along with digital resources as well. For praxis, this all is fantastic.

However, no review of mine should escape looking at theoretical issues and reading this book has made me once again consider how praxis can draw from theory.

For instance, an immediate question of mine was: Why is it important to include defining “comics” at all? The book — wonderfully titled — talks entirely about the process, what I would say “writing in visual language.” So, why spend additional space trying to shoehorn various social manifestations of “writing pictures” (i.e. manga, comics, graphic novels, etc.) into the umbrella of “comics”?

First off, why should people who are aiming to create visual stories necessarily care about the arguments for what are or are not "comics"? These issues may be important for scholars, but most who read this book can just go by the "I know it when I see it frame." Truly, if they needed to have their "horizons expanded" by a broader definition, they probably aren't the ones reading this book.

Furthermore, as I see it, inviting readers into the subcuture of “comics” is unimportant to the aims of the book. Especially since they do well to state the applications of the ideas beyond genres and styles, this book is not about “drawing comics” — it’s about learning to be a visual writer. To this end, mentioning “comics” as a cover term at all belies this broad goal. What better way can people expand their applications of “writing in pictures” than by not immediately being co-opted into a subculture that they might not desire being in? Let them learn the visual language, then let them decide what they want to do with it on their own terms.

With the course of instruction, I thought the emphasis on thumbnails and not on scripts was a great choice that isn’t focused on enough. This is the central place that “writing in pictures” happens, and the assembly line style with scripting skirts this step often. It nicely reinforces the theme that this book is teaching people to be authors, not cogs in a manufacturing wheel.

I do have some problems with the instruction of panel transitions as how authors are guided to think about their craftsmanship. Now, heavily influenced by McCloud as a teenager, I did go through a period where I thought in terms of panel transitions (or at least I thought I did), and I certainly benefited from it.

However, when I look back on my thought processes with a broader theory in mind, I realize that I wasn’t just looking just one panel ahead as transitions would have us think. Really, I planned for whole sequences, to the point where (in thumbnails) I might draw an expected panel later in the sequence before filling in the ones in between. Such a process would predict that we are thinking in terms of whole sequences not linear panel-to-panel relationships — as my theories of sequential images imply. I suspect that others have had similar experiences.

On the plus side, what transitions do allow is a directed focus on storytelling methods that convey aspects of the scene beyond just the actions. They provide a cover for people to think about whether to slow “time” down, show other characters or the environment, etc. However, if we were to develop a more robust vocabulary of types of panels to be used and the potential for what panels might contain, we might not need the bootstrap of “transitions” to couch it in.

In fact, while I think that learning theory can be beneficial to practice, perhaps what would be a more direct and simple learning tool would be to see that theory in action. For example, in section 3, they discuss rhythm and pacing — the decision making for what to show and when to show it and use several variations of stories where they demonstrate different pacings. This type of section could be expanded to account for the types of things covered by transitions.

I would also have enjoyed seeing a greater discussion of page layouts and the uses that one can make with them, especially regarding meaning and rhythm. While I disagree with McCloud that the size of panels has an effect on narrative time, I do believe it has an effect on pace — the rhythm and meter of reading. Elaborating on these concepts would be very useful for beginning authors.

Finally, while it has recently become a point of research for me, I found their comments on reading order a bit wanting. Despite their specific advocacy to not use “blockage” scenarios (where two panels are vertically stacked to the left of a long panel) they praise another page by Mike Mignola that uses this layout just three pages prior. The overall message of “using unambiguous layouts” rings true, but this inconsistency (along with personal knowledge that fluent readers don’t have ambiguity in such scenarios) was a little disappointing.

Despite these issues, this book is a fantastic resource and accomplishment given other books on the topic. I wish that I had it when I was a teenager, as I probably would have devoured it voraciously, doing all the exercises and then some. Indeed, while I probably would have gotten more mileage out of it then, I plan on using its resources here on out.

Visual Linguistics of "Comics" Course

I am ecstatic to say that I've just learned my course for teaching a "Visual Linguistics of 'Comics'" course next Spring semester '09 has been approved! This will be the first course of it's kind to cover my own visual language research and related studies in a complete package.

I'm beyond excited about it, so... if you're in the Boston area and might want to head over to Tufts for some spring classes, stay tuned.

Panels as Attention Units

I stumbled across this article recently about how current theories of perception are similar to what magicians have been exploiting for years. Essentially, the idea is that we can only "see" what our attention is focused on at a given time. They liken it to a "spotlight" which roams around and only let's you take in certain things under its view. Though in the case of vision all the things out of the "spotlight" are still within your visual field. You just don't "see" them.

As I discuss in this video, panels in the visual language used in comics serve to facilitate this same sort of focusing of attention. Most of the time though, panels serve to exclude all relevant information except for the elements that need to be focused on, or at least clearly distinguish what is relevant from irrelevant. This lets panels provide a graphic manifestation of this mental "spotlight," allowing the author to control that attention instead of the reader's wandering eyes (which is one of the reason's I formally call panels "Attention Units").

This ties into the argument for why you don't want to overload a panel with too much stuff, because it becomes too hard to disentangle the attentionally important from unimportant elements. (If you still want to pack info in, inset panels help facilitate this honing of attention).

Even more, when you have too much in several panels sequentially, it becomes too difficult to track all the changes and carry-overs from one panel to another. This is what gives way to things like "parallel cutting." By switching back and forth between two (or more) scenes, you can highlight the individual aspects of each in panels without risking it becoming overlooked for other information or overloading the system. Of course, doing so introduces other processing demands on the visual grammar, but at least your attention is focused exactly on what is intended to be conveyed.

Review: The System of Comics by Thierry Groensteen

Groensteen, Thierry. 2007. The System of Comics. Translated by B. Beaty and N. Nguyen: University of Mississippi Press.

Printer-friendly pdf

The best thing about the new translated work The System of Comics by Thierry Groensteen is that it hopefully reflects an increase of English translations of international works on comic theory. There are numerous offerings by European, Japanese, and South American authors that rarely make their way into American scholarship, and more exchange of ideas can only be fruitful to the field. Groensteen is regarded as one of Europe’s leading historians and theorists of comics, and in its original French, the Systeme de la Bande Desinée is heralded by many as a "must-read,” so it would be a logical starting point for such a trend.

Moreover, Groensteen’s book is offered by many as a “serious” alternative Scott McCloud’s Understanding Comics, which has gained massive popularity, especially in America, yet remains dismissed in many scholarly circles since McCloud himself is not an academic. Comparisons between the two works has become commonplace (including the book’s own introduction), and this review will be no exception. For his own part, Groensteen does not engage with McCloud's theories except in a singular passing endnote, which is a shame since their ideas share many similarities.

While my French is not sufficient to assess the overall quality of translation, it certainly did not read comfortably. At times, the English felt unnatural, and word-choice often seemed clumsy if not uninformed. For instance, calling the psychological device of an "eye tracker" an "eye path follower" betrays a lack of competency and/or desire to find the accurate vocabulary (whether on the part of the translators or author is unknown). Additionally, while the endnotes were often helpful, the absence of a reference section is quite noticeable, and the endnotes do not include all the cited bibliographic information. As an “academic” book, such an oversight is a bit peculiar.

Nevertheless, the writing should only be a surface issue to the actual ideas in the book. Perhaps the most appropriate place to start is where he does: with the definition of "comics.”

Problems with definitions

Groensteen begins with what has become a common exercise in the study of comics: defining “comics” itself (no doubt bande desinée originally). He carefully deconstructs the faults of various definitions that have been proposed by various realms of scholarship. He rightly shows it cannot be guided by a single "essence" like text/image interactions, and decries a definition of comics as episodic narratives, among others.

Rather, at the outset, he identifies "comics as a language" a “system” that arises out of the "combination of a… collection of codes,” most strongly motivated by “iconic solidarity” — a fancy term for the contribution of several images. While somewhat more flexible, this emphasis on the visuals sounds a great deal like McCloud’s more rigid “juxtaposed sequential images,” but lacks a reference or engagement with his ideas despite discussing many other scholars works who Groensteen disagrees with.

Truly, in its similarity, Groensteen’s definition falls into the same formalist trap as McCloud's in failing to separate the structural notions of creating images/writing from the socio-cultural role of “comics” as objects/artifacts. This habit may have been inherited by them both from Kunzle’s influential The Early Comic Strip, whose recasting of the term “comics” on pre-1800s sequential images engendered many (like McCloud) to seek out a definition to include all possible historical examples. However, this goal is a red-herring, and as Horrock’s points out, there is no reason for sequential images from diverse historical contexts to be bound to the same socio-cultural context of the contemporary notion of “comics.”

Indeed, "comics" as a social artifact refers to numerous qualities, including 1) physical objects (strips and books), 2) a collection of genres, 3) an industry, 4) a culture/community, and others that are all tied to a context of the modern era. On the other hand, sequential images do create a language: a “visual language” that combines with text to be used within those social objects called "comics." "Comics" are not this visual language. "Comics" are a social object written in a visual language that combines with text. If novels or magazines are written in English, why should “comics” be a language, instead of be written in a language?

While Groensteen strives to strike out an abstract notion for his definition, he frequently reinforces the conflation of "comics as a medium” with "comics as a social artifact” — particularly a physical object. He writes that comics owe their birth to the technological development of lithography (8). Further on, his analysis of both word balloons (69-75), panels as “measurable in square centimeters” (28), and the expressive power of sequential images through “arthrology” rest wholly on the physicality of and across pages. However, physical objects cannot be languages — especially in the abstract sense of them being a “code” — and his reinforcement of it runs contrary to his own definition of “comics” that claims to guide the broader work.

The "System"

As invoked by the title, Groensteen’s work seeks to sketch a “system” by which “comics” operate. To his credit, he carefully moves in analysis through varying levels of interactions in comics’ structure, accounting for most of its forms. He eschews the method of previous structuralist approaches to treating the form as a “language,” which aim to dissect the medium into its minimal units, instead aiming for its broader “articulation” — larger levels of structure. However, Groensteen’s grand system is little more than an extensive taxonomy with terms that essentially mean “the principle by which (pick your taxonomic portion) operates.”

Groensteen begins by discussing the “spatio-topical system,” the various spatial elements at play in comic pages. He first notes the “hyperframe” as the delineated space of the page, in contrast to the “multiframe” — meaning the relation of all frames that constitute a comic piece, including “the sum of the hyperframes.” This chapter also includes discussions of the functions of a panel, usefulness of margins, positioning of balloons, and various facets of page layout, such as inset panels, the “strip” of a tier of panels, and possible functional roles for types of layouts.

Now, it's not that I think Groensteen's theory of comics is invalid or wrong, but that it is uninteresting.

While at times insightful, he labors through discussing details of relatively lackluster observations, such as the various ways in which a balloon visually interacts with a panel border. It is understandable why a taxonomic distinction can be made between balloons that touch the panel border and those that overlap it, but does it really add substantially to knowledge about the function and understanding of balloons as a graphic/narrative element on its own?

If this discussion led to a substantial observation about the constraints that this interaction creates on such a relationship, this breakdown would seem more significant (for instance, that two balloons cannot cross tails such that their speakers are in opposite panels than the balloons). However, no revelation of this sort is reached, and most of his analysis focuses solely on the physical aspects of the relations of balloons to panels. Such is the features of most of his discussions. All of this leaves the question of what this analysis offers a theory of comics understanding except for surface descriptions of a (fairly banal) phenomenon?

The next two sections are devoted to the principle of “arthrology” to describe either the linear relations of panels to each other (restrained arthrology) or the relations that one panel might have to others in a non-linear sense (general arthrology). Unlike McCloud’s panel transitions, Groensteen does well to recognize that panels make connections beyond their immediately juxtaposed neighbors, yet does not give any hint as to how. McCloud’s transitions at least attempted to characterize relationships between panels, which is largely why the approach is so appealing, but Groensteen leaves such detail aside, preferring instead for gross scale abstract principles. He describes “braiding” as the principle guiding arthrology — essentially the function of making connections across the multiframe.

On the surface braiding and arthrology appear to be about the creation of meaning, like McCloud’s “closure,” or perhaps a comic version of Halliday and Hasan’s (1976) classic notions of local and global discourse coherence, which Saraceni (2000) also attempted to map to comics’ analysis. However, Groensteen does not even come close to talking about semantic concerns in this way — despite purporting to (the back cover describes the book as “An authoritative exploration of how the comics achieve meaning, form, and function.”).

Really, braiding and arthrology are a theory of compositional relationships. What Groensteen focuses on is not in any way a system of how the content of panels lends towards making meaning — it is primarily about the relationship of the visual composition within a panel to others on a page and other panels scattered throughout the broader work. In particular, his theory observes the recurrence of compositional or thematic similarities (such as motifs) across varying distances of space, be it a page or a whole work, and whether the position in layout of those panels has significance (such as in the first or last position of a page).

While these observations provide a very different perspective on comics pages, they do not even scratch the surface of the “holy grail” of questions about comic sequences that they appear to address: how do sequences of images create meaning? Indeed, Groensteen dismisses such aspects as merely about "storytelling."

Rather, arthrology, like the other aspects of Groensteen’s system, once again highlights a physical feature of comic pages in what is proposed as an abstract code of understanding. The theory treats the medium as an “art object” to be analyzed like a multi-canvas painting, as opposed to a communicative medium in the McCloudian sense. Indeed, such an approach would be akin in verbal language to recognizing that various patterns of sound appear throughout different words in a story — perhaps telling you something about phonetics, but nothing about meaning. All told, Groensteen’s system limits itself to only describing the surface aspects of the medium, without pushing towards any deeper constraining principles.

Despite the criticism of the workings of his system, Groensteen is nevertheless an astute observer of various components of the comics form, and his numerous insights on comic elements betray a deep commitment to dissecting the medium. The System of Comics does reveal gems of this intuition at times when discussing various components, though the overall theoretical architecture in which they are embedded is not commiserate with the value of their insight.

Scholarship and Paradigm Shifts

The most troubling thing I found about The System of Comics is the overall orientation to scholarship that this work represents. Throughout, Groensteen’s writing conveys an attitude as if the theories are entitled to be significant, echoed in the introduction where Beaty and Nguyen hail it for emerging from the rich semiotics tradition, as if that inherently legitimates its ideas. This is a direct knock to McCloud, who they note has “been criticized for…lack of theoretical sophistication” (1). Though, this criticism has only come from academics unfortunately perpetuating the stereotype of ivory tower snobbery, as if scholarship must be done in the academy. It is hypocritical at best for any scholar to deride McCloud for not engaging a broader literature while lauding a book with very similar ideas to McCloud’s without any citation or discussion of them. So much for the value of “engaging the literature.”

Truly, Groensteen is the anti-McCloud, keeping the keys to theory locked away in the ivory tower, reachable by only those willing to slog through the exposition to reach it. While McCloud’s work about comics is presented in its visual language, there is some irony that The System of Comics takes “iconic solidarity” of images as its crux, yet is almost devoid of graphic examples. Perhaps this is why McCloud is ridiculed so much by Groensteen-minded theorists: he willingly gives away the goods to the rabble without a fight.

If McCloud lies on one extreme of being too accessible (as if that’s a bad thing), Groensteen’s work is the inverse, reflecting the worst of academic jargon and inapplicability of “theoretical sophistication” — to the extent that the terminology obfuscates the actual theories (to academics and layfolk alike). Groensteen offers complicated names and lengthy descriptions to what are otherwise fairly facile observations about surface phenomena.

Moreover, numerous questions are left unanswered, for instance how this theory is useful or applicable to 1) describing how this medium of sequential images communicates, 2) contrasting various comics’ structure with each other, or 3) describing the relationship of the visual language in comics to other modes of human expression?

Groensteen’s “system” accomplishes none of these. These are all questions that are important to semiotics as a discipline, so why is Groensteen's approach unable to even broach them? It only states vague principles for nearly obvious observations, with fancy names and a semiotic tradition to validate its claims. It is scholarly hand-waving at its best, and a reflection of why the invocation of “semiotics” garners more eye-rolls than awe in many circles.

In Kuhn’s renowned discussion of paradigm shifts, he describes that prior to a major shift many similar theories will emerge in competition with each other, yet all pointing towards similar intuitions. In this case, Groensteen taps into the common intuition that the system found in comics is somehow similar to the system of language. Indeed, he begins with a strong statement about how comics are a language, yet his analysis paints a picture of the comic medium that is decidedly un-language-like.

While his stated focus is not on “minimal units” — thereby avoiding topics such as how the system is understood as a graphic domain and meaning-making for signs and symbols — he states nothing resembling a grammar for how the sequence creates meaning (though claiming that is his intent). If his aim is to describe a “code” that shows this is a language, one would think it would involve structures akin to language. All this leaves doubt as to whether Groensteen would actually know what being “a language” would entail in the first place (aside from metaphoric extension).

To this extant, Groensteen’s total work certainly marks a valiant attempt that can take its place next to other approaches that reflect the growing pains of a discipline, sharing the intuition that the comic form should be compared to language (including: Eisner 1985; McCloud 1993; Saraceni 2000; Cohn 2003, etc.). However, this piece and its theories are not the revolutionary new paradigm for comics that will sweep away all others, and will likely follow the path of the branch of semiotics from which it hails — being considered largely passé in the study of language.

References

Cohn, Neil. 2003. Early Writings on Visual Language. Carlsbad, CA: Emaki Productions.
Eisner, Will. 1985. Comics & Sequential Art. Florida: Poorhouse Press.
Halliday, M. A. K., and Ruqaiya Hasan. 1976. Cohesion in English. London: Longman.
McCloud, Scott. 1993. Understanding Comics: The Invisible Art. New York, NY: Harper Collins.
Saraceni, Mario. 2000. Language Beyond Language: Comics as Verbo-Visual Texts. Dissertation, Applied Linguistics, University of Nottingham.

Whatnots

Blogging has been slow lately for me what with the oncoming start of school next week. I've mainly been devoting my time to the set-up of my latest experiment, which, after a few behavioral studies, will finally start looking at people's brainwaves reading comic strips.

To give an idea of the scope of experiment preparation: I have to create 200 novel Peanuts strips using panels from existing strips, and then make an additional 600 that are not of the "normal" variety. So, 800 new strips for one study, which will all be rated and tested in three pre-experiment studies to make sure they will be acceptable to use. Oy!

I do, however, plan to post a few reviews in the coming weeks. First, I will finally post my review of Groensteen's System of Comics, though I'm debating how best to parse it up (one LONG post? several shorter ones...?), and I may actually get around to finishing my post of Abel and Madden's latest Drawing Words & Writing Pictures.

Stay tuned.

Note: Especially when things are slow around here, I'm always open to requested topics if people want to pick my brain. Feel free to send me conversation pieces!

Equivalences for "Language"

In claiming that the graphic form (especially in sequence) is structured as a language, it might help to parse out how to make the analogy reasonably. It's not as if no one has ever noted the similarities between the forms — in fact, it's fairly common. However, mistakes are often made in my opinion.

First, I assume that there is "Equivalence" between the different modalities, which can be summarized as "the expectation that the human mind/brain treats all modalities in an equal way, given modality specific constraints."

By this account, it essentially means that we would expect all modalities to feature the same sort of storage in memory for patterned signs of differing sizes and functions (from sound patterns of phonemes to sentence patterns of idioms) and feature ways to combine all those elements at different levels of structure (phonology through discourse). We would also expect development to be similar, with a critical learning period and drop offs after that.

However, this is not the take that most comparisons of the verbals and visual forms take. Rather, they often try to make direct superficial analogies between specific types of structures. For example, "such and such" is the equivalent of a "word" or "sentence." This is often why many want to claim that single images have "grammar" — because a single image has lots of information in it, like a "sentence" and unlike a "word" — even though composition within single images behaves nothing like a grammar. (...nor should we expect it to given the differences between sound and light!)

A similar endeavor has tried to find "minimal units" of the structure of the forms, following the school of Structuralism (most popular in American linguistics from around 1920-1960ish). However, again, just knowing minimal units doesn't tell you about the broader structure, and units larger than minimal units might also be useful and insightful. It also gives no beneficial comparison other than that "minimal units" exist in both domains.

All of this is an argument for looking beyond the superficial understandings of "language" and to look for comparisons in deeper, more fundamental aspects of structuring.

Diversity in Visuals

At the VaIL conference a few weeks ago, one of the frequent conversations revolved around the issue of creating a universal graphic system.

The belief that visuals are universal is not new, and largely stems from the fact that most drawings look like what they represent ("iconicity"). This is also a motivating factor behind so-called "universal writing systems" like Blissymbols, Icon Language.

However, when looking at drawings as reflecting patterns in the mind, then there is actually quite a lot of diversity. Patterned styles like those of Japanese manga compared to stereotypical American superhero comics reflect culturally diverse conventions of different populations. Even more relative are the sand drawings of native Australian communities, and the constraints they place on recognizing other graphic depictions. Several other stories exist of people not fully understanding iconic drawings as well, such as the story Mort Walker tells about natives (I can't recall where) who thought that a person's legs were cut off, despite just being "out of frame."

Given this, perhaps it's time that we get over the idea of a universal communication system, and we come to accept that populations of humans will always develop idiosyncratic and in-group tendencies.

While globalization especially has raised a desire for inter-cultural communication, diversity in communication systems may have had evolutionary advantages. Having an identity tied to your language separate from others' means you can identify your group members, and even more helpful that you can keep secrets from other groups. If it wasn't beneficial, we would all be speaking the same languages (or so the argument could be made... diversity could just be a useful "spandrel" for the way other cognitive functions happened to turn out).

Perhaps instead of attempting to create universal systems, we should instead acknowledge the diversity that comes along with the way our brains are wired for social interactions. By accepting it, we can then strive to work with the constraints that our cognition and diversity brings or allows. That way, it will cease the fighting against the tide of inevitable diversity with rose colored glasses of universality, and instead invites appreciation for relative systems and cooperation to meet common goals of communication.