Embodied Connections: Meaning-Making in a Multimodal Discourse

This past week in our graduate course on Information Architecture with Professor Bill Wolff, we’ve explored techniques and design philosophies centered on information display. Our discussion was informed by selected readings from Edward Tufte’s  Beautiful Evidence, as well as interactive and PDF versions of  Fighting to Live as the Towers Died, by the New York Times. A specific area of focus was the concept of  ‘mapping,’ within the context of  multimodal composition. (As a preface, it is important to make the distinction that, for purposes of discussion, our definition of mapping is not limited to ‘maps’ in the conventional sense of cartographic renderings–although, it could certainly include the use of them.)

According to Tufte, mapped pictures are representational images that combine scales, diagrams, overlays, numbers, words, and images (13).

Multimodal-mapping presents the audience with several different kinds of information, via a single, visual plane–through a process of layering. Each ‘layer’ serves to signify a different kind of knowledge. For writers, mapping affords a visual dissemination of  interconnected webs of knowledge and causal relationships. In terms of information architecture and as a form of composition, multimodal-mapping frames and contextualizes specific intersections within a greater body of knowledge.

If we reflect on the discipline of communication, as writers, at the heart of our craft is effective communication with our audience. What this means in terms of execution–is that in our ‘writing,’ the intended message should come across in a way that is consistent with precisely what we mean to say. In other words, it is paramount that our composition–and therefore our message–not transmit in a convoluted or disjointed way.

So, you may ask, how does multimodality come into play, and why is ‘mapping’–or incorporating other elements besides plain text–important?

As our author asserts, at times, it is necessary to include multiple sources and levels of data (78) in order to illustrate connections and relationships, as well as make different kinds of comparisons; the use of additional kinds of information assist to better explain what we mean to say. For example, the strategic arrangement of annotative and  typographic elements–lines and arrows–as well as colors, images, and text, all function together to transmit a message that is dynamic, visually engaging and robust. Important to note, is that these elements also function to provide context and directional navigation to the page–therefore, enabling the reader to clearly ‘decode,’ or ‘read’ the message.

Another facet of multimodal-mapping is that it offers the capability to take extremely complex, high-density information–such as that of cartography, brain research, molecular biology, physics and other high-resolution fields–and re-present it in a way that the linear data becomes embodied into a single, compact, visual representation. Tufte refers to this type of visual information as sparklines or data-words (58). This definition lends itself to that of semiotic metaphor. Another way to imagine sparklines or data-words, is to think of a ‘constellation of information’–or individual, interconnected pieces of knowledge that come together, making up a whole. An example of  a sparkline is the double helix of DNA encoding (13) or Tufte’s representation of all 65 of Galileo’s published observations of Jupiter and its satellites. (Pertaining to Tufte’s re-design of Galileo’s work, we note that when the original, ‘intervening’ text was omitted, a new architecture of information presented itself, resulting in the visual evidence becoming adjacent, sequential, linked, moving (108)–a continuity became present that was not there before.)

To further extrapolate on this concept, imagine an illustration of an adult, human skeleton. Now, in order for the message to be clearly understood, specificity is essential–what is needed is clear direction, scope, and scale. With a nod to the author, presenting ‘everything’ results in contextualizing nothing (31)–it conveys a message that is abstruse. So, we don’t want to focus on the ‘whole skeleton’ per say–instead, what we want to focus on is one area. Say, it’s the left leg–and even more specifically, the tibia. According to Tufte’s definition of beautiful evidence, every image presented must reside within the universal measurement grid (45). So, connecting back to the example I discussed above, what we are concerned with, is the portrayal of relationships within a given body of knowledge–showing things in context with one another provides for a local and precise way of understanding.

Tufte also discusses explanatory mappings and the practice of exploratory image analysis. Explanatory mappings appear in scientific research, newspapers, textbooks, technical manuals, legal proceedings, engineering reports and medical research (45). However, they are not constrained exclusively to these fields of discipline; they also extent into the arts and humanities and many times are interdisciplinary, because they represent complex information.

Below is image is from a journal article in Current Opinion in Neurobiology; an example of an explanatory mapping.

Networks for segregation in the human brain.

Below, this image  is a representative time-space map of the USA belonging to an article titled, The shrivelled USA: representing time–space in the context of metropolitanization and the development of high-speed transport from The Journal of Transport Geography.

During a discussion centered on one of the interactive mappings in Fighting to Live, one of my classmates commented, I don’t know when I’m supposed to stop and when I’m supposed to look at the images.

Regarding the page’s architecture of information, my classmate’s comment was reflective of bad design in terms of the page’s directional signifiers. In sum, just as a freeway has road signs and indicators of when to stop and when to go, when to turn here and exit there, multimodal mappings are beautiful evidence of what they represent only when elements therein properly signify or ‘point to’ what they were intended to, and result in a clear transmission of the message.

Mapped Images Provide Understanding

As Edward Tufte argues in Beautiful Evidence, mapped pictures combine “representational images with scales, diagrams, overlays, numbers, words, and images,” (p.13). Mapped pictures are very complex as they have multiple incorporations. It’s for this reason that mapped pictures are much better than just plain pictorial images. For instance, in “Fighting to Live as the Towers Died,” there are both mapped pictures, as well as simple pictorial pictures. On the very first page, there is a picture of the tower, with no added words, numbers, overlays, scales, or diagrams. While this picture does create a nice effect, adding aesthetic emotion, it doesn’t really advance any further than that. However, if you move to page two, there’s a nice outline of the North Tower that provides scales and words (labels) to help the reader associate with the text in the article. Instead of the article just talking about the different departments of the North Tower, there’s a diagram that further enforces the reality of what the article is saying. Furthermore, Tufte argues that an architectural drawing style that has a measurement scale, plain views, and labels shows that the object was “examined carefully” thus adding credibility to the image (p. 22-23). While the mapped images in the New York Times may not need credibility (as it’s the New York Times), the images do meet all of Lufte’s requirements for credibility.

Mapped images help the audience make a further connection with whatever lesson is at hand. As Tufte acclaims, a pile of “loosely related images” add up to a “coherent multiple viewpoint,” (p. 35). This claim is proven true in “Fighting to Live as the Towers Die,”.  There is a timeline of September 11^th, with arrows, pictures, words, and labels. Throughout the article there are pictures that display the faces of those that were lost (also with labels). The usage of many different mapped images adds up to show a coherent viewpoint which help aid in seeing and reading the article.

Tufte seems to think that labels should be used differently depending on the genre and context of an image. For instance, on page 42, Tufte claims that the diagram below the historical picture allows for a greater detail then placing the labels on the actual picture. I personally find this type of mapping confusing and I think it requires too much effort. I can barely figure out which shape represents each person, let alone realize that the dog is not labeled. Do you think it would’ve been better to label each person in the picture (using white labels perhaps)? I understand that this type of mapping allows for more details in the labels, but I question this too. For instance, on the next page, the dogs are labeled by name on the picture. Yes, this type of labeling is simplistic, but what if we wanted to put their breeders or some other information? I think we could also put that on the picture, underneath their names, without much hassle. Furthermore, Tufte acclaims that types of measurement should be placed directly on the photograph and that science should use this way of reporting standards (p.43). While I don’t disagree, I wonder if Tufte would agree to put measurements on the historical picture, if it were deemed necessary.

It’s interesting because, before I read Beautiful Evidence, I thought of mapped images just as the words state; an image of a map. During my reading, I thought that mapped images were strictly used in science and nature books, used to show the scales of animals and such. However, now I understand that it can and should be used virtually anywhere as it strengthens the writing it’s placed with.

All of this information helps us understand and evaluate the architecture of writing. Not only is the definition of writing becoming redefined, but the architecture of writing is also being redefined. A clear example of this is when I first started college. I took Composition 1 and 2, where I strived to write great essays and papers. The criterion for this was simple: write with a purpose, audience, and main idea in mind through clear and concise words. As I advanced my degree, it started to become much different. There were still papers and essays, but writing started to involve technology. I created a mashup video, which was basically similar to a persuasive essay. Yet I did this with only clips of videos. It’s still a form of writing, yet the architecture of it was very different from that of an essay. The involvement of mapped images in our writing will also become much more present as the idea of what writing is evolves.