Embodied Connections: Meaning-Making in a Multimodal Discourse

This past week in our graduate course on Information Architecture with Professor Bill Wolff, we’ve explored techniques and design philosophies centered on information display. Our discussion was informed by selected readings from Edward Tufte’s  Beautiful Evidence, as well as interactive and PDF versions of  Fighting to Live as the Towers Died, by the New York Times. A specific area of focus was the concept of  ‘mapping,’ within the context of  multimodal composition. (As a preface, it is important to make the distinction that, for purposes of discussion, our definition of mapping is not limited to ‘maps’ in the conventional sense of cartographic renderings–although, it could certainly include the use of them.)

According to Tufte, mapped pictures are representational images that combine scales, diagrams, overlays, numbers, words, and images (13).

Multimodal-mapping presents the audience with several different kinds of information, via a single, visual plane–through a process of layering. Each ‘layer’ serves to signify a different kind of knowledge. For writers, mapping affords a visual dissemination of  interconnected webs of knowledge and causal relationships. In terms of information architecture and as a form of composition, multimodal-mapping frames and contextualizes specific intersections within a greater body of knowledge.

If we reflect on the discipline of communication, as writers, at the heart of our craft is effective communication with our audience. What this means in terms of execution–is that in our ‘writing,’ the intended message should come across in a way that is consistent with precisely what we mean to say. In other words, it is paramount that our composition–and therefore our message–not transmit in a convoluted or disjointed way.

So, you may ask, how does multimodality come into play, and why is ‘mapping’–or incorporating other elements besides plain text–important?

As our author asserts, at times, it is necessary to include multiple sources and levels of data (78) in order to illustrate connections and relationships, as well as make different kinds of comparisons; the use of additional kinds of information assist to better explain what we mean to say. For example, the strategic arrangement of annotative and  typographic elements–lines and arrows–as well as colors, images, and text, all function together to transmit a message that is dynamic, visually engaging and robust. Important to note, is that these elements also function to provide context and directional navigation to the page–therefore, enabling the reader to clearly ‘decode,’ or ‘read’ the message.

Another facet of multimodal-mapping is that it offers the capability to take extremely complex, high-density information–such as that of cartography, brain research, molecular biology, physics and other high-resolution fields–and re-present it in a way that the linear data becomes embodied into a single, compact, visual representation. Tufte refers to this type of visual information as sparklines or data-words (58). This definition lends itself to that of semiotic metaphor. Another way to imagine sparklines or data-words, is to think of a ‘constellation of information’–or individual, interconnected pieces of knowledge that come together, making up a whole. An example of  a sparkline is the double helix of DNA encoding (13) or Tufte’s representation of all 65 of Galileo’s published observations of Jupiter and its satellites. (Pertaining to Tufte’s re-design of Galileo’s work, we note that when the original, ‘intervening’ text was omitted, a new architecture of information presented itself, resulting in the visual evidence becoming adjacent, sequential, linked, moving (108)–a continuity became present that was not there before.)

To further extrapolate on this concept, imagine an illustration of an adult, human skeleton. Now, in order for the message to be clearly understood, specificity is essential–what is needed is clear direction, scope, and scale. With a nod to the author, presenting ‘everything’ results in contextualizing nothing (31)–it conveys a message that is abstruse. So, we don’t want to focus on the ‘whole skeleton’ per say–instead, what we want to focus on is one area. Say, it’s the left leg–and even more specifically, the tibia. According to Tufte’s definition of beautiful evidence, every image presented must reside within the universal measurement grid (45). So, connecting back to the example I discussed above, what we are concerned with, is the portrayal of relationships within a given body of knowledge–showing things in context with one another provides for a local and precise way of understanding.

Tufte also discusses explanatory mappings and the practice of exploratory image analysis. Explanatory mappings appear in scientific research, newspapers, textbooks, technical manuals, legal proceedings, engineering reports and medical research (45). However, they are not constrained exclusively to these fields of discipline; they also extent into the arts and humanities and many times are interdisciplinary, because they represent complex information.

Below is image is from a journal article in Current Opinion in Neurobiology; an example of an explanatory mapping.

Networks for segregation in the human brain.

Below, this image  is a representative time-space map of the USA belonging to an article titled, The shrivelled USA: representing time–space in the context of metropolitanization and the development of high-speed transport from The Journal of Transport Geography.

During a discussion centered on one of the interactive mappings in Fighting to Live, one of my classmates commented, I don’t know when I’m supposed to stop and when I’m supposed to look at the images.

Regarding the page’s architecture of information, my classmate’s comment was reflective of bad design in terms of the page’s directional signifiers. In sum, just as a freeway has road signs and indicators of when to stop and when to go, when to turn here and exit there, multimodal mappings are beautiful evidence of what they represent only when elements therein properly signify or ‘point to’ what they were intended to, and result in a clear transmission of the message.

Categories: evidence, mapping, tufte | Tags: , , , , , , , , , , , , | 4 Comments

Post navigation

4 thoughts on “Embodied Connections: Meaning-Making in a Multimodal Discourse

  1. The comment “I don’t know when I’m supposed to stop and when I’m supposed to look at the images” in conjunction with what you’re talking about makes me think of something Professor Tweedie mentioned during my Intro to Writing Arts class. While studying Scott McCloud’s “Understanding Comics,” he said one of his former students was unable to follow the transition between words and images. Instead, he read all of the words first, then went back to look at the pictures.

    When it comes to creating a fluid transition between images and text, I think it’s important to consider not only the placement of the images but also how to ensure the reader looks at them in the correct order and at the right time. In the blog post above, it’s simple, since first there is an introduction, “Below this image is…” followed by the image in question. That’s simple and straightforward. However, in many textbooks the images are off to the side and separated from the text. Typically, I’ll pause at the end of a random paragraph to look at the pictures. However, if the pictures are incorporated more fluidly into the text, the reader will know when the RIGHT time to look at them is. That would certainly help make for a more connected and less disorderly experience.

  2. This goes back to the very engrained teaching methods we have. Images are too often thought to be most useful as supplements rather than an important way to further understanding. This is a Grammar A bias, and a control issue. Textbooks and presentations that fail to present beautiful evidence, not out of any sense of evil, but out of a flawed belief that this is simply how things are done, that they are the ones with the knowledge and the audience therefore should remain quiet and unquestioning. Being more vocal as an audience in calling out flaws is according to Tufte crucial, it is our end of a ethical bargain to make establishment aware that there are flaws, in such a rigid system it is the only way to affect change. Information doesn’t have to be presented in a certain way just because it always has been done that way. Of course, when making such an argument, we must have all our metaphorical ducks in a row.

  3. Devon

    I definitely think there are a lot of complexities when it comes to making fluid information design. To me, some of Tutfe’s ideas are not entirely helpful. For example, Tufte would prefer the original news article because the text and the pictures are right near each other. Of course, he would probably add arrows and make the layout a bit more logical, but I still don’t think crowding the text with images necessarily helps inform readers. To me it is distracting. I was probably the person who said I don’t know where to look–I prefer simplicity, just text–maybe some pictures when necessary. I have a hard time focusing with too many things on a page. In Beautiful Evidence I could concentrate because Tufte created his pages to be read more fluidly; he placed images where they could be seen, in proper order, as needed. They did not interrupt the text because he made sure to place them after periods, at the end of paragraphs. His design shows thoughtful layout planning.

  4. “Information doesn’t have to be presented in a certain way just because it always has been done that way” — Tom

    I read this, and my mind immediately went to “Fighting to Live as the Towers Died.” Why can’t a newspaper article, especially one of this magnitude, use arrows or different fonts like Tufte does? Because of convention? I was the one who originally preferred the online version of Fighting to Live because the multimedia images and audio were placed separate from the text, so I could consume one and then the other. After reading Beautiful Evidence, I can now see that authors can, in fact, integrate both seamlessly.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: