For researchers working with ‘modal ensembles’ (Kress 2010: 28), where meaning is made at the intersection of multiple semiotic modes (e.g. words and images), the ability to dynamically demonstrate the convergence and divergence of meaning between semiotic modes, both within and across multiple texts, has to date been a near impossible task. It is especially difficult to demonstrate such relations across large numbers of texts.
I address this issue using Kaleidographic, a visualization tool that is capable of demonstrating patterns within and across texts (inter- and intra-textual relations) and within and across semiotic modes (inter- and intra-semiotic relations). In the following I explain how this particular Kaleidographic works. If you would like to know more about the case study and the project more generally, click here.
The Kaleidographic View of Political Affiliation in the #ausvotes Corpus on Instagram demonstrates both inter- and intra-semiotic relations (between the visual and verbal modes) and inter-and intra-textual relations (both within a single text and across a number of texts – 2279 instances from the #ausvotes corpus that do affiliate or distance in some way). How Kaleidographic has been used in another multimodal project can be viewed here. And a reading explaining the motivation for the development of Kaleidographic can be downloaded from here.
How Kaleidographic works
By simply pressing ‘play’ you can watch the Kaleidographic scroll through the results of my analysis showing where (in image, verbal text, hashtag) in the Instagram post the affiliation or distancing strategies are deployed and which political party they connect with. If you speed up the player (to 10), you get a more holistic view of the spread of affiliation and distancing strategies across the dataset and where they tend to co-occur. Alternatively, you can manually scroll through each result (using the ‘next’ button) and explore the combination of strategies in each post in turn (see figure 1).
Individual segments are ‘coloured in’ when affiliating strategies are present. I have used official political party colours, where possible, to indicate when a post is affiliating with that particular party (e.g. green for the Greens, red for Labor, blue for Liberal). When a post distances itself from a party, I have used black to fill that segment for all parties. Figure 2 illustrates the colour scheme and names the political parties that were most frequently mentioned in this dataset. Support for invented parties (e.g. Vote 1 Democracy Sausage Party) and candidates (#pepperforpm, #Vote1RickAstley, #vote1cleavergreene) made a very strong showing in this corpus. Parties grouped under the ‘OTHER’ category include among others The Australian Sex Party, Animal Justice Party, Family First, Drug Law Reform Party, Australian Equality Party.
If you are interested in exploring how specific political parties are treated, you can grey out those segments you are not interested in (by pressing CTRL+Click in the segment you want to eliminate) and then hit ‘play’ to see the relations you are interested in. Figure 3 is an example showing only the ALP, Greens and Liberal/National parties.
Post number 1447 shows how this post affiliates with the Greens through both image and verbal text while at the same time it distances itself from the ALP and the Lib/Nat Coalition through the verbal text.
Likewise, if you are interested in exploring how affiliating strategies co-occur across elements, you can grey out the layer you are not interested in (by clicking SHIFT+Click to grey out the whole layer), and then hit ‘play’ to see the relations you are interested in. Figure 4 is an example showing only the image and the verbal text layers.
In Figure 4, the Hashtag layer is inactive, which allows the user to focus on how and when (dis)affiliating strategies co-occur across image and verbal text in each post. In this instance, post number 1829 endorses both the ALP and the Greens through the verbal text, while distancing itself from the Lib/Nat Coalition through the image.
Kress, G. (2010) Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge.
How to cite this page
Caple, H. (2017) ‘How Kaleidographic works’. Helen Caple. http://www.helencaple.com/kaleidographic/how-kaleidographic-works/