On Complex Documents

IMG_2004

Immersive VR … Google Cardboard (+ random biting cat): photo by @lynrobinson cc-by

Immersive Documents

I have written previously on the conceptual and likely practical relevance of immersive documents to the library and information science community. I have defined immersive documents as those which deliver an unreal reality to the ‘reader’. Reader, in this context is a loosely defined term, as the concept of the document is expanded to embrace the type of experience afforded by technologies such as virtual reality (VR), pervasive computing and the multisensory Internet. In this case, the reader may also be described as a viewer, a player, a user or a participant, but in writing from the perspective of LIS, the term reader seems to be an apt, all-encompassing descriptive.

This idea was sparked by my reading, some years ago, of Shuman’s scenario of the library as the ‘experience parlour’, (Shuman,1989).

Immersive documents, wherein reader engagement delivers the perception of an unreal, computer-generated world as indistinguishable from reality, do not yet exist.

This type of intangible document would, like other digital documents, exist only when its overarching computer program executes, and the associated file of coded content is read and processed. An immersive document differs from the familiar digital versions of texts, images, sound and film, (whether born digital or scanned), in that additionally, it allows for varying degrees of real-time reader behavior and interaction data to be processed along with the base content, and to influence the narrative outcome. The scope of participation or interaction could range from passive watching to full body telepresence with complete agency. The term ‘narrative’ can imply that the immersive document delivers the perception of being within a fictional novel, or a game. Whilst this is certainly one example of an immersive document, the ‘narrative’ could be a rendering of an historical event, a travel or news documentary, or a training scenario.

It is possible to regard the computer file, which contains the code for the immersive document, to be a document in its own right. It would have a physical materiality manifested in the computer storage media containing the binary codes. A sort of meta-document perhaps.

Additionally, should the specific outcome/modification from any participatory interaction be recorded, resulting in a new version of the original immersive experience, then this would constitute a related yet different document. This could in theory be played back by another participant, either as a passive, or further interactive experience.

Immersive documents comprise technology, software, and novel narratives, and developments in all three component areas will be needed for such documents to be realized, although an additional, important driving factor will be the strong desire for participatory experiences from readers.

At the time of writing, Spring 2016, we are anticipating the first wave of commercial VR head mounted displays, (HMDs), from Oculus, Samsung, HTC and Sony, which work with compatible computers and software to render stereoscopic, computer generated, virtual worlds, accompanied by sophisticated sound. These environments are compelling, and the ones I have tried deliver a realistic experience of being in a virtual world as an observer.

At this stage however, the reader is not able to fully interact with the environment or content; in programs which support telepresence, this only extends to feeling parts of the body. Doubtless, as technology advances, a fuller sense of body presence in the unreal reality will emerge, but this needs to be matched by the authoring of scripted worlds, to allow for more reader determined behavior and interaction with elements of the unreal world portrayed. [See, for example, http://motherboard.vice.com/read/tribeca-film-festival-2016-virtual-reality-film]

In support of an enhanced feeling of immersion, the HMD interface needs improvement, as at the moment it is rather clumsy and restricts the sense of full immersion, or the suspension of disbelief. Pervasive, wearable technology will undoubtedly improve to the point at which we become less/(un) aware of the interface, and we can look forward to contributions from the fields of neuroscience and psychology in reducing the friction between the reality/unreality interface. EEG headsets already allow merging of brain signals with the machine, brain-machine connection, so it will doubtless be an incremental step for this type of data recording to feed into immersive documents to simulate all five senses. [see for example, http://www.independent.co.uk/news/science/drones-brain-thoughts-controlled-bci-brain-computer-interface-brain-controlled-interface-a6996781.html]

See also, work on implants, leading to body–machine connection; cyborgs and biohacking.

Advances in multisensory transmission over the Internet, i.e. smell, taste and touch, will further enhance our ability to make the unreal, real, and at a distance.

Although fully immersive documents do not yet exist, it would be prudent for the LIS community to consider at this stage, whether the sector should play any part in the handling of these entities, and if so, in which ways. It will be easier to collect and record the documents as they emerge, if frameworks for understanding and description are already in place – thus avoiding the enormous retro-conversion efforts needed to redesign and extend current bibliographic data to enable semantic web functionality and promote discovery.

Partially Immersive Documents

Partially immersive documents do exist, and this prompts enquiry into how these can be recorded, stored, described, discovered, shared and preserved. Whilst LIS related work on these partially immersive entities is scattered amongst other disciplines, and in no way comprehensive, it is a worthwhile source of material relevant to the handling of future immersive documents, and is by its nature surely of interest to the LIS community.

Partially immersive documents may be distinguished from other analogue/physical or digital documents, because, like immersive documents, they allow for, and may even require some level of input or participation, from a data source or a reader. The distinguishing feature of these documents from other digital entities is that they are dynamic, not static.

These partially immersive documents may be divided into two categories.

Firstly, born digital entities, such as: visualizations, simulations, interactive narratives, videogames, virtual worlds, 360 digital video recordings and digital artworks. These documents all furnish the reader with varying degrees of unreal reality.

As with fully immersive documents, the level of participation within partially immersive documents can vary, from almost passive observation through to meaningful interaction – that is interaction which changes some aspect of the documentary experience.

In contrast to fully immersive documents, however, real world elements are present and noticeable, even if the reader is too ‘engrossed’ in the document to notice them.

These partially immersive document entities also exist as computer files containing content together with display or processing instructions, which require specific technological platforms on which to run. They exist only when the content data is acted upon by the software instructions. In many cases, such as with interactive narratives, there is scope for real-time data input from the reader, which generates novel content. Formats such as visualisations, simulations and digital art can all rely on other program data for input as well taking input from a human reader.

This complex, dynamic nature demands a more detailed approach to document handling than that used for digital files representing more conventional (often originally physical) types of static document, such as books, journals, manuscripts, datasets, sound, images or films, even though standard metadata for these more familiar documents may still need to be agreed, and issues of preservation in perpetuity remain.

Secondly, we need to consider partially immersive documents which are ephemeral, temporal, intangible, real world activities. Examples include theatre performances, dance performances and installation art. The level of reader participation may vary between passive reception of the content, to active engagement.

In these cases, the sense of unreality as reality is more related to suspension of reality in the mind, as the readers are at all times perceiving a real world event, even if a fantastical one. These documents need first to be recorded in order to be preserved for future access and understanding.

Augmented reality, and mixed reality events offer yet more time dependent document forms, blending physical world immersive events with the digital.

Complex Documents and the Information Communication Chain

Immersive and partially immersive documents may be thought of as ‘complex documents’, for which an interdisciplinary approach to their information communication chain journey may be beneficial, in contrast to the solely LIS focused efforts to record more usual document forms.

One area in which a significant amount or research has been done is that of preservation, and there are several disciplines which have started to consider how to describe and record complex documents within their domain.

The JISC (Joint Information Systems Committee) organized the POCOS project (Preservation of Complex Objects Symposia) in 2010, which considered complex documents as complex digital objects. To simplify the many types of objects under scrutiny, they were divided into three catagories and investigated from the perspectives of simulations and visualisations, software-based art, and gaming environments and virtual worlds. The original publications from the project are published in “Preserving Complex Digital Objects” edited by Janet Delve and David Anderson, Facet, 2014.

One of the key insights to the description of complex documents comes from consideration of the recording and preservation of dance. Firstly, we need to understand what has to be recorded. At first glance, it seems that we could perhaps merely take a video recording of the performance. But on further consideration, and indeed conversations with dancers, it becomes clear that there each performance (or dance), comprises many layers. It is likely that these layers exist for all complex documents. An initial list of things to be coded, and recorded in description and preservation systems includes:

  • 2D visual recording of the performance
  • 3D visual recording of the performance
  • coding of the movement, feelings, intention of the performers
  • sensory, (anatomical, electrical signal) recording from performers
  • mental, descriptive, narrative from performers
  • similar recordings from the creator, director
  • psychological, physical reaction from participants
  • overall impact of the performance

Perhaps the main challenge to handling complex documents within the information communication chain comes from reader participation.

Participatory behavior can take many forms, from allowing software to read facial expressions, or to measure pulse or heart rate, to allowing full interaction with objects, characters or avatars within the simulated environment. This data is complex to record and process with respect to the base document, but is also complex to add to document description. In some cases, there is more than one participant, adding to the complexity. For the purposes of recording and preservation, there is the question of what is being recorded, and how authentic is the replay?

In some games for example, there are hundreds of participants. Likewise for immersive, participatory theatre. Once we consider participation, we are faced with an additional layer to the recording, description and sharing of the document – do we record the viewpoint/experience of the participant? Can we? There are thus dual (possibly multiple) reader modes; that of a passive observer, a first time reader who is interacting with the document, or that of previous readers, engaging with something previously experienced..

It is perhaps surprising that the question ‘what is a document?’ is still unclear, five and a half thousand years after records began. Humanity nonetheless, still endeavors to record and preserve more and more layers of the human condition.

 

References

Shuman BA (1989). The Library of the future. Alternative scenarios for the information profession. Englewood CO: Libraries Unlimited.

LR update 2/05/16

 

On The Pogles: What makes an immersive document?

Here is an episode of The Pogles, a children’s television program from the 1960s. The film is in stop motion, made by the wonderful Oliver Postgate.

I write nowadays about immersive documents emerging from exotic technology, new narrative writing methods, and a populace madly keen to escape reality via participation in unreal realities. Except that, just sometimes, we can be transported into the unreal realm by far simpler mechanisms. I have written previously about immersive stories (see here), and in this small piece of writing I invite you to spend 9 minutes or so telling yourself that Pogles and magic plants don’t exist!

Note also:

1) In the 60s, alcoholic beverages are a *good thing*

2) In the 60s, there is no email and so plenty of time for adventure

Are the Digital Humanities and Library & Information Science the same thing?

LIS=DH? 2015

@lynrobinson cc-by

The following is a summary and further discussion around the presentation which I gave in May at ISI 2015, held at the University of Zadar, Croatia. The original paper, written with Ernesto Priego and David Bawden, was part of the stream on exploring the disciplinary boundaries of library & information science, and is published as: Robinson L, Priego E and Bawden D (2015). Library and information science and digital humanities: two disciplines, joint future? In: Pehar F, Schlögl C and Wolff C (eds.) Re-inventing information science in the networked society. Glückstadt: Verlag Werner Hülsbusch, 2015, pp 44-54. 

LIS:DH on GoogleThere is much talk of the relationship between digital humanities (DH) and library & information Science (LIS) these days. We know this, because a quick and dirty Google search throws up some links to serious thoughts on how the two disciplines are interrelated.

A more detailed search of the current literature further reveals an increasing number of papers within the LIS domain which discuss how the concerns of DH are linked to the traditional concerns of the LIS discipline, and to the core competencies of librarianship. This, added to the increasing focus on digital collections of major LIS players such as the British Library and the Wellcome Library, suggest it is timely for those of us involved with teaching and practice to examine the close relationship our discipline has with DH. Further encouragement can be gained from noticing the prominence in London of DH centres and courses, the transition to digital scholarship, and the critical mass of social media datasets available for analytics and visualisations.

LIS has always been described as a broad discipline (see Bawden and Robinson 2012). This has often been seen in a negative light, as its components are readily appropriated by related subject fields, such as computer science, linguistics, psychology, human computer interaction, management and publishing. However, if we focus on the fact that LIS is centred on the topic of information, instantiated as documents, we can see that LIS is about all the processes of the information communication chain (Robinson 2009). No other discipline is interested in documentation in this way. Further, it is noteworthy to consider how many members of related disciplines are, in reality, thinking and practicing LIS.

So to digital humanities. The following three questions present themselves:

  • How is DH linked to LIS as a discipline?
  • Can this be modeled?
  • How should evolving DH content be integrated into LIS masters courses?

In order to answer these questions, an analysis of a selection of the literature was carried out. Relevant articles were identified by searching for LIS and DH terms in the title, abstract and descriptor fields of Web of Science, LISA, LISTA and Google Scholar. As papers of this nature are relatively recent, no time limits were applied. The articles were then carefully read to elicit themes, models and suggestions for LIS curriculae. References at the end of each paper were scanned for further insight.

Links between DH and LIS disciplines

The following six themes emerged from  twenty two relevant papers. It is not suggested that these themes are definitive, they are however, indicative of the ways in which DH and LIS are related.

1) Place

  • DH research is often sited within collections institutions: libraries, archives, record centres, museums
  • Institutionally, LIS and DH are often located together in academic units

2) Documentation

  • LIS and DH are associated with the academic use of recorded information
  • The focus of DH and LIS is on documents, where the term is understood in the broadest sense, including datasets
  • The literature describes the categories of interest in documents for both LIS and DH as: resource creation, dissemination, search and retrieval, (digital libraries and archives), metadata and resource description, open access, linked data, collection management and curation, portals and repositories, preservation, interactivity and user experience to name but a few
  • There is a tension in both fields between the status as an academic discipline, and that of a support service

3) Education

  • Educational programmes in LIS are increasingly including DH material and vice versa. This is reflected in the requirements for skills and competencies from LIS accrediting and professional bodies

4) Journal Literature

  • Although DH has its own journals, and papers may appear in the literature of the humanities or computer science, DH research is often reported in journals primarily regarded as LIS resources
  • Sula (2013) showed a steady increase in DH related publications in the LIS sources in the LISTA database between 2005 and 2012.
  • A search for the phrase ‘digital humanites’ in the title, abstract or index terms in the Web of Science and LISTA databases, carried out in January 2015 for papers published in 2013 and 2014, found DH papers in a wide variety of LIS journals, emphasizing the continued relationship between the two disciplines

5) Pedagogy

  • LIS and DH have embedded pedagogical practice; LIS instantiated as information literacy and DH as champions of education for the use/development of related methods, tools and technologies (DH literacy).

6) Challenges

  • Considering the background and nature of LIS and DH, both are broad disciplines, focusing on information or recorded knowledge, instantiated as documents
  • Both disciplines lack agreed definition
  • Both disciplines face contested futures

Models

Cultural Informatics Model  for Digital Humanities and Libraries, Sula 2013

Cultural Informatics Model for Digital Humanities and Libraries, Sula 2013

One model showing the relationships between LIS and DH was found, Sula’s conceptual model (2013), based on a cultural informatics framework. This model shows the ways in which LIS and DH interlink across the dual spectrums of first-order to second-order content, and human to computer driven tasks. From this picture, we can see a close relationship between the activities comprising the two disciplines. However, the information communication model (Robinson, 2009), offered as a basis for LIS, could also represent the concerns of DH at a more fundamental level.

Information Communication Chain Model, Robinson 2009

Information Communication Chain Model, Robinson 2009

DH Content in LIS masters courses (and vice versa)

A look at the content of LIS and DH courses in the UK/US reveals crossover of content such as web content design and development, resource creation, organization, preservation, publishing and dissemination, metadata, data analytics and visualization, literacy and pedagogy, digital culture and policy. Again, we can see a close connection between the disciplines.

Conclusions

The original paper concluded with support for the well documented similarity between the two disciplines, and in anticipation of closer convergence in the near future, especially with regard to the content development for masters courses in LIS.

However, on putting together the slides for the verbal presentation, a stronger question emerged, asking whether DH and LIS could, in fact, be the same thing. This suggestion, understandably, provoked some further discussion amongst the delegates, and it seems worthwhile to record some aspects of the debate here.

Are the Digital Humanities and Library & Information Science the same thing? 

This is not a question that can be answered definitively in a single presentation, nor in a follow-up blog posting. But, on reflection, it is probable that the answer to the question is ‘sometimes’, depending upon the context.

Let us consider first, that LIS can be understood as a meta-discipline, one which can be applied to any subject or field (reflexively including itself). It is about the communication chain of information (instantiated as documents) within that field. LIS supports and underpins all disciplines in the act of collecting, indexing, disseminating, storing, preserving and sharing of knowledge, for the ultimate purpose of promoting understanding. It is arguable that without LIS, there would be no understanding, nor hence progress, in any discipline beyond that which can be carried within the mind of an individual.

A medical subject expert, physicist, mathematician or engineer, can subsequently take an interest in the communication aspects of their discipline, either with a desire to work as an information professional, or increasingly to function more adequately as a subject specialist. Likewise, experts and practitioners from the arts and humanities disciplines can also acquire the additional skills and understanding of the communication of documents within their field.

This traditional understanding of roles allows us to separate subject specialist expertise from the activities associated with the practice of LIS. Although the two are certainly complementary, an engineer is not necessarily an engineering information specialist, and a specialist in engineering information is not necessarily an engineer. The same argument goes for other subjects, including the humanities. (See Hjorland 2002).

This traditional, and often reassuring, distinction between subject specialism and LIS is now however, rather more blurry.

The definition of a document has evolved over the past 50 years to include digital media files, but perhaps it has changed most significantly over the last 10 years, with the advent of Web 2.0, where the separation of form from digital content spearheaded the creation of a myriad new resources. Additionally, the emergence of e-science resulted in new documents in the form of digital datasets, and most recently we are working with the move towards digital scholarship, recording not just data/metadata, but also process and methods, across all disciplines, not just in STEM subjects.

Accompanying this change has been a shift in the scope of LIS, to accommodate not only the handling of a more diverse spectrum of documents, but also to embrace an increasing emphasis on analysis, interpretation and explanation of document contents. Increasingly, LIS practitioners offer support to researchers in curation, understanding and interpretation of research data sets, often across disciplines. This may be seen as an extension of their longstanding role as champions of information literacy, potentially with echoes of support for evidence based medicine, and embedded librarianship. On the flip side, researchers too, are becoming more aware of the importance of LIS skills and techniques to record and facilitate their work, so that the boundary between subject specialist and LIS expert becomes less distinct. Consider bioinformatics, chemoinformatics and social informatics.

Indeed, LIS practitioners are themselves increasingly the creators of new documents, for example, in digitizing and disseminating local holdings. Driven by the digital, there is a need for all of us to be information literate in the 21st century.

Before moving to the relationship of LIS to the digital humanities, it is important to highlight that there is a difference between the practice of LIS, (librarianship, archival work, records management, information management) and the academic discipline of LIS, which has not been explicitly referred to thus far. In as much as discipline and practice are most helpfully regarded as symbiotic, this should not matter, but when we examine the aims, theories and research methods in order to look at how LIS compares and contrasts with DH, it is prudent to acknowledge the distinction.

The difference between academic research and professional practice seems harder to draw out in DH than in LIS, but this may be a subjective view, writing from a personal background in LIS, rather than DH. It is perhaps in what is considered practice, that there is most similarity between the two disciplines. Aspects such as the creation, curation, indexing, dissemination, and preservation, of documents, understood in the broadest sense, are common to both LIS and DH practice. Likewise the involvement of practitioners from both disciplines in teaching and learning, promoting the skills and understanding of tools to find, augment, analyze and share resources.

The point of department comes when we consider each discipline as an academic subject. Whereas the focus of LIS research is to examine the processes comprising the information communication chain in its entirety, the focus of digital humanities research is to further research in the humanities.

What is Library and Information Science?

Library and information science (LIS) is a long-standing academic discipline, with its own set of theories and perspectives. It focuses on the study of the communication chain of recorded information, and supports the practice of librarianship, information management, archiving and records management and other collection professions.

Although it makes full use of technology, LIS is rooted in the humanities and social sciences. Its origins are in bibliography, the attempt over several centuries to make published information organized and accessible, and in the documentation and special libraries movements of the early twentieth century, which sought to make specialised knowledge retrievable at a detailed level. It is therefore centred around an understanding of documents and the ways in which they are managed; particularly the new forms of digital and immersive documents now becoming available. City University London, 2015

MA Digital Humanities

Digital Humanities is a discipline born from the intersection of humanities scholarship and computational technologies. Its key purpose is to investigate how digital methodologies can be used to enhance research in disciplines such as History, Literature, Languages, Art History, Music, Cultural Studies and many others. Digital Humanities has a very strong practical component as it includes the concrete creation of digital resources for the study of specific disciplines, while at the same time having a strongly theoretical basis. King’s College London, 2015

The theories, methods and tools of the digital humanities, which are still evolving, are doubtless readily claimed by members of both the DH and LIS communites to differ from those of LIS. However, whilst LIS is considered by many to be a social science, it has clear connections to the humanities, deriving from its origins in bibliography and documentation.

This is not the only connection; the topic of book history is often found within LIS departments and syllabi. Book history can be studied from two main angles, both based in the humanities; the UK/US approach rooted within English Literature, and the Franco/European approach, growing from historical analysis and socio-cultural theory. LIS is a broad discipline, and it can accommodate and enjoy an equally broad scope in methods and tools.

It may thus be appropriate to acknowledge that contemporary times bring complications to the easy conclusion that LIS and DH differ from the starting point of theories, methods and tools. It seems likely that both disciplines investigate, and can themselves be investigated, by methods from both sides. It may be valid, for example, to study aspects of LIS (e.g. texts, Twitter datasets), using the methods and tools more usually associated with DH.

A next stage endeavor to this conversation, would be to identify, compare and contrast the theories and methods of LIS with those of DH, and the humanities in general. With the convergence of GLAM (galleries, libraries, archives and museums) sector activities, driven by born digital documents and digitization programmes, the disciplinary and sector boundaries of just ten years ago are being erased, and although this is most evident in practice, it is clear that our academic silos are fading, and we find ourselves within an era of interdisciplinarity, where digital data doesn’t care whether the theories and tools belong to LIS, DH or any other discipline.

LIS and DH may not be identical twins, but they are often seen wearing the same outfits.

References

Bawden D and Robinson L (2012) Introduction to Information Science. Facet Publishing: London.

Hjorland B (2002). Domain Analysis in Information Science. Eleven approaches – traditional as well as innovative. Journal of Documentation, 58(4), pp 422-462.

Robinson L, Priego E and Bawden D (2015). Library and information science and digital humanities: two disciplines, joint future? In: Pehar F, Schlögl C and Wolff C (eds.) Re-inventing information science in the networked society. Glückstadt: Verlag Werner Hülsbusch, 2015, pp 44-54.

Robinson L (2009). Information Science: the information chain and domain analysis. Journal of Documentation,  65(4), pp 578-591.

Sula C A (2013). Digital Humanities and Libraries: A Conceptual Model. Journal of Library Administration, 53, pp 10-26.