The Dynamic Review Journal: Support for Creating and Using the Literature

Leslie Carr, Gary Wills, Timothy Miles-Board, Guillermo Power, Christopher Bailey, Wendy Hall, and Simon Grange*

{lac,gbw,tmb,gp2,cpb,wh}@ecs.soton.ac.uk, simon.grange@btopenworld.com

Intelligence Agents Multimedia Group, University of Southampton, Southampton, UK

*Royal College of Surgeons of England, London, UK

Abstract

An archive of digital literature, together with its users and its contents, does not exist in isolated splendour. There is a cycle of activities which provides the context for the archive’s existence, and which the archive supports through its various roles of information access, discovery, storage, dissemination and preservation. This paper focuses on a specific framework, the Dynamic Review Journal, which supports the development and dissemination of literature by assisting authors in collating and analyzing experimental results, organizing internal project discussions, and producing papers. By bridging the gap between the undertaking of experimental work and the dissemination of its results through electronic publication, this work addresses the cycle of activity in which a digital archive rests.

1. Introduction

The accepted role of scientific and scholarly publication is to record research activity in a timely fashion, keeping others in the research community up-to-date with current developments. Until very recently, it has been the case that printed journals were the most efficient method for the dissemination and archival of research results. Technical advances in the past decade have allowed the process of scholarly communication to take other forms, particularly in the dissemination and storage of articles via the World Wide Web. Giving publishers a new medium for making their journal archives available [4], it also gave authors the means to break the so-called “Faustian bargain” and directly distribute their articles in pre- or post-publication form from their own Web pages [3] or in organised “e-print archives” [2]. The old medium allowed a paper to be published as the summarisation of a scientific activity - the discarded raw observations that led to the  article's conclusions are replaced by a description of the method for recreating the experiment. However, researchers are becoming more interested in the potential of the new medium for preserving experimental data as well as experimental conclusion: the ability to provide hypertext links between the article and the data to create an ‘audit trail’ for reviewers and thus facilitate further analyses and meta-analyses.

It may not be simply the technical support for reproducing and distributing articles electronically (electronic publishing, e-print archiving and digital archives/libraries), but also the emergence of technical support for improving human communication in the form of highly collaborative, large-scale activities and analyses (the Grid, Virtual Universities) that is likely to precipitate significant change in the field of scientific communication and significant changes in the way its communications are produced, curated and disseminated [6]. In the UK, the accepted term for scientific activities augmented by the Grid is ‘e-science’. This paper adopts that term when we wish to discuss the scientific activity augmented by computational support without unduly focusing on the enabling technology.  In this paper we present a system called the Dynamic Review Journal, a tool for assisting scientists to share data, write papers and discuss their findings.

2. Related Work

Dalgaard shifts the attention of scholarly hypertext away from hypertext being merely intertextual relationships between articles to the relationship between text and archive [1]. He points out that from its very inception, hypertext was thought of both at the level of the text and at the level of the network, arguing that in the context of the Web, hypertext has become the paradigmatic rhetorical structure of a global and distributed archive. Accordingly a scholarly archive is the collection of scholarly texts, and the catalogues and reference works giving access to them. Dalgaard observes that most navigational options are presented as texts (lists of works, authors names, references, etc) - this is the hypertextualization of the scholarly archive.

The historical image many people have of the authoring process is that of a lone scientist or small team working in a basement laboratory. A similar picture appears for the use of libraries, where researchers ferret away for vital missing bits of information. Levy and Marshall have examined the early underlying assumptions which have affected digital library development [5]. They challenge these images, especially that digital archives/libraries are used by individuals working alone. They point out that the work carried out by staff (in doing research) and library staff (in providing the service), is one of collaboration, and that digital systems should support formal and informal collaboration and communications. Similarly, Marchionini and Maurer point out that “digital libraries will allow learners of all types to share resources, time and energy, and experience to their mutual benefit” [8]. In their proposed future of digital libraries, sharing resources becomes an important factor supporting teaching, including the ability to share scientific data and other datasets. Many e-science projects have collected a vast amount of data: if the next generation e-scientists are to go beyond the present position they will have to have access to the raw data in their research and training. These early visions are slowly being realised, for example McGrath et al. have developed a system  to locate, navigate and retrieve astronomy data databases [10]. However, there is still a need for librarians/archivists and users to work together with those that have the technology skills to provide appropriate tools for handling, manipulating and analysing these large datasets [12].

Marchionini and Maurer also suggest that digital libraries should offer greater opportunities for users to deposit information. There are projects beginning to do this, for example the Digital Library for Earth Science Education (DLESE) project allows students to explore geospatial materials and Earth data sets, provides services to create and use materials, and facilitates peer reviewed teaching and learning resources [9]. The peer review of collections and peer comment is a significant part of the disseminating process, which adds value to any collection - Weatherley et al. have proposed a model that will aid reviewers in reviewing complex material or a digital collection [13]. Lyon sees the digital library in the context of an information grid as consisting of a collection of resources for learning and teaching, data repositories for research purposes, or as archives of diverse cultural heritage materials [7]. In her scenario researchers would undertake experiments, deposit raw data, and produce pre-prints using Web services.

3. The Dynamic Review Journal.

Recent advances in Web technologies allow these concepts to be realised. The Dynamic Review Journal (DRJ) has been implemented as a Web-based environment for supporting a critical subset of the e-science cycle (Figure 1): the collation and analysis of experimental results, the organisation of internal project discussions, and the production of appropriate outline papers depending on the requirements of conferences and journals selected for dissemination. The DRJ-Framework is the software which has been implemented to embody these processes. Figure 2 illustrates the DRJ-Framework concept; in the context of an e-science community Web site and existing (integrated) Web-based services:  eprint, discussion, and analysis services provide integrated support for e-print/reprint management, communication, and e-experimentation respectively. Figure 3 elaborates this picture in order to illustrate the major activity spaces in the DRJ-Framework and the workflow supported within these spaces.

Figure 1: Complete cycle of E-Science

The DRJ has been integrated as part of the Virtual Orthopaedic European University (VOEU), a community Web site dedicated to the ongoing education of orthopaedic surgeons in the EEC. Developed by a consortium of surgeons drawn from six European countries, VOEU aims to provide services to help ‘e-surgeons’ learn about and disseminate material on surgical techniques in orthopaedics, especially Image Guided Orthopaedics. The VOEU-managed learning environment for training surgeons consists of multimedia educational material (including problem cases and assessment), interactive simulators, and communication tools (moderated and asynchronous message boards) together with the Dynamic Review Journal.

Within VOEU, the Dynamic Review Journal has two main functions; to aid surgeons in preparing findings for publication, and to support the educational process. Orthopaedic surgical trials typically run for extended periods of up to 2 years, with postoperative assessment results being collected regularly. The collated results are then analyzed and discussed by a team of e-surgeons before being disseminated to the wider orthopaedic community. We have extended the UK arm[1] of the VOEU Web site to incorporate the DRJ-Framework components and support this process.

Figure 2: Conceptual overview of DRJ-Framework.

3.1. DRJ-Framework Schema Space.

The schema space is the mechanism by which the generic DRJ-Framework is specialised to a particular e-science community, through the formal specification of e-experimentation procedures relevant to that community. This specification is currently achieved using three different types of schema:

• Data schemas describe the exact nature of the experimental data (for example, specification of variable names, types, and possible values). In the case of  the VOEU, a number of data schemas have been defined for collecting orthopaedic clinical trial data.
• Experiment schemas describe experimental procedures or protocols. For example, a protocol could specify that any e-scientist conducting an experiment of type X needs to record an experiment description, statement of purpose and an outcome hypothesis. Human-readable guidelines are also included, to help scientists meet the requirements of the protocol and to help reviewers to ensure that the requirements have been met. As a simple example, the e-scientist guidelines for the experiment description may state “summarise the content of the experiment”, whereas the reviewer guidelines ask “is the content of the experiment adequately summarised?”.
• Publication schemas describe the required format for submitting experimental results to relevant journals/conferences (for example, abstract, introduction, background, experimental methods, results, conclusions). As with experiment schemas, human-readable guidelines are also included in publication schemas. In the case of VOEU, two publication schemas presenting the submission formats for the Journal of Bone & Joint Surgery (JBJS)[2] and the British Medical Journal (BMJ)[3] are currently defined.

Where possible, the publication schema also describes any mappings between experiment protocols and publication schemas (for example, specifying that the hypothesis of experiment type X should appear  in the experimental methods section of a JBJS preprint) - this allows outline preprint ‘previews’ to be generated automatically without requiring the e-scientist to copy and paste information between protocol and preprint.  Schemas are created using the Schema Builder interface of DRJ Framework (Figure 3).  In our experience of applying DRJ-Framework to the VOEU project, an elected subset of the community “bootstrap” the schema space with a representative set of data, experiment, and publication schemas, which are then augmented by individual e-scientists as needs arise.

Figure 3: Workflow in the DRJ Framework.

 

3.2. User Space

The user space is where e-scientists use the schema space to orchestrate practical data entry and collation, e-experimentation, and dissemination. The space is further subdivided into three personalised areas My Logbook, My Experiments, and My Papers.

My Logbook is an digital experiment logbook, in which surgical results can be entered (in accordance with a selected data schema). Logbook entries are subsequently added to the DRJ-Framework community database, making data available (anonymously) to other community members.

My Experiments is a workspace for e-experiments which the e-scientist works on. An e-scientist may be involved in an experiment in the capacity of lead investigator (initiates experiment and acts as co-ordinator and first point of contact for duration of experiment), associate (assistant), or reviewer (monitors the progress of the experiment and reviews its outcomes according to specified guidelines). Reviewers have read-only access to the experiment protocol and set-up. When a new experiment is initiated, a discussion facility is automatically set up to facilitate communication between the e-scientists involved (this is also the means by which reviewers can give feedback to the practitioners). Due to the clinical nature of the work in VOEU, the My Experiments work area has become My Trials. Figure 3 outlines the work process facilitated by My Experiments.

Finally, My Papers provides a simple shortcut allowing an e-scientist to quickly access all the papers produced by the various experiments worked on.

3.3 E-surgeon Scenario

To illustrate the DRJ-Framework in the VOEU context, this section outlines the process of managing e-experiments from the perspective of a fictional e-surgeon, Eddie[4]. Figure 4 shows Eddie’s view of the DRJ user space (note that this navigation menu is always available, but excluded from some figures for clarity—Figure 5). Eddie is currently working on three trials, undertaking a different role in each (note that roles are depicted using icons next to each trial). Eddie is the lead investigator in the “charcot joints” trial, an associate investigator in the “rotator cuff” trial, and reviews the “tear size” trial. Eddie has also entered several experimental records in the personal logbook (patient details, operative procedures, and assessment results), and so far has produced one pre-print.

To initiate a new trial, Eddie first selects the orthopaedic clinical trial protocol from the available experiment schemas. DRJ-Framework then uses this schema to generate a number of data entry forms in which Eddie enters specifics of the trial (Figure 5).

Guidelines for completing these forms are presented as “stretchtext links” [11] which can be viewed/hidden as required (Figure 5A). Eddie also specifies that the shoulder data schema will be used in the trial (Figure 5B) and the associative investigators and reviewers who will assist on the trial (Figure 5C).  When created, the new trial will appear in Eddie’s DRJ user space, and also in the user spaces of the associate investigators and reviewers. The experiment protocol may subsequently be updated by any of the investigators, perhaps as a result of critical comment from the reviewer.

Figure 4: DRJ user space

Figure 5: Entering trial protocol information

 

To create a dataset for the new trial, Eddie searches the database of cases. Since Eddie has already specified the shoulder data schema, only those cases matching this schema will be searched. DRJ-Framework uses the shoulder schema to generate a search form, so that Eddie can search for specific shoulder cases (Figure 6). Eddie and associates subsequently add 42 different shoulder cases to trial (see trial navigation menu in (Figure 7), which can be tabulated for visual comparison. This also provides the ‘audit trail’ back to the selected cases. To perform analyses on the dataset, Eddie and associates choose from statistical methods offered by a distributed Analysis Engine. Using the shoulder data schema and metadata from the Analysis Engine, the DRJ-Framework is able to generate an entry form for each statistical method, which Eddie can use to fine tune the analysis (specify test variables, groupings, etc.). The Analysis Engine queues the requested analysis and notifies the DRJ-Framework when results are available. These results then appear in Eddie’s DRJ user space, and can be viewed (Figure 8).

 

Figure 6: Searching the Virtual Observatory

Figure 7: Trial navigation.

 

Having obtained some significant results from the statistical analyses, Eddie then decides to create a pre-print in order to disseminate the results to the wider orthopaedic community. When Eddie selects the JBJS publication schema, the DRJ-Framework generates a pre-print template using the information Eddie entered in the trial protocol. Eddie fleshes out this template, following the JBJS guidelines provided (Figure 9), and specifies which analysis results should be included in the preprint. After previewing the pre-print, Eddie submits it: behind the scenes the DRJ-Framework submits the pre-print and its associated metadata to the community Eprints server (where it subsequently becomes available to the VOEU community), and makes the paper available in the user space of Eddie and associates.

Subsequent versions of the pre-print leading to submission, peer review, and reprint are managed by the Eprints server. Eddie and associates continue to discuss the development of the paper in the discussion forum. It should be noted that this workflow is not enforced as a linear progression from experiment protocol to pre-print; Eddie can make changes to the experiment protocol as the experiment progresses (for example, bringing a new associate investigator on board), return to the dataset at any point to add/remove experimental results or perform more analyses, and produce many different pre-prints describing different aspects of the experiment.

 

Figure 8: Viewing analysis results

Figure 9: Preparing pre-print.

4. Concluding Comments.

In this paper we have taken the position proposed by Dalgaard that scholarly hypertext is not merely the intertextual relationships between papers but the relationships between text and the wider archive [1]. In our view a digital archive is more than just a collection of digital copies of documents.  We argue for a broadening of this view, out from the document collections themselves, to the process of creating, collecting and deploying them. Parts of the scientific community (and the computing community) are currently obsessed by the idea of the Grid - the use of computers to improve scientific experimentation, analysis, and collaboration. However, we argue that publishing, dissemination, research, and learning are equally important (perhaps more important) parts of the scientific cycle and should not be left to unaided ‘mandraulic’ effort. This amounts to the hypertextualization of the scientific process.

We have presented our contribution to this ongoing effort, the Dynamic Review Journal, and described its integration and use within a Virtual University environment as an example of a system which deliberately crosses the barriers between these areas (experimentation, analysis, publishing, dissemination, discussion, and education). In providing support for the broader range of scientific activities, it is our hope that scientists can be made more effective in their work. Within VOEU the DRJ has 111 registered users, of which approximately 20 are currently active in working on clinical trials.  There are 6 data schemes for clinical trials, 3 of which are active. The shoulder schema has 42 patient cases; a hip schema 40 cases, and a dippy hip schema 2837 patient cases.

References

[1]	Dalgaard R. (2001) Hypertext and the Scholarly Archive: Intertext, Paratexts and Metatext at Work. In proceedings of the Twelfth ACM conference on Hypertext and Hypermedia. Aarhuus Denmark August 14-18, pp175-184.
[2]	P. Ginsparg. (1996) Winners and Losers in the Global Research Village. In Proceedings of the ’96 UNESCO Conference, Paris, Feb.
[3]	S. Harnad. (1995) Electronic Scholarly Publication: Quo Vadis? Serials Review, 21(1):70–72.
[4]	S. Hitchcock, L. Carr, and W. Hall. (1996) A Survey of STM Online Journals 1990-95: the Calm Before the Storm. In D. Mogge, editor, Directory of Electronic Journals, Newsletters and Academic Discussion Lists, pages 7–32. Association of Research Libraries, Washington, D.C., 6th  edition..
[5]	D. M. Levy and C. C. Marshall. (1995) Going digital: a look at assumptions underlying digital libraries. Communications of the ACM, 38(4):77–84, Apr.
[6]	R. Lucier. (1990)  Knowledge management: Refining roles in scientific communication. EDUCOM Review. Fall 1990: pp21-27
[7]	L. Lyon. (2002) Emerging information architectures for distributed digital libraries. In International Conference of Digital Library–IT Opportunities and Challenges in the New Millennium., 8 - 12 July 2002, Beijing, China.
[8]	G. Marchionini and H. Maurer. (1995) The roles of digital libraries in teaching and learning. Communications of the ACM, 38(4)..
[9]	M. Marlino, T. Sumner, D. Fulker, C. Manduca, and D. Mogk. (2001 The digital library for earth system education: building community, building the library. Communications of the ACM, 44(5):80–81..
[10]	R. E. McGrath, J. Futrelle, R. Plante, and D. Guillaume. (1999.) Digital library technology for locating and accessing scientific data. In Proceedings of the fourth ACM conference on Digital libraries, pages 188–194,.
[11]	T. H. Nelson.(1987) Computer Lib/Dream Machines. Tempus Books.
[12]	C. Nevill-Manning. (2001) The biological digital library. Communications of the ACM, 44(5):41–42.
[13]	J. Weatherley, T. Sumner, M. Khoo, M. Wright, and M. Hoffmann. (2002) Partnership reviewing: a cooperative approach for peer review of complex educational resources. In Proceeding of the second ACM/IEEECS joint conference on Digital libraries, pages 106–114, 2002.

 

---