User Centred Approach to Academic Information Management on the WWW

Hughes GV, Wills GB, Carr LA, Hall W, Hey AJG
Department of Electronics and Computer Science, Southampton University
Highfield, Southampton, United Kingdom. SO17 1BJ.
{gvh, gbw, lac, wh, ajgh, }@ecs.soton.ac.uk.

Technical Report No. M99-6
ISBN:- 085432703-7
Copyright © 1999, University of Southampton. All rights reserved.

Contents
Introduction
The AIMS Document Management System
Open Linking
Linking Items in Minutes
User Evaluation
Enhancing the Link Service
Beyond Generic Links
Related Work
Conclusion
References

Abstract: This paper presents a Web based document management system developed for use by administration personnel in an academic environment taking into account the variety of systems and processes already in use. Users do not need to know how to author Web pages as the source material for the system are files produced by common word processors. The system features a number of management tools to complement this concept written into a Lotus Domino application. The document management system is complemented by the use of an open linking service to dynamically cross-reference the documents by automatically generating links. An evaluation of the system was conducted to assess the impact of the introduction of the system and its ease of use by administrative personnel. The paper also presents the continuing research for providing users with a variety of link services and agents that enhance the basic content of the system.

Introduction

It is now common place for many large organisations to have a management information network, and the university environment is no exception. In addition, most academic institutions maintain a Web site of official information for staff and students. However, the management network is limit to specific university management functions, and the information on the official web site is often of a general nature. In addition, providing effective and controlled access to information for users from different levels within the academic environment, with different and varying computer skills, is also a major concern. Between these two systems lies the departmental information, that is often in a paper-based format, with all the associated difficulties in archiving, searching, auditing etc. What was required was a system whereby people could disseminate information effectively, that is easy to use and yet can be accessed by all that needed the information.

Hence, the AIMS (Academic Information Management System) project sets out to make it a very simple matter for the people who create paper documents within a department or organisation to contribute them directly to an automatically generated Web site. Advanced linking technology is used to enhance these online documents with as much relevant information as possible. The AIMS server becomes a fully featured Web-based repository and archive for the information. Users are expected to continue to work as normal on their own machines and submit finished versions of documents to AIMS.

Rather than just being a Web-based file store, AIMS has been designed to make use of the content of documents. An external link service enhances AIMS Web pages by embedding links on words or phrases found in AIMS pages. The links are not embedded in the documents on the server but are added to the Web pages on the fly by the link service. The link service and other agents attempt to reflect the structure of the department complementing the information contained in the documents. This paper describes the AIMS system, presents the results from the user evaluations and describes the continuing research.

Back to Contents List.


The AIMS Document Management System

This section contains a summary of the key technical features of the document management aspects of the system as implemented using a Lotus Domino server. Documents are submitted as word-processed files using a simple Web form. Every effort was made to ensure that the uploading of files was a quick and easy process, this achieved in part by collecting the minimum amount of metadata. A complex LotusScript program manages the creation of a document in the Notes database and conversion of file uploaded files of various formats (including Microsoft Word and WordPerfect) into the internal Notes format whilstalso storing the original file. The ability to work with the content of numerous formats was the main reason for choosing Lotus Domino as the application development environment. From the database of documents the Domino server can generate Web pages on the fly, as the server does not store static HTML. The overall design is targeted towards making a simple and fast process for putting normal word-processed documents online with the minimum of effort.

Features of the document management system include:

A fully operational AIMS site is in use within the author’s department. The AIMS approach is also being evaluated by a number of other departments at Southampton and other UK Universities. Currently there are three Domino servers running 5 separate AIMS databases.

Back to Contents List.


Open Linking

An initial requirement of the project was to provide useful links on this content of a document in a way that is easy to maintain, with effort going towards automatic link creation. This was made possible by making use of an open hypermedia link service developed within the department. Over time the service has been implemented in two flavours though both work along similar principles. The first is the Distributed Links Service (DLS) [Carr et al 1995] [Carr et al 1998] [Hill et al 1995] developed within the department mainly funded by the Open Journals Project [Hitchcock et al 1997] [Hitchcock et al 1998]. The commercial version, Webcosm, is developed by Multicosm Ltd, a company originally founded to exploit the Microcosm [Fountain et al 1990] system. Each brings differing advantages for the AIMS project, Webcosm has a larger feature count and will be used when the linking service is made available to users of AIMS whereas the DLS source is controlled by the department and the system can be altered at a low level for experimental purposes.

The link service is implemented as a Web proxy, becoming available to the user in the normal manner after they have configured their browser to use the link service proxy. When the browser requests a page, the link service will retrieve the page from the AIMS server and annotate the HTML with extra links before passing the enhanced Web page back to the user’s browser. Links are stored on the link server in flat file link databases or linkbases. An entry in a linkbase consists of the word or phrase to make an anchor, the destination URL and a description of the link. When the link service is in use, any occurrence of the word or phrase in pages the user views, will become a link to the destination document, even though this link is not in the original Web page. This is a generic or glossary link as first implemented in the Microcosm hypermedia system. The key advantage of using the system is ease of maintenance. For example, if the home page of our research group changes then the link can be updated in one simple action by altering the entry in the appropriate linkbase.

Linking Items in Minutes

Creating explicit links in lengthy academic minutes can, at the least, be time consuming, if not tedious. The minutes of the Department Board of Electronics and Computer Science contain item numbers of the form B.n where n is a number. Frequently items in minutes will be following on from Actions and Items in previous editions of the minutes. To reduce the burden on the document author from having to manually create explicit cross-referencing links, use was made of the structure of the minutes and the advanced features of the linking service. The Distributed Links Service has been used to link together these item numbers enabling readers to follow the trails of issues back through time, providing an effective audit trail. This is demonstrated in the diagram below [Fig. 1]. The Distributed Links Service has been designed to automatically link the items and actions in minutes together. This feature is being further developed to ensure that the techniques can be applied to more generic document types, using pattern matching. For pattern matching algorithms to be more successful users must be trained to be accurate in their use of the numbering schema. Users submitting documents could be constrained in what they can enter in the fields or more fields could be added to describe the document in more detail. However, this may quickly discourage users especially if they lack confidence and are concerned that their categorisation would be considered ‘incorrect’.

Figure 1: Using the Distributed Links Service to link items in minutes allowing the user to follow trails of issues back through time.

Back to Contents List.


Link Authoring From A Browser

Creating and editing of links, within Webcosm, is carried out using an off-line Java link editor. However, from a users perspective, they would prefer to create links whilst viewing a document. Modifying the context (or right click) menu on Microsoft Internet Explorer 4 allowed access to a simple authoring tool. The authoring tool consisted of a form, generated form JavaScript, whenever the user selected the additional entry on the context menu ‘Create Generic Link’. To create a link the user opens the destination document and selects the appropriate word in the text, to use as the anchor, and selects the ‘create link’ option. On completion, the link details are submitted directly to the Webcosm server, allowing users to directly author links between documents. The links will also have a high probability of still being relevant when new versions of documents are added, as the content rarely changes. That is, if a document containing an anchor is updated then the link will survive if the same word or word-pair still occurs in the content of the new version. In addition, if the destination document is updated then the link will point to the new version of the document. Whether these links are still relevant to the content is a more complicated issue, and is an area currently under investigation.

Back to Contents List.


User Evaluation

The evaluation was conducted in two stages [Wills 99], first a contextual review [Preece 94] of the working practices of the secretarial staff was undertaken. The second stage, was a usability study of the system, which was conducted in two parts. A structured expert review using discounted usability engineering [Nielsen 89, Nielsen 94] with an additional principle of 'Provide Navigational Aids' to ensure that the hypertext components of the system are reviewed [Wills 97]. The second half of the usability study was to conduct a time trial with users completing questionnaires. The questionnaires were used to measure the user subjective opinion of the system [Hirst 95] and their acceptance to use the system [Davis 93, Davis 96], an additional criteria of Navigation was added to measure the users opinion on how well they could move around and through the information space.

The contextual review showed that the main method of archiving was in the paper format, making it difficult for academic staff to obtain back copies of meetings to check on discussion or discussions made at previous meetings. The length of time required to photocopy, collate minutes, put them into envelopes, stick the address labels on (which have previously been printed) then walk them to each group, takes on average between six to eight hours.

Task Average time second (SD) Median
Task 1 Enter a new document

98.6

(10.0)

98.5

Task 2 Enter the next set of Minutes

60.8

(7.7)

63.5

Task 3 Edit document details

33.0

(6.3)

35.0

Task 4 Search for a document

40.9

(12.0)

43.5

Task 5 Using Webcosm to follow links.

31.5

(8.3)

32.0

Table 1 Average times to complete the tasks.

The average time to complete the tasks is shown in Table 1. The time taken to complete the tasks were consistent throughout the user group. The obvious saving of time comes from the ability to publish the set of minutes extremely quickly, when compared to the current method. The ability to disseminate information quickly would be a significant cost benefit to the department. However, this will only be an advantage if there is a paradigm shift. In that, academic member of staff will need to read the minutes on-line, or as last resorts print them off themselves to read them.

The general comments from the users showed that they were interested in the idea of a central repository for administrative information. This was reflected by the high score from the questionnaires given to the Impression Category in Table 2, and the ‘intention to use’ category from the Technology Acceptance Model criteria Table 3. All the users replied that they would recommend the system to their colleagues, and that the majority (87 %) could navigate the information space with ease.

The scores were normalised by dividing the score by the number of respondents and the number of questions they answered. The maximum normalised score is +1 indicating a very strongly agreed to –1 indicating a very strongly disagreed to all positively phrased questions (and visa-versa on all the negatively phrased questions). A score of 0.5 indicates that all the users agree with a positive statement, i.e. They agreed with ‘The AIMS system is one that I want to use on a regular basis.’ The questionnaire also showed that the scores given were not effected by the user access to the internet from home, or the users preference for using the internet.

Criteria

Normalised Score

Impression- user's feelings or emotions when using the software.

0.50

Command - the measure to which the user feels that they are in control.

0.46

Learnability - the degree to which the user feels that the application is easy to become familiar with.

0.44

Navigability - the degree to which the user can move around the application.

0.43

Helpfulness - the degree to which the application assists the user to resolve a situation.

0.43

Effectiveness - the degree to which the user feels that they can complete the task while using the system.

0.37

Over All

0.44

Table 2 Score from the questionnaire.

TAM Criteria Score Normalised Score
Intention to use

17

0.57

Perceived Usefulness

41

0.46

Perceived Ease of Use

80

0.41

Table 3 Technology Acceptance Model Scores.

Back to Contents List.


Enhancing the Link Service

Use of Webcosm and the DLS in a number of projects have shown that the implementation of the service as a Web proxy has a number of limitations. The primary limitation being that, users may not know how to set up a proxy or may not be allowed to make use of a proxy. In addition, the service will alter all documents irrespective of the context and the use of firewalls may also restrict its operation. This particular issue has been partially addressed by the DLS, which allows a linkbase to be used for a limited URL domain.

Alternative implementations do exist. Webcosm has also been developed as a plug-in library using the NSAPI interface for Apache Web servers. The link service is a library resident on the Web server and alters Web pages before being delivered to the browser. Consequently only documents on the server are enhanced, this can be seen as a definite advantage in AIMS. A similar low level interface to the Domino Web server does not currently exist, however, as Webcosm is implemented as a Web proxy component and a link server engine, it was possible to write programs that communicate directly with the link server, every time a document is requested. The link service returns only a list of links, which it would have placed into the document. The Domino server renders and delivers the document, with the links appearing in the margin, as a list of recommended sites for the user to view. While the links are not being embedded in the main flow of the document, the links are being generated on demand and in a context depending on the users chosen linkbases.

A further enhancement has been to try a process of pre-compilation of links. A system has been devised that can ‘pull’ the body of AIMS documents through the link service which adds the appropriate links. This HTML representation is stored in the database and served to users who are unaware that the main portion of the document is not actually being generated on demand. Though this has the obvious disadvantage that links are not being created on the fly the system now looks and feels like a normal Web site to users. This will make it easy to develop a system for creating point to point links between documents. The envisaged procedure would be that the document links are updated overnight as required. This is still in development and no conclusions have yet to be drawn from its use.

Back to Contents List.


Beyond Generic Links

The generation of too many generic links may at best become a distraction or worst overwhelm the user. Hence, to overcome the problem, the DLS can be modified to implement a concept of link priority. Different links are given a priority by the author and the system uses colours (or does not display) the links according to their priority setting.

Using pattern matching to create links in text has shown to be a promising way to generate link anchors without the overhead of explicit link authoring, as demonstrated by the example of linking the Board Minutes. Another application of such techniques would be to help with recognising names of people within documents, although such techniques are much harder to implement.

Pattern matching algorithms to solve this problem are generally complicated, and therefore compromise the performance of the service by slowing the access times of documents. Hence, an important implementation issue is how to create useful links for the documents without degrading the service. One method is to force the users to supply keywords and metadata for a document. However, this places unwanted demands on users and can be very difficult to implement. Alternatively, extracting data from other online systems, such as the staff database, allows linkbases to be automatically generated. While this is useful, it is limited to specific topics. A more interesting method to generate keywords automatically from the content of the documents, such a system has been developed by Multicosm, called Refindment, and is being tested with AIMS.

Back to Contents List.


Related Work

Other open hypermedia systems have evolved to encompass the Web in some respect and offer enhanced Web pages.

The Memoir [DeRoure 1998] project used an open architecture including proxies and link services to support researchers working with vast quantities of distributed information. The system not only used the DLS link services but also supported the trail as a first class object. Users could record their trails through documents whilst pursuing a particular task and the system could match trails thus connecting users with similar interests.

Hyperwave [Hyperwave 1999], now marketed as an advanced Web server, started life as an open hypermedia system Hyper-G [Andrews et al 1997], [Maurer 1996]. In it links are first class objects stored separately from documents and are also bi-directional. Links are merged into the documents when delivered to the browser.

Webvise [Grønbæck et al 1999] has evolved from the DHM/WWW integration project into an enhanced Web service offering contexts, links, annotations and guided tours stored in external databases. Users can collaboratively create links and other hypermedia content using the Microsoft Internet Explorer 4 Browser utilising COM technology. The same group is now drawing on the lessons of many of the systems listed here and working on the ARAKNE framework [Bouvin 1999], [Bouvin 1999a]. This is an object-oriented component-based, three layer model aimed at providing Web augmentation tools, a unified access to structure servers, proxies and Web browsers.

A number of systems have used Web server or proxy based methods to provide annotation services for applications such as discussion groups. Critsuite [Critsuite 1999] includes the CritLink link service component that will annotate existing documents with links. The author’s first mediator was Shodouka [Shodouka 1999], this proxy could render Japanese Web pages as images on Web browsers that did not support the Japanese language.

Back to Contents List.


Conclusion

We have developed a Web based document management system using Lotus Domino with facilities designed for academic users. The use of such a flexible development platform now allows us to bring these documents alive for users in ways that are not possible on paper or on a traditional Web site.

The focus of the research was in providing added value to the basic content of word processed documents submitted to AIMS. At its most basic AIMS simply stores files originally written on a word processor that were destined to be printed. These electronically frozen documents need to be kept alive by as many means possible to ensure their continuing relevance to the users and to make it easier for people to retrieve the information they need on a daily basis. By making use of the content of these documents a considerable advantage over the paper versions was achieved, as demonstrated by the results of the user evaluation. That is, the users were able to disseminate the information quickly and easily, giving a considerable time save (and hence cost). In addition the evaluation showed that the users found AIMS easy to use and would easily transfer over to using the technology.

Back to Contents List.


References

[Anderson 1997] Anderson, K. (1997). Integrating Open Hypermedia Systems with the World Wide Web. The Eighth ACM Conference on Hypertext and Hypermedia.

[Bouvin 1999] Bouvin, N. (1999). Unifying Strategies for Web Augmentation. The Tenth ACM Conference on Hypertext and Hypermedia .

[Bouvin 1999a] Bouvin, N., Schade, R. (1999). Integrating Temporal Media and Open Hypermedia on the World Wide Web. The Eighth International World Wide Web Conference, Toronto, Canada.

[Carr et al 1995] Carr, L., De Roure, D., Hall, W. and Hill, G. (1995). The Distributed Link Service: A Tool for Publishers, Authors and Readers. The Web Journal 1, 1, 647-656, OReilly and Associates.

[Carr et al 1998] Carr, L., De Roure, D., Hall, W. and Davis, H. (1998). Implementing an Open Link Service for the World Wide Web. The World Wide Web Journal 1, 2, 1998, OReilly and Associates.

[Critsuite 1999] Critsuite Critical Discussion Tools for the Web. The Foresight Institute. http://crit.org/http://crit.org/index.html.

[Davis 93] Davis FD. User acceptance of information technology: system characteristics, user perceptions and behavioural impacts. International Journal of Man Machine Studies (1993) Vol. 38 pp475-487.

[Davis 96] Davis FD. A critical assessment of potential measurement biases in the technology acceptance mode: three experiments. International Journal of Human Computer Studies (1996) Vol. 45, pp19-45.

[DeRoure 1998] DeRoure, D., Hall, W., Reich, S., Pirakis, A., Hill, G. and Stairmand, M. (1998). An Open Architecture for Supporting Collaboration on the Web. IEEE Seventh Internation Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises.

[Fountain et al 1990] Fountain A., Hall W., Heath I. and Davis H. (1990). Microcosm: an Open Model With Dynamic Linking. Hypertext: Concepts, Systems and Applications (Proceedings of the European Conference on Hypertext, INRIA, France, November 1990) 298 – 311.

[Grønbæck et al 1999] Grønbæck, K., Sloth, L., and Ørbæck, P. (1999). Webvise: Browser and proxy support for open hypermedia structuring mechanisms of the WWW. The Eighth International World Wide Web Conference, Toronto, Canada.

[Hill et al 1995] Hill, G., Hall, W., DeRoure, D. and Carr, L. (1995). Applying Open Hypertext Principles to the WWW. International Workshop on Hypermedia Design, Montpellier, France, 1-2 June 1995.

[Hitchcock et al 1997] Hitchcock, S., Carr, L., Harris, S., Hey, J. (1997). Citation linking: improving access to online journals. In Proceedings of Second ACM Conference on Digital Libraries, Philadelphia, pp.115-122.

[Hitchcock et al 1998] Hitchcock, S., Carr, L., Hall, W., Harris, S., Probets, S., Evans, D. and Brailsford, D. (1998). Linking electronic journals: Lessons from the Open Journal project. D-Lib Magazine, December 1998.

[Hirst 95] Hirst SJ, Corthouts J. Hyperlib Deliverable 5.1: Evaluation of the Hyperlib Interfaces. Hyperlib Electronic Document Store University of Antwerp 1995 available at http://143.169.20.1/MAN/WP51/root.html

[Hyperwave 1999] Hyperwave Information Management server. Hyperwave GmbH. http://www.hyperwave.com/

[Lotus Notes 1999] Lotus Notes workflow and information management system. Lotus Development Corporation. http://www.notes.net/

[Maurer 1996] Maurer, H., Ed. (1996). Hyper-G now Hyperwave: the next generation Web solution. Addison-Wesley.

[Nielsen 89] Nielsen J. Usability Engineering at a Discount. In: Designing and using Human-Computer Interfaces and Knowledge Based Systems. Editors, Salvengy G & Smith MJ. Elsvier1989, pp394-401.

[Nielsen 94] Nielsen J. Enhancing the Explanatory Power of Usability Heuristics. Factors in Computing Systems CHI ’94. Boston, Massachusetts, USA, April 24-28, 1994, p152-158.

[Preece 93] Preece J et al. A Guide To Usability, Human Factors in Computing. Addison Wesley, The Open University 1993.

[Preece 94] Preece J, Rogers Y, Sharp H, Benyon D, Holland S, Carey T. Human-Computer Interaction. Addison-Wesley 1994.

[Shodouka 1999] Shodouka enables Japanese text to be read on any Web browser. The Shodouka Corporation. http://www.shodouka.com/

[Webcosm 1999] Webcosm is a component used in the knowledge management products produced by Multicosm Ltd. http://www.multicosm.com/

[Wills 97] Wills G.B, Heath I, Crowder R.M, Hall W. Evaluation of a User Interface Developed for Industrial Applications. University of Southampton Technical report No M97-4 ISBN-0854326499.

[Wills 99] Wills GB, Hughes GV, Hall W. Evaluation of AIMS-Academic Information Management System. Multimedia Research Group Technical report No M99-X ISBN-085432700-2.

Back to Contents List.