Towards A Colorimetric Digital Image Archive for the Visual Arts

 

          Kirk Martinez and Anthony Hamber

 

Department of History of Art, Birkbeck College,

University of London, 43 Gordon Square, London WC1H 0PD

 

                            ABSTRACT

 

   The aim of this project is to produce a high-resolution, colorimetric and permanent digital archive of images taken directly from works of art. The proposed system is designed for use in education, research, galleries and museums. Tentative user requirements are examined with particular reference to resolution, image access and colorimetry. Existing technology and projects are considered. Some 3000x3000 pel images of paintings are used to illustrate the interrelationship between dimensions of the original, its inherent detail, scan resolution and display.

 

                        INTRODUCTION

 

   The project arose out of the increase in the scope of the requirements of those involved in the teaching, research and scientific examination of paintings. There is also a growing awareness of the limitations of conventional colour photography. 35mm colour transparencies have become the standard format for teaching and most research purposes, larger formats being impractical as well as far more costly. As the slide library in the Department of History of Art grew to its present levels of c.150,000 35mm slides so its management became increasingly problematic. Combined with these problems is the image degradation and variation in quality between film types. Image decay, through vastly increased use by both staff and students coupled with storage in far from archival conditions, was becoming increasingly apparent. Furthermore, access in order to photograph objects in either public or private collections has, in recent years, become more and more restrictive.

 

   Standard documentation techniques used in galleries and museums involve both images and complex textual data. Certain of these archives have become almost unmanageable through sheer volume and a digital alternative is awaited to replace photographic techniques.

 

        LIMITATIONS OF PHOTOGRAPHY

 

   Conventional colour emulsions are able to create a gamut of reproducible colour dependant on the spectral characteristics of the image dyes used.[OHTA]  Additive systems not only have linear characteristics but also a theoretically greater gamut of colour than the subtractive system, which is also non-linear. Colour film characteristics vary between film types, batches and manufacturers. The gamut of colour film neither equates with the gamut of four colour printing nor that of the phosphors of a CRT.

                       

   Temperature and humidity can be highly influential factors governing overall image quality and ultimately, however carefully film is stored and however exhaustive batching testing and calibration is, the processing of colour film can be the most influential factor on the overall accuracy of the reproduction of colour.

 

   Conventional colour photographic emulsions, depending on the test-object contrast, have an ability to resolve detail (on the emulsion) to between 40 and 120 lines/mm. In the case of black and white photographic materials, however, this may increase up to 400 lines/mm; some photographic plates produced for holography can theoretically resolve as many as 1000 lines/mm. The resolution captured by the film from the original object is dependent on the dimensions of this object with respect to the film size. Even large format transparencies cannot capture adequate detail from a large painting, as is illustrated below.

 

   The photographic negative enables multiple copies of a single original to be reproduced. It is not, however, possible to reproduce fully all the inherent information in a negative; especially the contrast. Moreover, the print from a negative is a second generation reproduction and often extremely difficult to produce and accurately assess for quality. For this reason colour transparencies are more commonly used in colour reproduction.

 

   Conventional colour photography is severely limited in terms of accessibility and retrieval systems. Photographic prints require large amounts of storage space as do 35mm slides. Access may be slow, even if there are electronic cataloguing systems; some of which may include a reference image. Unless there are multiple copies of each image, access is only possible by one individual at a time.

 

   The emulsions used in photographic colour materials use chemical dyes which are neither stable nor permanent. Light, temperature, humidity and other environmental factors affect stability and permanence. Teaching and research requirements often involve exposure to levels of light and temperature far in excess of the "average" use calculated by the manufacturers. The degredation of the image in a photographic transparency cannot be quantitatively determined because of the non-linear changes in the characteristics of the individual dye layers and the gelatin base.

 

 

                                     

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

                Table 1. User Requirements

 

 

                  USER REQUIREMENTS

 

   The major users of the archive are envisaged to be Art institutions and educational establishments. Art galleries and museums would use it for: administration, documentation, conservation, restoration, catalogues, reproductions and public access. Surface texture change and colour difference measurement have been identified in gallery scientific departments. Educational uses, at all levels, include: distance learning, computer-aided learning and instruction. Such an archive would be of particular use for specialised academic research. The impact of commercial use in publishing and broadcasting will undoubtedly be highly significant in the 1990s. There are clearly applications in completely different areas such as medical or satellite imaging.

 

   The textual information required for each image is varied but could be handled by conventional relational databases. It is the image requirements which cause major problems, in terms of acquisition, storage and retrieval. For recognition purposes only, a TV resolution image could be used (768x575), and image coding used to best effect. For detailed examination, small objects in the painting must be visible, such as flowers, birds or even the brushstrokes and craquelure of the paint itself. Consider a painting measuring 2x2m, a feature such as a hand could be 2cm long, so (assuming 10 pels minimum to resolve five fingers) at least 1000x1000 pels are needed purely to resolve the fingers. In reality a much higher resolution is needed to provide the level of detail seen when standing close to a painting and simple sampling theory is insufficient to provide good rendition in images.

 

   In certain cases photographs or transparencies are documents of the highest importance since they may be the only record of works of art which have been lost, destroyed, damaged, cleaned or restored. They therefore assume the position of an original rather than a reproduction. Thus if all the information recorded by a transparency is required for imaging and archiving, then for a 35mm slide (taking 60 lp/mm resolution on the emulsion, two pels per line-pair and 3bytes per pel) 35Mb is necessary but on 10x8" (a commonly used format in commercial reproduction), 2Gb would be necessary. Clearly the ability of large format transparencies to resolve such levels of detail is highly problematic. The acquisition storage and processing of the digital information scanned from a large format transparency is considerably more difficult than might be imagined.

 

   Scientific departments of museums and galleries require spectral and non-visible data, including infra red, ultra-violet and x-ray images. Many gallery archives include photographs of paintings taken in the IR and X-ray region, as well as raking light images, so there is a requirement for non-visible data. This brings similar problems to those found in remote sensing which requires the storage of a wide variety of spectral bands.

 

   Tristimulus scanning can not cover all of a colour space, while broadband spectral scanning covers a larger area. Both the colour space and the number of spectral bands recorded require carefully selection. Initial tests have shown that the transmittance of narrow band filters is so low that a good signal can not easily be recovered by a CCD without unacceptable levels of illumination. The alternative is to use a spectrometer technique, but the single scanning spot would cause speed problems. At some point this data must be converted into RGB for viewing, so if a sampled spectrum is stored it must be accurate enough for the RGB conversion to give precise colour rendition.

 

   If the archived images are for viewing only then tristimulus values, such as RGB, would be stored. The reason for storing in a standard colour space is that the transform from these systems to the RGB co-ordinates corresponding to commonly used phosphors is well known. The retreival and conversion of spectral data would greatly increase image access times. Unfortunately the transformation into XYZ (for example) discards spectral information that ought to be retained for scientific analysis of images. The transform will need to be confirmed using samples of well defined colour value/co-ordinates i.e. NPL tiles/Munsell chips etc. However some users require more information than the sensitivity of the eye, for example to measure colour or surface texture change, when the data should suit computer vision more than human vision.

 

   The major conflicts in the user requirements are those of tristimulus quantities for viewing and multi-spectral for scientific purposes. To satisfy both implies an ability to scan data in both formats, either directly, or from multi-spectral scanning with later conversion. The nature of the multi-spactral technique has yet to be investigated but clearly has major effects on hardware and software.

 

              AVAILABLE TECHNOLOGY

 

Imaging devices

 

   In terms of electronic imaging devices, CCDs are currently becoming the standard. They have good geometric stability and a better dynamic range than conventional video camera tubes. CCD devices with 4096 sensor linear arrays and 1035 x 1320 area arrays are commonly available.

 

   Many metal oxide semiconductor (MOS) CCDs are, however, less sensitive then a photodiode, particularly in the blue region of the visible spectrum. Neither a photodiode nor an MOS device have a response which equates to human perception of lightness or colour. Futhermore most CCDs are sensitive in the infra-red region which, for visible imaging, necessitates the use of a blocking filter. Other imaging techniques, such as micro-densitometers are capable of producing high-resolution output but are extremely slow.

 

Storage

 

   Although image archives are possible using magnetic tape systems, such processes have always had lengthy, sometimes unacceptable, access times and invariably occupy large amounts of space. Computers are now also much cheaper, smaller and more reliable. These factors bring the possibility of cheaper digital archives for use by institutions with restricted budgets.

 

   The storage required for high resolution colour images is most conveniently measured in giga-bytes (Gb). The data must be stored accurately, permanently and with a fast access time. The advent of WORM optical disks may provide a solution. Each disk, usually 5-12" in diameter, stores 2-8 Gb. Such disks have been incorporated into 'juke boxes' which can access information from a number of disks; for instance the Kodak system 6800 model D contains up to 150 disks with a total storage capacity of 1020 Gb. The drawback with this particular device is that there is but one drive for reading the disks. Ideally, the number of drives should be sufficient to cope with multiple requests to read data from the archive. Individual disks are relatively inexpensive compared to magnetic media (other than tapes) at around $600 each for an Optimem 2Gb disk. Unfortunately juke-boxes may have disk change times of tens of seconds. Winchester disks storing around 1Gb are now common and have data transfer rates of typically 3Mb/s. The disadvantage is the expense for large storage capacity, for example a CDC Sabre 1230 costs about £6,000 for 1.2Gb, and have poor long-term stability. Parallel-transfer disks can achieve extremely rapid rates (24Mb/s for multiple Sabre 2HPs) but are even more expensive, at around £10,000.

 

   To handle such large quantities of data, and more especially to carry out image processing operations, requires dedicated image processing hardware if response times are to be within the specifications of diverse user requirements. There are now many super-micros in the above 10 MIPS range, which can link to special hardware for I/O and processing. High speed networks now allow the distribution of different processors and storage systems for greater efficiency.

 

Displays

 

   All present display technologies have their limitations primarily in resolution and colour rendition. CRT technology is still the best available although the screen is not flat and the colour gamut is different from that of colour film, four colour printing or the information acquired by scanning. 4000 line colour monitors have recently become available (for example from Sony ) but are extremely expensive and require the use of very fast framestores. Large screen monitors which can show the scale of the original are not yet available and are unlikely to be of sufficient quality.

 

   In respect of display ergonomics, consideration must be taken of the viewing distance. Taking 50cm as the average viewing distance in a workstation environment indicates a theoretical resolution on the display of 8 pixels/mm. Unfortunately viewers tend to move as near as 25cm when looking closely at an object such as a painting. If this were to be the case when viewing a display then a display resolution of 16 pixels/mm would be necessary for the line structure to be invisible. This would be the equivalent of 4500 lines on a standard 18 inch monitor which is currently unrealistic.

 

   It is assumed here that the resolution on the scanned data will be higher than that of the display, thus a suitable viewing system is required. If a standard display resolution can be agreed (eg around 1kx1k) then zooming and panning of the viewport into the data would provide detailed examination of the original, with a subsampled version viewable full-screen.

 

   Hard copy output in the form of transparencies is also required, as these are transportable and more useful in many circumstances. 4000 line 35mm slide writers already exist to satisfy this requirement. For larger formats, 10x8" colour transparency writers are also available (e.g. Crosfield MAGNASCAN). Similarly instant print makers could provide immediate hardcopy. There are several new non-impact printing technologies being developed at present. For example the JetProof colour ink-jet printer has a resolution of 300 dpi.

 

Existing projects

 

   There are several numerous digital and analogue databases of images of works of art. The digital public access systems at the Musee d'Orsay, Paris is one of the most technologically advanced but the images are of fixed resolution and are taken from reproductions and not the original. The "Charters Monitoring System" at The National Archives and Records Administration, Washington, DC. uses a flat-bed scanner with a CCD camera in a climate chamber. The system examines small patches of the Declaration of Independence in monochrome and is probably the most adavnced scanner used to date.

 

   The Open University of The Netherlands makes extensive use of interactive electronic media including videodiscs and is about to complete a videodisc for an art history course (in collaboration with Birkbeck College, London). The J. Paul Getty Museum in Malibu has produced a touch-screen driven videodisc on its ancient Greek vase collection and there are several videodisc projects underway in museums and galleries in France and The Netherlands. The National Gallery, London is designing an information room which will use a digital image system and the Musee du Louvre is planning a public image access system for the new "auditorium" under the controversial glass pyramid.

 

   None of these projects combine high resolution and colorimetry to create a multi-purpose image database.

 

           POSSIBLE SCANNER DESIGNS

 

   There are fundamental problems to be solved in the scanning of paintings.  Some of these are purely technical while others are simply practical. For example many galleries have security systems which makes it expensive to remove paintings from the wall, even after permission has been obtained. Some works are impossible to move (such as wall paintings) and thus a portable scanner would be the only answer. Photographic recording as a first stage can be discounted on the grounds of inaccuracies, particularly in terms of colour and contrast. However, photographic transparencies can be digitised realitively cheaply and quickly on existing drum scanners, which convenient for transparencies of lost, destroyed, cleaned or restored paintings. Direct scanning of the painting should be the major goal if possible, as the accuracy of the data would be greatly enhanced.

 

   A scanner can be produced where the object is positioned horizontally or vertically (usually on a stable optical bench) and an X-Y bridge scans an imaging device (eg a CCD camera) over the surface. X-Y bridges used in manufacturing industries are capable of very fine positioning (0.25 micron for Leitz microscope small area after Berger Lahr crude move - REF) over substantial areas (say 3mx3m). Geometric correction would then mosaic the stripes or blocks in a similar way to remote sensing. In this way extremely fine details of small areas can be imaged and it is even possible to arrange for subsequent aligned scans (REF Washington) which is useful in detecting changes. The environment and lighting can be controlled to provide a standard scanning environment. Trade-offs in terms of bridge stabilisation time, number of pels scanned each move and correction workload have to be solved. For example using a 4k linear array CCD requires time consuming movements for each line but coarse movements of a 1kx1k area array might require more geometric correction work. The read-out rate from the CCD would also have to be considered.

 

   A scanning system could be fitted with different standard illumination and imaging devices to scan in the infra red for example, or even multiple channels simultaneously. Problems arise with specular reflection due to surface texture. Anomalous reflectance and fluorescence should be detectable and correctable where necessary.

 

                     SCAN RESOLUTION

 

   Assuming the eye can resolve up to 35 cycles per degree, at a 1m viewing distance, 0.5 mm cycles are seen, so 0.25mm pel coverage is needed. There is an inverse relationship between viewing distance and resolvable detail measured on the object. This suggests that at 1m viewing distance, 4kx4k pels resolution scanned for each square meter of painting would provide as much detail as could be seen 1m away from the object. This viewing distance/resolution relationship is crucial both for image acquisition and subsequent image display. If the scanning resolution is based on human resolving power then a decision has to be made on the closest viewing distance from the painting. However, those working in the restoration and conservation departments of galleries and museums often require higher levels of resolution than perceivable by the naked human eye.

 

   The variation in dimensions of three paintings in the National Gallery London indicates some of the problems to be encountered. "The Adoration of the Shepherds" by the 17th-Century italian painter Guido Reni has dimensions of 4.8 x 3.21 m; "The Adoration of the Magi" by Jan Gossaert (called Mabuse) is 1.77 x 1.61 m while the recently acquired painting by the 15th-century netherlandish painter Robert Campin of "The Virgin & Child in An Interior" measures a mere 0.225 x 0.154 m including the wooden frame.

 

   At 0.25 m viewing distance the eye can resolve around 0.06 mm pels which gives required coverage of 257 pels/mm2. If viewed from this distance and scanning at this resolution the Guido Reni painting would occupy 11Gb (Giga bytes, of RGB) while the Robert Campin would require a mere 25Mbyte. However, it has already been shown that there is 2Gb of information held in a 10x8" transparency. For the Campin, which is the correct or optimum resolution? The resolution of the transparency or the resolving power of the human eye? In terms of cost, the 11Gb Guido Reni would require in the order of $2700 to store on common WORM optical disks (Optimem 2402).

 

   Scan resolution tests on large format transparencies have been carried out in collaboration with The National Gallery, London and Crosfield Ltd. The first test was to scan a 10 x 8" transparency of the Campin at 2802 x 3237 which included part of the wooden frame and thus was slightly less than the resolution suggested above of 2484 x 3629. Figure 1 shows a full frame (575 pels high) view of the Campin

6:1 subsampled. Figure 2 shows a full screen detail (of the scanned pels) and should show the resolution seen by the eye at 0.25 m.

 

   This scanning resolution did not show the grain structure in the photographic emulsions. For non-scientific examination this scanning resolution appears to be adequate for this painting with real pel full-screen details appearing at eight times magnification compared to the original. Due to storage limitations of the cartridge tapes used (60 Mbyte) the 10 x 8" transparency of the Gossaert was scanned at 3071 x 3410 pels rather than the suggested 28500 x 26000 pels. While considerable detail can be seen on full screen display of the real pels it is clear that a higher scan resolution is necessary for the craquelure and fine detail such as hair.

 

                               CODING

 

   The use of image coding where degradation is introduced can only be justified if it did not affect scientific studies or become visible on close examination. If the visual system is to be exploited by compression algorithms then sub-threshold errors can be introduced.  For visual use only, sophisticated compression schemes such as hybrid quadtree/transform coding might be used.

 

   Lossless coding using previous pel differences followed by Huffman coding was tested on two paintings in The National Gallery, London: 'Campin', 2802x3237 pels and Rembrandt's "Adoration of the Shepherds", 2983x3621 pels (10x8" Ektachrome transpanencies were scanned on a Crosfield Magascan system). The highly detailed 'Campin' was compressed by only 8% (the blue component 14% as it is less detailed) whereas the 'Rembrandt' was reduced by 37% (B 28%, R 43%, G 39%). Although this is only a crude test it points to the value of this technique in compressing images with large 'smooth' areas.

 

   Coding of the colour information is even more critical, as human observers can be fooled but colourimetry studies are very sensitive. For example luminance with 2:1 subsampled chrominance could be used as in the digital image standard (CCIR Rec. 601), which produces errors invisible to the eye. Compression to 'the best' 256 colours, as is used on many graphics systems leads to unacceptable inaccuracies in most cases, and should only be considered for images displayed on cheaper PC systems where the cost is decisive. For multi-spectral images, designed for scientific analysis, errors could render then useless, so conventional techniques must be carefully tested.

 

              STORAGE REQUIREMENTS

 

   User requirements will determine the amount and type of storage. If RGB images of eight bits per component are used and scanning resolution of 1kx1k per square meter is selected, this would require 3Mb. Then 341 m2 could be stored in 1Gb. A resolution based on the 0.25 m viewing distance dramatically increases storage so that a square meter requires 735Mb.

 

   The Birkbeck College History of Art Department slide library contains 100000 35mm slides. Scanning these transparencies at 1kx1k (RGB) for a full screen display on a high resolution monitor would require 300Gb. If scanned at the full resolution of a transparency (35Mb) then 3400Gb. This, of course, has not taken into consideration the dimenisions of the original works of art. The resolution should be scaled with the size of the painting.

 

   Even modest resolution produces huge amounts of data. Currently, optical disks are the only practical system of storing such large data. It would be possible to scan at even higher resolution, but an over emphasis on this requirement would compromise others such as cost, speed and bulk. As storage cost is probably the major factor, the resolution could be determined by the storage budget.

 

   The question of data stability is crucial but as optical disks have a life expectancy of >30 years the data will have been transfered to new storage technology before then.

 

                        IMAGE ACCESS

 

   For a public access or CAL environment rapid access times are necessary, but if the images are retreived from a large, slow archive this may be impossible. The speed problems encountered in the Musee d'Orsay project due to multiple access should be avoided. The data rate of a typical WORM disk (OPTIMEM 2400) is 625kbyte/s, so 3Mb of data (a 1kx1k image) would take over 5s to read. Current 'juke boxes' of optical disks also have long disk change times which would be frustrating if they occured often. The typical data rate of a high capacity winchester disk is around 3Mb/s so the same image would only take 1s to read, but 1Gb disks are expensive for a large archive. Parallel transfer disks provide even higher data rates (c.20Mb/s) but are even more expensive and the hardware necesssary to receive this data is more complex. To provide fast display access the images would probably have to be stored on winchester disks, with reduced resolution and data-compression. The use of separate storage media for fast access also reduces contention due to multiple users; one disk per viewing workstation could also be used. New developments in parallel optical disks will improve the data rates but the problem of accessing many disks still exists.

 

   Another important aspect of image access is zooming and panning. A full screen version will be required, as well as more detailed versions. To produce a low resolution version it is not sufficient to skip samples, but some attempt at proper subsampling with low pass filtering is necessary. [aliasing pic?] Ideal filtering may be extremely compute-intensive as large convolution masks are required, for example an 11x11 gausian filter is required for a 6:1 sampling. Simpler block average filters are faster to implement and usually provide acceptable results. Special hardware should be provided for this function if variable 'zoom-out' is required, (for example Datacube convolution boards [Gruen88]). Alternatively, a full screen image could be stored permanently on winchester disk, as this is the most used data. For systems with low storage capacity, selected details could be stored to provide only multiple choice rather than full area zoom capability. The conventional technique of pel replication to produce zooming is easily provided in the framestore hardware; 2:1 being useful on high resolution displays, and greater for visibility of individual pels.

 

   Panning the detailed image around the full extent of the painting is simple if the framestore has enough storage (RAM) to hold all the data, which could be considerable. Parallel transfer disks can be used to rapidly read the new data into RAM.

 

                             Conclusions

 

   Large digital archives of paintings can be contructed using flat-bed scanners, CCD arrays and optical disks. However, inspite of the extremely high specifications of some users, there will have to be some trade-offs in order to produce a practical system. Scanning at the resolution required for scientific studies may be impractical for very large paintings for time/storage reasons, and may necessitate a reduction in scanning resolution. However, scanning for some scientific purposes may take place separately from the general archive scanning. Similarly, the resolution of displays and cameras should not dictate the form of the data.

 

   It is proposed that a standard scan resolution technique, possibly visually based, be selected for the archive; a linear resolution increase with picture size is suggested. To be pratical, a limit has to be imposed on the scan resolution of large paintings. A fourier study could be used as a guide to scan resolution or a quad-tree coding technique used to compress the data in 'smooth' portions of the image. The number, width and range of the sensor channels must also be selected. These then provide a core archiving system from which lower specification sub-systems can be produced: for CAL or public access for example.

 

   At present it is envisaged that the database would be specific to certain galleries/museums due to copyright and reproduction rights. Until these legal and political issues are resolved, practical networking of the data is also impossible.

 

                       Acknowledgements

 

Thanks to Dr. David Saunders of The National Gallery; Teresa Telus and Tony Johnson of Crosfield Ltd.

 

                              References

 

Ch. Lahanier, F. Heitz, Ch. de Couessin, G. Querré

"Digitization of Works of Art - Study of Scientific Images"

Reports of the 12th International symposium on The Conservation and Restoration of Cultural Property -Analysis and examination of an art object by Imaging technique Tokyo 1988 29 September - 1st October pp.1-47

 

A. Hamber Colour Photography vs Electronic Digital Imaging as a System for Recording Works of Art The Journal of Photographic Science Vol.34, 1987 pp.200-208

 

A. Hamber The Musee  d'Orsay Videodics system CHArt Newsletter Number 6 Autumn 1987 pp.11-18

 

A. Gruen "Towards Real-Time Photogrammetry" Photogrammetria, Vol.42, No.5/6 pp.209-244 (1988)

 

Berger Lahr, Product description 1988.