Towards A Colorimetric Digital Image Archive for the Visual Arts
History of Art,
The aim of this project is to produce a high-resolution, colorimetric and permanent digital archive of images taken directly from works of art. The proposed system is designed for use in education, research, galleries and museums. Tentative user requirements are examined with particular reference to resolution, image access and colorimetry. Existing technology and projects are considered. Some 3000x3000 pel images of paintings are used to illustrate the interrelationship between dimensions of the original, its inherent detail, scan resolution and display.
The project arose out of the increase in the scope of the requirements of those involved in the teaching, research and scientific examination of paintings. There is also a growing awareness of the limitations of conventional colour photography. 35mm colour transparencies have become the standard format for teaching and most research purposes, larger formats being impractical as well as far more costly. As the slide library in the Department of History of Art grew to its present levels of c.150,000 35mm slides so its management became increasingly problematic. Combined with these problems is the image degradation and variation in quality between film types. Image decay, through vastly increased use by both staff and students coupled with storage in far from archival conditions, was becoming increasingly apparent. Furthermore, access in order to photograph objects in either public or private collections has, in recent years, become more and more restrictive.
Standard documentation techniques used in galleries and museums involve both images and complex textual data. Certain of these archives have become almost unmanageable through sheer volume and a digital alternative is awaited to replace photographic techniques.
LIMITATIONS OF PHOTOGRAPHY
Conventional colour emulsions are able to create a gamut of reproducible colour dependant on the spectral characteristics of the image dyes used.[OHTA] Additive systems not only have linear characteristics but also a theoretically greater gamut of colour than the subtractive system, which is also non-linear. Colour film characteristics vary between film types, batches and manufacturers. The gamut of colour film neither equates with the gamut of four colour printing nor that of the phosphors of a CRT.
Temperature and humidity can be highly influential factors governing overall image quality and ultimately, however carefully film is stored and however exhaustive batching testing and calibration is, the processing of colour film can be the most influential factor on the overall accuracy of the reproduction of colour.
Conventional colour photographic emulsions, depending on the test-object contrast, have an ability to resolve detail (on the emulsion) to between 40 and 120 lines/mm. In the case of black and white photographic materials, however, this may increase up to 400 lines/mm; some photographic plates produced for holography can theoretically resolve as many as 1000 lines/mm. The resolution captured by the film from the original object is dependent on the dimensions of this object with respect to the film size. Even large format transparencies cannot capture adequate detail from a large painting, as is illustrated below.
The photographic negative enables multiple copies of a single original to be reproduced. It is not, however, possible to reproduce fully all the inherent information in a negative; especially the contrast. Moreover, the print from a negative is a second generation reproduction and often extremely difficult to produce and accurately assess for quality. For this reason colour transparencies are more commonly used in colour reproduction.
Conventional colour photography is severely limited in terms of accessibility and retrieval systems. Photographic prints require large amounts of storage space as do 35mm slides. Access may be slow, even if there are electronic cataloguing systems; some of which may include a reference image. Unless there are multiple copies of each image, access is only possible by one individual at a time.
The emulsions used in photographic colour materials use chemical dyes which are neither stable nor permanent. Light, temperature, humidity and other environmental factors affect stability and permanence. Teaching and research requirements often involve exposure to levels of light and temperature far in excess of the "average" use calculated by the manufacturers. The degredation of the image in a photographic transparency cannot be quantitatively determined because of the non-linear changes in the characteristics of the individual dye layers and the gelatin base.
Table 1. User Requirements
The major users of the archive are envisaged to be Art institutions and educational establishments. Art galleries and museums would use it for: administration, documentation, conservation, restoration, catalogues, reproductions and public access. Surface texture change and colour difference measurement have been identified in gallery scientific departments. Educational uses, at all levels, include: distance learning, computer-aided learning and instruction. Such an archive would be of particular use for specialised academic research. The impact of commercial use in publishing and broadcasting will undoubtedly be highly significant in the 1990s. There are clearly applications in completely different areas such as medical or satellite imaging.
The textual information required for each image is varied but could be handled by conventional relational databases. It is the image requirements which cause major problems, in terms of acquisition, storage and retrieval. For recognition purposes only, a TV resolution image could be used (768x575), and image coding used to best effect. For detailed examination, small objects in the painting must be visible, such as flowers, birds or even the brushstrokes and craquelure of the paint itself. Consider a painting measuring 2x2m, a feature such as a hand could be 2cm long, so (assuming 10 pels minimum to resolve five fingers) at least 1000x1000 pels are needed purely to resolve the fingers. In reality a much higher resolution is needed to provide the level of detail seen when standing close to a painting and simple sampling theory is insufficient to provide good rendition in images.
In certain cases photographs or transparencies are documents of the highest importance since they may be the only record of works of art which have been lost, destroyed, damaged, cleaned or restored. They therefore assume the position of an original rather than a reproduction. Thus if all the information recorded by a transparency is required for imaging and archiving, then for a 35mm slide (taking 60 lp/mm resolution on the emulsion, two pels per line-pair and 3bytes per pel) 35Mb is necessary but on 10x8" (a commonly used format in commercial reproduction), 2Gb would be necessary. Clearly the ability of large format transparencies to resolve such levels of detail is highly problematic. The acquisition storage and processing of the digital information scanned from a large format transparency is considerably more difficult than might be imagined.
Scientific departments of museums and galleries require spectral and non-visible data, including infra red, ultra-violet and x-ray images. Many gallery archives include photographs of paintings taken in the IR and X-ray region, as well as raking light images, so there is a requirement for non-visible data. This brings similar problems to those found in remote sensing which requires the storage of a wide variety of spectral bands.
Tristimulus scanning can not cover all of a colour space, while broadband spectral scanning covers a larger area. Both the colour space and the number of spectral bands recorded require carefully selection. Initial tests have shown that the transmittance of narrow band filters is so low that a good signal can not easily be recovered by a CCD without unacceptable levels of illumination. The alternative is to use a spectrometer technique, but the single scanning spot would cause speed problems. At some point this data must be converted into RGB for viewing, so if a sampled spectrum is stored it must be accurate enough for the RGB conversion to give precise colour rendition.
If the archived images are for viewing only then tristimulus values, such as RGB, would be stored. The reason for storing in a standard colour space is that the transform from these systems to the RGB co-ordinates corresponding to commonly used phosphors is well known. The retreival and conversion of spectral data would greatly increase image access times. Unfortunately the transformation into XYZ (for example) discards spectral information that ought to be retained for scientific analysis of images. The transform will need to be confirmed using samples of well defined colour value/co-ordinates i.e. NPL tiles/Munsell chips etc. However some users require more information than the sensitivity of the eye, for example to measure colour or surface texture change, when the data should suit computer vision more than human vision.
The major conflicts in the user requirements are those of tristimulus quantities for viewing and multi-spectral for scientific purposes. To satisfy both implies an ability to scan data in both formats, either directly, or from multi-spectral scanning with later conversion. The nature of the multi-spactral technique has yet to be investigated but clearly has major effects on hardware and software.
In terms of electronic imaging devices, CCDs are currently becoming the standard. They have good geometric stability and a better dynamic range than conventional video camera tubes. CCD devices with 4096 sensor linear arrays and 1035 x 1320 area arrays are commonly available.
Many metal oxide semiconductor (MOS) CCDs are, however, less sensitive then a photodiode, particularly in the blue region of the visible spectrum. Neither a photodiode nor an MOS device have a response which equates to human perception of lightness or colour. Futhermore most CCDs are sensitive in the infra-red region which, for visible imaging, necessitates the use of a blocking filter. Other imaging techniques, such as micro-densitometers are capable of producing high-resolution output but are extremely slow.
Although image archives are possible using magnetic tape systems, such processes have always had lengthy, sometimes unacceptable, access times and invariably occupy large amounts of space. Computers are now also much cheaper, smaller and more reliable. These factors bring the possibility of cheaper digital archives for use by institutions with restricted budgets.
required for high resolution colour images is most conveniently measured in
giga-bytes (Gb). The data must be stored accurately, permanently and with a
fast access time. The advent of WORM optical disks may provide a solution. Each
disk, usually 5-12" in diameter, stores 2-8 Gb. Such disks have been
incorporated into 'juke boxes' which can access information from a number of
disks; for instance the Kodak system 6800 model D contains up to 150 disks with
a total storage capacity of 1020 Gb. The drawback with this particular device
is that there is but one drive for reading the disks. Ideally, the number of
drives should be sufficient to cope with multiple requests to read data from
the archive. Individual disks are relatively inexpensive compared to magnetic
media (other than tapes) at around $600 each for an Optimem 2Gb disk. Unfortunately
juke-boxes may have disk change times of tens of seconds.
To handle such large quantities of data, and more especially to carry out image processing operations, requires dedicated image processing hardware if response times are to be within the specifications of diverse user requirements. There are now many super-micros in the above 10 MIPS range, which can link to special hardware for I/O and processing. High speed networks now allow the distribution of different processors and storage systems for greater efficiency.
All present display technologies have their limitations primarily in resolution and colour rendition. CRT technology is still the best available although the screen is not flat and the colour gamut is different from that of colour film, four colour printing or the information acquired by scanning. 4000 line colour monitors have recently become available (for example from Sony ) but are extremely expensive and require the use of very fast framestores. Large screen monitors which can show the scale of the original are not yet available and are unlikely to be of sufficient quality.
In respect of display ergonomics, consideration must be taken of the viewing distance. Taking 50cm as the average viewing distance in a workstation environment indicates a theoretical resolution on the display of 8 pixels/mm. Unfortunately viewers tend to move as near as 25cm when looking closely at an object such as a painting. If this were to be the case when viewing a display then a display resolution of 16 pixels/mm would be necessary for the line structure to be invisible. This would be the equivalent of 4500 lines on a standard 18 inch monitor which is currently unrealistic.
It is assumed here that the resolution on the scanned data will be higher than that of the display, thus a suitable viewing system is required. If a standard display resolution can be agreed (eg around 1kx1k) then zooming and panning of the viewport into the data would provide detailed examination of the original, with a subsampled version viewable full-screen.
Hard copy output in the form of transparencies is also required, as these are transportable and more useful in many circumstances. 4000 line 35mm slide writers already exist to satisfy this requirement. For larger formats, 10x8" colour transparency writers are also available (e.g. Crosfield MAGNASCAN). Similarly instant print makers could provide immediate hardcopy. There are several new non-impact printing technologies being developed at present. For example the JetProof colour ink-jet printer has a resolution of 300 dpi.
several numerous digital and analogue databases of images of works of art. The
digital public access systems at the Musee d'Orsay,
None of these projects combine high resolution and colorimetry to create a multi-purpose image database.
POSSIBLE SCANNER DESIGNS
There are fundamental problems to be solved in the scanning of paintings. Some of these are purely technical while others are simply practical. For example many galleries have security systems which makes it expensive to remove paintings from the wall, even after permission has been obtained. Some works are impossible to move (such as wall paintings) and thus a portable scanner would be the only answer. Photographic recording as a first stage can be discounted on the grounds of inaccuracies, particularly in terms of colour and contrast. However, photographic transparencies can be digitised realitively cheaply and quickly on existing drum scanners, which convenient for transparencies of lost, destroyed, cleaned or restored paintings. Direct scanning of the painting should be the major goal if possible, as the accuracy of the data would be greatly enhanced.
A scanner can
be produced where the object is positioned horizontally or vertically (usually
on a stable optical bench) and an X-Y bridge scans an imaging device (eg a CCD
camera) over the surface. X-Y bridges used in manufacturing industries are
capable of very fine positioning (0.25 micron for Leitz microscope small area
after Berger Lahr crude move - REF) over substantial areas (say 3mx3m).
Geometric correction would then mosaic the stripes or blocks in a similar way
to remote sensing. In this way extremely fine details of small areas can be imaged
and it is even possible to arrange for subsequent aligned scans (REF
A scanning system could be fitted with different standard illumination and imaging devices to scan in the infra red for example, or even multiple channels simultaneously. Problems arise with specular reflection due to surface texture. Anomalous reflectance and fluorescence should be detectable and correctable where necessary.
Assuming the eye can resolve up to 35 cycles per degree, at a 1m viewing distance, 0.5 mm cycles are seen, so 0.25mm pel coverage is needed. There is an inverse relationship between viewing distance and resolvable detail measured on the object. This suggests that at 1m viewing distance, 4kx4k pels resolution scanned for each square meter of painting would provide as much detail as could be seen 1m away from the object. This viewing distance/resolution relationship is crucial both for image acquisition and subsequent image display. If the scanning resolution is based on human resolving power then a decision has to be made on the closest viewing distance from the painting. However, those working in the restoration and conservation departments of galleries and museums often require higher levels of resolution than perceivable by the naked human eye.
The variation in dimensions of three paintings in the National Gallery London indicates some of the problems to be encountered. "The Adoration of the Shepherds" by the 17th-Century italian painter Guido Reni has dimensions of 4.8 x 3.21 m; "The Adoration of the Magi" by Jan Gossaert (called Mabuse) is 1.77 x 1.61 m while the recently acquired painting by the 15th-century netherlandish painter Robert Campin of "The Virgin & Child in An Interior" measures a mere 0.225 x 0.154 m including the wooden frame.
At 0.25 m viewing distance the eye can resolve around 0.06 mm pels which gives required coverage of 257 pels/mm2. If viewed from this distance and scanning at this resolution the Guido Reni painting would occupy 11Gb (Giga bytes, of RGB) while the Robert Campin would require a mere 25Mbyte. However, it has already been shown that there is 2Gb of information held in a 10x8" transparency. For the Campin, which is the correct or optimum resolution? The resolution of the transparency or the resolving power of the human eye? In terms of cost, the 11Gb Guido Reni would require in the order of $2700 to store on common WORM optical disks (Optimem 2402).
resolution tests on large format transparencies have been carried out in
collaboration with The National Gallery,
6:1 subsampled. Figure 2 shows a full screen detail (of the scanned pels) and should show the resolution seen by the eye at 0.25 m.
This scanning resolution did not show the grain structure in the photographic emulsions. For non-scientific examination this scanning resolution appears to be adequate for this painting with real pel full-screen details appearing at eight times magnification compared to the original. Due to storage limitations of the cartridge tapes used (60 Mbyte) the 10 x 8" transparency of the Gossaert was scanned at 3071 x 3410 pels rather than the suggested 28500 x 26000 pels. While considerable detail can be seen on full screen display of the real pels it is clear that a higher scan resolution is necessary for the craquelure and fine detail such as hair.
The use of image coding where degradation is introduced can only be justified if it did not affect scientific studies or become visible on close examination. If the visual system is to be exploited by compression algorithms then sub-threshold errors can be introduced. For visual use only, sophisticated compression schemes such as hybrid quadtree/transform coding might be used.
coding using previous pel differences followed by Huffman coding was tested on
two paintings in The National Gallery,
Coding of the colour information is even more critical, as human observers can be fooled but colourimetry studies are very sensitive. For example luminance with 2:1 subsampled chrominance could be used as in the digital image standard (CCIR Rec. 601), which produces errors invisible to the eye. Compression to 'the best' 256 colours, as is used on many graphics systems leads to unacceptable inaccuracies in most cases, and should only be considered for images displayed on cheaper PC systems where the cost is decisive. For multi-spectral images, designed for scientific analysis, errors could render then useless, so conventional techniques must be carefully tested.
User requirements will determine the amount and type of storage. If RGB images of eight bits per component are used and scanning resolution of 1kx1k per square meter is selected, this would require 3Mb. Then 341 m2 could be stored in 1Gb. A resolution based on the 0.25 m viewing distance dramatically increases storage so that a square meter requires 735Mb.
The Birkbeck College History of Art Department slide library contains 100000 35mm slides. Scanning these transparencies at 1kx1k (RGB) for a full screen display on a high resolution monitor would require 300Gb. If scanned at the full resolution of a transparency (35Mb) then 3400Gb. This, of course, has not taken into consideration the dimenisions of the original works of art. The resolution should be scaled with the size of the painting.
Even modest resolution produces huge amounts of data. Currently, optical disks are the only practical system of storing such large data. It would be possible to scan at even higher resolution, but an over emphasis on this requirement would compromise others such as cost, speed and bulk. As storage cost is probably the major factor, the resolution could be determined by the storage budget.
The question of data stability is crucial but as optical disks have a life expectancy of >30 years the data will have been transfered to new storage technology before then.
For a public
important aspect of image access is zooming and panning. A full screen version
will be required, as well as more detailed versions. To produce a low
resolution version it is not sufficient to skip samples, but some attempt at
proper subsampling with low pass filtering is necessary. [aliasing pic?] Ideal
filtering may be extremely compute-intensive as large convolution masks are
required, for example an 11x11 gausian filter is required for a 6:1 sampling.
Simpler block average filters are faster to implement and usually provide
acceptable results. Special hardware should be provided for this function if
variable 'zoom-out' is required, (for example Datacube convolution boards
[Gruen88]). Alternatively, a full screen image could be stored permanently on
Panning the detailed image around the full extent of the painting is simple if the framestore has enough storage (RAM) to hold all the data, which could be considerable. Parallel transfer disks can be used to rapidly read the new data into RAM.
Large digital archives of paintings can be contructed using flat-bed scanners, CCD arrays and optical disks. However, inspite of the extremely high specifications of some users, there will have to be some trade-offs in order to produce a practical system. Scanning at the resolution required for scientific studies may be impractical for very large paintings for time/storage reasons, and may necessitate a reduction in scanning resolution. However, scanning for some scientific purposes may take place separately from the general archive scanning. Similarly, the resolution of displays and cameras should not dictate the form of the data.
It is proposed that a standard scan resolution technique, possibly visually based, be selected for the archive; a linear resolution increase with picture size is suggested. To be pratical, a limit has to be imposed on the scan resolution of large paintings. A fourier study could be used as a guide to scan resolution or a quad-tree coding technique used to compress the data in 'smooth' portions of the image. The number, width and range of the sensor channels must also be selected. These then provide a core archiving system from which lower specification sub-systems can be produced: for CAL or public access for example.
At present it is envisaged that the database would be specific to certain galleries/museums due to copyright and reproduction rights. Until these legal and political issues are resolved, practical networking of the data is also impossible.
Thanks to Dr. David Saunders of The National Gallery; Teresa Telus and Tony Johnson of Crosfield Ltd.
Ch. Lahanier, F. Heitz, Ch. de Couessin, G. Querré
"Digitization of Works of Art - Study of Scientific Images"
Reports of the 12th International symposium on The
Conservation and Restoration of Cultural Property -Analysis and examination of
an art object by Imaging technique
A. Hamber Colour Photography vs Electronic Digital Imaging as a System for Recording Works of Art The Journal of Photographic Science Vol.34, 1987 pp.200-208
A. Hamber The Musee d'Orsay Videodics system CHArt Newsletter Number 6 Autumn 1987 pp.11-18
A. Gruen "Towards Real-Time Photogrammetry" Photogrammetria, Vol.42, No.5/6 pp.209-244 (1988)
Berger Lahr, Product description 1988.