Towards A
Colorimetric Digital Image Archive for the Visual Arts
Kirk
Department of
History of Art,
ABSTRACT
The aim of
this project is to produce a high-resolution, colorimetric and permanent
digital archive of images taken directly from works of art. The proposed system
is designed for use in education, research, galleries and museums. Tentative
user requirements are examined with particular reference to resolution, image
access and colorimetry. Existing technology and projects are considered. Some
3000x3000 pel images of paintings are used to illustrate the interrelationship
between dimensions of the original, its inherent detail, scan resolution and
display.
INTRODUCTION
The project
arose out of the increase in the scope of the requirements of those involved in
the teaching, research and scientific examination of paintings. There is also a
growing awareness of the limitations of conventional colour photography. 35mm
colour transparencies have become the standard format for teaching and most
research purposes, larger formats being impractical as well as far more costly.
As the slide library in the Department of History of Art grew to its present
levels of c.150,000 35mm slides so its management became increasingly
problematic. Combined with these problems is the image degradation and
variation in quality between film types. Image decay, through vastly increased
use by both staff and students coupled with storage in far from archival
conditions, was becoming increasingly apparent. Furthermore, access in order to
photograph objects in either public or private collections has, in recent
years, become more and more restrictive.
Standard documentation techniques used in
galleries and museums involve both images and complex textual data. Certain of
these archives have become almost unmanageable through sheer volume and a
digital alternative is awaited to replace photographic techniques.
LIMITATIONS
OF PHOTOGRAPHY
Conventional
colour emulsions are able to create a gamut of reproducible colour dependant on
the spectral characteristics of the image dyes used.[OHTA] Additive systems not only have linear
characteristics but also a theoretically greater gamut of colour than the
subtractive system, which is also non-linear. Colour film characteristics vary
between film types, batches and manufacturers. The gamut of colour film neither
equates with the gamut of four colour printing nor that of the phosphors of a
CRT.
Temperature
and humidity can be highly influential factors governing overall image quality
and ultimately, however carefully film is stored and however exhaustive
batching testing and calibration is, the processing of colour film can be the
most influential factor on the overall accuracy of the reproduction of colour.
Conventional
colour photographic emulsions, depending on the test-object contrast, have an
ability to resolve detail (on the emulsion) to between 40 and 120 lines/mm. In
the case of black and white photographic materials, however, this may increase
up to 400 lines/mm; some photographic plates produced for holography can
theoretically resolve as many as 1000 lines/mm. The resolution captured by the
film from the original object is dependent on the dimensions of this object
with respect to the film size. Even large format transparencies cannot capture
adequate detail from a large painting, as is illustrated below.
The
photographic negative enables multiple copies of a single original to be
reproduced. It is not, however, possible to reproduce fully all the inherent
information in a negative; especially the contrast. Moreover, the print from a
negative is a second generation reproduction and often extremely difficult to
produce and accurately assess for quality. For this reason colour
transparencies are more commonly used in colour reproduction.
Conventional
colour photography is severely limited in terms of accessibility and retrieval
systems. Photographic prints require large amounts of storage space as do 35mm
slides. Access may be slow, even if there are electronic cataloguing systems;
some of which may include a reference image. Unless there are multiple copies
of each image, access is only possible by one individual at a time.
The emulsions
used in photographic colour materials use chemical dyes which are neither
stable nor permanent. Light, temperature, humidity and other environmental
factors affect stability and permanence. Teaching and research requirements
often involve exposure to levels of light and temperature far in excess of the
"average" use calculated by the manufacturers. The degredation of the
image in a photographic transparency cannot be quantitatively determined
because of the non-linear changes in the characteristics of the individual dye
layers and the gelatin base.
Table
1. User Requirements
USER
REQUIREMENTS
The major
users of the archive are envisaged to be Art institutions and educational
establishments. Art galleries and museums would use it for: administration,
documentation, conservation, restoration, catalogues, reproductions and public
access. Surface texture change and colour difference measurement have been
identified in gallery scientific departments. Educational uses, at all levels,
include: distance learning, computer-aided learning and instruction. Such an
archive would be of particular use for specialised academic research. The
impact of commercial use in publishing and broadcasting will undoubtedly be
highly significant in the 1990s. There are clearly applications in completely
different areas such as medical or satellite imaging.
The textual
information required for each image is varied but could be handled by
conventional relational databases. It is the image requirements which cause
major problems, in terms of acquisition, storage and retrieval. For recognition
purposes only, a TV resolution image could be used (768x575), and image coding
used to best effect. For detailed examination, small objects in the painting
must be visible, such as flowers, birds or even the brushstrokes and craquelure
of the paint itself. Consider a painting measuring 2x2m, a feature such as a
hand could be 2cm long, so (assuming 10 pels minimum to resolve five fingers)
at least 1000x1000 pels are needed purely to resolve the fingers. In reality a
much higher resolution is needed to provide the level of detail seen when
standing close to a painting and simple sampling theory is insufficient to
provide good rendition in images.
In certain
cases photographs or transparencies are documents of the highest importance
since they may be the only record of works of art which have been lost,
destroyed, damaged, cleaned or restored. They therefore assume the position of
an original rather than a reproduction. Thus if all the information recorded by
a transparency is required for imaging and archiving, then for a 35mm slide
(taking 60 lp/mm resolution on the emulsion, two pels per line-pair and 3bytes
per pel) 35Mb is necessary but on 10x8" (a commonly used format in
commercial reproduction), 2Gb would be necessary. Clearly the ability of large
format transparencies to resolve such levels of detail is highly problematic.
The acquisition storage and processing of the digital information scanned from
a large format transparency is considerably more difficult than might be
imagined.
Scientific
departments of museums and galleries require spectral and non-visible data,
including infra red, ultra-violet and x-ray images. Many gallery archives
include photographs of paintings taken in the IR and X-ray region, as well as
raking light images, so there is a requirement for non-visible data. This
brings similar problems to those found in remote sensing which requires the
storage of a wide variety of spectral bands.
Tristimulus
scanning can not cover all of a colour space, while broadband spectral scanning
covers a larger area. Both the colour space and the number of spectral bands
recorded require carefully selection. Initial tests have shown that the
transmittance of narrow band filters is so low that a good signal can not
easily be recovered by a CCD without unacceptable levels of illumination. The
alternative is to use a spectrometer technique, but the single scanning spot
would cause speed problems. At some point this data must be converted into RGB
for viewing, so if a sampled spectrum is stored it must be accurate enough for
the RGB conversion to give precise colour rendition.
If the
archived images are for viewing only then tristimulus values, such as RGB,
would be stored. The reason for storing in a standard colour space is that the
transform from these systems to the RGB co-ordinates corresponding to commonly
used phosphors is well known. The retreival and conversion of spectral data
would greatly increase image access times. Unfortunately the transformation
into XYZ (for example) discards spectral information that ought to be retained
for scientific analysis of images. The transform will need to be confirmed
using samples of well defined colour value/co-ordinates i.e. NPL tiles/Munsell
chips etc. However some users require more information than the sensitivity of
the eye, for example to measure colour or surface texture change, when the data
should suit computer vision more than human vision.
The major
conflicts in the user requirements are those of tristimulus quantities for
viewing and multi-spectral for scientific purposes. To satisfy both implies an
ability to scan data in both formats, either directly, or from multi-spectral
scanning with later conversion. The nature of the multi-spactral technique has
yet to be investigated but clearly has major effects on hardware and software.
AVAILABLE
TECHNOLOGY
Imaging devices
In terms of
electronic imaging devices, CCDs are currently becoming the standard. They have
good geometric stability and a better dynamic range than conventional video
camera tubes. CCD devices with 4096 sensor linear arrays and 1035 x 1320 area
arrays are commonly available.
Many metal
oxide semiconductor (MOS) CCDs are, however, less sensitive then a photodiode,
particularly in the blue region of the visible spectrum. Neither a photodiode
nor an MOS device have a response which equates to human perception of
lightness or colour. Futhermore most CCDs are sensitive in the infra-red region
which, for visible imaging, necessitates the use of a blocking filter. Other
imaging techniques, such as micro-densitometers are capable of producing high-resolution
output but are extremely slow.
Storage
Although
image archives are possible using magnetic tape systems, such processes have
always had lengthy, sometimes unacceptable, access times and invariably occupy
large amounts of space. Computers are now also much cheaper, smaller and more
reliable. These factors bring the possibility of cheaper digital archives for
use by institutions with restricted budgets.
The storage
required for high resolution colour images is most conveniently measured in
giga-bytes (Gb). The data must be stored accurately, permanently and with a
fast access time. The advent of WORM optical disks may provide a solution. Each
disk, usually 5-12" in diameter, stores 2-8 Gb. Such disks have been
incorporated into 'juke boxes' which can access information from a number of
disks; for instance the Kodak system 6800 model D contains up to 150 disks with
a total storage capacity of 1020 Gb. The drawback with this particular device
is that there is but one drive for reading the disks. Ideally, the number of
drives should be sufficient to cope with multiple requests to read data from
the archive. Individual disks are relatively inexpensive compared to magnetic
media (other than tapes) at around $600 each for an Optimem 2Gb disk. Unfortunately
juke-boxes may have disk change times of tens of seconds.
To handle
such large quantities of data, and more especially to carry out image
processing operations, requires dedicated image processing hardware if response
times are to be within the specifications of diverse user requirements. There
are now many super-micros in the above 10 MIPS range, which can link to special
hardware for I/O and processing. High speed networks now allow the distribution
of different processors and storage systems for greater efficiency.
Displays
All present
display technologies have their limitations primarily in resolution and colour
rendition. CRT technology is still the best available although the screen is
not flat and the colour gamut is different from that of colour film, four
colour printing or the information acquired by scanning. 4000 line colour
monitors have recently become available (for example from Sony ) but are
extremely expensive and require the use of very fast framestores. Large screen
monitors which can show the scale of the original are not yet available and are
unlikely to be of sufficient quality.
In respect of
display ergonomics, consideration must be taken of the viewing distance. Taking
50cm as the average viewing distance in a workstation environment indicates a
theoretical resolution on the display of 8 pixels/mm. Unfortunately viewers
tend to move as near as 25cm when looking closely at an object such as a
painting. If this were to be the case when viewing a display then a display
resolution of 16 pixels/mm would be necessary for the line structure to be
invisible. This would be the equivalent of 4500 lines on a standard 18 inch
monitor which is currently unrealistic.
It is assumed
here that the resolution on the scanned data will be higher than that of the
display, thus a suitable viewing system is required. If a standard display
resolution can be agreed (eg around 1kx1k) then zooming and panning of the
viewport into the data would provide detailed examination of the original, with
a subsampled version viewable full-screen.
Hard copy
output in the form of transparencies is also required, as these are
transportable and more useful in many circumstances. 4000 line 35mm slide
writers already exist to satisfy this requirement. For larger formats,
10x8" colour transparency writers are also available (e.g. Crosfield
MAGNASCAN). Similarly instant print makers could provide immediate hardcopy.
There are several new non-impact printing technologies being developed at
present. For example the JetProof colour ink-jet printer has a resolution of
300 dpi.
Existing projects
There are
several numerous digital and analogue databases of images of works of art. The
digital public access systems at the Musee d'Orsay,
The Open
None of these
projects combine high resolution and colorimetry to create a multi-purpose
image database.
POSSIBLE
SCANNER DESIGNS
There are
fundamental problems to be solved in the scanning of paintings. Some of these are purely technical while
others are simply practical. For example many galleries have security systems
which makes it expensive to remove paintings from the wall, even after
permission has been obtained. Some works are impossible to move (such as wall
paintings) and thus a portable scanner would be the only answer. Photographic
recording as a first stage can be discounted on the grounds of inaccuracies,
particularly in terms of colour and contrast. However, photographic
transparencies can be digitised realitively cheaply and quickly on existing
drum scanners, which convenient for transparencies of lost, destroyed, cleaned
or restored paintings. Direct scanning of the painting should be the major goal
if possible, as the accuracy of the data would be greatly enhanced.
A scanner can
be produced where the object is positioned horizontally or vertically (usually
on a stable optical bench) and an X-Y bridge scans an imaging device (eg a CCD
camera) over the surface. X-Y bridges used in manufacturing industries are
capable of very fine positioning (0.25 micron for Leitz microscope small area
after Berger Lahr crude move - REF) over substantial areas (say 3mx3m).
Geometric correction would then mosaic the stripes or blocks in a similar way
to remote sensing. In this way extremely fine details of small areas can be imaged
and it is even possible to arrange for subsequent aligned scans (REF
A scanning
system could be fitted with different standard illumination and imaging devices
to scan in the infra red for example, or even multiple channels simultaneously.
Problems arise with specular reflection due to surface texture. Anomalous
reflectance and fluorescence should be detectable and correctable where
necessary.
SCAN
RESOLUTION
Assuming the
eye can resolve up to 35 cycles per degree, at a 1m viewing distance, 0.5 mm
cycles are seen, so 0.25mm pel coverage is needed. There is an inverse
relationship between viewing distance and resolvable detail measured on the
object. This suggests that at 1m viewing distance, 4kx4k pels resolution
scanned for each square meter of painting would provide as much detail as could
be seen 1m away from the object. This viewing distance/resolution relationship
is crucial both for image acquisition and subsequent image display. If the
scanning resolution is based on human resolving power then a decision has to be
made on the closest viewing distance from the painting. However, those working
in the restoration and conservation departments of galleries and museums often
require higher levels of resolution than perceivable by the naked human eye.
The variation
in dimensions of three paintings in the National Gallery London indicates some
of the problems to be encountered. "The Adoration of the Shepherds"
by the 17th-Century italian painter Guido Reni has dimensions of 4.8 x 3.21 m;
"The Adoration of the Magi" by Jan Gossaert (called Mabuse) is 1.77 x
1.61 m while the recently acquired painting by the 15th-century netherlandish
painter Robert Campin of "The Virgin & Child in An Interior"
measures a mere 0.225 x 0.154 m including the wooden frame.
At 0.25 m
viewing distance the eye can resolve around 0.06 mm pels which gives required
coverage of 257 pels/mm2. If viewed from this distance and scanning at this
resolution the Guido Reni painting would occupy 11Gb (Giga bytes, of RGB) while
the Robert Campin would require a mere 25Mbyte. However, it has already been
shown that there is 2Gb of information held in a 10x8" transparency. For
the Campin, which is the correct or optimum resolution? The resolution of the
transparency or the resolving power of the human eye? In terms of cost, the
11Gb Guido Reni would require in the order of $2700 to store on common WORM
optical disks (Optimem 2402).
Scan
resolution tests on large format transparencies have been carried out in
collaboration with The National Gallery,
6:1 subsampled. Figure 2 shows a full screen detail
(of the scanned pels) and should show the resolution seen by the eye at 0.25 m.
This scanning
resolution did not show the grain structure in the photographic emulsions. For
non-scientific examination this scanning resolution appears to be adequate for
this painting with real pel full-screen details appearing at eight times
magnification compared to the original. Due to storage limitations of the
cartridge tapes used (60 Mbyte) the 10 x 8" transparency of the Gossaert
was scanned at 3071 x 3410 pels rather than the suggested 28500 x 26000 pels.
While considerable detail can be seen on full screen display of the real pels
it is clear that a higher scan resolution is necessary for the craquelure and
fine detail such as hair.
CODING
The use of
image coding where degradation is introduced can only be justified if it did
not affect scientific studies or become visible on close examination. If the
visual system is to be exploited by compression algorithms then sub-threshold
errors can be introduced. For visual use
only, sophisticated compression schemes such as hybrid quadtree/transform
coding might be used.
Lossless
coding using previous pel differences followed by Huffman coding was tested on
two paintings in The National Gallery,
Coding of the
colour information is even more critical, as human observers can be fooled but
colourimetry studies are very sensitive. For example luminance with 2:1
subsampled chrominance could be used as in the digital image standard (CCIR
Rec. 601), which produces errors invisible to the eye. Compression to 'the
best' 256 colours, as is used on many graphics systems leads to unacceptable
inaccuracies in most cases, and should only be considered for images displayed
on cheaper PC systems where the cost is decisive. For multi-spectral images,
designed for scientific analysis, errors could render then useless, so
conventional techniques must be carefully tested.
STORAGE
REQUIREMENTS
User
requirements will determine the amount and type of storage. If RGB images of
eight bits per component are used and scanning resolution of 1kx1k per square
meter is selected, this would require 3Mb. Then 341 m2 could be
stored in 1Gb. A resolution based on the 0.25 m viewing distance dramatically
increases storage so that a square meter requires 735Mb.
The Birkbeck
College History of Art Department slide library contains 100000 35mm slides.
Scanning these transparencies at 1kx1k (RGB) for a full screen display on a
high resolution monitor would require 300Gb. If scanned at the full resolution
of a transparency (35Mb) then 3400Gb. This, of course, has not taken into
consideration the dimenisions of the original works of art. The resolution
should be scaled with the size of the painting.
Even modest
resolution produces huge amounts of data. Currently, optical disks are the only
practical system of storing such large data. It would be possible to scan at
even higher resolution, but an over emphasis on this requirement would compromise
others such as cost, speed and bulk. As storage cost is probably the major
factor, the resolution could be determined by the storage budget.
The question
of data stability is crucial but as optical disks have a life expectancy of
>30 years the data will have been transfered to new storage technology
before then.
IMAGE
ACCESS
For a public
access or
Another
important aspect of image access is zooming and panning. A full screen version
will be required, as well as more detailed versions. To produce a low
resolution version it is not sufficient to skip samples, but some attempt at
proper subsampling with low pass filtering is necessary. [aliasing pic?] Ideal
filtering may be extremely compute-intensive as large convolution masks are
required, for example an 11x11 gausian filter is required for a 6:1 sampling.
Simpler block average filters are faster to implement and usually provide
acceptable results. Special hardware should be provided for this function if
variable 'zoom-out' is required, (for example Datacube convolution boards
[Gruen88]). Alternatively, a full screen image could be stored permanently on
Panning the
detailed image around the full extent of the painting is simple if the
framestore has enough storage (RAM) to hold all the data, which could be
considerable. Parallel transfer disks can be used to rapidly read the new data
into RAM.
Conclusions
Large digital
archives of paintings can be contructed using flat-bed scanners, CCD arrays and
optical disks. However, inspite of the extremely high specifications of some
users, there will have to be some trade-offs in order to produce a practical
system. Scanning at the resolution required for scientific studies may be
impractical for very large paintings for time/storage reasons, and may
necessitate a reduction in scanning resolution. However, scanning for some
scientific purposes may take place separately from the general archive
scanning. Similarly, the resolution of displays and cameras should not dictate
the form of the data.
It is
proposed that a standard scan resolution technique, possibly visually based, be
selected for the archive; a linear resolution increase with picture size is suggested.
To be pratical, a limit has to be imposed on the scan resolution of large
paintings. A fourier study could be used as a guide to scan resolution or a
quad-tree coding technique used to compress the data in 'smooth' portions of
the image. The number, width and range of the sensor channels must also be
selected. These then provide a core archiving system from which lower
specification sub-systems can be produced: for CAL or public access for
example.
At present it
is envisaged that the database would be specific to certain galleries/museums
due to copyright and reproduction rights. Until these legal and political
issues are resolved, practical networking of the data is also impossible.
Acknowledgements
Thanks to Dr. David Saunders of The National Gallery;
Teresa Telus and Tony Johnson of Crosfield Ltd.
References
Ch. Lahanier, F. Heitz, Ch. de Couessin, G. Querré
"Digitization of Works of Art - Study of
Scientific Images"
Reports of the 12th International symposium on The
Conservation and Restoration of Cultural Property -Analysis and examination of
an art object by Imaging technique
A. Hamber Colour Photography vs Electronic Digital
Imaging as a System for Recording Works of Art The Journal of Photographic
Science Vol.34, 1987 pp.200-208
A. Hamber The Musee
d'Orsay Videodics system CHArt Newsletter Number 6 Autumn 1987
pp.11-18
A. Gruen "Towards Real-Time Photogrammetry" Photogrammetria,
Vol.42, No.5/6 pp.209-244 (1988)
Berger Lahr, Product description 1988.