The Pathetic Fallacy of RDF

mc schraefel and
David Karger
Position Paper for SWUI06

1. Introduction

The most popular visualization of RDF - the underlying language to represent the Semantic Web – is a Great Big Graph (GBG) or Big Fat Graph (BFG) if one prefers. By graph, we mean representations with nodes and edges to model the relationships within the space represented. Take a look at any of a variety of RDF representations: rdf-gravity renders ontologies. Another example of RDF as Graph default is frodo, a UML-like connected flow chart like graph; there’s also RDF Graphs with GSS. One other example of RDF-as-graph is a suite of redeployed classic Node style visualizers that have been modeled and applied anew on rdf (pdf). The above is just a light sampling of RDF-based visualizaitons which focus on the Great Big Graph as the default mechanism for representation. Why are they the default representation? The answer that is usually proferred is that the Semantic Web is a Graph, ipso facto we use graphs to represent it. This notion that data should be presented to the user as it is represented in the computer is what we propose as the pathetic fallacy of RDF. In the following discussion, we will look at some examples of the pathetic fallacy in terms of the interaction challenge of

• what knowledge these graphs communicate,
• what tasks they support, and
• whether these are the optimal paradigms for these representations/tasks.

From this context, we investigate the question, are graphs the main default representation for the Semantic Web and if not, how might we think about formalized representations for the Semantic Web in order to make accessible the promised benefits of the Semantic Web for knowledge building.

2. RDF Graph Exemplars: What are They Doing?

In the following section we consider four exemplars of the rdf bond with Great Big Graph representations, and question what they communicate and how they do so.

2.1 rdf-gravity

RDF Gravity renders ontologies. The exemplar shown is part.
What are the particular attributes of this kind of visualization that help in a given task? what is the task that is being supported, and is it really the best representation for that task? The main attributes of of rdf-gravity are:

• in terms of visualization, placement of nodes that are related (in the model) near each other on the screen, so can easily see “what’s related”.
• in terms of manipulation, easy multiple selection of a set of nodes near each other on the screen, which probably means they are all attached to each other.

When are these set of features useful? How do they differ from existing multiple selection? One answer: the graph is heterogeneous; one may be able to select wineTypes and regions_and_ vintages. Typical UIs only support selection of a group of winesTypes (or regions or vintages, as per Endeca’s demonstrator).

Building heterogeneous groups is extremely powerful: we do it all the time with filesystems, not worrying about filetype, in order to reflect what we need for a certain task, collecting it in a directory if we’re organized. But we do not get to do this for things that are not files, because we cannot refer to them outside their applications. This is one of the major wins of the Semantic Web: a format for reference to anything. And note that building groups requires only this, with no need for all the relation representation.

One might ask, once we pin down several things that suggest the graph is a useful representation, we need to ask, is it really the optimal representation? For instance, one of the obvious assets of such graphs is the ability to see clusters. If discovering clusters is important, however, why not build a tool that clusters and reports on what the clusters are (like
scatter/gather did long ago). If clusters can be reported, is the graph still relevant? Even for ontology viewing, is a graph the best tool for examining it? What user studies support such a claim where such graphs have been placed head to head against other interaction models?

2.2 RDF Graphs with GSS

In the case of RDF with GSS, using the W3C working papers as the data set, GSS starts from the assumption that graphs are a good way to display, then tries to make them better by making them less like graphs, incorporating other kinds of UI. but why the assumption that graphs are a good starting point? In the w3c working papers case, the graphs let you see that the typical model for the drafts is mainly linear but with occasional branching, so trees more generally. Once this model is established, however, what good is the graph visualization? Is there anything it could be used for afterwards? Or should that graph be a starting point to take what has been learned in order to choose an appropriate interaction model for effectively exploring the structure of the working drafts, or alternately might encourage other kinds of desired behaviours? What would such a tool look like?

2.3 Graphs redeployed on RDF

The GViz is used to present a visualization of the schema of the art in a the Rijksmuseum (rather than a visualization of the art itself).

In this work of applying node style graphs to RDF, let us take for example figure 9.5 (shown below).

The following text describes the image:

“In the above picture the edges with the label rdf:type are depicted in blue.
There are two red nodes to which these blue edges connect, one with the label
rdfs:Class and the other with the label rdf:Property, shown near the
nodes as balloon pop-up texts. We chose to depict the property nodes (laid out in a
large circular arc around the upper-left red node) in orange and the class nodes
(laid out in a smaller circle arc around the lower-right red node) in green. As it can
be noticed from the picture there are a lot of orange nodes which is in accordance
with the property-centric approach for defining RDFS schemas. In order to express
richer domain models we extended the RDFS primitives with the cardinality of
properties, the inverse of properties, and a media type system. These extensions are
showed in yellow edges (see also below) and yellow spheres (positioned at the
right end of the image). The yellow edges that connect to orange nodes represent
the inverse of a property. The yellow edges that connect an orange node with the
yellow rectangle labeled “multiple” (positioned at the middle of the figure bottom)
state that this property has cardinality one-to-many. The default cardinality is one-
to-one. Note that there are not many-to-many properties as we had previously
decomposed these properties in two one-to-many properties. The three yellow
spheres represent the media types: String, Integer, and Image. The light gray thin
edges denote the domain and the range of properties. Note that only range edges
can have a media node at one of its ends. “

Is this graph useful? Does it, in other words, tell us anything that the text is not telling us better? Indeed, would the graph make any sense at all, were the text not there to explain it? If that is the case, the graph seems a poor choice for exploring the data in the first place. Was the graph used to discover the things the text is telling us? Rather did those things had to be discovered first, and then the graph layout customized to support the discoveries.

Interestingly, a related group, CWI, in the Netherlands has come up with an interesting Semantic Web explorer for building up knowledge about the same collection. The ap is called “Topia” [pdf]. Not a graph in sight).

3. Aren’t Graphs the Natural Choice?

So what’s wrong with these Big Fat Graphs? After all, RDF is a graph.
That is true, but so is the Web (ps), and we do not commonly see people exploring the Web via it’s bowtie shaped nodes (pdf), do we? After all, “everything” can be represented by a graph, and yet we do not use graphs to represent “everything.” The idea that data should be presented the way it is would lead us to present (bitmap) images to a user as sequences of red-green-blue triples, and file directories as lists of (filename, inode) and (inode, fileinfo) pairs.

Indeed, the fact that the Semantic Web is so clearly a graph might be seen as a bit of a historical accident. The SW has two critical concepts: the naming of individual items, and the expression of relations between them. But who says relations have to be binary? The database community has been working with arbitrary arity relationships for decades, and there are many relations that are naturally high arity, where we have to do contortions with reification just so that they can be represented in rdf. So we might consider a parallel world, where the datbase people looked at Tim Berners-Lee’s web and said “good idea, let’s do it for databases!” They would have needed to recognize the importance of a common format for naming entities; fine. But once they named them, they could announce a standard tuple format for specifying arbitrary arity relations on these various entities.

This world would have pretty much resulted in the same semantic web as we have now. But, the exposure of high-arity tuples from the start might have prevented the growth of all these graph visualizers – and gotten us to a place where we were more focused on what we want to DO with that data as information, rather than on the easy correlation we currently have of rdf=gbg. Which brings us back to the issue of why are such graphs problematic.

So, in part, the Semantic Web community tends towards graph visualizations possibly because there is such an easy correspondence between (a) knowledge of the model structure and (b) knowledge of algorithms to map the data structure. This mapping brings us back to the fundamental question however of what do these representations tell us especially in terms of what we want to know/do with the information presented. Let us ask ourselves what Graphs such as these actually, therefore, do. .

3.1 Graphs. Uh. What are they Good For?

Density and Clusters. Great Big Graphs are known to address two things that are interesting but are limited. They show the Shape and Density of a given data space so that we can say things like “Oh that’s really big” or “there’s a lot of activity going on down there in that part of the graph, but not much up here.” It is not clear, however, when graphs are the best ways to communicate information about densities/clusters, as opposed to having the computer “see” them for us, for instance, and present those findings in ways that are more meaningful and manipulatable.

Focus + Context. Functionally, the classic issue with a graph, even when representing density/clusters, is that to be able to provide such an overview, detail gets lost. Conversely, if we zoom in to focus on a section of a graph, the context gets lost. This focus vs context issue of any graphical representation of information where the goal is to somehow show the whole thing and yet also provide detail is a classic problem in Human Computer Interaction, and the subject of considerable research. See the topic “focus + context” in the ACM digital library, for instance, which covers research in mechanisms to maintain sufficient Context around a focus, and to be able to shift focus easily. The focus of this research is indeed to look at mechanisms to support particular scenarios of use, particular tasks, which may be aided by this kind of particular approach. It is not clear that graphs/rdf are designed with this kind of task-visualization matching in mind. For instance isaviz, one RDF graph-based layout tool, offers rich ways to specify certain kinds of context. For example, which types and properties do I as an RDF editor care about right now.

Usability? The usability of the typical graph approach, however, is problematic in terms of the usablity/usefulness which graph interactions generally allow. We list a few of these below:

• graphs fail to take advantage of many of the cues we are used to for understanding information relationships in applications such as specific layouts (title above author, composer next to date above song title next “play it” button).
• they do not offer good ways to edit the data as graph (cannot drag a node onto another and have something reasonable happen)
• they are flat. All the nodes in view get treated the same way (though this may be changing with latest isaviz) instead of letting the person using the tool have a detailed focus on one object with broader context of “nearby” (from task perspective) objects
• they put next to each other the things that are connected by links. A task, however, may involve dealing with several items far apart in the graph. Then we have to fight with the graph tool to bring/keep them together. Or, we may want to see something that is a complex function of the data (vague). Where will that be shown in the graph visualization?
• they treat everything as a (primary) node, while one might want to think of certain nodes/types/items as primary, and other nodes as “attributes” of the primary nodes (represented by reference within the primary node.
• scale (when there is considerable data to represent) is handled poorly: as soon as what a system is trying to model as a complete entity takes up more than one page/screen, the detail/overview compromise kicks in. In graph tools, this compromise is done the same everywhere, whereas we want varying degrees of detail at different spots at the same time, which may be a fisheye view or some kind of aglomeration of focus+context techniques.

Whether the graphs are UML diagrams, flow charts, clustering graphs, maps of geographical areas or maps of streets or networks, scale forces a compromise between focus and context, but if the interest IS in exposing an entire domain at once graphs and the techniques for balancing focus+context can be extremely effective (consider the micro map of either a 3D gaming environment that shows where one is in a world from an overview perspective relative to one’s first person perspective; or the similar technique used in a Photoshop document where when zoomed in to work on a small section of image a map tool shows where one is working relative to the rest of the image. This map tile outline can also be used to navigate and reposition one’s work area in the image). With these kinds of approaches, we are not compromising; we are having it both ways: some info at detail, and other info at breadth for context, where we as the users decide what we want where.

If we can accept that while graphs may be easy algorithmically to map directly against RDF, but that they are not necessarily the most appropriate representaiton as interface for most things people may wish to do with rdf data, we may want to ask what new things are enabled by the RDF model and how should UIs take advantage of that”?

4. Moving towards the RDF Un-Graph

A fundamental question from the HCI perspective related to any interaction design problem is: what question/task/need is a given representation answering, whether graph or other visualization? To put the question another way, what visualizations/representations/interactions would best support the specified tasks?

To move from the general question to the more Semantic Web oriented question, we can push further to ask what, if anything, is special about what the Semantic Web enables such that existing UI paradigms do not suffice? This question in itself breaks into two distinct questions as they are in some sense orthogonal:

• (i) what old tasks get easier, with the right UIs, if you have the Semantic Web
• (ii) what new tasks become (a) doable or (b) necessary if we have Semantic Web?

We’ve suggested that Big Fat Graphs are not appropriate as a de facto way of presenting the Semantic Web because the tasks it supports are limited. This limitation is not in itself a bad thing. Our point with respect to most graphs of RDF is that there is not a strong match between what a BFG of RDF nodes provides and the kinds of information support people who use the Web have come to expect: try doing email or buying a book with a BFG. There is even some resistance within the RDF community itself towards graph-based ontology editing tools – that they are cool to look at but not great to use. This apparently was one of the motivations behind a non-graph RDF editor, SWOOP.

So, the question, to repurpose Freud somewhat may be not why are GBGs so poor for the Semantic Web, but “what does a (Semantic Web) user want?”

Another way of putting the question of what do we as Semantic Web users want may be: “what are we trying to do?” Ben Shneiderman, HCI guru at the U of Maryland, has more recently been framing the question as “what do you want to know?” Effectively, Shneiderman has said forget trying to show everything since we can never see all of everything at once anyway, and focus on the kinds of things that are of interest to the explorer. Much of Shneiderman’s work with his students, from Spotfire to the more current hierarchical clustering, has indeed focused on enabling researchers to focus on the kinds of questions of interest to them – such as being able to look at the results of a variety of functions when applied to sets of datas – thus being able to see for instance in what conditions are their outliers; in which matches to particular patterns.

The advantage of keeping the question as “what do we want to do” rather than “what do we want to know” may more explicitly capture one particular attribute of the Semantic Web which it has in common with many Web 2.0 applications: the desire to DO something on the Web with the data itself. To tag it; to edit it; to share it; to push it into new and or other representations. These attributes of edit/tag/share are possible with Web2 aps, which break one part of pre Web 2 models, where the web is interactively read only. However, the specific affordances and constraints, to use Don Norman’s terms, of the Semantic Web may take us beyond even these relatively new ways of interacting with information on the Web.

4.1. Difference at Source leading Difference at Interface

One of the interesting features of the Semantic Web that is not harnessed by simple Great Big Graphs of RDF is the fact that it is increasingly possible to break the paradigm of the page (called for in You’ve Got Hyperext) and actually enable people to choose a variety of representations for the information out there, depending again on what they want to do with it. This multiple different views over same data is also technically possible with any database-backed web site, though we cannot find an example that really demonstrates it. As well, the ideology of databases has been to maintain and almost protect the data source; the Semantic Web’s is to publish and share, thus making repurposing and likely therefore just as readily re-presenting the discovered data a raison d’etre of the paradigm.

Likewise, the immediate possibilities of how one set of data might be repurposed with another set of data automatically is also a remarkable and still largely untapped affordance (and possibly the central opportunity)of the Semantic web. This capacity is enabled by that same RDF that wraps up and makes communicatable the semantics of the data in relation to itself and to other data. Just as the schema of a database makes visualizations like Spotfire possible, the RDF of the semantic web will make richer mechanisms for engaging with data possible.

We see some of this page-breaking, cross-web, context sensitive flexible repurposing of data in Semantic Web Applications like Haystack, piggy bank, AKTive Futures and /facet (pronounced “slash facet”), and from Semantic Web/Web 2.0 hybrid applications like mSpace and mSpace mobile

AKTive Futures, for instance, uses a cartesian mapping as one facet of its interface presentation. The core interaction of the UI is to select countries for one axis and ranges of years for the other to look at trends in oil production in those places and times. By clicking on a spot on a line on the graph, the stories that are associated with those confluences are presented in a secondary window. In this case, the use of a particular kind of graph is appropriate for the task the designers of the application wish to support. Date and output data from MULTIPLE resources, (not just one database), are, as numeric data, represented in a numerically relevant fashion – not as static tables but on a graph where, in Shneiderman’s parlance, the person using the service is not presented with all data for all time, but is enabled to select the ranges of interest and focus on them with an appropriate format.

For this site, data is coming from all over the Web and converted where it doesn’t already exist in SW format into SW format (ie rdf most usually) so that it can be rendered appropriately for this kind of explorable user interface (UI). Indeed, the graph is used to help find trends of interest (not unlike Spotfire) and to use those relations of interest as the way to find the richly associated information to tease out what may have caused that particular moment.Likewise, the fact that data is sourced from many locations suggests it will be important for interfaces to give cues about sources (provenance) of diffent pieces of data in view.

4.2. Pre-existing Sites with a Purpose – Predefined Semantic Web UI aps

The above sites are examples of what happens when a site with A Purpose already exists. Haystack’s exemplar is the “universal information client” which integrates calendar information with other associated tasks like hotel and flight booking along with finding relevant and related email to support tasks in process. In this case, Haystack is showing the way of using the Semantic Web to do old tasks better, using familiar UI paradigms in new contexts to make it easier to do related tasks that typically draw down on information from a variety of applications: checking email for when a conference is in order to get the dates into the calendar and check out flights for those times.

mSpace is making classical music discoverable for people who know nothing about classical music. This discoverability is enabled by adding Preview Cues, or the ability to check out not just a piece, but the sound of an area of music, like sonatas or baroque, quickly and easily. This feature in itself is not driven by the semantic web, but it is powerfully supported by it. For instance, there are other affordances that the interface provides that go beyond online music explorers and into what makes the Semantic Web interesting: the browser automatically associates information from different sources about the music in the explorer with the music – choosing “period: baroque” yields a description of that content. This ap is another case of taking familiar and largely effective models for music library exploration and play back, and enhancing them to enable either improvement of previously doable but difficult or cumbersome tasks, and making it tractable to draw together new and related resources as they become available. For instance, with mSpace, new dimensions can be added to the domain as they become known; musicological data may be supplemented with technical recording data or historical data. Currently, this domain growth happens on the system side, discovering new information; work is under way to make new dimensions discoverable/addable by people exploring data resources themselves. The UI makes it possible (to use spreadsheet language) to pivot from one domain to another on a related term – so one moves from beethoven in the context of music to beethoven in the context of history. Sure yes one can do these pivots with databases and spreadsheets. Indeed, George Roberston’s Polyarchy work called “Visual Pivot” (pdf) in fact has shown exactly such pivoting in very interesting ways from one database table to another. One may suggest, however, that the Semantic Web has the potential to break from database scale to greater, messier, heterogeneous Web scale.

5. Dynamic, Free Form Semantic Web UI aps

The opportunity which both Haystack and mSpace demonstrate is that the approaches can unify any and all data into the user interface. Thus, they can bring together all and only the information that a user needs to tackle a particular task, without worrying about which application owns it or how it will be represented. The UI challenge within these kinds of applications is that they can unify any and all data into the UI. In other words, application/interaction designers may have no idea what data is going to show up and be necessary for the users’ tasks, so there is a challenge to determin how to present data that is not known in advance and therefore cannot be planned for. Haystack and mSpace, each in slightly different ways, are trying to exploit the opportunity/meet the challenge by creating a UI framework that can accept/display arbitrary data, thus offering all the benefits of task-driven data aggregation, while still looking and acting like a familiar application, rather than an exposure of the raw source data. For instance, suppose there is an interest in finding Jazz music and there is no pre-made mSpace Jazz explorer? or more intrigued yet, someone is interested in not only exploring the sounds of jazz but of seeing what is happening historically both politically and in architecture at the same time as different trends in music are occurring in order to explore the question what was influencing what when?

The above kind of questions means that a person may wish to be able to start exploring from a particular informaiton seed or set of seeds from which to start building and exploring relations (though even how to express these seeds may be challenging – another matter for interaction research innovation (ever know what you want but not the terms to express it so that you can find it on google?)). The above mix query means that samples of music need to be available so someone can audition the songs (we do not assume the Questor is a jazz expert) to hear what is of interest; engage historical political period data from different regions; enable this data to be contextualized not only by location but by time, and readily explorable by time and by location visualizations. What’s the ideal representation for this information as it is assembled? It is NOT a Great Big Graph (alone or primarily).

Web Founder and Semantic Web co-Founder Tim Berners-Lee has been developing an idea called the Tabulator (which one author has yet to be able to get to work). Conceptually, one starts with a specific known source of semantic web data, and then rather than in a graph, one selects cells in a tabular representation of the related RDF which expands into fresh tables, etc. The data collected in these expansions can then be re-visioned into either a map, a calendar or a timeline (note the term “or”). There is considerable potential here – currently selecting the source of the data is very geeky: data is expressed in rdf-ease triples like “colorPicture is mentioned in TAGmobile road trip BOS-> Amerst:photo” Qu’est-ce que se?

The tabulator also seems currently to be informed by the Web-as-Read-Only paradigm, where as the impetus of Web 2.0 (and the Semantic Web) is towards read/write/re-write – a very much more Ted Nelson-ish hypertext vision ( a good thing) than pre Web2 vision.

5.1 Mix and Match on the Fly

Some of the challenges for Semantic Web UI services besides de-geeking things like Tabulator will be to support data in formats so that the application has information that is relevant to what display options may be appropriate for it (dates, map coordinates, contacts). It is not clear what the solution is: micro formats is one approach; fresnel, defined as “a generic ontology for describing how to render RDF in a human-friendly manner,” where the style sheet for a data chunk effectively travels with that data, offers another. It will be interesting to see how these approaches work across heterogeneous data sources and distinct contexts. It will also mean being able to add new data/links/tags.

That observation of the context in which the data is discovered leads back to the earlier observation that UI’s for semantic web data, like all other human-usable systems, need to respect and support what the human wants to do with that data. Applications or higher level frameworks which start with this focus in mind have a greater likelihood of producing effective interactions. Being able to establish context for multiple intersecting data domains and data types may be as critical as being able to take advantage of a pre-asserted format for a particular data chunk. For example, Consider the Web 2.0 ap,
Live Clipboard that enables sharing of Web content between Web pages (technical information on how it does that). With Semantic Web, we may want to be able to pull together a variety of sources of data and data formats together for concurrent representation in appropriate models – data copied from various sources pasted into new representations to be used in new ways. How would a Semantic Web version of something like Live Clipboard work to deliver a dynamic version of AKTive futures?

6. Conclusions/Looking Forward

The bottom line is that Great Big Graphs have their place, but overall, it’s a fairly limited place. Great Big Graphs are generally also pretty easy. The algorithms for pumping data into many graphs are well known. As we hope we have begun to demonstrate, it is a pathetic fallacy to assert that because the data model is a graph the data should therefore be displayed as a graph. Using graphs by default, in usability terms, begs the question, whose interest is being served by providing graphs? What activity is actually being supported by such representations? As we have shown, the answers suggest that graphs have limited value, even for many of the tasks which they are supposed to support. The harder question coming from this interogation is “how do we elegantly support the range of possible interactions both in pre-defined Semantic Web applications and in dynamic explorations of Semantic Web resources?” We have only sketched out some examples of current SW applications to support old tasks better and in new ways enabled by the Semantic Web, and to explore more dynamically SW-RDF resources for user-determined exploration. As is evident, much more innovative work is possible and needs to be done.

People at the coal face of RDF and Ontology work mayn’t see it as their mission to consider that more human-oriented approach to representing information spaces for human usable, human-useful exploration. But why not? The result may well be the generation of a generic Semantic Web browser – a tool that would enable people both to explore and contribute to the rich associations possible in the ((increasingly Social and) Semantic) Web.