NFDI4Culture Data Story

Experiments in Data Alchemy

Or: A not-so-serious data integration story involving NFDI4Chem and NFDI4Culture

Abstract: This data story explores unconventional approaches to linking research data across two NFDI consortia: NFDI4Chem and NFDI4Culture. By experimenting with alchemical metaphors, we test how chemical and cultural datasets can be connected through shared standards, authority data, and semantic technologies. Three small “experiments” illustrate different modes of integration: visual (ICONCLASS and chemical structures), spatial (3D annotation tools applied to molecules), and auditory (mapping musical notes to chemical elements). Although the exercises are intentionally playful rather than strictly scientific, they reveal genuine opportunities for cross-domain data interoperability. Such connections may inspire novel applications in fields ranging from digital humanities to cultural heritage conservation, where material and chemical processes are integral to understanding and preserving artefacts.

Who is the Datafrog in Mainz ;-)

Introduction

In recent years, the National Research Data Infrastructure (NFDI) has advanced domain-specific strategies for data standardisation, interoperability, and open access. Within this broader framework, NFDI4Chem focuses on research data in chemistry, while NFDI4Culture addresses research data in the arts, humanities, and cultural heritage. At first glance, the two consortia inhabit very different epistemic worlds: chemical compounds on the one side, tangible and intangible cultural assets on the other.

Yet, the increasing use of semantic web technologies and authority data provides a shared basis for cross-domain integration. By applying the concept of “data alchemy”, this story stages a series of playful experiments that probe how cultural and chemical research data might be connected. These explorations are not intended as comprehensive solutions but as creative ideas: sketches of possible interoperability that highlight both the opportunities and the challenges of combining datasets across disciplinary boundaries.

Research questions

The use cases presented in this data story were guided by the following questions:

1. Connectivity: How can semantic identifiers and authority data enable links between chemical and cultural research datasets?
2. Methodological transfer: To what extent can tools developed in one domain (e.g. 3D annotation software from NFDI4Culture) be applied to data in another (e.g. molecular models from NFDI4Chem)?
3. Multimodal representation: What new perspectives emerge when cultural and chemical data are combined in visual, spatial, or auditory modes?

Beyond playful experimentation, what potential applications could such integrations support — for instance, in the conservation of cultural heritage, where chemical processes are central?

First Experiment: VISION

We begin our first experiment with an object from the NFDI4Chem search service: a molecule that could hardly be more “alchemical” — it contains gold.

What do we see? The molecule is represented by its InChI identifier. This enables precise and standardised identification, sharing, and linking across databases — the foundation for interoperability in chemistry.

Next, we turn to the world of art history: an etching by Pieter Bruegel the Elder showing an alchemist at work.

What do we see? At first glance, merely pixels — but in the context of art historical data, such images are semantically enriched using the ICONCLASS classification system.

Just as InChI provides a canonical identifier for molecules, ICONCLASS codes provide canonical identifiers for iconographic motifs. For Bruegel’s work, we can, for example, assign:

• 49E393 “alchemistic equipment”
• 49E391 “alchemist at work”
• 42B742, 43C786, 31B8, 49C3 (further motifs present in the scene)

Thus, the image data becomes a semantically described scene.

The parallel between InChI in chemistry and ICONCLASS in art history suggests a basis for integration. Both consortia, NFDI4Chem and NFDI4Culture, make their data available via SPARQL endpoints, using shared vocabularies. For the cultural domain, the Culture Knowledge Graph relies on NFDIcore and the Culture Top-Level Ontology (CTO) to describe resources and their semantic context.

With this infrastructure, we can query for cultural heritage resources that reference chemistry or alchemy by their ICONCLASS codes. The following query retrieves resources connected to a predefined set of chemistry-related ICONCLASS concepts:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

PREFIX rdfs:        <http://www.w3.org/2000/01/rdf-schema#>
PREFIX nfdicore:    <https://nfdi.fiz-karlsruhe.de/ontology/>
PREFIX cto:         <https://nfdi4culture.de/ontology/>
PREFIX iconclass:   <https://iconclass.org/>

SELECT ?code ?iconLabel ?resource (SAMPLE(?lbl) AS ?resourceLabel)
WHERE {
  VALUES (?code ?iconLabel) {
    ("49E32"   "inorganic chemistry")
    ("49E33"   "organic chemistry")
    ("49E3"    "chemistry")
    ("49E39"   "alchemy")
    ("49E31"   "chemical symbols")
    ("49E393"  "alchemistic equipment")
    ("49E394"  "alchemistic substances")
    ("49E395"  "alchemistic processes")
    ("49E392"  "alchemistic symbols")
    ("49E3949" "other substances")
    ("49E3931" "alchemistic vessels")
    ("49E3932" "alchemistic furnace")
    ("49E3939" "other alchemistic equipment")
    ("49E3941" "Sulphur ~ alchemistic substance")
    ("49E3942" "Mercury ~ alchemistic substance")
    ("49E3943" "Sal ~ alchemistic substance")
    ("49E3946" "Elixir Vitae")
    ("49E3953" "multiple processes combined")
    ("49E3933" "alchemistic bath (balneum)")
    ("49E3944" "Prima Materia")
    ("49E391"  "alchemist at work")
    ("49E39311" "bottles (ampullae)")
    ("49E39312" "flasks (cucurbitae)")
    ("49E39421" "Philosophical Mercury ~ alchemistic substance")
    ("49E3945"  "Philosopher's Stone (Lapis philosophorum)")
    ("49E39461" "Panacea ~ alchemistic substance")
    ("49E3954"  "interaction or reaction of alchemistic substances")
    ("49E39313" "pots, jars (ollae)")
    ("49E3952"  "Magnum opus: additional stages of transmutation")
    ("49E39451" "Quintessence")
    ("49E3951"  "Magnum opus: the four original stages of transmutation")
    ("49E3913"  "alchemist failing at work")
    ("49E39452" "the White Stone")
    ("49E39453" "the Red Stone")
    ("49E39542" "evaporation of solvent")
    ("49E394211" "Bird of Hermes")
    ("49E39543"  "interaction between (pairs of) opposites")
    ("49E3911"   "alchemist trying to make gold")
    ("49E39511"  "nigredo (blackening)")
    ("49E39512"  "albedo (whitening)")
    ("49E39513"  "citrinitas (yellowing)")
    ("49E39514"  "rubedo (reddening)")
    ("49E39541"  "exchange of chemical properties (e.g. wet for dry)")
    ("49E3912"   "alchemist looking for the Philosopher's Stone")
  }

  BIND( IRI(CONCAT(STR(iconclass:), ?code)) AS ?concept )
  ?resource cto:CTO_0001026 ?concept .
  OPTIONAL { ?resource rdfs:label ?lbl }
}
GROUP BY ?code ?iconLabel ?resource
ORDER BY ?code ?resource

The query returns a wide range of resources from image databases and digital collections. A few examples include:

• “Darstellung einer fraktionierten Destillation zur Trennung von Flüssigkeitsgemischen”, http://www.bildindex.de/document/obj20344132
• “Alchemistische Experimente” (Bibliotheca Hertziana, Fotoarchiv), https://foto.biblhertz.it/media/obj/08003830/bh_gern_143965_post
• “COME L’ORO NEL FOCO” (Ceiling painting, motif: alchemist trying to make gold), https://www.deckenmalerei.eu/31187757-ec65-4fcb-9e9d-c2e7735c2643

Through this experiment, we demonstrate how standardised semantic identifiers (InChI, ICONCLASS) allow us to connect two seemingly disparate domains: molecular structures and iconographic scenes. What unites them is not their medium (chemical diagram vs. copperplate engraving) but the fact that both can be semantically described and queried as data. This creates the conditions for linked data applications that move fluidly across disciplinary boundaries — a core ambition of NFDI.

Second Experiment: SHAPE

In this experiment we turn from the vision of alchemy to its shapes. Instead of images and iconography, we now consider the three-dimensional structures of molecules and cultural artefacts.

3D molecular information in chemistry

Molecular geometry is commonly stored and exchanged in the SDF (Structure Data File) format. SDF is a widely used standard for encoding 3D atomic coordinates, bonds, and conformers of molecules. Many chemistry repositories, such as PubChem, provide SDF files directly through their APIs.

For our running example — the molecule from Experiment 1 containing gold — we can retrieve its SDF data from PubChem:

146026886
  -OEChem-09192503073D

 73 79  0     0  0  0  0  0  0999 V2000
    3.9980   -0.0810   -0.7400 P   0  0  0  0  0  0  0  0  0  0  0  0
   -7.6306   -1.4194    0.3245 N   0  0  0  0  0  0  0  0  0  0  0  0
    1.6192   -2.3375   -0.7629 C   0  0  0  0  0  0  0  0  0  0  0  0
    1.5401   -1.1949    0.2390 C   0  0  0  0  0  0  0  0  0  0  0  0
   -1.2639    2.5225   -0.1927 C   0  0  0  0  0  0  0  0  0  0  0  0
    1.1742   -1.9577   -2.2251 C   0  0  0  0  0  0  0  0  0  0  0  0
   -0.1574    2.5755    0.9268 C   0  0  0  0  0  0  0  0  0  0  0  0
   -1.0091    1.4948   -1.2856 C   0  0  0  0  0  0  0  0  0  0  0  0
    0.6284    1.2840    1.1007 C   0  0  0  0  0  0  0  0  0  0  0  0
    2.3638   -0.0745    0.0905 C   0  0  0  0  0  0  0  0  0  0  0  0
    0.2128   -0.7812   -2.3081 C   0  0  0  0  0  0  0  0  0  0  0  0
   -1.6802    0.2618   -1.2396 C   0  0  0  0  0  0  0  0  0  0  0  0
    1.9032    1.1716    0.5217 C   0  0  0  0  0  0  0  0  0  0  0  0

[...]

Open this link to get the full SDF data.

This data encodes the molecule’s 3D conformation: atom types, bond orders, and spatial coordinates. It is the chemical equivalent of a digital 3D model.

GLB, Semantic Kompakkt and annotation practices

In the cultural heritage domain, however, a different 3D standard is dominant: GLB, the binary form of the glTF (GL Transmission Format). GLB has become the standard for delivering 3D content on the web, supported by lightweight viewers and widely adopted in European digital heritage projects. GLB enables efficient rendering, exchange, and annotation of sculptures, artefacts, and architectural fragments.

Within NFDI4Culture, Semantic Kompakkt plays a central role in the management and use of 3D cultural heritage data. Semantic Kompakkt is a web-based framework for hosting, rendering, and linking 3D models and integrates with knowledge graphs and metadata repositories, enabling scholarly workflows. Crucially, it allows annotations directly on the 3D surface. Scholars in the arts and humanities use this to mark features of sculptures, architectural details, or artefacts.

In chemistry, annotation of molecules in 3D is far less common. Yet the same affordance applies: atoms, bonds, or functional groups can be semantically marked up, linked, and shared. This is where the synergy between NFDI4Chem and NFDI4Culture becomes clear.

From SDF to GLB: a pipeline for synergy

What if there were an easy way to transform SDF molecular data into GLB files, so that molecules could be loaded into Semantic Kompakkt just like cultural heritage objects?

To test this idea, we developed a small script that converts PubChem SDF data into GLB:

Using this tool, our gold-containing test molecule was successfully transformed and uploaded into Semantic Kompakkt:

Towards sound: Sonification of 3D molecules

The exploration of 3D molecular data does not need to remain purely visual. An inspiring example is provided by the Intangible Realities Laboratory with their project “Sonification of Molecules”. In this work, 3D atomic coordinates are algorithmically transformed into musical structures, allowing molecules to be experienced not only through sight but also through sound.

Such approaches illustrate the aesthetic and epistemic potential of sonification: turning molecular geometries into auditory forms can both open new modes of perception and provide novel ways of analysing complex spatial patterns.

Having demonstrated that SDF data from NFDI4Chem can be transformed into GLB and displayed in Semantic Kompakkt within NFDI4Culture, we can envision a similar pipeline for sound. Once molecules are available in interoperable 3D formats, they could be processed by sonification algorithms comparable to those used in the Intangible Realities Laboratory. In this way, annotation and sonification could complement each other: annotated features in 3D space could be mapped to sonic events, rhythms, or tonal structures.

Our second experiment demonstrates how 3D workflows in chemistry and cultural heritage can converge:

• Chemistry provides 3D conformer data in SDF.
• Cultural heritage provides 3D viewing and annotation infrastructure via GLB and Semantic Kompakkt.
• By bridging the formats, annotation practices can be transferred: what is a sculptural detail in art history can become an atomic site in chemistry.

Just as alchemists once sought to reveal the hidden essence of matter, we can now annotate and share the gold atom within our digital molecule.

Third Experiment: SOUND

The third experiment turns from vision and shape to sound. Its inspiration lies in Atom Tones, a project developed by Jill Linz. Atom Tones explores the idea that every element in the periodic table can be assigned a distinctive auditory identity, derived from its atomic emission spectrum. Instead of treating the periodic table as an abstract schema of numbers and letters, Linz transforms the physical properties of atoms into tones that can be played and combined. In this way, data about atomic structure becomes music, and chemistry is made audible through sonification. This concept of giving each element a characteristic “voice” based on spectral information illustrates how scientific data can be translated into an entirely different sensory modality, opening up unexpected perspectives.

Building on this idea, we asked: what if the periodic table could be played like a piano keyboard? Each white or black key would be mapped to one or more elements (e.g. C → Carbon, G → Gallium/Germanium/Gadolinium, A → Gold). Pressing a key would trigger and a federated SPARQL query across NFDI4Culture and NFDI4Chem.

The result of this idea is the Culture Chemical Composer a small web application that takes the metaphor of the keyboard literally, but uses it as an input device for linked data rather than as a musical instrument. On the screen, a piano keyboard is placed next to a periodic table. By clicking on the keys, the user generates a sequence of note letters, which are at the same time mapped to corresponding elements in the periodic table. The notes typed in are displayed as a running string, and can be corrected by deleting the last input. The periodic table highlights which elements are active, so that the user sees immediately which part of chemistry is being called into play when a particular musical note is pressed.

With each interaction, the application formulates a federated SPARQL query that reaches out simultaneously to the NFDI4Culture and NFDI4Chem endpoints. On the cultural side, it searches the Culture Knowledge Graph for notated openings that contain the selected note pattern.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

PREFIX schema: <http://schema.org/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX cto: <https://nfdi4culture.de/ontology/>

SELECT ?title ?pattern ?score {
  ?incipit a cto:CTO_0001024 .
  ?incipit schema:associatedMedia/cto:CTO_0001021 ?score .
  ?incipit cto:CTO_0001063 ?pattern . FILTER regex(?pattern, "A") 
  ?source cto:CTO_0001065 ?incipit .
  ?source rdfs:label ?title .
} 
LIMIT 100

On the chemical side, it queries the NFDI4Chem Knowledge Graph for entries whose InChIs contain the element symbols mapped from the notes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

PREFIX schema: <http://schema.org/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX nfdicore: <https://nfdi.fiz-karlsruhe.de/ontology/>
PREFIX cto: <https://nfdi4culture.de/ontology#>
PREFIX dcat: <http://www.w3.org/ns/dcat#>
PREFIX dct:  <http://purl.org/dc/terms/>

SELECT
    ?dataset ?title ?description ?keyword
WHERE {
  SERVICE <https://service.tib.eu/lablsk-sparql/fuseki/NFDI4Chem/query> {
    SELECT ?dataset ?title ?description ?keyword
    WHERE {
      ?dataset a dcat:Dataset ;
               dct:description ?description ;
               dct:title ?title ;
               dcat:keyword ?keyword .
      FILTER (
        REGEX(STR(?keyword), "\\bAu\\.", "i")
      )
    }
  } 
}

The results of this combined query are then displayed together on the page: musical incipits with their titles, score renderings on one side, and chemical datasets with their titles, descriptions and keywords on the other. The interface thus stages a juxtaposition of two data worlds that would normally never be seen together.

Try it out: https://nfdi4culture.de/chemical-composer.html

An important point to note is that the Chemical Composer does not generate or play sounds. It is not a synthesiser, but an experiment in data integration, making the links between chemistry and culture visible by placing them in a single search interaction. The playful effect lies in the fact that the simple gesture of pressing a piano key brings back both a piece of music and a molecule, enacting in miniature what it means to query across the boundaries of two NFDI consortia.

Summary

The experiments presented in this data story were intentionally playful, but they nonetheless demonstrate substantive points about data integration across domains. In the first experiment (VISION), we showed that cultural artefacts and chemical structures can both be described with semantic identifiers. InChI in chemistry and ICONCLASS in art history function as parallel authority data systems, making it possible to connect molecular structures with iconographic scenes of alchemy. What at first appears to be only an analogy between identifiers becomes a concrete basis for federated queries that retrieve data across the two infrastructures.

The second experiment (SHAPE) extended this perspective from the visual to the spatial. By converting SDF molecular data into GLB and importing it into Semantic Kompakkt, we demonstrated that workflows originally developed for cultural heritage 3D objects can also be applied to molecules. The ability to annotate surfaces, long familiar in the arts and humanities, opens new opportunities for chemistry, where semantic annotation of 3D structures is not yet common practice. The example of the Intangible Realities Laboratory further suggested how sonification could complement annotation, allowing molecular geometries to be experienced through sound as well as vision.

The third experiment (SOUND) took this idea further by staging a multimodal interaction in which the periodic table becomes a keyboard. The Culture Chemical Composer translates keystrokes into federated SPARQL queries that simultaneously search for incipits in NFDI4Culture and for molecular datasets in NFDI4Chem. The results are displayed side by side, showing in real time how a single interaction can traverse disciplinary boundaries. The application does not produce audio, but its playful interface makes visible the links that can be forged across consortia once common standards and query mechanisms are in place.

Taken together, the experiments underline three important insights. First, interoperability depends on shared semantics: without identifiers such as InChI and ICONCLASS, no meaningful integration would be possible. Second, methodological transfer across domains is a powerful way to test interoperability: annotation tools, 3D pipelines, and interactive front ends can be reused in novel contexts. Third, creative experiments are not just whimsical; they can reveal genuine integration scenarios with relevance for fields such as cultural heritage conservation, where chemical processes and cultural data are closely intertwined. What we called “data alchemy” is therefore more than a metaphor: it illustrates the promise of linked infrastructures to connect domains of knowledge that, until now, have rarely been seen together.