Open Data – ross spencer :: exponentialdecay.digipres

Image of the foundations of a new building being erected in Wellington New Zealand, circa 2017.

File format building blocks: primitives in digital preservation

A primitive in software development can be described as:

a fundamental data type or code that can be used to build more complex software programs or interfaces.

– via https://www.capterra.com/glossary/primitive/ (also Wiki: language primitives)

Like bricks and mortar in the building industry, or oil and acrylic for a painter, a primitive helps a software developer to create increasingly more complex software, from your shell scripts, to entire digital preservation systems.

Primitives also help us to create file formats, as we’ve seen with the Eyeglass example I have presented previously, the file format is at its most fundamental level a representation of a data structure as a binary stream, that can be read out of the data structure onto disk, and likewise from disk to a data structure from code.

For the file format developer we have at our disposal all of the primitives that the software developer has, and like them, we also have “file formats” (as we tend to understand them in digital preservation terms) that serve as our primitives as well.

"Bei der Buche", a landscape architectural installation by landscape architect and photographer Karina Raeck. Created in 1993 in the Wartberg area north-east of Stuttgart.

wikidata + mediawiki = wikidata + provenance == wikiprov

Today I want to showcase a Wikidata proof of concept that I developed as part of my work integrating Siegfried and Wikidata.

That work is wikiprov a utility to augment Wikidata results in JSON with the Wikidata revision history.

For siegfried it means that we can showcase the source of the results being returned by an identification without having to go directly back to Wikidata, this might mean more exposure for individuals contributing to Wikidata. We also provide access to a standard permalink where records contributing to a format identification are fixed at their last edit. Because Wikidata is more mutable than a resource like PRONOM this gives us the best chance of understanding differences in results if we are comparing siegfried+Wikidata results side-by-side.

I am interested to hear your thoughts on the results of the work. Lets go into more detail below.

René Magritte's The Lovers, Paris 1928 (Photographed at MoMA, NYC in 2017

Unrealized ideas: Unintentional Secrecy in the Era of Openness

Tyler recently posted this quote:

“History unprocessed is opportunity unrealized”

It reminds me of an unrealized article I wasn’t able to get written and into the wild, but it’s an important thought I would like to share nonetheless.

Proposed for James Lowry’s ACARM Symposium in 2015, I wanted to discuss when government is unable to adequately fund day-to-day effort, and research and development in the archive sector, leading to inefficient and potentially ineffective processing pipelines for records of archival value accessioned from government agencies and commissions.

It was just an abstract, but maybe folks have thoughts about this? Have we moved on since the early to mid 2010’s? What modern metrics do we have available to us today to see the progress? What does the advent of the new US administration mean for issues like this? As well as increasing worldwide authoritarianism?

Making DROID work with Wikidata

Wikidata is a good service, Wikibase (on which Wikidata is built) is a better platform.

I have spoken before about its potential to be added into the file-format registry ecosystem in a federated model.

If we are to use it as a registry that can perhaps complement the pipelines going into PRONOM, e.g. in vendor’s digital preservation platforms such as the Rosetta Format Library, a Wikidata should be able to output different serializations of signature file for tools such as Siegfried, DROID or FIDO.