Open Source Archives - ross spencer :: exponentialdecay.digipres

Image from July 2021 depicting the fields around Ravensburg in Southern Germany. There is a sign with some graffiti on which depicts a car sliding off the road, presumably because there is very little curb and it's likely you will career into the grass if you're not careful.

When you can’t pay for things the currency of payment is psychic…

Contributing back to the commons in digital preservation hasn’t been for everyone.

We know the famous XKCD that touches on the underappreciated work of maintainers in obscurity. When you, or your institutions, or services are using free and open source software, or other information and data in the commons, and you’re not contributing back, you’re perpetuating this, and what’s more, there’s a virtuous cycle that we’re missing out on.

I read something the other day and it felt like a red flag.

Image of the foundations of a new building being erected in Wellington New Zealand, circa 2017.

File format building blocks: primitives in digital preservation

A primitive in software development can be described as:

a fundamental data type or code that can be used to build more complex software programs or interfaces.

– via https://www.capterra.com/glossary/primitive/ (also Wiki: language primitives)

Like bricks and mortar in the building industry, or oil and acrylic for a painter, a primitive helps a software developer to create increasingly more complex software, from your shell scripts, to entire digital preservation systems.

Primitives also help us to create file formats, as we’ve seen with the Eyeglass example I have presented previously, the file format is at its most fundamental level a representation of a data structure as a binary stream, that can be read out of the data structure onto disk, and likewise from disk to a data structure from code.

For the file format developer we have at our disposal all of the primitives that the software developer has, and like them, we also have “file formats” (as we tend to understand them in digital preservation terms) that serve as our primitives as well.

"Bei der Buche", a landscape architectural installation by landscape architect and photographer Karina Raeck. Created in 1993 in the Wartberg area north-east of Stuttgart.

wikidata + mediawiki = wikidata + provenance == wikiprov

Today I want to showcase a Wikidata proof of concept that I developed as part of my work integrating Siegfried and Wikidata.

That work is wikiprov a utility to augment Wikidata results in JSON with the Wikidata revision history.

For siegfried it means that we can showcase the source of the results being returned by an identification without having to go directly back to Wikidata, this might mean more exposure for individuals contributing to Wikidata. We also provide access to a standard permalink where records contributing to a format identification are fixed at their last edit. Because Wikidata is more mutable than a resource like PRONOM this gives us the best chance of understanding differences in results if we are comparing siegfried+Wikidata results side-by-side.

I am interested to hear your thoughts on the results of the work. Lets go into more detail below.

René Magritte's The Lovers, Paris 1928 (Photographed at MoMA, NYC in 2017

Unrealized ideas: Unintentional Secrecy in the Era of Openness

Tyler recently posted this quote:

“History unprocessed is opportunity unrealized”

It reminds me of an unrealized article I wasn’t able to get written and into the wild, but it’s an important thought I would like to share nonetheless.

Proposed for James Lowry’s ACARM Symposium in 2015, I wanted to discuss when government is unable to adequately fund day-to-day effort, and research and development in the archive sector, leading to inefficient and potentially ineffective processing pipelines for records of archival value accessioned from government agencies and commissions.

It was just an abstract, but maybe folks have thoughts about this? Have we moved on since the early to mid 2010’s? What modern metrics do we have available to us today to see the progress? What does the advent of the new US administration mean for issues like this? As well as increasing worldwide authoritarianism?

A tree at sunset photographed from the train on the Bodendsee in Southern Germany

Versioning as memory?

So, it turns out my theme of the moment is code hygiene (or maybe memory?).

Today I am thinking about versioning, especially in relation to its impact on digital preservation; both software preservation and the impact of versions on long-term preservation efforts in other contexts.

Silver birth tree in Ravensburg featuring its characteristic eye-like bark.

Code as memory?

It is very poetic to think about code as it containing the memory of its maintainers. I don’t entirely disagree with the idea, but it’s overly poetic and the reality of maintenance on systems that have become too unwieldy is anything but poetic.

Demystify-lite is upgraded and demystify 2.0.0 is finally here!

Demystify-lite and Demystify 2.0.0 have been released featuring the denylist for the first time, plus some bonus features! With thanks…

Stop, Look, Listen, retro game style advertising for safety at a Houston Bus Stop

Linting as understanding

I have been working on a Python template repository as part of my day-job at Orcfax.

It is based on the popular pypa sample project and adds important tooling that supports the quality assurance of projects that many developers are expected to engage with.

In my template repository I add editor defaults, linting, and prepare the repository for unit tests, and then deployment.

I have migrated a copy of the template I created for Orcfax to a new file format organisation I have created to capture work I am doing around tools such as ffdev.info (the PRONOM signature development utility).

The new template repository can be found here: ffdev-info/template.py.

I want to talk about how this tooling can be used as a way of understanding legacy, or new code that you are going to be looking at. Looking at how linting can be useful for learning and understanding.

Using a custom Wikibase with Siegfried

In March I was invited by the LD4 Wikidata Affinity Group to talk about my experiences using Wikibase with Siegfried, the file format identification tool. I don’t think I’ve talked about that work on here before but you can find links to my iPRES talk on my ORCID page.

Let’s look at the abstract and the content of the talk below.

Information Maintainers talk: Something something twenty years open source…

It was back in May, yes, way back when, that Jordan Hale of the Information Maintainers group put the following to me:

I write today to ask if you’d be interested in being our special guest on the next Information Maintainers call … we thought your perspective on working within and maintaining decentralized, small-group systems and development infrastructures would be really rad to hear about. What do you think?

I am a big fan of the Information Maintainers and so I was pretty stoked to be asked. Of course, I jumped at the chance and wrote about “Something something twenty years open source…”

Tag: Open Source

When you can’t pay for things the currency of payment is psychic…

File format building blocks: primitives in digital preservation

wikidata + mediawiki = wikidata + provenance == wikiprov

Unrealized ideas: Unintentional Secrecy in the Era of Openness

Versioning as memory?

Code as memory?

Demystify-lite is upgraded and demystify 2.0.0 is finally here!

Linting as understanding

Using a custom Wikibase with Siegfried

Information Maintainers talk: Something something twenty years open source…

Follow ross spencer :: exponentialdecay.digipres :: blog