Shortened links are a digital preservation and web archiving nightmare. You can imagine how they need to work:
Create a unique short code for a given (target) URL (like a hash, but far far shorter)
Pair the short code with your URL in a database.
Create a redirect rule on the URL shortening server from the new source URL to the target URL.
Send the shortened link to the caller, e.g. shortURL.com/123badf00d
In-perpetuity: continue to pay for your domain; maintain the database; look after redirect rules during server migrations; ensure duplicate short-codes are not created.
Last year I wrote about pitfalls in modern journalism, especially with regards to receiving documents and information from whistleblowers without offering them adequate protection.
The tl;dr is that you, as a whistleblower, need to protect yourself; and you, as an editor or journalist, need to protect your whistleblowers.
Steganographic fingerprints might be one method adopted to detect someone leaking information. Steganographic characters replace common textual characters with unusual but hard to detect variants, e.g. they look the same to the human eye, or are actually invisible. Using a tool called SafeText by David Jacobson we can identify these hidden fingerprints in the content that you share.
I firmly believe we can find clues about what is important to preserve, or learn to preserve, when we analyse the content of the digital record and not just the (file) format of the digital record.
A file can contain many different features and these are all challenges to their future interpretation, and thus preservation.
I wanted to use SafeText in some of my other non-Python tooling and so I decided to port the code to Golang as a composable module and binary.
By coincidence at the time I started writing this I had also just written about revisiting tikalinkextract and so I thought I would write this small explanation about how you might combine Tika and SafeText to perform some content analysis of your own.
Who knows, maybe we will find a conspiracy. Maybe we’ll find secret codes in our own digital records. Maybe we’ll learn something new about our records…
Lets have a look at putting Tika and SafeText together and see where it goes.
For a while back then I was into space flight again. Scientists, science communicators, and engineers were all excited for a new era of rocket launches and the potential unification of the human race as we look towards the future.
During that time I discovered Colin Fries’ work in the NASA History Division to document all NASA “Wake-up calls”. A wake-up call is simply a piece of music used to wake astronauts on missions, a different piece of music, daily, for the duration of the flight.
Take, for example, the last Space Shuttle mission (Space Transportation System) STS-135; it was in flight for 13 days, and the wake-up call on day one was Coldplay’s Viva la Vida, while on day 13 it was Kate Smith singing God Bless America.
As a huge music buff who has the radio or music television on 18 hours a day, I really wanted to delve into this further. While Colin’s work is great, it’s just a PDF file (@wtfpdf). A PDF is not an ideal file format for querying data and gleaning new insights. So, while I wanted to explore it, I first decided to turn it into a true dataset. The result was a set of resources, a website, a JSON, a CSV, and an SQLite database which are each more functional and more maintainable over time.
Just released on the No Time to Wait (NTTW) YouTube channel is my presentation from NTTW8 in Karlsruhe, Germany. (Slides also available here).
The presentation follows up on my proposal for iPRES 2024 and allowed me to present parts of what was, in the end, a pretty significant paper (in terms of word count).
Some of my reflections on the presentation are below.
File Formats As Emoji (0xFFAE or 0xffae) might be my most random file format hack yet. Indeed, it is a random page generator! But it generates random pages of file formats represented as Emoji.
The idea came in 2016 with radare releasing a new version that supported an emoji hexdump. I wondered whether I could do something fun combining file formats and the radare output to create a web-page.
Along came a spare moment one weekend, some pyscript, and bit of sqlite, et voilà. File Formats as Emoji (0xFFAE) was made a reality.
Today I want to showcase a Wikidata proof of concept that I developed as part of my work integrating Siegfried and Wikidata.
That work is wikiprov a utility to augment Wikidata results in JSON with the Wikidata revision history.
For siegfried it means that we can showcase the source of the results being returned by an identification without having to go directly back to Wikidata, this might mean more exposure for individuals contributing to Wikidata. We also provide access to a standard permalink where records contributing to a format identification are fixed at their last edit. Because Wikidata is more mutable than a resource like PRONOM this gives us the best chance of understanding differences in results if we are comparing siegfried+Wikidata results side-by-side.
I am interested to hear your thoughts on the results of the work. Lets go into more detail below.
I want to let you into a secret: I enjoy corruption. Corrupting digital objects leads to undefined behavior (C++’s definition is fun). And flipping bits in objects can tell us something both about the fragility, and robustness of our digital files and the applications that work with them.
I had a pull-request for bitflip accepted the other day. Bitflip is by Antoine Grondin and is a simple utility for flipping bits in digital files. I wrote in my COPTR entry for it that it reminds me of shotGun by Manfred Thaller. The utility is exceptionally easy to use (and of course update and maintain written in Golang) and has some nice features for flipping individual bits or a uniform percentage of bits across a digital file.
My pull-request was a simple one updating Goreleaser and its GitHub workflow to provide binaries for Windows and FreeBSD. I only needed to use Windows for a short amount of time thankfully, but it’s an environment I believe is prevalent for a lot of digital preservationists in corporate IT environments.
Bitflip is a useful utility to improve your testing of digital preservation systems, or simply for outreach, but let’s have a quick look at it in action.
Today I am thinking about versioning, especially in relation to its impact on digital preservation; both software preservation and the impact of versions on long-term preservation efforts in other contexts.
It is very poetic to think about code as it containing the memory of its maintainers. I don’t entirely disagree with the idea, but it’s overly poetic and the reality of maintenance on systems that have become too unwieldy is anything but poetic.
