Not your first paper from iPRES2024: Design patterns in Digital Preservation: Declarative software for digital preservationists

Well folks, my paper for iPRES2024 was rejected. but the good news is that you get to read it here now! (PDF)

The paper is titled “Design patterns in Digital Preservation: Declarative software for digital preservationists”. It is a title inspired by Andy Jackson’s “Design Patterns in Digital Preservation: Understanding Information Flows” (available on the British Library’s website). Design patterns providing a neat way to introduce and talk about a concept and its implementations and impact.

The abstract:

A network effect is defined by Wikipedia as a positive feedback system whereby users derive more value from a product as more users participate in the same network. We rely on network effect in digital preservation to help manage the sheer number of tasks involved with collecting and maintaining digital content. Network effect is used to develop tooling that helps users with the same problems to solve them. Network effect also helps us to check the veracity of solutions where misplaced and corrupt bytes are so detrimental to the community’s reputations and collections. Software is used throughout the digital preservation technology stack to store, to encode, to check, to analyze, to replay. These functions exist on a continuum and are repeated over and over by the many institutions engaging with digital preservation today. This paper looks at a style of programming called declarative programming that we explain we already take advantage of during the analysis of digital content and from which we take huge advantages from network effect. We look at similar design patterns that share the same properties as declarative programming – declarative-style approaches. We ask whether or not declarative approaches can be used elsewhere in the digital preservation stack. As more initiatives appear to engage with programming skills at a professional level within the GLAM sector we ask readers to understand the declarative pattern more in their software architecture choices and potentially choose to adopt it to enhance the all-important network effect that the community needs to keep on improving its own capabilities.

Declarative programming asks the user to focus on what software does vs. how it does it (see imperative programming). Declarative programming is more high-level than its counterpart in imperative but designed well it can provide its users important network effect, for example, as we see in PRONOM — users contribute declarative regular expressions to a database that everyone benefits from once they are included in a PRONOM signature release.

While one reviewer suggests the paper is one-sided and they cannot find the problem definition in the paper. Rather than creating another false dichotomy in the field, suggesting one approach over and above another, my focus is on describing a paradigm for a community and describing its benefits; drawing a contrast between styles only as much as is necessary.

In designing declarative programming interfaces we benefit more from deliberate design thinking and thinking about executing a solution to a problem together rather than in piecemeal and bespoke ways. Importantly a declarative approach to writing code can lower the barrier to participation in the development (and improvement cycle) democratizing the software development process. Writing about this should add to a body of digital literacy in the topic of software development and solutions design in digital preservation which I do feel we need more of.

Written for the ‘scaling up’ track, and written as a community piece in general, the writing tries to combine existing declarative approaches in digital preservation with new approaches we are yet to explore collectively in more detail. 

The paper draws out examples from the field of digital preservation over the last decade or so that already use a declarative approach or can be adopted to use one. We might know those projects but we might not recognize the paradigm that they adopted. The paper looks at PRONOM; Johan van der Knijff’s work with Schematron in the early 10’s; work by Richard Lehane at State Records New South Wales; Blewer et al’s ffmprovisr, and FQ and Kaitai from developers Mattias Wadman and Mikhail Yakshin, respectively.

I always enjoy trying to amplify others’ work and demonstrate its continuing relevance to our community and have tried to do this here. I also enjoy trying to start new, focused discussion on topics, as I have also tried to do here.

One reviewer wrote:

I also see his point to make the turn towards declarative programming and find it certainly worth the discussion in our community.

Which I appreciate. I would love to be part of that discussion sometime down the line.

I have left the other reviewer comments below for those interested or considering writing for iPRES in future.

They don’t help me too much, iPRES do not allow you to edit and re-submit a paper (while also asking for complete papers to be submitted seven months in advance of the conference before delivering abstracts.) REVIEWER 1 certainly isn’t happy with the paper. REVIEWER 3 seems to have received a mojibaked web-version of the PDF I provided, so I won’t be able to fix the layout issues (read also, the ACM format needs to be gone (and the less said about PDF the better)).

What do you think of the paper? The call to discuss declarative approaches in more detail? and more generally, a collaborative approach to enumerating and solving digital preservation issues? instead of each and everyone writing their own custom scripts?

Please enjoy Design patterns in Digital Preservation: Declarative software for digital preservationists, it was a fun one to write, it would have been lots of fun to talk about and engage with people on at iPRES.

DESIGN PATTERNS IN DIGITAL PRESERVATION: DECLARATIVE SOFTWARE FOR DIGITAL PRESERVATIONISTS

---

REVIEWER 1 Introduction is too long. A lot of detailed, sometimes redundant information, especially for the introduction. The reader has difficulties to find the aim / problem definition for the article. For instance the detailed discussion of programming paradigms / styles should be move to different /its own chapter, which introduces new concepts used in this article. – Section ||: – introduce “hub-and-spoke model” – Condensing sections II – IV into a single chapter would benefit readability. To many details, but without being clear why it is necessary to be read. Are all examples really necessary and what do I learn for the actual approach of this article? – Formal issues: – citation style is not correct. please use the [X] where X is the number of the reference found in the references list. The template provides a few examples. – Paper is too long – Paper requieres reformating – The paper is very prosaic with many anecdotes, a more dense language and less anecdotes I find more appropriate for a format like this. – Style: – I’d rather remove all footnotes containing text only. Either the text is necessary or not. Using additional text in footnotes makes reading difficult and is usually a sign of structural issues of the text. – You may reduce the number of direct quotes, as this article addresses an informed reader (regarding basic aspects of dig. preservation). – The article lacks structure and a clear focus. – General: I can understand the general idea but if have difficulties understanding the goal of this article. By presenting too many detailed examples it is hard to not get lost. I can also understand many arguments made for a declarative approach, but the discussion seems one-sided. How do we deal with issues like predictive results. From an imperative view, for instance, issuing an ffmpeg command with a bunch of parameters will produce a multifaceted predictable outcome, mostly through implicit parameters settings or implicit configurations. To reproduce this result a huge amount of details have to be declared up-front. I agree this would be ideal, but most likely for many cases not realistic. A discussion along these lines would be fruitfull in my view.

REVIEWER 2 The contribution is an “ode” (if one may write so) to the declarative approach to programming and supports this way with different applications and ideas. This in a very clear, pedagogic and precise way. I mostly liked how the author explained the concepts and the ideas with examples and already existing open technologies and software. I also see his point to make the turn towards declarative programming and find it certainly worth the discussion in our community. As it is rather a practice paper than a study, I find it difficult to evaluate the validity. I however think the paper fits well into the scope of iPRES and would certainly give food for discussion. My open questions would here be: What is necessary to make this shift happen? How to convince / bring people in our field to turn more towards declarative programming or to use the tools mentioned in the contribution? What is the current research state of delarative programming? Is there feedback from other communities? Community building: Does this contribution reinforce development & collaboration in the field? Yes, the contribution asks for turning towards the declarative approach to programming to democratize the software development process and gives examples of software to use that would underline this approach.

REVIEWER 3 The paper certainly makes an interesting case for declarative software. Nevertheless, it suffers from a few readability issues due to introducing concepts or abbreviations without explaining them, or explaining them only later the in paper. More so, tables or figures are referenced but have then to be found on certain webpages or some other place in the paper itself. It could also do with another proofreading for some small style and structure issues. The UNIX statement for example on page 12 starts at 2. At the end the author also makes a small comment referencing large language models and the like. This seems very much worth expanding upon more substantially than a few sentences even though one can fill multiple papers on the subject. Nevertheless, I recommend this paper to be accepted.

REVIEWER 4 A purely technical paper with a nice selection of programming paradigms. Nonetheless, it is written in an informal tone with grammar and syntax errors. Also, the proposal does not follow the iPRES 2024 template (i.e. in-text citations). Further, the paper does not include a clear structure. Therefore, the text at times appears to be incomprehensible and rather confusing for the non-specialist. The paper would be a much more enjoyable read if there was a connection between different paragraphs.