Declarative all the way down: Building PRONOM signatures with JSONID

PRONOM signatures are a form of declarative language, you describe the anticipated behavior in PRONOM’s regular expression syntax and tools like DROID, FIDO, and Siegfried will interpret those instructions and attempt to match them against different files to return a file format identification.

Normally, you will write PRONOM signatures by hand but doing so for file formats based on other file format building blocks can lead to inconsistencies. Bertrand Caron previously also recognized this in the XML formats that are described with PRONOM signatures on Wikidata.

XML can use single quotes ' (hex: 0x27) and double quotes " (hex: 0x22) for attribute data, and so, do we make a PRONOM signature with multiple sequences anticipating the use of either?

The answer is more often than not likely to be yes, because the appearance of these values are often helpful for identifying boundaries for strings that we know must exist.

But the more file formats that we need to add to PRONOM that are based on foundational formats like XML, or JSON, or similar, the more inconsistencies will creep in, such as sequences that are looking specifically for one byte sequence over another.

The issue extends further if file formats allow data to appear at the beginning of file, or we need to account for a variable amount of white-space, or we want to start thinking about multi-byte character encoding.

We can, and in the XML issue described by myself and Caron, I think the recommendation is very much to create editorial standards for signatures for file formats based on other baseline, structured data formats like XML, JSON, YAML, and so on.

And standards are well and good, but what if tooling could help us?

For JSON this is exactly what I have tried to do in JSONID.

What does this feature look like? And what does it get us? Let’s take a look.

JSONID for PRONOM development

JSONID is described on COPTR and I was able to introduce it at No Time To Wait in Dublin last year.

I will also be sharing a short paper at this years iPRES conference in Copenhagen.

In short, JSONID provides a declarative language to describe data structures that can be represented in JSON, YAML, TOML, and JSONL—structures that you can recognize as a document type or a “file format” that you might want to identify in your collections, analogous to why we use PRONOM.

Example syntax

A simple example of JSONID syntax might be as follows:

    [
        { "KEY": "name", "IS": "value" },
        { "KEY": "schema", "CONTAINS": "/schema/version/1.1/" },
        { "KEY": "data", "IS": "more data" },
    ]

The pseudo code for this is:

1. Navigate to the JSON key “name” and determine that its value is “value”.
2. If 1. Navigate to key “schema” and test that its value contains: “/schema/version/1.1/”.
3. If 2. Navigate to key “data” and determine that its value is “more data”.

This declarative structure will match JSON with these properties, e.g. one that looks as follows:

{
    "name": "value",
    "schema": "_____/schema/version/1.1/_____",
    "any_data": [
        1, 2, 3, 4
    ],
    "data": "more data"
}

JSONID records 85 similar sequences at time of writing.

JSONID’s key benefit is that it will work on YAML,, TOML, or JSONL without having to define any other signature, and PRONOM in comparison has a harder time with these other formats because they don’t have as much structure as JSON.

The focus of this blog remains JSON because we can ask one very nice question:

Can we take JSONIDs signature definitions and turn them into PRONOM compatible signatures?

The answer is yes!

JSONID to PRONOM

This is described in more detail on the GitHub repository, but given our example snippet above we can use JSONID’s companion tool on our example above as follows:

python json2pronom.py --path json_id_blog/patterns_example.json

And a its output looks as follows:

<?xml version="1.0" ?>
<FFSignatureFile xmlns="http://www.nationalarchives.gov.uk/pronom/SignatureFile" Version="1" DateCreated="2026-07-21T19:54:50Z">
  <InternalSignatureCollection>
    <InternalSignature ID="1" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}7B" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="226E616D6522{0-16}76616C7565" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="VAR" Sequence="22736368656D6122{0-16}3A{0-16}22*2F736368656D612F76657273696F6E2F312E312F*22" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="VAR" Sequence="226461746122{0-16}6D6F72652064617461" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="EOF" Sequence="7D{0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>
    <InternalSignature ID="2" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}7B00" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="22006E0061006D0065002200{0-16}760061006C0075006500" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="VAR" Sequence="220073006300680065006D0061002200{0-16}3A00{0-16}2200*2F0073006300680065006D0061002F00760065007200730069006F006E002F0031002E0031002F00*2200" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="VAR" Sequence="220064006100740061002200{0-16}6D006F007200650020006400610074006100" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="EOF" Sequence="7D00{0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>
    <InternalSignature ID="3" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}007B" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="0022006E0061006D00650022{0-16}00760061006C00750065" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="VAR" Sequence="00220073006300680065006D00610022{0-16}003A{0-16}0022*002F0073006300680065006D0061002F00760065007200730069006F006E002F0031002E0031002F*0022" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="VAR" Sequence="002200640061007400610022{0-16}006D006F0072006500200064006100740061" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="EOF" Sequence="007D{0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>
    <InternalSignature ID="4" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}7B000000" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="220000006E000000610000006D0000006500000022000000{0-16}76000000610000006C0000007500000065000000" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="VAR" Sequence="22000000730000006300000068000000650000006D0000006100000022000000{0-16}3A000000{0-16}22000000*2F000000730000006300000068000000650000006D000000610000002F00000076000000650000007200000073000000690000006F0000006E0000002F000000310000002E000000310000002F000000*22000000" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="VAR" Sequence="220000006400000061000000740000006100000022000000{0-16}6D0000006F00000072000000650000002000000064000000610000007400000061000000" MinOffset="" MaxOffset=""/>
      <ByteSequence Reference="EOF" Sequence="7D000000{0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>
  </InternalSignatureCollection>
  <FileFormatCollection>
    <FileFormat ID="1" Name="JSONID2PRONOM Conversion (UTF-8)" PUID="jsonid2pronom/1" Version="" MIMEType="application/json" FormatType="structured text">
      <InternalSignatureID>1</InternalSignatureID>
      <Extension>json</Extension>
    </FileFormat>
    <FileFormat ID="2" Name="JSONID2PRONOM Conversion (UTF-16)" PUID="jsonid2pronom/2" Version="" MIMEType="application/json" FormatType="structured text">
      <InternalSignatureID>2</InternalSignatureID>
      <Extension>json</Extension>
      <HasPriorityOverFileFormatID>1</HasPriorityOverFileFormatID>
    </FileFormat>
    <FileFormat ID="3" Name="JSONID2PRONOM Conversion (UTF-16BE)" PUID="jsonid2pronom/3" Version="" MIMEType="application/json" FormatType="structured text">
      <InternalSignatureID>3</InternalSignatureID>
      <Extension>json</Extension>
      <HasPriorityOverFileFormatID>2</HasPriorityOverFileFormatID>
      <HasPriorityOverFileFormatID>1</HasPriorityOverFileFormatID>
    </FileFormat>
    <FileFormat ID="4" Name="JSONID2PRONOM Conversion (UTF-32LE)" PUID="jsonid2pronom/4" Version="" MIMEType="application/json" FormatType="structured text">
      <InternalSignatureID>4</InternalSignatureID>
      <Extension>json</Extension>
      <HasPriorityOverFileFormatID>2</HasPriorityOverFileFormatID>
      <HasPriorityOverFileFormatID>3</HasPriorityOverFileFormatID>
      <HasPriorityOverFileFormatID>1</HasPriorityOverFileFormatID>
    </FileFormat>
  </FileFormatCollection>
</FFSignatureFile>

A PRONOM signature containing:

Consistent beginning of file (BOF) and end of file (EOF) offsets.
Consistent encoding of whitespace.
Outputs for multiple different encodings including UTF-16 and UTF-32.
Variably positioned keys where keys can appear anywhere in the JSON map without changing the document type.

What does this mean for you?

JSONID can already output the majority of its 85 signature patterns to PRONOM.

JSONID will also output consistent baseline sequences for basic JSON identification.

Using the companion utility I have described above, if YOU are trying to write JSON based signatures, you can consider writing them first in JSONID’s basic syntax, and output strong, variably encoded PRONOM signatures as a by-product.

You can focus on identifying the important structure of your JSON documents and JSONID can do the heavy lifting and output PRONOM signatures for you to select and test and submit to the PRONOM team.

Declarative all the way down. We’ve successfully built PRONOM from JSONID.

A prototype open for discussion

JSONID will output suggested defaults for baseline JSON identification e.g. UTF-8:

    <InternalSignature ID="0" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}(7B|5B)" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="22{0-16}3A" MinOffset="0" MaxOffset="0"/>
      <ByteSequence Reference="EOF" Sequence="(7D|5D){0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>
    <InternalSignature ID="0" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}(7B|5B)" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="[30:39]" MinOffset="0" MaxOffset="0"/>
      <ByteSequence Reference="EOF" Sequence="(7D|5D){0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>
    <InternalSignature ID="0" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}(7B|5B)" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="(74727565|66616C7365)" MinOffset="0" MaxOffset="0"/>
      <ByteSequence Reference="EOF" Sequence="(7D|5D){0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>
    <InternalSignature ID="0" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}(7B|5B)" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="6E756C6C" MinOffset="0" MaxOffset="0"/>
      <ByteSequence Reference="EOF" Sequence="(7D|5D){0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>

And UTF-16:

    <InternalSignature ID="0" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}(7B00|5B00)" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="2200{0-16}3A00" MinOffset="0" MaxOffset="0"/>
      <ByteSequence Reference="EOF" Sequence="(7D00|5D00){0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>
    <InternalSignature ID="0" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}(7B00|5B00)" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="[30:39]" MinOffset="0" MaxOffset="0"/>
      <ByteSequence Reference="EOF" Sequence="(7D00|5D00){0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>
    <InternalSignature ID="0" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}(7B00|5B00)" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="(7400720075006500|660061006C0073006500)" MinOffset="0" MaxOffset="0"/>
      <ByteSequence Reference="EOF" Sequence="(7D00|5D00){0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>
    <InternalSignature ID="0" Specificity="Specific">
      <ByteSequence Reference="BOF" Sequence="{0-4095}(7B00|5B00)" MinOffset="0" MaxOffset="4095"/>
      <ByteSequence Reference="VAR" Sequence="6E0075006C006C00" MinOffset="0" MaxOffset="0"/>
      <ByteSequence Reference="EOF" Sequence="(7D00|5D00){0-4095}" MinOffset="0" MaxOffset="4095"/>
    </InternalSignature>

Baseline JSON suggestions differ from the signature development workflow described above. The primary approach is that JSONID will build PRONOM signatures based solely on the data you describe. Baseline JSON patterns are designed as a catch-all for all JSON documents, and these have a lower identification priority (or weighting) compared to more specific JSON document types.

The defaults may or may not be optimal and I hope that the continuing conversation about this utility will open these lines of discussion perhaps allowing a broader number of the signature development community to find consensus and sensible default settings.

Feel free to engage in this discussion via GitHub’s Issues or Discussions.

Additional thoughts

JPEG

JPEG is another example of a format on which a lot of other “formats” are based. The recent discussion about extending the size of the end of file sequence demonstrates the importance of standardizing PRONOM signature encoding practices when dealing with pluralistic base formats. A standard PRONOM EOF sequence should probably be applied to all formats in PRONOM that use the JPEG (or JPEG File Interchange Format (JFIF)).

PRONOM research on this blog

The ability to write PRONOM based signatures from JSONID’s syntax has evolved out of my research into PRONOM’s alternative simplified signature syntax. The simplified syntax makes it easier to programmatically write PRONOM compatible XML.

You can read more about this research in the articles below.

simpledroid: completing the loop

Making DROID work with Wikidata

PRONOM as Emoji

While you’re reading this blog don’t forget you can take a light-hearted look at PRONOM’s data through another of my side projects through the lens of Emoji. More about that in the article below.

File formats as Emoji: 0xffae

2 thoughts on “Declarative all the way down: Building PRONOM signatures with JSONID”

Jack O'Sullivan says:

2026-07-22 at 11:15

@exponentialdecay I've always thought that XML particularly, but probably JSON as well, should be handled more like container formats, i.e. we do an initial parse that says this is XML, then use more rigourous XML parsing to identify exactly what it is based on things like namespace validation, because that kind of thing is very hard to describe in a PRONOM signature
1. Ross Spencer says:
  
  2026-07-22 at 12:08
  
  I hope it’s a thread worth following. In the iPRES paper I cover most of the prior art I could find and talk about where this could go. I’d love to see the same for XML. I still have a vague memory of a desire to record schema in PRONOM or maybe Planets Core Registry, I’d love to make that memory concrete if there are still resources about. Looking forward to talking about it more. There are a few directions it could go including closer alignment with JSON schema but for identification.