{"id":266,"date":"2012-11-22T21:48:42","date_gmt":"2012-11-22T21:48:42","guid":{"rendered":"http:\/\/exponentialdecay.co.uk\/blog\/?p=266"},"modified":"2023-07-10T18:18:55","modified_gmt":"2023-07-10T17:18:55","slug":"hacking-the-droid-signature-file-keep-it-simple-stupid","status":"publish","type":"post","link":"https:\/\/exponentialdecay.co.uk\/blog\/hacking-the-droid-signature-file-keep-it-simple-stupid\/","title":{"rendered":"Hacking the DROID Signature File: Keep It Simple Stupid!"},"content":{"rendered":"

One of the nicest features introduced by the DROID 6.01 development team (Matt Palmer, Richard Flitcroft, Richard Brennan and Alok Dash) was a simplified signature file format. The format cuts out DROID shift bytes<\/em> reducing the pre-processing required to output a signature file by any system that chooses to do so. It\u00a0is a feature that is yet to be adopted as it results in the generation of signature files that are only compatible with DROID 6.01 onward and would require a change to PRONOM stored procedures but you can see traces of it in DROID container signature files which appear in 6.01.<\/p>\n

Once again, using the Eyeglass sample format<\/a> as our example format and making use of the\u00a0signature files<\/a>\u00a0created last week, we can demonstrate the difference this simplification makes like so:<\/p>\n

The contraction of the shift bytes using DROID 6.01 syntax can turn a byte sequence (within any particular signature) from this:<\/p>\n

<ByteSequence Reference=\"BOFoffset\">\r\n <SubSequence MinFragLength=\"0\" Position=\"1\" ... >\r\n <Sequence>BB0D0A657965676C6173731A0AAB<\/Sequence>\r\n <DefaultShift>15<\/DefaultShift>\r\n <Shift Byte=\"BB\">14<\/Shift>\r\n <Shift Byte=\"0D\">13<\/Shift>\r\n <Shift Byte=\"0A\">2<\/Shift>\r\n <Shift Byte=\"65\">9<\/Shift>\r\n <Shift Byte=\"79\">10<\/Shift>\r\n <Shift Byte=\"67\">8<\/Shift>\r\n <Shift Byte=\"6C\">7<\/Shift>\r\n <Shift Byte=\"61\">6<\/Shift>\r\n <Shift Byte=\"73\">4<\/Shift>\r\n <Shift Byte=\"1A\">3<\/Shift>\r\n <Shift Byte=\"AB\">1<\/Shift>\r\n <RightFragment MaxOffset=\"1\" MinOffset=\"1\" Position=\"1\">\r\n  [!00:01]\r\n <\/RightFragment>\r\n <\/SubSequence>\r\n<\/ByteSequence><\/pre>\n
To this:<\/p>\n
<ByteSequence Reference=\"BOFoffset\">\r\n <SubSequence MinFragLength=\"0\" Position=\"1\" ... >\r\n <Sequence>BB0D0A657965676C6173731A0AAB{1}[!00:01]<\/Sequence>\r\n <\/SubSequence>\r\n<\/ByteSequence><\/pre>\n
As you can see, the DROID regular expression syntax,\u00a0‘{1}[!00:01]’,<\/em> is now encoded in the primary ‘<Sequence>’ tag and the shift bytes no longer exist.<\/p>\n
Converting all the signatures in the original signature file created last week<\/a> results in a new, more simplified signature file<\/a>. When we run the signature file across the Eyeglass sample files in DROID we get the same format identifications and characterizations. The difference in size between the two pretty-printed files is\u00a06410<\/em> vs. 3570<\/em> bytes<\/em>. At almost half the size, using extremely basic signatures, the effect on the entire DROID signature file (currently at 1,339,853 bytes) will be even bigger.<\/p>\n
Having worked through the pre-processing algorithm to create the PRONOM Signature Development Utility<\/a> (mirrored here<\/a>), I can say that the amount of string manipulation required is phenomenal. It creates a barrier for anyone who wants to create a DROID signature file or who is attempting to interpret one or reuse it in their own identification engine.<\/p>\n
The simplified approach:<\/p>\n