Digital Preservation: Digitisation dilemmas

Digitisation dilemmas

[caption id="" align="alignright" width="201"] Operating a digital camera via the laptop[/caption]



There are substantial benefits to be gained by digitising paper records. As a result, the number of organisations undertaking digitisation has increased dramatically in recent years.



However, if you want to introduce digitisation projects or programs you need to be aware that there are also a variety of risks. Being aware of these risks can help your organisation to built mitigation measures into program/project planning and management.



Types of digitisation



Organisations usually conduct either (1) business process digitisation programs, where they scan incoming correspondence as it is received (2) back-capture digitisation projects, where accumulated paper records are digitised.



Benefits of digitisation



When programs and projects are planned and managed well and in line with best practice, digitisation can bring many benefits. Depending on the program or project, digitisation may allow you to:

• improve the accessibility of records within the organisation



• publish records online and/or facilitate information sharing more widely



• integrate records with existing EDRMS and business systems



• Improve efficiency by reducing reliance on hybrid systems



• Automate workflow



• Enable and encourage data reuse



• Promote the preservation of original paper records by using digital images instead



• Reduce physical storage costs by destroying paper records after digitisation.

Risks of digitisation



Below are some examples of risks that may apply, depending on the nature of the program or project, and how risks can be mitigated:



1. Risk: That money will be wasted by poor selection of records for digitisation



Some organisations get so excited about digitisation that they want to digitise everything! Digitisation is an expensive undertaking and should only be done when substantial and realistic benefits can be made.



It is also worth noting that some records do not digitise well e.g. fragile records, records with faint inks etc. Others, such as large maps and plans, will require specialist (and often more expensive) digitisation techniques and equipment.



Risk mitigation: Selection decisions for digitisation programs or projects should be carefully defined in planning.



2. Risk: That the complexities of digitisation will be underestimated



Digitisation is inherently complex. It is not enough just to buy the software and hardware and start scanning, or sign a contract with a service provider and send them paper records. If care is not taken the results could be disastrous, the digital images unusable and public money wasted.



Risk mitigation: All digitisation activity needs to be carefully planned and managed. Research needs to be conducted and decisions need to be made and documented about a variety of matters including:

• what records should be digitised and why



• what benchmarks are needed



• what technical standards to use



• what metadata is required, and



• what quality assurance measure to apply.

Staff members need to be trained and supported to perform their allocated roles.



If contracting-out digitisation, organisations need to ensure that controls are communicated to service providers and built into contracts. The performance of the contractor must be monitored and the paper and digital records protected at all times. For more information about responsibilities when contracting-out see Accountable outsourcing: Recordkeeping considerations of outsourcing NSW Government business (Guideline 16).



3. Risk: That costs will be underestimated



Digitisation involves a range of upfront and ongoing costs:

• Upfront costs can include the purchase of hardware and software, training and support of staff, staff time to define benchmarks, contract set-up etc. In some cases costs might include conservation treatment for fragile paper records prior to digitisation.



• Ongoing costs include staff time to digitise and perform quality assurance, the cost of maintaining digital images (and possibly original paper records as well) over time, managing variables and the maintenance and update of systems.

In particular, the ongoing costs (and complexity) of preserving digital images in place of original paper records over long periods of time should not be underestimated. This will require a significant investment of funds over a long period of time which could make some projects unviable.



If outsourcing, some upfront costs can be reduced, but the organisation will incur other costs from the service provider.



Risk mitigation: As part of planning, organisations need to clearly define both upfront and ongoing costs, including costs for maintaining some digital images in the long term, and ensure that they have adequate resources over the lifetime of the program or project. If they cannot guarantee resources, project aims and selection decisions may have to be reviewed.



4. Risk: That original paper records, including State archives, will be destroyed without authorisation



Any organisations undertaking digitisation in order to destroy original paper records need to be aware that not all records are authorised for destruction after digitisation. If a reduction in storage costs is one of the promised benefits in their business case, this may not always be deliverable.



Where original paper records can be destroyed, the digital images will become the primary records and need to be managed accordingly and kept for the full retention periods specified in retention and disposal authorities.



For more information on disposal see the General retention and disposal authority – Imaged records.



Risk mitigation: Organisations should clearly understand disposal requirements for the original paper records selected for digitisation before embarking on a digitisation program or project. They should also build in staff training and quality assurance measures to ensure that no unauthorised disposal takes place. Any authorised destruction should be secure and in line with organisational records policy.



5. Risk: That original paper records will be damaged during the digitisation process



Back-capture digitisation projects require that original paper records be retrieved from storage, handled and sometimes unbound. On occasions pages may need to be removed (e.g. in order to apply different scanner settings). In the case of outsourcing, original paper records will need to be transported to and from service providers.



There is a risk that the paper records will be damaged in this process or returned to storage in the incorrect order.  This may create problems and compromise their ability to function as evidence, especially for records of high risk or high value that are being retained after digitisation.



Risk mitigation: Organisations need to ensure that original paper records are handled carefully and put back in their original order. Staff members need to be trained in handling and supervised where necessary and quality assurance measures built in to ensure records are not damaged.



Important note: If records are required as State archives, advice from State Records should be sought before digitisation commences.



6. Risk: That digitisation will not produce a full and accurate representation of the original paper records



The aim of digitisation is to create digital images of the original paper records. The quality of these images need to be fit for purpose. In some cases it may be appropriate to settle for smaller files of lesser quality e.g. for the delivery of images online.



However, there are occasions where the quality of the digital images cannot be compromised, for example, where the original paper records are to be destroyed and the digital records are to replace them as evidence of the business activity (and could potentially be required in court).



Risk mitigation: Organisations need to identify what the essential characteristics of the records are that must be retained, and create quality benchmarks to ensure that these remain intact. The definition of technical and metadata standards, along with staff training and quality assurance checks (either of images or samples of images) should help to ensure that image quality is fit for purpose. Procedures and standards need to be developed and documented as these will help to demonstrate that digital images are trusted copies of the originals.



For more information see the General retention and disposal authority – Imaged records and Digital imaging and recordkeeping (RIB 11).



7. Risk: That digital images are not stored or protected appropriately



If the original paper records are destroyed the digital images become the official records. Digital images must be stored securely, remain tamper-proof and unaltered, and be kept for the retention period defined for the original paper records. Poor storage or protection of the digital images may result in the loss of essential evidence, which can have far-reaching consequences for the organisation.



Even if the original paper records are retained, there are risks to the organisation if digital images are not subject to the same controls as the original paper records. For example, access and security regimes for paper records with sensitive content should equally be applied to the digital images.



Risk mitigation: Organisations should confer with their records management staff to ensure that digital images are stored and controlled appropriately.



Ideally, digital images should be captured into a recordkeeping system, where they can be accessed only by authorised users, secured from unauthorised alteration or deletion, in context with related records, protected from disaster and kept for their required retention periods.



If the organisation does not have a recordkeeping system, digital images need to be kept on a secure drive with stringent security, access controls and backup until the organisation can move them into a recordkeeping system.



Note: Master digital images should not be stored on removable media due to their inherent risks. For more information see Managing removable media in the Managing digital records guidelines.



8. Risk: That duplicates are not managed appropriately



When original paper records are retained, the digital (master) image is already a duplicate. Organisations may also potentially require multiple digital copies at different levels of quality, depending on their purpose.



If these are inefficiently managed, it will be difficult to ascertain which version is the ‘official’ record and to ensure that it survives for as long as it is needed.  The existence of many derivative images may also add to storage and management costs and cause problems in relation to access and security.



Risk mitigation: Organisations should confer with their records management staff to ensure that digital images are managed appropriately. They will also need to adopt a well defined metadata schema with relationships between records to ensure that it is very clear whether the original paper record exists, which is the ‘master’ image and which are derivatives and for what purpose they are to be used.



9. Risk: That digital images will not survive and remain accessible for as long as they are required



Digital images are inherently unstable. Technological obsolescence and media degradation can render them unreadable in relatively short time frames. If digital images are replacing original paper records, and are required as evidence for long periods of time, the organisation will need to consider how to ensure  they survive for as long as they are required.



Risk mitigation: Organisations should confer with their records management staff to ensure that digital records are managed as part of a records management program and that their longevity is planned for. For example, the organisation may be required to plan for, test and implement data and systems migration on an average of 5-7 years in order to maintain information accessibility and integrity. Funds for ongoing management should be defined in business cases and made available when required. Technological and storage decisions will impact on the longevity of images and should be made with care.



The above risks, and any other risks identified, should be considered as part of the planning and management of digitisation programs or projects. If mitigation strategies are built in, organisations can enjoy the benefits digitisation can offer, while avoiding the pitfalls.



 Guidelines



State Records NSW intends to develop detailed guidelines, both for business process digitisation programs and back-capture digitisation projects, which will address how to manage projects, optimise benefits and reduce risks. These will be released in 2012.



In the meantime, please feel free to provide us with feedback through the blog or by email at govrec@records.nsw.gov.au if:

• you want to discuss any particular digitisation problems you are encountering



• you believe you have established a best practice digitisation program or project



• you have found innovative solutions to common digitisation problems



• you would to share your business case for digitisation.

Such information will help us to ensure that the guidelines are beneficial for all NSW organisations undertaking digitisation.

Also of interest:

• Digitisation does not equal digital preservation!



• Exposure draft of digitisation guidance now available