Digital Preservation: Digitisation dilemmas

Digitisation dilemmas

[caption id="" align="alignright" width="201"] Operating a digital camera via the laptop[/caption]



There are substantial benefits to be gained by digitising paper records. As a result, the number of organisations undertaking digitisation has increased dramatically in recent years.



However, if you want to introduce digitisation projects or programs you need to be aware that there are also a variety of risks. Being aware of these risks can help your organisation to built mitigation measures into program/project planning and management.



Types of digitisation



Organisations usually conduct either (1) business process digitisation programs, where they scan incoming correspondence as it is received (2) back-capture digitisation projects, where accumulated paper records are digitised.



Benefits of digitisation



When programs and projects are planned and managed well and in line with best practice, digitisation can bring many benefits. Depending on the program or project, digitisation may allow you to:

• improve the accessibility of records within the organisation



• publish records online and/or facilitate information sharing more widely



• integrate records with existing EDRMS and business systems



• Improve efficiency by reducing reliance on hybrid systems



• Automate workflow



• Enable and encourage data reuse



• Promote the preservation of original paper records by using digital images instead



• Reduce physical storage costs by destroying paper records after digitisation.

Risks of digitisation



Below are some examples of risks that may apply, depending on the nature of the program or project, and how risks can be mitigated:



1. Risk: That money will be wasted by poor selection of records for digitisation



Some organisations get so excited about digitisation that they want to digitise everything! Digitisation is an expensive undertaking and should only be done when substantial and realistic benefits can be made.



It is also worth noting that some records do not digitise well e.g. fragile records, records with faint inks etc. Others, such as large maps and plans, will require specialist (and often more expensive) digitisation techniques and equipment.



Risk mitigation: Selection decisions for digitisation programs or projects should be carefully defined in planning.



2. Risk: That the complexities of digitisation will be underestimated



Digitisation is inherently complex. It is not enough just to buy the software and hardware and start scanning, or sign a contract with a service provider and send them paper records. If care is not taken the results could be disastrous, the digital images unusable and public money wasted.



Risk mitigation: All digitisation activity needs to be carefully planned and managed. Research needs to be conducted and decisions need to be made and documented about a variety of matters including:

• what records should be digitised and why



• what benchmarks are needed



• what technical standards to use



• what metadata is required, and



• what quality assurance measure to apply.

Staff members need to be trained and supported to perform their allocated roles.



If contracting-out digitisation, organisations need to ensure that controls are communicated to service providers and built into contracts. The performance of the contractor must be monitored and the paper and digital records protected at all times. For more information about responsibilities when contracting-out see Accountable outsourcing: Recordkeeping considerations of outsourcing NSW Government business (Guideline 16).



3. Risk: That costs will be underestimated



Digitisation involves a range of upfront and ongoing costs:

• Upfront costs can include the purchase of hardware and software, training and support of staff, staff time to define benchmarks, contract set-up etc. In some cases costs might include conservation treatment for fragile paper records prior to digitisation.



• Ongoing costs include staff time to digitise and perform quality assurance, the cost of maintaining digital images (and possibly original paper records as well) over time, managing variables and the maintenance and update of systems.

In particular, the ongoing costs (and complexity) of preserving digital images in place of original paper records over long periods of time should not be underestimated. This will require a significant investment of funds over a long period of time which could make some projects unviable.



If outsourcing, some upfront costs can be reduced, but the organisation will incur other costs from the service provider.



Risk mitigation: As part of planning, organisations need to clearly define both upfront and ongoing costs, including costs for maintaining some digital images in the long term, and ensure that they have adequate resources over the lifetime of the program or project. If they cannot guarantee resources, project aims and selection decisions may have to be reviewed.



4. Risk: That original paper records, including State archives, will be destroyed without authorisation



Any organisations undertaking digitisation in order to destroy original paper records need to be aware that not all records are authorised for destruction after digitisation. If a reduction in storage costs is one of the promised benefits in their business case, this may not always be deliverable.



Where original paper records can be destroyed, the digital images will become the primary records and need to be managed accordingly and kept for the full retention periods specified in retention and disposal authorities.



For more information on disposal see the General retention and disposal authority – Imaged records.



Risk mitigation: Organisations should clearly understand disposal requirements for the original paper records selected for digitisation before embarking on a digitisation program or project. They should also build in staff training and quality assurance measures to ensure that no unauthorised disposal takes place. Any authorised destruction should be secure and in line with organisational records policy.



5. Risk: That original paper records will be damaged during the digitisation process



Back-capture digitisation projects require that original paper records be retrieved from storage, handled and sometimes unbound. On occasions pages may need to be removed (e.g. in order to apply different scanner settings). In the case of outsourcing, original paper records will need to be transported to and from service providers.



There is a risk that the paper records will be damaged in this process or returned to storage in the incorrect order.  This may create problems and compromise their ability to function as evidence, especially for records of high risk or high value that are being retained after digitisation.



Risk mitigation: Organisations need to ensure that original paper records are handled carefully and put back in their original order. Staff members need to be trained in handling and supervised where necessary and quality assurance measures built in to ensure records are not damaged.



Important note: If records are required as State archives, advice from State Records should be sought before digitisation commences.



6. Risk: That digitisation will not produce a full and accurate representation of the original paper records



The aim of digitisation is to create digital images of the original paper records. The quality of these images need to be fit for purpose. In some cases it may be appropriate to settle for smaller files of lesser quality e.g. for the delivery of images online.



However, there are occasions where the quality of the digital images cannot be compromised, for example, where the original paper records are to be destroyed and the digital records are to replace them as evidence of the business activity (and could potentially be required in court).



Risk mitigation: Organisations need to identify what the essential characteristics of the records are that must be retained, and create quality benchmarks to ensure that these remain intact. The definition of technical and metadata standards, along with staff training and quality assurance checks (either of images or samples of images) should help to ensure that image quality is fit for purpose. Procedures and standards need to be developed and documented as these will help to demonstrate that digital images are trusted copies of the originals.



For more information see the General retention and disposal authority – Imaged records and Digital imaging and recordkeeping (RIB 11).



7. Risk: That digital images are not stored or protected appropriately



If the original paper records are destroyed the digital images become the official records. Digital images must be stored securely, remain tamper-proof and unaltered, and be kept for the retention period defined for the original paper records. Poor storage or protection of the digital images may result in the loss of essential evidence, which can have far-reaching consequences for the organisation.



Even if the original paper records are retained, there are risks to the organisation if digital images are not subject to the same controls as the original paper records. For example, access and security regimes for paper records with sensitive content should equally be applied to the digital images.



Risk mitigation: Organisations should confer with their records management staff to ensure that digital images are stored and controlled appropriately.



Ideally, digital images should be captured into a recordkeeping system, where they can be accessed only by authorised users, secured from unauthorised alteration or deletion, in context with related records, protected from disaster and kept for their required retention periods.



If the organisation does not have a recordkeeping system, digital images need to be kept on a secure drive with stringent security, access controls and backup until the organisation can move them into a recordkeeping system.



Note: Master digital images should not be stored on removable media due to their inherent risks. For more information see Managing removable media in the Managing digital records guidelines.



8. Risk: That duplicates are not managed appropriately



When original paper records are retained, the digital (master) image is already a duplicate. Organisations may also potentially require multiple digital copies at different levels of quality, depending on their purpose.



If these are inefficiently managed, it will be difficult to ascertain which version is the ‘official’ record and to ensure that it survives for as long as it is needed.  The existence of many derivative images may also add to storage and management costs and cause problems in relation to access and security.



Risk mitigation: Organisations should confer with their records management staff to ensure that digital images are managed appropriately. They will also need to adopt a well defined metadata schema with relationships between records to ensure that it is very clear whether the original paper record exists, which is the ‘master’ image and which are derivatives and for what purpose they are to be used.



9. Risk: That digital images will not survive and remain accessible for as long as they are required



Digital images are inherently unstable. Technological obsolescence and media degradation can render them unreadable in relatively short time frames. If digital images are replacing original paper records, and are required as evidence for long periods of time, the organisation will need to consider how to ensure  they survive for as long as they are required.



Risk mitigation: Organisations should confer with their records management staff to ensure that digital records are managed as part of a records management program and that their longevity is planned for. For example, the organisation may be required to plan for, test and implement data and systems migration on an average of 5-7 years in order to maintain information accessibility and integrity. Funds for ongoing management should be defined in business cases and made available when required. Technological and storage decisions will impact on the longevity of images and should be made with care.



The above risks, and any other risks identified, should be considered as part of the planning and management of digitisation programs or projects. If mitigation strategies are built in, organisations can enjoy the benefits digitisation can offer, while avoiding the pitfalls.



 Guidelines



State Records NSW intends to develop detailed guidelines, both for business process digitisation programs and back-capture digitisation projects, which will address how to manage projects, optimise benefits and reduce risks. These will be released in 2012.



In the meantime, please feel free to provide us with feedback through the blog or by email at govrec@records.nsw.gov.au if:

• you want to discuss any particular digitisation problems you are encountering



• you believe you have established a best practice digitisation program or project



• you have found innovative solutions to common digitisation problems



• you would to share your business case for digitisation.

Such information will help us to ensure that the guidelines are beneficial for all NSW organisations undertaking digitisation.

Also of interest:

• Digitisation does not equal digital preservation!



• Exposure draft of digitisation guidance now available

Read More

Grenada and UNESCO: Caribbean Knowledge Societies

Building Caribbean Knowledge Societies: IFAP Conference in Grenada

Over 50 delegates from 13 Caribbean countries gathered for the First UNESCO Regional IFAP Conference in Saint George's, Grenada during the 15th and 17th June 2011. Under the theme "Building Caribbean Knowledge Societies", the Conference aimed to enable Caribbean countries to develop a realistic action plan that would address their development needs and foster the creation of knowledge societies in the region.

Introduction

The concept of Knowledge Societies is founded on the notion of societies which use information and communication tools and resources in a manner that is people-centered, inclusive, and equitable. In Knowledge Societies everyone can freely create, access, utilize, share and disseminate information and knowledge, so that individuals, communities, and peoples are empowered to improve their quality of life and to achieve their full potential.



International thinking about Knowledge Societies came to the fore during the World Summit on the Information Society (WSIS) in 2003 and 2005, which set a series of related goals for 2015. The WSIS follow-up framework provides today a forum in which multiple stakeholders including international organizations, governments, the private sector and civil society can discuss the opportunities of the new information and communication environment.



This consultation will combine the WSIS multi-stakeholder approach with the Information For All Programme (IFAP), which is an intergovernmental body of UNESCO created in 2001 to foster the creation of Knowledge Societies in Member States. IFAP supports knowledge-based processes for attaining national development goals by elaborating and implementing national information policy frameworks in the following five priority areas: Information accessibility, Information for Development, Information Preservation, Information Literacy and Information Ethics. In addition, by disseminating best practices, supporting projects in IFAP’s priority areas and other strategic activities IFAP addresses Member States’ needs in this area.



The event, organized by UNESCO in collaboration with the Government of Grenada, featured the following topics:

• Concept of knowledge societies in the Caribbean context;



• Role of IFAP in building inclusive knowledge societies;



• Outcomes of the World Summit of Information Society (WSIS); and



• UNESCO's work in the follow-up to WSIS.

The delegates are expected to adopt a political declaration on building knowledge societies in the Caribbean and to develop an action plan that would address the specific needs of Caribbean countries in the framework of IFAP. Isidro Fernandez-Aballi, from UNESCO's Kingston Office, expressed his satisfaction over the fact that UNESCO facilitates a conference of such importance in the region. According to him, the time has come for Caribbean leaders to think strategically about building knowledge societies and creating a culture of information for all.



Special guests at the Conference included our Prime Minister of Grenada and Chairman of CARICOM, Tillman Thomas, and the President of the UNESCO General Conference, Davidson Hepburn.

Trying to close the digital divide in the Caribbean

For the second time in a month, a senior Grenada government official has appealed to the region to “fully embrace” Information Communication Technology (ICT) as a tool for the development of the Caribbean region.



Minister of Finance and Acting Prime Minister Nazim Burke on the 15th June 2011 told participants attending the UNESCO Building Caribbean Knowledge Societies Conference that ICT will empower all citizens of the Caribbean Community (CARICOM) to improve their quality of life and achieve their full potential.

“There is little doubt that Information Communication Technology acts as a vehicle in bringing services to our people where ever they are. These services include e-banking, e-government, e-learning and a gamut of people-centered interactions,” he said. “Even more importantly, ICT brings with it a myriad of opportunities for employment creation, innovation and entrepreneurship hitherto unimaginable in our region.”

“We cannot and must not wait to be pulled along, kicking and screaming, by the forces of globalization and international competition. ICT and its related benefits have been with us for far too long, for our small countries to lose this opportunity for forward movement,” Burke implored participants.

The two-day conference brought together policy makers, ICT experts, civil society representatives, international organizations and other industry leaders to identify the issues and concerns that hinder the region from utilizing ICT to meet the stated objective of developing “knowledge societies.”



This conference cames on the heels of the May 24-27 thirty sixth meeting of the Council for Trade and Economic Development on Information and Communications Technologies, which discussed the draft Regional Digital Development Strategy and the Draft Regional Implementation Plan for adoption by CARICOM member states.



Minister Burke challenged the meeting to find ways of utilizing Information and Communication Technology to “build a society, in which all our people can access, create, utilize, share and disseminate information and knowledge.”



He noted that while the region has already conceptualized its own Knowledge Network, there is an urgent need to make it operational, with the implementation of CARIBNET in the CARICOM region and OECS Knowledge Network at the sub-regionally level.

“From all indicators, this region is yet to fully embrace the platform created by the ICT revolution. For us in Grenada, we see no meaningful option or alternative,” the Finance Minister stated, noting that the government views ICT as a “transformational sector.”

He outlined a number of steps government has taken that will help to move the sector forward.



Among those measures are the establishment of an ICT Center of Excellence and Innovation, an agreement with the World Bank for Grenada to serve as the pilot in the Caribbean Regional Communications Infrastructure Programme (CARCIP) under which the country will complete the establishment of its national ICT backbone and Grenada’s own Internet Exchange Point (IXP).



Burke said the government is also pushing ahead with plans, announced in its 2011 budget, for the establishment of a new public-private partnership to provide managed IT services, IT infrastructure management, managed network security services, connectivity solutions, data centre services and the programme management of telephony services.



"We cannot continue to run in analogue mode when the world is digital.



To build an effective knowledge society, we must bridge the digital the divide," Education Minister, Senator Franka Bernadine, said at the opening ceremony of the two-day conference at the Grenadian by Rex Resorts.



The conference focuses on building Caribbean knowledge societies.



One way to achieve this, Sen. Bernadine said, is to reduce the digital divide that limits regional nationals from optimizing their full potential.



It requires allowing them more than having access to technology, and greater effort by educators to alert people to the benefits of technology, the Grenadian Minister said.

"Having technology is just the first step.



When government puts laptops and desktops into the hands of teachers and students, they would have only completed the first part of the process," she said.



Other barriers will also have to be broken if we have to build knowledge societies.



It is hoped that the IFAP conference will help to develop a Caribbean action plan with well articulated policies to close the digital divide; aid the regional movement toward knowledge-based societies; and build awareness and understanding of the role IFAP."

IFAP was developed specifically by the executive of the United Nations Educational, Scientific and Cultural Organization (UNESCO) in 2009 to provide a framework for international corporation and partnership in building an information society for all.



IFAP was created "as a concrete response to the growing technological challenges and opportunities of the nineties which ushered in the age of an information societyand widened the digital divide," said Bryan Raloykow of UNESCO's Paris-based headquarters.



Conference participants, who are drawn from throughout the Caribbean, were urged to translate their deliberations into concrete measures.



"Unless ICT for development initiatives are people-centered and explicitly founded on human rights, they run the risk of reinforcing and exacerbating existing problems," said Davidson Hepburn, President of the General Conference of UNESCO.



You can download the related documents:-

• List of Actions for Building Caribbean Knowledge Societies



• Final Program IFA - Building Caribbean Knowledge Societies (revised 12 June 2011)

* UNESCO's Information for All Programme (IFAP) is an intergovernmental body created in 2000 to assist Member States in the formulation of national information policy frameworks. Through IFAP governments have pledged to harness the new opportunities of the information age to create equitable societies through better access to information.

Read More

Archive Talk - Birth dates and the British Empire

I always feel a mild sense of archivist euphoria when I encounter a document bearing my birth date: 9 July. Certainly the oldest such document I’ve uncovered while working on the ARCHIVE project lives at the Rosenbach Museum and Library: a 9 July, 1657 letter written to John Thurloe, British secretary of state under Oliver Cromwell, from one of his officials on the island of Jamaica. “Imagine,” I thought, staring at the letter, “this was written 330 years to the date before I was born!”



Then, when I read a bit of the letter and discovered it was a sobering missive relating early British colonizers’ attempts to “subdue” the island’s indigenous population. The letter-writer—apparently a British soldier named William Brayne—requests that Thurloe dispatch to the island “bloodhounds” to assist soldiers in “finding and killing” Jamaica’s “wild negroes.” The letter continues:

“I am Confident [that] if his Highness did but know how useful they [the bloodhounds] might be here he would cause some to be speedily sent” (Volume 3, p. 121).

The topic of the letter, though cold and detached, even clinical in the way in which Brayne discusses the topic made me appreciate how the Caribbean islands have grown and developed through a hard history even more. It’s not every day that you read a coolly written letter requesting the tools by which to police (some would say subjugate) an unstable civilization.



Perhaps I shouldn’t have been too surprised, given the years covered by the John Thurloe papers—1655 to 1660. By this time, the British Empire had established its dominion: in parts of the present-day United States, in many of the smaller Caribbean islands, and in Asia, Africa, and other regions. It had also, by this time, established and consolidated a number of trading companies, like the British East India Company, to administer the colonies and capitalize upon their economic possibilities. Furthermore, the Empire had just signed the Treaty of Westminster, ending the first conflicts in the Anglo-Dutch Wars, and was well-embroiled in the Anglo-Spanish War (1654-1660), which was sparked by commercial rivalry and resulted in the English takeover of Jamaica in 1655.



As an aside, more of a note of fact, the "wild negroes" referred to were a people traveling the smaller islands, stretching from St. Thomas to Tobago, a warlike and indomitable race, collectively known as "Charaibes" or "Caribs," who resisted every attempt at colonization on the part of Europeans, and preferred death to the withering slavery that became the fate of the natives of the larger islands. So stubborn was the resistance offered by these dauntless people that settlements in the places held by them were not effected until long after the other islands begun their flourishing plantation and civilized communities.  The group should not be confused with the race of people, who at this same period, lived Cuba, San Domingo, Jamaica, and the other larger islands in the West Indies, they were a mild and timid race, generally called Arouagues by Labat, Du Tetre and other French historians of the 17th century.



By the time the Protectorate collapsed and Thurloe lost his job in 1660, paving the way for the return of the monarchy and the further expansion of the British Empire, the “wild negroes” of Jamaica had been sold into slavery, exploited by British trade groups as free labor for the burgeoning sugar cane and coffee industries. (Enslaved Africans were also transported to the island.) A century later, the Jamaican slaves—who by then well outnumbered their white masters—mounted Tacky’s Revolt, an attempt to overthrown the colonial government. Untypically of many Caribbean islands, Jamaica have more than two centuries of violence and political upheaval.  However, with the changing politics of the Sugar world Jamaica, as with most Caribbean islands, could finally become independent and they achieved this on 6 August, 1962—a mere twenty-five years before my birth, in 1987.



The Thurloe papers at Rosenbach are chock-a-block full of interesting insights into Protectorate-era England (at least for those who can decipher seventeenth-century script).

Read More