Date and version

This document relates to version 2 of the Community Heritage Digit Archive and is current as at August 2013.[/reveal]

Introduction

This document provides an introduction to the Community Heritage Digital Archive which at the time of writing is used in the historic web sites of: Bow Brickhill, Hanslope, Olney, the Two village of Milton Keynes and Broughton, Yardley Gobion, and Yardley Hastings; all of which are members of Milton Keynes Heritage Association.  The primary audience for this document is anyone wanting to get an overview understanding of the architecture of the System, particularly anyone who might in future need to become involved in maintaining the System.

The Community Heritage Digital Archive is software which allows communities to store photographs and information from documents and index these in ways that allow visitors to interact with the community web site to explore the history of the community.  All information stored in the system is input using web pages which are specifically designed to be easy to use by people with limited experience of using computers.

The main components of the System are:

• a database which is used to store information and the indexes used in searches for information
• pages to be used by visitors to the web site to make enquiries and receive responses
• pages to be used by the team which inputs and amends information in the system.

The system is intended to be embedded within a parent web site to provide the archiving and indexing capability.  It is not intended as a complete web site in itself.  The above components reside on each community’s web site on the Internet (i.e. there is a separate database and separate pages for each community using the System).

The system is intended to be available at very low cost to the communities that use it.  It is not intended to be industrial strength software for a mission critical application.  Rather, it is conceived as software to support a hobby activity that results in information that is interesting to a wide audience, but is not critical.

User identities and passwords

The arrangements described above require user identities and passwords to be set up for each community group for each of the following purposes.

• access the database
• access web entry pages
• upload pages to the server

A document recording the ids and passwords for each of the groups set up to date will be handed to two coordinators of groups using the Archive software (subject to agreement of all), to be used in an emergency, such as might occur if no one was available with a required authority.

Referral database

In addition to the separate database set up for each community to store its data, there is one further database.  This supports the referral of visitors between the web sites of communities, as described below.

Currently referral only operates for census information.  If the “where born” field in the census displays the name of another community which is using the Community Heritage Digit Archive, a message is displayed and the visitor has the option to open a page with the results of a search for the person named in the census in the other community web site.

This functionality is provided by JavaScript in the census detail page which uses an XMLHTTPRequest to send a message to the Referral database where a check is made whether the “where born” name is that of a participating community, and if so the web address of that community is returned.

The referral database has two tables:

• one to hold the web address of each participating community
• the other to hold each alternative spelling of the community name.

When a new community is added, a record must be added to the first table by a coordinator.  Communities can add the alternative spellings of their town/village name using their web entry pages (using web entry page masterReferral.php accessed from main menu Reference Data item, then “alternate spelling of name” on page that appears).  Communities should only do this when they have census data available.

Setting up a new Community

Appendix  D contains a checklist of tasks to set up a new installation for a community and help them get started.

The main steps are as follows.

1. Set up an empty database with initial user ids and passwords, load empty table definitions  For MKHA members this is done using Host-It windows admin panels
2. (optionally) Set up phpMyAdmin and/or Access links to the database with password
3. Load visitor and web entry pages to the community’s web site.  This required FTP software with passwords to access the community’s web site.
4. Create the link between the web pages and the database in page coWebEntry.php by adding the database name and password.
5. Tailor the appearance of visitor pages and, separately, web entry pages using the facilities provided in the web entry pages.
6. Add village parameters to the database using the web entry pages.  Parameters include name of community, name of village, logo and favicon files, names of pages to return to on the community web site when visitor leaves the Community Heritage Digit Archive pages.
7. Add Google analytics codes if the community uses Google analytics to provide statistics of web page usage.

Then it is a matter of selecting a source to work on (census, church records, etc.), and learning by doing data entry.  The pages are designed to be intuitive to use, but it undoubtedly easier to have someone on hand with experience to help a new team get started.

If a community already has data in digital form, it should be possible to load that data to the database.  An effort will be required to “map” the data available to the fields available in the database tables.  Generally, it is best to get data onto spreadsheets, then upload to the database from the spreadsheets.  Loading can be done via either phpMyAdmin or Microsoft Access.  Access is probably easier to use, but phpMyAdmin may handle large volumes faster.

 

A. Tables used in Community Heritage Digit Archive

Table	Key	Type	Content
accountsBooks	bookID	source	each accounts book
accountsDetail	entryID	source	each entry in accounts book
accountsGroup	groupID	source	grouping of accounts books (for use where there are several similar books, such as rent books of one landlord – enables summarisation across group rather than book alone)
buildingData	buildingID	building	each building
buildingImages	itemID	building	link to each image of building in imageDetail
buildingPeople	masterID	derived	each person identified with a building
buildingStreet	streetSideID	building	each “street”, treating two sides of a street separately, to allow sequencing of buildings on a street
buildingSubdivision	buildingSubID	building	each subdivision of a building (apartments within a block; each hereditament with a terraced houses)
censusDistrict	districtID	source	each district within a census year (allows possibility of different arrangement of districts between census years)
censusHousehold	householdID	source	each household within a census district
censusPeople	recorded	source	each person with a household
censusYear	year	source	each census year
churchBaptisms	registerID	source	each baptism record within a book of baptism records
churchBooks	bookID	source	each book of church records
churchBurials	registerID	source	each burials record within a book of baptism records
churchMarriages	registerID	source	each marriage record within a book of baptism records
churchRegister	registerID	source	each record within a book of mixed baptism, marriage and burial records (typically early books with records on unformatted pages)
churchRegisterLines	type	technical	entry for each type of record allowed in table churchRegister (used to format forms for data entry)
directoryData	directoryID	source	each business directory
directoryDetail	entryID	source	each person mentioned in a business directory
docsBuildings	recorded	source	each building mentioned in a document
docsDetail	itemID	source	each document (for documents for which there is no specific provision, such a census, wills, etc); expected to included deeds, press cuttings, booklets, catalogues, and many others)
docsImages	imageID	source	each image of a document (typically one for each page)
docsKeywordIndex	recorded	derived	each keyword records for each document; used to provide fast response to keyword searches
docsPeople	peopleID	source	each person mentioned within a documents
docsPeopleBuilding	entryID	source	link of persons mentioned in a documents to a building mentioned in a document (e.g. linking an owner of a tenant to a building)
docsRelated	entryID	source	link of one document to another related document
electionRegister	entryID	source	each person included in an election register
imageDetail	imageID	source	each photograph
imageKeywordIndex	recorded	derived	each keyword records for each photograph; used to provide fast response to keyword searches
imagePeople	itemID	source	each person in a photograph
imageSet	setID	source	convenient grouping of photographs, to facilitate selection of photographs for viewing (loosely equivalent to a photo album)
imageSetContent	imageSetID	source	each photograph in each image set
mapData	mapID	source	each map to be displayed using Google maps software
mapMarkers	itemID	source	each marker on each map
masterAltFamNames	itemID	reference	each alternative spelling for each family name
masterAltFstNames	itemID	reference	each alternative spelling for each first name
masterArchives	sourceID	reference	each Archive in which documents are stored; mainly used by documents tables; to be used by wills tables
masterCounty	countyCode	reference	each county (and some countries); used in census (where born county) and in wills.
masterFamilyNames	itemID	derived	representation of alternative spellings of family names for used in fast look-up responding to enquiries (required because masterAltFamNames will have no entry for names which have no alternative spelling)
masterFirstNames	itemID	derived	representation of alternative spellings of first names for used in fast look-up responding to enquiries
masterParameters	itemID	reference	on entry for parameters used to tailor system for a village/community; ranges from village name to colour and appearance.
masterPeople	masterID	derived	each reference to a person in each source table
masterReference	recordID	reference	users authorised to perform web entry, including password which is encrypted
masterRelationships	relationCode	reference	two digit relationship codes used to represent relationships such a wife, son, niece, etc; use mainly in census for relation to head of household and in wills
masterSource	sourceID	technical	each major source of information (census, church records, etc); also each “other” source recognised in table docsDetail
masterSourceEvent	itemID	technical	each type of entry recognised within each source (e.g. with marriage source: bride, groom, father of bride, groom, witness)
monumentData	monumentID	source	each churchyard gravestone or other monument
monumentImages	imageID	source	each image of a monuments
monumentPeople	personID	source	each person mentioned in a monument
otherPeople	otherPeopleID	source	each entry of sundry information about a person (intended for information such as what happened to people who leave the community)
schoolBooks	bookID	source	each register book of school records
schoolRegister	entryID	source	each entry in a register
surveyData	surveyed	source	each survey (main use for 1910 Inland Revenue survey)
surveyDetail	entryID	source	each person/property entry in a survey
willData	willID	source	each will
willImages	recorded	source	each page image of a will
willPeople	entryID	source	each person mentioned in a will

 

The 58 tables in the database can be grouped as follows.

1. source tables used to store data according to type of source (38 tables):

• census data stored in 4 tables
• church records information stored in 5 tables (churchRegisterLines is classified as a “technical table” – see below)
• accounts records stored in 3 tables intended for old accounts books, including rent books
• directory records in 2 tables for business directories
• document records in 6 tables for use for any document not specifically provided for under other headings, including deeds, court records, newspaper cuttings, booklets and catalogues
• election registers in 1 table
• map information in 2 tables
• monument information in 3 tables for gravestones and similar monuments
• photographs in 4 tables which provide for grouping photographs into “sets” rather like photo albums, but allowing one photograph to be included in several sets
• other people information in one table for sundry information about a person that does not fit anywhere else
• school books records from school registers in 2 tables
• surveys and rate records in 2 tables
• will information in 3 tables

1. building information in 4 tables which provide for storing architectural and historical information and indexing photographs.  The table also provide for sub-divisions of buildings such as a terraces or a tenement blocks.
2. master reference data shared by several of the above areas, stored in 7 tables
3. derived tables which summarise information from the above areas to provide faster response to enquiries
   • a table, masterPeople with a record for each person mentioned in any of the Source tables listed above, and a link back to the source table
   • a table buildingPeople with a record for each person mentioned in any Source table above who can be identified with a specific property, and a link back to the source table
   • two tables with alternative spellings of family names and first names, respectively
   • image and document keyword indexes
4. technical tables used to provide flexibility to make changes to the software functions (3 tables: masterSource, masterSourceEvent, and churchRegisterLines)

The diagram below illustrated as the tables and the relationships between them.

B.  Web pages

Page versions

The lists on the next page records the version number of each page at 26 August 2013.  To open a spreadsheet containing these lists click on this icon:

 

 

Pages used for Web Entry

The diagrams below illustrate the relationships between the pages for web entry for each source of information, and provide an indication of the role of each page.

 

C.  Change Requests

Click on the link below to open spreadsheet summary of enhancement requests.

 

D.  New Installation Task

These note are intended to provide a checklist of tasks to set up a database, install pages, and tailor the site to integrate with a village’s own pages.

Creating the Database

This section assumes the database will be set up as part of the Milton Keynes Heritage Association (MKHA) web site hosted by Internet Service Provider Host-IT.  If another Internet Service Provider is used, the steps here must be modified in consultation with that provider.  The minimum requirement is that there is access to a MySQL database.  It is also highly desirable that the tool phpMyAdmin is provided to access the database.

This step requires access to a user name and password to access the Host-It controller.

Host-It controller user name:_______________________    password:___________________

These can be obtained from the chairman of MKHA.

1. Open the Host-It web page:

https://secure3.host-it.co.uk/

and enter the user name and password

1. Click the Domain icon on the User Main Menu
2. Click the mkheritage.co.uk domain name on the Domains page
3. Click the Database Manager icon towards the bottom of the Domain Menu
4. Click the Add New button on the Databases page
5. Select MySQL radio button, enter a database name, and click the Save button on the Databases page.  Keep a record of the name set up for the database name:

Database name:________________________________________

1. Make a note of the Connection Information (MySQL Host) on the Update Database page.

Connect information (server name):______________________________________

1. Click the Add New button in the Database Users box of on the Update Database page.
2. Enter the database user name and password supplieddatabase user name:________________________    password:_______________________and click the Save button on the Add Database User page.
   (This name will be deleted once the database set up – see step 18 below).This returns to the Update Database page.
3. Add further users, as required.  At least one must be added.
   Very few user names should be set up.  These are only used for accessing the database directly.  The users here are not the users of the web site.  It may be sufficient to set up two users:

1. A person who will be the coordinator for the web site.
2. A name and password to be used by the site web pages to authenticate their access to the database.

Keep a record of the user names and passwords set up.

Database user name	password

 

1. Click the word ‘here’ in the sentence: ‘See here for phpMyAdmin panel’.

This opens a phpMyAdmin page.

Keep a record of the web address of the phpMyAdmin page:

Web address:_____________________________________

Or add it to your favourites.

1. Click the name of the database just set up in the left panel.
2. Click the word Import in the menu near the top of the right panel.
3. Click the Browse button and locate the file masterALL_YYYYmmdd.sql
   (YYYYmmdd is date file created) which has been supplied.
4. Click the Go button in the bottom right on the page.
   The database table should now load.
   Note any error messages that appear.
5. Check the following:
   1. 55 tables have been created, starting with accountsBooks, and ending with willSources.
   2. The following tables should hold data:
      (Check by clicking on the database name in the left hand panel.  The tables should be listed in the right panel, with the number of records shown in the column Records.  To see the content of a table, click on the browse icon, the first icon on the line beside a table name.  If there is no data, the structure of the database will appear in the right panel.)

1. censusYear should hold 8 records for each of census years from 1841 to 1911
2. churchRegisterLines should hold 8 records.

• masterAltFamNames and masterAltFstNames should hold the alternative spellings of names developed at Hanslope. If it is preferred to start new content from scratch, the content of these tables can be deleted by first clicking on the table name in the list of all table in the left panel, then clicking Empty in the menu at the top of the right panel, then OK in the confirmation popup.

1. masterCounty should hold approximately 175 records of allowed county and country names.
2. masterReferences should hold one record (a user name to allow initial access to set up more names. The original name should be deleted later – see below).
3. masterRelationships should hold approximately 125 records of allowed relationship which can be recorded between people

• masterSource should hold approximately 17 records of sources of information which the system can display
• masterSourceEvent should hold approximately 69 records of “events” related to the various sources in the previous table

1. surveyData should hold one record, for the 1910 survey

All other tables should be empty.

1. (After clicking the database name in the left panel, so the tables are listed in the right panel)
   Below the tables in the right panel, there should be a list of  approximately 23 Routines, starting with clearIndex and ending with nameIndexFirst.  (These routines are used to update the derived tables after new data has been entered.)  It may be necessary to click the word Routine to cause the list to appear, depending on the version of phpMyAdmin.
1. Log out of phpMyAdmin by clicking the Exit icon near the top of the left panel.  Close this window.
2. Delete the temporary database user name set up in step 9 above.
   1. Get to the Host-It  Update Database page:This may still be open.  However, Host-It may have logged out automatically.  In which case log in again by following steps 1 to 4 above.  Then click on the name of the new database on page Databases to arrive at the Update Database page.
   2. Click on the temporary name entered in step 9 in the Database Users box
   3. Click the Delete button on the Update Database User page.
   4. Click the OK button on the confirmation popup.
3. Log out of the Host-It controller page by clicking on the words Log Off in the very small menu bar just below the page titles.  Close this window.

Using Microsoft Access to view the Database

It is possible to use Microsoft Access as a means of examining the tables in the database.

This is only recommended for those who already has Access on their PC, and who are already familiar with Access.

To use Access in this way, the tables in the new database are “linked” as “external tables” to Access.  This requires the setting up of the new database as an ODBC Data Source.

The step below are for Access 2002 running in Windows XP.  They are likely to be slightly different for other versions of Access and Windows.

To do this open Access, then follow the steps below.

1. Select File – New, then click Blank Database.  A File New Database panel appears.
   1. Enter a file name for the Access file.
   2. Select a folder in which to save the file
   3. Click the Create button.
2. Select File-Get External Data – Link Tables.  A Link panel opens.
3. In the ‘File of type’ at the bottom of the panel select ‘ODBC Databases()’ from the drop-down list.  A Select Data Source panel appears.
4. Select the machine Data Source tab, then click the New button.  Another panel opens with a wizard to guide the next steps.
5. Click the ‘System Data Source’ radio button; click Next button.
6. Select MySQL ODBC 5.1 Driver from list that appears in another panel; click Next button.

If this driver does not appear on the list search the Web for “mysql odbc 5.1 download” which should find a MySQL site which allows the driver for Windows to be downloaded.

1. Click the Finish button.  Several panels close and a new panel opens headed MySQL Connector/ODBC.
2. Complete the form:
   1. Data Source Name is any name you care to give: say: ‘Web database’
   2. Description is optional; say: ‘Link to xxx database on web’ (xxx is your name for database)
   3. Server: enter the name recorded in step 7 of  Creating the Database above
   4. User: enter the name set up in step 10b of  Creating the Database above
   5. Password: enter password set up in step 10b of  Creating the Database above
   6. Database: enter name set up in step 6 of  Creating the Database above
   7. Click the Test button to check the software can make the connection with the information provided.  If there is an error, check all the information entered.  After that ….
   8. Click the OK button.  The panel ‘Select Data Source’ should be exposed, with the source just added now in the Machine Data Source list.
3. Select the Data Source Name for the Machine Data Source list; click the OK button
   A Link Tables panel should appear listing all the tables in the MySQL database.
4. Select the table to be shown in Access by clicking on each.  When all required tables are selected, click the OK button.

Note that as a result of the above steps, the new database tables should be visible in the Access Tables panel.  The data is still held on the server.  No data is held in Access.  All Access holds is information about how to link to the tables.

The tables may now be opened in Access and the content of tables can be changed, just as local tables are.  Queries can be written using the query-by-example features of Access.

Obviously, care should be taken not to introduce error to the database by ill thought out changes.

It is always better to make changes with the facilities provided on the user pages.

Loading the pages

This section assumes the new pages will be added to an existing web site which is part of the MKHA site hosted by Internet Service Provider Host-It.  If this is not the case the steps below must be adapted to whatever environment is intended.

The pages are provided in two folders: visitorPages and WebEntry.  Each folders is intended to be loaded as a child folder of the folder in which the index page of the web site is stored.

Load the new pages using the normal method used for loading pages, such as FTP.

Load one folder at a time.  The content of that folder should be loaded with it.

If there is an existing folder with the same name as one of the folders to be loaded, load the new pages for this folder into the existing folder.

Spreadsheet pageVersions.xls contains list of pages provided by folder.

Tailoring the pages

Tailoring is by changing the following pages:

• coWebEntry.php in folders visitorPages/shared/ and webEnry/shared/
• styles3.php in folder visitorPages/shared/
• weStyle.php in folder webEntry/shared/
• analytics.php in folder sharedFiles

The role of each of these pages and what needs to be done to tailor them is set out below.

coWebEntry.php

This file contains the information which all other pages use to connect to the database.

Use any text editor (such as Notepad, or GoLive) to add the following to this file:
(in each case between the quotation marks “”)

• hostname: enter name set up in step 7 of  Creating the Database above
• database: enter name set up in step 6 of  Creating the Database above
• username: enter the name set up in step 10b of  Creating the Database above
• password: enter the corresponding password set up in step 10b of  Creating the Database above

Save the file and load it to the server.

To test the connection, try to load the following file:

http://www.mkheritage.co.uk/xxxx/census/censusPageSelect.php

where xxxx is replaced by the site address within MKHA site.

If the connection is established, the drop-down box for year should be showing each of the census years from 1841 to 1911.  If they do not appear, the connection has not been established.  Check each of the pieces of information entered to coWebEntry.php, and make sure the changed page has been saved and loaded to the server.

If that does not solve the problem, you will need help.

style3.php and weStyle.php

These two files contain the styles information to change the colour and appearance of pages.

• styles3.php is used for pages seen by visitors to the site
• weStyle.php is used for the page used by team members for web wntry

Both these pages contain php code which loads session variables into the style sheet, most to modify colours.  These session variables are loaded at the beginning of a visitor or web entry session by page vvParameters.php in the shared folder.  The page vvParameters.php is loaded into the first page the used accesses when starting a session.  Page vvParameters.php reads the variables from the database table masterParameters.

It is recommended that initially, changes to colour and appearance are made using the web entry pages under the menu item Reference Data.  If this s sufficient, there will be no need to modify the style sheet directly.

The style sheet have some notes on them.  Probably the best way to modify them is to experiment, initially making small change which are easily visible, say to colour.  It will be much easier to make these changes using an HTML/CSS editor which can display colours to choose, rather than trying to find the numbers that represent a colour.

analytics.php

This file provides a Google analytics code to each visitor page, to enable Google analytics to accumulate statistics on page usage.  If the site is using Google analytics, the site account code with Google must be inserted in the file.  Otherwise the code on this page should be commented out, as delivered.

Adding links to the pages in existing web site

To make these pages available to visitors to the web site, links to them must be added in the existing pages.  In addition, it is probably best to add some complete introductory pages which act as linking pages.

The most important pages which are candidates to be link to are:

• for persons: personSearch.php which is the page by which a user submit search criteria to find a person
• for images: imageSelect.php which allows users to select images to view, by either selecting a ‘set’, or by entering a few keywords
• for buildings: buildingStreetSelect.php which lists streets for the user to choose one to move on to thumbnails of the houses in the street as a means of selecting a building.  If the alternative method of selecting from a map is to be used, a page will be required to hold the map, and hotspots added to create the links.  Another alternative is to import a Google map (see example on Sherington web site).

The following pages could also be linked, but if they are not, the information will be available by other means:

• accountsSelect.php which allows the user to select a book of accounts to review
• censusPageSelect.php which is the starting page for searching the census only.  It asks the user to select a census year, district and page to be displayed.
• churchBooksList.php which will list the books available and allows the user to click on a book to open the first page of the book.
• directoriesList.php which allows user to select a directory to view
• docsFind.php which allows the user to search for a document
• electRegister.php which allows the selection of an electoral register to view
• mapSelect.php which lists maps available to view and allows selection of one
• monumentResults.php which list all monuments such as tombstones
• poorRatePage.php to select a poor rate book to view
• willList.php which lists all wills indexed and allow the user to select one to view
• 1910Page.php  to view the 1910 Inland Revenue survey