Collaborative Manuscript Transcription: What does it mean to "support TEI" for manuscript transcription?

Collaborative Manuscript Transcription: What does it mean to "support TEI" for manuscript transcription? T-PEN is a case study here.

Seeking a New Developer

The Center for Digital Theology, a research unit in the Department of Theological Studies of Saint Louis University, seeks to hire a Senior Web Developer.  This successful candidate  will be part of a team who design web-based applications that support advanced research in the humanities.  Reporting to the Center’s Director, the Senior Developer will be responsible for backend development including database design and management as well as system architecture. The developer will be responsible for ensuring all applications meet W3C standards for interoperability.  Part of the position’s tasks includes acting as a liaison concerning technical operations for partnering institutions.

The Center seeks a creative and self-motivated developer who will take the initiative in system design and implementation.  S/He will have strong communication skill, possess the ability to work in a team, and work according to set deadlines.  S/He will work within an agile development method, participate in debugging and work closely with use case testers and contracting clients.

The successful candidate will have a minimum 2 years in software development with Java.  S/He will also have some experience with JavaScript. Knowledge of other coding languages (Ruby, Python, etc.) would also be an asset.

The salary range is $42,600-$64,000 and will be based on qualifications and experience.  This position comes with a full range of benefits offered to full-time employees of Saint Louis University.

Applicants can learn more about the Center at its website and see its most recent project at www.t-pen.org

Apply online at jobs.slu.edu/applicants/Central?quickFind=65031 (this link breaks, search by selecting Theological Studies from the Department drop down box).

If It's Tuesday, It Must Be Vienna: T-PEN Abroad

Now that T-PEN has had several months of public use and is functional and fun, we are turning our attention to another sort of building, not of software, but of collaborative relationships.  Although it is possible to upload privately held images under conditions of restricted access, we have always hoped that T-PEN would encourage better use of the many, many images of handwritten materials that libraries and manuscript repositories are making available on the web.  We remain inspired by the sometimes massive releases of digitised manuscripts and documents (one of our current favourites is Europeana Regia).  Seeing those images makes us want to transcribe the texts!  We wonder whether one of the consequences of the proliferating digital resources will be that texts will be liberated from the institutions and codices in which they have been cloistered.  Although the digital libraries allow unprecedented access to the texts, it is still rather awkward for scholars, editors, and creators of digital projects to work with them; we hope, of course, that T-PEN is a satisfying bridge between the images and further research or preparation of editions (remember, one can export from T-PEN in several formats!).

When T-PEN and manuscript repositories collaborate, scholarship can advance in powerful ways.  We can draw upon the profound expertise of the curators, cataloguers, and creators of edifying exhibitions; we can combine that expertise with the analytical, interpretative, and editorial work of academic researchers.

Abigail Firey, T-PEN's Co-PI,  made a little tour to meet with prospective partners, in hopes that we can realise this vision of bringing together digital library resources and those who use them.  With delight, T-PEN found -- over and over again-- wonderful generosity, interest, and energy in the repositories we approached.  Naturally, there are various legal and technical considerations yet to be addressed.  We cannot make definitive statements at this time, and even when the ink does start drying on the agreements, we shall first implement pilot projects of a dozen or so manuscripts to ensure that the data transfers are smoothly implemented.  Nevertheless, we would like to acknowledge the exceptional courtesy shown to our little dinosaur at

• the Bayerische Staatsbibliothek (BSB) 
• the Österreichische Nationalbibliothek (ÖNB)
• the Staatsbibliothek at Bamberg
• the Staatsbibliothek zu Berlin Preussischer Kulturbesitz (SPK)
• the Walters Art Museum
• the University of Kentucky Special Collections

We shall continue to build collaborative relationships in the coming months.  If you are involved with digitising a collection, or contemplating digitising one, and are interested in the potential of T-PEN to enhance the utility of the images you will be making public, please do get in touch with us.

Major Fix for T-PEN

Several recent posts and notices have explained trouble we've had with our T-PEN interface. Today the team was happy to push an out-of-cycle update to restore functionality for all browsers and implement some minor feature improvements. If your experience seems unimproved, you may need to clear the cache on your browser.

Thank you for your patience as we continue to improve the user experience.

Trouble for T-PEN: Known Issues and a Request for Feedback

At T-PEN, we spend lot of our time recently trying to keep up with the rapid development of modern web browsers. Transcription and parsing within a web browser is accomplished through a complex set of interactions that can lead to some frustrating problems when it does not behave as expected. Please read below for some known issues and how to provide the best feedback when you find problems we might not know about.

Known Issues

• Firefox focus - Recent versions of Firefox will not move the cursor automatically from one line to another when using keystrokes (Tab, Enter) or the mouse (Next Line buttons, clicking within the full page image). The interface updates correctly, showing the correct line, but the user must click within the textarea to begin typing.
• Chrome focus - Some installations of Chrome suffer the same problems as Firefox above, especially on Mac computers.
• Safari pageLoad - As mentioned in a previous post, when the transcription environment is first loaded in Safari, the textarea is missing altogether. After some awkward acrobatics (outlined in previous post), the page behaves appropriately, but still needs to be reset each time a new page loads.
• Opera display - Opera reports slightly different sizes for on-screen elements than other browsers causing a minor (1 or 2 pixel) displacement for lines and boundaries, causing small gaps in the parsing tool or minor shifts in position indications when transcribing. This does not have a significant effect under normal use. Though the issue is known and some corrections are already in place, there is more work being done.
• JavaScript errors - Several users have reported alerts or errors from the browser indicating that a particular script has broken or is trapped in an infinite loop. In cases where we have been able to repeat the problem, it was a simple and quick fix, but as these are inconsistent, we rely on feedback to begin looking for solutions.

Important Feedback

If something peculiar happens when you are using T-PEN, let us know.

How to Report

The "Contact" button

At the top right of the transcription workspace, there is a button labeled "Contact" which will send the user to the Account Management page, where a contact form is available. Options include Ask a Question, Suggest a Feature, or Report a Bug.

When reporting a bug, please include as much information as possible. At a minimum, try to provide the URL from the address bar of your browser, which will often include identifying information about the page and project you are accessing. Other helpful information includes:

• description of what failed or is broken - describe what happened and what you had expected to happen;
• text of any error messages displayed - exact messages are the most helpful;
• steps to repeat the problem - if you can make it happen repeatedly, it is the most likely to be fixed quickly;
• operating environment - the operating system and version of browser will help narrow down the scope of problems; and
• anything additional - if you have screenshots of the problem or have logs of the JavaScript errors... more information is always better.

*We may contact you about your report to get more information or let you know about a possible fix. If this contact is unwelcome, please let us know in the initial report.

What to Report

In a tool such as T-PEN, the threshold for reporting can be a challenge for the user. A good rule of thumb is this: 

If the tool does not behave as you expect, let us know what you expected.

Many of the features of T-PEN entered the workflow as bug reports from users who expected that it would be simple to move the workspace around the screen or keep a reminder to close XML tags. Most reports will fall into these categories:

• T-PEN is Broken - There is an error message or erratic behavior. The display may fail to put an image on the screen or make room for a tool with no content.
• This is frustrating - Errors such as the vanishing cursor (above) don't break the tool, but can lead to extra steps to complete work. By reporting these, we can either address a bug or start work on a needed feature.
• What happened? - Sometimes, there seems to be no cause, but something has gone wrong. Perhaps a folio transcribed the previous day turns up blank today or the text you were linebreaking before lunch looks corrupted when you returned to your work.
• I wish T-PEN could... - This is the most popular report and it may be a new repository to court or external tool to integrate. These invaluable suggestions make sure that T-PEN remains relevant for the most people.

Conclusion

My First Sandbox Environment

Throughout our development, our team has relied upon the input of real scholars doing real work. We understand the importance of communicating what we are working on so every user can work confidently, without fear of data or software abandonment. We not only welcome feedback - we require it. We invite you to leave a comment below, contact us on the website, or even (if you happen to be in Saint Louis, or run across a member of the team at large) tell us in person.

My name is Patrick Cuba and I am responsible for making your interaction with t-pen.org intuitive, reliable, and consistent. I check the reports every day, or you can write to me at cubap@slu.edu.

JavaScript Bug in Safari 5.1.5

Update: The latest Safari release 5.1.7 still has these issues. The T-PEN team is hard at work finding a more graceful solution for our Safari users. A separate update to T-PEN, however, means that once you have restored the interface, all the tools will work correctly.

The T-PEN development team has confirmed that there is a bug in the latest release of Safari, causing previously working pages to stop working. It seems that it can break sites that use some versions of java applets entirely, but is more frustrating on sites like T-PEN where piecemeal failures interrupt expected behaviors without offering fair warnings. In this case, Safari displays the image for transcription without any regions for text entry or functional tools.

In most installations, Safari updates automatically with no simple ability to rollback. The 5.1.5 release (beginning March 26) on Mac and Windows contain this bug. Since the applications update on a staggered schedule, it would be at some point over the following few weeks that the transcription page would stop displaying correctly. This has been the principal reason why it has taken us a while to determine the problem: being able to repeat a problem is essential in diagnosing it.

We are working on some fixes, and we hope in the same way that Apple quickly released 5.1.5 to fix problems with 5.1.4 (released March 12) they will fix this as well.

Chrome and Firefox still behave correctly. If switching browsers is an option, it is the best solution at this time. T-PEN is designed to function in all modern browsers and we are addressing this problem as a priority.

In the interim, this somewhat clunky workaround will restore full functionality, though the process must be repeated on each page.

1. Load up your page to transcribe;
2. Click on a tool and immediately press the ESC key;
• Right after you click on the tool, you should see the bar that splits the screen on the far right - you'll need to hit ESC before the bar fades out again
• If successful, you will see a split screen with the right side completely blank - the left side will show the correct display
3. Press ESC again to restore the screen to full size - clicking the tools will now behave correctly.

If you notice other bugs or display issues as you are using T-PEN, please let the team know by logging into T-PEN and clicking on the Contact Us link on the transcription or home pages. Regular bug fixes are explained in periodic release notes; updates to this issue will be posted here as well.

T-PEN version 2.0

T-PEN reached a milestone this month by releasing version 2.0 at www.t-pen.org.  We've debugged over 600 bugs during the last 18 months and have added numerous features.  For this release, we offer four substantial new features:

1. Users can now upload their own image set for transcriptions.  These sets must be comprised of jpeg files and have to be zipped together.  This kind of project is automatically designated as private.  The manuscript shelf mark is not listed in T-PEN's catalog and the project leader can add a maximum of  five additional users to the project.  The images are hosted on T-PEN under the terms of the Fair Use section of the US Millenium Copyright Act.  To upload your images, select the "Advanced' tab on the landing page and follow the instructions.

2. T-PEN now fully supports crowd-sourcing projects.  These projects can be fully public (so both the project attributes and content can be modified by anyone) or have limited access.  These projects are listed under the "Public Projects" tab on the landing page.  Any project can be designated as public using the options in the Project Management page.  All projects that have been T-PEN use cases at the the Center for Digital Theology will be listed as public projects (content will be readable only).

3. T-PEN has been providing access to support tools for transcribers.  These tools are available on a split screen display. To date we have provided access to various language dictionaries, a dictionary of Latin abbreviations, and the Latin Vulgate Bible.  We know that soon many T-PEN users will want tools for transcribing manuscripts that are not pre-modern Latin.  To that end, T-PEN now permits users to identify a URL for a tool that a user may want to have displayed.  Under the "options" tab on project management, a user can add the name and the URL of the tool and that will be added to the buttons on the Transcription User Interface.   Please note: Users should be aware that this "i-frame" set up does provide full browser functionality.  If you are using a tool which requires authentication in advance (especially if it is via an institutional proxy), we recommend you establish the authentication first in that browser before loading that tool.   You can do this by loading the tool in anther tab and completing the authentication process (login, etc.).  When T-PEN is loaded in the same browser, but different tab, that authenticiation will persist.

4. The final feature is experimental: Glyph matching, a palegraphical analytical tool into T-PEN.  This tool will compare the page you have in the Transcription UI to other pages of that MS.  At present, this tool only compares to pages that are in T-PEN's image cache, and so the entire manuscript will not be part of the comparison process.  As we develop this tool and increase the speed of the comparative work, full page to manuscript comparison will become available.  At present, T-PEN compares glyph to glyph.  Sometimes that will be individual letters, sometimes that will be ligatures or letters written very close together.  Matches are hightlighted in blue. If you click on them, you will see a list of the matches on other pages.  Clicking on any those thumbnail images will take you to the originating page so you can see that glyph in context.  It is our aim that this tool will facilitate analysis of changes in hand in a manuscript and eventually be way to build data sets that demonstrate common hands in two or more manuscripts.  The tool will be heavily developed over the Summer of 2012.

Re-Tooling T-PEN's Logo

As we get ready for the last stage of T-PEN's planned development, Patrick Cuba, T-PEN's GUI developer has been re-designing our logo.  He was assigned this task not only because he has greater artistic talent than Jim Ginther and Abigail Firey (who developed the initial logo), but also there was a lack of clarity about the IPR of the old T-Rex image itself.  We are therefore very pleased to headline our new logo here.

Our mascot holds both quill and scroll as he surges ahead ready to consume any and all manuscripts with scholarly mastication!  This image will sit on T-PEN's landing page with the next release due at the end of February 2012.

All other pages will have a more modest image:

The T-Rex image has been licensed from Mineo Shiraishi at dinosaurcentral.com .  The background image is from Cambridge, Corpus Christi College, MS 415, p. 171.

Release Notes for T-PEN v1.3

Release Date: 5 February 2012

Contact: ginthej@slu.edu
Site: t-pen.org

Bug Fixes in T-PEN v.1.3

Transcription UI:

• Restored the ability to zoom into a line with SHIFT + CTRL
• Resolved issue with some XML tags entering the Transcription stream without the angle brackets surrounding them or missing parameters

Linebreaking UI:

• Corrected some encoding issues which did not render non-alphanumeric characters as intended. Files of the .xml and .txt format can now be read and linebroken correctly.
• Now checks filename extensions to warn against uploading .doc and .docx files

Project Management:

• Resolved naming issue that prevented buttons from saving correctly if they had been reordered.
• Resolved an issue that was preventing private collections hosted by TPEN from modifying  image names by the manuscript's administrator.

New Features

Project Management:

1.     Export: User's first and last names are included with the exported information when submitted to a QENI Conduit.

2.     Button Management: When a user enters an invalid XML tag name in the tag field, a warning flag will alert the user of the rule being violated. It does not prohibit the use of invalid tag names, in case XML validation is not important to the user.

3.     Image Sequencing: Updates:

o    Select All & Invert Selection buttons

o    Compact list of pages with meaningful and color-clarified titles

o    Click to select, shift-click to select groups

o    Context-relevant controls in the middle for replacing, holding, or discarding groups of images

o    Always visible options to discard, hold, or preview images

Transcription UI:

1.     Display: A narrow workspace will display a condensed version of the transcription tool which features icons in place of buttons and navigation. If the browser window is widened or an oversized tool is closed, the workspace will return to normal.

2.     Lost Connection: if a connection to T-PEN is lost because of a session timeout, a server failure, or other communication issue, the page will alert the user to the problem and interrupt work that cannot be saved. In most cases, simply logging back in will allow the user to continue where they left off.

3.     New Annotation Tool: With the new Annotation Tool, users can create or update simple annotations on any project image. The Tool permits users to draw a rectangle around a significant area on a manuscript image and attach a text-only annotation. .

Release Notes for T-PEN v1.2

Release Date: 5 January 2012
Contact: ginthej@slu.edu
Site: t-pen.org 

Bug Fixes Implemented in T-PEN v1.2

Transcription UI:

• Corrected a math error in the handling of oversized lines that led to incorrect bounding by the interface especially for the last line of a column or page.
• “Changes Saved” message was obscuring manuscript image and could not be dismissed. Now, if the “Changes Saved” message does not leave of its own accord, a mouseover will dismiss it.

Transcription Preview:

• Resolved problem with preview tool not showing latest edits and also added real-time updating to preview tool, even if the tool is currently open.
• Preview tool now remains in sync with the transcription, even when lines are added or deleted with the parsing tool.

Linebreaking Tool:

• Corrected issue with the linebreaking tool that prevented the insertion of text when some button sequences were used.

Image Parsing

• Corrected issue that triggered the parsing tool when a page image was loaded. T-PEN had previously checked to see if the first line contained data; now all possible lines are checked for content.

New Features in T-PEN v1.2

Transcription UI: The previous line text will scale to the size of the screen. When on the first line of the page, available text from the last line of the previous page will be shown.

Visual Cue for Closing Tag Proximity (Transcription UI): Closing tags originating on previous pages are faded to indicate their distance from the current line of transcription.

Transcription Preview: The preview tool now applies a simple color filter to <tags> to allow for a more clear view of your transcription.

New Project Option Tools: Users can now add a French/English dictionary, a dictionary for Middle English and the Dictionary of Old English (subscription required). User can select which tools will be available in the Transcription UI’s i-frame. All tools can also be renamed by the user on the project management page.

Exporting Transcriptions: When exporting xml, users can now select or deselect all the tags for inclusion at once.

Line History (Versioning): T-PEN now distinguishes between all versions of any transcription line. Users can view previous versions of a given line. A change in a line is defined as addition or deletion of a single character (including white space) or modifications to the line parsing coordinates. A previous version can be selected to replace the current version. All versioning activity is recorded in the project log, which can be viewed on the project management page under the Collaboration tab.