View source for TechnicalSpecification

= Overview =
Based on the requirements identified as part of the Functional Specification, the AnnoSys system will be implemented along the Technical Specification described in this document. Thereby, AnnoSys will be built up on following interacting system components:
* Repository
* Data Sharing
* Message System
* User Interface
* Security

[[Image:Technical_Specification_Overview.png|frame|none|''Overview of the technical specification draft'']]

The repository component provides functionalities to persist any annotations and a copy of the relating XML record document as presented to the user at creation time. The data sharing component provides service interfaces to enable external clients or services to conduct searches within the repository or to retrieve any kind of data from it. The task of the message system component is to organise information flows such as informing annotators, curators or other users about current status changes in annotation workflows. A web based, but desktop oriented user interface shall enable annotators, curators and users to intuitively manage partly complex annotation workflow functionalities. Finally, the security component is responsible for secure authentication, authorisation and data privacy concerns within the system.

The following chapters will first enlighten the annotation context model. Based on that context model, the underlying data model and data exchange format will be introduced. Follow-up, a detailed description of the system components will be given.

= Annotation context modelling =
AnnoSys' annotation context modelling approach founds on the presumption that annotations will be related to any kind of data records being uniquely identifiable by [http://wiki.tdwg.org/GUID Globally Unique Identifiers (GUIDs)]<ref name="TDWG_GUIDs">TDWG, ''Welcome to the Globally Unique Identifiers (GUID) Wiki'', http://wiki.tdwg.org/GUID (2009), accessed 22 Oct 2012<br/></ref><ref name="TDWG_GUID_AS">Richards, Kevin; ''TDWG GUID Applicability Statement'', 09 September 2009, http://www.tdwg.org/standards/150/download/, accessed 19 December 2011<br/></ref>). Beyond that, it is presumed that these records can be made accessible as XML documents presented in a well defined standard format like [http://wiki.tdwg.org/twiki/bin/view/ABCD/ ABCD]<<ref name="TDWG_ABCD">Holetschek, Jörg, ''ABCD - Access to Biological Collection Data'', http://wiki.tdwg.org/twiki/bin/view/ABCD/ (02 March 2010), accessed 05 Dec 2011<br/></ref> or [http://rs.tdwg.org/dwc/index.htm Darwin Core]<ref name="TDWG_DarwinCore">Wieczorek, John, Döring, Markus, De Giovanni, Renato, Robertson, Tim, Vieglais, Dave, ''Darwin Core'', http://rs.tdwg.org/dwc/index.htm (2009), accessed 05 Dec 2011<br/></ref>. Several use cases like e.g. annotating geolocations require combining multiple record data elements with each's new value proposals and/or an annotator comments into a single annotation. That is, the annotation's context model has to integrate general meta information determining e.g. the annotated record with an aggregated list of annotated elements proposing new values and/or making natural language comments on a selected set of data record elements.

Therefore, in AnnoSys the annotation context will be defined by 
* GUID of a unit for which a relating data record exists (tripleID)
* An XML representation of the data record
* A selection of XML elements (XPath expression) within the XML data record representation affected by the annotation

The following sections will introduce the AnnoSys' interpretation and implementation of the  above mentioned annotation context elements.

=== GUIDs in AnnoSys ===
With regard to biodiversity collections, the term GUID initially reflects data records describing physical unit objects in museum collections. As such, a GUID is usually represented as a tripleID constructed by the following elements:
* institutionID
* collectionID
* unitID
As such, TDWG XML record document standards like [http://wiki.tdwg.org/twiki/bin/view/ABCD/ ABCD]<ref name="TDWG_ABCD"/> or [http://rs.tdwg.org/dwc/index.htm Darwin Core]<ref name="TDWG_DarwinCore"/> define elements reflecting that data triple.

In general, AnnoSys' basic annotation workflow starts by downloading and analysing an XML record document. One part of the analysing process the system conducts is to extract the tripleID from the document and compare it with the most recent document version available for that tripleID within the repository. If there are any differences detected, a new revision of the data record will be assumed and thus, a new version for the related XML record document has to be added to the repository by the system.
The latter is required since any changes within the XML record document may invalidate any annotations created before. Thus, annotations in AnnoSys must be connected with a specific version of the annotated XML record document. Therefore, AnnoSys uses [http://www.omg.org/cgi-bin/doc?dtc/04-05-01 LSIDs]<ref name="OMG_LSID">TDWG; ''LSID Authority Identifications'', www.omg.org/cgi-bin/doc?dtc/04-05-01, ???, not accessible 19 December 2011<br/></ref><ref name="TDWG_LSID_AS">Pereira, Ricardo; Richards, Kevin; Hobern, Donald, Hyam, Roger; Belbin, Lee; Blum,Stan;''TDWG Life Sciences Identifiers (LSID) Applicability Statement'', 03 September 2009, http://www.tdwg.org/standards/150/download/, accessed 19 December 2011<br/></ref> that extend the former introduced notion of tripleID by the desired version information. 

The same information may be available in different data formats. AnnoSys can only correctly reproduce annotations, if the standard of the corresponding annotation's XML record document has also been recorded. Therefore, AnnoSys again has extended the LSID specification by adding the namespace URI of the annotated XML record document. Thus, currently a persistent identifier within AnnoSys is defined as follows:
* urn:annosys:institutionID:collectionID:unitID:version:format

While, the URN namespace "annosys" has been chosen to avoid different interpretations of the term guid, subsequent redefinitions or adoptions to other persistent identifier schemas may be implemented instead.

As all namespace specific strings forming the GUID URN may contain special characters, it is required to transform them according to [http://tools.ietf.org/html/rfc2141 RFC 2141]<ref name="RFC_2142">Moats, R.;''URN Syntax'', May 1997, http://tools.ietf.org/html/rfc2141, accessed 24 October 2012<br/></ref>.

=== XML Data Record representations ===
AnnoSys expects XML data record documents in one of the following formats:
* ABCD 2.06
* Darwin Core 1.2

Other formats or versions may be added as required.

=== Annotation context selection ===
An annotator may identify several items within a data record to be worth annotating. While AnnoSys is planned to work with XML record documents, a mechanism to identifiy the corresponding data elements within the relating XML document has to be implemented. Evaluation results turn out that the usage of XML diff representations or inserting annotated values and/or natural language comments as XML comments into XML record documents results in either cryptic result representations, low performance or conceptual difficulties. Therefore, AnnoSys introduces the idea of a mechanism realising a sort of "recorder" functionality to capture any XML data record elements addressed by annotators.

As XPath  [http://www.w3.org/TR/xpath20/ XPath]<ref name="W3C_XPath">Berglund, Anders, Boag, Scott, et al., ''XML Path Language (XPath) 2.0 (Second Edition) - W3C Recommendation 14 December 2010 (Link errors corrected 3 January 2011)'', http://www.w3.org/TR/xpath20/ (14 December 2010), accessed 05 Dec 2011<br/></ref> expressions are universally applicable within the XML domain, regardless of the XML data record document format, XPath can be used to define adequate expressions selecting (sets of) record data elements. These selectors may uniquely identify a single dedicated data element, but can also encompass sets of data elements like 
* "all elements named "xxx" located below the element "yyy"" 
* "all elements named "xxx" with a value "zzz""
The range of use cases includes e.g. typo corrections in a single data element to mass annotations of errorneous geographic location references. 

'''Note:'''
XPath filter expressions may become highly complex. So please note that not any XPath (e.g. manually or automatically produced) filter expression can be adequately presented in a graphical user interface presenting an annotation's differences in relation to the original document.

= Data Model =
The data model specified within the next sections will mainly be built up on the results evaluated in chapter [[#Annotation context modelling|Annotation context modelling]]. As such, annotations will be described within the data model by two interrelated parts:
* Annotation Metadata Specification
* Annotated Elements Specification
While the metadata specification provides general meta information (e.g. determining the annotated record and document), it  will be augmented by a list of elements recording the annotation's context selections. The following sections will provide more detailed information on both parts.

[[Image:Annotation_Elements.png|frame|none|''Annotation Data Model'']]

== Annotation Metadata Specification ==
Annotation metadata provide general information about
* annotation creation
* annotated record
* annotating agent
* optional annotation aspects

Except optional annotation aspects, any data described within the following sections is mandatory in order to instantiate a valid annotation.

=== Annotation creation ===
Any creation of an annotation entails the generation of the following data elements
* Motivation
* Datetime (of the creation)
* GUID (of the annotation)

Usually, annotations will be proposed for well determined reasons. In the first instance, these reasons were thought to be included as "annotation type" outlining particular intentions of the given modification or comment. Inspired by the [http://wiki.tdwg.org/twiki/bin/view/AnnotationsIG/DataExtensionsToAO extensions]<ref name="TDWGAnnotationsIG_AODExtensions">Morris, Paul J., ''Proposed AOD Extensions to AO'', http://wiki.tdwg.org/twiki/bin/view/AnnotationsIG/DataExtensionsToAO (06 July 2011), accessed 05 Dec 2011<br/></ref> proposed by the [http://wiki.tdwg.org/AnnotationsIG/ TDWG Annotations Interest Group]<ref name="TDWGAnnotationsIG">Morris, Paul J., ''TDWG Wiki AnnotationsIG'', http://wiki.tdwg.org/AnnotationsIG (2011), accessed 05 Dec 2011<br/></ref> the term motivation was introduced synonymously to the former notion of annotation type. The term shall express the reason for which an annotator has made the annotation. Currently, AnnoSys supports the following motivations (annotation types):
* Determination
* Locality of duplicates
* Gathering
* Nomenclatural type
* Sequence
* Record basis
* Scientific name
* Label text
* Other

The creation's datetime describes the point in time the annotation was published by the system. This time is usually different to the time, the annotation was created within the system for user processing. The time's granularity is given in milliseconds.

The GUID of the annotation shall uniquely identify any annotation created. In accordance to the extended [[#GUIDs in AnnoSys|GUID interpretation in AnnoSys]], the GUID's "institutionID" should match the organisation running the AnnoSys instance creating the annotation. The "collectionID" should match the name of the annotation repository (e.g. "AnnoSys"). While "unitID" expresses here the time in milliseconds since 01/01/1970 the annotation was first instantiated in the system for processing, revision is represented by the time in milliseconds since 01/01/1970 the annotation was published by the system. Finally, the format is determined by the namespace URI identifiying the standard format and version used to publish the annotation (e.g. Open Annotation: http://www.w3.org/ns/openannotation/core/).

=== Annotated record ===
The term annotated record circumscribes information regarding the underlying data record representation of the annotated unit:
* record GUID
* record document format
* record document source
* record document revision
The recordGUID identifies the collection unit, on which any kind of data records may rely. In general, that recordGUID is expected to be represented in tripleID format as described in section [[#GUIDs in AnnoSys|GUIDs in AnnoSys]]. Optionally, other formats may be supported.

The record document format describes the format of the annotated record document. It is expected to be represented by the namespace URI of the relating XML standard format including its version (e.g. ABCD: http://www.tdwg.org/schemas/abcd/2.06http://www.tdwg.org/schemas/abcd/2.06).

The document source states the source from which the annotated record document has been downloaded. Usually, this will be represented in URL form. Optionally, any URI denoting any standard transport or retrieval protocol like FTP, or DOI's may be supported.

The revision information reflects any kind of version information regarding the annotated record document. If no revision information can be extracted from the record document based on the given document format, the time in milliseconds since 01/01/1970 when downloading the document first into the system will be set as revision information.

=== Annotating agent ===
An agent can be either a person or a machine conducting the annotation. Conforming to the common practice in botanical museums, only a minimum set of data has to be captured:
* name
* email
* institution

The name field should include first name(s) and last name(s) of the annotating person, or the name and version of the software or service used to create the annotation.

The email address should hold a valid email address of the annotating person, or the email address of the annotating machine's administrator.

The institution denotes the organisation the annotating agent belongs to. If the institution field is empty, the authorisation component will consider the agent as member of the "private taxonomist" group.

=== Optional annotation aspects ===
The provision of the following aspects is not mandatory for annotations. Mainly, they provide additional information regarding the annotation management processed by the collection manager or the access control component of the issuing annotation system.
* evidence
* expectation
* constraints
The "evidence" aspect provides a means to justify an annotation by substantial proof arguments. For instance, the evidence may be provided by a list of references, links to multimedia content or other instruments of evidence. It is a point for discussion whether evidence should be provided entirely or to each annotated element separately.

Using the "expectation" aspect, an annotator may express the outcome of the annotation especially with regard to the holder(s) of the annotated record unit(s). Mainly, the expectation aspect is part of this data model for conformance reasons to the proposal of the [http://wiki.tdwg.org/AnnotationsIG/ TDWG Annotations Interest Group]<ref name="TDWGAnnotationsIG">Morris, Paul J., ''TDWG Wiki AnnotationsIG'', http://wiki.tdwg.org/AnnotationsIG (2011), accessed 05 Dec 2011<br/></ref>  Anyway, in terms of AnnoSys the general expectation might be assumed to express the annotators' willing that the curator enters the annotated data into his collection database.

The "constraints" aspect may be used by annotators to define constraints (e.g. endangered species, planned scientific publication), to prevent e.g. unwanted publication, unintentional usage, time, copyright or other restrictions assigned to that specific annotation. This aspect mainly applies to the system creating annotations on behalf of the annotator. That is, authorisation restrictions must apply before the system is showing annotation data to users or exchanging annotation data with other services.

== Annotated Elements Specification ==
According to the definition described in chapter [[#Annotation context modelling|Annotation context modelling]], any entry on the annotated element list mentioned earlier combines the following information to propose a new value and/or comment on a selected set of data record elements to the data collection manager:
* Element selector (XPath expression)
* Value proposal
* Text comment

While both, "value proposal" and "text comment" are optional, one of both must be given in order to provide valuable information. That is, annotation data that does neither include a value proposal nor a free text comment has to be considered as invalid.

The "element selector" will be realised in form of an XPath expression according to section [[#Annotation context selection|Annotation context selection]]. Therewith, the following special use cases will be interpreted by AnnoSys' selector implementation as follows : 
# If the XPath expression selects an XML element for which no value is assigned in the relating XML Schema, only a text comment may be given in order to make an annotation in natural language. 
# Selecting the root element ("/") will be recognised as to address the entire data record. 

"Value proposal" enables annotators to propose a new value to any XML element selected by the element selector. While the intended usage is to propose the replacement of the selected XML element values by the given value, curently the value proposal is expected to be represented as text string.
'''Note:'''
An empty value proposal will be interpreted as the annotator's propose to remove all selected elements from the record.

"Text comment" permits annotators to annotate or to make comments on the selected set of record elements in natural language. Text comment is also considered to be represented as text string.

= Exchange Format =
In comparison to AnnoSys' evaluation phase, the activities of the [http://www.w3.org/community/openannotation/ W3C Open Annotation Community Group]<ref name="W3COpenAnnotationCommunityGroup">W3C, ''W3C Community and Business Groups - Open Annotation Community Group'', http://www.w3.org/community/openannotation/ (2012), accessed 25 Oct 2012<br/></ref> alleviate the decision towards a suitable storage and exchange format for annotations significantly. [http://www.w3.org/community/openannotation/ W3C Open Annotation Community Group] pools forces of two former initiatives:
* [http://code.google.com/p/annotation-ontology/wiki/Homepage Annotation Ontology]<ref name="AnnotationOntology">Google, ''annotation-ontology'', http://code.google.com/p/annotation-ontology/wiki/Homepage (2011), accessed 05 Dec 2011<br/></ref>
* [http://www.openannotation.org/spec/beta/ Open Annotation Model]<ref name="OpenAnnotationBetaModel">Sanderson, Robert; Van de Sompel, Herbert, ''Open Annotation: Beta Data Model Guide'', http://www.openannotation.org/spec/beta/ (10 August 2011), accessed 25 Oct 2011<br/></ref>.
The consolidation of both initiatives under the umbrella of W3C prospects for a long term sustainability of the upcoming standard. As such, the specifications [http://www.openannotation.org/spec/core/ W3C Open Annotation Core Data Model]<ref name="OpenAnnotationCore">Sanderson, Robert; Ciccarese, Paolo; Van de Sompel, Herbert, ''W3C Open Annotation Core Data Model - Community Draft (09 May 2012)'', http://www.openannotation.org/spec/core/, accessed 25 Oct 2012<br/></ref> and [http://www.openannotation.org/spec/extension/ W3C Open Annotation Extension Specification]<ref name="OpenAnnotationExtension">Sanderson, Robert; Ciccarese, Paolo; Van de Sompel, Herbert, ''Open Annotation Extension Specification - Community Draft (09 May 2012)'', http://www.openannotation.org/spec/extension/, accessed 25 Oct 2012<br/></ref> provide powerful means to express any of AnnoSys' basic requirements defined earlier. Additionally, they provide for a bunch of further concepts for potential system enhancements. 

Even though AnnoSys' primary focus is on XML record documents, W3C Open Annotation is a semantic approach defining an ontology for annotation use cases in general. Therefore, annotation data has to be stored using [http://www.w3.org/RDF/ RDF]<ref name="W3C_RDF">W3C RDF Working Group, ''Resource Description Framework (RDF)'', http://www.w3.org/RDF/, 10 February 2004, accessed 13 Dec 2011<br/></ref> terminology. Due to the early stage of W3C Open Annotation Community Group, the further directions of standard specification can actively be influenced by providing use cases to the W3C Open Annotation Community Group. As such, some extension proposals discussed within the [http://wiki.tdwg.org/AnnotationsIG/ TDWG Annotations Interest Group]<ref name="TDWGAnnotationsIG"/> and AnnoSys' implementation concerns have already raised group discussions and influenced decisions toward the next standard release scheduled for the end of 2012. Thus, W3C Open Annotation is the standard AnnoSys will support and contribute to.

As mentioned above, the upcoming version of the W3C Open Annotation will yield some substantial changes. In particular, these changes will provide solutions to problems not adequately solveable with the current Community Draft 1. The following diagrams will show the current interpretation of Open Annotation Community Draft 1 as implemented in the first AnnoSys protoype. Due to the pending changes, it will not be discussed in detail. Nevertheless, some comments regarding necessary extensions and the mentioned implementation issues will be given.
[[Image:OpenAnnotation-simplified.png|frame|none|''W3C Open Annotation representation of Annotation Metadata'']]

While the application of W3C Open Annotation regarding the annotation meta data defined in the AnnoSys data model could have been implemented in a quite straight forward manner, the realisation of the list of annotated elements caused some problems. 

First, AnnoSys introduced the extension ''XPathSelector'' simply representing XPath expression as a new selector type for specific target resources. 

Second, W3C Open Annotation supports addressing multiple (specific) targets. Unfortunately, it lacks to offer an opportunity to relate different bodies to individual targets by limiting the total number of bodies in an annotation to one. Thus, AnnoSys had to create multiple OA annotations in order to represent the data model's list of annotated elements. The relationship between the annotated elements and the original annotation holding the metadata information has been realised by "misusing" the ''oa:hasSemanticTag'' property "linking" annotated elements to meta annotation. As a consequence, a number of unrequired annotation instances would have to be created within the RDF model. The upcoming standard release will permit multiple bodies and provide means to related bodies to specific targets.

[[Image:OpenAnnotationElement-simplified.png|frame|none|''W3C Open Annotation representation of Annotated Elements'']]

= Repository =
The [[#Exchange Format|former chapter]] defines [http://www.w3.org/community/openannotation/ W3C Open Annotation Community Group]<ref name="W3COpenAnnotationCommunityGroup"></ref> as exchange and storage format for annotations in AnnoSys. While W3C Open Annotation produces metadata in RDF format, the record documents are predetermined to be delivered in standardised XML formats (like ABCD or Darwin Core). Both kinds of data (record documents and annotations) are closely related to each other. Thus, both have to be archived within the AnnoSys repository in order to be precisely reproducable at some point in time later. Besides XML and RDF data, the AnnoSys repository also includes user profile information including login credentials and authorisation details.
== Record documents ==
With regard to the experience gained from the current implementation of the BioCASE annotation system, the file system based, hierarchical storage architecture used here proofed to be an acceptable solution for efficiently storing XML record documents. Therefore, the AnnoSys repository stores XML record documents in a configurable location on the server's file system as well. Using ABCD terms for the unit's tripleID the following file path creation scheme is used.
* $RECORD_BASE_DIR/SourceInstitutionID/SourceID/format/UnitID.revision.xml

Thereby, the tripleID information (SourceInstitutionID, SourceID and UnitID) will be extracted from the XML record document in due consideration of the relating standard format. While the format will be represented by the XML namespace prefix of the record document's XML namespace URI, a predefined mapping scheme between namespace URI and namespace prefix will be used by all AnnoSys system components. Finally, the revision information will describe the datetime when the XML record document was first downloaded into the repository. According to respective data elements defined within the [#Data Model|Data Model], the revision's value will be given by time in milliseconds since 01/01/1970.

Optionally, if required for some substantial reasons, record documents may also be stored in an dedicated XML database in later revisions.

== Annotations ==
As AnnoSys uses [http://www.w3.org/community/openannotation/ W3C Open Annotation Community Group]<ref name="W3COpenAnnotationCommunityGroup"></ref>, which is an RDF based standard format to manage annotations, annotations will be stored in so called RDF stores.
RDF stores (aka. triplestores) can be viewed as specific databases for storage and retrieval of RDF data (triples). Comparable to relational database systems, information stored in a triplestore can be retrieved using the [http://www.w3.org/TR/rdf-sparql-query/ SPARQL]<ref name="W3C_SPARQL">Eric Prud'hommeaux and Andy Seaborn (2008). SPARQL Query Language for RDF.<br/></ref> query language.

Some recent evaluation reports and RDF store benchmarking studies <ref name="EUROPEANA_RDFSTOREREPORT">Bernhard Haslhofer, Elaheh Momeni, et al. (2011). Europeana RDF store report.<br/></ref><ref name="BSBM_2011_02">Chris Bizer and Andreas Schultz (2011). "BSBM V3 Results (February 2011) ". Retrieved 15 February 2012, 2012, from http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/results/V6/index.html.<br/></ref><ref name="BSBM_2009_03">Chris Bizer and Andreas Schultz (2009). "Berlin SPARQL Benchmark Results." Retrieved 15 February 2012, 2012, from http://www4.wiwiss.fu-berlin.de/bizer/BerlinSPARQLBenchmark/results/index.html.<br/></ref> provide a serious overview of suitable RDF store implementations. In particular, these reports evaluate that native stores are significantly outperforming RDBMS backed RDF stores. Even though, these benchmarks show that with regard to native RDF stores individual strengths and weaknesses have to be considered. In AnnoSys, the focus can be seriously narrowed down to the most relevant and performing open source solutions supported by the Apache Jena<ref name="APACHE_JENA">The Apache Software Foundation (2012). "Apache Jena - Welcome to Jena." Retrieved 15 February, 2012, from http://incubator.apache.org/jena/.<br/></ref> framework.

The benchmarks show that Apache Jena's [http://incubator.apache.org/jena/documentation/tdb/ TDB]<ref name="APACHE_JENA_TDB">The Apache Software Foundation (2012). "Apache Jena - TDB." Retrieved 15 February, 2012, from http://incubator.apache.org/jena/documentation/tdb/.<br/></ref> model is an adequately performing solution for development and prototype implementations. As  the number of triples to be stored approaches 50 millions, the most performant triple store turned out to be [http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/ Virtuoso Universal Server]<ref name="VIRTUOSO">OpenLink Software (2011). "Virtuoso Open-Source Edition." Retrieved 15 February 2012, 2012, from http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/.<br/></ref>. The latter will be the case if the number of annotations approximates about 500.000 annotations. Though, initially AnnoSys will start implementing using the Apache TDB triple store, and reserves switching to Virtuoso if the system's performance suggests to do so.

The following sections will provide a brief introduction to both triple store implementations.

=== [http://incubator.apache.org/jena/documentation/tdb/ TDB] ===
TDB<ref name="APACHE_JENA_TDB"/> is the native triplestore implementation of the Apache Jena framework. Thus, it is supported by the full range of Jena APIs and is advertised by Jena as "high performance RDF store on a single machine". Nevertheless, benchmarks are showing that TDB has quite a good performance regarding the import of triples, but query and update performance is just adequate.

Due to the file based approach of TDB, replication, backup or transactions are not directly provided by TDB. On the other hand, SPARQL and full-text queries as well as inference engines are well supported through the Jena API.

=== [http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/ Virtuoso Universal Server] ===
Virtuoso Universal Server<ref name="VIRTUOSO">OpenLink Software (2011). "Virtuoso Open-Source Edition." Retrieved 15 February 2012, 2012, from http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/.<br/></ref> represents a hybrid storage solution, which can be used for RDF as well as for XML, relational data and free text documents. Through its unified storage, it can serve as an integration point for heterogeneous data sources. While it is offered as an open source product, there are several license models for commercial usage as well. Virtuoso is known for hosting the large datasets of e.g. [http://dbpedia.org DBpedia]<ref name="DBPEDIA">Pablo Mendes (2011). "The DBpedia Knowledge Base." Retrieved 15 February 2012, 2012, from http://dbpedia.org.<br/></ref>. 

Benchmarks show that Virtuoso outperforms in most query and update disciplines. Apparently, the advantage has grown in the last few years. Regarding loading times, Virtuoso shows linear scaling, but is to some degree slower than other competitors.

Further on, Virtuoso supports transactions, replication, backup, inference engines as well as SPARQL and full-text queries. Moreover, Virtuoso offers back-end providers for being seamlessly integrated with Apache Jena.

== User Profiles ==
The term "user profiles" aggregates the following information concerning user agents:
* metadata of annotating agents
* authentication credentials
* private annotation store
=== Metadata of annotating agents ===
"Metadata of annotating agents" addresses the storing of information derscribed within the [[#Annotating agent | Data Model]]. In consideration of the best practices recommendation provided in the [http://www.openannotation.org/spec/extension/ W3C Open Annotation Extension Specification]<ref name="OpenAnnotationExtension">Sanderson, Robert; Ciccarese, Paolo; Van de Sompel, Herbert, ''Open Annotation Extension Specification - Community Draft (09 May 2012)'', http://www.openannotation.org/spec/extension/, accessed 25 Oct 2012<br/></ref>, the FOAF vocabulary will be used to describe agents as well as institutions. The metadata will be stored along with the annotation data with AnnoSys' triple store. Also, any of these resources will be referenced within AnnoSys according to the extended [[#GUIDs in AnnoSys|GUID interpretation in AnnoSys]]. Thereby, the URN of an agent resource in AnnoSys might be similar to the following expression:
* urn:annosys:bgbm:annosys:1347628344528:1347628344528
Please note that if user profile data changes, a new resource with an updated version datetime will be generated in order to retain annotator information as they were available at annotation creation time.

Furthermore, an agent resource may be represented by the following properties:
{| class="wikitable" style="text-align: left;"
! property
! definition
|-
| rdf:type || agent type (foaf:Person, foaf:Organization or dcTypes:Software)
|- 
| foaf:name || agent's name
|- 
| foaf:mbox || agent's mailbox
|-
| foaf:member || optional list of members (for agent type foaf:Organization only)
|}

=== Authentication credentials ===
Authentication credentials contain the login information for a given agent, as well as a copy of the current agents' metadata information. These information will be stored in a SQLite database where the '''users''' table defines the following columns:
{| class="wikitable" style="text-align: left;"
! column
! description
|-
| uid || agent's login name
|- 
| credential || agent's password
|- 
| resourceid || agent's resource URI in AnnoSys triple store
|-
| name || agent's name
|-
| mailbox || agent's mailbox
|}

=== Private annotation store ===
AnnoSys maintains a private annotation store for any registered agent within the system. Initially, these stores hold any annotation data created by the user. Agents must explicitly publish annotations in order to move these annotations from the agent's private store to the AnnoSys triple store and make them generally accessible. After publication, the annotation will be removed from the agent's private triple store.

The private triple stores will be created in a separate directory named according to the agents resource id. Thereby, any colons (":") will be replaced by underscores ("_") in the directory name. The private triple stores will be created using Apache Jena's TDB implementation.

= Data Sharing =
In order to permit external services accessing annotations stored in the AnnoSys, there must be interfaces to access RDF based annotation information as well as XML record documents. Internally, annotations refer to XML record documents through the GUID defined for the XML record document instance as described in the [[#Record documents | repository]] specification. The next sections will describe how access is provided to both kinds of information.
== RDF ==
Access to the AnnoSys triple store will be given by an open [http://www.w3.org/TR/rdf-sparql-query/ SPARQL]<ref name="W3C_SPARQL"></ref>. endpoint. A SPARQL endpoint enables clients to conduct any SPARQL query on the AnnoSys triple store over common web protocols like HTTP. Therefore, Apache Jena (Fuseki) as well as Virtuoso provide those SPARQL endpoints.
== XML ==
Along with the relating annotations, the AnnoSys repository for XML record documents must be opened for queries executed by external services as well. This is a necessary feature, because the representation of annotated elements depends on the XML record representation available at the time of annotation creation. There, the following options have been considered:
* WebDAV/FTP connection to repository files
* BioCASE Provider Software
Providing [http://webdav.org/ WebDAV]<ref name="WebDAV">L. Dusseault, Ed., ''HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV) (June 2007)'', http://tools.ietf.org/html/rfc4918, accessed 07 Nov 2012<br/></ref> or [http://tools.ietf.org/html/rfc959 FTP]<ref name="FTP">J. Postel, Ed.; J. Reynolds, ''FILE TRANSFER PROTOCOL (FTP) (October 1985)'', http://tools.ietf.org/html/rfc959, accessed 07 Nov 2012<br/></ref> services to enable direct read access to the file based hierachy of the record repository is an easy and lightweight solution. The mapping scheme from a record document's GUID to the relating file path in the record repository is described in section [[#Record documents| Record documents]]. Thus, external services simply have to implement that simple mapping scheme in order to access annotation related record documents. 

Alternatively, the [http://www.biocase.org/products/provider_software/ BioCASE Provider Software] implementation has to be adopted in order to enable the delivery of XML record documents from the record repository's file hierachie.

= Message System =
The message system provides mechanisms to inform system actors like collection managers(curators), annotators or other users about the current status of various annotation workflow processings. The message system supports notifications for the following annotation workflow activities:
* inform collection managers whenever annotations referencing their collection(s) has been issued
* inform annotators regarding the acceptance or rejection of issued annotations on behalf of the collection manager
* notify users about ongoing workflow activities

The following sections provide an overview of the technical components used to implement the AnnoSys message system and describe the integration of the AnnoSys message system concept into the technical environment.

== Technical components ==
Technically, the message system builds on the [http://activemq.apache.org/ Apache ActiveMQ]<ref name="APACHE_ACTIVEMQ">Apache Software Foundation (2011). "ActiveMQ." Retrieved 30 January, 2012, from http://activemq.apache.org/.<br/></ref> which is a provider implementation of the [http://www.oracle.com/technetwork/java/jms-136181.html Java Message Service (JMS)]<ref name="ORACLE_JMS">Oracle. "Java Message Service (JMS)." Retrieved 30 January, 2012, from http://www.oracle.com/technetwork/java/jms-136181.html.<br/></ref>. Thereby, the implementation of system external message transport protocols like email will be realised through [http://camel.apache.org/ Apache Camel]<ref name="APACHE_CAMEL">Claus Ibsen and Jonathan Anstey (2010). "Camel in Action", Manning.<br/></ref>. The following sections will briefly introduce the basic concepts of Java Message Service(JMS), Apache ActiveMQ and Apache Camel.

=== Java Message Service (JMS) ===
The [http://www.oracle.com/technetwork/java/jms-136181.html Java Message Service (JMS)]<ref name="ORACLE_JMS">Oracle. "Java Message Service (JMS)." Retrieved 30 January, 2012, from http://www.oracle.com/technetwork/java/jms-136181.html.<br/></ref> standard was developed by the Java Community Process and defined within the specification JSR 914<ref name="ORACLE_JSR914">Rich Burridge Mark Hapner, Rahul Sharma, Joseph Fialli, Kate Stout (2002) Java Message Service.<br/></ref> . JMS is an application programming interface (API) for message communication between one or more software components or client applications.

The term messaging describes a loosely coupled form of reliable, asynchronous, distributed message exchange between software components. Herein, senders don't need to have precise knowledge of their receivers and vice versa. Nevertheless, the message format must be arranged by communication partners. Further on, communication partners are not required to be online at the same time. The reliability term determines that a message is delivered once and only once, even in case of system failures.

JMS provides two communication models:
* point-to-point
* publish/subscribe
Using point-to-point communication, the producer sends a message to a queue which is connected to a dedicated receiver. If the receiver is currently not available, the message will be stored by the message service and the receiver can fetch it when he reconnects.

The publish/subscribe model requires the publisher to create a message topic, where an arbitrary number of clients can subscribe to. Messages sent to that topic must then actively be consumed through subscription or will be lost. Optionally, subscribers may decide for durable-subscription. Thereby, messages will be persistently stored by the message service and redistributed to subscribers on every reconnect.

In addition, JMS provides several types of messages to be sent to queues or topics like text, byte streams or (mapped)Java objects. 

In order to use JMS, a JMS provider implementation like Apache ActiveMQ is required providing functionalities to manage topics, queues and sessions. JMS roviders may operate in stand-alone or embedded mode. While in stand-alone mode, the JMS provider runs in its own process, it will run in the same process (Java Virtual Machine(JVM)) as client applications like e.g. AnnoSys.

=== Apache ActiveMQ and Camel ===
Apache ActiveMQ is one of the most popular open source JMS provider implementation and fully compliant to the JMS API specification. It also includes an administration tool permitting to maintain queues, topics and messages externally. Moreover, it supports different transport mechanisms like e.g. TCP(OpenWire), SSL or VM(in-process connections for embedded broker) and integrates different platforms and programming languages like Java, C/C++ or .Net.

In combination with [http://camel.apache.org/ Apache Camel]ref name="APACHE_CAMEL">Claus Ibsen and Jonathan Anstey (2010). "Camel in Action", Manning.<br/></ref> a bunch of more than 70 network transport protocols (e.g. email, news-feeds, FTP, HTTP) can be configured to transport messages by using [http://camel.apache.org/components.html Apache Camel Components]. Apache Camel routes messages from or to destinations (e.g. queues, topics) to communication endpoints defined by URIs.

Thereby, [http://camel.apache.org/ Apache Camel]<ref name="APACHE_CAMEL"></ref> provides a Java API completely supporting any of the 65 [http://www.enterpriseintegrationpatterns.com/eaipatterns.html Enterprise Integration Patterns(EIPs)]<ref name="EIP">Gregor Hohpe and Bobby Woolf (2003). "Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions", Addison-Wesley. ISBN 0-321-20068-3.<br/></ref> to configure routing and mediation rules. EIPs represent a catalogue of accepted solutions to recurring problems in enterprise applications. In particular, messaging, routing and transformation solutions are addressed like e.g. content based message routing.

== Integrating annotation workflows and Java Message Service (JMS) ==
Basically, the AnnoSys message system requires the creation of message queue instances for
* agents
* records
* topic subscriptions
As naming of JMS queues and topics is based on free text any naming scheme could be introduced to map specimen records or annotations related messages to individual destinations. It is common practice using dot based separators to indicate hierarchical levels in naming context (e.g. Java packages, domain names), adopting a similar schema for annotation queues might also be fine. Thus, AnnoSys names agent and record message queues according to their GUID representations.

Agent message queues serve as point-to-point communication with users and will be backed by email endpoints through the relating camel component. That way, agents, curators and topic subscribers will be notified by email of events they have been registered or subscribed to. Optionally, any other message transport protocol provided by Apache Camel (e.g. news-feeds) may be integrated.

Messages considering annotated records or annotations will be directed to topics named analogously to their corresponding GUIDs. 

Due to this GUID based schema, event driven topic subscriptions related to collections, records or annotations can be managed through wild-card selectors supported by ActiveMQ. That way, subscriptions like "keep me informed about events regarding unitID x in any institution and collection" (e.g. "*.*.unitID"). 

Through further extensions of that schema, e.g. by annotation types, users might subscribe for issued annotations of a specific type (e.g. in general "*.*.*.*.*.annotationType" or record related "instId.collId.1234.*.*.annotationType"). Alternatively, that feature could also be implemented through content based routing rules in Apache Camel. However, for more specific and record data oriented topics like genus, geographic locations etc., a suitable approach will have to be developed and is subject of further research.
=== Message types ===
AnnoSys includes the following messages types:
{| class="wikitable" style="text-align: left;"
! message type
! definition
|-
| ANNOTATION_ISSUED || annotation issued by an annotator
|- 
| ANNOTATION_ACCEPTED || annotation accepted by collection manager
|- 
| ANNOTATION_PARTLY_ACCEPTED || annotation partly accepted by collection manager
| ANNOTATION_REJECTED || annotation rejected by collection manager
|}

ANNOTATION_ISSUED type messages will automatically be generated and sent to the related collection manager as well as to the record's topic. All other messages types have to be explicitly submitted by collection managers when they have updated (or not) the collection database record related to a given annotation. Therewith, they may either reject the annotation, or accept it by completely or partially updating the corresponding data record.

While annotators can be automatically addressed through their user profile data, collection managers will have to be recognised based on their role approved by the AnnoSys authentication system. Thus, whenever a new annotation is issued, the message system requests a current list of users who are registered as curators for the given collection from the authentication system. A similar request to the authentication system ensures that only users registered as curators for a given collection may issue acceptance or rejection messages towards annotators.

For any message type, notifications will not only be sent to the curator(s) or annotator(s) respectively, but also to the message queue of the annotated records. That way, those status messages are automatically propagated to related topic subscribers.

= User Interface =
Providing and managing annotations may become a complex job either to annotators or data curators. For instance, annotators have to gather specimen data, analyse a potentially large number of data objects and annotate or comment specific data elements accurately. Again, data curators have to analyse these annotations, accept or reject them possibly along with individual comments and have to reintegrate them back into their data collections. Further on, any of these activities should be documented and communicated back to annotators and other subscribing members of the community.

As the main focus of both annotators and curators should be on preparing accurate information, a suitable user interface should give advice and support users through all these workflows. Furthermore, it should allow a clear presentation of information, adoption to individual user preferences and provide convenient guidance and support through the most common workflow activities. While common web interfaces are doing their job quite well, they are feeling mostly a little bit static and inflexible in comparison to modern user interfaces of desktop applications. 

Fortunately, due to recent web toolkit developments based on the [http://www.adaptivepath.com/ideas/ajax-new-approach-web-applications Asynchronous JavaScript and XML (Ajax)]<ref name="AJAX">Garrett, J. J. (2005, 18 February 2005). "Ajax: A New Approach to Web Applications." from http://www.adaptivepath.com/ideas/ajax-new-approach-web-applications.<br/></ref> technology, desktop alike applications can be implemented for the web now as well. Therefore, AnnoSys decided to use [http://eclipse.org/rap/ Eclipse Rich Ajax Platform (RAP)] for developing the desktop oriented user interface.

The aim of the [http://eclipse.org/rap/ Eclipse Rich Ajax Platform (RAP)]<ref name="ECLIPSE_RAP">The Eclipse Foundation (2012). "Enabling modular business apps for desktop, browser and mobile." Retrieved 08 February, 2012, from http://www.eclipse.org/rap/.<br/></ref> is to make the [http://eclipse.org/rcp/ Rich Client Platform (RCP)]<ref name="ECLIPSE_RCP">The Eclipse Foundation (2012). "Rich Client Platform." Retrieved 08 February, 2012, from http://www.eclipse.org/rcp/.<br/></ref> available for the web. That is, desktop applications originally developed with Eclipse RCP can be executed within a client's web browser. Currently, some parts like individual manipulation of drawing context and working with styled text components are not achieved. However, any other kinds of widget required to realise comfortable user interfaces are available. 

Eclipse RAP applications are completely running in an Java EE server container. Thereby, AnnoSys uses the Jetty 8 platform as recommended by the Eclipse RAP developers.

= Security =
The security component tasks comprise the following functionalities:
* verified user registration including user profile information
* secure user login
* role based authorisation
As the validity of each user's email address is a prerequisite for the communication workflow within the system, the verification of the claimed address is the basic functionality of the user registration procedure. Moreover, the registration procedure is accompagnied by appropriate measures to reduce spam bot registrations. With respect to data privacy protection, the repository stores user profile data in a location inaccessible from system external services.

A password based, secure user login will be provided in order to seriously ascertain a user's identity. Optionally, logins initiated by federated identity management systems such as SAML or OpenID may be integrated if seriously requested.

AnnoSys implements functional access restriction to certain functions based on a user's role assignment. In particular, AnnoSys defines the following roles:
{| class="wikitable" style="text-align: left;"
! role
! definition
|-
| annotator || default role enabling annotating workflow activities
|- 
| administrator || role for system administrators
|- 
| manager_collectionX || role for curators of a given collection X
|}

The technical demand is an easy integratibility with other AnnoSys components like repository, message system and user interface. Since any of these components are realised based on the [http://www.oracle.com/technetwork/java/javaee/index.html JavaEE]<ref name="JAVA_EE">Oracle; ''Java EE at a Glance'', http://www.oracle.com/technetwork/java/javaee/index.html, 2012, accessed 04 January 2012.<br/></ref> platform, in particular the solution must be interoperable with Java Web Application Containers like Jetty. Based on evaluation results, AnnoSys prefers [http://shiro.apache.org/ Apache Shiro]<ref name="APACHE_SHIRO">The Apache Software Foundation (2012). "Welcome to Apache Shiro." Retrieved 20 February, 2012, from http://shiro.apache.org/.<br/></ref> towards the common solution [http://docs.oracle.com/javase/6/docs/technotes/guides/security/jaas/JAASRefGuide.html Java Authentication and Authorisation Service (JAAS)]<ref name="ORACLE_JAAS">Oracle (2011). "JavaTM Authentication and Authorization Service (JAAS) Reference Guide." Retrieved 20 February, 2012, from http://docs.oracle.com/javase/6/docs/technotes/guides/security/jaas/JAASRefGuide.html.<br/></ref>. 

The main advantage of Apache Shiro is its easy to use, understand and configure production quality security framework supporting authentication, authorisation, session management and cryptography. In particular, Apache Shiro facilitates the definition of complex authorisation rules and session management issues.

= References =
<references />