View source for Download Metrics

=Download Metrics in EPrints=

==Capturing Usage==

Eprints uses the Apache web server and mod_perl (an accelerator for Perl CGI scripts). mod_perl provides hooks to the logging and control features of the Apache server. One of these hooks is to the access logging function. A ''log handler'' is registered with Apache by using the ''[http://perl.apache.org/docs/2.0/user/handlers/http.html#PerlLogHandler PerlLogHandler]'' pragma e.g.

<code>PerlLogHandler EPrints::Apache::LogHandler</code>

This handler is called after a response has been generated by the web server (i.e. in response to requests coming in from web clients).

In the initial implementation we are only interested in requests to eprint objects. This ignores requests to static content (e.g. the home page), searches and user-specific content (e.g. depositing processes). This is because our current requirements are to capture usage to determine download metrics for objects contained in a repository, rather than analysing how a repository is used.

For the remainder of this section we will use the following example request (as you would otherwise see in an Apache access log file):

<code>127.0.0.1 - - [25/May/2007:13:07:24 +0100] "GET /12614/01/Semantic_Web_Revisted.pdf HTTP/1.1" 200 130951 "http://www.w3.org/2001/sw/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"</code>

The pertinent parts of this entry are:

{| class="wikitable"
|+Fields in a HTTP request.
|-
!field
!example
|-
|requesting host
|127.0.0.1
|-
|date/time of request (by server time)
|25/May/2007:13:07:24 +0100
|-
|page requested
|/12614/01/Semantic_Web_Revisted.pdf
|-
|HTTP response code (200 is 'ok')
|200
|-
|referring web page
|http://www.w3.org/2001/sw/
|-
|user's web browser (or web crawler id)
|Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
|}

In the Eprints log handler any non-200 ''HTTP response code'' requests are ignored i.e. any errors or redirects. A simple heuristic is applied to separate out the requests into three potential types: non-eprint requests, eprint abstract requests and full-text requests. This is based on the ''page requested'' field:

{| class="wikitable"
|+Parts of a page request URL.
|-
!align="center" |eprint identifier
!align="center" |document identifier
!align="center" |file requested
|-
|/12614/
|/01/
|Semantic_Web_Revisted.pdf
|}

Requests without an eprint identifier part are ignored. Requests with no document identifier part are defined as abstract requests. Requests with both eprint and document identifier parts are defined as full-text requests.

A simple filter is implemented in the capture code to handle requests to multi-file documents (e.g. HTML pages with inlined images). Any full-text requests where the referring web page (see above) is also a request to the same full-text are ignored. Assuming the referring web page is correctly supplied by the web client this will count requests to any part of a document, but ignore multiple requests resulting from complex media.

==Storing Usage Data==

Requests are stored in the Eprints ''access'' dataset, see [[Access_Object]] for a description of the dataset's fields. This is partially-processed mirror of the normal Apache log file, but storing the data in a table enables much simpler exposure and re-use of the data.

The Eprints dataset API provides a unique identifier for every request, which is an incremented integer starting at 1. As a dataset requests can be searched or iterated over using the Eprints API.

==Exposing Usage==

Eprints 3 includes a script that can expose access logs through an [http://www.openarchives.org/ OAI-PMH] interface. This interface is located at <tt>/cgi/oai_accesslogs</tt>. The interface contains an <code>EPrints::abort</code> at the top that prevents all access - this is because usage logs are potentially sensitive data: a repository will want to implement some kind of access restriction before providing access to their logs.

The OAI interface uses Eprints export plugins to convert access log objects (as read from the Eprints database) to XML suitable to be embedded in the OAI output. From an Eprints architecture perspective the mechanism to expose access log records is nearly identical to the exposure of eprint records (they are both ''datasets'' in Eprints).

{| class="wikitable"
|+accesslog/eprint OAI comparison
!
!eprint
!accesslog
|-
|identifier
|oai:<archive name>:<item id>
|oai:<archive name>:<item id>
|-
|export plugins
|(predefined list in cfg.d/oai.pl)
|can_accept => "list/access"
|-
|from/until
|last_mod field
|datestamp field
|-
|OAI sets
|arbitrary fields defined in cfg.d/oai.pl
|not-supported
|}

An OAI ListRecords request may contain date and set filters, although sets aren't yet supported for access logs. Using the Eprints search API a result set of records is generated that matches the OAI arguments e.g. all access log records made after a certain time. Each record is converted to XML, depending on the export plugin requested (via a mapping of the metadataPrefix OAI argument to plugin identifier), and output as an XML document.

Currently Eprints supports exporting access log records in OpenURL ContextObject format. To be OAI-compliant an interface should also support Dublin Core (but access logs don't make a lot of sense in DC). The following is our example request, exposed as an XML ContextObject in an OAI response:

<pre>
<ctx:context-object timestamp="2007-05-25T13:07:19Z" >
<ctx:referent>
<ctx:identifier>oai:eprints.ecs.soton.ac.uk:12614</ctx:identifier>
</ctx:referent>
<ctx:referring-entity>
<ctx:identifier>http://www.w3.org/2001/sw/</ctx:identifier>
</ctx:referring-entity>
<ctx:requester>
<ctx:identifier>urn:ip:127.0.0.1</ctx:identifier>
<ctx:private-data>Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)</ctx:private-data>
</ctx:requester>
<ctx:service-type>
<ctx:metadata-by-val>
<ctx:format>info:ofi/fmt:xml:xsd:sch_svc</ctx:format>
<sv:svc-list>
<sv:fulltext>yes</sv:fulltext>
</sv:svc-list>
</ctx:metadata-by-val>
</ctx:service-type>
</ctx:context-object>
</pre>

==Analysing Usage==

TODO: epstats