21.028 peer-review criteria

                Humanist Discussion Group, Vol. 21, No. 28.
       Centre for Computing in the Humanities, King's College London
  www.kcl.ac.uk/schools/humanities/cch/research/publications/humanist.html
                        www.princeton.edu/humanist/
                     Submit to: humanist_at_princeton.edu

   [1] From: Martyn Jessop <martyn.jessop_at_kcl.ac.uk> (44)
         Subject: Re: 21.026 peer-review criteria?

   [2] From: Michael Fraser <mike.fraser_at_computing- (28)
                 services.oxford.ac.uk>
         Subject: Re: 21.026 peer-review criteria?

--[1]------------------------------------------------------------------
         Date: Thu, 17 May 2007 08:51:10 +0100
         From: Martyn Jessop <martyn.jessop_at_kcl.ac.uk>
         Subject: Re: 21.026 peer-review criteria?

I'm working on something similar but focussed specifically on
visualization projects in the digital humanities. I've found the
London Charter very useful. It deals with one aspect of humanities
computing, 3D visualization, applied to a very specific subject area;
cultural heritage but I am using it as a framework for a wider study.
The URL of the charter is http://www.londoncharter.org/.

It aims to produce a set of principles to ensure intellectual
integrity, reliability, transparency, documentation, standards,
sustainability and access. These are surely the qualities that should
be assessed in peer review. There is also a purpose beyond peer
review "to ensure that such work is intellectually and technically
rigorous, and for its potential in this domain to be realised, there
is a need both to establish standards responsive to the particular
properties of 3d visualization, and to identify those that it should
share with other methods." It sets out nine principles concerning

1. Subject Communities
2. Aims and Methods
3. Sources
4. Transparency Requirements
5. Documentation
6. Standards
7. Sustainability
8. Access

My paper at Digital Humanities 2007 in June will present some of the
work that I have based upon it for visualization projects in the humanities
as a whole.

My next point is rather more tightly focussed than you requested but I
think it is a particularly important aspect of peer review, (also
covered in the charter). It concerns 'paradata', i.e. documentation
about the research process; why particular decisions were made etc. I
recently attended a combined arts/sciences/humanities workshop and
the discussion of the importance of lab books to scientists struck a
chord. They fulfill a far more important role in scientific research
than I had realised. Artists document their research processes in
different ways but again for similar ends. Perhaps it's a practice we
should adopt in the digital humanities?

----------------------
Martyn Jessop
Centre for Computing in the Humanities
King's College London
Strand
London WC2R 2LS

email: martyn.jessop_at_kcl.ac.uk
Phone: 0207-848-2470
Fax: 0207-848-2980

--[2]------------------------------------------------------------------
         Date: Thu, 17 May 2007 08:51:49 +0100
         From: Michael Fraser <mike.fraser_at_computing-services.oxford.ac.uk>
         Subject: Re: 21.026 peer-review criteria?

On Wed, 16 May 2007, Humanist Discussion Group (by way of Willard
McCarty <willard.mccarty_at_kcl.ac.uk>) wrote:

>I'd very much appreciate help in locating lists and discussions of
>criteria for the peer-review of digital scholarly objects. In
>simplest form, such criteria overlap with those for evaluating
>websites, but what I'm really interested in is how one might go about
>reviewing complex digital objects. Best would be those at a
>sufficiently high level of abstraction as to suit whatever one might
>encounter, but those specific to particular kinds of objects would
>also be quite useful, as would articles on the process of reviewing
>digital objects.

Dear Willard,

The following AHRC-funded project (and its final report) should be of interest:

"Peer review and evaluation of digital resources for the arts and humanities"

http://www.history.ac.uk/digit/peer/

Best wishes,

Mike

---
Dr Michael Fraser
Co-ordinator, Research Technologies Service
Director, Intute: Arts and Humanities
Oxford University Computing Services
13 Banbury Road
Oxford OX2 6NN
Tel: 01865 283 343
Fax: 01865 273 275
http://www.oucs.ox.ac.uk/rts/
http://www.intute.ac.uk/artsandhumanities/
http://users.ox.ac.uk/~mikef/