Humanist Discussion Group, Vol. 37, No. 511.
        Department of Digital Humanities, University of Cologne
                      Hosted by DH-Cologne
                       www.dhhumanist.org
                Submit to: humanist@dhhumanist.org




        Date: 2024-03-26 17:16:03+00:00
        From: Dieuwertje Luitse <d.luitse@uva.nl>
        Subject: CfP Topical Collection DISO: Politics of Machine Learning Evaluation

CfP Topical Collection in Digital Society: Politics of Machine Learning
Evaluation

Guest Editors: Dieuwertje Luitse 
<https://www.uva.nl/profiel/l/u/d.luitse/d.luitse.html>, 
Anna Schjøtt Hansen 
<https://www.uva.nl/en/profile/h/a/a.s.hansen/a.s.hansen.html#Publications> &
Tobias Blanke <https://www.uva.nl/en/profile/b/l/t.blanke/t.blanke.html#About>
(University of Amsterdam)

Is the data good enough for training purposes? Does the model perform accurately
enough? Is the error rate low enough? Such questions of ‘good enough’ are at the
very core of the process of Machine Learning (ML) evaluation and can also be
considered a highly political process in the development of ML systems. There is
already a growing interest in the political implications of ML in relation to,
for example, dataset construction and the political capacities of specific ML
models or foundational algorithmic techniques. However, there has been less
focus on the politics of evaluation practices and techniques in ML. To further
explore this issue, we invite contributions to a topical collection on ‘The
Politics of Machine Learning Evaluation’ in Digital Society. We invite papers
that engage with conceptual, methodological, and political questions in relation
to topics, such as but are not limited to:


  *   Dataset construction
  *   Data labelling practices
  *   Ground truths and benchmarks
  *   Biases in evaluation
  *   Metrics
  *   Errors and error analysis
  *   Evaluation techniques

Concretely, we invite papers that engage in conceptualising or historizing
machine-learning evaluation as a politically contested practice. In addition, we
are interested in papers that provide methodological approaches to the study of
evaluation techniques or empirical studies into ML evaluation in practice. For
more information, see also: https://link.springer.com/collections/bbbehaibcj

This topical collection emerged as part of a workshop hosted by the University
of Amsterdam in November 2023 and will feature articles presented during the
event, but we also welcome additional contributions to the topic.

Timeline and submission details:

Abstracts should be between 300-500 words, excluding references. Abstracts
should be sent to a.s.hansen@uva.nl and d.luitse@uva.nl, with the subject line 
‘CFP: Politics of Machine Learning Evaluation’. The deadline for submission of 
abstracts is April 26, 2024. Notifications of invitations to submit a full paper 
will be sent by mid-May.

Final papers are to be submitted via Digital Society’s submission
system <https://link.springer.com/journal/44206>, which will be open for
submission between October 18 to November 1, 2024. Please indicate that the
submission is part of the topical collection.

Although initially accepted, all submissions will be subject to peer review
following the peer-review procedure of Digital Society. We expect the submitting
authors to be the reviewer for a different paper in the collection.

If authors miss the deadline for abstract submission, they should still contact
the guest editors before sending their manuscript to Digital Society.


_______________________________________________
Unsubscribe at: http://dhhumanist.org/Restricted
List posts to: humanist@dhhumanist.org
List info and archives at at: http://dhhumanist.org
Listmember interface at: http://dhhumanist.org/Restricted/
Subscribe at: http://dhhumanist.org/membership_form.php