2nd Call for Papers: Workshop on Human-in-the-loop Data Curation at CIKM 2022
YANG Jie
Workshop URL: https://hilworkshops.github.io/hil-dc2022/
Although data quality is a long-standing and enduring problem, it has recently received a resurgence of attention due to the fast proliferation of data analytics, machine learning, and decision-support applications built upon the wide-scale availability and accessibility of (big) data. The success of such applications heavily relies on not only the quantity, but also the quality of data. Data curation, which may include ingestion, annotation, cleaning, integration, etc., is a critical step to provide adequate assurances on the quality of analytics and machine learning results. Such data preparation activities are recognised as time and resource intensive for data scientists as data often comes with a number of challenges that need to be tackled before it can be used in practice. Data re-purposing and the resulting distance between design and use intentions of the data, is a fundamental issue behind many of these challenges. These challenges include a variety of data issues such as noise and outliers, incompleteness, representativeness or biases, heterogeneity of format or semantics, etc. Mishandling these challenges can lead to negative and sometimes damaging effects, especially in critical domains like healthcare, transport, and finance. An observable distinct feature of data quality in these contexts is the increasingly important role played by humans, being often the source of data generation and the active players in data curation. This workshop will provide an opportunity to explore the interdisciplinary overlap between manual, automated, and hybrid human-machine methods of data curation.
This full-day workshop on Oct 21, 2022 will include the following three parts:
Part 1 features plenary sessions, including the keynotes, invited talks, and panel.
Part 2 features selected presentations from speakers whose papers are peer-reviewed and who attend in person.
Part 3 features lightning talks including tool/ demo presentations from online presenters for extended abstracts that are not formally peer-reviewed.
We thus invite submissions for novel research papers around the following topics:
- Quality control for crowdsourced data curation
- Data worker incentivization and engagement, including techniques from citizen science and collective intelligence
- Expertise finding and engagement for data curation
- Supporting crowd workers and experts in data task completion
- Supporting data curation task design for data requesters
- Collaborative data work among humans and between humans and AI
- Human studies into the transparency, reliability, and biases in manual and hybrid data curation
- Interaction techniques for manual, collaborative, and hybrid human-machine data curation, e.g., conversational interfaces
- Database and machine learning techniques for supporting large-scale and hybrid data curation
- Human intervention in data cascades and machine learning lifecycle management
- Benchmarks in machine learning, AI, and related areas
- Privacy and security issues of data quality, e.g., data poisoning attacks
Research papers must describe original work that has not been previously published, not accepted for publication elsewhere, and not simultaneously submitted or currently under review in another journal or conference.
Submissions of research papers must be in English, in PDF format, and be at most 2-4 pages (including figures, tables, proofs, appendixes, acknowledgments, and any content except references) in length and 2 pages for demos, with unrestricted space for references, in the current ACM two-column conference format. Suitable LaTeX, Word, and Overleaf templates are available from the ACM Website (use “sigconf” proceedings template for LaTeX and the Interim Template for Word).
Submissions must be anonymous and should be submitted electronically via EasyChair:
https://easychair.org/conferences/?conf=hildc2022
At least one author of each accepted paper for part 2 of the workshop is required to register for, and present the work at the workshop.
Best papers will be invited to submit an extended version to a special issue of the ACM Journal of Data and Information Quality to be published in Q3 2023.
Important dates (23:59 Anywhere on Earth):
- August 15, 2022: Paper submission deadline
- September 15, 2022: Paper acceptance notification
- October 15, 2022: Final paper submission
- October 21, 2022: Full-day workshop at CIKM 2022.
ACM JDIQ Special Issue on Human-in-the-loop Data Curation timeline:
- Submission deadline: February 2023
- First-round review decisions: June 2023
- Deadline for revision submissions: August 2023
- Notification of final decisions: October 2023
- Camera-ready Manuscript: November 2023
- Tentative publication: December 2023
--
Jie Yang
Assistant Professor
Human-Enhanced Data Management
Human-Centered Machine Learning
TU Delft
Web Information Systems, EEMCS Faculty
Room - 4.W.900, Building 28
Van Mourik Broekmanweg 6
2628 XE Delft, The Netherlands
Email: j.ya...@tudelft.nl
Webpage: http://yangjiera.github.io