Data Quality, Data Centric AI

Feb 13, 2023

Data scientists receive little training in how to collect data, leaving them unprepared when they need to quickly gather annotations.

My background in survey data collection has taught me how to collect high quality data; my expertise can save data scientists money and time in model development.

Stephanie Eckman
Stephanie Eckman
Researcher & Data Scientist

I collect high quality data for social science and model training

Publications

Annotation Sensitivity: Training Data Collection Methods Affect Model Performance
When training data are collected from human annotators, the design of the annotation instrument, the instructions given to annotators, …
Stephanie Eckman
PDF Project DOI
Annotation Sensitivity: Training Data Collection Methods Affect Model Performance
Improving Labeling Through Social Science Insights: Preliminary Results and Research Agenda
Although often seen as a gold-standard, human labeled training data is not error free. Decisions in the design of labeling tasks can …
Jacob Beck, Stephanie Eckman, Rob Chew, Frauke Kreuter
PDF Project DOI
Improving Labeling Through Social Science Insights: Preliminary Results and Research Agenda

Events

Annotation Sensitivity: Training Data Collection Methods Affect Model Performance
5 minute lightening talk on methods to collect annotations for NLP models
Aug 22, 2023 6:30 PM Arlington, VA
Stephanie Eckman
Project Slides Video
Annotation Sensitivity: Training Data Collection Methods Affect Model Performance
Improving Label Collection Through Social Science Insights
As many in the AI field transition from a model-centric to a data-centric approach, the quality of training data is receiving increased …
May 11, 2022 8:30 AM
Stephanie Eckman
Project Slides Video stephnie
Improving Label Collection Through Social Science Insights
Data Quality in Data Science
Data scientists are increasingly turning their attention to the collection of high quality training data. Those of us with expertise in data collection can apply lessons in web survey design to them.
Aug 5, 2021 10:00 AM
Stephanie Eckman
Project Slides Video stephnie
Data Quality in Data Science