Data Collection and Management for Research on Teaching

Collecting and Managing Data

Once you have determined your research question and identified which learning context you want to examine, it is time to consider which data collection methods would be most informative for your context. Your research question will inform what data collection methods you decide to use.

This resource gives an overview of some data collection methods used in education research. This resource then provides some starting guidelines for data management as you collect and analyze your data.

Data Collection in Scholarship of Teaching and Learning

Depending on your research question(s), you may be collecting quantitative or qualitative data, or a combination of both. You may also be collecting from multiple sources or via multiple methods.

Often there are trade-offs between different methods. For instance, data collection methods vary in how much time and effort are involved for both the researcher and the participants. Resultant data may vary in complexity or detail based on factors such as participation rate.

Types of Data You Might Collect

MethodData CollectedAnalysis Type
Survey or Questionnaire Responses to specific questions asked.

Questions can be open-ended (e.g., What was one thing that helped you learn the material in this course?) or closed-ended (e.g., Using the following scale, how much did the instructor’s explanation help you learn the material in this course? Not at all, a small amount, a moderate amount, a large amount).

Depending on the types of questions, you may have quantitative analyses, looking at the frequency of specific answer choices. You may also have qualitative analyses for open-ended questions.

You may also have questionnaires at specific timepoints to compare responses over time.

InterviewsRecords and/or transcripts of a conversation or exchange between the interviewer and interviewee. Interviews allow interviewees to reflect and articulate their insights about specific learning experiences and provide an opportunity for the interviewer to follow-up on specific responses.Typically, qualitative analyses will be used to identify patterns or themes across multiple interviews.
Think aloudsRecordings and/or transcripts of a participant articulating their process as they are doing a specific task (e.g., a learner talks through how they are solving a math problem in real time so the listener can get a sense of how they work through this process).Typically, qualitative analyses will be used to identify patterns across multiple think alouds.
Analyzing student work Artifacts that are generated from participants’ work as part of their course or learning experience. This can include course assessments or student reflections.Both quantitative and qualitative analyses might be used to analyze this data.

For example, you may use a rubric or scoring system to obtain quantitative data from student work. You may also utilize qualitative methods to find overarching patterns across artifacts.

Data Management

As you collect data from multiple participants, potentially across multiple methods, it will be important to have a system where data is secure, organized, and well-documented. 

Secure

  • Ensure that any data sharing follows any procedures and constraints as specified by your IRB protocol (if applicable), the Office of Human Subjects Research Protection, and FERPA policies. 
  • De-identify data when possible. This means separating any personal identifying information (e.g., name, DOB, phone number, etc.) from the rest of your data. This might involve replacing participant information with an ID number or other identifier in your data and then maintaining a separate participant log that contains the participant name and their ID number. This participant log should be a separate file from the rest of your data that has password protection (see here for password protection on Excel spreadsheets).   
  • Store data in a secure location. See the University policy on data classification and storage here:

Organized

  • Always keep a clean, untouched copy of all raw data. This is critical to making sure your work is reproducible as well as an important step for having an accessible backup file when necessary. For instance, if collecting questionnaire responses via Qualtrics or using transcripts from an interview, download an untouched version of these files to keep as a reference that will not be edited. 
  • Establish folder structure to organize your data and other research materials. Name folders so that they appear in the order that you would like them to in your finder (e.g., put numbers at the start of folder names to dictate the order). An example folder structure might be:
    • 01 – Study Materials → Contains research plan and protocol, the text of questions, etc. 
    • 02 – Original/Raw Data → Untouched copies of raw data 
    • 03 – Analysis Documents → Annotated data, spreadsheets with transformed data, and other analyses that you are doing
    • 04 – Output/Dissemination → Documents or files relating to dissemination of your work (e.g., data visualizations, abstracts/write ups based on results, etc.)  
  • A consistent filename structure. Consider how files will be sorted in your finder and how you want files to be organized. For example, if you want files sorted by participant ID, you might start filenames with the ID number (e.g. 001_interview.txt). You may also want to add other information into the filename as appropriate (e.g., if you want to distinguish between interviews at different timepoints, you might include this in the filename. E.g., 001_interview_1.txt, etc.).

Well-documented

  • Think about what information you might need if you were looking at your data for the first time and document it! This will be helpful to any present or future collaborators and yourself if you revisit a project.
  • Keep a record of what you have done and changes you have made to your research protocol. This is especially important if:
    • You make changes to a protocol or questionnaire between different data collection timepoints. 
    • You adjust a pre-existing or previously-validated questionnaire. 
  • Other things you may want to document:
    • Your folder and file naming conventions
    • Any abbreviations or shorthand used (e.g., color codes for highlighting/annotations, a description of categories or codes used) 
    • Dates that you collected data 
    • You may want to also create a data dictionary that has individual descriptions for each type of data you collected, the structure of different spreadsheets. 

Northeastern Library has a great Data Management Checklist with these principles. 

Next Steps

Learn more about Scholarship of Teaching and Learning projects that have been done at Northeastern from the essays of previous Teaching & Learning Scholars (links to essay booklets are in the right hand column!). As you read, think about the relationship between the research questions and the data collection methods of previous scholars. 

To start thinking about your project, reach out to CATLR for a consultation.

Other resources

Data Management for Research from Northeastern Library.

Broman, K. W., & Woo, K. H. (2018). Data organization in spreadsheets. The American Statistician, 72(1), 2-10.

Chick, N. L. (2018). SoTL in action: Illuminating critical moments of practice (First edition.).

Tenopir, C., Allard, S., Douglass, K., Aydinoglu, A. U., Wu, L., Read, E., Manoff, M., & Frame, M. (2011). Data sharing by scientists: Practices and perceptions. PloS one, 6(6), e21101.