EA - How to use AI speech transcription and analysis to accelerate social science research by AlexanderSaeri

Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How to use AI speech transcription and analysis to accelerate social science research, published by AlexanderSaeri on January 31, 2023 on The Effective Altruism Forum.SummaryAI tools like OpenAI Whisper and GPT-3 can be used to improve social science research workflows by helping to collect and analyse speech and text data.In this article, I describe two worked examples where I applied AI tools to (1) transcribe and (2) conduct basic thematic analysis of a research interview, and provide enough detail for readers to replicate and adapt my approach.OpenAI Whisper (example) created a high quality English transcription of a 30 minute research interview at a ~70x cost saving compared to a human transcriber.GPT-3 (text-davinci-003; example) answered a research question and identified relevant themes from a transcribed research interview, after providing a structured prompt and one example.These tools, when chained together with human oversight, can be considered an early, weak example of PASTA (Process for Automating Scientific and Technological Advancement).Social science research workflows involve a lot of speech and text data that is laborious to collect and analyseThe daily practice of social science research involves a lot of talking, reading and writing. In my applied behaviour science research consulting role at Monash University and through Ready Research, I generate or participate in the generation of a huge amount of speech and text data. This includes highly structured research activities such as interviews, surveys, observation and experiments; but also less structured research activities like workshops and meetings.Some fictionalised examples of work I’ve done in the past year:Research interviews with 20 regular city commuters to understand what influences their commuting behaviour post-COVID, to assist a public transit authority in planning and operating its services efficientlyPractitioner interviews with staff from city, regional and rural local governments to assess organisational readiness for flood preparation and responseWorkshop of 5-10 people involved in hospital sepsis care, each representing a different interest (e.g., patients, clinicians, researchers, funders) to identify priority areas to direct $5M research fundingSurvey of 5,000 Australians to understand the impacts and experiences of living under lockdown in Melbourne, Australia during COVID-19Evaluation interviews with 4 participants in the AGI Safety Fundamentals course to understand the most significant change in their knowledge, skills, or behaviours as a result of their participationTo make this data useful it needs to be collected, processed, organised, structured and analysed. The typical workflow for these kinds of activities involves taking written notes during the research activity, or recording the audio / video research activity and reviewing the recording later. Interviews are sometimes transcribed by a paid service for later analysis. Other times they are transcribed by the researcher.The amount of speech and text data generated during research activity is large - each research activity yields thousands of words. The sheer volume of data can be overwhelming and daunting, making it difficult to carry out analysis in any meaningful way. In addition, sometimes data just isn’t collected (e.g., during an interview or workshop) because the researcher is busy facilitating / listening / processing / connecting with research participants.Even for data that is collected, managing and analysing it is a challenge. Specialised programs such as nVivo are used in social science to manage and analyse text data, but less structured research activities would almost never be managed or analysed through this kind of program, because of the time and skills required. Open text data i...