Application and Review Process#
The application, review, and selection processes have been developed to provide a fair opportunity for all the interested members to show their interest in attending our book dash events.
In this subchapter, we have provided details on the book dash’s application and reviewing process.
Call for applications#
The book dash event is organised for a minimum of 15 participants. The event is divided into multiple sessions with small groups of participants attending in their time zone. This allows the core team members to offer specific mentorship to the invited participants and make this an interactive, intimate, fun, and productive event.
To fairly make this selection, a call for applications is opened at least two months in advance so that interested members can share their skills, interest, and expected outcomes through a short application. The applications are kept open for a minimum of 4 weeks and the timeline is communicated clearly through our newsletters, X (formerly Twitter) feed, and application form.
This application can take 30-45 minutes to complete. We have created this template document with all the questions to prepare a draft before submission: https://tinyurl.com/tw-bookdash-template.
We continue to think of ways to ensure privacy and data protection. Currently, all these responses are securely handled under the GDPR (General Data Protection Regulation) law (learn about creating GDPR-proof surveys). All our applicants must agree with the following GDPR statement when sending their application:
We need to store your e-mail address to contact you. The information provided in this form will be shared only with The Turing Way Book Dash organisers to help them review your application fairly. These details will be stored in a Google Drive that core Turing Way team members have access to. The servers are based in the USA. You can ask for your e-mail address or any information to be removed from our database at any time. If you have any questions please contact us by emailing turingway@gmail.com. You may edit your responses after submission until 21 May 2022. Please select the “I understand and I accept” option before proceeding.
We ask if applicants have previously participated in a Book Dash so that they can express their interest to join the Book Dash review and planning committee for this event:
We encourage returning applicants to The Turing Way Book Dash to join the planning committee. They will take on advisory and leading roles for the Book Dash events in 2022 by getting involved in application Review, selection process, organisation meeting, session host and mentorship or accessibility-related support they can offer based on their willingness, availability and interest. For any questions, please email theturingway@gmail.com.
In 150 words each, applicants respond to the following mandatory questions:
What could you contribute to The Turing Way during the book dash event?
What would you gain from being part of The Turing Way book dash event?
We ask an optional question for returning members to express their interest in taking on leadership roles in designing and organising Book Dashes:
How would you like to collaborate with other participants, or what support will you invite from them during the Book Dash event?
These responses to the questions in the application form help us select a group of people who will be able to work effectively together and cover a broad set of possible contributions to The Turing Way. We don’t expect that the candidates might have previously contributed to the project, but some familiarity with our project infrastructure such as GitHub-based contributions will be helpful. We are looking to support ideas on what our participants think is important and helpful for others doing reproducible research and data science in the current scenario.
We then ask a few demographic questions. We want to make sure that the selected group of participants brings multiple skills, knowledge, and experience to The Turing Way. We aim to involve members from diverse groups, especially those who are traditionally excluded from data science based on gender, ethnicity, and career stage. Therefore, we ask our applicants to indicate if they consider themselves to be a member of one or multiple historically underrepresented groups. Here they can add any other information about themselves that they think the core team should know when reviewing their application.
We seek to remove any barriers that might prevent our participants from attending the event. Hence, we use this form to assess what support we can offer, including funding requirements, to our participants in ensuring that they can comfortably attend the event. Applicants can mention their travel, accommodation, childcare, and accessibility requirements for their participation.
Finally, we provide details regarding the event schedule and ask our applicants to indicate which time slots for collaborative co-working and social sessions would work for their time zones.
Reviewing process#
Applications for The Turing Way book dashes are reviewed by at least 2 members of The Turing Way core team. A response form created by the organising members using the rubrics described below, the response from which are shared among all the reviewers during a designated meeting between the panel members. Scoring for each criterion is done between 1 to 3, where 3=criteria fully met, 2=criteria partially met, and 1=criteria not met. To avoid personal bias, each of these scores is clearly defined for every question in the review response form, which is explained below. This rubric aims to evaluate the application across multiple aspects and avoid any personal bias panel members may have.
This rubric is adapted from the Mozilla Open Leadership and the Open Life Science programs.
Rubrics for scoring applications#
The invited reviewers will rate the participants across multiple dimensions with scores of 1, 2, or 3.
We ask our reviewers to consider the following questions:
Has the applicant answered the application questions?
Does the applicant have a clear, feasible, and relevant idea of how they will contribute to the book dash? This scoring corresponds generally to reviews of “not ready”, “enthusiastic” and “clear” respectively:
Is the applicant enthusiastic about reproducibility and the aims of The Turing Way?
Based on their evaluation of the applications they will provide scores for different criteria that will be assessed as follows:
Applicants who score mostly 1s do not have a clear idea/topic ready for contribution or do not have a clear understanding of the overall goals of The Turing Way and book dash in general.
Applicants who score mostly 2s are enthusiastic if not wholly suitable for the project/event, for example, they may come with some ideas for a contribution that is currently not in the scope of the project/event.
Applicants who score mostly 3s are clearly ready, goal-oriented, interested in contributing to the project, and excited to learn from others at the event.
Table summarising our scoring rubric and criteria for each score:
This scoring rubric is based on the Mozilla Open Leaders and Open Life Science (OLS) selection criteria as described in the ‘Review Rubrics’ on this GitHub repository.
Sections |
Score 1 |
Score 2 |
Score 3 |
|---|---|---|---|
Readiness for the book dash event |
(not ready) Does not provide enough information or seems to misunderstand the nature of The Turing Way project and this event in general |
(enthusiastic) Seems to have a clear understanding of The Turing Way project and this event and brings along a specific content for contributions |
(clear) Seems to have a clear understanding of The Turing Way project and this event and a clear understanding of how they can contribute and collaborate with each other at this event |
Goals for the project contributions: |
(not ready) Shares vague or general ideas that are unrelated to The Turing Way project and this event, or no goals at all |
(enthusiastic) Shares clear, overly ambitious ideas for The Turing Way project and this event that can likely be refined in a collaboration cafe, or the day before the event during the brainstorm session |
(clear) Shares clear, achievable contribution/development ideas for this event that fits The Turing Way project and are likely to be achieved through the applicant’s participation |
Purpose of participation and what they will get out of the book dash: |
(not ready) Purposes for participation at the book dash seem almost entirely self-centered and about the applicant’s status, rather than about participating in The Turing Way community to develop the project |
(enthusiastic) Purposes for the participation at the book dash event are not completely clear from the application or are limited (even though useful), such as typo or bug fixing |
(clear) Purposes for participation at the book dash event are valuable in many ways and are likely to help the applicant to become an active contributor and take ownership of their work in The Turing Way project and in the broader ecosystem in their own rights |
Willingness to collaborate and contribute after the book dash: |
(not ready) Seems closed to collaborative ways of working or more interested in only one aspect of data science, research or related topic |
(enthusiastic) Seems excited to learn from others and The Turing Way project, but in a general way without much understanding of what those things mean yet |
(clear) Seems excited to collaborate with others and is motivated to contribute to The Turing Way community |
The following sections are aimed at collecting open-ended response by the reviewers:
We ask all reviewers to finish the review by describing the application, representation aspects of the applicant and the motivations for their scores to facilitate conversation during the selection panel discussion under these questions.
- Please highlight any diversity/minority groups this applicant belong to, that should be represented at this event
- Please provide 1-2 summary sentences about this application to facilitate discussion during the selection panel
Reviewers are also asked to disclose if any conflict of interest may have influenced their review leading to any bias in their decision.
- Please state if you have any conflict of interest with the proposal described in this application or do you know this applicant personally that may lead to a biased decision
Finally, reviewers are asked whether they would recommend an applicant. The available answers are “No”, “Yes”, “Unsure” or “Request for open discussion in the panel”. Candidates who have ratings of “No” or “Unsure” across all reviewers will be rejected. All other candidates will be discussed at the panel selection.