Personal stories - Large Collaborative Research Community

Personal stories - Large Collaborative Research Community#

Discussion with Remi Gau and Isil Poraz Bilgin from the Brainhack community#

Brainhack is an open science community that has an innovative meeting format to promote scientific collaboration and education in an open, inclusive environment. Remi, Isil and their colleagues have been dealing with the issue of how to determine contributions and authorship in such a large community.

More information about Remi Gau:

More information about Isil Poyraz Bilgin:

1. What is the normal practice of authorship for academic papers in your discipline? or used to be the normal practice?

Remi: My background is in Biology and Psychology. Most papers are multi-author and so have more than one author. Single authorship can happen but not for experimental work. This work usually involves at least a Principal Investigator with a Postdoctoral Researcher or PhD student.

Single author papers don’t have a bad reputation. They tend to be theoretical, summarizing articles or opinion pieces. However, they are more unusual now. This is because of a need to collaborate as projects are more complex and therefore require more skills and hence more people.

There are still some problematic practices happening such as providing materials for the project because of wanting to get on the paper. So yes, these people are providing something that makes the project possible, such as a raw material needed for the project, but are not making a substantial contribution to the project in any other way.

In terms of the order of authors on papers, usually the two first names are very important and the two last names. First names are the people who have completed the most work on the project and the last two names have supervisory roles. The middle ranking authors can be random or alphabetical.

There is a slow move towards using the CRediT system now; giving a description of what the contributions have been. The issue with this is that it is a very detailed description of the different roles and there is a lot of push back about this and trying to do things differently.

The increase in large collaborative papers and also the generation of software in projects means authorship needs to be dealt with in a different way.

Isil: I am a mathematician and have a Computer Science MSc, so I have more insights from the engineering practices due to my background, in which the authorship models follow a more compact authorship line-up. This means authorship that mainly constitutes of PhD or Postdoctoral students who actively work in the project as the first authors and the most senior person, such as the supervisors, principal investigators as the last author. However, with the development of more interdisciplinary research in the field, we have seen an increase in the number of authors that are included in papers.

As the general rule of thumb, the author lists are created by those who made a ‘significant intellectual contribution’ to scientific research. However the complication comes from having no clear definition or metrics to quantify the significance of the contribution that has been made.

The only definition that has been made to this day is by the American Institute of Physics (AIP) as “Authorship should be limited to those who have made a significant contribution to the concept, design, execution, or interpretation of the research study”, which still does not give a clear guideline regarding how to evaluate the contributors in the fairest and consistent way across journals, institutions and laboratories.

2. When did you discuss authorship in your project?

Remi: This was done very early in the project. We spent some of the first three meetings, in which the two main questions were finding out how to deal with authorship and the journal we wanted to aim for. We also discussed how we were dividing up the tasks.

Even with this early approach, there were some headaches. The main issues were whether authorship should be all alphabetical, random or ranking.

The order does let people know to what level they want to contribute and has implications on different peoples input into the project.

Alphabetical authorship order would not seem very radical in some groups, such as an open science crowd. But in other settings it is very radical and may mean that some researchers are less keen to contribute if they don’t get the recognition for their work that they think they should get.

Isil: In addition to these, there have been some discussions regarding using consortium authorships with some examples given from previous papers by the community members as in SciPy paper or as in Open Brain Consent.

But going back to what Remi mentioned, through the discussions, the feeling was that there had been some missing links in the general understanding regarding what the audience of a community paper should be in comparison to a scientific paper. How the authorship model and the expectations regarding the contributions should be justified and what they should be formed like.

Therefore, these consortium authorship models remained a topic that needed more extensive discussions, which was not possible due to the pressure of time.

3. Who had the discussion about authorship in your project?

Remi: Everybody was included in the discussions at the beginning of the project and we had a vote on it.

These general guidelines were documented but were to be revisited at a later stage of the project.

Problems arose when more people joined the project. Most people agreed with the framework and guidelines but I’m not sure that they were made aware of this fully when onboarding. It was a rush and the process was rushed.

Later on there were more discussions about authorship.

The time scale of the project, and the writing of articles, is going to affect the authorship process.

It will affect who gets a say and if the decisions are made correctly. If you have to move fast, then having a democratic decision takes a long time in a large community and might not be possible. To speed up the process, we found some core members needed to make decisions, but this undermines the process.

Isil: Also the discussions, the voting to be done, the decisions taken or to be taken were all shared through the public communication channels and a google email list that includes all the contributors to make sure everyone had a chance to see this information or follow the process one way or another.

4. What approach have you decided to take in your project?

Remi: Our guidelines set out that we list authors into three groups:

  • Group 1 are a core team of people that are actively working in the project and writing the paper. For example, the actual writing of the paper, brainstorming and recording agendas for meetings. We are using an all contributors model like The Turing Way’s but with our own emojis in a checklist. This core team were ranked in terms of their contributions but this did create an issue of how to decide the value of different contributions and I would suggest that this should be decided firmly at the beginning.

  • Group 2 are those people that did not actively write the paper but are active in the Brainhack community so they ran events, gave talks or led a project. These members were listed alphabetically.

  • Group 3 is the last author. It was decided to be the Brainhack community (not by name but part of the community). There were some discussions about a web list of names.

We did have to revisit these guidelines.

Isil: Just to add to this, the groups were formed organically as a bottom-up process and kept open to participation at any point in time through data collection and writing processes.

Due to the personal time schedules and commitments, some people were only able to be present at the beginning of the discussions and some were present throughout the whole time or some were able to contribute at the end. However, at the end of the day, we tried to do our best to quantify any contributions given at any time point throughout the process independent of the amount of presence that had been shown.

5. How have you gone about paper writing practically?

Remi: We started by brainstorming to break the paper into different sections and then created sub teams, where each team worked on one section. There were at least 4 teams.

There was also a master editing team to unify all these sections and another team to work on the figures, tables and glossary.

Very early on we realised that the paper would be this fixed thing. However, we wanted there to be an evolving version. We were restricted with the word limit to the paper and we wanted to spill over with all the ideas and keep evolving so we started a Jupyter book. So there is also a team working on the Jupyter book.

Once all of the sections were written and then passed to the editing team, they were sent to the co-authors.

We took lots of notes in meetings and fed this back to all co-authors. All meetings were recorded and videos made available.

Once the paper was finished we sent it out for Community review. This was done by putting the pre-print online for any contributors to check.

Then another meeting took place with the core team to finalise these comments. The community review followed the example of the INCF - A standards organisation for open and FAIR neuroscience.

Isil: The teams for the sections were self-organized by those members who wanted to share their ideas and experiences in that particular domain such as educational aspect, community building or projects. There was no clear cut division between these section teams, the members were fluid across the teams as needed.

Although this was helpful given the race we were in against the time in completing the tasks at hand, making this organisation more explicit between the members of the teams might have given greater opportunities for contributions from more contributors who represent a larger variety of backgrounds and opinions.

6. What have been the problems with doing this?

Remi: We under specified the ranking system for the core team. We did not want the core team to do the ranking, we wanted more of a bottom up approach. Like a tiering system of what their contributions were.

We wanted to use the contributions checklist to rank people such as the number of boxes that they had ticked. We tried to go about this by ranking people in subteams, but this made people very uncomfortable. The community ethos was against a ranking system and also it was unclear that they would not rank themselves. This lack of clarity made it hard to decide how to make a change to this system once it was in place. The process needed to be slowed down and discussed more thoroughly about how to rank people fairly. Decisions were finally made about how to rank authors by a core team. But it was time constraints and the ever increasing people onboarding to the community that caused an issue.

For other publications we will probably just use the ‘Brainhack community’ as consortium authorship. Each person’s name will appear underneath the community name as a list and therefore all named authors will get the citation. This will make our authorship more equitable.

Isil: Remi has touched upon the many important points above.

Just to add, one of the difficulties we also had was to have a full representation of the Brainhack community in this beautiful work. The nature of the Brainhack community has made it a real challenge to trace back all the contributors and have them involved in the publication process. It is a voluntary based organization that has had events all around the world since 2012. Sometimes these events are run as stand-alone events, or under the roof of the yearly organized Brainhack Global or even by one of the biggest international neuroscience society - Organization for Human Brain Mapping (OHBM) - through their yearly conference.

Because of his past leadership and many other involvements in OHBM Brainhack events, Remi at the OHBM side and I personally at the Brainhack Global side, we spent weeks tracing back to these Brainhack organizers and attendees, chasing the changed affiliations, and persuading them they should be in this project and why it matters for the future of the Open science and the community itself. There were, however, still missing members or data to be represented in this whole process.

I believe this is a difficulty in writing a paper that strives to engage and embrace an inclusive and representative approach. We want differing ideas, backgrounds and experiences at its heart but we have to deal with the aforementioned struggles.

Therefore, I also strongly believe that adopting an accumulative ‘Brainhack community’ consortium authorship model would create better, fair and equal opportunities for the representation of every single member of the community. This will create a better chance of incentivizing and acknowledging the years of commitments and efforts given by each member of the community.

7. What collaborative tools did you use for the writing and projects?

Remi: We used Google docs and sheets. Also forms for information from authors. But some Google tools are not accessible in some countries such as China. So it is good to have a discussion about what tools to use so that everyone has access to the same software and tools from the start.

Isil: Also since we had a lot of supplementary details regarding the years-long events and projects that were run under Brainhack events, that was not able to go into the paper due to the limitation of the word count. We created a supplementary Jupyterbook for which we had a public repository on Github, that is constituted of markdown files for the content of the book, .json files for the data to process and many graphics to summarise the data.

Recent article from the Brainhack community: Gau R, Noble S, Heuer K, Bottenhorn KL, Bilgin IP, Yang YF, Huntenburg JM, Bayer JMM, Bethlehem RAI, Rhoads SA, Vogelbacher C, Borghesani V, Levitis E, Wang HT, Van Den Bossche S, Kobeleva X, Legarreta JH, Guay S, Atay SM, Varoquaux GP, Huijser DC, Sandström MS, Herholz P, Nastase SA, Badhwar A, Dumas G, Schwab S, Moia S, Dayan M, Bassil Y, Brooks PP, Mancini M, Shine JM, O’Connor D, Xie X, Poggiali D, Friedrich P, Heinsfeld AS, Riedl L, Toro R, Caballero-Gaudes C, Eklund A, Garner KG, Nolan CR, Demeter DV, Barrios FA, Merchant JS, McDevitt EA, Oostenveld R, Craddock RC, Rokem A, Doyle A, Ghosh SS, Nikolaidis A, Stanley OW, Uruñuela E; Brainhack Community. Brainhack: Developing a culture of open, inclusive, community-driven neuroscience. Neuron. 2021 Jun 2;109(11):1769-1775. doi: 10.1016/j.neuron.2021.04.001. Epub 2021 Apr 30. PMID: 33932337.

This interview was conducted by Emma Karoune.

Contents