Pilot studies
CON²PHYS was piloted at three major neuroscience meetings: the Bernstein Conference, the Pre-COSYNE Brainhack, and the Pre-FENS Brainhack. These events reached communities with substantially different scientific backgrounds, from computational and theoretical neuroscience to predominantly experimental research. Despite these differences, the same broad pattern emerged across all three pilots: teams frequently used different operational definitions and analytical strategies, leading to highly heterogeneous answers and very limited consensus across most questions.
Together, the pilot studies suggest that the conceptual variability observed in CON²PHYS is not restricted to a particular conference, subfield, or methodological community. Instead, it appears to be a reproducible feature of how neuroscientists translate shared concepts into concrete analyses of the same data.
Pilot study 1 – Bernstein Conference 2025 (PhD Symposium)
The first pilot study ran as a hackathon during the Bernstein Conference 2025 PhD Symposium, with roughly 80 participants divided into small teams. The teams tackled five questions from the CON²PHYS questionnaire that the participants had previously selected, and worked on them for a full-day hackathon using the shared dataset.
Overall, very little consensus emerged across the teams on any of the five questions. Answers diverged both in terms of chosen analyses and in the resulting quantitative estimates, confirming that conceptual variability was substantial.
Pie charts of the answers to 2 multiple-choice questions asked at the 2025 Bernstein pilot study. The questions were on spike-spike correlations and directed functional connectivity. The distribution illustrates the low consensus across teams.
At the same time, many participants explicitly said that this was the first time they had been forced to think about “consistency” from this practical perspective, something that they described as refreshing and thought-provoking. This pilot strongly motivated us to continue with the full-scale project.
Pilot study 2 – Pre-COSYNE Brainhack
The second pilot study ran as a two-day hackathon at the Pre-COSYNE Brainhack at the Champalimaud Foundation in Lisbon, on March 10–11, in collaboration with the International Brain Laboratory (IBL). Roughly 40 researchers, organized in teams of two or three, analyzed the same anonymized Neuropixels dataset and answered four multiple-choice questions.
Results largely replicated what was observed at Bernstein: directed functional connectivity and pairwise spike interactions produced near-total disagreement, with answers spread roughly evenly across all available options. Questions on neural decoding were a notable exception, yielding substantially more agreement across teams.
The aggregated results from the Bernstein and Pre-COSYNE pilot studies are shown below. These figures predate the Pre-FENS event and therefore do not include its responses.
Pie charts showing the distribution of answers across teams for two multiple-choice questions — pairwise spike interactions (N=29 teams) and directed functional connectivity (N=28 teams) — aggregated across the Bernstein Conference 2025 and Pre-COSYNE Brainhack pilot studies. Answers are spread across all available options, indicating minimal consensus. Answers are anonymized on purpose.
Pilot study 3 – Pre-FENS Brainhack
The third and final pilot study of the CON²PHYS electrophysiology cycle was held at the Pre-FENS Brainhack 2026 on July 5, 2026, the day preceding the FENS Forum in Barcelona, Spain, at the Facultat de Medicina of the Universitat de Barcelona.
The event brought together approximately 40 participants. Compared with the previous pilot studies, this group had a more predominantly experimental neuroscience background, providing an opportunity to examine whether the patterns observed at Bernstein and COSYNE would also extend to a differently composed scientific audience.
The results were strikingly consistent with those of the previous events. Across the questions addressed during the hackathon, teams adopted substantially different definitions and analytical approaches and arrived at widely divergent answers. No clear consensus emerged for most questions. The recurrence of this pattern in a more experimentally oriented group suggests that conceptual and analytical variability is not confined to a particular neuroscience subfield.
The event was organized in collaboration with the International Brain Laboratory, Andrea Navas-Olive, and Helena Lluís Sanchez-Lafuente, with support from the International Brain Laboratory, F.R.S.-FNRS, the NOMIS Foundation, and the Universitat de Barcelona.
Participating in both a pilot study and the full CON²PHYS project?
If you participated in any of the three pilot studies, this does not disqualify you from participating in the full CON²PHYS project. On the contrary, we strongly encourage you to do so, since you have already completed a meaningful part of the study and are familiar with the dataset and questionnaire. We only ask that you clearly indicate your participation in the relevant section of the submission form and answer the associated questions.