Why ONS? From a discussion with friends

I agree that the key is open data, and it really only helps if that data is discoverable, and better if it’s in a central repository. The open notebook community is pretty focused on figuring out ways to automate and link the data from the notebook. For instance, many solubilities recorded for chemicals given in wikipedia automatically link to the open notebook entries describing the experiment wherein the data was collected, check out https://en.wikipedia.org/wiki/Benzoic_acid.

There are lots of other reasons though.

  1. Ensuring that key results are reproducible. Papers need to be linked to their data, and the data to the notebook.

  2. Notebook is a great learning tool. I’ve learned how others keep and organize notes, and what software tools they use that facilitate this through their open notebooks. New students in my department read my notebook to learn how to get started & get organized for research, particularly theorists where the work is less structured.

  3. Standardization: Because I learn what tools and conventions others use to keep notes, record and link data, the approaches I use are more standardized than everyone having a secret custom solution.

  4. Null results: People also don’t have time to publish null results, which wouldn’t be such a problem if it were in the notebook.

  5. Publications are slow. I know what groups are working on interesting problems in my field from conferences before I see their papers, but the notebook would make it instantaneous.

  6. Avoid needless repetition: Lots of labs waste effort doing the same thing as another group, trying to beat them to the punch. That’s a waste of tax money – why work on a question if someone else will be able to answer it if you just sit back and wait?

  7. Good for collaborators – Has helped me find collaborators, keep them updated on my work, and get feedback.

  8. Good for my advisors – easy for them to see what I’m working on

  9. Catch and correct mistakes – I make fewer mistakes because I’m more careful knowing it’s public, but others also point out when I make mistakes.

  10. Better motivation – I feel pressured to get things done fast and well because my tardiness or sloppiness would be visible.

  11. Increase my impact/name recognition – My notebook is better read and better referenced/linked to/quoted than either of my publications. Why? I don’t know, but their are lots of journals, but not lots of notebooks; because my notebook is a web-native object that is open and easy to discover and access, because open notebooks are controversial and hence interesting to discuss.

The Social Lab Notebook?

I’m a theorist.  A student.  An ecologist.  I work with ideas.  I also conduct experiments – the numerical kind.  Theoretically, these are perfectly replicable, all the way down to the random number generation.  In practice: not so much.  I develop code. I’m fortunate to build my code upon a rich set of general purpose ( = quite well developed and documented) and highly specialized (= buggy) open source tools.  The code implements methods I invent.  That’s my “academic” contribution – the ideas.  That’s what counts.  It requires some math.  It requires big computers.  It requires an understanding of the scientific questions, and how to translate them into the language of math and computers.

I was never taught to keep a notebook – the cost of not being in a wet lab.  Tracking the different elements of this process is something of a challenge.  Good tools exist to track

Discussion ideas

  • Use dedicated tools for scientists or general purpose tools?

  • How do you document the thoughts behind the process?

  • Scalable research model

  • Sharing

Scale

Read. Do. Write. Reading about what’s being done and writing up what has been done appear to compete with doing science. Can we integrate these processes through tools such as a social lab notebook? Avoid lag & limited reporting.

Communication

The academic journal was a big breakthrough in scientific communication.  Before then, when you discovered something of significant interest, you had to sit down and write a letter to any  colleague who didn’t work down the hall, explaining what you had done.  Then they might read your letter to some of their colleagues.  If you were lucky, this might be at  a meeting of many scientists, such as the Royal Society of London, and you wouldn’t have to write them all separately.  Then in 1665, the Royal Society published those letters so they could be read by those who couldn’t be there in person – significantly advancing the ease and speed at which scientific knowledge was disseminated.  345 years later, little has changed.   We have more journals, and consequently spend more time figuring out which letters go in which journals, but the pulse and scale of the scientific enterprise remains set by the beating of these organs.

(Several studies have shown most submissions that are rejected are eventually published elsewhere:) Wijnhoven, B. P. L., & Dejong, C. H. C. 2010. Fate of manuscripts declined by the British Journal of Surgery. British Journal of Surgery 97: 450-454. https://dx.doi.org/10.1002/bjs.6880 McDonald, R. J., Cloft, H. J., & Kallmes, D. F. 2009. Fate of Manuscripts Previously Rejected by the American Journal of Neuroradiology: A Follow-Up Analysis. American Journal of Neuroradiology 30: 253-256. https://dx.doi.org/10.3174/ajnr.A1366 Groves, T. 2009. Nine in 10 articles rejected by NEJM appear in another journal. BMJ 339: b3777. https://dx.doi.org/10.1136/bmj.b3777 Hall, S. A., & Wilcox, A. J. 2007. The Fate of Epidemiologic Manuscripts: A Study of Papers Submitted to Epidemiology. Epidemiology 18: 262-265. https://dx.doi.org/10.1097/01.ede.0000254668.63378.32 Liesegang, T. J., Shaikh, M., & Crook, J. E. 2007. The outcome of manuscripts submitted to the American Journal of Ophthalmology between 2002 and 2003. American Journal of Ophthalmology 143: 551-560. https://dx.doi.org/10.1016/j.ajo.2006.12.004 Opthof, T., Furstner, F., van Geer, M., & Coronel, R. 2000. Regrets or no regrets? No regrets! The fate of rejected manuscripts. Cardiovascular Research 45: 255-258. https://dx.doi.org/10.1016/s0008-6363(99)00339-9 Cronin, B., & McKenzie, G. 1992. Documentation note: the trajectory of rejection. Journal of Documentation 48: 310-317.