The Great Electronic Lab Notebook Challenge, pt. I

This is the first in a series of posts on searching for an ELN suitable for use in my group.  This question is related to a series of hardware, workflow and "data management" questions.  These I will address elsewhere.  In this post I lay out what I'm looking for in an ELN, and how it fits my ideal group workflow.  Subsequent posts will address my experiences with MacJournal and DEVONthink Pro Office.

My research is almost entirely computational.  (For details on the research itself, visit the Beck Research Group page.) I have a small group, who are expected to use their laptops as their primary research gateway/tool.  The ideal research workflow for my group members looks something like this:

1. Students take notes of discussions on laptop (ELN)
2. Literature search with reference manager (SDB)
3. Background study and hypothesis development (ELN)
• Takes notes about (and on?) references (ELN/SDB)
• Note thoughts, ideas, plans, etc., on laptop (ELN)
4. Generate "Design of Calculations" preliminary report (SO)
5. Preliminary calculations on local or production compute resources (DO)
• Data analysis of preliminary results (ELN)
6. "Pre-production Calculations" report, including: (SO)
• Convergence parameters (ELN)
• Comparison to prior results
• Estimate of production calculation resource requirements (ELN)
7. Production calculations on local or external resources (DO)
8. Data analysis (ELN)
9. "Draft results" report (SO)
10. Discussion and further analysis *** (ELN)
11. Paper preparation (ELN/SO/C)
• Figure and chart preparation
• Iterative and collaborative text preparation
12. Archival of results, reports and paper (A)

For each of these steps, I have indicated a rough idea of the nature of the data/information storage environment required.

• Electronic Lab Notebook (ELN) - Primary and complete legal record of the research. Personally created and recorded by the researcher, but the formal responsibility of the PI.  Must be continuously and constantly available.  Data should be recorded to the researcher's laptop to guarantee offline access, but synched to a central database available for online access from any computer.  I very much prefer an "IMAP" model to a "web-form" model (see below). Data itself  must be backed-up, and should be secure against tampering/re-writing.  The ELN itself should be permanently stored by the PI and the researcher.
• Shared Database (SDB) - For external content and data that is of use to the whole group.  The obvious example is the PDF library of reference literature. Content is added by individual researchers and can be tagged/notated by individuals, but the collection itself should be group accessible.
• *Shared Output (SO) - This is content prepared by individual researchers, but that final versions of must be available (read-only) to the group.  Drafts and information needed during preparation reside in the ELN or as DO (see below).
• *Data Output (DO) - Content generated by calculations on local or external production resources.  Data must be staged physically on production resource, but must migrate to a central, group readable location.  Data should migrate to read-only.

• Special categories:
• Collaborative (C) - This is content that should have parallel multi-individual access to allow collaborative/interactive content generation (e.g., paper drafts).
• Archive (A) - Not a primary end point for data, but both Shared Output and Data Output (indicated with asterisks, above) should eventually be archived.

Based on the above, an ELN must be able to handle the following either in the ELN app itself, or via importing of third-party files.  The ELN must allow users to:

• Take notes, anytime and anywhere
• Handle scanned input, e.g. of handwritten notes or diagrams.  The inclusion of a working OCR pathway would be beneficial as well.  from school: including input of scans of handwritten stuff, allow diagramming.
• Enable both symbolic and numerical Math, spreadsheet capabilities, plotting and graphing, curve fitting
• Support presentation and text report generation, as well as image and diagram preparation
• Collect, and allow searching of the generated files, allow links pointing to specific locations of DO and SO objects.

More at the next update....