Electronic and other old fashioned lab notebooks in and out of formal settings

[Dan Kolis]

Here's an open procedural; question....

Whether its a formal institution trying to rally work of hundreds or people or a BIYer, notes in life sciences matter.

So, Im working on software and a side component of it is to populate electronic notebooks. The only one I can visualize in Jupyter really. the commercial ones I have looked at seem pretty yawn ready and expensive re ongoing costs...

Also a modality is real time completely, so some work has audio and probably video 'open wire; and shared ( started anyway ). I see both as adjunct functionality to the baseline work.

Anybody run into taking notes, the technology ofit, seriously ? Like a blog maybe, master document for planning etc.

Planned in is also forensic identity verification, so any sequence or PDB etc can't be mistakenly altered. That is, any change can be detected witout people overhead as it occurs.

I think serious researchers would generally like each others notebooks rather then formal papers most of the time. the magic is in the greasy uncertainty.

DIY here as a theme is understood, however, the base of life scienc people can be pretty docrinaire, this is an open ... :Huh first question.

ex question, are pair notebooks, one notes, versus Jupyter, etc .... worth it ? What about old timer sorta, who like paper ? Is a endless fat paper book in chrono order just scanned better then a Jupyter ? Is fussing with little programs a waste of time compared to paragraphs of vague maybe this maybe that, reasoning ?

Audio inserts.

Regards to all,
Daniel B. Kolis

[Abizar Lakdawalla]

or configure a web based soln such as smartsheets, notion, filemaker

--
-- You received this message because you are subscribed to the Google Groups DIYbio group. To post to this group, send email to diy...@googlegroups.com. To unsubscribe from this group, send email to diybio+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/diybio?hl=en
Learn more at www.diybio.org
---
You received this message because you are subscribed to the Google Groups "DIYbio" group.
To unsubscribe from this group and stop receiving emails from it, send an email to diybio+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/diybio/f55939b7-7064-474e-8daf-7660e9b0f5e2n%40googlegroups.com.

[Timothy Fallon]

Hi Daniel,

I think this is a critical issue, thank you for raising it. 

I personally use Onenote as my electronic laboratory notebook (ELN), which is fairly unstructured. It would be nice to have something with more structure (i.e. templates), and a very flexible API for plugging in workflows and other pieces of software.

I think there are very interesting developments in the “notebook” space, i.e. Notion (https://www.notion.so ; https://en.wikipedia.org/wiki/Notion_(productivity_software)) , Roam (https://roamresearch.com ), Obsidian (https://obsidian.md ), Craft (https://www.craft.do).

Presumably, life scientists & DIY biologists don’t need to reinvent the wheel to take electronic notes. 

I think, for a long time there were concerns about having “signability” and “traceability” for patents, so commercial ELNs used in Pharma had to include those. But, from my understanding now, at least in the USA & much of the world, patents are “first to file”, rather than “first to invent”, so I’m not sure how relevant these patent-induced notebook requirements are anymore. 

Over the the Laboratory Open Protocol (LabOP) language project, https://bioprotocols.github.io/labop/ , we have been trying to come up with a way to describe biological protocols in software.

A absolutely beautiful thing, would be if executing the protocol, also automatically put the results into your “electronic lab notebook”, but we (LabOP) are a very long ways off from that I think. Anyway, I mention it as I’d be happy to continue the discussion, both in the context of LabOP, and outside of it...

All the best,

-Tim

___________________________________

Timothy R. Fallon, PhD

NIH NRSA Postdoctoral Fellow - Moore Laboratory

Center for Marine Biotechnology and Biomedicine 
Scripps Institution of Oceanography 
University of California, San Diego 

tfa...@ucsd.edu
Google Scholar publications and citations
Website: http://photocyte.github.io

Pronouns: (he/him/his)

** I may send emails at odd hours due to my own flexible schedule. I respect your working patterns and do not expect responses outside of your working hours. **

[Dan Kolis]

Thank you for those line items. here are top level URLS and a one line summary from my peeking.

https://www.notion.so/
  Workflow GUI 'who does what, when' group tool.

https://www.smartsheet.com/
  Nested PERT charts as orthag. speadsheets.

https://www.claris.com/filemaker/
  Database add in tool, report generator.

I note { Pfizer-BioNTech or Moderna } spend > 1 Billion USF to get a mRNA vax shippable. Yet the essence of why there stuff was approached and AstraZeneca flopped would occupy maybe 6 printed page faces. Not opening the box of XX Billions in annual sales ... I think a notebook system scalable from free for students to supporting projects like this needs to grasp FDA sorta-kinda-traceability and where the files live matters greatly to decision makers.

Enabling both open ended research and "Do it because the boss says its right" is probably too much scope of work in a single IDE, but the underlying struct of knowledge might do this. I specifically 'diss the SAS model some lovely people who dont charge much or are free ( ex ad supported ) can be trusted with content. including all sorts of black hats. Jupyter has this utility... ipyng could live in highly secure boxes.. And starts free with no rules up to locked down like Nat. security is an interesting scope of work.

Interest in this is expanding. I note you cant send DNA clips via FB, I was corresp with a fine guy and tried and it bounced. was RNA, detected with there s/w and a polite note sent back and message suppressed. Don't try it if you don't want to endure annoyances... So maybe lab notebooks esp merged with real time people hookups will encounted this security in depth idea, too...  Of course some shared notebooks might automatically obfusticate DNA sequences and conf of Atoms like PDBs, etc so the shared recip would be denied some of the copy and paste of it entirely... But still get the higher level goals. presumably they could ask for more as required.... One thing about a paper book is it can be poked int a vault. Of course it can also be photocopied.

Im not trying to proposed to write one. I am proposing to avoid doing that but interface unusually well with one. Thermo Fischer has a nice one, kind of expensive but not terriblly so ( $ 1K / user seat per year )... A wrapper around Jupyter seems a possibility.

Regards,

Daniel B. Kolis

[Anthony VanHoy]

I struggled with this some time back. I agree with Timothy and settled on MS Onenote. I use Mathematica Home Edition for visualization as I am an academic mathematician. I have used Python and the Jupyter system.

Anthony

Get Outlook for Android

---

From: diy...@googlegroups.com <diy...@googlegroups.com> on behalf of Dan Kolis <dank...@gmail.com>
Sent: Monday, October 24, 2022 11:57:52 AM
To: DIYbio <diy...@googlegroups.com>
Subject: Re: [DIYbio] Electronic and other old fashioned lab notebooks in and out of formal settings

 

--
-- You received this message because you are subscribed to the Google Groups DIYbio group. To post to this group, send email to diy...@googlegroups.com. To unsubscribe from this group, send email to diybio+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/diybio?hl=en
Learn more at www.diybio.org
---
You received this message because you are subscribed to the Google Groups "DIYbio" group.
To unsubscribe from this group and stop receiving emails from it, send an email to diybio+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/diybio/7cc8ce8f-6df5-4646-b262-cb60fdb1d901n%40googlegroups.com.

[Dan Kolis]

Timothy R. Fallon said:
> I personally use Onenote as my electronic laboratory notebook (ELN), which is
> fairly unstructured. It would be nice to have something with more structure
> (i.e. templates), and a very flexible API for plugging in workflows and other
> pieces of software.

Dan says:

One notes is definitely reasonably capable. Have you ever tried Jupyter ? When you wish for workflows suddenly this really busts out the urge to be some sort of groupware; ex shared on somebody elses WWW for instance.

> I think, for a long time there were concerns about having “signability” and
> “traceability” for patents, so commercial ELNs used in Pharma had to
> include those. But, from my understanding now, at least in the USA & much of
> the world, patents are “first to file”, rather than “first to invent”,
> so I’m not sure how relevant these patent-induced notebook requirements are
> anymore.

Treaty of Paris international patents slowly wiped out first to invent. https://www.intepat.com/blog/patent/paris-convention-vs-patent-cooperation-treaty-pros-and-cons/ Slowly the PCT addenda rto the Paris convention has wiped out 'who invented this ?" fights with bound notebooks. 

> The Open Protocol (LabOP) language project,
> https://bioprotocols.github.io/labop/ , has been trying to come up with a way

> to describe biological protocols in software.
> A absolutely beautiful thing, would be if executing the protocol, also
> automatically put the results into your “electronic lab notebook”, but we
> (LabOP) are a very long ways off from that I think. 

A subsystem in a lap notebook for machine readable protocols is an obvious software component. Because its desirability is obvious does not make it easier to invent and make truly an asset in general.

Anthony said:
> I struggled with this some time back. I agree with Timothy and settled on MS
> Onenote. I use Mathematica Home Edition for visualization as I am an academic
> mathematician. I have used Python and the Jupyter system.

Dan says:

Do you think between Jupyter and One note either is much better for life science notebooks ?

Do you think multiple user issues etc is the deal breaker ... Execution of code in Jupyter is pretty fancy. Wow.

Regards,

Daniel B. Kolis

 

[Anthony VanHoy]

Hi Dan,

For most diybio folks I think OneNote would/might work great. If you need number crunching (biostatistics), then python/jupyter would be needed. 

Personally,  I have only been playing with diybio for the past year and wanted to start with something simple. I ended up developing an interest in replicating the biotech industries Nucleic Acid extraction technologies. So I really don't need anything more than something like Onenote.

Anthony

Get Outlook for Android

---

From: diy...@googlegroups.com <diy...@googlegroups.com> on behalf of Dan Kolis <dank...@gmail.com>

Sent: Monday, October 24, 2022 7:16:35 PM

To view this discussion on the web visit https://groups.google.com/d/msgid/diybio/d64b1ee0-e39b-4677-a8e5-26e358af9de7n%40googlegroups.com.

[Dan Kolis]

OneNote is fine if your in the Microsoft Ghetto. I've used it. It's a fine product. all this drivel: "Version 2.hsjsgj can only read Version fdfj if you have MS 10.6 or 11 or eat oatmeal, or have the Norwegian version... or: paid for for the PAYED-or-locked-out-399 version", etc. Is somewhere between distressing and amusing. Or the sadness of watching people realise, very, very slowly a spreadsheet is no way to consolidate most machine readable complicated ideas.
 
I sort of visualise a DIY movement as a leading edge of a broader base of practices generally. However, this corresp. has made me visualise a specific dichotomy. Its "DIY" somewhat equated to "loner" as opposed to formal science without exception sizable groups ? Certainly the rubber hits the road on a notebook most when one of two things happen:

• Its lost.
• Its shared.

Leaving an embedded running program in Jupyter may be a slight distraction as opposed to a passive formula for the maker(s), but its pretty instructive. I am sure of ONE thing for sure in my humble, finite life: I would rather have; Anybody's place-mat doodles and notebooks rather then any formal publishing output for any idea, whatsoever. That applies to patents or 'papers', anything. So I am in the wow ! The notebook is magic camp.

Maybe I visualize DIYers as more like moonlighters; that there daily roles are sciencey, or students of it anyway ( same thing from a content-of- head perspective, almost ).

Regards,

Daniel B. Kolis

[Timothy Fallon]

Hi folks,

Thanks for the continuing discussion on this. I participated in a seminar yesterday that was purporting the use of “Semantic Wikis”, for electronic lab notebooks, and other types of data organization. It seemed quite promising, and indeed is not the “Microsoft” or typical solution.

I am still learning about the concept, but am sharing the link to the seminar recording, in case others might also find it insightful:

https://twitter.com/openneurosci/status/1585579923917373441 

All the best,

-Tim

___________________________________

Timothy R. Fallon, PhD

NIH NRSA Postdoctoral Fellow - Moore Laboratory

Center for Marine Biotechnology and Biomedicine 
Scripps Institution of Oceanography 
University of California, San Diego 

tfa...@ucsd.edu
Google Scholar publications and citations
Website: http://photocyte.github.io

Pronouns: (he/him/his)

** I may send emails at odd hours due to my own flexible schedule. I respect your working patterns and do not expect responses outside of your working hours. **

--
-- You received this message because you are subscribed to the Google Groups DIYbio group. To post to this group, send email to diy...@googlegroups.com. To unsubscribe from this group, send email to diybio+un...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/diybio?hl=en
Learn more at www.diybio.org
---
You received this message because you are subscribed to the Google Groups "DIYbio" group.
To unsubscribe from this group and stop receiving emails from it, send an email to diybio+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/diybio/43d3f90f-82b9-4552-9952-5ce75c14638bn%40googlegroups.com.

[Dakota Hamill]

This is the best ELN I've found.  https://www.labarchives.com/

To view this discussion on the web visit https://groups.google.com/d/msgid/diybio/7C2BD3EA-3938-4FD0-8F2A-448E0E2869DB%40ucsd.edu.

[Dan Kolis]

Thanks Dakota,

seems a product with a vast heap of features. 

I think I will lean into Jupyter as the log subsystem with a way to add notes without struggle. How uncertainty is absorbed into notes is the central issue to me I think. My stuff expects the worker to be reasonably distracted and will try to absorb cause and effect of searches and genomic operations without any attention by the worker. 

Since this product takes lot fo file 'formats' I think the minimal exports I plan or have implemented will enable the motovated worker to use this well.

I can't quite visualize how educational institutions  andor 'other' ones look at costs, both instantaneously and ongoing. Any of the costs seem to be justifiable in the technical sense. but the cost per se is coupled to the commitment to really rely on it nearly indefinitely, really.

Im not yet ready to think this out but one Idea I have entertained is a sale of a few TerrByte HD which shows up completely ram jammed full of genomic data and search tools and any number of users can use it. for a one time cost, just the HD which is a linux system ready to plop into any computer.

The point is to avoid external DB for many things, for a few reasons like avoiding sneak peeks by other companies, speed, etc.

Paying people to think for infrastructure is expensive and delays ridden, the notion might be the variable facts are stored on a different system entirely. then periodically the entire system is replaced, some weekend in/out and the newer features and db are all there with no excuses, no installs, no new learning curve required, at all.  Notebooks are a small but important part of the offering(s).

Regards,
Daniel B. Kolis

my ref: chW27a 27 Oct 2022

[John Griessen]

On 10/26/22 11:27, Dan Kolis wrote:
> I sort of visualise a DIY movement as a leading edge of a broader base of practices generally. However, this corresp. has made me
> visualise a specific dichotomy. Its "DIY" somewhat equated to "loner" as opposed to formal science without exception sizable
> groups ? Certainly the rubber hits the road on a notebook most when one of two things happen:
>

> * Its lost.
> * Its shared.

>
>
> Leaving an embedded running program in Jupyter may be a slight distraction as opposed to a passive formula for the maker(s), but

> its pretty instructive. I am sure of ONE thing for sure in my humble, finite life: I would rather have; /*Anybody's
> place-mat doodles and notebooks rather then any formal publishing output*/ for any idea, whatsoever. That applies to patents or
> 'papers', anything. So I am in the wow ! _The notebook is magic camp_.

The sharing aspect is a main thing. So, that makes me think of a little database project for tracking your inventory of parts
called partkeepr, and a new rewrite called Limas: https://github.com/Lopo/Limas that is built with a toolset called Symfony.
It's changing, and not ready yet, but has been useful to people wanting to efficiently track myriad things and query the database
about them. Mostly what I find about query jupyter is running queries from jupyter, not on jupyter content.

Having a structure and queries possible is important. (Whether each content creator uses it or not.)
Searchability and structure makes finding a reason to collaborate easier.

[Dan Kolis]

You mention Jupyter seems to be visualized as populated with endless little Python programs and graphs, maybe some tables, etc and not "maybe this maybe that" reasoning.

I notice for USA NIH R&D Jan 29 2023 is the deadline for a new requirement for grants to explain in considerable precision how results are shared. 

Your example of maybe an embedded inventory control data base is interesting. Labelling, barcodes ( whatever ) and sample naming is pretty essential. 

"Sharing" with people on site in your group, in the 'group' but in facilities at a distance, and explaining to 3rd parties trying to graft the new knowledge onto their projects are really, really different sorts of communication.

Probably your line item of control is accessible via some program like Python script or from a notebook.  

So, does a project end up with a 'master' 'lab book' with all of them poured together, or what ?

Why do little programs instead of paragraphs of indecision end up in notebooks ? A: Notebooks feel more like personal assets than project line items...

I suggested to an insider ISO 900X would be a better credential for science then these little vows of happy talk. He said: "Too expensive", I disagree. but not all science could be written up in that form. Theres no single answer to the open questions, none at all... But there can still be incremental improvement..

The new requrement:
Beginning in January 2023, the Final NIH Policy for Data Management and Sharing (NOT-OD-21-013) will require researchers to include a data management and sharing (DMS) plan in funding applications. In preparation for the policy implementation, NIH has launched a Scientific Data Sharing Website.

Interesting stuff,

Daniel B. Kolis