Converting to JSON

249 views
Skip to first unread message

Tahor SuiJuris Ben

unread,
Nov 21, 2018, 3:00:37 PM11/21/18
to openscr...@googlegroups.com
Depending upon what format the data, once converted to CSV then ... Quick and easy, using simple Excel VBA to convert to JSON.





Tahor SuiJuris Ben Berith
Yadah Yasha Ahayah
h3034 h3467 h1961
A MANDATORY verbal "confession" in sequence Colossians 3:17 / John 14:6)
Proverbs 15:32
Acts 17:26, 27 / Judges 2:3, 22
• a TIME limit for all to SEEK •
Deuteronomy 4:28, 29 / Isaiah 55:6, 7
---------------------------------------------------------------------
Electronic Privacy Notice. This e-mail, and any attachments, contains information that is, or may be, covered by electronic communications privacy laws, and is also confidential and proprietary in nature. If you are not the intended recipient, please be advised that you are legally prohibited from retaining, using, copying, distributing, or otherwise disclosing this information in any manner. Instead, please reply to the sender that you have received this communication in error, and then immediately delete it. Thank you in advance for your cooperation.
---------------------------------------------------------------------




‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, November 21, 2018 11:48 AM, Peter Passchier <pe...@passchier.net> wrote:

Loads of options for XML to JSON:

Blessings,
Peter

On Wednesday, November 21, 2018 at 11:01:49 AM UTC-8, Eric S. Wong wrote:
@Ben, Oh so I just took a look at the link you sent me and XML is definitely prevalent. I must've misread but you meant that we convert the XML to JSON. Yeah there's got to be a way!

On Wed, Nov 21, 2018 at 10:52 AM Eric S. Wong <erics...@gmail.com> wrote:
Thank you for all the replies. I'm glad that you guys are active in this work.

I'll take a look at the resources mentioned.

@Peter, I'm not sure what you're saying. But the only source I found was on Ben's opensource javascripture. Other than that you guys mentioned some resources that I'm going to take a look at.

@Ben, yeah if you can share with me the fork that would be great! Thank you for the link to you source. I'm not sure how crosswire's module works but it doesn't necessarily seem too bad if we just cross match strong's words with a Westminster Leningrad Codex for the Hebrew then something like a Codex Sinaiticus for Greek. That way we can have most if not all the words having a strong tag from our algorithm. From there we can cross check from a more accurately done file by human filter. Yeah we can definitely run a script to convert to JSON. I'm literally just using iOS's JavaCore Framework, changing your variable and using your whole file as a massive JSON import into my app lol!

@All - Definitely make a slack, this would make the process a lot smoother. Setup is so fast... Hit us up and I'll join asap!




On Wed, Nov 21, 2018 at 10:30 AM Jesse Griffin <jag...@gmail.com> wrote:
Yes, this is a good place for that discussion Tahor.

The Open Scriptures Hebrew Bible is tagged with Strong's numbers and has most of the forms parsed, you can download it from https://github.com/openscriptures/morphhb/releases .

We'll have another release soon (maybe January time frame) with 100% of the forms parsed.

Thank you,
Jesse Griffin


On Wed, Nov 21, 2018 at 12:52 PM 'Tahor SuiJuris Ben' via Open Scriptures <openscr...@googlegroups.com> wrote:
I need to create a database with both the original languages with English parallel including Strongs #.

I am very hopeful that the data resources may prove a solution.

Thank you Ben, shall view the github link you shared.

My question actually was about a central discussion area, would the Open Scriptures Google groups be an appropriate place to discuss the data?

Tahor SuiJuris Ben Berith
Yadah Yasha Ahayah
h3034 h3467 h1961
A MANDATORY verbal "confession" in sequence Colossians 3:17 / John 14:6)
Proverbs 15:32
Acts 17:26, 27 / Judges 2:3, 22
• a TIME limit for all to SEEK •
Deuteronomy 4:28, 29 / Isaiah 55:6, 7
---------------------------------------------------------------------
Electronic Privacy Notice. This e-mail, and any attachments, contains information that is, or may be, covered by electronic communications privacy laws, and is also confidential and proprietary in nature. If you are not the intended recipient, please be advised that you are legally prohibited from retaining, using, copying, distributing, or otherwise disclosing this information in any manner. Instead, please reply to the sender that you have received this communication in error, and then immediately delete it. Thank you in advance for your cooperation.
---------------------------------------------------------------------




‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, November 21, 2018 9:11 AM, Ben Dwyer <b...@scruffian.com> wrote:

The code for javascripture is open source, so anyone can access it here: 

On Wed, 21 Nov 2018 at 16:34, Peter Passchier <pe...@passchier.net> wrote:
Why have these most important recources not been shared in a more accessible way..? It seems Christians in the past have tended towards proprietary solutions and formats, which is very sad. Let this be the generation to change this all, and make the word and crucial tools and utilities accessible as openly and freely as possible, so collaboration and access gets easier and more abundant.

Blessings on you all,
Peter


--
You received this message because you are subscribed to the Google Groups "Open Scriptures" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscripture...@googlegroups.com.
To post to this group, send email to openscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Open Scriptures" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscripture...@googlegroups.com.
To post to this group, send email to openscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Open Scriptures" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscripture...@googlegroups.com.
To post to this group, send email to openscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to a topic in the Google Groups "Open Scriptures" group.
To unsubscribe from this group and all its topics, send an email to openscripture...@googlegroups.com.
To post to this group, send email to openscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
-Eric Wong


--
-Eric Wong


--
You received this message because you are subscribed to the Google Groups "Open Scriptures" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscripture...@googlegroups.com.
To post to this group, send email to openscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Troy A. Griffitts

unread,
Nov 21, 2018, 4:06:23 PM11/21/18
to openscr...@googlegroups.com

Hi guys,

May I make a humble suggestion with regard to converting XML to JSON?

If you'd rather have a JSON structure as opposed to an XML structure, I don't think you will be served well by a blind conversion.

JSON alone doesn't capture well (if even sanely), all things from our (CrossWire / OSIS) documents.  Here is a simple example of a few verses with words of Christ marked, and a few footnotes.  I could come up with a much more difficult example which would include lemma and morphology tagging, along with poetry and paragraph, but I wanted to keep it simple for an example:

<div type="book" osisID="Matt">

...

<div type="chapter" osisID="Matt.4>

...

<verse osisID="Matt.4.18" n="18">

<title subType="x-preverse" type="section">The First Disciples</title> <note n="A" osisID="Matt.4.18.xref.A" type="crossReference"><reference osisRef="Matt.4.18-Matt.4.20">Matt 4:18-22: <hi type="italic">Mark 1:16-20</hi></reference>; <reference osisRef="Luke.5.2-Luke.5.11">Luke 5:2-11</reference>; <reference osisRef="John.1.40-John.1.42">John 1:40-42</reference></note>Now as Jesus was walking by <note n="B" osisID="Matt.4.18.xref.B" type="crossReference"><reference osisRef="Matt.15.29">Matt 15:29</reference>; <reference osisRef="Mark.7.31">Mark 7:31</reference>; <reference osisRef="Luke.5.1">Luke 5:1</reference>; <reference osisRef="John.6.1">John 6:1</reference></note> the Sea of Galilee, He saw two brothers, <note n="C" osisID="Matt.4.18.xref.C" type="crossReference"><reference osisRef="Matt.10.2">Matt 10:2</reference>; <reference osisRef="Matt.16.18">16:18</reference>; <reference osisRef="John.1.40-John.1.42">John 1:40-42</reference></note> Simon who was called Peter, and Andrew his brother, casting a net into the sea; for they were fishermen.

</verse>

<verse osisID="Matt.4.19" n="19">

<transChange type="tenseChange">And He said</transChange> to them, <q who="Jesus">“<note n="1" osisID="Matt.4.19.note.1" type="explanation">Lit <hi type="italic">Come here after Me</hi></note> Follow Me, and I will make you fishers of men.”</q>

</verse>

...

</chapter>

...

</div>

A straight XML to JSON conversion will change all the quotes and references within a verse to json properties, maybe split the verse up into an array of verses because the text segments are divided by these... I actually have no idea of any kind of sanity JSON alone would produce.

As I mentioned before, you probably just want the book chapter verse structer to be converted to JSON.  All the other markup, you are better off leaving as XML within your JSON structure or changing to markdown or something else.

I know the cool kids these days like JSON, but for representing rich document markup, I am not sure how much sense this actually makes.

If easy verse retrieval is what you are after, you should be able to simply say something like:

$('verse [osisID=Matt.4.18]') to retrieve the content directly from the document... but I would suggest even that.

Our opensource tools allow you to do something like:

var book = SWORD.mgr.getModuleByName("NASB");

book.setKeyText("Matt.4.18");

var verseHTML = book.getRenderText();

I hope this helps and might encourage you to investigate our free open-source tools and large library of resources.

Blessings and Happy Thanksgiving!!!

Troy

Tahor SuiJuris Ben

unread,
Nov 21, 2018, 4:17:36 PM11/21/18
to openscr...@googlegroups.com
Excellent point Troy, thus the introduction of the possibility of NoSQL.
Specifically MarkLogic.

Tahor SuiJuris Ben Berith
Yadah Yasha Ahayah
h3034 h3467 h1961
A MANDATORY verbal "confession" in sequence Colossians 3:17 / John 14:6)
Proverbs 15:32
Acts 17:26, 27 / Judges 2:3, 22
• a TIME limit for all to SEEK •
Deuteronomy 4:28, 29 / Isaiah 55:6, 7
---------------------------------------------------------------------
Electronic Privacy Notice. This e-mail, and any attachments, contains information that is, or may be, covered by electronic communications privacy laws, and is also confidential and proprietary in nature. If you are not the intended recipient, please be advised that you are legally prohibited from retaining, using, copying, distributing, or otherwise disclosing this information in any manner. Instead, please reply to the sender that you have received this communication in error, and then immediately delete it. Thank you in advance for your cooperation.
---------------------------------------------------------------------




‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

llew...@vdm.io

unread,
Nov 28, 2018, 6:23:44 PM11/28/18
to Open Scriptures
Troy to say
XML to JSON conversion will change all the quotes and references within a verse to json properties, maybe split the verse up into an array of verses because the text segments are divided by these... I actually have no idea of any kind of sanity JSON alone would produce

Is simply not right, I mean that dealing with quotes in the text was maybe the first thing they resolved when they punched out the JSON conventions in many of the modern langues that work so easily with JSON. It is not about being cool either it is just practical. 

Here is a line I quickly dumped from the pysword lib (used the KJV, not the NASB) just to prove the point. All I did in python is json.dump() with the dictionary object, not even the extreme measures from jq... just json.dump() . JQ does the same but in c

{
  "chapter": 4,
  "verse": 19,
  "name": "Matthew 4:19",
  "text": "<w lemma=\"strong:G2532 lemma.TR:και\" morph=\"robinson:CONJ\" src=\"1\">And</w> <w lemma=\"strong:G3004 lemma.TR:λεγει\" morph=\"robinson:V-PAI-3S\" src=\"2\">he saith</w> <w lemma=\"strong:G846 lemma.TR:αυτοις\" morph=\"robinson:P-DPM\" src=\"3\">unto them</w>, <q marker=\"\" who=\"Jesus\"><w lemma=\"strong:G1205 lemma.TR:δευτε\" morph=\"robinson:V-XXM-2P\" src=\"4\">Follow</w> <w lemma=\"strong:G3694 lemma.TR:οπισω\" morph=\"robinson:ADV\" src=\"5\"/><w lemma=\"strong:G3450 lemma.TR:μου\" morph=\"robinson:P-1GS\" src=\"6\">me</w>, <w lemma=\"strong:G2532 lemma.TR:και\" morph=\"robinson:CONJ\" src=\"7\">and</w> <w lemma=\"strong:G4160 lemma.TR:ποιησω\" morph=\"robinson:V-FAI-1S\" src=\"8\">I will make</w> <w lemma=\"strong:G5209 lemma.TR:υμας\" morph=\"robinson:P-2AP\" src=\"9\">you</w> <w lemma=\"strong:G231 lemma.TR:αλιεις\" morph=\"robinson:N-APM\" src=\"10\">fishers</w> <w lemma=\"strong:G444 lemma.TR:ανθρωπων\" morph=\"robinson:N-GPM\" src=\"11\">of men</w>.</q>"
}


{
  "chapter": 4,
  "verse": 19,
  "name": "Matthew 4:19",
  "text": "And he saith unto them, Follow me, and I will make you fishers of men."
}

Why not allow the cool kids access, to a technological pipeline of text distribution that is clearly more popular. It is not like we are saying take away XML, just JSON also.


Ben Dwyer

unread,
Nov 29, 2018, 7:51:12 AM11/29/18
to Open Scriptures
> Why not allow the cool kids access, to a technological pipeline of text distribution that is clearly more popular. It is not like we are saying take away XML, just JSON also.

Yes please!

Troy A. Griffitts

unread,
Nov 29, 2018, 10:53:03 AM11/29/18
to openscr...@googlegroups.com
Hi guys. I am not sure you read carefully enough my email. I was recommending against brute force convertors recommended in earlier emails which would try to convert MORE THAN JUST book, chapter, and verse structure to JSON. A brute Force XML to JSON convertor would also try to convert the quote tags (please note the example given <q who="Jesus">) and all the other reference tags (footnotes, crossrefs, etc).

In my email, my recommendation was NOT to use a brute force convertor, but to be more intelligent and simply convert only the book, chapter, and verse structure to JSON and to leave the rest of the XML tags as-is or convert them to markdown. This is exactly what you have done, so I believe we are in agreement to that point.

The small bit we might still disagree on is:
How is this any more useful? You still need to parse all the rich markup to do anything useful, like show the footnotes or cross-references or color Christ's words red or show hover help using Strong's numbers. And what benefit do you gain having the book chapter verse structure in JSON? Don't you still want to retrieve a verse or array of verses from a web service? You don't simply want to load the 20MB KJV text plus entire Greek lexicon plus all other aids, into the user's browser as a JSON structure do you?

The point is, a richly marked up document is not well suited to be completely converted from XML to JSON. Only the navigation structure, but then what does that give you? A web service is better suited to serve the structure on demand.

Hope this clears things up a bit.

Troy
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

llew...@vdm.io

unread,
Nov 29, 2018, 12:42:39 PM11/29/18
to Open Scriptures
Okay, so we still use xhtml in the text and that is totally normal. I see your hint to blind conversion as meaning brute Force XML to JSON conversion. Okay, point taken, but it also seems like you did not from what I wrote get my point.

The point is why not JSON, since there is no real reason not to use it, it is that simple. The fact that you are comfortable with xml does not mean we all must be.. its a free country :) and JSON can do the work, because no one I know, who really prefers to work with JSON, and has done so for many years will want to reinvent the wheel with some kind of brute force conversion. Saying it is a bad choice is not really true, but a preference really, it more like we should have the freedom to choose, and it should not even be a issue that the choice is being looked at, but just normal.

I would like to think that since I like to work outside of the GUI, in the command line, and therefore have a much clearer perspective on the possibilities and freedom of the shell. This does not make those who want the use cPanel to manage a web-server wrong. They want to use that, sure why not... I use docker and deploy scripts that I have honed in over time for my own use and ease. Yes I think they are in overkill and paying for what can be done for free. But it is their choice in the freedom of the open market.

So it seemed to me from your first post like all those who want to use JSON are kids who must still grow up, and I disagree. Sorry for thinking that was what you meant. If your point was simply hey if you use JSON take care of doing it right... then my bad... I feel ashamed and I am sorry... really, please forgive me. But if you still think xml has some kind of edge on JSON then I think you are entitled to think that, like some like one car brand over another. But JSON is not inferior, just different, and when it comes to weight on a page the difference is not that big either.

Most will say it depends on the use case (check out the link), and so having both available in the most important document in the world is to me like of course why not.

I rest my case and would agree with, new is not always better, but could be if given a chance.

Craig

unread,
Nov 30, 2018, 9:54:04 AM11/30/18
to Open Scriptures
I rarely chime in on these discussions but just want to make a small point: XML is a mark-up languages that is made for what you're doing here. JSON is a way of representing objects. They solve two different problems that are arguably unrelated.

Using JSON syntax within the XML document to represent book/chapter/verse references would be like saying you're going to use XML for all the structural components of the document (like denoting where each book of the Bible begins) but the text itself is going to be PDF. Or that everything is going to be XML except that words of Christ are going to be rendered in red, captured as a JPEG, then inserted as a Base64 string into the attribute value of an XML tag.

It's also like saying that the chapter and verse references are going to be XML snippets in a JSON document, like {"book":"<book name=\"Matthew\">", "chapter":"<chapter name=\"1\">"}. It's unnecessary and unhelpful.

I apologize if I'm completely misunderstanding the motivation for the suggestion to represent Bible references in JSON in an XML document. It was just too strange of an idea for me to skim past without saying something. :-)

Craig

ref...@gmx.net

unread,
Nov 30, 2018, 10:37:00 AM11/30/18
to openscr...@googlegroups.com
One aspect has not been mentioned. For OSIS a verse is not by necessity a balanced XML fragment. Biblical documents have sections and paragraphs, there is poetry and prose. All these things can cut through a verse, leaving the individual verse unbalanced and not valid XML.

Troy is more dismissive than I am at this moment, I am happy to put the work in to make Libsword spit out JSONified data, but only if there is a real need and a valid concept presented which shows a decent understanding of what our library does. To do a verse per line JSON collection of XML chunks seems like a lot of effort for little gain, you could just as well run the out put of the diatheke (a command-line front-end of CrossWire) through a simple filter and have what is asked for so far.

Peter


Sent from my mobile. Please forgive shortness, typos and weird autocorrects.
--

Peter PilMung

unread,
Jan 1, 2019, 11:02:11 PM1/1/19
to openscr...@googlegroups.com
I have a bible version(the rights belong to us).
When correct typos we found, we have to correct in InDesign, and the app, the web- one after another.

To save time and energy, anyone can help how to change in only one file/platform and it will change  to all the rest.

Any idea? Any technique?

Thanks.

Peter

ref...@gmx.net

unread,
Jan 2, 2019, 2:33:38 AM1/2/19
to openscr...@googlegroups.com
The most sensible format to do all these things from is probably USFM as so many pathways from there are well defined and sorted. The alternative is to use OSIS. Latter is ultimately more ideal, but lack of exports into commercial tools like InDesign require more expertise to do things yourself.

If you have money, but no time or expertise, USFM, if you want total control of all platforms and pathways using open source tools and have time and expertise to contribute to the community where pathways s Nd tools lack features etc, OSIS

Ben Dwyer

unread,
Jan 3, 2019, 7:04:42 AM1/3/19
to Open Scriptures
I have created a version of the sword-to-json module that allows it to generate JSON with embedded strongs numbers and morphology. You can use it here: https://github.com/danday74/sword-to-json/pull/3

Let me know if you have questions on how to use it, or suggestions.

Tahor SuiJuris Ben

unread,
Jan 5, 2019, 12:06:47 AM1/5/19
to openscr...@googlegroups.com
Hello Ben,

How exciting!

Do you have a front end that displays a visual?

How is the data structure of the relationship to the English text to Strong's and the morphology?  Are you using SQL or noSQL on the backend?

Tahor SuiJuris Ben Berith
Yadah Yasha Ahayah
h3034 h3467 h1961
A MANDATORY verbal "confession" in sequence Colossians 3:17 / John 14:6)
Proverbs 15:32
Acts 17:26, 27 / Judges 2:3, 22
• a TIME limit for all to SEEK •
Deuteronomy 4:28, 29 / Isaiah 55:6, 7
---------------------------------------------------------------------
Electronic Privacy Notice. This e-mail, and any attachments, contains information that is, or may be, covered by electronic communications privacy laws, and is also confidential and proprietary in nature. If you are not the intended recipient, please be advised that you are legally prohibited from retaining, using, copying, distributing, or otherwise disclosing this information in any manner. Instead, please reply to the sender that you have received this communication in error, and then immediately delete it. Thank you in advance for your cooperation.
---------------------------------------------------------------------




‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

d...@vdm.io

unread,
Jan 11, 2019, 11:59:32 AM1/11/19
to Open Scriptures
Ben this is exactly what I dido, and I was stopped like three months ago. I had to delete all my work... so my head is still spinning.... 

All I did more, is make the result accessible to the open web via an url, for free... then I got taken down.

I still have all the code local backed-up and also the JSON files, I can share access to it with you of gitLab, so send me your username (for gitlab) and I will give you access.

ref...@gmx.net

unread,
Jan 11, 2019, 12:54:02 PM1/11/19
to openscr...@googlegroups.com
Among the Bibles you publish this way is at least one under copyright, the ESV, possibly more.

Please take these down. There is no justification to do this.

Sent from my mobile. Please forgive shortness, typos and weird autocorrects.


-------- Original Message --------
Subject: Re: Converting to JSON

Ben Dwyer

unread,
Jan 11, 2019, 5:00:44 PM1/11/19
to Open Scriptures
These are already publish in JSON in that repo. My change just adds extra data to the JSON. If you have a problem with this I suggest you contact the original author of sword-to-json, but I really don't understand what the objection is.

ref...@gmx.net

unread,
Jan 11, 2019, 5:06:33 PM1/11/19
to Ben Dwyer, Open Scriptures
The objection is to publishing texts obtained from our repositories for which you do not hold permissions and which are not in the public license domain or otherwise freely licensed.

Peter

Ben Dwyer

unread,
Jan 11, 2019, 5:19:12 PM1/11/19
to openscr...@googlegroups.com
Which repos are you referring to? The crosswire ones?

d...@vdm.io

unread,
Jan 11, 2019, 5:20:23 PM1/11/19
to Open Scriptures
Ben it is simple, the love of money is the root of all evil. Men have added copyright to the Bible so that they can sell it, and collect royalties from others who want to do the same. It is a mess I tell you... so we are at the point were you are blocked by the SWORD project from giving public access to the scripture (public like in open JSON api)

Sure if the scripture is only on one device or one desktop all is fine, and they allow that, but the moment you let other developers (JavaScript developers) access it without directly querying the modules of the SWORD project then there seems to be issues.

I personally have been trying to understand this for a few months now.. and all I come to conclude is that the SWORD project obtained permission to use copyrighted Bibles but in their closed system (not for the web) so if we want to share it with the web, we must again get direct access (written permission) from these translations.

I then asked it they could point out to us which of these translations are able to be used publicly (so we can only use those), but then the whole debate of whether JSON is truly able to port scripture correctly came up. Round and round we go... welcome to open scripture that is not really open.

Hey if anything I said is not true, please correct me, I mean it will be like music to my ears....

d...@vdm.io

unread,
Jan 11, 2019, 5:28:33 PM1/11/19
to Open Scriptures

Troy A. Griffitts

unread,
Jan 11, 2019, 7:01:16 PM1/11/19
to openscr...@googlegroups.com

Dear OpenScriptures,

Just a short note about CrossWire and what we do.

All software we write is open source.

We are NOT the authoritative source or copyright holder for any Biblical material (save a very few exceptions, e.g., the KJV2003 Project which we spearheaded to produce the first open and freely available KJV text with Strongs markup http://crosswire.org/sword/kjv2003/ )

CrossWire exists as a community of engineers who wish to come together and build free software to support Bible researchers, translators, missionaries, and other ministry organizations.  We are a non-income organization.

The important point is that we are NOT a source for Bible texts themselves.  We never hold an authoritative source of texts on our server (excluding a very few (one or two) as mentioned above).

Don't forget there is a difference between "storage format" and "markup format." A SWORD module is typically a compressed and index blob which can be used with our open source engine.  That blob contains text which is marked up (often richly) in a number of different formats: OSIS, ThML, GBF, TEI are the major markup formats our engine supports.  Our engine can get you the data for any verse and all the metadata associated with that verse and output that verse in a number of output formats we support: HTML/CSS, RTF, LaTeX.

There is not 1 installation source for CrossWire modules.  Our open source software engine supports autodiscovery of Remote Installation Repositories and we encourage publishers to host their own.  Here is a helpful diagram of data flow at CrossWire:

http://crosswire.org/sword/develop/dataflow/

We try to ALWAYS list in our data module where we obtained the text and where we recommend anyone else obtain the text who wishes to use the same text outside of our community.

We don't want to discourage you from working outside of our community.  We only wish to discourage you from using CrossWire as an authoritative data source for biblical material outside of our community, and it is only from good intentions.  We have already converted the material from the authoritative source and often reconvert the material when we find errors in our tools or when the authoritative source makes an update.  We don't wish to encourage a conversion to another format from our conversion of the authoritative format.

Again, we don't with to discourage you from working outside our community.  You are always welcome to share with us, or do your own thing, but please don't consider our project a data source of biblical material.  That is not our purpose and our desire for you to not use our converted data isn't because we are selfish.  It is because we don't want to propagate conversion errors we may have made and we don't want to BECOME or BE SEEN as an authoritative data source of biblical material.  We are not.  You should contact the publishers of you wish to use their text directly and gain their permission, if they require it.  You are welcome to use our module library as a means to find all those sources.  We try to make that easy for other projects by including a "Source=" entry in each module's .conf file.

Finally, there are other projects who may consider themselves as a source for biblical material.  Please see Michael Johnson's great work at https://ebible.org/download.php

Sharing in Christ's work,

Troy

d...@vdm.io

unread,
Jan 11, 2019, 9:00:33 PM1/11/19
to Open Scriptures
Thanks Troy for your kindness in answering, and giving some more light to the subject.

Please I know Crosswire/Sword Project is not the copyright holders... I also know that you are not making money with the scripture. Yet I don't agree that anyone on earth today can claim ownership on the Holy Word of God. They are at best just the translators, and that does not make them the owners. Yet I also know that the debate is not in scope of the Sword Project, as you said it is a community of developers who build a table to place the Bibles on.

My question has only been simply this, can I place this table so that the public can reach it. That simple... we have projects like Pocket Sword and it is a Public App, and it can download all the translations on to any device, no problem. Why is that so different from a website? or an API. No one who reads the Bible in the Pocket Sword is asking permission... So to me it does not make sense that only none programmers can access these text without permission, and the moment it is on a website it is copywriter infringement.

So now we must ask permission... when did Pocket Sword ask permission, o yes it is part of the projects in the community. So what should be done to build an API that is also part of the community, so we can give access to developers to via a HTTP query get scripture back to add as quote, or load a chapter, or a book.... I mean a simple API that continues to do what you said you are doing:  support Bible researchers, translators, missionaries, and other ministry organization

Why cant anyone just give us a list, saying these translations are able to work this way in an open API, because I know that some of the copyright notices in the crosswire models say only the crosswire has permission to use it, and yet we find these translations being used everywhere.

So if I understand this correctly you are saying the community has these modules and they can be hosted in any project, but then at the same time we need to ask permission from the publishers again if that project is not part of the Crosswire projects. I mean do I understand you correctly, and then you can't really say this list do not require that you ask permission, because the community don't want to really be seen as distributors.

Wow okay hmm lets say you were going to build a website for your local Church, and they would like people to read the Bible on their website, what translation would you use? I mean seeing they want as many on the website as possible... but don't really want to go around asking permissions and sign contracts. Just plain simple reading the Bible on many languages on your local Church website, what list of translation will you use.

That is simple what I want to provide for Churches and ministries, an easy way to add the scripture on their websites. Just like Pocket Sword.... I install it and use it, and I never need to agree to anything.... or ask permission. So the Churches and Ministries can install a component unto there various platforms, and via this API get the scripture.

Open Scripture!

My first strong remark earlier "money is the root of all evil" is aimed at the fact that there are already API's like these, but they ask money. So my point, if there is money suddenly it can all be done. If we say we are going to do it for free... then hmmm there is this issue and that issue. So I am saying this must change, this is wrong. I am willing to commit the time for free to make this happen. There must be a free API that is as powerful and convenient as the paid options out there.

Freely you have received freely give, we must do this.

Michael Johnson

unread,
Jan 11, 2019, 11:04:29 PM1/11/19
to openscr...@googlegroups.com
Copyright law and the way Bible copyright owners treat it is a real problem. No matter how strongly you feel about every Bible translation needing to be open for access and use by all in any way they like, the laws of almost all countries and some international treaties say differently. A long time ago, I asked God about this, and he answered with profound wisdom. (Of course.) He told me to do a Bible translation. To make a long story very short, that is why the World English Bible is in the Public Domain. I strongly support and encourage the use of Public Domain Bibles and Creative Commons licenses with Bibles, especially CC-BY and CC-BY-SA. With those, I can freely share Bibles in downloadable and sharable formats, with no limits on distribution and redistribution. Those are the Bibles you see without the gray background at https://eBible.org/find/. The ones with the gray background are generally limited to distribution in just some formats and/or from specific web sites. Some of those that are totally missing from that list, like the ESV, NIV, Spanish RVR 1960, and Czech KMS, are missing because of copyright owner restrictions.

So what do we do? I recommend focusing on what we CAN do legally (at least according to the laws of the countries we are in and the applicable treaties), in so far as the law of the land is not contrary to God's law. (In some places, just owning or sharing a Bible is against the law. I don't censor my own Bible sites to accommodate such laws that apply in other countries, and I anonymize all of my access records for statistics, then delete the original server logs to protect people in those places. But I digress...)

Therefore, if Crosswire gets permission to distribute the ESV in their main repository for Sword Project use, that is great! But it does not imply that anyone else (me included) can use it for anything else but in Sword apps. BUT, if there is an open access license on a Bible Translation, or it is in the Public Domain, then by all means, use it as the license allows, or if it is in the Public Domain, use it in any way that would please God. But please don't make it harder for Crosswire and others (including me) to get permission to distribute copyrighted, proprietary-licensed Bible translations by appropriating them for use and distribution under terms that violate the license of the copyright owner. Once someone posted a copy of a Cambodian Khmer Bible on a blog in Australia, then someone at UBS noticed and asked me to take my copy down as well as theirs. (I couldn't take their copy down, but I took mine down.) I later cleared up the permission I had from the Bible Society in Cambodia and was able to put it back up, but only in the format that is most difficult to copy. I could mention a couple of other incidents, but you get the idea.

Focus on what you can do legally... and leave the rest for later. Maybe the copyright owners will have a change of heart, or maybe someone will come up with an open source Bible translation in that language. Check out https://unfoldingword.bible/ for some hope.
--
You received this message because you are subscribed to the Google Groups "Open Scriptures" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscripture...@googlegroups.com.
To post to this group, send email to openscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/openscriptures.
For more options, visit https://groups.google.com/d/optout.


--

Aloha,
Michael Johnson
PO BOX 881143 • PUKALANI HI 96788-1143 • USA
mljohnson.org • Phone: +1 808-333-6921 • Skype: kahunapule

ref...@gmx.net

unread,
Jan 12, 2019, 1:58:47 AM1/12/19
to openscr...@googlegroups.com
Llewellyn, You ask why can CrossWire not give a list of texts you can do what you like with.

Every single module of ours has a line DistributionLicense explaining exactly and in as little words as possible and as standardized as possible what you can do and what you can not. It also has a line TextSource which says where we got our text from. And I have pointed you at it. Several times over. You can read these lines, think it through and act upon it. You claim this is too difficult???

It is only laziness on your side, Llewellyn to ask for more. It is great that Michael tries to really do make it easy by colour coding his webpage, but, really???? You tell me that you are able to open up a module with your API , but are not able to read and think through what a line in the conf file called DistributionLicense means????

You tell me that a line "Copyrighted by XYZ. Distribution permitted to CrossWire" gives you sleepless nights to understand????? If it was not so tragic you would make me laugh.

If you take a copyrighted text for which you have not been given a permission to distribute and put it in a website, or on your GitHub account then you are in breach of copyright. You might not like that, I might not like it, but in the end you are acting illegally and at the same time you damage those who try to give access to the text legally. As Michael has quite eloquently pointed out. He lost access to a text because a fly-by-night decided to ignore copyright and republish a text Michael had done the legwork and obtained permission for.

We two, Llewellyn have been in a long and somewhat painful private conversation and I think I would like to bring this to a public conclusion now. It is your responsibility alone and not ours or mine to figure out what you can do with a given text under copyright. We are already doing way more for you than most others who distribute texts by both giving you a clear line in the conf file for each text and pointing you as much as possible at the origin and source of our texts.

Finally, for those who somehow do not grasp why it is a stupid and irresponsible thing to take a bunch of modules and convert then into another format (here json) and redistribute them, quite irrespective of legalities, please have a look at our bug tracker. Right now there are 130 open bugs on our modules. They go from the minor to the devastating. Some are really upstream, some are systematic and reflect upon our own processes. You take these modules, subject them to yet another conversion and republish them you are compounding our errors and make them worse.

If people use our modules, every time they refresh their apps' module store they see that a text has been updated and corrected. Once you break this connection you are assuming responsibility for the correctness of the texts you distribute. Fine if what you share are fairy tales. Not fine if it is the word of God.

Peter



Sent from my mobile. Please forgive shortness, typos and weird autocorrects.


-------- Original Message --------
Subject: Re: Converting to JSON
--

d...@vdm.io

unread,
Jan 12, 2019, 5:16:37 AM1/12/19
to Open Scriptures
Thank you Michael, and Thank you Peter

That was very helpful,

Just for the record, which Peter forgot to mention, and what I think should give me some credit. I have deleted my work (years of work, months of work and weeks of work), and removed my result set from the public domain, and I have taken many steps back because of Peter's good council on this matter. Further more I am also still trying to fix the problem, so calling me lazy is not the correct observation. Asking for a list, is to avoid assumption but instead have an easy public list not just for me but for everyone who is like Ben asking the same question.

I have not changed in my desire, and that is to eventually create an open, free API. Which work as good as all the paid options out there. So to be angry with me or disappointed is not helping, I want to do this legally, even while I don't agree with it. I understand the trap we are in and I will openly say that I think it is wrong what has happened, the enemy has had a huge victory against the people of God.

I also understand that there are huge cost in translation, but at the same time the Biblical example was always not to burden the people of God, but to fund your own labor when it comes to ministry.

Peter to look at me and think tragic case... is very very close to the truth, I am a tragic case and I also weep over what I see. But in His great love He has given me hope, so if you have given up hope... then at least pray for me the tragic case. But I do in fact do see my responsibility or I would not have deleted my work, and again be here speaking about my responsibility.

So how shall we putt this, my responsibility it to make the Bible freely available to all developers, and therefore to all people. This is the first, and second is to do it in good conscience towards all men, submitting my self unto every ordinance of men as far as it does not conflict with the ordinance of God.

Further more me talking to you on a open forum is again clearly communicating that I want to really do this alongside other who have the same responsibilities.

Peter von Kaehne

unread,
Jan 12, 2019, 5:29:00 AM1/12/19
to openscr...@googlegroups.com
Just to be clear - to create an API - even one which does not look too
clever in the eyes of e.g. CrossWire developers like Troy is not a
problem. Publish all your sword2json python etc routines as much as you
like.

The step which I have objected to - for legal reasons in some cases,
for other reasons (mainly text reliability) in the rest - is to take it
a step further and create text dumps of Bible modules CrossWire
publishes.

If you do the former without the latter, you will not ever fall foul of
the law (unless the law is really dumb) and you will push the
responsibility for the use of the texts firmly into the hands of the
final end users (church websites have been mentioned).

If they then go and obtain our modules and convert them into JSON or
database dumps or whatever else and use that as a base of a website,
then so be it. If they in the process violate copyright - again, so be
it. That is their responsibility then.

Peter

Troy A. Griffitts

unread,
Jan 12, 2019, 11:53:20 AM1/12/19
to openscr...@googlegroups.com
Dear Llewellyn,

I think what you want to do is what Michael at eBible.org has already
begun: A repository of Bible data downloadable by developers.  In
addition, you'd like to add a web service on top of Michael's data
repository which can retrieve chunks of data in real time.

May I suggest that if the real time retrieval is to faciliate churches
embedding Scripture on their website, a quick search shows a couple of
the more popular Bible study web sites facilitate this already:

https://biblia.com/plugins

https://www.biblegateway.com/share/#widget

Also, our open source web tools make this easy; have a look at our web
examples here and note particularly the lookup example:

http://crosswire.org/study/examples/

It would take 5 minutes to parameterize the Bible and Verse range and
you'd have your web service.  No conversions necessary. No parsing OSIS,
ThML, GBF necessary.

The argument, as I see it, for Michael to not add this real time lookup
functionality to his server, is this would dramatically pivot what
Michael does from being a Resource Repository, to being a Compute
Service.  Biblia and Bible Gateway are well funded services.  They pay
for the resources to keep their service up and running and available as
a central service.  If you are successful in getting churches to move
all of their traffic away from Biblia, Bible Gateway, et al., your
successful end is simply paying for enough balancing and redundancy to
keep your service online.  I'm not sure Michael's goal is to be a
Compute Service.  In this scenario if his server goes away, all clients
are disabled.  As he currently is-- a Resource Repository-- clients
serving the Gospel are decentralized; if his server goes away, all
clients have already obtained the resource and continue to share the
Gospel.  I don't think Michael wants to encourage a single point of
failure dependency on his server, which is what a real time retrieval
service would encourage.

Serving together,

Troy

d...@vdm.io

unread,
Jan 12, 2019, 1:21:50 PM1/12/19
to Open Scriptures
I want to fund this, I mean I will setup the server and the maintain its availability. I am working on many projects, and this desire to provide this "real time lookup functionality"  is what I feel I want to fund, develop and maintain. I agree with Peter, that we want to build a system that is fully actively and dynamically connected to the models to stay in perfect step with all revisions as they get pushed out to the master branch. I also think it does not need to be only JSON, but also XML, and all the other options.

I work with Amazon Web Services every day, I have many client servers in my domain of maintenance. So for me to have to run this kind of back-bone "real time lookup functionality" will be completely possible and well funded.

Once again, thank you for your feedback on this Troy! I will take a closer look.

Ben Dwyer

unread,
Jan 17, 2019, 4:59:56 PM1/17/19
to Open Scriptures
I have taken down the generated files for the ESV and the KJV from my pull request. There are other generated files in the repo, but I have no access to them, sorry. If you want them taken down you'll need to contact the original author.


On Friday, 11 January 2019 17:54:02 UTC, Peter von Kaehne wrote:

Peter von Kaehne

unread,
Jun 20, 2019, 3:56:03 AM6/20/19
to openscr...@googlegroups.com
On Thu, 2019-01-17 at 13:59 -0800, Ben Dwyer wrote:
> I have taken down the generated files for the ESV and the KJV from my
> pull request. There are other generated files in the repo, but I have
> no access to them, sorry. If you want them taken down you'll need to
> contact the original author.

They are still up.

https://github.com/danday74/sword-to-json/blob/master/bibles/en/ESV/ESV.json

This has now done real damage.

Crossway has asked us to remove the ESV from CrossWire's servers,
mentioning specifically people who have downloaded from us and
redistribute the text.

Thank you guys

Peter


Pro- wave

unread,
Jan 8, 2020, 3:15:39 AM1/8/20
to openscr...@googlegroups.com
Hello?
Any one with .ewb easy worship Swahili/Kiswahili Bible plugin or anyone who knows how to by-pass the purchase option and have the Bible

--
You received this message because you are subscribed to the Google Groups "Open Scriptures" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openscripture...@googlegroups.com.
To post to this group, send email to openscr...@googlegroups.com.
Visit this group at https://groups.google.com/group/openscriptures.
To view this discussion on the web visit https://groups.google.com/d/msgid/openscriptures/9ea5a0e938777ac155d9bc34ad06e1640b6682a4.camel%40gmx.net.
Reply all
Reply to author
Forward
0 new messages