Problems importing JSON file

[Robert Jopling]

Hello

TW5. I am trying to import a JSON file which contains the following so as to create two new tiddlers. When I drag and drop the file on to the green bar in either Firefox, Chrome or TiddlyDesktop I get zero tiddlers.

Does anyone know why?

[
    {
        "condition": "Destroyed",
        "current_use": "none",
        "date visited": null,
        "fb_number": "",
        "historic_use": "3rd order",
        "name": "Pennyvenie Colliery",
        "osgb_gridref": "NS 48772 06904",
        "osgb_gridsq": "NS 48 06",
        "physical_type": "Bolt",
        "sq_easting": 48,
        "sq_letters": "NS",
        "sq_northing": 6,
        "visited": null,
        "waypoint": "TP15832"
    },
    {
        "condition": "Good",
        "current_use": "none",
        "date visited": null,
        "fb_number": "",
        "historic_use": "3rd order",
        "name": "Dalmellington Church Tower Centre",
        "osgb_gridref": "NS 48039 06085",
        "osgb_gridsq": "NS 48 06",
        "physical_type": "Intersected Station",
        "sq_easting": 48,
        "sq_letters": "NS",
        "sq_northing": 6,
        "visited": null,
        "waypoint": "TP15840"
    }
]

Cheers

Rob

[Robert Jopling]

I am a newbie to TW5 and JSON so please can you explain how I achieve that. The JSON file was an export from a database app.

On Sun, 21 Jun 2020, 23:00 Saq Imtiaz, <saq.i...@gmail.com> wrote:

Each tiddler needs a unique title attribute.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/18c7a284-5ab2-442c-a48e-f46fea2b2112o%40googlegroups.com.

[john.edw...@gmail.com]

Lacking a "title" per entry. Also I don't think "date visited" works for a field but "date_visited" will.

[Eric Shulman]

On Sunday, June 21, 2020 at 3:27:20 PM UTC-7, Robert Jopling wrote:

I am a newbie to TW5 and JSON so please can you explain how I achieve that. The JSON file was an export from a database app.

To *export* tiddlers from TiddlyWiki using JSON format go to $:/AdvancedSearch -> Filters tab, and enter a list of tiddlers, then press the "export" button and choose JSON format.

The result is a file with contents like this:

[
   {
     "created":"20200621234617853",
      "text":"",
      "tags":"",
      "title":"TiddlerOne",
      "modified":"20200621234625900"
   },
   {
      "created":"20200621234629958",
      "text":"",
      "tags":"",
      "title":"TiddlerTwo",
      "modified":"20200621234637103"
   },
   {
      "created":"20200621234639688",
      "text":"",
      "tags":"",
      "title":"Tiddler Three with spaces",
      "modified":"20200621234652570"
   }
]

You can then take that file and drag-and-drop it into another TiddlyWiki and you will be prompted to import all three tiddlers.

Note that you don't actually need *all* those fields in the JSON file.  You can actually get away with this minimal JSON:

[
   {
      "title":"TiddlerOne"
   },
   {
      "title":"TiddlerTwo"
   },
   {
      "title":"Tiddler Three with spaces"
   }
]

However... while this minimal JSON can be imported into TiddlyWiki, it lack a few fields that make things work better.  Specifically, because the tiddlers in this JSON file don't have any "created" or "modified" fields defined, they won't initially show up in the Recent tab in the TW sidebar (they can still be found by using the "More > All" tab in the sidebar).

To make things work well right from the start, here's what I consider to be an effective minimal JSON file:

[
   {
      "created":"20200621234617853",
      "title":"TiddlerOne",
      "modified":"20200621234625900"
   },
   {
      "created":"20200621234629958",
      "title":"TiddlerTwo",
      "modified":"20200621234637103"
   },
   {
      "created":"20200621234639688",
      "title":"Tiddler Three with spaces",
      "modified":"20200621234652570"
   }
]

Note that each tiddler has both a "created" and "modified" field defined.  The timestamp format for these fields is:

YYYY0MM0DD0hh0mm0ss0XXX

That is:

4 digit year

2 digit month

2 digit day

2 digit hours

2 digit minutes

2 digit seconds

3 digit millisecond

Note that all values (except for the year) are zero-padded... i.e., July 4th 2020 at 12:15:30pm would be: "20200704121530000"

For your purposes, you could just use midnight on the current day, i.e. "20200621000000000".

Also, as already noted by John Edwards, field names cannot have spaces, but can have underscores instead, so you will have to fix that as well.

Hopefully, the above info is enough to get your data successfully imported as tiddlers.

Let me know how it goes...

enjoy,

-e

Eric Shulman

TiddlyTools.com: "Small Tools for Big Ideas!" (tm)

InsideTiddlyWiki: http://TiddlyTools.com/InsideTW

[Robert Jopling]

Hi Eric

Thank you for your response which was a great help. The only problem now is that when I try a test JSON structured as suggested I get the following error message

JSON error: SyntaxError: JSON.parse: unexpected character at line 8 column 1 of the JSON data

That character is the closing square bracket. 

This is the content of the JSON file

[
{
"created": "2.02E+16","text": "55.2521","Tags": "","Title": "Ailsa Craig Lighthouse","modified": "2.02E+16"
},
{
"created": "2.02E+16","text": "56.72716667","Tags": "","Title": "Ardnamurchan Lighthouse","modified": "2.02E+16"
},
]

What i actually want to do is to insert the data columns in my csv file into fields in each tiddler. I set up a tiddler in the format that I want and exported it to see the structure and then created the following file JSON2 with the additional fields after the 'modified' field.

[
{"created": "2.02E+16","text": "","Tags": "","Title": "Ailsa Craig Lighthouse","": "","modified": "2.02E+16","Name": "Ailsa Craig Lighthouse","Area": "Ayrshire","Lat": "55.2521","Long": "5.108716667","Island_mainland": "Ailsa Craig","Year_built": "1886","Built_by": "Thomas & David A Stevenson","Operated_by": "NLB","Tower_height": "11m","Focal_height": "18m","Range": "17 nm"},
{"created": "2.02E+16","text": "","Tags": "","Title": "Ardnamurchan Lighthouse","": "","modified": "2.02E+16","Name": "Ardnamurchan Lighthouse","Area": "Highland","Lat": "56.72716667","Long": "6.226116667","Island_mainland": "mainland","Year_built": "1849","Built_by": "Alan Stevenson","Operated_by": "NLB","Tower_height": "35m","Focal_height": "55m","Range": "24 nm"},
{"created": "2.02E+16","text": "","Tags": "","Title": "Auskerry","": "","modified": "2.02E+16","Name": "Auskerry","Area": "Orkney Islands","Lat": "59.02595","Long": "2.572783333","Island_mainland": "Auskerry","Year_built": "1867","Built_by": "David & Thomas Stevenson","Operated_by": "NLB","Tower_height": "34m","Focal_height": "34m","Range": "18 nm"}
]

On import nothing happens!

Any explanation?

Rob

On Mon, 22 Jun 2020 at 01:12, Eric Shulman <elsd...@gmail.com> wrote:

On Sunday, June 21, 2020 at 3:27:20 PM UTC-7, Robert Jopling wrote:

I am a newbie to TW5 and JSON so please can you explain how I achieve that. The JSON file was an export from a database app.

To *export* tiddlers from TiddlyWiki using JSON format (go to $:/AdvancedSearch -> Filters tab, and enter a list of tiddlers, then press the "export" button and choose JSON format)

However... while this minimal JSON can be imported into TiddlyWiki, it lack a few fields that make things work better.  Specifically, because the tiddler in this JSON file don't have any "created" or "modified" fields defined, they won't initially show up in the Recent tab in the TW sidebar (they can still be found by using the "More > All" tab in the sidebar).

To make things work well right from the start, here's what I consider to be an effective minimal JSON file:

[
   {
      "created":"20200621234617853",
      "title":"TiddlerOne",
      "modified":"20200621234625900"
   },
   {
      "created":"20200621234629958",
      "title":"TiddlerTwo",
      "modified":"20200621234637103"
   },
   {
      "created":"20200621234639688",
      "title":"Tiddler Three with spaces",
      "modified":"20200621234652570"
   }
]

Note that each tiddler has both a "created" and "modified" field defined.  The timestamp format for these fields is:

YYYY0MM0DD0hh0mm0ss0XXX

That is:

4 digit year

2 digit month

2 digit day

2 digit hours

2 digit minutes

2 digit seconds

3 digit millisecond

Note that all values (except for the year) are zero-padded... i.e., July 4th 2020 at 12:15:30pm would be: "20200704121430000"

For your purposes, you could just use midnight on the current day, i.e. "20200621000000000".

Also, as already noted by John Edwards, field names cannot have spaces, but can have underscores instead, so you will have to fix that as well.

Hopefully, the above info is enough to get your data successfully imported as tiddlers.

Let me know how it goes...

enjoy,

-e

Eric Shulman

TiddlyTools.com: "Small Tools for Big Ideas!" (tm)

InsideTiddlyWiki: http://TiddlyTools.com/InsideTW

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/a766e2f7-d65f-44ae-bb4b-d645d13baa49o%40googlegroups.com.

[Eric Shulman]

On Wednesday, June 24, 2020 at 7:14:13 AM UTC-7, Robert Jopling wrote:

When I try a test JSON structured as suggested I get the following error message

JSON error: SyntaxError: JSON.parse: unexpected character at line 8 column 1 of the JSON data

That character is the closing square bracket. 

JSON parsing can be somewhat finicky if the input is not 100% well-formed.  The problem here is that you have a trailing comma on the last entry!

I set up a tiddler in the format that I want and exported it to see the structure and then created the following file JSON2 with the additional fields after the 'modified' field.

On import nothing happens!
Any explanation?

To help me see what might be happening, I took the first entry in your JSON2 file, and split it on the commas to show one field per line... here's the result:

 

[{

"created": "2.02E+16",
"text": "",
"Tags": "",
"Title": "Ailsa Craig Lighthouse",
"": "",
"modified": "2.02E+16",
"Name": "Ailsa Craig Lighthouse",
"Area": "Ayrshire",
"Lat": "55.2521",
"Long": "5.108716667",
"Island_mainland": "Ailsa Craig",
"Year_built": "1886",
"Built_by": "Thomas & David A Stevenson",
"Operated_by": "NLB",
"Tower_height": "11m",
"Focal_height": "18m",
"Range": "17 nm"
},

I think I see three separate problems with the above JSON input:

1) All fieldnames in TW must be *lowercase* only.  I did a quick test, and TW ignores any JSON fieldnames that are not lowercase. Thus, because "Title" is capitalized, none of your entries actually have a valid title field, so they are not imported.

2) In between the "Title" and "modified" fields, there is an entry with *no name* (and no value).  This doesn't prevent the entries from being imported (once the capitalization problem is fixed), but in the resulting imported tiddlers, there is a *field with no name*, which appears when you edit any of those tiddlers, but which then cannot be deleted using the trashcan icon next to the field in the TW edit view.

3) The "created" and "modified" field values seem to have been converted into scientific notation (i.e., "2.02E+16").  For these to be used as TW field values, they need to be the full 17-digit numbers (e.g., "20200624000000000").  Note that if you can't get your application to output the full 17-digit values without converting them to scientific notation, you *can* just omit those fields entirely.  However, when imported, none of your tiddlers will appear in the sidebar "Recent" list, though they will still appear in the list shown under "More>All".  When you then edit/save each of those tiddlers, a proper "modified" field will be automatically added, but the "created" field will still be missing.

So... give it another try, where your JSON file has:

1) no capitalized fieldnames

2) no blank fieldnames

3) 17-digit date/time stamps for "created" and "modified"

Again, let me know how it goes...

-e

[Robert Jopling]

Thanks so much for your help Eric.

I will do as you suggested and let you know what happens.

Rob

On Mon, 22 Jun 2020, 01:12 Eric Shulman, <elsd...@gmail.com> wrote:

On Sunday, June 21, 2020 at 3:27:20 PM UTC-7, Robert Jopling wrote:

I am a newbie to TW5 and JSON so please can you explain how I achieve that. The JSON file was an export from a database app.

To *export* tiddlers from TiddlyWiki using JSON format (go to $:/AdvancedSearch -> Filters tab, and enter a list of tiddlers, then press the "export" button and choose JSON format)

However... while this minimal JSON can be imported into TiddlyWiki, it lack a few fields that make things work better.  Specifically, because the tiddler in this JSON file don't have any "created" or "modified" fields defined, they won't initially show up in the Recent tab in the TW sidebar (they can still be found by using the "More > All" tab in the sidebar).

To make things work well right from the start, here's what I consider to be an effective minimal JSON file:

[
   {
      "created":"20200621234617853",
      "title":"TiddlerOne",
      "modified":"20200621234625900"
   },
   {
      "created":"20200621234629958",
      "title":"TiddlerTwo",
      "modified":"20200621234637103"
   },
   {
      "created":"20200621234639688",
      "title":"Tiddler Three with spaces",
      "modified":"20200621234652570"
   }
]

Note that each tiddler has both a "created" and "modified" field defined.  The timestamp format for these fields is:

YYYY0MM0DD0hh0mm0ss0XXX

That is:

4 digit year

2 digit month

2 digit day

2 digit hours

2 digit minutes

2 digit seconds

3 digit millisecond

Note that all values (except for the year) are zero-padded... i.e., July 4th 2020 at 12:15:30pm would be: "20200704121430000"

For your purposes, you could just use midnight on the current day, i.e. "20200621000000000".

Also, as already noted by John Edwards, field names cannot have spaces, but can have underscores instead, so you will have to fix that as well.

Hopefully, the above info is enough to get your data successfully imported as tiddlers.

Let me know how it goes...

enjoy,

-e

Eric Shulman

TiddlyTools.com: "Small Tools for Big Ideas!" (tm)

InsideTiddlyWiki: http://TiddlyTools.com/InsideTW

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/a766e2f7-d65f-44ae-bb4b-d645d13baa49o%40googlegroups.com.

[Robert Jopling]

Hi Eric

I did as you suggested and it works. Thanks.

I have even managed to display editable fields in the tiddler!

Rob