Justin Baacke

unread,

Aug 22, 2018, 8:57:48 PM8/22/18

to TiddlyWikiDev

Hello everyone,

This is my first time posting so please let me know if there is additional context or conventions I should be providing. I also posted this in the TiddlyWiki group as I didn't know where the line is drawn between the types of questions.

I am trying to retrieve a json object in the format they are imported and exported by TiddlyWiki from AWS S3 and then create a tiddler with the response. I am new to Javascript and am very open to suggestions for other improvements.

My code which successfully retrieves the object and writes the json to the console is as follows:

```

(function(){

"use strict";

exports.name = "getobject";

exports.params = [

{key: ""}

];

exports.run = function(key) {

function handleresult(err, data) {

if(err)throw err;

console.log(data.Body.toString('utf-8'));

// Make a tiddler out of data.Body.toString('utf-8') here

return data

}

AWS.config.update({

accessKeyId: 'MyAccessKey',

secretAccessKey: 'MySecretKey',

region: 'us-east-2'

});

var s3 = new AWS.S3();

var params = {

Bucket: 'testtiddlybucket',

Key: key

}

var result = s3.getObject(params, handleresult);

return "Results from list objects call with prefix " + result

};

})();

```

Is there a Javascript macro I can call to implement the Tiddler creation?

Thanks in advance!

Jeremy Ruston

unread,

Aug 23, 2018, 10:27:35 AM8/23/18

to tiddly...@googlegroups.com

Hi Justin

It may not be exactly what you need, but there is an AWS plugin in the core plugin library that includes commands for use under Node.js to load, save and render tiddlers from files stored on S3. I've pasted the docs below.

(Stuff like this can't be done as a JS macro: they are limited to returning a single string, and are not allowed to modify the tiddler store).

Best wishes

Jeremy

Commands

Perform operation on Amazon Web Services

--aws <sub-command> [<parameter> ...]

"s3-load" subcommand

Load tiddlers from files in an S3 bucket.

--aws s3-load <region> <bucket> <filename>...

region: AWS region
bucket: name of the bucket containing the files
filename: one or more filenames of the files to load

The content in the files is deserialized according to the content type reported by S3.

"s3-savetiddler" subcommand

Save a raw tiddler to a file in an S3 bucket.

--aws s3-savetiddler <title> <region> <bucket> <filename> <zipfilename> <savetype>

title: title of the tiddler to save
region: AWS region
bucket: name of the bucket to save the saved file
filename: filename of the saved file
zipfilename: optional; the file will be packed into a ZIP file with the specified name
savetype: optional; the MIME type for the saved file (defaults to type or "text/html")

"s3-savetiddlers" subcommand

Save raw tiddlers matching a filter to an S3 bucket.

--aws s3-savetiddlers <filter> <region> <bucket> <filenamefilter> <savetypefilter>

filter: filter identifying tiddlers to render
region: AWS region
bucket: name of the bucket to save the files
filenamefilter: filter for converting tiddler titles to filepaths (eg [encodeuricomponent[]addprefix[files/]addsuffix[.html]])
savetypefilter: optional; a filter that is passed the title of the tiddler being saved and should yield the MIME type for the saved file (defaults to a filter that yields the value of the type field)

"s3-rendertiddler" subcommand

Save the results of rendering a tiddler to a file in an S3 bucket.

--aws s3-rendertiddler <title> <region> <bucket> <filename> <type> <template> <zipfilename> <savetype>

title: title of the tiddler to render
region: AWS region
bucket: name of the bucket to save the rendered file
filename: filename of the rendered file
type: optional render type (defaults to "text/html")
template: optional template (defaults to directly rendering the tiddler without using a template)
zipfilename: optional; the file will be packed into a ZIP file with the specified name
savetype: optional; the MIME type for the saved file (defaults to type or "text/html")

"s3-rendertiddlers" subcommand

Save the results of rendering tiddlers identified by a filter to files in an S3 bucket.

--aws s3-rendertiddlers <filter> <template> <region> <bucket> <filenamefilter> <type> <savetypefilter>

filter: filter identifying tiddlers to render
template: template for rendering each tiddler
region: AWS region
bucket: name of the bucket to save the rendered file
filenamefilter: filter for converting tiddler titles to filepaths (eg [encodeuricomponent[]addprefix[files/]addsuffix[.html]])
type: optional render type (defaults to "text/html")
savetypefilter: optional; a filter that is passed the title of the tiddler being rendered and should yield the MIME type for the saved file (defaults to the value of the type field via [is[tiddler]get[type]])

--

Jeremy Ruston

jer...@jermolene.com

https://jermolene.com

--
You received this message because you are subscribed to the Google Groups "TiddlyWikiDev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywikide...@googlegroups.com.
To post to this group, send email to tiddly...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywikidev.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/756be27c-77cb-482b-95b4-9a32d0be3f28%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Justin Baacke

unread,

Aug 23, 2018, 1:07:38 PM8/23/18

to tiddly...@googlegroups.com

Jeremy,

From what I can tell this requires either my wiki to be hosted in a node.js lambda or to be running from a computer with with configured credentials. I can't seem to find where to set the credentials for a browser based session inside the aws plugin code. (I am also having trouble actually running the aws commands from tiddlywiki desktop but I think that is more easily attributed to user error). I don't feel I understand how the lambda hosting works well enough to set it up that way, I am looking for a tutorial but haven't had much luck.

As another possible structure is it possible to create a tiddler based on json returned from a REST API? That would be fairly easy to set up in front of the bucket and I could build some logic into the proxy lambda.

I guess the core of my question is what would be the best way to generate tiddlers based off results of a query of an external service the user can trigger with input parameters. Is there a better way I can be thinking about this?

Also this is an example of what my calls are currently returning:

[

{

"created": "20180706194034282",

"text": "My content ",

"title": "Sample Tiddler to Save",

"tags": "atag [[a spaced tag]]",

"modified": "20180706194122357",

"type": "text/html",

"afieldname": "Some Field Content",

"tmap.id": "95ca0589-0bf1-4442-80e0-d07db2f26036"

}

]

Thanks,

Justin

Jeremy Ruston

unread,

Aug 23, 2018, 1:23:12 PM8/23/18

to TiddlyWikiDev

Hi Justin

From what I can tell this requires either my wiki to be hosted in a node.js lambda or to be running from a computer with with configured credentials. I can't seem to find where to set the credentials for a browser based session inside the aws plugin code. (I am also having trouble actually running the aws commands from tiddlywiki desktop but I think that is more easily attributed to user error). I don't feel I understand how the lambda hosting works well enough to set it up that way, I am looking for a tutorial but haven't had much luck.

That’s right, the plugin isn’t currently designed for use in the browser.

As another possible structure is it possible to create a tiddler based on json returned from a REST API? That would be fairly easy to set up in front of the bucket and I could build some logic into the proxy lambda.

I guess the core of my question is what would be the best way to generate tiddlers based off results of a query of an external service the user can trigger with input parameters. Is there a better way I can be thinking about this?

That makes sense: importing data from JSON apis is something that should really be in the core. For the moment, you’d have to write a startup module that responds to a widget message “tm-call-my-api”. The widget message can take whatever parameters you need (e.g. credentials, URL, etc.) and must also take the title of the tiddler to which the data should be saved (unless that’s going to be derived from the incoming data).

A simple example of a startup module that listens for a widget message would be:

https://github.com/Jermolene/TiddlyWiki5/blob/master/plugins/tiddlywiki/text-slicer/modules/startup/slicer-startup.js

Let me know how you get on,

Best wishes

Jeremy.

Thanks,

Justin

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywikidev/6a62ee63-f5ec-4d26-b824-6e19f6c6771d%40googlegroups.com.

Justin Baacke

unread,

Aug 23, 2018, 8:17:26 PM8/23/18

to tiddly...@googlegroups.com

Jeremy,

I did it! But not quite in the way you described and I don't entirely understand what it is I have done. My current "handleresult" function from above is the only thing that has changed:

function handleresult(err, data) {
  if(err)throw err;
  console.log(data.Body.toString('utf-8'

));
  var tiddlersjson = JSON.parse(data.Body.toString('utf-8'));
  console.log(tiddlersjson[0]);
  //$tw.wiki.addTiddlers([new $tw.Tiddler($tw.wiki.getModificationFields(), {"title": "Tiddler1","text": "Hello"})])
  $tw.wiki.addTiddlers([new $tw.Tiddler($tw.wiki.getModificationFields(), tiddlersjson[0])]);
  return data;
}

I was going through the page you linked to when I saw the $tw.wiki.addTiddlers function which was perfect. I was able to find a few other examples of people using it so I twisted until I got it right with the format being passed back. I couldn't find any documentation on it though so I am wondering what the first argument does. Is there documentation on this function and others like it I could go through?

On a related note, what does the $tw.wiki.getModificationFields() function do? Is that just an arguement stating what is about to come is jsonified fields?

I am going to do some code cleanup and try to account for any number of tiddlers being in the object I get back, but if you have any other suggestions for improvement they would be very welcome!

How to create a tiddler from json inside javascript macro

Justin Baacke

Jeremy Ruston

Commands

"s3-load" subcommand

"s3-savetiddler" subcommand

"s3-savetiddlers" subcommand

"s3-rendertiddler" subcommand

"s3-rendertiddlers" subcommand

Justin Baacke

Jeremy Ruston

Justin Baacke