[Server App Inventor] JSON e XML files for components in Designers and blocks in Blocks

[Lorenzo Vitale]

Hello everyone. I'm trying to understand how app inventor source code works to make changes about project. My difficulty is related to how the App Server Inventor saves a project with all the components on the Designer and the blocks in the block editor. From the documentation found on the internet, it appears that the server saves every two seconds about two files: a json file containing the components and an xml file containing the blocks. 

1. I can not find these files in any way. 

2. From the code it seems that app inventor organizes components and blocks in a node tree (ProjectNode) starting from a root node. These nodes are objects transferred via RPC between server and client and therefore are serialized objects. I thought then that they contained information about components and blocks but the ProjectNode interface returns, as relevant attributes, only id and name. Could someone help me to understand how the server saves the project data and how the client receives them in order to manipulate them?

Thanks so much

[Evan Patton]

Hi Lorenzo,

Regarding 1, every project file (AIA) is a ZIP file with the project content. If you want to see what the files look like, you can rename a .aia file to .zip and extract it to look at the contents, include the Screen's JSON contents and the block XML.

The answer for 2 is a bit more complex. First, there are service definitions in appengine/src/com/google/appinventor/shared/rpc, and you want to look specifically at project/ProjectService.java. On the client, this interface is represented by an asynchronous version called ProjectServiceAsync (accessed via Ode) and on the server side by ProjectServiceImpl. Google Web Toolkit takes care of managing the RPC format and serialization. On the server side, ProjectServiceImpl in turn calls through to CommonProjectService, under appengine/src/com/google/appinventor/server/project. Direct manipulation of the server-side storage is done via ObjectifyStorageIo under appengine/src/com/google/appinventor/server/storage.

If you run App Inventor in the dev server, you can view the contents of the data store by going to http://localhost:8888/_ah/admin. If you deploy to App Engine, you can view them through the Datastore section in the Google Cloud Console.

Regards,

Evan

[Lorenzo Vitale]

Hi Evan thank you for your precious help. 

Meanwhile, I must say that the address http: // localhost: 8888 / _ah / admin was very useful for me to understand the structure of the data.

Going with order, I still have unresolved issues:

1. With regard to point 1, what is still unclear to me is whether the files of the user's projects are automatically created and saved somewhere (locally or on the server?) Or necessarily and only by exporting them manually.

2. With regard to point 2 instead, I browsed through the traceability of the data stored by focusing on the FileData structure. Specifically, I know that for each project there are three files associated with each screen: [screenname_1] .bky, [screenname_1] .scm, [screenname_1] .yail, [screenname_2] .bky, ... .scm, ... .yail , [screenname_n] .scm, etc. etc. all located under a path src / appinventor / ai_test / [username] / [screenname_1] .bky (which however does not exist locally). Intuitively I am led to think that for every screen the .bky file contains the blocks inserted in the block editor and that .scm can contain the components of the designer editor ignoring the contents of the .yail files. If so, then what I ask is: if I want to take this information and manipulate it into some integration of the app inventor that I'm going to create in what way should I do it? Let me explain: let's imagine I want to run time to delete a block from the block editor of a screen without a specific user action but in the background. I imagine that by obtaining the FileData object I can access the set of screens and the elements contained in them in a field that I saw called "content" and be of the blob type. I have to assume that these blobs can be converted into data structures like xml or json type that can be modified and then converted and saved in the original format. I am wrong?

[Evan Patton]

Hi Lorenzo,

Here's a more detailed explanation. When you create a new project or add a screen, the server generates initial .scm and .bky contents. The client retrieves these "files" (resources might be a better term since they are not stored in a file system), and renders them in the editors. Likewise, when you open a project in App Inventor, the project's contents are retrieved from the server. After a change is made, the editor is marked dirty and a timer is set 5 seconds in the future to post the new contents back to the server (see saveDirtyEditors in EditorManager.java). The local state is ephemeral, e.g., if your browser crashes before the scheduled save completes, then the data are lost. 

I think the big challenge with what you want to do is when do you want to make your modifications? If you want to make them while the user has the project open, you will need to manipulate the project state client-side because the client only loads the project from the server when it is first opened. It does not reload on every change, so a server-side manipulation would never appear to the user unless they refresh (not to mention any server-side changes would be overwritten by the autosave feature). If you want to make changes when the user is offline, then you would want to do it server-side by manipulating the FileData objects in the datastore. The contents are just the JSON (designer) and XML (blocks) serializations of the screen contents.

Regards,

Evan

[Lorenzo Vitale]

Hi Evan so if I understand correctly when I do client side changes in the block editor or designer after 5 seconds is the server that takes care of saving. At this point while the project is open the server will never send data to the client.

If the scm and bky blocks contain information about components and blocks, what are the yail files? They're important?

 

I do not know if I'm off topic.

I noticed that in Blockly the events Blockly.Events.CREATE, Blockly.Events.MOVE, etc. capture user actions on the block editor. But I do not know how these events act as a bridge with java. For example, in the action of creating a block I get the resultant xml in the browser console, after which it is unclear how and where that xml is converted into a java and serialized object.

I also can not find the point where the component creation actions are captured in the Designer editor.

So the next step for understanding for me would be to understand how to intercept these actions by working in gwt da java. This is because specifically I would like to be able to add additional information to the datastore that extends the functionality of app inventor useful for my purpose.

[Evan Patton]

Hi Lorenzo,

App Inventor does not generate Java code from the blocks. Rather, it generates code in a language called YAIL, which is Scheme with additional macros to simplify the blocks language. The YAIL files are the generated code that gets compiled when you build an APK.

Blocks changes are propagated from the blocklyeditor (JavaScript) code back to the Java code through the workspaceChanged(JavaScriptObject) in BlocklyPanel.java, and the associated save and update of the companion code occurs in onWorkspaceChange(BlocklyPanel, JavaScriptObject) in YaBlocksEditor.java. For the designer side, saves are scheduled by onFormStructureChange() in YaFormEditor.java, which is triggered from the various onComponent* methods in the same file.

Cheers,

Evan