Work in progress: Moving blocks to improved JSON

[Andrew n marshall]

Hey everyone,

I just want to let you know about some work that a few of you have already noticed on the develop branch.

TLDR:

The blocks found in blocks/ of the web project (and the playground) are being rewritten into an extended JSON syntax. Blocks using the new definitions are fully backward compatible, but if you're using modifications of those blocks and want to continue tracking changes, this will be an important update. Blocks defined with the prior JSON notation will also continue to work, but if you're interested in building Blockly apps for multiple platforms, this new features will also be relevant.

Details:

A little over a year ago we introduce the JSON syntax for blocks. This gave us a block definition syntax that we could use on Android and iOS. However, our usage in web Blockly wasn't strictly JSON, in that we used a number of constants. Additionally, anything that required a function, like a mutator or a dynamically generated element, could not be represented in that syntax.

Over the last couple of weeks I've been working on these problems. We can now reference message in the Blockly.Msg string table. For instance, "%{BKY_MSGNAME}" will be replaced with Blockly.Msg['MSGNAME']. We are using the BKY_ prefix to allow future compatibility with additional string tables, such as app-specific messages (though much needs to be done to complete this, and this is much lower priority).

Message references are usable in block messages ('message0', etc), field labels, dropdown labels, tooltips, help URLs, and eventually values. I also use it in the block's 'colour' attribute, replacing the former JavaScript constant.

I've also added extensions, functions that are run as the last step of block initialization. Each extension has a name, and the 'extensions' JSON attribute is list of extensions applied in order. Each extensions will require some platform specific code, but at least the name and the rest of the JSON block structure can be shared across platforms. This make it easy to reuse our extensions, and write your own extensions for your own blocks.

Combined, these changes should be enough to support all the existing blocks on the playground.

Support for these new JSON features has not yet begun on Android or iOS, but it will be a priority following the block rewrite process.

You can explore the first few blocks ported to the new format in develop's blocks/colours.js and blocks/math.json.  Please let me know what you think of the new syntax.

--
Andrew

[Murray Lang]

Thanks for your work on this Andrew.
Your timing is impeccable!
Murray

[Murray Lang]

Hi Andrew,

Regarding app-specific messages, I am looking at reproducing msg/messages.js in demos/<myapp>/msg/, then modifying build.py to pull in those messages by calling _rebuild() again in run().
How far removed is this approach to what you had in mind?
Does it ring any alarm bells for you?

Regards,

Murray

On Wednesday, January 25, 2017 at 5:12:24 AM UTC+8, Andrew n marshall wrote:

[Murray Lang]

Andrew,

I am coming up against a problem with tooltips for math blocks with the new  JSON message placeholder scheme.

The browser reports a number of errors like...WARNING: No message string for %{BKY_MATH_TRIG_TOOLTIP_ACOS}....for all of the messages defined in

Blockly.Blocks.math.TOOLTIPS_BY_OP_.

A breakpoint in Blockly.Extensions.buildTooltipForDropdown() revealed that Blockly.MSG has no messages defined at the time that it is called. It appears that the "math_op_tooltip"
extension is being called too soon in the scheme of things.

Do you have this problem?

Regards,
Murray

[Tim Dawborn]

Hi Murry,

My 2c, but at a guess, you're including the messages file(s) into your HTML your block definitions file. The messages file(s) need to come before the blocks when you're using the JSON format, as they're utilised at file execution file instead of at object-construction time as they were before when blocks were classes.

Cheers,

Tim

--
You received this message because you are subscribed to the Google Groups "Blockly" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blockly+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tim Dawborn]

On 26 January 2017 at 22:22, Tim Dawborn <tim.d...@gmail.com> wrote:

My 2c, but at a guess, you're including the messages file(s) into your HTML your block definitions file.

Whoops, I accidentally a word there. That was meant to say "... into your HTML after your block definitions file(s)".

[Murray Lang]

Hi Tim,

I don't have a general problem with message substitution into the JSON block definitions. They all work as expected except for the ones referenced from within the "math_op_tooltip" extension. Putting a breakpoint in that extension then stepping through it, I can see that there are no messages whatsoever in the Blockly.Msg object. It would appear that extensions are called before messages are loaded.

There is nothing that I can have changed because it's all in the blockly_compressed.js and blocks_compressed.js files as cloned from the develop branch. The same error occurs in the Code demo. I can also see that none of the demos in the develop branch have been modified recently, so there is no evidence of a need for new inclusions in index.html or a change in their order.

Murray

To unsubscribe from this group and stop receiving emails from it, send an email to blockly+u...@googlegroups.com.

[Andrew n marshall]

I haven't yet pulled up the code this morning, but this does strike me as odd. All my tests, and I manually checked the tooltips in the case of dynamic tooltips. The lookup into Blockly.Msg table should occur at Block initialization time, same as before, due to the tooltip table using the same reference notation.

I will investigate.

--

You received this message because you are subscribed to the Google Groups "Blockly" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blockly+unsubscribe@googlegroups.com.

[Murray Lang]

Thanks Andrew,

If you can do a fresh clone of the develop branch and run the Code demo without this error then the problem is clearly mine.

By the way, I open Blockly as file://. Don't know if that's relevant.

I'm away from home for a day or so now, and will not be able to further investigate my end until I return. I'm thinking I should do a fresh clone myself and try again.

Regards,
Murray

[Andrew n marshall]

Fresh clone works for me.  Tested both master (which shouldn't have a change) and develop. Code demo page and block therein load fine. If I change the operator on math_arithmetic, the tooltip changes appropriately.  Chrome on Mac, but I don't think that should matter in this case.

--
Andrew

[Andrew n marshall]

Wait..  After a rebuild, I'm seeing this error.  I will investigate.

--
Andrew

[Andrew n marshall]

Okay. This is a false negative warning.  The extension is registered before the messages are loaded, and the system attempts to be proactive in warning about the missing messages. However, by the time you create a block, the messages are loaded and this isn't an issue. All of my above tests work.

My intention was to ensure developers are warned about such issues as early as possible, even if they never opened the toolbox or otherwise created the one of the affected blocks. I felt this would help avoid future errors, but obviously I'm checking the stuff too early for Blockly pages that load blocks via <script> tags.

I'll see if I can push this validation into onload.

--
Andrew

[Murray Lang]

My my math blocks aren't showing. Only the first literal math block.
You see all of your math blocks?

[Murray Lang]

Sorry, you said so.
OK, I have another problem as well then. I'll get back to you on this in a couple of days.
Cheers,
Murray

[Andrew n marshall]

I fixed the validation and incorrect warning.  Try resyncing to develop and let me know.

--
Andrew

[Murray Lang]

Thanks Andrew,

After pulling your changes, my missing math blocks reappeared.
Maybe that was a browser version issue(?) Anyway, onwards and upwards.

Cheers,
Murray

On Friday, January 27, 2017 at 9:42:27 AM UTC+8, Andrew n marshall wrote:

I fixed the validation and incorrect warning.  Try resyncing to develop and let me know.

--
Andrew

On Thu, Jan 26, 2017 at 2:22 PM, Murray Lang <murray...@gmail.com> wrote:

Sorry, you said so.
OK, I have another problem as well then. I'll get back to you on this in a couple of days.
Cheers,
Murray

--
You received this message because you are subscribed to the Google Groups "Blockly" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blockly+u...@googlegroups.com.

[Tim Dawborn]

Hi Andrew,

With this new format, there's no way currently to define a dropdown field whose options are defined by a function (currently done by passing a function to the FieldDropdown constructor). This can't currently be done with an extension as there's no public API for changing an existing FieldDropdown instance's menuGenerator_. The other option would be for the extension to set a field on a Block by name, but there is no API for this either.

I'm happy to implement either (probably the first one) and submit a PR. Do you have a preference as to which, or do you intend to support this functionality in a different way?

Cheers,

Tim

[Burak Bilgehan]

Hello Andrew,

So far we had hard times defining blocks with mutators. Most of those blocks with mutators were erroneous. Wİll this update offer some good things about the mutators? 

[Andrew n marshall]

Hey Tim,

You can do this by creating the FieldDropdown object in the extension, and then adding it to the block by calling .appendField(..) on some input. By creating it at that time, the extension can pass in the generator as a constructor option.

field_dropdown args have some additional processing. The static message strings are processed to extract prefix and suffix label fields (see docs), a feature not available on generated dropdown options. Having the ability to add an option generator to an existing dropdown after it has gone through this prefix/suffix extraction is probably a recipe for unexpected results.

Given that, I think the current status is sufficient. It may feel a little odd to write the blocks that use such extensions, but it is certainly possible.  It is probably worth a demo and some additional docs, though.

--
Andrew

--

You received this message because you are subscribed to the Google Groups "Blockly" group.

To unsubscribe from this group and stop receiving emails from it, send an email to blockly+unsubscribe@googlegroups.com.

[Tim Dawborn]

On 30 January 2017 at 14:35, Andrew n marshall <a...@anm.me> wrote:

You can do this by creating the FieldDropdown object in the extension, and then adding it to the block by calling .appendField(..) on some input. By creating it at that time, the extension can pass in the generator as a constructor option.

field_dropdown args have some additional processing. The static message strings are processed to extract prefix and suffix label fields (see docs), a feature not available on generated dropdown options. Having the ability to add an option generator to an existing dropdown after it has gone through this prefix/suffix extraction is probably a recipe for unexpected results.

Given that, I think the current status is sufficient. It may feel a little odd to write the blocks that use such extensions, but it is certainly possible.  It is probably worth a demo and some additional docs, though.

Right. So, to clarify, the preferred way to do this is to have a translation string with %N's in it, and for each %N that you want to be a dropdown whose menuGenerator_ is a function or an array that's derived at runtime, use a named input_dummy in the argsN slot for the field, and in the extension for the block, get the dummy input by name and append to it the manually constructed FieldDropdown instance?

That seems reasonable, albeit a little counterintuitive/not obvious that that's the best way to do things. An example definitely would not go astray :)

[Andrew n marshall]

Add issue #889 to track the creation of a dropdown demo and update of documentation (which needs to occur for all these with these new JSON extensions).

--
Andrew

[Andrew n marshall]

Hi Burak,

Mutators remain advance block functionality, and these JSON changes do not make them any easier.

While the details of how mutator methods are applied changes, the mutator methods themselves and the capabilities of mutated blocks have not changed. There are a couple of examples of how we are suggesting developers do this using goog.mixin() in the math.js, and with that example it should be relatively straight forward to port a working mutator.

--
Andrew

On Sun, Jan 29, 2017 at 9:56 AM, Burak Bilgehan <burakb...@gmail.com> wrote:

Hello Andrew,

So far we had hard times defining blocks with mutators. Most of those blocks with mutators were erroneous. Wİll this update offer some good things about the mutators? 

--

[Andrew Stratton]

Before the JSON format became more known/available, I started out on a similar road myself. I'm not sure if some of my concepts would be useful - but I did include ways to handle generated text (in my case - always javascript) and menus generated by javascript as well. e.g.

        quando_editor.defineAction({
            name: 'Show "', title: 'Show Text',
            interface: [{ name: ID_GREETING, title: '"', text: '.type your text here..' }, { title: '"' }],
            javascript: function (block) {
                return 'quando.text("' + quando_editor.getText(block, ID_GREETING) + '");\n';
            }
        });

  And a more complex example:

        quando_editor.defineBlock({
            // TODO must be in a vitrine...?
            name: LABEL_TO_BLOCK, title: 'Label', category: 'dig', colour: self.CONFIG.RULE_COLOUR,
            interface: [
                {
                    name: LABEL_TO_MENU,
                    menu: _label_menu
                },
            ],
            javascript: _label_javascript,
        });

This relies on:

        let _label_javascript = function (block) {
            let menuid = quando_editor.getMenu(block, LABEL_TO_MENU);
            // find when block on id, then get it's title
            let whenblock = Blockly.mainWorkspace.getBlockById(menuid);
            let title = quando_editor.getText(whenblock, WHEN_VITRINE_TEXT);
            var result = `quando.addLabel("${menuid}", "${title}");\n`;
            return result;
        }

and:

        let _label_menu = function () {
            let topBlocks = Blockly.mainWorkspace.getAllBlocks();
            let choices = [['-----', 0]];
            for (var block of topBlocks) {
                if (block.type == PREFIX + WHEN_VITRINE_BLOCK) {
                    let text = quando_editor.getText(block, 'title');
                    choices.push([text, block.id]);
                }
            }
            return choices;
        }

I then have an injector, that iterates through the JSON and injects the Blocks, with generators and menu functions, into Blockly - but I guess you would do things differently...

I know that the Blockly JSON has gone a different route - but I thought this might be useful (but I have been know to be wrong in the past)

Best wishes

  Andy