Large Blockly project with more than 7K blocks

[jimmy joseph]

I have been experimenting with Blockly and found that randomly adding blocks more than 5k or so significantly slows down and freezes the UI. I totally understand that, in reality we might not need more than 5K blocks in a single blockly project file. Nevertheless, I would love to hear from others here - are there any techniques to prevent the UI from freezing after adding large number of blocks?  How can we handle and deal with large blockly project files (let's keep splitting into multiple files away for time being). 

[Neil Fraser]

Can you recreate steps using the playground?

  https://blockly-demo.appspot.com/static/tests/playground.html

Performance is important to us, but we aren't seeing UI freezes at 5k blocks.

There are currently too many unknowns around this issue.  Maybe they are using Angular.  Maybe they have 640kb of memory.  Let's get a repeatable test case nailed down.

Am Fr., 18. Nov. 2022 um 15:49 Uhr schrieb jimmy joseph <jim...@gmail.com>:

I have been experimenting with Blockly and found that randomly adding blocks more than 5k or so significantly slows down and freezes the UI. I totally understand that, in reality we might not need more than 5K blocks in a single blockly project file. Nevertheless, I would love to hear from others here - are there any techniques to prevent the UI from freezing after adding large number of blocks?  How can we handle and deal with large blockly project files (let's keep splitting into multiple files away for time being). 

--
You received this message because you are subscribed to the Google Groups "Blockly" group.
To unsubscribe from this group and stop receiving emails from it, send an email to blockly+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/blockly/949542cb-fece-48d7-801f-c4286efb9b7an%40googlegroups.com.

--

Neil Fraser, Switzerland
https://neil.fraser.name

[jimmy]

Here is the blocks xml file saved from playground.

Thanks
_____________________
Jimmy

To view this discussion on the web visit https://groups.google.com/d/msgid/blockly/CAEx9HWGt_3ejDui%3DTyXcsX9%3DvKgKwyaSMLv5%3DoaEQ_Yu5VVmUg%40mail.gmail.com.

[Neil Fraser]

Thanks for that.  First finding is that it's lightning quick in Firefox.  No issues there.  Likewise, Safari is smooth.

Google Chrome on the other hand is problematic.  First it's very choppy (but usable).  An examination of the console shows that most of the CPU is going to 'rendering', not Blockly's JavaScript.  But leave the page alone for a few minutes, and it becomes utterly unusable.  So in addition to everything else, there's a major memory leak happening.  This is going to be fun.

To view this discussion on the web visit https://groups.google.com/d/msgid/blockly/CANsYDt_-43jsbdHmrFq%3DpBFrLU6wBHQ8zO%3D8%2BZ_K4SW6Mf5cgg%40mail.gmail.com.

[jimmy]

Thanks a ton Neil for responding.

Yes, we primarily use Chrome or embed blockly via MS Edge webview in our windows app. Everything works perfectly - except it cannot handle blocks  > 5K. Yes it can; but the experience is not so pleasant unfortunately. 

Thanks
_____________________
Jimmy

To view this discussion on the web visit https://groups.google.com/d/msgid/blockly/CAEx9HWFFsbxDx-yJQQxHZhKx1K4nq%2BfdnGLHMBv5dZ4bsqEQbA%40mail.gmail.com.

[Ewald Ramp]

We had some very large blockly configurations that caused us to run into a similar problem (in a ReactJs context), while I have no idea how many blocks in total were in there, we have one configuration, when exported to JSON, that was over 60MB.

I can confirm that the rendering is the problem. Loading that many blocks in a virtual workspace has little to no impact.

What we ended up doing was keep 2 workspaces around, one rendered, one virtual. 

The virtual workspace would have the complete document.

On the rendered workspace, we added a change listener, whenever a block is collapsed, we remove all the children. When it is expanded again, we recreate the child blocks in from the non-rendered workspace. As long as you don't need to keep many blocks open at the same time, it works.  But you'll have to add a lot of extra logic to replicate normal behavior. Copying blocks for instance needs to copy the virtual copy of the block, not the one the user actually had selected. As otherwise you'd loose the collapsed children.

Obviously, it still is a problem when you need a lot of blocks open at the same time, but if not, it helps it be manageable.

Note, recently we have refactored our use of blockly, allowing us to break up the large blockly configurations in much smaller sets, which obviously mitigates the problem.

Hope that helps?

Ewald

[jimmy]

Thanks Ewald - yes splitting one program into multiple chunks definitely works. Nevertheless, it would be an awesome feature, if blocky could handle large set of blocks. 

To view this discussion on the web visit https://groups.google.com/d/msgid/blockly/6aa3154e-ff7f-4a6e-90b1-a8a7db6f4efcn%40googlegroups.com.

[Neil Fraser]

The team here at Google are dying to know what your use cases are.  If it's private, feel free to email us off the list.

To view this discussion on the web visit https://groups.google.com/d/msgid/blockly/CANsYDt_-2YDq4-j-5iyBiydfsks2Xa2MGKZGpTH9_WqC15phhg%40mail.gmail.com.

[jimmy]

Hi Neil, 

We use blocky to code pure python  for a rule engine. At present there are no python code that are larger than 1k blocks. And we could split large source file to multiple chunks - this issue is not a show stopper. 

We love blocky and wanted to see that it could handle large number of blocks. :-)

To view this discussion on the web visit https://groups.google.com/d/msgid/blockly/CAEx9HWF2ou0uuDota9h-ZBjpsGQ_t9ZokcfpD8UWSku9KQSkhA%40mail.gmail.com.

[Christopher Allen]

I have filed #6654 to track this issue.

[Ewald Ramp]

Hi Neil,

I think David Welling, a colleague of mine, might have shared our use case in the past, though our usage definitely has grown since. 

We use Blockly probably well outside its original envisioned use.

We use it somewhat like a low/no code style, where we allow non-devs to define user interfaces, with various features like access permissions, data sourcing and viewport adaptation.

For this, we defined about 120 or so custom blocks, and we use a customized JavaScript generator to generate a JSON "metadata" file.

That metadata then is used to render a UI dynamically (in ReactJS on the web, and React Native on devices), it drives api access and payload validation, and even influences business logic/workflows run on the data.

All of this runs as a multi-tenant SaaS environment with multiple users per tenant, with tenant-specific customizations expressed in blockly as well. The biggest application has about 80 different screens defined that way. 

A screenshot might help envisioning it:

Hope that gives some idea of our use? 

Let me know if you want to hear more. 

We love Blockly, and while perhaps not originally designed for this, it really works for us.

Ewald