V Batch Hack

[Daisy Hughlett]

Ijust built an extensive Batch Macro (nested within another Batch Macro) with 36 Control Parameters that will Dynamically Filter / Apply Formula Changes based on settings from an Excel Controller. I thought that I could use a Union Tool or GroupBy the Batch ID in order to get it to loop the records through for the next batch, but unfortunately I can only output duplicate records for each batch or filter for just the changed records for each batch... I am coming to the unfortunate realization that it should have been built as an Iterative Macro in order to loop records through to be assessed for further changes.

I am really hoping that there is some cool trick for Iterating a Batch Macro, or passing formula tool actions into an Iterative Macro (if it is nested in a Batch Macro) that would allow me to convert this over to an Iterative Macro? The only way I have been able to get a formula into an Iterative Macro for evaluation so far is when I created an Iterative version of the Dynamic Replace Tool (which has a basic formula Eval built in). While I am on the topic... Why don't we also have Control Parameters in Iterative Macros in the first place?

Unfortunately, there is no magic bullet to convert from a batch to an iterative macro, but you can simulate a control parameter in an interactive macro. Add another Macro Input and pass in a dataset that has a RecordID field, i.e. starts at 0 with an increment of 1, as well as all the control parameter fields. Use a filter after this input with [RecordID] = [Engine.IterationNumber] to use the row that corresponds to the current iteration.

For your situation, you should probably look at converting your outer macro to an iterative one that calls the inner batch macro passing the the row mentioned above as the control parameters. The output of you inner macro goes to the iterative Output of the outer one so the modified Input records can be used on the next iteration, with the control parameters that correspond to row of the second Input.

To stop looping, check to see if the result of the filter on the second input has 0 rows. If so, you've looped through all the "Control" records so pass the output of the inner macro to data output and no rows to the iterative output.

Converting the outer macro would be much easier than converting the inner macro. The outer macro is quite simple and is really just doing some controller data transformation to break back out incoming concatenated values so that I reduce the number of Macro Field settings from 36 to 5. I will try to implement your recommendations and let you know how it goes. Thank you very much for your quick and easily understood response!

"The output of you inner macro goes to the iterative Output of the outer one so the modified Input records can be used on the next iteration, with the control parameters that correspond to row of the second Input."

I was definitely envisioning the flow of data incorrectly. I had assumed that the inner macro would cycle through its own input / output until completion (Unless it was waiting for the outer macro to batch send control parameter data to an input.) I now understand how an iterative outer macro could solve the issue.

I have a Survey123 survey form, via which data is logged. For example, staff would enter information on Device usage and errors for the company pool devices used. Each survey would collect data like device name, device type, date & time of usage or errors encountered etc. These data on the device usage are only entered while the devices are active - once they reach their lifespan, the devices are retired and replaced.

I have a ArcGIS dashboard linked to the Survey123 data, which show the number of active devices, number of errors encountered daily, amount of daily usage etc. I only want to display data for the active devices on the dashboard - however that means having to manually filter out the retired device names, in each of the charts inside the dashboard. This would be very time consuming.

So rather than manually actioning the filter, it seems I could create a new field inside the Survey that automatically gives each submission a default value of 'ACTIVE'. If I use the Batch Attribute Editor, I could batch-edit the 'ACTIVE' value of all existing survey submission data of device AAA111 to 'RETIRED', and use this to mass-filter the Dashboard charts. Is my understanding correct?

Additionally, the main chunk of my survey data is in a repeat layer of the survey. I have been trying to test out the Web Appbuilder and the Batch Attribute Editor widget, but the repeat layer is not showing up on the widget configuration page for selection at all. Any idea what I might be doing wrong, or is the Batch Attribute Editor not compatible with repeat layers?

I believe the selection of features has to be performed using the selection tools within the Batch Attribute Editor widget, and any selections you've performed using other widgets (e.g. Select widget) are not used.

Based on your screenshot, I think you may need to make an adjustment in the Batch Attribute Editor widget configuration. You will want to enable the option to Select by Area, with the tools to be used for the selection.

Thank you for sharing! Based on your XLS, I see there is no Geopoint question. Is your repeat layer a related table? Technically, the Batch Attribute Editor widget only supports layers with a geometry that can be found in the web map (so no related tables).

Ahh this is perfect, thank you so much for your help! I thought that since I'm selecting only by specific value of a specific field (rather than selecting by location) I had to use the select by feature. Thank you again, this is really helpful!

Hi @Anonymous User , right now there isn't, but this will be possible in the near future with the new editing experiences in Experience Builder. There are plans to allow for batch editing and also editing features from a selection - from a list or a table.

Batch macros can be intimidating, but the intention of this course is to dispell your worries. Learn when to consider a batch macro, identify pre-existing processes that could be turned into batch macros, and create your very first batch macro!

Featured Tools:

Macro Input, Macro Output, Control Parameter

To circumnavigate this behavior, you'll see the text input tool setup as just 0 - so if no count returns from the summarize above it, it will become the output value. If something is returned from the summarize, you'll see that it is ordered so the 0 will always be sampled out.

To give more context: i have 150+ different DB related to different entities. However in these file i do not have an entire column with the code/name of the entity, but this information can be found at the end of each file. I have built a standard macro to make the process.

Same here, validating for more than 9 hours, but successfully completed a while ago. It just took a few minutes for the actual completion, though. The system restored service at around 12pm est for both of my batch jobs submitted 9 and 1 hours ago, respectively.

Same problem, i am gonna have to use a charger far from equipped to charge my batch 6 until either the 240w comes out, which im still hoping is any day now, or buy a charger when they become available

The Amazon link you provided comes up with this in the listing. But looking at it again it does also say NVMe as the Hard Disk Interface, but I would expect the Connectivity Technology field to show PCIe.

Ray-tracing is on, but off for sun-shadows since it used to cause crashes for me a few builds ago when I first gave it a go with my newer RTX card upgrade. I also rely mainly on artificial light sources, so soft exterior sun shadows not a big deal for my use and those seemed the most problematic a while back. The upgraded reflections are a big benefit, so I would like to keep "basic" RT on if at all possible.

DLSS and DeNoiser are on. Haven't tried toggling those yet. Honestly, because I don't want to spend the day troubleshooting when I need the batch to just work from time to time when I need it for these larger jobs. DeNoiser is something I want to keep on, since it does help results in my experience.

To me, it seems to be the processing time with SketchUp transitioning from scene to scene in my larger models. Some scenes have more complex geometry and SketchUp lags, which I bet causes Enscape to run into a timeout issue or something like that. Still, that doesn't explain the simply longer render times using the batch render option. Yes, it's still "fast" considering it's rendering 20+ scenes vs something like VRay. It's just "slow" compared to what I'm used to doing single renders in Enscape already.

The data is 3 channel time series of celltrack/PI/AnnexinV for looking at appoptosis. We have a Fiji macro that has served us well over the years but StarDist is much faster and more robust at pulling out the nuclei.

Hey guys @uschmidt83 @Cameron.Nowell ,

I was very happy to find this thread as I was thinking to run StarDist in a loop to segment a Z-stack. Now I am wondering if that is possible. I am trying to call StarDist from a macro for loop:

}

which does not work (I guess since StarDist does not work on stacks). If I change the image properties to make a time series and not a Z-stack it blows every segment for every frame. Is it possible to look a function that only accepts 2D to analyze 3D? What I am missing?

For my time series stuff i just give it the time stack and StarDist processes it all (code example below for that function in Fiji macro). I have it generate ROIs only and associate them with the frame position. Then dump them into a blank temp image with the same dimensions and paint them all (ssemed to be the only way to get it to work in batch mode).

3a8082e126