Stdin Issues with External Tools

36 views
Skip to first unread message

Hil tronix

unread,
Jun 22, 2021, 12:50:50 PMJun 22
to PyScripter

Hello,

I'm having some issues with line length buffer size when writing scripts for the Pyscripter External Tools.  For example, I've written a JSON Prettify tool, to quickly format a json string into something readable.  But when the line of json characters is greater than 1024, there is an error, that I think is being caused by the input being limited by a buffer size of what is being passed in the "Selection" of External Tools.

The reason I would sometimes have JSON strings longer than 1024, is that when I work with IoT devices, and I want to review the output during development of interfaces, the JSON results are often very long.  It's convenient to use Pyscripter External Tools to write scripts that solve small problems, in this case formatting JSON to human readable.  It's inconvenient to have to manually find break points in the string that are less than 1024 characters.

So my question is, is the stdin limitation of 1024 characters I'm finding an issue with External Tools selection input passing, or is this with Windows or Python?  If it's with Pyscripter, can this line length buffer size be increased?

I have shared the External Tools parameters below, and a sample JSON string to test.  If you line break the JSON string in half less than 1024 characters, and select both lines when you run the tool, you'll see the script works as expected and pretty formats the JSON.

Thank you.

Example External Tools Properties:

Application: $[PythonExe-short]
Parameters: -c "import sys,json;data=json.loads(sys.stdin.read());results=json.dumps(data, sort_keys=True, indent=4);sys.stdout.write(results);sys.stdout.write('\n')"
Std. Input: Selection
Std. Output: Selection

Here is the sample JSON string:

{"device":{"type":"SHBLB-1","mac":"XXXXXXXXXXXX","hostname":"shellybulb-XXXXX","num_outputs":1},"wifi_ap":{"enabled":false,"ssid":"shellybulb-XXXXXX","key":""},"wifi_sta":{"enabled":true,"ssid":"xxx","ipv4_method":"static","ip":"192.168.0.51","gw":null,"mask":"255.255.255.0","dns":"192.168.0.1"},"wifi_sta1":{"enabled":false,"ssid":null,"ipv4_method":"dhcp","ip":null,"gw":null,"mask":null,"dns":null},"ap_roaming":{"enabled":false,"threshold":-70},"mqtt": {"enable":true,"server":"192.168.0.15:1883","user":"ShelleyBtn1","id":"shellybulb-42104E","reconnect_timeout_max":60.000000,"reconnect_timeout_min":2.000000,"clean_session":true,"keep_alive":60,"max_qos":2,"retain":false,"update_period":30},"coiot": {"enabled":true,"update_period":15,"peer":"","execute_enable":false},"sntp":{"server":"192.168.0.1","enabled":true},"login":{"enabled":true,"unprotected":false,"username":"Automate"},"pin_code":"","name":null,"fw":"20210429-095954/v1.10.4-g3f94cd7","discoverable":true,"build_info":{"build_id":"20210429-095954/v1.10.4-g3f94cd7","build_timestamp":"2021-04-29T09:59:54Z","build_version":"1.0"},"cloud":{"enabled":false,"connected":false},"timezone":"America/Winnipeg","lat":500.000000,"lng":500.000000,"tzautodetect":false,"tz_utc_offset":-21600,"tz_dst":false,"tz_dst_auto":true,"time":"","unixtime":0,"debug_enable":false,"allow_cross_origin":false,"actions":{"active":false,"names":["out_on_url","out_off_url"]},"hwinfo": {"hw_revision":"prod-1.3","batch_id":1},"mode":"white","lights":[{"ison":true,"red":25,"green":254,"blue":48,"white":0,"gain":100,"temp":4750,"brightness":100,"effect":0,"default_state":"on","auto_on":0.00,"auto_off":0.00,"power":0.00,"schedule":false,"schedule_rules":[]}]}


PyScripter

unread,
Sep 19, 2021, 6:27:52 PMSep 19
to PyScripter
Not sure.  I will have a look about this limitation.

PyScripter

unread,
Sep 19, 2021, 7:04:47 PMSep 19
to PyScripter
Here with PyScripter v.4 the above file works after I format it.  I have tried with a Json file up to 5Kbs and it worked.  Not sure thought whether the limitation is on the line length.  It appears that it is not on the total size.  Will try...

PyScripter

unread,
Sep 19, 2021, 7:08:37 PMSep 19
to PyScripter
Yes the limitation of 1024 characters is on the line length.  I will see what I can do about it.

PyScripter

unread,
Sep 19, 2021, 8:00:20 PMSep 19
to PyScripter
If you want this fixed, could you please open an issue at  Issues · pyscripter/pyscripter (github.com)?

PyScripter

unread,
Oct 10, 2021, 9:05:57 PMOct 10
to PyScripter
Fixed (see  External tools input line length limitation · Issue #1136 · pyscripter/pyscripter (github.com)).  The fix will be available with the next release.

Hil tronix

unread,
Oct 20, 2021, 7:30:24 PM (6 days ago) Oct 20
to PyScripter
Thank you!  I only saw your replies today.  For some reason I'm not getting notifications from groups.  Unfortunately I'm getting errors when trying to close 4.1 after upgrading from 4.0.  And code completion no longer works after upgrading to 4.1.

PyScripter

unread,
Oct 20, 2021, 7:34:18 PM (6 days ago) Oct 20
to PyScripter
Regarding the error on shutdown, I am releasing a new version tomorrow.  4.1.0.4 that fixes that and a couple of other issues.  Please download again and try.  On the other hand code completion should work.  Are you using the installed version or the portable one? (zip file)?

PyScripter

unread,
Oct 20, 2021, 9:44:10 PM (6 days ago) Oct 20
to PyScripter
PyScripter v4.1.1 was released.  Please try and report whether it fixes the error on shutdown.   Regarding code completion, if you are using the portable version make sure you put it in a directory with write access (e.g. not in Program Files).
Reply all
Reply to author
Forward
0 new messages