dealing with big .py files in editor

[Steve Byrnes]

Hi everyone,

I have a particular .py file that I'm using essentially to store data. It has a line of text with ~1 million characters. It's syntactically correct and python can run it instantaneously. But the editor in spyder cannot open it (at least, not within the 20 minutes I waited).

The editor in spyder CAN open it if I change the file extension to nonsense ".pyy", or even if I just comment out (add the character "#" at the beginning of) the million-character line. When I do that, it's a bit slow in spyder (compared to using a dedicated text editor) but usable.

So I conclude that the possible problems are:

(A) Syntax-highlighting is choking on the large .py file,

(B) Code introspection is choking on the large .py file,

(C) Something else I'm not thinking of.

(B) seems to be ruled out because the problem persists when I turn off all the code introspection / analysis in the Preferences menu. (A) seems possible. I tried to open the same file in a few dedicated syntax-highlighting text editors ... geany and notepad++ loaded it with no problem at all, emacs was slghtly slow, but gedit could not load it. And of course (C) is also possible.

Any ideas, advice, experiences?

Thanks in advance!

Steve Byrnes

Postdoc, Applied Physics, Harvard University

http://sjbyrnes.com/

[Big Stone]

this is clearly a bug: Syntax-highlighting and code-introspection should disable themselves if a line exceed 9999 characters.
(which is already way of 10 times 80)

[Denis Akhiyarov]

for large files nothing can beat sublime (except maybe vim :) ), I regularly open 1 gb text  files

[anatoly techtonik]

On Sunday, August 30, 2015 at 4:16:52 AM UTC+3, Denis Akhiyarov wrote:

for large files nothing can beat sublime (except maybe vim :) ), I regularly open 1 gb text  files

Do you know how to add that to Spyder? 

[Denis Akhiyarov]

no, spyder and sublime text do not look compatible, sublime comes with its own set of extensions. i use because it is blazing fast, very versatile, and has small installation.