csv better than dlm?

[Jonathan Peirce]

Hi there,

I've just realised that the new wide text format is saving as
tab-delimitted files with ending .dlm rather than comma-separated-value
files ending .csv. It isn't a major difference but I was intending to
save csv rather than dlm files. Apart from anything else MS Excel
automatically registers itself to open csv files but not dlm (it can
open dlm too if you 'tell it').

I feel pretty sure this is worth changing in Builder to the better
default of csv files. But I could leave the code versions (saveAsText()
and saveAsWideText()) to stick with default=dlm to minimise changes?
There at least it's trivial to control manually.

(Probably I'm trying getting opinions where people just don't care! ;-) )

Jon

--
Jonathan Peirce
Nottingham Visual Neuroscience

http://www.peirce.org.uk


This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham.

This message has been checked for viruses but the contents of an attachment
may still contain software viruses which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.

[Erik Kastman]

Hi Jon,

The only advantage of saving as tab-delimted by default is the escaped-delimter problem Alex Milner and others have run into.

Perhaps a better thing from a UI/UX standpoint would be a format drop-down in Builder's Settings: save wide text format as -> {comma-delimited CSV, tab-delimted DLM } and set a default to .csv?

Erik

> --
> You received this message because you are subscribed to the Google Groups "psychopy-users" group.
> To post to this group, send email to psychop...@googlegroups.com.
> To unsubscribe from this group, send email to psychopy-user...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Michael MacAskill]

On 25 Sep, 2012, at 14:22, Erik Kastman <erik.k...@gmail.com> wrote:

> The only advantage of saving as tab-delimted by default is the escaped-delimter problem Alex Milner and others have run into.

The other big advantage is that tab-delimited files are MUCH more readable than CSV when opened in a plain text editor. Often one wants to view and edit files in one of those rather than Excel, for fear of what Excel might do to it when saving. But also they can offer support for regular expressions etc if the files need manipulating. But any sizeable CSV file generally looks like crap in a text editor.

Any stats program will open either so it doesn't really matter, but CSV is probably more prone to problems with escaped delimiters as Erik notes.

> Perhaps a better thing from a UI/UX standpoint would be a format drop-down in Builder's Settings: save wide text format as -> {comma-delimited CSV, tab-delimted DLM } and set a default to .csv?

Agreed, it would be good for users to be able to choose this. I'd have to say though, that .dlm is not a suffix I've ever actually seen in the wild. I think .txt would be a better choice for tab delimited files, as they will then almost universally be double-clickable to something that will open them without any configuration.

Cheers,

Mike

[Jeremy Gray]

(Probably I'm trying getting opinions where people just don't care! ;-) )

ok, I'll bite: meh, tabs commas whatever :-)

I used tab-sep for years (in part for readability) but have become sort of spooked by empty cells having a tendency to cause havoc if lines are accidentally parsed using a separator rule of "white space" rather than 'a single tab char' as the separator (which happens more than it should--you get ragged edges in an excel sheet and have to try to reconstruct what one was missing). tab-sep is more readable than csv, but csv has its own issues.

For clarity & robustness, more is needed than the choice of separator, and so I cooked up a whole system (Open Behavioral Format, https://github.com/jeremygray/obf ). So far not even I have been tempted to use it, but in principle its a good thing because its really clear what is going on :-) I envision it being useful mostly as a hidden layer, so you can slurp things in if they are in this format and rewrite a copy back out as tab-sep, comma-sep, or whatever. Sol and I were talking about extending the format to include binary blobs as well, which would be a big addition.

--Jeremy

[Jonathan Peirce]

I've just uploaded a 1.75.00 release for you to test. Keenies can
download the zip file and use the update tool in install it. Or wait a
few days and I'll set the auto-update mechanism to run, once we more
sure there are no lurking gremlins.

This release will have the following data outputs by default:
- the Builder will start outputting files in wide-text csv, rather
than tab-delimited format
- a single psydat file for the experiment (but not one for each
separate loop)
- no Excel files (which were always summary files and had one sheet
per loop)
These files will be generated even if the subject aborts (unless they do
something like force-quitting the app)

There are quite a lot of user-interface improvements and a few fixes
too, as below.

best wishes and happy experimenting!
Jon


* IMPROVED: Experiment info dialog box easier to control now from
experiment settings (user doesn't need to write a dictionary by hand any
more)
* IMPROVED: Components in the Builder are now arranged in categories,
including a special 'Favorites' category
* IMPROVED: Code Components now support full syntax highlighting and
code folding (but still aren't quite big enough!)
* ADDED: Builder undo/redo now gives info about what is going to be
un/redone
* ADDED: Window now supports a `stereo` flag to provide support for
quad-buffers (advanced graphics cards only)
* FIXED: bug with copying/pasting Routines that was breaking Flow in
certain situations and corrupting the experiment file
* FIXED: fatal typo in QuestHandler code (Gary Lupyan)
* FIXED: data outputs for multiple key/mouse presses
* ADDED: Microphone now supports `stop` to abort recording early (Jeremy
Gray)
* ADDED: beginning of error reporting when generating Builder
experiments (thanks Piotr Iwaniuk)
* FIXED: csv files now generated from Builder as expected not dlm files
(tab-delimited)