Views, Copies, and the SettingWithCopiesWarning Issue

[Nick Eubank]

Hello All,

As pandas approaches its 1.0 release, I would like to raise a concern about one aspect of the pandas architecture that I think is a threat to its widespread adoption: how pandas works with copies and views when setting values (what I will refer to here as the SettingWithCopyWarning issue). 

I've posted a new issue that I would love general input on here, but the summary of my concerns are:

1. SettingWithCopyWarning is a threat to data integrity 
   
2. It is unreasonable to expect the average user to avoid a `SettingWithCopyWarning` issue, as doing so requires keeping track of the plethora of factors that determine what generates a copy and what generates a view. 
1. Views made sense in `numpy`, but not in `pandas`. 
2. Chain-indexing is a much more subtle problem than suggested in the `pandas` docs. 
3. Given (1) and (2), data integrity in `pandas` relies on users noticing a non-exception warning in the flow of their output. 
   
4. Even aside from the threat to data integrity, this behavior is unpythonic, and likely to frustrate alienate lots of potential users of `pandas`. 
   
5. I think solutions can be found that would have only limited effects on performance for the majority of users 

If you have any thoughts on these issues, please consider visiting the github issue I've posted and weigh in!

Nick

[Joris Van den Bossche]

Hi Nick,

I will try to response later on the github issue, but I just already wanted to say: thanks a lot for writing this up!

Such detailed feedback and concerns from the community is really valuable, and important in making pandas better!

Regards,

Joris

--
You received this message because you are subscribed to the Google Groups "PyData" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pydata+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.