Convert NullableArray w/o nulls to an Array

[Douglas Bates]

I remember someone (Milan?, Alex?) writing a short cal to convert a NullableArray to an Array but I can't seem to find it now.  Can someone refresh my memory, please?

[Milan Bouchet-Valat]

What do you mean exactly? This works AFAICT:
julia> convert(Array, NullableArray(1:3))
3-element Array{Int64,1}:
 1
 2
 3


Regards

[Douglas Bates]

Thanks.  I didn't try that because I thought that a NullableArray was an Array so that conversion would be a no-op.  I should have tested it.

[Douglas Bates]

Of course, this brings me back to another problem, which is replacing a column of a dataframe with an Array without having the Array converted to a DataArray or a NullableArray.  I have a large CSV file in which over 99% of the columns do not contain nulls.  Because some of the columns contain nulls I can't use CSV.read(..., nullable = false) but I do want to convert those columns that do not contain nulls to Arrays.

If I try to replace the column as

mydf[1] = Array(mydf[1])

I get a DataArray in the release version of DataFrames.  (I'm not sure what happens in your nl/nullable branch).

The way I avoid this is right now is by working with mydf.columns directly but I always caution others not to work directly with fields of composite types and I should probably follow my own advice.

[Tom Short]

For avoiding unwanted conversions, DataFramesMeta has a PassThrough type with an alias P. You should be able to do:

mydf[1] = P(Array(mydf[1]))

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Douglas Bates]

On Wednesday, September 14, 2016 at 4:42:39 PM UTC-5, tshort wrote:

For avoiding unwanted conversions, DataFramesMeta has a PassThrough type with an alias P. You should be able to do:

mydf[1] = P(Array(mydf[1]))

True but the approach has a bit of a "burning the toast and then scraping it" feel to it.  (That was a favorite expression of Deming's re quality control.)  I think it would be better for setindex! in a DataFrame to skip the conversion to a DataArray or a NullableArray. 

On Sep 14, 2016 5:17 PM, "Douglas Bates" <dmb...@gmail.com> wrote:

Of course, this brings me back to another problem, which is replacing a column of a dataframe with an Array without having the Array converted to a DataArray or a NullableArray.  I have a large CSV file in which over 99% of the columns do not contain nulls.  Because some of the columns contain nulls I can't use CSV.read(..., nullable = false) but I do want to convert those columns that do not contain nulls to Arrays.

If I try to replace the column as

mydf[1] = Array(mydf[1])

I get a DataArray in the release version of DataFrames.  (I'm not sure what happens in your nl/nullable branch).

The way I avoid this is right now is by working with mydf.columns directly but I always caution others not to work directly with fields of composite types and I should probably follow my own advice.

On Wednesday, September 14, 2016 at 4:07:21 PM UTC-5, Douglas Bates wrote:

Thanks.  I didn't try that because I thought that a NullableArray was an Array so that conversion would be a no-op.  I should have tested it.

On Wednesday, September 14, 2016 at 4:02:04 PM UTC-5, Milan Bouchet-Valat wrote:

Le mercredi 14 septembre 2016 à 13:56 -0700, Douglas Bates a écrit :
> I remember someone (Milan?, Alex?) writing a short cal to convert a
> NullableArray to an Array but I can't seem to find it now.  Can
> someone refresh my memory, please?
What do you mean exactly? This works AFAICT:
julia> convert(Array, NullableArray(1:3))
3-element Array{Int64,1}:
 1
 2
 3


Regards

--
You received this message because you are subscribed to the Google Groups "julia-stats" group.

To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats...@googlegroups.com.

[Tom Short]

I agree.

To unsubscribe from this group and stop receiving emails from it, send an email to julia-stats+unsubscribe@googlegroups.com.