티스토리 뷰

Flux in Julia/Learning Julia (Intro_to_Julia_DFs)

12. pitfalls

딥스탯 2018. 10. 19. 14:12
12_pitfalls

Introduction to DataFrames

Bogumił Kamiński, Apr 21, 2018

Reference

Series

In [1]:
using DataFrames

Possible pitfalls

Know what is copied when creating a DataFrame

In [2]:
x = DataFrame(rand(3, 5))
Out[2]:
x1x2x3x4x5
Float64Float64Float64Float64Float64
10.3790610.537440.7761470.2539680.0953152
20.5362930.9076520.3711860.2832860.0397663
30.009140990.5461850.4984170.3649730.453046
In [3]:
y = DataFrame(x)
x === y # no copyinng performed

┌ Warning: In the future DataFrame constructor called with a `DataFrame` argument will return a copy. Use `convert(DataFrame, df)` to avoid copying if `df` is a `DataFrame`.
│   caller = top-level scope at In[3]:1
└ @ Core In[3]:1
Out[3]:
true
In [4]:
y = copy(x)
x === y # not the same object
Out[4]:
false
In [5]:
all(x[i] === y[i] for i in ncol(x)) # but the columns are the same
Out[5]:
true
In [6]:
x = 1:3; y = [1, 2, 3]; df = DataFrame(x=x,y=y) # the same when creating arrays or assigning columns, except ranges
Out[6]:
xy
Int64Int64
111
222
333
In [7]:
y === df[:y] # the same object
Out[7]:
true
In [8]:
typeof(x), typeof(df[:x]) # range is converted to a vector
Out[8]:
(UnitRange{Int64}, Array{Int64,1})

Do not modify the parent of GroupedDataFrame

In [9]:
x = DataFrame(id=repeat([1,2], outer=3), x=1:6)
g = groupby(x, :id)
Out[9]:
GroupedDataFrame with 2 groups based on key: :id
First Group: 3 rows
│ Row │ id    │ x     │
│     │ Int64 │ Int64 │
├─────┼───────┼───────┤
│ 1   │ 1     │ 1     │
│ 2   │ 1     │ 3     │
│ 3   │ 1     │ 5     │
⋮
Last Group: 3 rows
│ Row │ id    │ x     │
│     │ Int64 │ Int64 │
├─────┼───────┼───────┤
│ 1   │ 2     │ 2     │
│ 2   │ 2     │ 4     │
│ 3   │ 2     │ 6     │
In [10]:
x[1:3, 1]=[2,2,2]
g # well - it is wrong now, g is only a view
Out[10]:
GroupedDataFrame with 2 groups based on key: :id
First Group: 3 rows
│ Row │ id    │ x     │
│     │ Int64 │ Int64 │
├─────┼───────┼───────┤
│ 1   │ 2     │ 1     │
│ 2   │ 2     │ 3     │
│ 3   │ 1     │ 5     │
⋮
Last Group: 3 rows
│ Row │ id    │ x     │
│     │ Int64 │ Int64 │
├─────┼───────┼───────┤
│ 1   │ 2     │ 2     │
│ 2   │ 2     │ 4     │
│ 3   │ 2     │ 6     │

Remember that you can filter columns of a DataFrame using booleans

In [11]:
using Random
Random.seed!(1)
x = DataFrame(rand(5, 5))
Out[11]:
x1x2x3x4x5
Float64Float64Float64Float64Float64
10.2360330.2109680.5557510.2094720.0769509
20.3465170.9519160.4371080.2513790.640396
30.3127070.9999050.4247180.02037490.873544
40.007909280.2516620.7732230.2877020.278582
50.4886130.9866660.281190.8595120.751313
In [12]:
x[x[:x1] .< 0.25] # well - we have filtered columns not rows by accident as you can select columns using booleans
Out[12]:
x1x4
Float64Float64
10.2360330.209472
20.3465170.251379
30.3127070.0203749
40.007909280.287702
50.4886130.859512
In [13]:
x[x[:x1] .< 0.25, :] # probably this is what we wanted
Out[13]:
x1x2x3x4x5
Float64Float64Float64Float64Float64
10.2360330.2109680.5557510.2094720.0769509
20.007909280.2516620.7732230.2877020.278582

Column selection for DataFrame creates aliases unless explicitly copied

In [14]:
x = DataFrame(a=1:3)
x[:b] = x[1] # alias
x[:c] = x[:, 1] # also alias
x[:d] = x[1][:] # copy
x[:e] = copy(x[1]) # explicit copy
display(x)
x[1,1] = 100
display(x)
abcde
Int64Int64Int64Int64Int64
111111
222222
333333
abcde
Int64Int64Int64Int64Int64
110010010011
222222
333333
┌ Warning: indexing with colon as row will create a copy in the future use df[col_inds] to get the columns without copying
│   caller = top-level scope at In[14]:3
└ @ Core In[14]:3
저작자표시 비영리 변경금지

'Flux in Julia > Learning Julia (Intro_to_Julia_DFs)' 카테고리의 다른 글

13. extras  (0) 2018.10.20
12. pitfalls (한글)  (0) 2018.10.19
11. performance (한글)  (0) 2018.10.18
11. performance  (0) 2018.10.18
10. transforms (한글)  (0) 2018.10.16
공지사항
최근에 올라온 글
최근에 달린 댓글
Total
Today
Yesterday
링크
TAG
more
«   2025/05   »
일 월 화 수 목 금 토
1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31
글 보관함