data.table - R - write.table() creates table that isn't read properly by fread() -
i have data.table in r i'm trying write out .txt file, , input r.
it's sizeable table of 6.5m observations , 20 variables, want use fread().
when use
write.table(data, file = "data.txt")
a table of 2.2gb written in data.txt. in manually inspecting it, can see there column names, it's separated " ", , there quotes on character variables. should fine.
however,
data <- fread("data.txt")
returns data.table of 6.5m observations , 1 variable. ok, maybe reason fread() isn't automatically understanding separator string:
data <- fread("data.txt", sep = " ")
all data in proper variables now,
- r has added unnecessary row-number column
- in 1 (only one) of columns nas have been replaced 9218868437227407266
- all variable names missing
maybe fread() isn't recognizing header, somehow.
data <- fread("data.txt", sep = " ", header = t)
now first set of observations column names. not useful.
i'm baffled. understand what's happening here?
edit:
row.names = f
solved names problem, ananda mahto.
ran
datasub <- data[runif(1000,1,6497651), ] write.table(datasub, file = "datasub.txt", row.names = f) fread("datasub.txt")
fread()
seems work fine smaller dataset.
edit:
here subset of data created above:
https://github.com/cbcoursera1/exploratorydataanalysisproject2/blob/master/datasub.txt
this data comes national emissions inventory (nei) , made available epa. more information available here:
http://www.epa.gov/ttn/chief/eiinformation.html
edit:
i can no longer reproduce issue. may row.names = f
solved issue, or possibly restarting r/clearing environment/something random fixed problem.
Comments
Post a Comment