data.table - R - write.table() creates table that isn't read properly by fread() -


i have data.table in r i'm trying write out .txt file, , input r.

it's sizeable table of 6.5m observations , 20 variables, want use fread().

when use

write.table(data, file = "data.txt") 

a table of 2.2gb written in data.txt. in manually inspecting it, can see there column names, it's separated " ", , there quotes on character variables. should fine.

however,

data <- fread("data.txt") 

returns data.table of 6.5m observations , 1 variable. ok, maybe reason fread() isn't automatically understanding separator string:

data <- fread("data.txt", sep = " ") 

all data in proper variables now,

  • r has added unnecessary row-number column
  • in 1 (only one) of columns nas have been replaced 9218868437227407266
  • all variable names missing

maybe fread() isn't recognizing header, somehow.

data <- fread("data.txt", sep = " ", header = t) 

now first set of observations column names. not useful.

i'm baffled. understand what's happening here?

edit:

row.names = f solved names problem, ananda mahto.

ran

datasub <- data[runif(1000,1,6497651), ] write.table(datasub, file = "datasub.txt", row.names = f) fread("datasub.txt") 

fread() seems work fine smaller dataset.

edit:

here subset of data created above:

https://github.com/cbcoursera1/exploratorydataanalysisproject2/blob/master/datasub.txt

this data comes national emissions inventory (nei) , made available epa. more information available here:

http://www.epa.gov/ttn/chief/eiinformation.html

edit:

i can no longer reproduce issue. may row.names = f solved issue, or possibly restarting r/clearing environment/something random fixed problem.


Comments

Popular posts from this blog

asp.net mvc - SSO between MVCForum and Umbraco7 -

Python Tkinter keyboard using bind -

ubuntu - Selenium Node Not Connecting to Hub, Not Opening Port -