python - Do file size requirements change when importing a CSV file to MongoDB? -


background:

i'm attempting follow tutorial in i'm importing csv file that's approximately 324mb

enter image description here

to mongolab's sandbox plan (capped @ 500mb), via pymongo in python 3.4.

the file holds ~ 770,000 records, , after inserting ~ 164,000 hit quota , received:

raise operationfailure(error.get("errmsg"), error.get("code"), error)  operationfailure: quota exceeded  

question:

would accurate json-like structure of nosql takes more space hold same data csv file? or doing screwy here?

further information:

here database metrics:

enter image description here

here's python 3.4 code used:

import sys import pymongo import csv   mongodb_uri = '***credentials removed***'   def main(args):      client = pymongo.mongoclient(mongodb_uri)      db = client.get_default_database()      projects = db['projects']      open('opendata_projects.csv') f:         records = csv.dictreader(f)         projects.insert(records)      client.close()   if __name__ == '__main__':     main(sys.argv[1:]) 

not accounting things compression, set of json documents take more space csv, because field names repeated in each record, whereas in csv field names in first row.

the way files allocated factor:

in filesize section of database metrics screenshot attached, notice says first file allocated 16mb, next 1 32mb, , on. when data grew past 240mb total, had 5 files, of 16mb, 32mb, 64mb, 128mb, , 256mb. explains why filesize total 496mb, though data size 317mb. next file allocated 512mb, put way past 500mb limit.


Comments

Popular posts from this blog

shopping cart - Page redirect not working PHP -

php - How to modify a menu to show sub-menus -

python - Installing PyDev in eclipse is failed -