python - Do file size requirements change when importing a CSV file to MongoDB? -
background:
i'm attempting follow tutorial in i'm importing csv file that's approximately 324mb

to mongolab's sandbox plan (capped @ 500mb), via pymongo in python 3.4.
the file holds ~ 770,000 records, , after inserting ~ 164,000 hit quota , received:
raise operationfailure(error.get("errmsg"), error.get("code"), error) operationfailure: quota exceeded question:
would accurate json-like structure of nosql takes more space hold same data csv file? or doing screwy here?
further information:
here database metrics:

here's python 3.4 code used:
import sys import pymongo import csv mongodb_uri = '***credentials removed***' def main(args): client = pymongo.mongoclient(mongodb_uri) db = client.get_default_database() projects = db['projects'] open('opendata_projects.csv') f: records = csv.dictreader(f) projects.insert(records) client.close() if __name__ == '__main__': main(sys.argv[1:])
not accounting things compression, set of json documents take more space csv, because field names repeated in each record, whereas in csv field names in first row.
the way files allocated factor:
in filesize section of database metrics screenshot attached, notice says first file allocated 16mb, next 1 32mb, , on. when data grew past 240mb total, had 5 files, of 16mb, 32mb, 64mb, 128mb, , 256mb. explains why filesize total 496mb, though data size 317mb. next file allocated 512mb, put way past 500mb limit.
Comments
Post a Comment