python 2.7 - What does reduce() do without mapper() in MRJob? -
i new python , trying build recommendation system following instruction http://www.yekeren.com/blog/archives/1005,
what confuses me :
def reducer3_init(self): self.pop = { } file = open(self.options.item_pop, "r") line in file.readlines(): movieid_jstr, pop_jstr = line.strip().split("\t") movieid = json.loads(movieid_jstr) pop = json.loads(pop_jstr) self.pop[movieid] = pop file.close() def reducer3(self, key, values): yield key, sum(values) / math.sqrt(self.pop[key[0]] * self.pop[key[1]])
the reduce3
has no corresponding mapper
,how execute ? , json.load() do? lot thanks!!
the documentation says:
class mrjob.step.mrstep(**kwargs)
used mrjob.steps. see multi-step jobs sample usage.
accepts following keyword arguments. parameters:
mapper – function same function signature mapper(), or none identity mapper.
it idiom map defaults identity , reduce flatten.
Comments
Post a Comment