python - scrapy append to linkextractor links -


i using crawlspider linkextractor crawl links.

how go appending parameters links linkextractor finds? add '?pag_sortorder=0&pag_perpage=999' each link linkextractorextracts.

this perfect use case using process_value argument:

from scrapy.contrib.linkextractors import linkextractor  addition = "?pag_sortorder=0&pag_perpage=999"  linkextractor(process_value=lambda x: x + addition) 

Comments