twitter - Train customized corpus, with NLTK for Python -


i try train corpus own documents. documents structured in same way original movie_reviews corpus data, 1k positive text files in folder 'pos' , 1k negative text files in folder 'neg'. each textfile contains 25 lines of tweets, cleaned, in: urls, usernames, capital letters, punctuation removed.

how can adjust code use own text data instead of movie_reviews?

import nltk.classify.util nltk.classify import naivebayesclassifier nltk.corpus import movie_reviews collections import defaultdict import numpy np  # define split of % training / % test split = 0.8  def word_feats(words):     return dict([(word, true) word in words])   posids = movie_reviews.fileids('pos') negids = movie_reviews.fileids('neg')  negfeats = [(word_feats(movie_reviews.words(fileids=[f])), 'neg') f in negids] posfeats = [(word_feats(movie_reviews.words(fileids=[f])), 'pos') f in posids]  cutoff = int(len(posfeats) * split)  trainfeats = negfeats[:cutoff] + posfeats[:cutoff] testfeats = negfeats[cutoff:] + posfeats[cutoff:]  print 'train on %d instances\ntest on %d instances' % (len(trainfeats),len(testfeats))  classifier = naivebayesclassifier.train(trainfeats) print 'accuracy:', nltk.classify.util.accuracy(classifier, testfeats)  classifier.show_most_informative_features()

you can login root user , change directory path this:

/usr/local/lib/python2.7/dist-packages/nltk/corpus/__init__.py

in document can find existing movie_reviews corpora loaded using lazycorpusloader:

movie_reviews = lazycorpusloader(     'movie_reviews', categorizedplaintextcorpusreader,     r'(?!\.).*\.txt', cat_pattern=r'(neg|pos)/.*')

then try adding thing similar this:

my_movie = lazycorpusloader(     'my_movie', categorizedplaintextcorpusreader,     r'(?!\.).*\.txt', cat_pattern=r'(neg|pos)/.*')

where my_movie name have created movie reviews. once done save , exit.

finally place corpus in nltk directory can find movie_review corpus.

try performing this:

from nltk.corpus import my_movie  # newly created own corpus

hope work.


Comments

Post a Comment

Popular posts from this blog

shopping cart - Page redirect not working PHP -

im trying redirect user when click "empty cart". redirect working "add cart" not "empty cart". ideas great. in main .php file set current url seen $current_url = base64_encode("http://$_server[http_host]$_server[request_uri]"); then below cartaction.php file handles actions such "add cart" , "empty cart" <?php session_start(); include_once 'config.php'; //add item in shopping cart if(isset($_post["add"])) { $product_id = $_post["product_id"]; //product id $return_url = base64_decode($_post["return_url"]); //return url $qty = $_post["qty"]; $product = $mysqli->query("select product_name, unit_price, unit_quantity products product_id = '$product_id'"); $obj = $product->fetch_object(); if(!isset($_session['products'])) { $_session['products'] = array(); } if(array_key_exists($product_id...
Read more

php - How to modify a menu to show sub-menus -

i have custom php menu made person, displays categories menu. however, need display subcategories well. tried adding hierarchical parameter didn't work. hoping assistance! here code: <ul> <?php /* categories wp e-commerce products */ $wpec_product_categories = get_terms( 'wpsc_product_category', 'hide_empty=0&parent=0&exclude=298'); $myarray = array(); foreach($wpec_product_categories $wpec_categories){ $myarray[$wpec_categories->term_id]['name'] = $wpec_categories->name; $myarray[$wpec_categories->term_id]['slug'] = $wpec_categories->slug; $myarray[$wpec_categories->term_id]['term_id'] = $wpec_categories->term_id; } foreach($myarray $wpec_categories1){ $wpec_term_name = $wpec_categories1['name']; $wpec_term_slug = $wpec_categories1['slug']; $wpec_term_id = $wpec_categories1['term_id']; echo '<li> <a href="'.get_page_link(19)....
Read more

python - Installing PyDev in eclipse is failed -

unable read repository @ http://pydev.sf.net/updates/content.xml. sun.security.validator.validatorexception: pkix path building failed: sun.security.provider.certpath.suncertpathbuilderexception: unable find valid certification path requested target i installing pydev in eclipse using python, doesn't install successfully. gives above error. happened? how solve error? i had error when trying install on laptop behind corporate firewall. couldn't tell setup of firewall, not sure of root cause. i got passed error installing pydev eclipse when out behind firewall. another way got around have eclipse on non-firewalled machine, install pydev eclipse, , copy whole eclipse directory non-firewalled machine firewalled machine. there better way, , way fix properly, didn't have time work out firewall doing. this on windows 7 machine (yeah know) not work *nix versions installed through package manager yum or apt.
Read more