asp.net - Handle large number of PUT requests to a rest api -

i have been trying find way make task more efficient. consuming rest based web service , need update information on 2500 clients.

i using fiddler watch requests, , i'm updating table update time when complete. i'm getting 1 response per second. expectations high? i'm not sure define 'fast' in context.

i handling in controller , have tried running multiple web requests in parallel based on examples around place doesn't seem make difference. honest don't understand enough , trying build. suspect still waiting each request complete before firing again.

i have increased connections in web config file per suggestion no success:

 <system.net>     <connectionmanagement>       <add address="*" maxconnection="20" />     </connectionmanagement>   </system.net> 

my controllers action method looks this:

public async task<actionresult> updatemattersasync()         {            //only  matters haven't synced yet             list<matterclientrepair> repairlist = data.get.allunsyncedmatterclientrepairs(true);         //take next 500         list<matterclientrepair> subrepairlist = repairlist.take(500).tolist();                    finalisedmatterviewmodel vm = new finalisedmatterviewmodel();           using (applicationdbcontext db = new applicationdbcontext())         {             int jobcount = 0;             foreach (var job in subrepairlist)             {                 // if not yet synced - shouldn't ever be!!                 if (!job.synced)                 {                     jobcount++;                     // set authentication fields                     var oauth = new oauth.manager();                     oauth["access_token"] = session["accesstoken"].tostring();                     string uri = "https://app.com/api/v2/matters/" + job.matter;                      // prepare json object body                     matterclientjob jsonbody = new matterclientjob();                     jsonbody.matter = new matterforupload();                     jsonbody.matter.client_id = job.newclient;                     string jsonstring = jsonbody.tojson();                                             // send off. returns whole object updated - don't                     matter result = await oauth.update<matter>(uri, oauth["access_token"], "put", jsonstring);                      // update our entities                                            var updatejob = db.matterclientrepairs.find(job.id);                     updatejob.synced = true;                     updatejob.update_time = datetime.now;                     db.entry(updatejob).state = system.data.entity.entitystate.modified;                     if (jobcount % 50 == 0)                     {                         // save every 50 changes                         db.savechanges();                     }                 }             }              // if there remaining files save             if (jobcount % 50 != 0)             {                 db.savechanges();             }              return view("finalisedmatters", data.get.allmatterclientrepairs());         }     } 

and of course update method handles web requesting:

  public async task<t> update<t>(string uri, string token, string method, string json)                 {                   var authzheader = generateauthzheader(uri, method);                 // prepare token request                  var request = (httpwebrequest)webrequest.create(uri);                 request.headers.add("authorization", authzheader);                 request.method = method;             request.contenttype = "application/json";             request.accept = "application/json, text/javascript";             byte[] bytes = system.text.encoding.ascii.getbytes(json);             request.contentlength = bytes.length;             system.io.stream os = request.getrequeststream();             os.write(bytes, 0, bytes.length);             os.close();             webresponse response = await request.getresponseasync();              using (var reader = new system.io.streamreader(response.getresponsestream()))             {                 return jsonconvert.deserializeobject<t>(reader.readtoend());              }          } 

if it's not possible more 1 request per second i'm interested in looking @ ajax solution can give user feedback while processing. in current solution cannot give user feedback while action method hasn't reached 'return' yet can i?

okay it's taken me few days (and lot of trial , error) i've worked out. can others. found silver bullet. , place should have started: msdn: consuming task-based asynchronous pattern

in end following line of code brought light.

string [] pages = await task.whenall(from url in urls select  downloadstringasync(url)); 

i substituted few things make work put request follows:

httpresponsemessage[] results = await task.whenall(from p in toupload select client.putasync(p.uri, p.jsoncontent)); 

'toupload' list of myclass:

  public class myclass     {         // uri should relative base pase         // (ie: /api/v2/matters/101)         public string uri { get; set; }          // string in json format, being body of put request         public stringcontent jsoncontent { get; set; }     } 

the key stop trying put putasync method inside loop. new line of code still blocking until responses have come back, wanted. also, learning use linq style expression create task list on fly immeasurably helpful. won't post code (unless wants it) because it's not nicely refactored original , still need check whether response of each item 200 ok before record saved in database. how faster it?

results

i tested sample of 50 web service calls local machine. (there saving of records sql database in azure @ end).

original synchronous code: 70.73 seconds

asynchronous code: 8.89 seconds

that's gone 1.4146 requests per second down mind melting 0.1778 requests per second! (if average out)

conclusion

my journey isn't over. i've scratched surface of asynchronous programming , loving it. need work out how save results have returned 200 ok. can deserialize httpresponse returns json object (which has unique id can etc.) or use task.whenany method, , experiment interleaving.