Excessive memory usage with C# client in CouchbaseNetClient 2.0.3.1 -
i using couchbasenetclient 2.0.3.1 c# client , ran out of memory situations when started threading application. while have not tried identify every suspect situation, have created simple scenario demonstrates consider excessive memory usage.
my couchbase server has empty bucket. c# client, open bucket , surprised see application's memory usage on 340 mb @ point. bucket opened, attempt delete 10 invalid items database. memory usage stays same. start running removal of items concurrently , start see increase in memory usage , see levels off around 1 gb.
i concerned high memory usage around, processing concurrently not seem option seeing. there can prevent such high memory usage, or issue addressed?
class program { static void printmemoryusage(string label) { var mb = 1024 * 1024; var gc = gc.gettotalmemory(true) / mb; var pm = process.getcurrentprocess().pagedmemorysize64 / mb; console.writeline($"{label}: gc={gc} mb pm={pm} mb"); } static void exampleofthreadedmemoryissue() { var cluster = new couchbase.cluster(); var poolconfig = cluster.configuration.poolconfiguration; poolconfig.maxsize = environment.processorcount; poolconfig.buffersize = 1024 * 1024 * 20; printmemoryusage("created cluster"); var bucket = cluster.openbucket("default"); printmemoryusage("opened bucket"); (int = 0; < 10; ++i) { bucket.remove("completelyinvalidkey"); } printmemoryusage("removed 10 items serially"); (int concurrent = 1; concurrent <= 10; ++concurrent) { var tasks = new list<task>(); (int = 0; < concurrent; ++i) { tasks.add(task.run(() => { bucket.remove("completelyinvalidkey"); })); } task.waitall(tasks.toarray()); printmemoryusage($"removed {concurrent} items concurrently"); } console.writeline("\npress key continue."); console.readkey(false); } static void main(string[] args) { exampleofthreadedmemoryissue(); } } here results of running example on machine:
created cluster: gc=0 mb pm=18 mb opened bucket: gc=321 mb pm=346 mb removed 10 items serially: gc=321 mb pm=346 mb removed 1 items concurrently: gc=321 mb pm=346 mb removed 2 items concurrently: gc=481 mb pm=508 mb removed 3 items concurrently: gc=641 mb pm=668 mb removed 4 items concurrently: gc=801 mb pm=829 mb removed 5 items concurrently: gc=961 mb pm=991 mb removed 6 items concurrently: gc=961 mb pm=991 mb removed 7 items concurrently: gc=961 mb pm=990 mb removed 8 items concurrently: gc=961 mb pm=990 mb removed 9 items concurrently: gc=961 mb pm=990 mb removed 10 items concurrently: gc=961 mb pm=990 mb press key continue.
the excessive memory usage because of size of buffer pool have configured: 1024 * 1024 * 20 (times maxsize)
poolconfig.buffersize = 1024 * 1024 * 20; the default 1024*16, reason increasing this? largest document size supported couchbase server 20mb.
Comments
Post a Comment