linq to sql - Out of memory when creating a lot of objects C# -
i'm processing 1 million records in application, retrieve mysql database. i'm using linq records , use .skip() , .take() process 250 records @ time. each retrieved record need create 0 4 items, add database. average amount of total items has created around 2 million.
iqueryable<object> objectcollection = datacontext.repository<object>(); int amounttoskip = 0; ilist<object> objects = objectcollection.skip(amounttoskip).take(250).tolist(); while (objects.count != 0) { using (datacontext = new linqtosqlcontext(new datacontext())) { foreach (object objectrecord in objects) { // create 0 - 4 random items (int = 0; < random.next(0, 4); i++) { item item = new item(); item.id = guid.newguid(); item.object = objectrecord.id; item.created = datetime.now; item.changed = datetime.now; datacontext.insertonsubmit(item); } } datacontext.submitchanges(); } amounttoskip += 250; objects = objectcollection.skip(amounttoskip).take(250).tolist(); }
now problem arises when creating items. when running application (and not using datacontext) memory increases consistently. it's items never getting disposed. notice i'm doing wrong?
thanks in advance!
ok i've discussed situation colleague of mine , we've come following solution works!
int amounttoskip = 0; var finished = false; while (!finished) { using (var datacontext = new linqtosqlcontext(new datacontext())) { var objects = datacontext.repository<object>().skip(amounttoskip).take(250).tolist(); if (objects.count == 0) finished = true; else { foreach (object object in objects) { // create 0 - 4 random items (int = 0; < random.next(0, 4); i++) { item item = new item(); item.id = guid.newguid(); item.object = object.id; item.created = datetime.now; item.changed = datetime.now; datacontext.insertonsubmit(item); } } datacontext.submitchanges(); } // cumulate amounttoskip processamount don't go on same items again amounttoskip += processamount; } }
with implementation dispose skip() , take() cache everytime , don't leak memory!
Comments
Post a Comment