java - What is the best way to iterate over a large result set in JDBC/iBatis 3? -

-

we're trying iterate on large number of rows database , convert objects. behavior follows:

  • result sorted sequence id, new object created when sequence id changes. object created sent external service , have wait before sending 1 (which means next set of data not used)
  • we have invested code in ibatis 3 ibatis solution best approach (we've tried using rowbounds haven't seen how iteration under hood).
  • we'd balance minimizing memory usage , reducing number of db trips.
  • we're open pure jdbc approach we'd solution work on different databases.

update 1:

  • we need make few calls db possible (1 call ideal scenario) while preventing application use memory. there other solutions out there type of problem may pure jdbc or other technology?

update 2

  • query backend driven , have 1 instance executing @ given time.

thanks , hope hear insights on this.

it seems need sort of pagination. ibatis through standard limit/offset parameters in query (and rowbounds in ibatis 3 ).

but seems (if right) using group feature of ibatis, select returning n records n1 distint "idx" fields result in creation of n1 "parent" objects each 1 having several children objects (with total of n children objects created). or that.

unfortunately (and understandably) both things do not mix well.

i dont' see silver bullet here, 1 can think of many approaches, each has shortcomings - hard evaluate without more information.

if main objects "big" (many records) , each 1 processed individually (with trip remote server) might want ad-hoc pagination, object per page, remembering internally previosuly read id (something select ... ... id = (select min(id) .... id > #lastid# ) )


Comments

Post a Comment

Popular posts from this blog

javascript - Enclosure Memory Copies -

-
function myclass() { //lots , lots of vars , code here. this.bar = function() { //do numerous enclosed myclass vars } this.foo = function() { alert('hello'); //don't enclosed variables. } } each instance of myclass gets own copy of bar , foo, why prototyped methods use less memory. however, know more memory use of inner methods. it seems obvious me (1) below must true. agree? not distinct myclass instances have own distinct copies of bar, must have own distinct copies of myclass enclosure. (or else how bar method keep myclass variables straight on per instance basis?) now (2) question i'm after. since inner foo method doesn't use in myclass enclosure natural qustion is: javascript smart enough not keep myclass enclosure in memory use of foo? this depend entirely on implementation not language. naive implementations keep far more around others. answer true e.g. v8 (chrome's js engine) may not true spider...
Read more

php - Replacing tags in braces, even nested tags, with regex -

-
example preg_replace('/\{[a-za-z.,\(\)0-9]+\}/', 'replaced', 'lorem ipsum dolor sit {tag1({tag2()})}, consectetur adipiscing elit.'); the result: lorem ipsum dolor sit {tag1(replaced)}, consectetur adipiscing elit. question as can see "tag2" has been replaced, want replace "tag1" know how can this? (in cases might this: {tag1({tag2({tag3()})})}) , on.) btw using preg_replace_callback, easier show preg_replace here site can test code: http://www.spaweditor.com/scripts/regex/index.php you need add curly braces character set. here's pattern used: /\{[a-za-z.,\(\)\{\}0-9]+\}/ and here result: "lorem ipsum dolor sit replaced, consectetur adipiscing elit."
Read more