python - How to update records in SQL Alchemy in a Loop -

-

i trying use sqlsoup - sqlalchemy extention, update records in sql server 2008 database. using pyobdc connections. there number of issues make hard find relevant example.

i reprojection geometry field in large table (2 million + records), many of standard ways of updating fields cannot used. need extract coordinates geometry field text, convert them , pass them in. fine, , individual pieces working.

however want execute sql update statement on each row, while looping through records 1 one. assume places locks on recordset, or connection in use - if use code below hangs after updating first record.

any advice on how create new connection, reuse existing one, or accomplish way appreciated. 

s = select([text("%s fid" % id_field),             text("%s.stastext() wkt" % geom_field)],            from_obj=[feature_table])  rs = s.execute()  row in rs:     new_wkt = reprojectfeature(row.wkt)      update_value = "geometry :: stgeomfromtext('%s',%s)" % (new_wkt, "3785")     update_sql = ("update %s set geom3785 = %s %s = %i" %                   (full_name, update_value, id_field, row.fid))      conn = db.connection()     conn.execute(update_sql)     conn.close() #or not - no effect..

updated working code looks this. works fine on few records, hangs on whole table, guess reading in data. 

db = sqlsoup(conn_string) #create outer query  session = sessionmaker(autoflush=false, bind=db.engine) session = session() rs = session.execute(s)  row in rs:      #create update sql...     session.execute(update_sql) session.commit()

i connection busy errors.

dbapierror: (error) ('hy000', '[hy000] [microsoft][odbc sql server driver]connection busy results hstmt (0) (sqlexecdirectw)')

it looks problem odbc driver - http://sourceitsoftware.blogspot.com/2008/06/connection-is-busy-with-results-for.html

further update:

on server using profiler, shows select statement first update statement "starting" neither complete. if set select statement return top 10 rows, complete , updates run. 

sql: batch starting   select... sql: batch starting   update...

i believe issue pyodbc , sql server drivers. if remove sql alchemy , execute same sql pyodbc hangs. if create new connection object updates.

i tried sql server native client 10.0 driver meant allow mars - multiple active record sets made no difference. in end have resorted "paging results" , updating these batches using pyodbc , sql (see below), thought sqlalchemy have been able me automatically.

try using session.

rs = s.execute() becomes session.execute(rs) , can replace last 3 lines session.execute(update_sql). i'd suggest configuring session autocommit off , call session.commit() @ end.


Comments

Post a Comment

Popular posts from this blog

javascript - Enclosure Memory Copies -

-
function myclass() { //lots , lots of vars , code here. this.bar = function() { //do numerous enclosed myclass vars } this.foo = function() { alert('hello'); //don't enclosed variables. } } each instance of myclass gets own copy of bar , foo, why prototyped methods use less memory. however, know more memory use of inner methods. it seems obvious me (1) below must true. agree? not distinct myclass instances have own distinct copies of bar, must have own distinct copies of myclass enclosure. (or else how bar method keep myclass variables straight on per instance basis?) now (2) question i'm after. since inner foo method doesn't use in myclass enclosure natural qustion is: javascript smart enough not keep myclass enclosure in memory use of foo? this depend entirely on implementation not language. naive implementations keep far more around others. answer true e.g. v8 (chrome's js engine) may not true spider...
Read more

php - Replacing tags in braces, even nested tags, with regex -

-
example preg_replace('/\{[a-za-z.,\(\)0-9]+\}/', 'replaced', 'lorem ipsum dolor sit {tag1({tag2()})}, consectetur adipiscing elit.'); the result: lorem ipsum dolor sit {tag1(replaced)}, consectetur adipiscing elit. question as can see "tag2" has been replaced, want replace "tag1" know how can this? (in cases might this: {tag1({tag2({tag3()})})}) , on.) btw using preg_replace_callback, easier show preg_replace here site can test code: http://www.spaweditor.com/scripts/regex/index.php you need add curly braces character set. here's pattern used: /\{[a-za-z.,\(\)\{\}0-9]+\}/ and here result: "lorem ipsum dolor sit replaced, consectetur adipiscing elit."
Read more