Update table rows in Postgresql taking too much time - postgresql

I am trying to update around 6000 rows in a table but my query never finishes.
I have put the data to be updated in a temp table and using a join to update rows.
This was working pretty fast in Sql Server but in Postgresql it never finishes.
I am updating around 40 columns.
Here is the sql I am running.
UPDATE "STG_magento_de".sales_flat_order
SET customer_id = b.customer_id
,created_at = b.created_at
,updated_at = b.updated_at
,coupon_code = b.coupon_code
,box_id = b.box_id
,beautytrends_glossydots = b.beautytrends_glossydots
,billing_address_id = b.billing_address_id
,shipping_address_id = b.shipping_address_id
,base_discount_amount = b.base_discount_amount
,base_discount_canceled = b.base_discount_canceled
,base_discount_invoiced = b.base_discount_invoiced
,base_discount_refunded = b.base_discount_refunded
,base_grand_total = b.base_grand_total
,base_shipping_amount = b.base_shipping_amount
,base_shipping_canceled = b.base_shipping_canceled
,base_shipping_invoiced = b.base_shipping_invoiced
,base_shipping_refunded = b.base_shipping_refunded
,base_shipping_tax_amount = b.base_shipping_tax_amount
,base_shipping_tax_refunded = b.base_shipping_tax_refunded
,base_subtotal = b.base_subtotal
,base_subtotal_canceled = b.base_subtotal_canceled
,base_subtotal_invoiced = b.base_subtotal_invoiced
,base_tax_amount = b.base_tax_amount
,base_tax_canceled = b.base_tax_canceled
,base_tax_invoiced = b.base_tax_invoiced
,base_tax_refunded = b.base_tax_refunded
,base_to_global_rate = b.base_to_global_rate
,base_to_order_rate = b.base_to_order_rate
,base_total_canceled = b.base_total_canceled
,base_total_invoiced = b.base_total_invoiced
,base_total_invoiced_cost = b.base_total_invoiced_cost
,base_total_offline_refunded = b.base_total_offline_refunded
,base_total_online_refunded = b.base_total_online_refunded
,base_total_paid = b.base_total_paid
,base_total_qty_ordered = b.base_total_qty_ordered
,base_total_refunded = b.base_total_refunded
,increment_id = b.increment_id
,order_type = b.order_type
,STATUS = b.STATUS
,is_chargerun = b.is_chargerun
,chargeback_flag = b.chargeback_flag
,gift_message_id = b.gift_message_id
,dispatch = b.dispatch
FROM "STG_magento_de".sales_flat_order a
JOIN "STG_magento_de".sales_flat_order_temp b ON a.entity_id = b.entity_id

From the manual:
Note that the target table must not appear in the from_list, unless you intend a self-join (in which case it must appear with an alias in the from_list).
(emphasis mine)
So you actually want:
UPDATE "STG_magento_de".sales_flat_order
SET customer_id = b.customer_id,
....
from sales_flat_order_temp b --<< do NOT repeat the target table here
where "STG_magento_de".sales_flat_order = b.entity_id`
Unrelated, but: you should really avoid those dreaded quoted identifiers. They are much more trouble than they are worth.

Related

Extract fields from Structure Array to put into another Structure Array

I have a structure array with a large number of fields that I don't care about, so I want to extract the limited number of fields I DO care about and put it into a separate structure array.
For a structure array of size one, I've done this by creating the new array from scratch, for example:
structOld.a = 1;
structOld.b = 2;
structOld.usefulA = 'useful information';
structOld.usefulB = 'more useful information';
structOld.c = 3;
structOld.d = 'words';
keepFields = {'usefulA','usefulB'};
structNew = struct;
for fn = keepFields
structNew.(fn{:}) = structOld.(fn{:});
end
which gives
structNew =
usefulA: 'useful information'
usefulB: 'more useful information'
Is there a more efficient way of doing this? How can I scale up to an structure array (vector) of size N?
N = 50;
structOld(1).a = 1;
structOld(1).b = 2;
structOld(1).usefulA = 500;
structOld(1).usefulB = 'us';
structOld(1).c = 3;
structOld(1).d = 'ef';
structOld(2).a = 4;
structOld(2).b = 5;
structOld(2).usefulA = 501;
structOld(2).usefulB = 'ul';
structOld(2).c = 6;
structOld(2).d = 'in';
structOld(3).a = 7;
structOld(3).b = '8';
structOld(3).usefulA = 504;
structOld(3).usefulB = 'fo';
structOld(3).c = 9;
structOld(3).d = 'rm';
structOld(N).a = 10;
structOld(N).b = 11;
structOld(N).usefulA = 506;
structOld(N).usefulB = 'at';
structOld(N).c = 12;
structOld(N).d = 'ion';
In this case, I'd like to end up with:
structNew =
1x50 struct array with fields:
usefulA
usefulB
Keeping elements with empty usefulA/usefulB fields is fine; I can get rid of them later if needed.
Using rmfield isn't great because the number of useless fields far outnumbers the useful fields.
You can create a new struct array using existing data as follows:
structNew = struct('usefulA',{structOld.usefulA},'usefulB',{structOld.usefulB});
If you have an arbitrary set of field names that you want to preserve, you could use a loop as follows. Here, I'm first extracting the data from strcutOld into a cell array data, which contains each of the arguments the the struct call in the previous line of code. data{:} is now a comma-separated list of these arguments, the last line of code below is identical to the line above.
keepFields = {'usefulA','usefulB'};
data = cell(2,numel(keepFields));
for ii=1:numel(keepFields)
data{1,ii} = keepFields{ii};
data{2,ii} = {structOld.(keepFields{ii})};
end
structNew = struct(data{:});

Looping through columns in Postgresql

I want to execute the following query 3 times for 3 different columns
(pc = t1.postal_code, t1.postal_code_2, t1.postal_code_3).
update addresses t1 set
admin_name1 = t.admin_name1,
admin_name2 = t.admin_name2,
admin_name3 = t.admin_name3,
admin_name4 = t.admin_name4,
from (
select * from all_countries) t
where t.postal_code = pc and t1.country_ = t.country ;
Is there are way of writing the code only once, e.g. by means of a loop, and how can I implement it?

PowerShell accessing data from hashtable inside hashtable

I have 5 hash tables:
$Monday = #{RUG = "";NRH1 = "";NRH2 = "";ELM = "";BAGVAGT = ""}
$Tuesday = #{RUG = "";NRH1 = "";NRH2 = "";ELM = "";BAGVAGT = ""}
$Wednesday = #{NRH1 = "";NRH2 = "";ELM = "";BAGVAGT = ""}
$Thursday = #{NRH1 = "";NRH2 = "";ELM = "";BAGVAGT = ""}
$Friday = #{NRH1 = "";NRH2 = "";ELM = ""}
That get filled with data. And I can get data out from these either one at at time with $Monday.RUG or all with $Monday | out-string. No problem there.
I'm going to combine those in another hash table 100 times with different data.
So it will be like this:
$Week = #{
1 = #{mo=$Monday;tu=$Tuesday;we=$Wednesday;th=$Thursday;fr=$Friday;val=$value}
2 = #{mo=$Monday;tu=$Tuesday;we=$Wednesday;th=$Thursday;fr=$Friday;val=$value}
}
And so on, until I have 100 different weeks with different values (value will be a calculated number)
But the question is. How do I access the items in the hashtables inside the $week hashtable?
Is there a direct way like $week.1.mo ? or do you need to use a loop?
You can access it using:
$Week[1].mo

matlab - combine all the output using celldisp

Below is my code.. I'm trying to display each of the seq using celldisp..but it comes out one by one..
%mycode
seq1 = {'acc';'gct';'tta';'ccc';'ttc';'aaa';'ttg';'gta';'gtg';'act'};
seq2 = {'act';'gcc';'tta';'cca';'aac';'aaa';'ttg';'gta';'gtg';'acc'};
seq3 = {'ttc';'tgc';'atg';'ggc';'ccg';'aat';'aag';'ggt';'tga';'agt'};
seq4 = {'cag';'ccg';'ttg';'caa';'aat';'agc';'ttg';'ggg';'gct';'att'};
seq5 = {'ccc';'ggg';'tta';'cag';'aac';'aaa';'ttg';'gta';'gac';'acc'};
seq6 = {'acc';'gct';'ata';'ccc';'ttc';'taa';'ttg';'gtc';'gtg';'acc'};
for mutseq = {seq2,seq3,seq4,seq5,seq6}
a = strcmp(seq1,mutseq);
celldisp(mutseq);
end
%endcode
the output is :
mutseq{1}{1} =
act
mutseq{1}{2} =
gcc
mutseq{1}{3} =
tta
mutseq{1}{4} =
cca
mutseq{1}{5} =
aac
mutseq{1}{6} =
aaa
mutseq{1}{7} =
ttg
mutseq{1}{8} =
gta
mutseq{1}{9} =
gtg
mutseq{1}{10} =
%and it repeat for next seq
did anyone know how to combine it into one?

Converting classic ASP ADODB script to TSQL

Hi i inherited a load of old classic ASP code that makes some updates to some tables in what i think was old MS access db.
The databases have now been converted to SQL and work OK, however i have a need to convert some old ASP code to the equivalent TSQL. TSQL is NOT my strong point and would appreciate some help converting the vb script to the equivalent TSQL for a stored procedure called 'UpdateCircuitOrdersComments' its not the basic syntax i'm struggling with, its more what can be done IE FOR's IF's Cases, loops etc in order to achieve the below in sql.
I know the code below is not great or could be done far better, but that's what i inherited sorry.
Every field is available to me via c# parameters passed to the sproc using ajax apart from the "Contract Period"
Looking forward to learning from any sound advice from you guys...
The script is below:
Dim connect2, Class2, Query2, Counter
Dim indate1, indate2, indate3, aday1, amonth1, ayear1, aday2, amonth2, ayear2, aday3, amonth3, ayear3, length, maintrate, equiprate, stqrrate, startserial, liveserial, endserial
Dim splitArray
Set Connect = Server.CreateObject("ADODB.Connection")
Connect.Open QuotebaseDB
Set Class1 = Server.CreateObject("ADODB.Recordset")
Query = "SELECT * FROM circuits WHERE ID=" & Request("ID")
Class1.Open Query,Connect,adOpenDynamic,adLockOptimistic
if Class1("Contract Period") <> "" Then
length = Class1("Contract Period")
splitArray = split(length, " ")
length = CInt(splitArray(0))
else
length = 1
end if
Class1("actual live date") = Request("actuallivedate")
Class1("lastupdater") = Session("username")
Class1("lastupdate") = Date()
Class1("Contract Period") = length
Class1.Update
'=========================================
' Add Rate Information - Based On Actuals
'=========================================
indate1 = Request("actuallivedate")
indate2 = Request("actuallivedate")
indate3 = Request("actuallivedate")
aday1 = Left(indate1, 2)
amonth1 = Mid(indate1, 4, 2)
ayear1 = Right(indate1, 2)
aday2 = Left(indate2, 2)
amonth2 = Mid(indate2, 4, 2)
ayear2 = Right(indate2, 2)
aday3 = Left(indate3, 2)
amonth3 = Mid(indate3, 4, 2)
ayear3 = Right(indate3, 2) + length
startserial = dateserial(ayear1, amonth1, aday1)
liveserial = dateserial(ayear2, amonth2, aday2)
endserial = dateserial(ayear3, amonth3, aday3)
'========================================================
' Check that current maintenance rate will be superseded
'========================================================
Dim MaintConnect1, MaintRS1, MaintQuery1
Set MaintConnect1 = Server.CreateObject("ADODB.Connection")
MaintConnect1.Open QuotebaseDB
Set MaintRS1 = Server.CreateObject("ADODB.Recordset")
MaintQuery1 = "SELECT * FROM maintenancetable WHERE CircuitID=" & Request("ID")
MaintRS1.Open MaintQuery1,MaintConnect1,adOpenDynamic,adLockOptimistic
Do while not MaintRS1.eof
If (MaintRS1("startdate") < startserial) OR (MaintRS1("enddate") > endserial) Then
MaintRS1("startdate") = StartSerial
MaintRS1("enddate") = EndSerial
MaintRS1("circuitnum") = Class1("circuit number")
MaintRS1("modifieddate") = now
MaintRS1.Update
End If
MaintRS1.movenext
Loop
MaintRS1.close
set MaintRS1 = nothing
MaintConnect1.close
set MaintConnect1 = nothing
'========================================================
' Check that current equipment rate will be superseded
'========================================================
Dim EquipConnect1, EquipRS1, EquipQuery1
Set EquipConnect1 = Server.CreateObject("ADODB.Connection")
EquipConnect1.Open QuotebaseDB
Set EquipRS1 = Server.CreateObject("ADODB.Recordset")
EquipQuery1 = "SELECT * FROM [equipment table] WHERE CircuitID=" & Request("ID")
EquipRS1.Open EquipQuery1,EquipConnect1,adOpenDynamic,adLockOptimistic
Do while not EquipRS1.eof
If (EquipRS1("startdate") < startserial) OR (EquipRS1("enddate") > endserial) Then
EquipRS1("startdate") = StartSerial
EquipRS1("enddate") = EndSerial
EquipRS1("circuitnum") = Class1("circuit number")
EquipRS1("modifieddate") = now
EquipRS1.Update
End If
EquipRS1.movenext
Loop
EquipRS1.close
set EquipRS1 = nothing
EquipConnect1.close
set EquipConnect1 = nothing
'========================================================
' Check that current rental rate will be superseded
'========================================================
Dim STQRConnect1, STQRRS1, STQRQuery1
Set STQRConnect1 = Server.CreateObject("ADODB.Connection")
STQRConnect1.Open QuotebaseDB
Set STQRRS1 = Server.CreateObject("ADODB.Recordset")
STQRQuery1 = "SELECT * FROM STQRtable WHERE CircuitID=" & Request("ID")
STQRRS1.Open STQRQuery1,STQRConnect1,adOpenDynamic,adLockOptimistic
Do while not STQRRS1.eof
If (STQRRS1("startdate") < startserial) OR (STQRRS1("enddate") > endserial) Then
STQRRS1("startdate") = StartSerial
STQRRS1("enddate") = EndSerial
STQRRS1("circuitnum") = Class1("circuit number")
STQRRS1("modifieddate") = now
STQRRS1.Update
End If
STQRRS1.movenext
Loop
STQRRS1.close
set STQRRS1 = nothing
STQRConnect1.close
set STQRConnect1 = nothing
'===========================================
' Update Connection Charge As A One Off Charge
'===========================================
Dim OneConnect, OneRS, OneQuery
Set OneConnect = Server.CreateObject("ADODB.Connection")
OneConnect.Open QuotebaseDB
Set OneRS = Server.CreateObject("ADODB.Recordset")
OneQuery = "SELECT * FROM OneOffCharges WHERE chargetype = 'Connection Charge' and CircuitID=" & Request("ID")
OneRS.Open OneQuery,OneConnect,adOpenDynamic,adLockOptimistic
If not oners.eof Then
OneRS("chargedate") = startserial
OneRS("modifieddate") = now
OneRS("chargetype") = "Connection Charge"
OneRS.Update
End If
OneRS.close
set OneRS = nothing
OneConnect.close
set OneConnect = nothing
Class1.close
set Class1 = nothing
Connect.close
set Connect = nothing
The loops don't appear to do much other then iterate over the record set. It is unlikely that you will need to do this in T-SQL as you can probably deduce everything from queries.
Most of it just looks like update statements
-- Check that current maintenance rate will be superseded
UPDATE maintenancetable
SET StartDate = #startSerial,
EndDate = #endSerial,
CircuitNum = #circuitNumber,
ModifiedDate = CURRENT_TIMESTAMP
WHERE CircuitID=#circuitId
AND (Startdate < #startSerial OR EndDate > #endSerial)
The contract period is a bit confusing it seems to be getting the first item of an array so maybe this is the equivalent of
SELECT TOP 1 ContractPeriod FROM circuits WHERE ID=#id
-- ? ORDER BY ContractPeriod ?