Merge rows (same values) in PostgreSQL based on row difference - postgresql

I have a table my_tbl in my PostgreSQL 9.5 (x64 Windows) database, which contains data as shown below.
grp id low high avg
1 7 292 322 18.8
1 8 322 352 18.8
1 9 352 22 18.8
1 10 22 52 18.8
1 11 52 82 18.8
1 12 82 112 18.8
4 1 97 127 19.0
4 2 127 157 11.4
4 3 157 187 11.4
4 4 187 217 19.6
4 5 217 247 19.6
4 6 247 277 19.6
4 10 7 37 19.5
4 11 37 67 19.5
4 12 67 97 19.5
6 6 182 212 0.0
6 7 212 242 0.0
6 8 242 272 0.0
6 9 272 302 21.4
6 10 302 332 21.4
6 11 332 2 0.0
6 12 2 32 0.0
7 5 275 305 0.0
7 6 305 335 0.0
7 7 335 5 0.0
7 8 5 35 0.0
7 9 35 65 21.2
7 10 65 95 21.2
7 11 95 125 21.2
7 12 125 155 21.2
Now I would like to merge rows in the above data in the following way. For each grp, if the difference between avg values (the preceding and the succeeding ones) is zero (same value), then all such rows should be merged with the low value of the first row to high value of the last row (where merging should stop).
My expected output is:
grp id low high avg
1 {7,8,9,10,11,12} 292 112 18.8
4 {1} 97 127 19.0
4 {2,3} 127 187 11.4
4 {4,5,6} 187 277 19.6
4 {10,11,12} 7 97 19.5
6 {6,7,8} 182 272 0.0
6 {9,10} 272 332 21.4
6 {11,12} 332 32 0.0
7 {5,6,7,8} 275 35 0.0
7 {9,10,11,12} 35 155 21.2
Does someone care to help or suggest on how this could be achieved using SQL/PLPGSQL?

You can use ARRAY_AGG function and FIRST_VALUE function:
SELECT
grp, ARRAY_AGG(id) AS id, low, high, avg
FROM (
SELECT
grp,
id,
FIRST_VALUE(low) OVER (PARTITION BY grp, avg ORDER BY id) AS low,
FIRST_VALUE(high) OVER (PARTITION BY grp, avg ORDER BY id DESC) AS high,
avg
FROM my_tbl ORDER BY id
) t
GROUP BY grp, avg , low, high
ORDER BY grp;
DB Fiddle

Related

How to apply an "interface" method to a set of rows in kdb?

Sorry if this is a newbie question again.
I am trying to replicate the functionality of interfaces as seen in c++, rust etc. in kdb as is shown in a simple demonstration below:
q).iface.a.fun:{x*y+z}
q).iface.b.fun:{x*x+y+z}
q)ifaces:`a`b; // for demonstration purposes
q)tab:([]time:`datetime$();kind:`ifaces$();x:`long$();y:`long$();z:`long$());
q)n:10;
q)tab,:flip(n#.z.z;n?ifaces;n?10;n?10;n?10)
Now you would assume that the kind would be able to reference the `a`b fun methods of the iface interface as follows:
q)?[`tab;();0b;`max`ifaceval!((max;`x);(`.iface;`kind;`fun;`x;`y;`z))]
evaluation error:
fun
[0] ?[`tab;();0b;`max`ifaceval!((max;`x);(`.iface;`kind;`fun;`x;`y;`z))]
^
Obviously the functional nature of the select inhibits referencing the fun method on account of the symbol type field declarations.
You can avert this error by using enlist as follows:
q)?[`tab;();0b;`max`ifaceval!((max;`x);(`.iface;`kind;enlist`fun;`x;`y;`z))]
max ifaceval ..
-----------------------------------------------------------------------------..
9 77 154 95 65 0 128 153 126 60 49 77 154 95 65 0 128 153 126 60 49 77 154 ..
However this duplicates the result of fun for each row.
How might one effectively go about this without getting the above malformed responses?
Thanks again.
Selecting ifaceval first will ensure each row is returned. max x is a scalar, which forces all the ifaceval entries into one row. The scalar will be expanded across all rows if a vector column precedes it.
q)?[`tab;();0b;`ifaceval`max!((`.iface;`kind;enlist`fun;`x;`y;`z);(max;`x))]
ifaceval max
-------------------------------------
160 11 126 28 32 60 76 10 112 168 8
96 10 77 24 16 35 60 6 63 104 8
96 10 77 24 16 35 60 6 63 104 8
96 10 77 24 16 35 60 6 63 104 8
96 10 77 24 16 35 60 6 63 104 8
160 11 126 28 32 60 76 10 112 168 8
96 10 77 24 16 35 60 6 63 104 8
160 11 126 28 32 60 76 10 112 168 8
160 11 126 28 32 60 76 10 112 168 8
160 11 126 28 32 60 76 10 112 168 8
I'm not sure if this is exactly what you're looking for though. If you want to calculate ifaceval for each row in the table, this should work.
q)?[tab;();0b;`ifaceval`max!(((';(`.iface;::;enlist`fun));`kind;`x;`y;`z);(max;`x))]
ifaceval max
------------
160 8
10 8
77 8
24 8
16 8
60 8
60 8
10 8
112 8
168 8
One point to make is that it's probably best to avoid using kdb keywords for column names. Although it works in functional queries, it does not for qSQL ones.
q)select max:max x from tab
'assign
[0] select max:max x from tab
^

Unity: Unable to convert classes into dex format Plugins: Unity IAP 1.16 Facebook SDK 7.13.0 [duplicate]

This question already has answers here:
Too many field references: 70613; max is 65536
(2 answers)
Closed 4 years ago.
got a problem with Facebook SDK. When i installed this i got error: Unable to convert classes into dex format.
There are installed plagins: Unity IAP 1.16, Facebook SDK 7.13.0, Google Play Service 0.9.50, Appodeal 2.8.45.
Unity 2017.1.2p3.
jdk1.8.0_181 and latest version of sdk tools
No same files or files with diffrent version there.
Here is the link for pastebin with error
CommandInvokationFailure: Unable to convert classes into dex format.
C:/Program Files/Java/jdk1.8.0_181\bin\java.exe -Xmx2048M -Dcom.android.sdkmanager.toolsdir="C:/Program Files (x86)/Android\tools" -Dfile.encoding=UTF8 -jar "C:\Program Files\Unity\Editor\Data\PlaybackEngines\AndroidPlayer/Tools\sdktools.jar" -
stderr[
trouble writing output: Too many field references to fit in one dex file: 86260; max is 65536.
You may try using multi-dex. If multi-dex is enabled then the list of classes for the main dex list is too large.
References by package:
3 android.accounts
30 android.app
1 android.bluetooth
2 android.content
60 android.content.pm
10 android.content.res
3 android.database
49 android.graphics
2 android.graphics.drawable
4 android.hardware
2 android.media
1 android.media.browse
4 android.net
6 android.net.wifi
28 android.os
3 android.print
4 android.provider
13 android.support.annotation
1546 android.support.compat
1546 android.support.coreui
1546 android.support.coreutils
1648 android.support.customtabs
1546 android.support.fragment
1705 android.support.graphics.drawable
1546 android.support.graphics.drawable.animated
1546 android.support.mediacompat
1546 android.support.v4
13 android.support.v4.accessibilityservice
17 android.support.v4.animation
885 android.support.v4.app
114 android.support.v4.content
1 android.support.v4.content.pm
1 android.support.v4.content.res
11 android.support.v4.graphics
41 android.support.v4.graphics.drawable
4 android.support.v4.hardware.display
12 android.support.v4.hardware.fingerprint
10 android.support.v4.internal.view
388 android.support.v4.media
379 android.support.v4.media.session
10 android.support.v4.net
19 android.support.v4.os
63 android.support.v4.print
10 android.support.v4.provider
55 android.support.v4.text
6 android.support.v4.text.util
118 android.support.v4.util
434 android.support.v4.view
138 android.support.v4.view.accessibility
8 android.support.v4.view.animation
495 android.support.v4.widget
409 android.support.v7.app
1546 android.support.v7.appcompat
1546 android.support.v7.cardview
8 android.support.v7.content.res
18 android.support.v7.graphics.drawable
1 android.support.v7.text
2 android.support.v7.transition
77 android.support.v7.view
249 android.support.v7.view.menu
856 android.support.v7.widget
6 android.text
1 android.text.util
16 android.util
22 android.view
8 android.view.accessibility
3 android.webkit
23 android.widget
2 bitter.jnibridge
158 bolts
1540 com.CODvinn.GameOfWords
1540 com.adcolony.adcolonysdk
1270 com.adcolony.sdk
1616 com.amazon.device.ads
1560 com.android.vending.billing
4 com.applovin.sdk
1037 com.appodeal.ads
128 com.appodeal.ads.a
129 com.appodeal.ads.b
80 com.appodeal.ads.c
98 com.appodeal.ads.d
174 com.appodeal.ads.e
58 com.appodeal.ads.f
146 com.appodeal.ads.g
150 com.appodeal.ads.native_ad
107 com.appodeal.ads.native_ad.views
49 com.appodeal.ads.networks
68 com.appodeal.ads.networks.a
45 com.appodeal.ads.networks.vpaid
177 com.appodeal.ads.utils
19 com.appodeal.ads.utils.a
47 com.appodeal.ads.utils.b
4 com.appodeal.ads.utils.c
1540 com.appodeal.appodeal.unity
1540 com.appodeal.inmobi.unity
1540 com.appodeal.ogury.unity
1 com.appodeal.sdk
1540 com.appodeal.startapp.unity
37 com.appodeal.unity
1540 com.appodeal.yandexmetrica.unity
31 com.appodealx.applovin
28 com.appodealx.mraid
66 com.appodealx.sdk
17 com.appodealx.vast
2 com.chartboost.sdk.Libraries
180 com.evernote.android.job
5 com.evernote.android.job.gcm
10 com.evernote.android.job.util
16 com.evernote.android.job.util.support
16 com.evernote.android.job.v14
2 com.evernote.android.job.v19
12 com.evernote.android.job.v21
3 com.evernote.android.job.v24
2 com.evernote.android.job.v26
3 com.facebook.ads
1540 com.google.android.gms
20 com.google.android.gms.actions
4 com.google.android.gms.ads
21 com.google.android.gms.ads.identifier
1594 com.google.android.gms.auth
5 com.google.android.gms.auth.account
1563 com.google.android.gms.auth.api
68 com.google.android.gms.auth.api.accounttransfer
97 com.google.android.gms.auth.api.credentials
6 com.google.android.gms.auth.api.phone
1540 com.google.android.gms.auth.api.phone.license
37 com.google.android.gms.auth.api.proxy
59 com.google.android.gms.auth.api.signin
34 com.google.android.gms.auth.api.signin.internal
1540 com.google.android.gms.auth.license
1540 com.google.android.gms.base
1540 com.google.android.gms.base.license
102 com.google.android.gms.common
98 com.google.android.gms.common.api
338 com.google.android.gms.common.api.internal
41 com.google.android.gms.common.data
44 com.google.android.gms.common.images
160 com.google.android.gms.common.internal
1540 com.google.android.gms.common.license
35 com.google.android.gms.common.stats
27 com.google.android.gms.common.util
1688 com.google.android.gms.drive
54 com.google.android.gms.drive.events
1540 com.google.android.gms.drive.license
10 com.google.android.gms.drive.metadata
20 com.google.android.gms.drive.metadata.internal
39 com.google.android.gms.drive.query
49 com.google.android.gms.drive.query.internal
8 com.google.android.gms.drive.widget
25 com.google.android.gms.dynamic
20 com.google.android.gms.dynamite
2 com.google.android.gms.dynamite.descriptors.com.google.android.gms.flags
16 com.google.android.gms.flags.impl
1915 com.google.android.gms.games
25 com.google.android.gms.games.achievement
13 com.google.android.gms.games.event
155 com.google.android.gms.games.internal
128 com.google.android.gms.games.internal.api
12 com.google.android.gms.games.internal.experience
46 com.google.android.gms.games.internal.player
61 com.google.android.gms.games.leaderboard
1540 com.google.android.gms.games.license
61 com.google.android.gms.games.multiplayer
50 com.google.android.gms.games.multiplayer.realtime
49 com.google.android.gms.games.multiplayer.turnbased
56 com.google.android.gms.games.quest
34 com.google.android.gms.games.request
46 com.google.android.gms.games.snapshot
15 com.google.android.gms.games.stats
40 com.google.android.gms.games.video
3 com.google.android.gms.iid
1518 com.google.android.gms.internal
5 com.google.android.gms.location.places
1547 com.google.android.gms.nearby
72 com.google.android.gms.nearby.connection
1540 com.google.android.gms.nearby.license
95 com.google.android.gms.nearby.messages
2 com.google.android.gms.nearby.messages.audio
163 com.google.android.gms.nearby.messages.internal
6 com.google.android.gms.security
1588 com.google.android.gms.tasks
1540 com.google.android.gms.tasks.license
1540 com.google.example.games.mainlibproj
1581 com.google.games.bridge
15 com.inmobi.a
1076 com.inmobi.ads
70 com.inmobi.ads.cache
7 com.inmobi.commons.a
39 com.inmobi.commons.core.a
24 com.inmobi.commons.core.b
13 com.inmobi.commons.core.c
62 com.inmobi.commons.core.configs
10 com.inmobi.commons.core.d
43 com.inmobi.commons.core.e
12 com.inmobi.commons.core.f
66 com.inmobi.commons.core.network
29 com.inmobi.commons.core.utilities
6 com.inmobi.commons.core.utilities.a
37 com.inmobi.commons.core.utilities.b
10 com.inmobi.commons.core.utilities.uid
160 com.inmobi.rendering
53 com.inmobi.rendering.a
83 com.inmobi.rendering.mraid
29 com.inmobi.sdk
104 com.inmobi.signals
10 com.inmobi.signals.a
13 com.inmobi.signals.activityrecognition
14 com.inmobi.signals.b
53 com.integralads.avid.library.adcolony
2 com.integralads.avid.library.adcolony.activity
2 com.integralads.avid.library.adcolony.base
4 com.integralads.avid.library.adcolony.processing
5 com.integralads.avid.library.adcolony.registration
3 com.integralads.avid.library.adcolony.session
45 com.integralads.avid.library.adcolony.session.internal
19 com.integralads.avid.library.adcolony.session.internal.jsbridge
12 com.integralads.avid.library.adcolony.session.internal.trackingwebview
14 com.integralads.avid.library.adcolony.utils
26 com.integralads.avid.library.adcolony.video
14 com.integralads.avid.library.adcolony.walking
21 com.integralads.avid.library.adcolony.walking.async
1 com.integralads.avid.library.adcolony.weakreference
53 com.integralads.avid.library.inmobi
2 com.integralads.avid.library.inmobi.activity
2 com.integralads.avid.library.inmobi.base
4 com.integralads.avid.library.inmobi.processing
5 com.integralads.avid.library.inmobi.registration
3 com.integralads.avid.library.inmobi.session
45 com.integralads.avid.library.inmobi.session.internal
19 com.integralads.avid.library.inmobi.session.internal.jsbridge
12 com.integralads.avid.library.inmobi.session.internal.trackingwebview
14 com.integralads.avid.library.inmobi.utils
26 com.integralads.avid.library.inmobi.video
14 com.integralads.avid.library.inmobi.walking
21 com.integralads.avid.library.inmobi.walking.async
1 com.integralads.avid.library.inmobi.weakreference
2 com.ironsource.mediationsdk
218 com.moat.analytics.mobile.inm
2 com.moat.analytics.mobile.inm.a.b
1 com.mobvista.msdk
1 com.squareup.okhttp
345 com.squareup.picasso
6 com.startapp.a.a.a
4 com.startapp.a.a.b
32 com.startapp.a.a.c
7 com.startapp.a.a.d
5 com.startapp.a.a.e
3 com.startapp.a.a.f
13 com.startapp.a.a.g
3 com.startapp.android.publish
6 com.startapp.android.publish.a
45 com.startapp.android.publish.ads.a
9 com.startapp.android.publish.ads.b
55 com.startapp.android.publish.ads.banner
87 com.startapp.android.publish.ads.banner.banner3d
35 com.startapp.android.publish.ads.banner.bannerstandard
2 com.startapp.android.publish.ads.c.a
5 com.startapp.android.publish.ads.c.b
111 com.startapp.android.publish.ads.list3d
49 com.startapp.android.publish.ads.nativead
123 com.startapp.android.publish.ads.splash
146 com.startapp.android.publish.ads.video
12 com.startapp.android.publish.ads.video.a
31 com.startapp.android.publish.ads.video.b
48 com.startapp.android.publish.ads.video.tracking
338 com.startapp.android.publish.adsCommon
31 com.startapp.android.publish.adsCommon.Utils
25 com.startapp.android.publish.adsCommon.a
7 com.startapp.android.publish.adsCommon.activities
26 com.startapp.android.publish.adsCommon.adListeners
90 com.startapp.android.publish.adsCommon.adinformation
11 com.startapp.android.publish.adsCommon.b
8 com.startapp.android.publish.adsCommon.c
10 com.startapp.android.publish.adsCommon.d
48 com.startapp.android.publish.adsCommon.e
13 com.startapp.android.publish.adsCommon.f
11 com.startapp.android.publish.adsCommon.g
105 com.startapp.android.publish.cache
163 com.startapp.android.publish.common.metaData
121 com.startapp.android.publish.common.model
17 com.startapp.android.publish.html
40 com.startapp.android.publish.inappbrowser
42 com.startapp.common
54 com.startapp.common.a
3 com.startapp.common.b
3 com.startapp.common.c
1546 com.unity.purchasing
34 com.unity.purchasing.common
1724 com.unity.purchasing.googleplay
1 com.unity3d.ads
1 com.unity3d.ads2
203 com.unity3d.player
2 com.vungle.warren
212 com.yandex.metrica
455 com.yandex.metrica.impl
18 com.yandex.metrica.impl.interact
924 com.yandex.metrica.impl.ob
29 com.yandex.metrica.impl.utils
3 com.yandex.mobile.ads
364 for
21 for.do
26 for.do.byte
8 for.do.case
12 for.do.char
46 for.do.do
23 for.do.for
43 for.do.if
26 for.do.int
177 for.do.new
1 for.do.try
63 if.do.do
3 if.do.do.for
82 if.do.do.if
111 if.do.do.if.do
39 if.do.do.int
71 int
135 io.presage
47 io.presage.actions
8 io.presage.actions.do
6 io.presage.activities
24 io.presage.activities.do
12 io.presage.activities.handlers
73 io.presage.ads
10 io.presage.ads.controller
35 io.presage.byte
14 io.presage.case
76 io.presage.char
27 io.presage.char.do
26 io.presage.else
44 io.presage.else.do
16 io.presage.finder
56 io.presage.finder.model
14 io.presage.flatbuffers
7 io.presage.for
47 io.presage.formats
45 io.presage.formats.multiwebviews
13 io.presage.formats.multiwebviews.video
8 io.presage.goto
17 io.presage.helper
8 io.presage.if
2 io.presage.int
59 io.presage.long
22 io.presage.long.do
27 io.presage.model
15 io.presage.new
32 io.presage.provider
9 io.presage.receiver
35 io.presage.this
20 io.presage.try
2 java.io
15 java.lang
11 java.lang.annotation
5 java.net
2 java.nio
6 java.util
6 java.util.concurrent
5 java.util.logging
1 javax.xml.xpath
14 net.vrallev.android.cat
3 net.vrallev.android.cat.instance
1 org.apache.http.conn.ssl
20 org.fmod
1 org.json
8 org.nexage.sourcekit
239 org.nexage.sourcekit.mraid
14 org.nexage.sourcekit.mraid.internal
6 org.nexage.sourcekit.mraid.nativefeature
18 org.nexage.sourcekit.mraid.properties
36 org.nexage.sourcekit.util
40 org.nexage.sourcekit.vast
139 org.nexage.sourcekit.vast.activity
114 org.nexage.sourcekit.vast.model
6 org.nexage.sourcekit.vast.processor
22 org.nexage.sourcekit.vast.view
]
stdout[
processing archive C:\Users\ZaAz\Desktop\GameOfWords2017\GameOfWords2017\Temp\StagingArea\android-libraries\GoogleAIDL\libs\.\classes.jar...
processing com/android/vending/billing/BuildConfig.class...
processing com/android/vending/billing/IInAppBillingService.class...
processing com/android/vending/billing/IInAppBillingService$Stub.class...
processing com/android/vending/billing/IInAppBillingService$Stub$Proxy.class...
processing archive C:\Users\ZaAz\Desktop\GameOfWords2017\GameOfWords2017\Temp\StagingArea\android- libraries\GooglePlay\libs\.\classes.jar...
processing com/unity/purchasing/googleplay/ActivityLauncher.class...
processing com/unity/purchasing/googleplay/BillingServiceManager.class...
processing com/unity/purchasing/googleplay/BillingServiceManager$1.class...
processing com/unity/purchasing/googleplay/BillingServiceManager$1$1.class...
processing com/unity/purchasing/googleplay/BillingServiceManager$1$2.class...
processing com/unity/purchasing/googleplay/BillingServiceManager$2.class...
processing com/unity/purchasing/googleplay/BillingServiceProcessor.class...
processing com/unity/purchasing/googleplay/BuildConfig.class...
processing com/unity/purchasing/googleplay/Consts.class...
processing com/unity/purchasing/googleplay/Consts$PurchaseState.class...
processing com/unity/purchasing/googleplay/Consts$ResponseCode.class...
processing com/unity/purchasing/googleplay/GooglePlayBillingUnAvailableException.class...
processing com/unity/purchasing/googleplay/GooglePlayPurchasing.class...
processing com/unity/purchasing/googleplay/GooglePlayPurchasing$1.class...
processing com/unity/purchasing/googleplay/GooglePlayPurchasing$2.class...
processing com/unity/purchasing/googleplay/GooglePlayPurchasing$3.class...
processing com/unity/purchasing/googleplay/GooglePlayPurchasing$4.class...
processing com/unity/purchasing/googleplay/GooglePlayPurchasing$5.class...
processing com/unity/purchasing/googleplay/GooglePlayPurchasing$6.class...
processing com/unity/purchasing/googleplay/GooglePlayPurchasing$7.class...
processing com/unity/purchasing/googleplay/GooglePlayPurchasing$Features.class...
processing com/unity/purchasing/googleplay/IActivityLauncher.class...
processing com/unity/purchasing/googleplay/IBillingServiceManager.class...
processing com/unity/purchasing/googlep<message truncated>
I had the same problem a few days ago. This is your error:
Too many field references to fit in one dex file: 86260; max is 65536.
The plugins you are using all have their own dex files and the combined size, especially while using something like GooglePlayGames, can ramp up quickly.
You can bypass this by switching your build type in Build Settings from Internal to Gradle.
If you are still not able to build it correctly, you may want to consider removing some Plugins if you can. (I understand this is probably not your desired approach).
I ended up removing the GooglePlayGames Plugin, which was also slowing my build and app down.

Problems with matrix indexing using for loop and if condition

I to do some indexing, something like what follows:
for c=1:size(params,1)
for d=1:size(data,2)
if params(c,8)==1
value1(c,d)=data(params(c,11),d);
elseif params(c,8)==2
value2(c,d)=data(params(c,11),d);
elseif params(c,8)==3
value3(c,d)=data(params(c,11),d);
end
end
end
The problems with this is that if we have params(:,8)=1,3,1,3,2,3,1... then value1 will contain all zeros in rows 2, 4, 5, 6, etc. These are the rows that do not have 1 in column 8 in params. Similarly, value2 will contains all zeros in rows 1, 2, 3, 4, 6, 7... and value3 will contain all zeros in row 1, 3, 5, 7, .... Could anyone tell me how to index so I don't have 'gaps' of zeros in between rows? Thanks!
Edit; below is a sample dataset:
data (1080x15 double)
168 432 45 86
170 437 54 82
163 423 52 83
178 434 50 84
177 444 42 87
177 444 58 85
175 447 48 77
184 451 59 86
168 455 52 104
174 437 62 88
175 443 55 85
179 456 51 92
168 450 73 82
175 454 60 68
params (72x12 double - we are interested in only column 8 and 11 ) so I'm showing only column 8-11 for the sake of space:
1 10 15 1
3 12 16 16
2 10 15 32
3 12 16 47
1 8 14 63
2 10 15 77
2 8 14 92
3 10 15 106
1 12 16 121
3 8 14 137
2 10 15 151
The expected output for value1, value2, and value3 should be 24x15. This is because there are 15 columns in data and value 1, 2, 3 occur 24 times each in column 8 in params.
You can use bsxfun to avoid for-loop (note that it is actually not vertorizing):
value1 = bsxfun(#times,data(params(:,11),:),(params(:,8)==1));
value2 = bsxfun(#times,data(params(:,11),:),(params(:,8)==2));
value3 = bsxfun(#times,data(params(:,11),:),(params(:,8)==3));
But it still gives you the results with zero rows. So you can remove zero-rows by:
value1(all(value1==0,2),:)=[];
value2(all(value2==0,2),:)=[];
value3(all(value3==0,2),:)=[];
You can also use above commands to remove zero-rows in your results without using bsxfun. It is not always good to loose the transparency.

Extracting a submatrix that contains some sub-columns from A

20 4 4 74 20 20 74 85 85 85 2 1
A = 36 1 1 11 36 36 11 66 66 66 4 1
77 1 1 15 77 77 15 11 11 11 1 4
3 4 2 6 7 8 10 10 15 17 1 5
20 4 85
B = 36 1 66
77 1 11
How from the matrix A, I can extract the submatrix whose coloumns contains the vectors B(:,i): ​​[20 36 77] , [4 1 1] and [85 66 11]?
The desired result:
20 4 4 20 20 85 85 85
36 1 1 36 36 66 66 66
77 1 1 77 77 11 11 11
3 4 2 7 8 10 15 17
Now that you've explained what you want, transpose the matrices, select only the three top rows of A, and then use the third argument of ismember to specify that you want to compare entire rows:
A(:,ismember(A(1:3,:).', B.', 'rows').')
20 4 4 20 20 85 85 85
36 1 1 36 36 66 66 66
77 1 1 77 77 11 11 11
3 4 2 7 8 10 15 17

Functional addition of Columns in kdb+q

I have a q table in which no. of non keyed columns is variable. Also, these column names contain an integer in their names. I want to perform some function on these columns without actually using their actual names
How can I achieve this ?
For Example:
table:
a | col10 col20 col30
1 | 2 3 4
2 | 5 7 8
// Assume that I have numbers 10, 20 ,30 obtained from column names
I want something like **update NewCol:10*col10+20*col20+30*col30 from table**
except that no.of columns is not fixed so are their inlcluded numbers
We want to use a functional update (simple example shown here: http://www.timestored.com/kdb-guides/functional-queries-dynamic-sql#functional-update)
For this particular query we want to generate the computation tree of the select clause, i.e. the last part of the functional update statement. The easiest way to do that is to parse a similar statement then recreate that format:
q)/ create our table
q)t:([] c10:1 2 3; c20:10 20 30; c30:7 8 9; c40:0.1*4 5 6)
q)t
c10 c20 c30 c40
---------------
1 10 7 0.4
2 20 8 0.5
3 30 9 0.6
q)parse "update r:(10*c10)+(20*col20)+(30*col30) from t"
!
`t
()
0b
(,`r)!,(+;(*;10;`c10);(+;(*;20;`col20);(*;30;`col30)))
q)/ notice the last value, the parse tree
q)/ we want to recreate that using code
q){(*;x;`$"c",string x)} 10
*
10
`c10
q){(+;x;y)} over {(*;x;`$"c",string x)} each 10 20
+
(*;10;`c10)
(*;20;`c20)
q)makeTree:{{(+;x;y)} over {(*;x;`$"c",string x)} each x}
/ now write as functional update
q)![t;();0b; enlist[`res]!enlist makeTree 10 20 30]
c10 c20 c30 c40 res
-------------------
1 10 7 0.4 420
2 20 8 0.5 660
3 30 9 0.6 900
q)update r:(10*c10)+(20*c20)+(30*c30) from t
c10 c20 c30 c40 r
-------------------
1 10 7 0.4 420
2 20 8 0.5 660
3 30 9 0.6 900
I think functional select (as suggested by #Ryan) is the way to go if the table is quite generic, i.e. column names might varies and number of columns is unknown.
Yet I prefer the way #JPC uses vector to solve the multiplication and summation problem, i.e. update res:sum 10 20 30*(col10;col20;col30) from table
Let combine both approach together with some extreme cases:
q)show t:1!flip(`a,`$((10?2 3 4)?\:.Q.a),'string 10?10)!enlist[til 100],0N 100#1000?10
a | vltg4 pnwz8 mifz5 pesq7 fkcx4 bnkh7 qvdl5 tl5 lr2 lrtd8
--| -------------------------------------------------------
0 | 3 3 0 7 9 5 4 0 0 0
1 | 8 4 0 4 1 6 0 6 1 7
2 | 4 7 3 0 1 0 3 3 6 4
3 | 2 4 2 3 8 2 7 3 1 7
4 | 3 9 1 8 2 1 0 2 0 2
5 | 6 1 4 5 3 0 2 6 4 2
..
q)show n:"I"$string[cols get t]inter\:.Q.n
4 8 5 7 4 7 5 5 2 8i
q)show c:cols get t
`vltg4`pnwz8`mifz5`pesq7`fkcx4`bnkh7`qvdl5`tl5`lr2`lrtd8
q)![t;();0b;enlist[`res]!enlist({sum x*y};n;enlist,c)]
a | vltg4 pnwz8 mifz5 pesq7 fkcx4 bnkh7 qvdl5 tl5 lr2 lrtd8 res
--| -----------------------------------------------------------
0 | 3 3 0 7 9 5 4 0 0 0 176
1 | 8 4 0 4 1 6 0 6 1 7 226
2 | 4 7 3 0 1 0 3 3 6 4 165
3 | 2 4 2 3 8 2 7 3 1 7 225
4 | 3 9 1 8 2 1 0 2 0 2 186
5 | 6 1 4 5 3 0 2 6 4 2 163
..
You can create a functional form query as #Ryan Hamilton indicated, and overall that will be the best approach since it is very flexible. But if you're just looking to add these up, multiplied by some weight, I'm a fan of going through other avenues.
EDIT: missed that you said the number in the columns name could vary, in which case you can easily adjust this. If the column names are all prefaced by the same number of letters, just drop those and then parse the remaining into int or what have you. Otherwise if the numbers are embedded within text, check out this other question
//Create our table with a random number of columns (up to 9 value columns) and 1 key column
q)show t:1!flip (`$"c",/:string til n)!flip -1_(n:2+first 1?10) cut neg[100]?100
c0| c1 c2 c3 c4 c5 c6 c7 c8 c9
--| --------------------------
28| 3 18 66 31 25 76 9 44 97
60| 35 63 17 15 26 22 73 7 50
74| 64 51 62 54 1 11 69 32 61
8 | 49 75 68 83 40 80 81 89 67
5 | 4 92 45 39 57 87 16 85 56
48| 88 34 55 21 12 37 53 2 41
86| 52 91 79 33 42 10 98 20 82
30| 71 59 43 58 84 14 27 90 19
72| 0 99 47 38 65 96 29 78 13
q)update res:sum (1+til -1+count cols t)*flip value t from t
c0| c1 c2 c3 c4 c5 c6 c7 c8 c9 res
--| -------------------------------
28| 3 18 66 31 25 76 9 44 97 2230
60| 35 63 17 15 26 22 73 7 50 1551
74| 64 51 62 54 1 11 69 32 61 1927
8 | 49 75 68 83 40 80 81 89 67 3297
5 | 4 92 45 39 57 87 16 85 56 2582
48| 88 34 55 21 12 37 53 2 41 1443
86| 52 91 79 33 42 10 98 20 82 2457
30| 71 59 43 58 84 14 27 90 19 2134
72| 0 99 47 38 65 96 29 78 13 2336
q)![t;();0b; enlist[`res]!enlist makeTree 1+til -1+count cols t] ~ update res:sum (1+til -1+count cols t)*flip value t from t
1b
q)\ts do[`int$1e4;![t;();0b; enlist[`res]!enlist makeTree 1+til 9]]
232 3216j
q)\ts do[`int$1e4;update nc:sum (1+til -1+count cols t)*flip value t from t]
69 2832j
I haven't tested this on a large table, so caveat emptor
Here is another solution which is also faster.
t,'([]res:(+/)("I"$(string tcols) inter\: .Q.n) *' (value t) tcols:(cols t) except keys t)
By spending some time, we can decrease the word count as well. Logic goes like this:
a:"I"$(string tcols) inter\: .Q.n
Here I am first extracting out the integers from column names and storing them in a vector. Variable 'tcols' is declared at the end of query which is nothing but columns of table except key columns.
b:(value t) tcols:(cols t) except keys t
Here I am extracting out each column vector.
c:(+/) a *' b
Multiplying each column vector(var b) by its integer(var a) and adding corresponding
values from each resulting list.
t,'([]res:c)
Finally storing result in a temp table and joining it to t.