In the following example, why does the first series(f,n=infinity,9); return a series of order 8, but it returns a series of order 9 only after executing series(f,n=infinity,10);?
|\^/| Maple 17 (X86 64 LINUX)
._|\| |/|_. Copyright (c) Maplesoft, a division of Waterloo Maple Inc. 2013
\ MAPLE / All rights reserved. Maple is a trademark of
Waterloo Maple Inc.
| Type ? for help.
> f := 1 + (n-1)*6/n^3 - (1+6/n^3)^(n-2) * (1+6/n^3/(1+(n^2/2)*6/n^3));
6 (n - 1) / 6 \(n - 2) / 6 \
f := 1 + --------- - |1 + ----| |1 + ------------|
3 | 3 | | 3 |
n \ n / \ n (1 + 3/n)/
> series(f,n=infinity,8);
198 810 1
--- - --- + O(----)
6 7 8
n n n
> series(f,n=infinity,9);
198 810 1
--- - --- + O(----)
6 7 8
n n n
> series(f,n=infinity,10);
198 810 2952 11070 1
--- - --- + ---- - ----- + O(---)
6 7 8 9 10
n n n n n
> series(f,n=infinity,9);
198 810 2952 1
--- - --- + ---- + O(----)
6 7 8 9
n n n n
The third paragraph in the Description section of the help-page for the series command mentions that the order parameter specifies the truncation order for the calculation and is not necessarily the order of the returned series.
Also, there is a remember table in play, so that a higher order returned result may be quickly truncated when a subsequent lower truncation order query is computed. (The Order environment variable is being treated somewhat analogously to the Digits environment variable, in that better or more accurate saved results might be truncated/rounded quickly when a coarser query is made.)
After requesting clearance of a remember table for series, the lower order query must once more be actually computed (using the specified truncation order of the calculation) rather than merely looked up and truncated.
restart:
kernelopts(version);
Maple 17.02, X86 64 WINDOWS, Sep 5 2013, Build ID 872941
f := 1 + (n-1)*6/n^3 - (1+6/n^3)^(n-2) * (1+6/n^3/(1+(n^2/2)*6/n^3)):
series(f,n=infinity,9);
198 810 /1 \
--- - --- + O|--|
6 7 | 8|
n n \n /
series(f,n=infinity,10);
198 810 2952 11070 / 1 \
--- - --- + ---- - ----- + O|---|
6 7 8 9 | 10|
n n n n \n /
series(f,n=infinity,9);
198 810 2952 /1 \
--- - --- + ---- + O|--|
6 7 8 | 9|
n n n \n /
forget(series);
series(f,n=infinity,9);
198 810 /1 \
--- - --- + O|--|
6 7 | 8|
n n \n /
Related
I've got a query of the form:
select lR cor mR from j
This works fine, and I get a single number.
When I try to use my custom function:
center:{x - avg raze x}
fit1:{
xc: 0^center[x];
yc: 0^center[y];
beta: raze yc lsq xc;
intercept: (avg raze y) - (beta * avg raze x);
raze beta
}
select fit1[lR;mR] from j
I get a column of numbers, as it appears to have applied the function row-wise, rather than column wise.
Two questions:
How can I go about fixing this?
Is it possible to see the source code of cor so I can learn, or is it closed?
Edit
t:([] date:tt; x:xx; y:yy)
t x y
--------------
2021.01.01 0 4
2021.01.02 1 4
2021.01.03 2 4
2021.01.04 3 4
2021.01.05 4 4
2021.01.06 5 4
2021.01.07 6 4
2021.01.08 7 4
2021.01.09 8 4
2021.01.10 9 4
select fit1[x;y] from t
Updated now with a proper example:
center:{x - avg raze x}
fit1:{
xc: 0^center[x];
yc: 0^center[y];
beta: raze yc lsq xc;
intercept: (avg raze y) - (beta * avg raze x);
raze beta
}
tt: 2021.01.01 + til 10
xx: 9h$til 10
yy: ((xx)*3) + 4
t:([] date:tt; x:xx; y:yy)
date x y
---------------
2021.01.01 0 4
2021.01.02 1 7
2021.01.03 2 10
2021.01.04 3 13
2021.01.05 4 16
2021.01.06 5 19
2021.01.07 6 22
2021.01.08 7 25
2021.01.09 8 28
2021.01.10 9 31
q)fit1[enlist xx;enlist yy]
,3f
q)select fit1[x;y] from t
y
----
-4.5
-3.5
-2.5
-1.5
-0.5
0.5
1.5
2.5
3.5
4.5
d
I don't think there is enough information provided to help give a definitive fix for your issue but your function creates an intercept variable which it doesn't use in the final return. This line is currently pointless. Your custom function is not designed to return an atom like cor.
Some keywords provide the underlying k code if you just enter the keyword in the q console. If you get the key word back then it is hidden but for cor and cov the equivalent code is provided in the docs.
https://code.kx.com/q/ref/cor/
https://code.kx.com/q/ref/cov/
To answer your first question you can do:
q)fit1[j`lR;j`mR]
I have a dataset of measurements of "Y" at different locations, and I am trying to determine how variable Y is influenced by variables A, B, and D by running a lmer() model and analyzing the results. However, when I reach the post hoc step, I receive an error when trying to analyze.
Here is an example of my data:
table <- " ID location A B C D Y
1 1 AA 0 0.6181587 -29.67 14.14 168.041
2 2 AA 1 0.5816176 -29.42 14.21 200.991
3 3 AA 2 0.4289670 -28.57 13.55 200.343
4 4 AA 3 0.4158891 -28.59 12.68 215.638
5 5 AA 4 0.3172721 -28.74 12.28 173.299
6 6 AA 5 0.1540603 -27.86 14.01 104.246
7 7 AA 6 0.1219355 -27.18 14.43 128.141
8 8 AA 7 0.1016643 -26.86 13.75 179.330
9 9 BB 0 0.6831649 -28.93 17.03 210.066
10 10 BB 1 0.6796935 -28.54 18.31 280.249
11 11 BB 2 0.5497743 -27.88 17.33 134.023
12 12 BB 3 0.3631052 -27.48 16.79 142.383
13 13 BB 4 0.3875498 -26.98 17.81 136.647
14 14 BB 5 0.3883785 -26.71 17.56 142.179
15 15 BB 6 0.4058061 -26.72 17.71 109.826
16 16 CC 0 0.8647298 -28.53 11.93 220.464
17 17 CC 1 0.8664036 -28.39 11.59 326.868
18 18 CC 2 0.7480748 -27.61 11.75 322.745
19 19 CC 3 0.5959143 -26.81 13.27 170.064
20 20 CC 4 0.4849077 -26.77 14.68 118.092
21 21 CC 5 0.3584687 -26.65 15.65 95.512
22 22 CC 6 0.3018285 -26.33 16.11 71.717
23 23 CC 7 0.2629121 -26.39 16.16 60.052
24 24 DD 0 0.8673077 -27.93 12.09 234.244
25 25 DD 1 0.8226558 -27.96 12.13 244.903
26 26 DD 2 0.7826429 -27.44 12.38 252.485
27 27 DD 3 0.6620447 -27.23 13.84 150.886
28 28 DD 4 0.4453213 -27.03 15.73 102.787
29 29 DD 5 0.3720257 -27.13 16.27 109.201
30 30 DD 6 0.6040217 -27.79 16.41 101.509
31 31 EE 0 0.8770987 -28.62 12.72 239.036
32 32 EE 1 0.8504547 -28.47 12.92 220.600
33 33 EE 2 0.8329484 -28.45 12.94 174.979
34 34 EE 3 0.8181102 -28.37 13.17 138.412
35 35 EE 4 0.7942685 -28.32 13.69 121.330
36 36 EE 5 0.7319724 -28.22 14.62 111.851
37 37 EE 6 0.7014828 -28.24 15.04 110.447
38 38 EE 7 0.7286984 -28.15 15.18 121.831"
#Create a dataframe with the above table
df <- read.table(text=table, header = TRUE)
df
# Make sure location is a factor
df$location<-as.factor(df$location)
Here is my model:
# Load libraries
library(ggplot2)
library(pscl)
library(lmtest)
library(lme4)
library(car)
mod = lmer(Y ~ A * B * poly(D, 2) * (1|location), data = df)
summary(mod)
plot(mod)
I now need to determine what variables significantly influence Y. Thus I ran Anova() from the package car (output pasted here).
Anova(mod)
# Analysis of Deviance Table (Type II Wald chisquare tests)
#
# Response: Y
# Chisq Df Pr(>Chisq)
# A 8.2754 1 0.004019 **
# B 0.0053 1 0.941974
# poly(D, 2) 40.4618 2 1.636e-09 ***
# A:B 0.1709 1 0.679348
# A:poly(D, 2) 1.6460 2 0.439117
# B:poly(D, 2) 5.2601 2 0.072076 .
# A:B:poly(D, 2) 0.6372 2 0.727175
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
This suggests that:
A significantly influences Y
B does not significantly influence Y
D significantly influences Y
So next I would run a post hoc test for each of these variables, but this is where I run into issues. I have tried using both emmeans and multcomp packages below:
library(emmeans)
emmeans(mod, list(pairwise ~ A), adjust = "tukey")
# NOTE: Results may be misleading due to involvement in interactions
# Error in if ((misc$estType == "pairs") && (paste(c("", by), collapse = ",") != :
# missing value where TRUE/FALSE needed
pairs(emmeans(mod, "A"))
# NOTE: Results may be misleading due to involvement in interactions
# Error in if ((misc$estType == "pairs") && (paste(c("", by), collapse = ",") != :
# missing value where TRUE/FALSE needed
library(multcomp)
summary(glht(mod, linfct = mcp(A = "Tukey")), test = adjusted("fdr"))
# Error in h(simpleError(msg, call)) :
# error in evaluating the argument 'object' in selecting a method for function 'summary': Variable(s) ‘depth’ of class ‘integer’ is/are not contained as a factor in ‘model’.
This is the first time I've run an ANOVA/post hoc test on a lmer() model, and though I've read a few introductory sites for this model, I'm not sure I am testing it correctly. Any help would be appreciated.
If I am looking at the data correctly, A is the variable that has values of 0, 1, ..., 7. Now look at your anova table, where you see that A has only 1 d.f., not 7 as it should for a factor having 8 levels. That means your model is taking A to be a numerical predictor -- which is rather meaningless. Make A into a factor and re-fit he model. You'll have better luck.
I also think you meant to have + (1|location) at the end of the model formula, rather than having the random effects interacting with some of the polynomial effects.
Given a keyed table, e.g.:
q)\S 7 / seed random numbers for reproducibility
q)v:flip (neg[d 0]?`1)!#[;prd[d]?12] d:4 6 / 4 cols 6 rows
q)show kt:([]letter:d[1]#.Q.an)!v
letter| c g b e
------| ----------
a | 11 0 3 9
b | 11 8 10 0
c | 7 2 2 3
d | 8 4 9 6
e | 0 0 5 0
f | 1 0 0 11
How to calculate an average for each row --- e.g. (c+g+b+e)%4 --- for any number of columns?
Following on from your own solution, note that you have to be a little careful with null handling. Your approach won't ignore nulls in the way that avg normally would.
q).[`kt;("a";`g);:;0N];
q)update av:avg flip value kt from kt
letter| c g b e av
------| ---------------
a | 11 3 9
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
To make it ignore nulls you have to avg each row rather than averaging the flip.
q)update av:avg each value kt from kt
letter| c g b e av
------| -------------------
a | 11 3 9 7.666667
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
Solution 1: q-sql
q)update av:avg flip value kt from kt
letter| c g b e av
------| ---------------
a | 11 0 3 9 5.75
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
Solution 2: functional q-sql
tl;dr:
q)!enlist(avg;)enlist,cols[`kt]except cols key`kt
letter| c g b e av
------| ---------------
a | 11 0 3 9 5.75
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
let's start with a look how the parse tree of a non-general solution would look like:
q)parse"update av:avg (c;g;b;e) from kt"
!
`kt
()
0b
(,`av)!,(avg;(enlist;`c;`g;`b;`e))
(note that q is a wrapper implemented in k, so the , prefix operator in the above expression is the same as enlist keyword in q)
so all the below are equivalent (verify with ~). relying on projection: (x;y)~(x;)y, we can further improve the readability by reducing the distance between parens:
q)k)(!;`kt;();0b;(,`av)!,(avg;(enlist;`c;`g;`b;`e)))
q)(!;`kt;();0b;(enlist`av)!enlist(avg;(enlist;`c;`g;`b;`e)))
q)(!;`kt;();0b;(1#`av)!enlist(avg;(enlist;`c;`g;`b;`e)))
q)(!;`kt;();0b;)(1#`av)!enlist(avg;)(enlist;`c;`g;`b;`e)
let's evaluate the parse tree to check:
q)eval(!;`kt;();0b;)(1#`av)!enlist(avg;)(enlist;`c;`g;`b;`e)
letter| c g b e av
------| ---------------
a | 11 0 3 9 5.75
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
(enlist;`c;`g;`b;`e) in the general case is:
q)enlist,cols[`kt]except cols key`kt
enlist
`c
`g
`b
`e
so let's plug in and check:
q)eval(!;`kt;();0b;(1#`av)!enlist(avg;)enlist,cols[`kt]except cols key`kt)
letter| c g b e av
------| ---------------
a | 11 0 3 9 5.75
b | 11 8 10 0 7.25
c | 7 2 2 3 3.5
d | 8 4 9 6 6.75
e | 0 0 5 0 1.25
f | 1 0 0 11 3
also:
q)![`kt;();0b;(1#`av)!enlist(avg;)enlist,cols[`kt]except cols key`kt]
q)!enlist(avg;)enlist,cols[`kt]except cols key`kt
I have some question related the RISC V manual
It has different types of instruction encoding such as R-type,I-type.
Just like the MIPS encoding.
* R-type
31 25 24 20 19 15 14 12 11 7 6 0
+------------+---------+---------+------+---------+-------------+
| funct7 | rs2 | rs1 |funct3| rd | opcode |
+------------+---------+---------+------+---------+-------------+
* I-type
31 20 19 15 14 12 11 7 6 0
+----------------------+---------+------+---------+-------------+
| imm | rs1 |funct3| rd | opcode |
+----------------------+---------+------+---------+-------------+
* S-type
31 25 24 20 19 15 14 12 11 7 6 0
+------------+---------+---------+------+---------+-------------+
| imm | rs2 | rs1 |funct3| imm | opcode |
+------------+---------+---------+------+---------+-------------+
* U-type
31 11 7 6 0
+---------------------------------------+---------+-------------+
| imm | rd | opcode |
+---------------------------------------+---------+-------------+
But it also have something called immediate format:
such as I-immediate, S-immediate and so on
* I-immediate
31 10 5 4 1 0
+-----------------------------------------+-----------+-------+--+
| <-- 31 | 30:25 | 24:21 |20|
+-----------------------------------------+-----------+-------+--+
* S-immediate
31 10 5 4 1 0
+-----------------------------------------+-----------+-------+--+
| <-- 31 | 30:25 | 11:8 |7 |
+-----------------------------------------+-----------+-------+--+
* B-immediate
31 12 11 10 5 4 1 0
+--------------------------------------+--+-----------+-------+--+
| <-- 31 |7 | 30:25 | 11:8 |z |
+--------------------------------------+--+-----------+-------+--+
* U-immediate
31 30 20 19 12 11 0
+--+-------------------+---------------+-------------------------+
|31| 30:20 | 19:12 | <-- z |
+--+-------------------+---------------+-------------------------+
* J-immediate
31 20 19 12 11 10 5 4 1 0
+----------------------+---------------+--+-----------+-------+--+
| <-- 31 | 19:12 |20| 30:25 | 24:21 |z |
+----------------------+---------------+--+-----------+-------+--+
According to the manual, it say those immediate is produced by RISC-V instruction but how are the things related?
What is the point to have immediate format?
The 2nd set of diagrams is showing you how the immediate bits are concatenated and sign-extended into a 32-bit integer (so they can work as a source operand for normal 32-bit ALU instructions like addi which need both their inputs to be the same size).
For I-type instructions it's trivial, just arithmetic right-shift the instruction word by 20 bits, because there's only one immediate field, and it's contiguous at the top of the instruction word.
For S-type immediate instructions, there are two separate fields in the instruction word: [31:25] and [11:7], and this shows you that they're in that order, not [11:7, 31:25] and not with any implicit zeros between them.
B-type immediate instructions apparently put bit 7 in front of [30:25], and the low bit is an implicit zero. (So the resulting number is always even). I assume B-type is for branches.
U-type is also interesting, padding the 20-bit immediate with trailing zeros. It's used for lui to create the upper bits of 32-bit constants (with addi supplying the rest). It's not a coincidence that U-type and I-type together have 32 total immediate bits.
To access static data, lui can create the high part of an address while lw can supply the low part directly, instead of using an addi to create the full address in a register. This is typical for RISC ISAs like MIPS and PowerPC as well (see an example on the Godbolt compiler explorer). But unlike most other RISC ISAs, RISC-V has auipc which adds the U-type immediate to the program counter, for efficient PIC without having to load addresses from a GOT (global offset table). (A recent MIPS revision also added an add-to-PC instruction, but for a long time MIPS was quite bad at PIC).
lui can encode any 4k-aligned address, i.e. a page-start address with 4k pages.
In matlab, I want to fit a piecewise regression and find where on the x-axis the first change-point occurs. For example, for the following data, the output might be changepoint=20 (I don't actually want to plot it, just want the change point).
data = [1 4 4 3 4 0 0 4 5 4 5 2 5 10 5 1 4 15 4 9 11 16 23 25 24 17 31 42 35 45 49 54 74 69 63 46 35 31 27 15 10 5 10 4 2 4 2 2 3 5 2 2];
x = 1:52;
plot(x,data,'.')
If you have the Signal Processing Toolbox, you can directly use the findchangepts function (see https://www.mathworks.com/help/signal/ref/findchangepts.html for documentation):
data = [1 4 4 3 4 0 0 4 5 4 5 2 5 10 5 1 4 15 4 9 11 16 23 25 24 17 31 42 35 45 49 54 74 69 63 46 35 31 27 15 10 5 10 4 2 4 2 2 3 5 2 2];
x = 1:52;
ipt = findchangepts(data);
x_cp = x(ipt);
data_cp = data(ipt);
plot(x,data,'.',x_cp,data_cp,'o')
The index of the change point in this case is 22.
Plot of data and its change point circled in red:
I know this is an old question but just want to provide some extra thoughts. In Maltab, an alternative implemented by me is a Bayesian changepoint detection algorithm that estimates not just the number and locations of the changepoints but also reports the occurrence probability of changepoints. In its current implementation, it deals with only time-series-like data (aka, 1D sequential data). More info about the tool is available at this FileExchange entry (https://www.mathworks.com/matlabcentral/fileexchange/72515-bayesian-changepoint-detection-time-series-decomposition).
Here is its quick application to your sample data:
% Automatically install the Rbeast or BEAST library to local drive
eval(webread('http://b.link/beast')) %
data = [1 4 4 3 4 0 0 4 5 4 5 2 5 10 5 1 4 15 4 9 11 16 23 25 24 17 31 42 35 45 49 54 74 69 63 46 35 31 27 15 10 5 10 4 2 4 2 2 3 5 2 2];
out = beast(data, 'season','none') % season='none': there is no seasonal/periodic variation in the data
printbeast(out)
plotbeast(out)
Below is a summary of the changepoint, given by printbeast():
#####################################################################
# Trend Changepoints #
#####################################################################
.-------------------------------------------------------------------.
| Ascii plot of probability distribution for number of chgpts (ncp) |
.-------------------------------------------------------------------.
|Pr(ncp = 0 )=0.000|* |
|Pr(ncp = 1 )=0.000|* |
|Pr(ncp = 2 )=0.000|* |
|Pr(ncp = 3 )=0.859|*********************************************** |
|Pr(ncp = 4 )=0.133|******** |
|Pr(ncp = 5 )=0.008|* |
|Pr(ncp = 6 )=0.000|* |
|Pr(ncp = 7 )=0.000|* |
|Pr(ncp = 8 )=0.000|* |
|Pr(ncp = 9 )=0.000|* |
|Pr(ncp = 10)=0.000|* |
.-------------------------------------------------------------------.
| Summary for number of Trend ChangePoints (tcp) |
.-------------------------------------------------------------------.
|ncp_max = 10 | MaxTrendKnotNum: A parameter you set |
|ncp_mode = 3 | Pr(ncp= 3)=0.86: There is a 85.9% probability |
| | that the trend component has 3 changepoint(s).|
|ncp_mean = 3.15 | Sum{ncp*Pr(ncp)} for ncp = 0,...,10 |
|ncp_pct10 = 3.00 | 10% percentile for number of changepoints |
|ncp_median = 3.00 | 50% percentile: Median number of changepoints |
|ncp_pct90 = 4.00 | 90% percentile for number of changepoints |
.-------------------------------------------------------------------.
| List of probable trend changepoints ranked by probability of |
| occurrence: Please combine the ncp reported above to determine |
| which changepoints below are practically meaningful |
'-------------------------------------------------------------------'
|tcp# |time (cp) |prob(cpPr) |
|------------------|---------------------------|--------------------|
|1 |33.000000 |1.00000 |
|2 |42.000000 |0.98271 |
|3 |19.000000 |0.69183 |
|4 |26.000000 |0.03950 |
|5 |11.000000 |0.02292 |
.-------------------------------------------------------------------.
Here is the graphic output. Three major changepoints are detected:
You can use sgolayfilt function, that is a polynomial fit to the data, or reproduce OLS method: http://www.utdallas.edu/~herve/Abdi-LeastSquares06-pretty.pdf (there is a+bx notation instead of ax+b)
For linear fit of ax+b:
If you replace x with constant vector of length 2n+1: [-n, ... 0 ... n] on each step, you get the following code for sliding regression coeffs:
for i=1+n:length(y)-n
yi = y(i-n : i+n);
sum_xy = sum(yi.*x);
a(i) = sum_xy/sum_x2;
b(i) = sum(yi)/n;
end
Notice that in this code b means sliding average of your data, and a is a least-square slope estimate (first derivate).