Create trendline for a plot of a set of disordered data? - matlab

I have a totally random set of data with around 500 inputs obtained from a physics experiment in excel format. I plotted it successfully but I am not able to create a trendline for my plot. I need to find the trendline for this data.
excel data can be found from this google drive link.
Alternatively, these are the first 2 columns of the Excel file:
0_x 0_y
0.321816 -82.594828
1.595041 -85.396122
2.868266 -82.085502
3.632201 -84.632133
4.905426 -83.868143
6.178651 -84.122807
7.197231 -85.905448
7.961166 -84.37747
8.470457 -85.141459
9.743682 -82.340165
10.252972 -84.886796
11.271552 -82.594828
13.054067 -84.886796
13.054067 -83.104154
14.327292 -84.632133
15.345872 -86.160112
16.364452 -84.886796
17.383032 -86.160112
18.401612 -83.868143
19.165547 -85.905448
20.438773 -82.849491
21.457353 -85.396122
22.221288 -83.104154
22.475933 -85.650785
23.494513 -85.650785
24.513093 -84.632133
25.277028 -85.141459
26.550253 -84.122807
27.314188 -85.141459
28.587413 -82.849491
29.860638 -84.886796
31.133864 -82.340165
31.897799 -84.886796
32.916379 -83.868143
33.934959 -84.37747
34.698894 -84.886796
35.717474 -84.122807
37.245344 -85.141459
38.263924 -83.104154
39.282504 -84.122807
40.301084 -81.576175
40.810374 -84.122807
42.0836 -82.085502
43.356825 -84.886796
44.12076 -85.396122
45.13934 -85.141459
45.903275 -86.669438
47.1765 -84.632133
47.940435 -85.396122
48.70437 -83.358817
49.977595 -85.396122
50.486885 -82.085502
51.25082 -84.632133
52.2694 -83.868143
52.524045 -83.868143
53.287981 -85.905448
55.070496 -84.37747
56.343721 -86.160112
57.107656 -84.122807
57.871591 -86.160112
58.635526 -83.61348
59.144816 -85.905448
60.163396 -84.122807
61.181976 -85.396122
62.709846 -86.160112
62.964491 -85.396122
64.492362 -86.669438
65.001652 -85.141459
65.765587 -85.905448
66.784167 -84.122807
68.566682 -86.160112
69.585262 -83.358817
70.349197 -85.650785
71.113132 -84.37747
72.386357 -84.632133
73.150292 -85.141459
73.914227 -83.868143
74.932807 -85.141459
75.951388 -82.594828
77.224613 -85.396122
78.752483 -82.340165
79.516418 -85.141459
80.280353 -84.122807
81.553578 -85.905448
82.317513 -85.650785
82.826803 -84.886796
83.845383 -85.905448
83.336093 -83.868143
81.808223 -85.905448
80.534998 -83.104154
79.516418 -85.905448
78.497838 -83.868143
76.969968 -85.396122
75.696743 -86.160112
74.678162 -85.396122
73.659582 -86.669438
72.641002 -85.141459
71.367777 -86.160112
69.585262 -83.61348
68.566682 -86.160112
67.548102 -84.122807
66.274877 -86.160112
65.765587 -85.650785
65.001652 -85.905448
63.728426 -86.414775
61.945911 -84.632133
60.927331 -86.160112
60.418041 -83.61348
59.144816 -85.905448
58.380881 -82.594828
57.362301 -84.886796
56.089076 -82.849491
55.325141 -85.141459
54.561206 -85.141459
53.797271 -83.61348
52.524045 -85.141459
51.505465 -83.358817
50.486885 -85.141459
49.21366 -83.104154
48.19508 -84.886796
47.431145 -82.340165
46.15792 -84.886796
45.13934 -84.37747
44.12076 -85.396122
42.847535 -85.141459
41.319664 -84.886796
40.301084 -85.396122
39.537149 -84.632133
38.009279 -85.905448
37.245344 -82.849491
35.972119 -85.396122
34.953539 -83.868143
34.189604 -85.141459
32.152444 -84.632133
31.388509 -84.632133
30.115283 -86.160112
29.351348 -83.61348
28.078123 -85.396122
27.568833 -82.594828
26.804898 -85.396122
25.786318 -82.085502
24.513093 -85.141459
23.494513 -85.141459
22.730578 -84.37747
21.711998 -84.632133
20.438773 -84.122807
19.674838 -84.632133
18.401612 -82.594828
17.383032 -84.886796
16.619097 -82.594828
15.855162 -84.122807
15.091227 -83.61348
13.818002 -84.122807
12.799422 -84.122807
11.780842 -84.122807
11.016907 -84.886796
9.743682 -82.849491
8.215811 -84.632133
7.706521 -81.830838
6.942586 -84.632133
5.414716 -83.868143
4.141491 -85.396122
3.122911 -85.396122
2.358976 -84.886796
1.085751 -85.905448
0.321816 -84.37747
-0.696764 -85.905448
-1.969989 -83.358817
-2.98857 -85.650785
-4.51644 -82.594828
-5.53502 -85.905448
-6.808245 -85.396122
-7.57218 -86.160112
-8.59076 -86.414775
-10.11863 -85.141459
-10.62792 -86.669438
-11.901145 -84.122807
-12.919725 -86.669438
-13.17437 -83.61348
-13.938306 -85.650785
-15.211531 -84.122807
-15.975466 -85.396122
-17.248691 -86.414775
-18.267271 -85.141459
-19.285851 -86.160112
-20.049786 -85.141459
-21.323011 -84.886796
-22.341591 -82.594828
-23.105526 -85.141459
-24.124106 -82.085502
-25.397332 -84.632133
-26.161267 -84.632133
-27.179847 -84.632133
-27.434492 -85.905448
-28.707717 -84.632133
-29.471652 -85.905448
-30.999522 -83.358817
-32.527392 -85.396122
-33.291327 -82.340165
-34.309907 -85.396122
-35.073842 -83.61348
-35.583132 -85.141459
-37.620293 -85.396122
-38.638873 -84.122807
-39.402808 -85.650785
-40.421388 -84.122807
-41.694613 -86.160112
-43.222483 -82.849491
-44.241063 -86.160112
-45.259643 -83.868143
-46.023578 -85.650785
-47.551449 -85.905448
-48.315384 -85.141459
-49.588609 -85.141459
-50.607189 -84.122807
-51.625769 -85.141459
-52.644349 -82.340165
-53.662929 -84.886796
-54.426864 -82.594828
-54.936154 -84.37747
-56.209379 -83.104154
-56.718669 -83.868143
-57.737249 -84.37747
-59.26512 -83.104154
-60.029055 -84.37747
-61.30228 -81.830838
-62.066215 -83.868143
-62.32086 -82.085502
-63.594085 -84.122807
-64.612665 -82.849491
-66.140535 -84.122807
-66.649825 -84.122807
-68.177695 -83.868143
-69.45092 -85.396122
-70.214856 -83.61348
-71.233436 -85.141459
-72.252016 -82.594828
-73.015951 -84.886796
-74.289176 -82.594828
-76.071691 -84.632133
-76.326336 -85.650785
-77.854206 -84.122807
-78.618141 -86.160112
-79.891366 -85.905448
-80.655301 -86.414775
-81.928527 -83.868143
-82.692462 -86.414775
-83.965687 -83.358817
-84.984267 -85.141459
-86.002847 -83.868143
-86.766782 -84.122807
-87.785362 -84.886796
-88.803942 -83.868143
-89.313232 -85.141459
-90.331812 -83.358817
-91.350392 -85.650785
-92.368973 -82.594828
-93.132908 -84.37747
-94.406133 -83.104154
-95.424713 -84.632133
-96.188648 -85.141459
-96.697938 -84.37747
-97.461873 -85.141459
-98.480453 -84.37747
-99.499033 -85.650785
-100.517613 -83.104154
-101.790838 -85.650785
-102.554773 -83.358817
-103.573354 -85.650785
-104.591934 -85.396122
-105.865159 -85.396122
-107.647674 -85.650785
-108.156964 -84.632133
-108.920899 -85.905448
-110.703414 -83.358817
-111.721994 -86.414775
-112.485929 -83.868143
-113.759154 -85.905448
-114.52309 -85.141459
-115.54167 -86.414775
-116.305605 -86.414775
-117.57883 -85.396122
-118.08812 -86.414775
-118.852055 -84.886796
-120.12528 -86.160112
-120.889215 -83.358817
-121.907795 -85.905448
-122.926375 -83.868143
-123.69031 -85.396122
-124.963535 -85.396122
-125.727471 -84.632133
-126.746051 -85.905448
-127.764631 -83.868143
-128.783211 -84.632133
-129.037856 -82.594828
-130.056436 -85.141459
-131.075016 -82.085502
-132.093596 -84.37747
-133.112176 -84.37747
-133.876111 -84.37747
-135.149336 -84.632133
-136.677207 -83.104154
-137.441142 -85.650785
-138.459722 -83.61348
-139.223657 -84.886796
-140.496882 -82.340165
-141.515462 -84.632133
-142.534042 -83.358817
-143.297977 -84.37747
-144.825847 -85.141459
-145.844427 -84.886796
-146.863007 -86.160112
-147.881588 -84.37747
-148.900168 -85.396122
-149.918748 -83.358817
-150.682683 -85.650785
-151.701263 -83.358817
-152.719843 -85.141459
-153.738423 -85.396122
-153.993068 -84.122807
-155.011648 -84.886796
-156.030228 -84.37747
-157.303453 -84.632133
-158.067388 -82.340165
-159.340614 -85.396122
-159.849904 -82.085502
-159.595259 -84.122807
-158.576678 -82.849491
-157.303453 -84.122807
-156.030228 -84.37747
-155.011648 -83.358817
-153.229133 -84.37747
-152.210553 -82.849491
-151.446618 -84.632133
-149.664103 -81.321512
-148.645523 -84.886796
-147.881588 -82.594828
-146.608362 -84.37747
-145.589782 -84.886796
-144.571202 -84.122807
-143.552622 -85.396122
-143.043332 -84.122807
-142.024752 -85.396122
-140.751527 -83.358817
-139.478302 -85.905448
-138.459722 -83.358817
-137.441142 -85.905448
-136.677207 -85.141459
-135.403981 -85.141459
-134.640046 -85.905448
-134.130756 -84.632133
-132.857531 -84.632133
-131.838951 -83.868143
-130.820371 -85.905448
-129.547146 -82.594828
-128.783211 -85.650785
-127.764631 -83.868143
-125.982116 -85.141459
-124.963535 -85.650785
-123.944955 -84.632133
-122.926375 -85.396122
-121.907795 -83.868143
-120.889215 -85.141459
-119.870635 -82.594828
-118.59741 -85.141459
-117.833475 -82.849491
-116.305605 -85.905448
-114.777735 -85.141459
-114.013799 -84.632133
-113.504509 -85.141459
-112.740574 -83.868143
-111.467349 -85.396122
-110.703414 -83.868143
-109.684834 -85.650785
-108.666254 -83.104154
-107.647674 -86.160112
-106.629094 -84.37747
-105.355869 -85.396122
-104.337289 -85.905448
-103.318709 -85.141459
-102.300128 -86.160112
-102.045483 -83.868143
-101.026903 -86.669438
-100.008323 -83.868143
-98.735098 -86.669438
-97.716518 -84.122807
-96.952583 -85.905448
-95.934003 -86.669438
-94.406133 -85.905448
-93.387553 -85.905448
-92.368973 -84.632133
-91.350392 -85.905448
-90.331812 -83.358817
-89.313232 -85.650785
-88.294652 -83.104154
-87.530717 -85.141459
-86.512137 -83.358817
-85.238912 -84.632133
-84.474977 -84.886796
-83.201752 -84.122807
-82.437817 -84.886796
-80.655301 -84.122807
-79.891366 -85.141459
-78.618141 -82.340165
-77.599561 -84.886796
-76.835626 -83.358817
-75.817046 -85.141459
-75.053111 -85.141459
-73.779886 -84.37747
-73.015951 -86.160112
-72.506661 -83.868143
-71.488081 -85.905448
-70.469501 -82.085502
-69.196275 -85.141459
-67.92305 -83.104154
-67.41376 -85.396122
-66.649825 -85.141459
-65.3766 -85.141459
-65.121955 -85.905448
-64.103375 -84.122807
-63.084795 -84.886796
-62.066215 -83.358817
-61.047635 -85.141459
-60.029055 -82.085502
-59.010475 -84.886796
-58.246539 -83.358817
-56.973314 -84.886796
-55.700089 -85.141459
-54.681509 -83.868143
-53.408284 -84.37747
-52.644349 -83.104154
-51.880414 -84.37747
-50.861834 -81.830838
-49.588609 -84.122807
-48.824674 -81.830838
-47.806094 -84.37747
-46.787513 -84.632133
-45.259643 -84.886796
-43.986418 -84.886796
-43.222483 -83.61348
-42.203903 -85.905448
-41.185323 -83.61348
-40.166743 -86.160112
-39.148163 -83.104154
-38.384228 -85.650785
-37.365648 -85.141459
-36.856358 -84.886796
-35.837777 -84.886796
-35.073842 -83.358817
-33.800617 -84.886796
-33.036682 -82.594828
-31.763457 -85.905448
-31.508812 -83.104154
-30.490232 -86.414775
-29.471652 -84.122807
-28.453072 -86.160112
-27.434492 -86.414775
-26.415912 -85.396122
-25.397332 -86.414775
-24.378751 -84.632133
-23.105526 -86.414775
-21.832301 -83.868143
-21.068366 -85.650785
-19.540496 -83.868143
-19.031206 -85.141459
-18.012626 -85.141459
-16.994046 -84.632133
-15.466176 -85.905448
-14.956886 -83.868143
-13.68366 -85.141459
-12.919725 -82.849491
-11.901145 -84.886796
-10.62792 -82.340165
-9.863985 -85.141459
-8.336115 -83.61348
-7.57218 -84.886796
-6.04431 -86.414775
-5.53502 -85.141459
-5.02573 -85.396122
-4.00715 -83.868143
-2.479279 -85.396122
-1.206054 -82.849491
-0.696764 -85.396122
-0.187474 -83.868143
1.085751 -86.160112
1.849686 -86.160112
3.122911 -85.905448
4.141491 -86.414775
4.905426 -84.632133
5.924006 -86.160112
6.178651 -83.358817
7.197231 -85.905448
8.215811 -82.594828
9.234392 -85.141459
10.252972 -84.632133
11.271552 -84.632133
13.054067 -85.396122
13.563357 -84.122807
14.836582 -85.141459
15.855162 -83.868143
I tried polyval, polyfit, Basic fitting (plot->tootls->basic fitting)
datasheet=xlsread('08_08_2019-11_08_58_.xlsx','Worksheet');
x=datasheet(:,1);
y=datasheet(:,2);
plot(x,y)

What exactly do you understand under trendline? A linear line p1*x+p2 can be achieved by the following code, which assumes that you have the curve fitting toolbox installed. The last part concerning polyfit() does not need this toolbox.
If you check these (fit) and these (fitoptions) docs, you'll find out that also other options, for quadratic polynomials, exponentials or fourier/sin-cos functions (and many more) are available when using the toolbox. The xand y of the data you posted looks like sinusoidal to me, so I'd go with the second option below.
Linear fit p1*x+p2
data = xlsread('08_08_2019-11_08_58_.xlsx');
x=data(:,1);
y=data(:,2);
[xData, yData] = prepareCurveData(x, y);
% Set up fittype and options.
ft = fittype('poly1')
% Fit model to data.
[fitresult, gof] = fit(xData, yData, ft);
% Plot fit with data.
figure
h = plot(fitresult, xData, yData);
legend(h, 'data', 'linear 1st order polynomial', 'Location', 'NorthEast');
xlabel('x'); ylabel('y'); grid on
% Extract the equation(ft) and the coefficients
coeffnames(fitresult)
coeffvalues(fitresult)
Using a Nonlinear Least Squares to fit a Fourier model a0 + a1*cos(x*w) + b1*sin(x*w)
data = xlsread('08_08_2019-11_08_58_.xlsx');
x=data(:,1);
y=data(:,2);
[xData, yData] = prepareCurveData(x, y);
% Set up fittype and options.
ft = fittype('fourier1')
opts = fitoptions('Method', 'NonlinearLeastSquares');
% Fit model to data.
[fitresult, gof] = fit(xData, yData, ft, opts);
% Plot fit with data.
figure
h = plot(fitresult, xData, yData);
legend(h, 'data', 'Fourier model fit', 'Location', 'NorthEast');
xlabel('x'); ylabel('y'); grid on
% Extract the equation(ft) and the coefficients
coeffnames(fitresult)
coeffvalues(fitresult)
Using the polyfit() and polyval() functions for p1*x+p2:
% using polyfit and polyval (no toolbox needed)
data = xlsread('08_08_2019-11_08_58_.xlsx');
x=data(:,1);
y=data(:,2);
p = polyfit(x, y, 1)
pp = polyval(p, x);
h = plot(x, y, 'o', x, pp);
legend(h, 'data', 'linear 1st order polynomial', 'Location', 'NorthEast');
xlabel('x'); ylabel('y'); grid on
Without Curve fitting toolbox, using your mentioned in-plot method
data = xlsread('08_08_2019-11_08_58_.xlsx');
x=data(:,1);
y=data(:,2);
plot(x, y)
In the figure window: Tools -> Basic Fitting -> linear/polynomial
also seems to work for your given case.

Related

How to do repeated measures Anova and Friedman test in gtsummary?

I have repeated measures data for fasting glucose of few patients which are measured in Day 0, 3 and 7 during a clinical trial. I want to do friedman.test (and repeated measures anova for another data). Is it possible in gtsummaryto do Friedman test and repeated measures Anova?
This is my code
library(gtsummary)
library(dplyr)
smpl %>% select(c(Participant.ID, value, Day)) %>%
tbl_summary(by = Day, include = -Participant.ID) %>% add_p()
By default gtsummary does Kruskal-Wallis rank sum test for the three groups in my data (Day 0, 3 and 7). As it is a repeated measures I wanted to do friedman.test on these groups. So I tried the following code
my_friedman <- function(data, variable, by, random, ...) {
data <- data[c(variable, by)] %>% dplyr::filter(complete.cases(.))
friedman.test(variable ~ data[[by]] | random, data = data)
}
smpl %>% select(c(Participant.ID, value, Day)) %>%
tbl_summary(by = Day) %>%
add_p(test = value ~ "my_friedman", random = Participant.ID)
But that gives an error..
this is the dput for my sample data smpl
structure(list(Participant.ID = c(1002, 1007, 1010, 1017, 1018,
1022, 1044, 1050, 1051, 1052, 1002, 1007, 1010, 1017, 1018, 1022,
1044, 1050, 1051, 1052, 1002, 1007, 1010, 1017, 1018, 1022, 1044,
1050, 1051, 1052), Randomization = c("Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone"
), Day = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L), .Label = c("0", "3", "7"), class = "factor"),
value = c(31.92, 53.25, 34.69, 33.32, 25.94, 34.69, 22.65,
33.32, 24.33, 25.94, 31.92, 46.54, 44.74, 145.12, 37.34,
39.89, 22.65, 27.5, 25.94, 25.94, 37.34, 53.25, 29.02, 37.34,
25.94, 34.69, 22.65, 27.5, 29.02, 31.92)), row.names = c(NA,
-30L), class = c("tbl_df", "tbl", "data.frame"))
Can someone kindly help?
Here is the direct call to friedman.test
> friedman.test(value ~ Day | Participant.ID, data = aa)
Friedman rank sum test
data: value and Day and Participant.ID
Friedman chi-squared = 1.2667, df = 2, p-value = 0.5308
Your code was close! I made some small adjustments. See below!
library(gtsummary)
#> #BlackLivesMatter
smpl <-
structure(list(Participant.ID = c(1002, 1007, 1010, 1017, 1018,
1022, 1044, 1050, 1051, 1052, 1002, 1007, 1010, 1017, 1018, 1022,
1044, 1050, 1051, 1052, 1002, 1007, 1010, 1017, 1018, 1022, 1044,
1050, 1051, 1052), Randomization = c("Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone",
"Pioglitazone", "Pioglitazone", "Pioglitazone", "Pioglitazone"
), Day = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L,
3L, 3L, 3L, 3L), .Label = c("0", "3", "7"), class = "factor"),
value = c(31.92, 53.25, 34.69, 33.32, 25.94, 34.69, 22.65,
33.32, 24.33, 25.94, 31.92, 46.54, 44.74, 145.12, 37.34,
39.89, 22.65, 27.5, 25.94, 25.94, 37.34, 53.25, 29.02, 37.34,
25.94, 34.69, 22.65, 27.5, 29.02, 31.92)), row.names = c(NA,
-30L), class = c("tbl_df", "tbl", "data.frame"))
my_friedman <- function(data, variable, by, group, ...) {
# construct the formula
formula <- stringr::str_glue("{variable} ~ {by} | {group}") |> as.formula()
# perform Friedman test
friedman.test(
formula = formula,
data = data
) |>
broom::tidy()
}
my_friedman(data = smpl, variable = "value", by = "Day", group = "Participant.ID")
#> # A tibble: 1 × 4
#> statistic p.value parameter method
#> <dbl> <dbl> <dbl> <chr>
#> 1 1.27 0.531 2 Friedman rank sum test
tbl <-
smpl %>%
select(Participant.ID, value, Day) %>%
tbl_summary(
by = Day,
include = -Participant.ID
) %>%
add_p(test = value ~ "my_friedman", group = Participant.ID)
Created on 2023-01-08 with reprex v2.0.2

Any other options besides the traditional CLD bar graph?

I am looking for an alternative approach to plotting results from pairwise comparisons than traditional bar plots. If possible, I would like to create a plot like the one shown below [1], but for a model that includes an interaction effect. R code for the plot below is online [2]. Is there a way to revise or add onto this code to include an interaction effect?
Example of my data set (too large to include in its entirety but I can send upon request) and the model used:
aq <- tibble::tribble(
~trt, ~season, ~site, ~sp,
"herbicide", "early", 1L, 0.120494496,
"herbicide", "early", 1L, 0.04057757,
"herbicide", "early", 1L, 0.060556802,
"herbicide", "early", 1L, 0.050567186,
"herbicide", "early", 1L, 0.110504881,
"herbicide", "early", 1L, 0.090525649,
"herbicide", "early", 1L, 0.100515265,
"herbicide", "early", 1L, 0.030587954,
"herbicide", "early", 1L, 0.080536033,
"herbicide", "early", 1L, 0.010608723,
"herbicide", "early", 1L, 0.080536033,
"herbicide", "early", 1L, 0.04057757,
"herbicide", "mid", 1L, 0.050567186,
"herbicide", "mid", 1L, 0.050567186,
"herbicide", "mid", 1L, 0.04057757,
"herbicide", "mid", 1L, 0.04057757,
"herbicide", "mid", 1L, 0.140473728,
"herbicide", "mid", 1L, 0.030587954,
"herbicide", "mid", 1L, 0.150463344,
"herbicide", "mid", 1L, 0.020598339,
"herbicide", "mid", 1L, 0.120494496,
"herbicide", "mid", 1L, 0.04057757,
"herbicide", "mid", 1L, 0.050567186,
"herbicide", "late", 1L, 0.090525649,
"herbicide", "late", 1L, 0.070546417,
"herbicide", "late", 1L, 0.150463344,
"herbicide", "late", 1L, 0.070546417,
"herbicide", "late", 1L, 0.220390654,
"herbicide", "late", 1L, 0.120494496,
"herbicide", "late", 1L, 0.150463344,
"herbicide", "late", 1L, 0.130484112,
"herbicide", "late", 1L, 0.090525649,
"herbicide", "late", 1L, 0.020598339,
"herbicide", "late", 1L, 0.170442575,
"herbicide", "late", 1L, 0.050567186,
"herbicide", "early", 1L, 0.010608723,
"herbicide", "early", 1L, 0.060556802,
"herbicide", "early", 1L, 0.000619107,
"herbicide", "early", 1L, 0.050567186,
"herbicide", "early", 1L, 0.030587954,
"herbicide", "early", 1L, 0.010608723,
"herbicide", "early", 1L, 0.000619107,
"herbicide", "early", 1L, 0.000619107,
"herbicide", "early", 1L, 0.020598339,
"herbicide", "early", 1L, 0.000619107,
"herbicide", "early", 1L, 0.030587954,
"herbicide", "early", 1L, 0.010608723,
"herbicide", "mid", 1L, 0.04057757,
"herbicide", "mid", 1L, 0.050567186,
"herbicide", "mid", 1L, 0.010608723,
"herbicide", "mid", 1L, 0.010608723,
"herbicide", "mid", 1L, 0.04057757,
"herbicide", "mid", 1L, 0.010608723,
"herbicide", "mid", 1L, 0.050567186,
"herbicide", "mid", 1L, 0.010608723,
"herbicide", "mid", 1L, 0.010608723,
"herbicide", "mid", 1L, 0.070546417,
"herbicide", "mid", 1L, 0.020598339,
"herbicide", "mid", 1L, 0.060556802,
"herbicide", "late", 1L, 0.030587954,
"herbicide", "late", 1L, 0.030587954,
"herbicide", "late", 1L, 0.070546417,
"herbicide", "late", 1L, 0.04057757,
"herbicide", "late", 1L, 0.010608723,
"herbicide", "late", 1L, 0.080536033,
"herbicide", "late", 1L, 0.000619107,
"herbicide", "late", 1L, 0.010608723,
"herbicide", "late", 1L, 0.010608723,
"herbicide", "late", 1L, 0.030587954,
"mow", "early", 1L, 0.050567186,
"mow", "early", 1L, 0.050567186,
"mow", "early", 1L, 0.04057757,
"mow", "early", 1L, 0.04057757,
"mow", "early", 1L, 0.080536033,
"mow", "early", 1L, 0.050567186,
"mow", "early", 1L, 0.020598339,
"mow", "early", 1L, 0.060556802,
"mow", "early", 1L, 0.000619107,
"mow", "early", 1L, 0.04057757,
"mow", "early", 1L, 0.050567186,
"mow", "early", 1L, 0.020598339,
"mow", "mid", 1L, 0.020598339,
"mow", "mid", 1L, 0.020598339,
"mow", "mid", 1L, 0.070546417,
"mow", "mid", 1L, 0.020598339,
"mow", "mid", 1L, 0.04057757,
"mow", "mid", 1L, 0.04057757,
"mow", "mid", 1L, 0.020598339,
"mow", "mid", 1L, 0.020598339,
"mow", "mid", 1L, 0.030587954,
"mow", "mid", 1L, 0.010608723,
"mow", "mid", 1L, 0.010608723,
"mow", "late", 1L, 0.04057757,
"mow", "late", 1L, 0.020598339,
"mow", "late", 1L, 0.04057757,
"mow", "late", 1L, 0.020598339,
"mow", "late", 1L, 0.020598339,
"mow", "late", 1L, 0.020598339,
"mow", "late", 1L, 0.030587954,
"mow", "late", 1L, 0.030587954,
"mow", "late", 1L, 0.020598339,
"mow", "late", 1L, 0.000619107,
"mow", "late", 1L, 0.030587954,
"mow", "late", 1L, 0.030587954,
"mow", "early", 1L, 0.050567186,
"mow", "early", 1L, 0.010608723,
"mow", "early", 1L, 0.100515265,
"mow", "early", 1L, 0.110504881,
"mow", "early", 1L, 0.04057757,
"mow", "early", 1L, 0.030587954,
"mow", "early", 1L, 0.050567186,
"mow", "early", 1L, 0.04057757,
"mow", "early", 1L, 0.050567186,
"mow", "early", 1L, 0.010608723,
"mow", "early", 1L, 0.010608723,
"mow", "early", 1L, 0.000619107,
"mow", "mid", 1L, 0.060556802,
"mow", "mid", 1L, 0.010608723,
"mow", "mid", 1L, 0.000619107,
"mow", "mid", 1L, 0.030587954,
"mow", "mid", 1L, 0.060556802,
"mow", "mid", 1L, 0.020598339,
"mow", "mid", 1L, 0.050567186,
"mow", "mid", 1L, 0.04057757,
"mow", "mid", 1L, 0.020598339,
"mow", "mid", 1L, 0.04057757,
"mow", "mid", 1L, 0.030587954,
"mow", "mid", 1L, 0.030587954,
"mow", "late", 1L, 0.050567186,
"mow", "late", 1L, 0.050567186,
"mow", "late", 1L, 0.010608723,
"mow", "late", 1L, 0.030587954,
"mow", "late", 1L, 0.010608723,
"mow", "late", 1L, 0.010608723,
"mow", "late", 1L, 0.060556802,
"mow", "late", 1L, 0.020598339,
"mow", "late", 1L, 0.050567186,
"mow", "late", 1L, 0.04057757,
"mow", "late", 1L, 0.010608723,
"mow", "late", 1L, 0.070546417,
"herbicide", "early", 2L, 0.04057757,
"herbicide", "early", 2L, 0.450151817,
"herbicide", "early", 2L, 0.000619107,
"herbicide", "early", 2L, 0.500099896,
"herbicide", "early", 2L, 0.010608723,
"herbicide", "early", 2L, 0.190421807,
"herbicide", "early", 2L, 0.180432191,
"herbicide", "early", 2L, 0.130484112,
"herbicide", "early", 2L, 0.020598339,
"herbicide", "early", 2L, 0.360245275,
"herbicide", "early", 2L, 0.010608723,
"herbicide", "early", 2L, 0.030587954,
"herbicide", "mid", 2L, 0.050567186,
"herbicide", "mid", 2L, 0.370234891,
"herbicide", "mid", 2L, 0.010608723,
"herbicide", "mid", 2L, 0.250359502,
"herbicide", "mid", 2L, 0.050567186,
"herbicide", "mid", 2L, 0.080536033,
"herbicide", "mid", 2L, 0.04057757,
"herbicide", "mid", 2L, 0.050567186,
"herbicide", "mid", 2L, 0.050567186,
"herbicide", "mid", 2L, 0.16045296,
"herbicide", "mid", 2L, 0.000619107,
"herbicide", "mid", 2L, 0.000619107,
"herbicide", "late", 2L, 0.050567186,
"herbicide", "late", 2L, 0.540058359,
"herbicide", "late", 2L, 0.04057757,
"herbicide", "late", 2L, 0.260349117,
"herbicide", "late", 2L, 0.070546417,
"herbicide", "late", 2L, 0.120494496,
"herbicide", "late", 2L, 0.030587954,
"herbicide", "late", 2L, 0.070546417,
"herbicide", "late", 2L, 0.020598339,
"herbicide", "late", 2L, 0.120494496,
"herbicide", "late", 2L, 0.04057757,
"herbicide", "late", 2L, 0.000619107,
"herbicide", "early", 2L, 0.010608723,
"herbicide", "early", 2L, 0.050567186,
"herbicide", "early", 2L, 0.010608723,
"herbicide", "early", 2L, 0.010608723,
"herbicide", "early", 2L, 0.060556802,
"herbicide", "early", 2L, 0.04057757,
"herbicide", "early", 2L, 0.210401038,
"herbicide", "early", 2L, 0.060556802,
"herbicide", "early", 2L, 0.100515265,
"herbicide", "early", 2L, 0.090525649,
"herbicide", "early", 2L, 0.010608723,
"herbicide", "early", 2L, 0.000619107,
"herbicide", "mid", 2L, 0.060556802,
"herbicide", "mid", 2L, 0.020598339,
"herbicide", "mid", 2L, 0.030587954,
"herbicide", "mid", 2L, 0.010608723,
"herbicide", "mid", 2L, 0.000619107,
"herbicide", "mid", 2L, 0.010608723,
"herbicide", "mid", 2L, 0.030587954,
"herbicide", "mid", 2L, 0.070546417,
"herbicide", "mid", 2L, 0.020598339,
library(tidyverse)
library(betareg)
library(emmeans)
library(lmtest)
library(multcomp)
library(lme4)
library(car)
library(glmmTMB)
trt_key <- c(ctrl = "Control", mow = "FallMow", herbicide = "SpotSpray")
aq$trt <- recode(aq$trt, !!!trt_key)
aq$trt <- factor(aq$trt, levels = c("Control", "FallMow", "SpotSpray"))
season_key <- c(early = "Early", mid = "Mid", late = "Late")
aq$season <- recode(aq$season, !!!season_key)
aq$season <- factor(aq$season, levels=c("Early","Mid","Late"))
glm.soil <- glmmTMB(sp ~ trt + season + trt*season + (1 | site), data = aq,
family = list(family = "beta", link = "logit"), dispformula = ~trt)
#Interaction
lsm <- emmeans(glm.soil, pairwise ~ trt:season, type="response", adjust = "tukey")
lsmtab <- cld(lsm, Letter=letters, sort = F)
colnames(lsmtab)[1] <- "Treatment"
colnames(lsmtab)[2] <- "Season"
colnames(lsmtab)[8] <- "letter"
df <- as.data.frame(lsmtab)
print(df)
This is my first post, so I apologize in advance if I've overlooked any posting protocols. Thanks!
[1]: https://i.stack.imgur.com/GJ8VA.png
[2]: https://schmidtpaul.github.io/DSFAIR/compactletterdisplay.html
I'm the author of the plot/code you linked.
You are not the first one asking how to create an analogous plot when interactions are present. I am suggesting two options below using your data.
(Note that in the following reprex I deleted the part of the code with the data, because my post was reaching the character limit.)
# packages ---------------------------------------------------------------
library(emmeans)
library(glmmTMB)
library(car)
library(multcomp)
library(multcompView)
library(scales)
library(tidyverse)
# format ------------------------------------------------------------------
trt_key <- c(Control = "ctrl", FallMow = "mow", SpotSpray = "herbicide")
season_key <- c(Early = "early", Mid = "mid", Late = "late")
aq <- aq %>%
mutate(
trt = trt %>% fct_recode(!!!trt_key) %>% fct_relevel(names(trt_key)),
season = season %>% fct_recode(!!!season_key) %>% fct_relevel(names(season_key))
)
# model -------------------------------------------------------------------
glm.soil <-
glmmTMB(
sp ~ trt + season + trt:season + (1 | site),
data = aq,
family = list(family = "beta", link = "logit"),
dispformula = ~ trt
)
Anova(glm.soil) # interaction is significant!
#> Analysis of Deviance Table (Type II Wald chisquare tests)
#>
#> Response: sp
#> Chisq Df Pr(>Chisq)
#> trt 48.422 2 3.057e-11 ***
#> season 18.888 2 7.916e-05 ***
#> trt:season 16.980 4 0.001951 **
#> ---
#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Here the ANOVA tells us that there are significant interaction effects between treatment and season. Simply put, this means that treatments behave differently depending on the season. In the extreme case this could mean that the same treatment could have the best performance in one season, but the worst performance in another season. Therefore it would be misleading to simply estimate one mean per treatment across seasons (or one mean per season across treatments). Instead, we should look at the means of all treatment-season combinations. In this scenario, there are 9 season-treatment combinations and we can estimate them using emmeans() via either ~ trt:season or ~ trt|season. These are the two options I was talking about above. Again, both estimate the same means for the same 9 combinations, but what is different is which of these means are compared to each other. None of the two appraoches is more correct than the other. Instead, you as the analyst must decide which approach is more informative for what you are trying to investigate.
# emmean comparisons ------------------------------------------------------
emm1 <- emmeans(glm.soil, ~ trt:season, type = "response")
emm2 <- emmeans(glm.soil, ~ trt|season, type = "response")
cld1 <- cld(emm1, Letter=letters, adjust = "tukey")
cld2 <- cld(emm2, Letter=letters, adjust = "tukey")
~ trt:season Comparing all combinations to all other combinations
Here, all combinations are compared to all other combinations. In order to plot this, I create a new column trt_season that represents each combination (and I also sort its factor levels according to their estimated mean) and put it on the x-axis. Note that I also filled the boxes and colored the dots according to their treatment, but this is optional.
cld1
#> trt season response SE df lower.CL upper.CL .group
#> Control Early 0.0560 0.00540 950 0.0428 0.0730 abc
#> FallMow Early 0.0316 0.00248 950 0.0254 0.0393 d
#> SpotSpray Early 0.0498 0.00402 950 0.0397 0.0622 abc
#> Control Mid 0.0654 0.00610 950 0.0504 0.0845 a
#> FallMow Mid 0.0427 0.00305 950 0.0350 0.0520 b
#> SpotSpray Mid 0.0609 0.00473 950 0.0490 0.0754 a c
#> Control Late 0.0442 0.00464 950 0.0330 0.0590 bcd
#> FallMow Late 0.0437 0.00314 950 0.0358 0.0533 b
#> SpotSpray Late 0.0630 0.00482 950 0.0508 0.0777 a c
#>
#> Confidence level used: 0.95
#> Conf-level adjustment: sidak method for 9 estimates
#> Intervals are back-transformed from the logit scale
#> P value adjustment: tukey method for comparing a family of 9 estimates
#> Tests are performed on the log odds ratio scale
#> significance level used: alpha = 0.05
#> NOTE: Compact letter displays can be misleading
#> because they show NON-findings rather than findings.
#> Consider using 'pairs()', 'pwpp()', or 'pwpm()' instead.
cld1_df <- cld1 %>% as.data.frame()
cld1_df <- cld1_df %>%
mutate(trt_season = paste0(season, "-", trt)) %>%
mutate(trt_season = fct_reorder(trt_season, response))
aq <- aq %>%
mutate(trt_season = paste0(season, "-", trt)) %>%
mutate(trt_season = fct_relevel(trt_season, levels(cld1_df$trt_season)))
ggplot() +
# y-axis
scale_y_continuous(
name = "sp",
limits = c(0, NA),
breaks = pretty_breaks(),
expand = expansion(mult = c(0, 0.1))
) +
# x-axis
scale_x_discrete(name = "Treatment-Season combination") +
# general layout
theme_bw() +
theme(axis.text.x = element_text(
angle = 45,
hjust = 1,
vjust = 1
),
legend.position = "top") +
# black data points
geom_point(
data = aq,
aes(y = sp, x = trt_season, color = trt),
#shape = 16,
alpha = 0.5,
position = position_nudge(x = -0.2)
) +
# black boxplot
geom_boxplot(
data = aq,
aes(y = sp, x = trt_season, fill = trt),
width = 0.05,
outlier.shape = NA,
position = position_nudge(x = -0.1)
) +
# red mean value
geom_point(
data = cld1_df,
aes(y = response, x = trt_season),
size = 2,
color = "red"
) +
# red mean errorbar
geom_errorbar(
data = cld1_df,
aes(ymin = lower.CL, ymax = upper.CL, x = trt_season),
width = 0.05,
color = "red"
) +
# red letters
geom_text(
data = cld1_df,
aes(
y = response,
x = trt_season,
label = str_trim(.group)
),
position = position_nudge(x = 0.1),
hjust = 0,
color = "red"
) +
labs(
caption = str_wrap("Black dots represent raw data. Red dots and error bars represent backtransformed estimated marginal means ± 95% confidence interval per group. Means not sharing any letter are significantly different by the Tukey-test at the 5% level of significance.", width = 100)
)
~ trt|season Comparing all trt combinations only within each season
Here, fewer mean comparisons are made, i.e. only 3 comparisons (Control vs. FallMow, FallMow vs. SpotSpray, Control vs. SpotSpray) for each of the seasons. This means that e.g. Early-Control is never compared to Mid-Control. Moreover, the letters from the compact letter display are also created separately for each season. This means that e.g. the a assigned to the Early-Control mean has nothing to do with the a assigned to the Mid-Control mean. This is crucial and I made sure to explicitly state this in the plot's caption. I also used facettes which separate the results for the seasons visually.
That being said, presenting the results in this way may actually be more suited for your investigation. (Obviously you could use colors per treatment or season here as well)
cld2
#> season = Early:
#> trt response SE df lower.CL upper.CL .group
#> Control 0.0560 0.00540 950 0.0444 0.0704 a
#> FallMow 0.0316 0.00248 950 0.0262 0.0381 b
#> SpotSpray 0.0498 0.00402 950 0.0410 0.0603 a
#>
#> season = Mid:
#> trt response SE df lower.CL upper.CL .group
#> Control 0.0654 0.00610 950 0.0522 0.0816 a
#> FallMow 0.0427 0.00305 950 0.0360 0.0506 b
#> SpotSpray 0.0609 0.00473 950 0.0505 0.0733 a
#>
#> season = Late:
#> trt response SE df lower.CL upper.CL .group
#> Control 0.0442 0.00464 950 0.0344 0.0567 a
#> FallMow 0.0437 0.00314 950 0.0368 0.0519 a
#> SpotSpray 0.0630 0.00482 950 0.0524 0.0755 b
#>
#> Confidence level used: 0.95
#> Conf-level adjustment: sidak method for 3 estimates
#> Intervals are back-transformed from the logit scale
#> P value adjustment: tukey method for comparing a family of 3 estimates
#> Tests are performed on the log odds ratio scale
#> significance level used: alpha = 0.05
#> NOTE: Compact letter displays can be misleading
#> because they show NON-findings rather than findings.
#> Consider using 'pairs()', 'pwpp()', or 'pwpm()' instead.
cld2_df <- cld2 %>% as.data.frame()
ggplot() +
facet_grid(cols = vars(season), labeller = label_both) +
# y-axis
scale_y_continuous(
name = "sp",
limits = c(0, NA),
breaks = pretty_breaks(),
expand = expansion(mult = c(0, 0.1))
) +
# x-axis
scale_x_discrete(name = "Treatment") +
# general layout
theme_bw() +
# black data points
geom_point(
data = aq,
aes(y = sp, x = trt),
shape = 16,
alpha = 0.5,
position = position_nudge(x = -0.2)
) +
# black boxplot
geom_boxplot(
data = aq,
aes(y = sp, x = trt),
width = 0.05,
outlier.shape = NA,
position = position_nudge(x = -0.1)
) +
# red mean value
geom_point(
data = cld2_df,
aes(y = response, x = trt),
size = 2,
color = "red"
) +
# red mean errorbar
geom_errorbar(
data = cld2_df,
aes(ymin = lower.CL, ymax = upper.CL, x = trt),
width = 0.05,
color = "red"
) +
# red letters
geom_text(
data = cld2_df,
aes(
y = response,
x = trt,
label = str_trim(.group)
),
position = position_nudge(x = 0.1),
hjust = 0,
color = "red"
) +
labs(
caption = str_wrap("Black dots represent raw data. Red dots and error bars represent backtransformed estimated marginal means ± 95% confidence interval per group. For each season separately, means not sharing any letter are significantly different by the Tukey-test at the 5% level of significance.", width = 100)
)
So these are the two options I think about when I have a significant two-way interaction and want to compare means. Note that you could also switch treatment and season to ~ season|trt and also plot it the other way around.
Further reading
emmeans documentation
chapter on Compact Letter Display (CLD) - you will also find a brief discussion on the weakness of CLDs that Russ Lenth mentioned in his comment
related stackoverflow post #1
related stackoverflow post #2
Bonus: Raincloud plot
This has nothing to do with your question, but I'd like to point out that you have much more data points than I have in the the plot you linked. Because of this, the original plotting approach could potentially be improved because geom_point() leads to many dots being plotted on top of each other so that we have no way seeing how much data there really is. Therefore, you could simply replace geom_point() with geom_jitter() or even go further and create these raincloud plots mentioned in this blogpost. I've created one (without the emmeans part) below that is analogous to the first plot above.
ggplot() +
# y-axis
scale_y_continuous(
name = "sp",
limits = c(0, NA),
breaks = pretty_breaks(),
expand = expansion(mult = c(0, 0.1))
) +
# x-axis
scale_x_discrete(name = "Treatment-Season combination") +
# general layout
theme_bw() +
theme(axis.text.x = element_text(
angle = 45,
hjust = 1,
vjust = 1
),
legend.position = "top") +
# add half-violin from {ggdist} package
ggdist::stat_halfeye(
data = aq,
aes(y = sp, x = trt_season, fill = trt),
adjust = .5,
width = .5,
.width = 0,
justification = -.2,
point_colour = NA,
show.legend = FALSE
) +
# boxplot
geom_boxplot(
data = aq,
aes(y = sp, x = trt_season, fill = trt),
width = 0.1,
outlier.shape = NA
) +
# add justified jitter from the {gghalves} package
gghalves::geom_half_point(
data = aq,
aes(y = sp, x = trt_season, color = trt),
side = "l",
range_scale = .4,
alpha = .2
)
Created on 2022-01-26 by the reprex package (v2.0.1)
IMO, almost anything is better than a CLD. They display non-findings rather than findings.
I suggest presenting the simple comparisons in tabular form
lsm = emmeans(glm.soil, ~ trt:season, type = "response")
pairs(lsm, by = "trt")
pairs(lsm, by = "season")
If you really want the diagonal comparisons, you could consider pwpp() or pwpm() which can display quite a few comparisons compactly. See the documentation in emmeans

org.postgresql.util.PSQLException: Large Objects may not be used in auto-commit mode Error with JBPM 4.4

We are using Jbpm 4.4 as our 3rd party Business Process Management tool with Java 6.x. However So far we used it with Oracle DB and it worked well, but now we want to run it with PostgreSQL 12.x version DB.
So we integrated postgresql-42.2.19.jre6.jar (JDBC driver) and try to run it.
We have encountered below error in the run time.
Can anyone suggest what need to be done here to resolve the issue, specially with JBPM 4.4
We have already set
<prop key="hibernate.connection.autocommit">false</prop>
But that did not resolved our issue.
2021-05-05 06:41:57,670 ERROR [o-8443-exec-154] .AbstractFlushingEventListener portaladmin#10.100.250.41 - Could not synchronize database state with session
org.hibernate.exception.GenericJDBCException: could not insert: [org.jbpm.pvm.internal.lob.Lob]
at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:126) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:114) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2295) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2688) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.action.EntityInsertAction.execute(EntityInsertAction.java:79) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:279) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:263) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:64) [hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:996) [hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1141) [hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102) [hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.jbpm.pvm.internal.query.AbstractQuery.execute(AbstractQuery.java:93) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.query.ProcessDefinitionQueryImpl.execute(ProcessDefinitionQueryImpl.java:67) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.query.AbstractQuery.untypedList(AbstractQuery.java:67) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.query.ProcessDefinitionQueryImpl.list(ProcessDefinitionQueryImpl.java:157) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.repository.ProcessDeployer.checkKey(ProcessDeployer.java:133) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.repository.ProcessDeployer.deploy(ProcessDeployer.java:92) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.repository.DeployerManager.deploy(DeployerManager.java:46) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.repository.RepositorySessionImpl.deploy(RepositorySessionImpl.java:62) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.cmd.DeployCmd.execute(DeployCmd.java:47) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.cmd.DeployCmd.execute(DeployCmd.java:33) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.svc.DefaultCommandService.execute(DefaultCommandService.java:42) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.tx.SpringCommandCallback.doInTransaction(SpringCommandCallback.java:45) [jbpm-pvm-4.4.jar:4.4]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130) [spring-tx-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.jbpm.pvm.internal.tx.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:49) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.svc.EnvironmentInterceptor.executeInNewEnvironment(EnvironmentInterceptor.java:53) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.svc.EnvironmentInterceptor.execute(EnvironmentInterceptor.java:40) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.svc.RetryInterceptor.execute(RetryInterceptor.java:56) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.svc.SkipInterceptor.execute(SkipInterceptor.java:43) [jbpm-pvm-4.4.jar:4.4]
at org.jbpm.pvm.internal.repository.DeploymentImpl.deploy(DeploymentImpl.java:90) [jbpm-pvm-4.4.jar:4.4]
at com.abc.def.portal.processes.jbpm.JbpmProcessDefinitionRepository.deployProcess_aroundBody18(JbpmProcessDefinitionRepository.java:108) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessDefinitionRepository.deployProcess_aroundBody19$advice(JbpmProcessDefinitionRepository.java:92) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessDefinitionRepository.deployProcess_aroundBody20(JbpmProcessDefinitionRepository.java:1) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessDefinitionRepository.deployProcess_aroundBody22(JbpmProcessDefinitionRepository.java:106) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessDefinitionRepository.deployProcess_aroundBody23$advice(JbpmProcessDefinitionRepository.java:80) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessDefinitionRepository.deployProcess(JbpmProcessDefinitionRepository.java:1) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessService.deployProcess_aroundBody46(JbpmProcessService.java:178) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessService.deployProcess_aroundBody47$advice(JbpmProcessService.java:92) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessService.deployProcess_aroundBody48(JbpmProcessService.java:1) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessService.deployProcess_aroundBody50(JbpmProcessService.java:178) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessService.deployProcess_aroundBody51$advice(JbpmProcessService.java:80) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessService.deployProcess_aroundBody52(JbpmProcessService.java:1) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessService.deployProcess_aroundBody53$advice(JbpmProcessService.java:61) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.processes.jbpm.JbpmProcessService.deployProcess(JbpmProcessService.java:1) [com.abc.def.portal.processes-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.partner.client.task.TaskController.handleFormUpload_aroundBody128(TaskController.java:611) [TaskController.class:na]
at com.abc.def.portal.partner.client.task.TaskController.handleFormUpload_aroundBody129$advice(TaskController.java:58) [TaskController.class:na]
at com.abc.def.portal.partner.client.task.TaskController.handleFormUpload_aroundBody130(TaskController.java:1) [TaskController.class:na]
at com.abc.def.portal.partner.client.task.TaskController.handleFormUpload_aroundBody131$advice(TaskController.java:92) [TaskController.class:na]
at com.abc.def.portal.partner.client.task.TaskController.handleFormUpload_aroundBody132(TaskController.java:1) [TaskController.class:na]
at com.abc.def.portal.partner.client.task.TaskController.handleFormUpload_aroundBody134(TaskController.java:605) [TaskController.class:na]
at com.abc.def.portal.partner.client.task.TaskController.handleFormUpload_aroundBody135$advice(TaskController.java:102) [TaskController.class:na]
at com.abc.def.portal.partner.client.task.TaskController.handleFormUpload_aroundBody136(TaskController.java:1) [TaskController.class:na]
at com.abc.def.portal.partner.client.task.TaskController.handleFormUpload_aroundBody137$advice(TaskController.java:55) [TaskController.class:na]
at com.abc.def.portal.partner.client.task.TaskController.handleFormUpload(TaskController.java:1) [TaskController.class:na]
at com.abc.def.portal.partner.client.task.TaskController$$FastClassByCGLIB$$2349406.invoke(<generated>) [cglib-nodep-2.1_3.jar:na]
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149) [cglib-nodep-2.1_3.jar:na]
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689) [spring-aop-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) [spring-aop-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:67) [spring-security-core-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) [spring-aop-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622) [spring-aop-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at com.abc.def.portal.partner.client.task.TaskController$$EnhancerByCGLIB$$4f295537.handleFormUpload(<generated>) [cglib-nodep-2.1_3.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_45]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) ~[na:1.6.0_45]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_45]
at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_45]
at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96) [spring-webmvc-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617) [spring-webmvc-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578) [spring-webmvc-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80) [spring-webmvc-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923) [spring-webmvc-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852) [spring-webmvc-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882) [spring-webmvc-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789) [spring-webmvc-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:646) [servlet-api.jar:na]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api.jar:na]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) [catalina.jar:7.0.53]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.53]
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.53]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.53]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:369) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at com.abc.def.portal.partner.client.security.IncompleteUserProfileFilter.doFilterInternal_aroundBody4(IncompleteUserProfileFilter.java:108) [IncompleteUserProfileFilter.class:na]
at com.abc.def.portal.partner.client.security.IncompleteUserProfileFilter.doFilterInternal(IncompleteUserProfileFilter.java:89) [IncompleteUserProfileFilter.class:na]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:97) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:100) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:78) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:35) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:187) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at com.abc.def.portal.ui.servlet.SsoRequestHeaderAuthenticationFilter.doFilter_aroundBody2(SsoRequestHeaderAuthenticationFilter.java:63) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.ui.servlet.SsoRequestHeaderAuthenticationFilter.doFilter(SsoRequestHeaderAuthenticationFilter.java:58) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:79) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:109) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:381) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:168) [spring-security-web-3.0.7.RELEASE.jar:3.0.7.RELEASE]
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.53]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.53]
at com.abc.def.portal.partner.client.security.SSOAutoLoginFilter.doFilterInternal_aroundBody0(SSOAutoLoginFilter.java:67) [SSOAutoLoginFilter.class:na]
at com.abc.def.portal.partner.client.security.SSOAutoLoginFilter.doFilterInternal(SSOAutoLoginFilter.java:63) [SSOAutoLoginFilter.class:na]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.53]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.53]
at com.abc.def.portal.ui.csrf.CsrfFilter.doFilterInternal_aroundBody0(CsrfFilter.java:86) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.ui.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:57) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.53]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.53]
at com.abc.def.portal.ui.csrf.AjaxTimeoutFilter.doFilterInternal_aroundBody0(AjaxTimeoutFilter.java:45) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.ui.csrf.AjaxTimeoutFilter.doFilterInternal(AjaxTimeoutFilter.java:31) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.53]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.53]
at com.abc.def.portal.ui.timing.TimingServletFilter.doFilter_aroundBody2(TimingServletFilter.java:71) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.ui.timing.TimingServletFilter.doFilter(TimingServletFilter.java:63) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.53]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.53]
at com.abc.def.portal.ui.servlet.XFilter.doFilterInternal_aroundBody0(XFilter.java:56) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.ui.servlet.XFilter.doFilterInternal_aroundBody1$advice(XFilter.java:64) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at com.abc.def.portal.ui.servlet.XFilter.doFilterInternal(XFilter.java:51) [com.abc.def.portal.ui-2.1.NOPSE19C.1.jar:na]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.53]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.53]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) [spring-web-3.1.2.RELEASE.jar:3.1.2.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.53]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.53]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) [catalina.jar:7.0.53]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) [catalina.jar:7.0.53]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:610) [catalina.jar:7.0.53]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) [catalina.jar:7.0.53]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) [catalina.jar:7.0.53]
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) [catalina.jar:7.0.53]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) [catalina.jar:7.0.53]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) [catalina.jar:7.0.53]
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040) [tomcat-coyote.jar:7.0.53]
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607) [tomcat-coyote.jar:7.0.53]
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313) [tomcat-coyote.jar:7.0.53]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [na:1.6.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [na:1.6.0_45]
at java.lang.Thread.run(Thread.java:662) [na:1.6.0_45]
Caused by: org.postgresql.util.PSQLException: Large Objects may not be used in auto-commit mode.
at org.postgresql.largeobject.LargeObjectManager.createLO(LargeObjectManager.java:284) ~[postgresql-42.2.19.jre6.jar:42.2.19.jre6]
at org.postgresql.largeobject.LargeObjectManager.createLO(LargeObjectManager.java:272) ~[postgresql-42.2.19.jre6.jar:42.2.19.jre6]
at org.postgresql.jdbc.PgPreparedStatement.createBlob(PgPreparedStatement.java:1159) ~[postgresql-42.2.19.jre6.jar:42.2.19.jre6]
at org.postgresql.jdbc.PgPreparedStatement.setBlob(PgPreparedStatement.java:1200) ~[postgresql-42.2.19.jre6.jar:42.2.19.jre6]
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.setBlob(NewProxyPreparedStatement.java:495) ~[c3p0-0.9.1.2.jar:0.9.1.2]
at org.hibernate.type.BlobType.set(BlobType.java:72) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.type.BlobType.nullSafeSet(BlobType.java:140) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.persister.entity.AbstractEntityPersister.dehydrate(AbstractEntityPersister.java:2025) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:2271) ~[hibernate-core-3.3.1.GA.jar:3.3.1.GA]
... 160 common frames omitted
Though jbpm 4.4 is a very old version (currently at 7.54), try to update your schema and use bytea type for postgresql large objects.
If you're using JTA datasource, auto-commit setting is always true and it can not be changed. Try to change it to an xa-datasource
Solution:
I was able to find out a solution for this.
I have changed the DB column to Bytea type in PostgreSQL and change the JBPM 4.4 implementation to use byte[] over java.sql.Blob (in org.jbpm.pvm.internal.lob.Lob class).

How to create a xml file per records in spark scala

I have a file that has records like below
1_107570667_ANA_2C68EF2F-AB17-40EF-9095-387DE1D5D745_App.xml|<CAudit><ai2aiinst nT="LevFcf#A0" auNdSTy="Analytics" auNdTy="Identifier" ndNo="1" aId="1" conDes="Levered Free Cash Flow" conCd="LevFcf" aiaGUId="1_107570667_ANA_2C68EF2F-AB17-40EF-9095-387DE1D5D745" aiaId="1" aiKey="2990569588" aiId="14" pEndDt="2013-Dec-31" perCd="A" isYr2Dt="False" ><AudNode aId="1" ndNo="2" auNdTy="Operation" auNdSTy="-" nV="2626287569.000000000000000" ><AudNode aId="1" ndNo="3" auNdTy="Operation" auNdSTy="-" nV="2825849069.000000000000000" ><AudNode aId="1" ndNo="4" auNdTy="Identifier" auNdSTy="Standardized" nT="STD.SEBITDA#A0" nV="3130019939.000000000000000" ><ai2si nV="3130019939.00000" nT="STD.SEBITDA#A0" auNdSTy="Standardized" auNdTy="Identifier" ndNo="4" aId="1" inId="1035" conDes="Earnings before Interest, Taxes, Depreciation & Amortization (EBITDA)" conCd="SEBITDA" stdaGUId="841_107570667_STD_2C68EF2F-AB17-40EF-9095-387DE1D5D745" stdIaId="841" siKey="12004131416271429" siId="413" sLiCurIso="KRW" sCurIso="KRW" stCurIso="KRW" stTyCd="INC" sId="1" pEndDt="2013-Dec-31" pId="2" fId="192730348494" fbId="1" /></AudNode><AudNode aId="1" ndNo="5" auNdTy="Identifier" auNdSTy="Standardized" nT="STD.STAX#A0" nV="304170870.000000000000000" ><ai2si nV="304170870.00000" nT="STD.STAX#A0" auNdSTy="Standardized" auNdTy="Identifier" ndNo="5" aId="1" inId="968" conDes="Income Taxes" conCd="STAX" stdaGUId="807_107570667_STD_2C68EF2F-AB17-40EF-9095-387DE1D5D745" stdIaId="807" siKey="120038112041962629" siId="381" sLiCurIso="KRW" sCurIso="KRW" stCurIso="KRW" stTyCd="INC" sId="1" pEndDt="2013-Dec-31" pId="2" fId="192730348494" fbId="1" /></AudNode></AudNode><AudNode aId="1" ndNo="6" auNdTy="Operation" auNdSTy="SUM" nV="199561500.000000000000000" ><AudNode aId="1" ndNo="7" auNdTy="Identifier" auNdSTy="Standardized" nT="STD.SCEX#A0" nV="199561500.000000000000000" ><ai2si nV="199561500.00000" nT="STD.SCEX#A0" auNdSTy="Standardized" auNdTy="Identifier" ndNo="7" aId="1" inId="888" conDes="Capital Expenditures - Total" conCd="SCEX" stdaGUId="704_107570667_STD_2C68EF2F-AB17-40EF-9095-387DE1D5D745" stdIaId="704" siKey="12002771860094347" siId="277" sLiCurIso="KRW" sCurIso="KRW" stCurIso="KRW" stTyCd="CAS" sId="1" pEndDt="2013-Dec-31" pId="2" fId="192730348494" fbId="1" /></AudNode><AudNode aId="1" ndNo="8" auNdTy="Constant" nV="0.000000000000000" /></AudNode></AudNode></ai2aiinst></CAudit>
3_107570667_ANA_2C68EF2F-AB17-40EF-9095-387DE1D5D745_App.xml|<CAudit><ai2aiinst nT="ExcessCashMargin#A0" auNdSTy="Analytics" auNdTy="Identifier" ndNo="1" aId="3" conDes="Excess Cash Margin - %" conCd="ExcessCashMargin" aiaGUId="3_107570667_ANA_2C68EF2F-AB17-40EF-9095-387DE1D5D745" aiaId="3" aiKey="2990569579" aiId="5" pEndDt="2013-Dec-31" perCd="A" isYr2Dt="False" ><AudNode aId="3" ndNo="2" auNdTy="Operation" auNdSTy="*" nV="2.257160458878393" ><AudNode aId="3" ndNo="8" auNdTy="Identifier" auNdSTy="PseudoFinancialConcept" nT="PERCENTSCALE#A0" nV="100.000000000000000" /><AudNode aId="3" ndNo="3" auNdTy="Operation" auNdSTy="//" nV="0.022571604588784" ><AudNode aId="3" ndNo="7" auNdTy="Identifier" auNdSTy="Standardized" nT="STD.STLR#A0" nV="68201182151.000000000000000" ><ai2si nV="68201182151.00000" nT="STD.STLR#A0" auNdSTy="Standardized" auNdTy="Identifier" ndNo="7" aId="3" inId="990" conDes="Revenue from Business Activities - Total" conCd="STLR" stdaGUId="813_107570667_STD_2C68EF2F-AB17-40EF-9095-387DE1D5D745" stdIaId="813" siKey="12003871970759396" siId="387" sLiCurIso="KRW" sCurIso="KRW" stCurIso="KRW" stTyCd="INC" sId="1" pEndDt="2013-Dec-31" pId="2" fId="192730348494" fbId="1" /></AudNode><AudNode aId="3" ndNo="4" auNdTy="Operation" auNdSTy="-" nV="1539410116.000000000000000" ><AudNode aId="3" ndNo="6" auNdTy="Identifier" auNdSTy="Standardized" nT="STD.SNIC#A0" nV="438846856.000000000000000" ><ai2si nV="438846856.00000" nT="STD.SNIC#A0" auNdSTy="Standardized" auNdTy="Identifier" ndNo="6" aId="3" inId="1055" conDes="Net Income after Minority Interest" conCd="SNIC" stdaGUId="856_107570667_STD_2C68EF2F-AB17-40EF-9095-387DE1D5D745" stdIaId="856" siKey="120043012135950005" siId="430" sLiCurIso="KRW" sCurIso="KRW" stCurIso="KRW" stTyCd="INC" sId="1" pEndDt="2013-Dec-31" pId="2" fId="192730348494" fbId="1" /></AudNode><AudNode aId="3" ndNo="5" auNdTy="Identifier" auNdSTy="Standardized" nT="STD.STLO#A0" nV="1978256972.000000000000000" ><ai2si nV="1978256972.00000" nT="STD.STLO#A0" auNdSTy="Standardized" auNdTy="Identifier" ndNo="5" aId="3" inId="924" conDes="Net Cash Flow from Operating Activities" conCd="STLO" stdaGUId="719_107570667_STD_2C68EF2F-AB17-40EF-9095-387DE1D5D745" stdIaId="719" siKey="12002951348701451" siId="295" sLiCurIso="KRW" sCurIso="KRW" stCurIso="KRW" stTyCd="CAS" sId="1" pEndDt="2013-Dec-31" pId="2" fId="192730348494" fbId="1" /></AudNode></AudNode></AudNode></AudNode></ai2aiinst></CAudit>
5_107570667_ANA_2C68EF2F-AB17-40EF-9095-387DE1D5D745_App.xml|<CAudit><ai2aiinst nT="Cf#A0" auNdSTy="Analytics" auNdTy="Identifier" ndNo="1" aId="5" conDes="Cash Flow" conCd="Cf" aiaGUId="5_107570667_ANA_2C68EF2F-AB17-40EF-9095-387DE1D5D745" aiaId="5" aiKey="2990569577" aiId="3" pEndDt="2013-Dec-31" perCd="A" isYr2Dt="False" ><AudNode aId="5" ndNo="2" auNdTy="Operation" auNdSTy="-" nV="898935497.000000000000000" ><AudNode aId="5" ndNo="6" auNdTy="Constant" nV="0.000000000000000" /><AudNode aId="5" ndNo="3" auNdTy="Operation" auNdSTy="+" nV="898935497.000000000000000" ><AudNode aId="5" ndNo="5" auNdTy="Identifier" auNdSTy="Standardized" nT="STD.STDAE#A0" nV="460088641.000000000000000" ><ai2si nV="460088641.00000" nT="STD.STDAE#A0" auNdSTy="Standardized" auNdTy="Identifier" ndNo="5" aId="5" inId="956" conDes="Depreciation, Depletion & Amortization - Total" conCd="STDAE" stdaGUId="796_107570667_STD_2C68EF2F-AB17-40EF-9095-387DE1D5D745" stdIaId="796" siKey="120036611860540497" siId="366" sLiCurIso="KRW" sCurIso="KRW" stCurIso="KRW" stTyCd="INC" sId="1" pEndDt="2013-Dec-31" pId="2" fId="192730348494" fbId="1" /></AudNode><AudNode aId="5" ndNo="4" auNdTy="Identifier" auNdSTy="Standardized" nT="STD.SIAT#A0" nV="438846856.000000000000000" ><ai2si nV="438846856.00000" nT="STD.SIAT#A0" auNdSTy="Standardized" auNdTy="Identifier" ndNo="4" aId="5" inId="1018" conDes="Net Income after Tax" conCd="SIAT" stdaGUId="831_107570667_STD_2C68EF2F-AB17-40EF-9095-387DE1D5D745" stdIaId="831" siKey="120040511473155197" siId="405" sLiCurIso="KRW" sCurIso="KRW" stCurIso="KRW" stTyCd="INC" sId="1" pEndDt="2013-Dec-31" pId="2" fId="192730348494" fbId="1" /></AudNode></AudNode></AudNode></ai2aiinst></CAudit>
I need to make a xml file for each row .
The name of the xml file wouldl be the first column before |
So in this case i will have 3 xml file like below
1_107570667_ANA_2C68EF2F-AB17-40EF-9095-387DE1D5D745_App.xml
3_107570667_ANA_2C68EF2F-AB17-40EF-9095-387DE1D5D745_App.xml
5_107570667_ANA_2C68EF2F-AB17-40EF-9095-387DE1D5D745_App.xml
And each xml file will contain the record after |.
Like this i will have 500000 rows and i need to create xml file for each row .
First, you need to create a paired RDD containing file name and file content as tuple and then use that paired RDD to write the individual files to disk/hadoop.
You can have a look at the following code snippet:
val input = sparkSession.sparkContext.textFile("<your_input_file>")
val pairedRDD = input.map(row => {
val split = row.split("\\|")
val fileName = split(0)
val fileContent = split(1)
(fileName, fileContent)
})
import org.apache.hadoop.io.NullWritable
import org.apache.spark.HashPartitioner
import org.apache.hadoop.mapred.lib.MultipleTextOutputFormat
class RddMultiTextOutputFormat extends MultipleTextOutputFormat[Any, Any] {
override def generateActualKey(key: Any, value: Any): Any = NullWritable.get()
override def generateFileNameForKeyValue(key: Any, value: Any, name: String): String = key.asInstanceOf[String]
}
pairedRDD.partitionBy(new HashPartitioner(1000)).saveAsHadoopFile("<output_path>", classOf[String], classOf[String], classOf[RddMultiTextOutputFormat])
output:

nltk parses parenthesis incorrectly

I'm tagging text to search for nouns and adjectives:
text = u"""Developed at the Vaccine and Gene Therapy Institute at the Oregon Health and Science University (OHSU), the vaccine proved successful in about fifty percent of the subjects tested and could lead to a human vaccine preventing the onset of HIV/AIDS and even cure patients currently on anti-retroviral drugs."""
nltk.pos_tag(nltk.word_tokenize(text))
This results in:
[('Developed', 'NNP'), ('at', 'IN'), ('the', 'DT'), ('Vaccine',
'NNP'), ('and', 'CC'), ('Gene', 'NNP'), ('Therapy', 'NNP'),
('Institute', 'NNP'), ('at', 'IN'), ('the', 'DT'), ('Oregon', 'NNP'),
('Health', 'NNP'), ('and', 'CC'), ('Science', 'NNP'), ('University',
'NNP'), ('(', 'NNP'), ('OHSU', 'NNP'), (')', 'NNP'), (',',
','), ('the', 'DT'), ('vaccine', 'NN'), ('proved', 'VBD'),
('successful', 'JJ'), ('in', 'IN'), ('about', 'IN'), ('fifty', 'JJ'),
('percent', 'NN'), ('of', 'IN'), ('the', 'DT'), ('subjects', 'NNS'),
('tested', 'VBD'), ('and', 'CC'), ('could', 'MD'), ('lead', 'VB'),
('to', 'TO'), ('a', 'DT'), ('human', 'NN'), ('vaccine', 'NN'),
('preventing', 'VBG'), ('the', 'DT'), ('onset', 'NN'), ('of', 'IN'),
('HIV/AIDS', 'NNS'), ('and', 'CC'), ('even', 'RB'), ('cure', 'NN'),
('patients', 'NNS'), ('currently', 'RB'), ('on', 'IN'),
('anti-retroviral', 'JJ'), ('drugs', 'NNS'), ('.', '.')]
Is there a built in way of correctly detecting parenthesis when tagging sentences?
If you know what you want to return as the tag value for the parens, then you can use a RegexpTagger to match the parens and fallback to the preferred tagger for all else.
import nltk
from nltk.data import load
_POS_TAGGER = 'taggers/maxent_treebank_pos_tagger/english.pickle'
tagger = load(_POS_TAGGER) # same tagger as using nltk.pos_tag
regexp_tagger = nltk.tag.RegexpTagger([(r'\(|\)', '--')], backoff = tagger)
regexp_tagger.tag(nltk.word_tokenize(text))
Result:
[(u'Developed', 'NNP'), (u'at', 'IN'), (u'the', 'DT'), (u'Vaccine',
'NNP'), (u'and', 'CC'), (u'Gene', 'NNP'), (u'Therapy', 'NNP'),
(u'Institute', 'NNP'), (u'at', 'IN'), (u'the', 'DT'), (u'Oregon',
'NNP'), (u'Health', 'NNP'), (u'and', 'CC'), (u'Science', 'NNP'),
(u'University', 'NNP'), (u'(', '--'), (u'OHSU', 'NNP'), (u')', '--'),
(u',', ','), (u'the', 'DT'), (u'vaccine', 'NN'), (u'proved', 'VBD'),
(u'successful', 'JJ'), (u'in', 'IN'), (u'about', 'IN'), (u'fifty',
'JJ'), (u'percent', 'NN'), (u'of', 'IN'), (u'the', 'DT'),
(u'subjects', 'NNS'), (u'tested', 'VBD'), (u'and', 'CC'), (u'could',
'MD'), (u'lead', 'VB'), (u'to', 'TO'), (u'a', 'DT'), (u'human', 'NN'),
(u'vaccine', 'NN'), (u'preventing', 'VBG'), (u'the', 'DT'), (u'onset',
'NN'), (u'of', 'IN'), (u'HIV/AIDS', 'NNS'), (u'and', 'CC'), (u'even',
'RB'), (u'cure', 'NN'), (u'patients', 'NNS'), (u'currently', 'RB'),
(u'on', 'IN'), (u'anti-retroviral', 'JJ'), (u'drugs', 'NNS'), (u'.',
'.')]