Really I need to solve a data cleaning, my target is keep the older date according with variable 'codCie'
I have indexed my dataset with the columns as appear with the command:
df_temp2.set_index(['idCliente','sexo', 'edad'], inplace = True)
OUTPUT:
codCie fechaConsulta
idCliente sexo edad
100020 Femenino 64.0 I10X 2010-07-03
64.0 I10X 2012-03-23
64.0 R060 2016-11-24
64.0 R060 2016-11-25
64.0 R060 2016-11-26
64.0 R060 2016-11-27
64.0 R060 2016-11-28
100073 Femenino 69.0 I471 2006-07-31
69.0 I471 2007-02-21
69.0 I471 2007-04-03
69.0 I471 2007-04-04
69.0 I471 2007-05-02
69.0 I471 2008-01-02
69.0 I471 2008-11-05
69.0 I471 2009-02-18
69.0 I499 2009-06-26
69.0 R000 2020-02-07
69.0 R074 2005-04-18
However I need to remove for each 'idCliente' where in variable 'codCie' is appearing different value, remove the new one on the column 'fechaConsulta'. My target is that output should be as follow:
codCie fechaConsulta
idCliente sexo edad
100020 Femenino 64.0 I10X 2010-07-03
64.0 R060 2016-11-24
100073 Femenino 69.0 I471 2006-07-31
69.0 I499 2009-06-26
69.0 R000 2020-02-07
69.0 R074 2005-04-18
Somebody can help me?
Related
InfluxDB provides movingAverage() or timedMovingAverage(), but what is the most efficient way to get the moving min/max?
For example, if I have data like:
_time
_value
01-01
-1
01-02
4
01-03
2
01-04
9
01-05
-4
01-06
3
01-07
8
01-08
1
The following would return:
|> movingMinMax(n: 3) or |> timedMovingMinMax(every: 3d, period: 8d)
_time
min
max
01-03
-1
4
01-04
2
9
01-05
-4
9
01-06
-4
9
01-07
-4
8
01-08
1
8
I want to decompose a signal into multiple discrete frequencies.
^ Here the top wave is composed of the bottom two discrete waves. It is my understanding that it is possible to use fourier transforms to separate that top signal into multiple underlying frequencies. I want to obtain those separate frequencies (D294, A440) given my 1d array below.
Import Data: I have a 1D array of temperature readings over 1575 timesteps.
import numpy as np
temp_arr_1d = np.array([10. 7.4 7.16666667 8.66666667 6. 7.
7. 8.85714286 14. 11. 15.71428571 14.
15.83333333 12.83333333 14.71428571 13.83333333 16.5 13.83333333
12.5 11.28571429 11.2 9.5 14. 13.83333333
12.25 12.66666667 12.85714286 14.83333333 14.125 14.71428571
16.2 16. 16.28571429 18. 17.42857143 16.625
16.66666667 15.6 14. 15.42857143 15.25 15.875
15.33333333 16.28571429 17.33333333 19.16666667 14.42857143 13.66666667
15.6 15.85714286 17.71428571 20. 20.5 17.42857143
16.85714286 16.875 17.85714286 20.8 19.42857143 17.33333333
19. 19.33333333 17.6 20.875 20.85714286 23.42857143
24.16666667 25.42857143 23.14285714 24. 23.5 21.5
22.33333333 24.16666667 20.33333333 22.66666667 23.42857143 22.5
29.16666667 23.83333333 25.25 27.375 27. 23.5
24.14285714 21. 22.42857143 21.25 23.5 23.2
25.375 25.16666667 26.2 24.6 25.6 25.85714286
29.14285714 28.71428571 30.16666667 30. 30. 28.85714286
30.2 28.25 28.25 32.125 29.2 30.28571429
28.28571429 30.625 27.66666667 27.375 28.625 30.28571429
31.14285714 29.875 31.14285714 30.57142857 32.125 31.14285714
31.85714286 29.83333333 28.57142857 32.85714286 32.625 32.75
32.875 34.5 34.28571429 34. 30.75 29.85714286
31.71428571 32.28571429 33. 33. 32.83333333 31.4
35.33333333 36.4 36. 36.75 37.5 38.42857143
38.71428571 37.8 35.85714286 35.33333333 34.14285714 32.2
33.625 32. 32.4 35.6 35.85714286 37.16666667
31.28571429 34. 34.2 36.16666667 36.625 30.16666667
34.14285714 29.83333333 30.14285714 30.71428571 27. 26.875
28.4 29.85714286 33. 34.83333333 35.6 35.16666667
33.14285714 30.57142857 30.66666667 31.42857143 31.5 33.25
32.83333333 33.85714286 33.14285714 31.57142857 32.375 32.8
31. 31.16666667 29.83333333 29. 29.75 32.5
28.875 31.75 30.75 31.75 29.875 31.125
31.75 30.5 26.83333333 28.2 29.5 31.85714286
29.71428571 28.33333333 30. 30.14285714 32.28571429 31.
30.5 28.83333333 30. 31.57142857 32.5 32.5
29.5 27.16666667 28.375 28.5 27.16666667 29.42857143
30. 29.5 30.75 29.66666667 27.71428571 26.6
27.42857143 28.33333333 28.16666667 27.375 28.33333333 29.16666667
31.28571429 32. 29. 32. 32.14285714 30.14285714
31.5 29.16666667 29. 30. 31.85714286 32.75
31.71428571 31.66666667 30.57142857 30.83333333 28. 31.
29.66666667 30.85714286 31.16666667 31.14285714 30.5 31.625
29.66666667 29.25 29.14285714 29.8 28.66666667 25.2
28.33333333 30.28571429 30.75 28.57142857 28.2 29.
30. 28.14285714 26.85714286 28.28571429 29.2 28.6
24.83333333 28.5 29.75 29.71428571 29.2 30.83333333
29.42857143 23.85714286 26.14285714 27.16666667 27.71428571 25.85714286
26.42857143 26.85714286 26.33333333 24.57142857 24.33333333 24.875
27.8 25. 23.85714286 22.8 23. 22.875
22.66666667 23. 22.85714286 23.66666667 23.42857143 21.625
20.625 21.16666667 18.83333333 20.57142857 20.14285714 18.
19.85714286 16.83333333 18.85714286 16.57142857 17.75 17.625
17. 16.625 15.57142857 18.5 17.875 18.25
17.875 17.625 18.14285714 19.125 21.25 21.25
19.125 18.625 17.75 17.875 18. 17.25
17.5 17.14285714 17.14285714 16.125 16. 16.5
16.125 15.5 16.625 17.75 17.14285714 16.14285714
15.5 15.25 14.75 14.875 16.125 15.375
14.75 15.25 14.25 13.5 13.66666667 12.125
11.875 10.875 10.57142857 12.375 14.5 13.375
11. 12.5 12.875 12.375 11.42857143 12.14285714
11.875 12.83333333 12.375 12.125 12.875 14.
14.875 12. 12.28571429 12. 12.5 14.5
14.625 13.57142857 15.25 15.625 13.875 11.75
14.375 16.625 16.625 14.875 14.5 14.75
14. 15.25 18.42857143 18.85714286 17. 18.5
19.625 13.625 12.875 12.625 13.375 13.25
15.25 13. 12.375 13.5 15.125 15.25
15.125 16.125 17.125 16.625 16.5 16.375
18.125 18.75 18.625 15.625 16.14285714 16.125
17.875 18.75 18.875 18.375 18.5 19.875
22. 20. 20.71428571 20. 19. 19.625
21.625 23.125 26.25 24.85714286 21.75 21.5
22.5 25. 22.75 22.375 24.625 25.125
23.5 24.42857143 25.875 24.125 24.125 24.875
26.75 25. 25.85714286 27.375 30.125 27.25
25.75 25.75 26.375 28.14285714 29.125 26.875
27.625 29. 27.125 26.375 24.25 25.625
27. 28. 29.375 29.125 29.375 30.625
31.5 30.625 31.5 32.125 33.25 34.875
34.375 31.75 31.875 29.75 29.57142857 31.75
31.375 30.5 30. 29.625 28. 25.25
26.875 29.5 31.125 29.875 31.5 32.375
31.125 33. 33.25 34.625 31.5 31.5
30.125 31.875 33.625 35.25 33.125 34.625
32.75 32.75 34.875 35.25 36.875 37.5
38.5 37.625 37.875 37.25 37.625 32.875
30.25 30.5 33.875 36.375 36.875 33.875
35.75 38. 36.875 34.5 32.125 33.
32.25 32.5 34.75 34.875 33. 32.
31.375 29.5 28.875 31.38888889 34.70588235 30.625
34. 35. 36. 36.125 36.625 35.875
31.625 32.375 33.125 32.625 30.75 27.25
29.875 30.875 32.125 31.625 30.625 30.5
30.75 32.5 31.71428571 30. 30.25 31.875
32.5 33.25 30.125 31.5 30.5 29.75
32.125 31.125 31. 29.23529412 28.5 31.125
32.125 32.125 30.375 31.125 30.42857143 31.375
31.875 33. 33.25 32.5 33. 33.125
33.125 33.875 34.125 33.85714286 29.625 30.375
28.625 27.625 29.42857143 29.25 28.625 27.875
26.5 28.5 31. 31.25 31. 28.75
26.57142857 28.5 29.375 29.125 29.625 30.625
30.625 30.5 31. 31.125 30.5 30.625
31. 29.625 30. 30. 30.5 30.25
30.5 30.625 30.42857143 30.375 30.5 31.
30.625 30.625 31. 28.75 27.625 27.125
25.71428571 26. 27.625 24.5 24.5 23.
25. 24. 23.75 24.75 25.625 25.75
25.25 25.5 25.25 25.125 24.875 25.125
24.75 23.875 23.375 22.5 22.875 23.375
23.625 22.75 23. 24. 23.875 24.125
21.375 19.25 19.625 18.5 18.375 18.625
18.25 17.75 17.625 17.5 20.375 18.25
19.875 17.125 16.875 16.5 19.625 18.125
19.375 20.25 19.75 20. 21.375 25.
21.25 17.75 18.25 17.25 17. 15.28571429
21.5 19.625 17.5 16. 16.875 16.625
17.75 12.875 11.75 11.75 11.25 9.5
9.375 9.625 9.875 9.25 10.75 10.375
9. 11.125 11.625 12.375 14.75 14.875
15.125 14.125 14. 12. 9.625 10.
10.625 11.125 11. 10.625 12.25 12.
12.375 13. 13.5 13. 13.75 13.375
13.25 12.625 14.625 12.375 13.25 13.75
13.25 11.71428571 11.5 12.75 13.75 15.
17.375 16.75 14.625 14.625 15.125 15.875
16.625 16.875 15.75 16. 15.5 17.25
19.375 18.875 20.625 21.14285714 21.25 22.75
22.75 21.875 22.14285714 21.5 22.875 23.125
20. 19.25 21.25 17.375 17.5 16.25
16.75 18.75 20. 18.375 19.5 18.125
17.75 18.25 20.5 21.375 21.5 17.625
19.85714286 20.875 20.25 21.125 22.5 24.875
25.125 24.75 25.75 26.625 28.5 27.375
27. 25. 23.5 25.25 26.42857143 28.71428571
24.5 21. 23.57142857 25. 23.75 25.625
26. 27.125 28.25 24.375 24.75 26.25
27.625 25.25 28.125 30.125 32.125 32.875
31.75 30.75 30.75 31.125 30.5 28.75
30.125 31. 31.25 32.625 31.25 30.125
31. 30.875 32.125 34.125 35.42857143 34.125
35.125 35.125 34. 33.125 27.75 29.625
28.5 30. 33.25 35.375 31.5 33.125
35.625 36.25 37.375 36. 37.75 35.25
33.75 34.75 35.125 32.25 34.25 29.375
31.625 28.625 31.875 32.625 33.375 35.
36.5 37.625 36.625 35.625 35.875 31.125
27. 31. 32.375 33.875 35.125 35.875
33.125 30.375 32.125 31.5 37.25 26.625
31.125 32.625 34.25 31.125 30.875 32.
33.75 33. 34.125 35.71428571 27.875 28.125
31.25 29.625 26.625 25.5 25.5 29.875
30.875 33.375 31.75 30.75 29.5 30.125
29.125 31.375 32.25 32.125 32. 30.85714286
30. 29.625 29.5 28.5 29. 29.
28.25 29.625 31.5 29.42857143 29.59375 29.5
29.25 28.875 27.375 30.375 28.375 30.75
32. 30.875 28.5 28.625 30.75 29.125
31.25 30.75 29.375 30.25 29.875 30.75
32. 32. 32.5 32. 32.25 32.625
31.85714286 32.125 31.875 31.625 32. 30.875
31.25 31.5 31.625 31. 31.25 30.75
32.14285714 32.125 31.875 33. 32.375 31.375
31. 30.625 29.14285714 30.125 30.25 28.625
28.25 28.5 29.25 28.71428571 28.25 29.
29. 28.125 28.28571429 30. 29. 28.875
28.375 28.625 28.375 28.5 28.5 28.57142857
29.25 29.5 28.125 27.5 27.875 28.
26.85714286 28.14285714 27.71428571 26.5 24.75 24.
24.875 26.375 25.625 24.75 20.875 22.125
22.375 22.375 22.875 23.375 24. 23.75
22.125 22.875 21.875 21.875 21.75 22.75
21.875 21.875 22.375 22.375 22.5 21.875
19.875 18.5 18.5 18.5 18.75 18.625
17. 18.5 19.375 20. 18.5 18.75
18.125 19.5 19.25 18.125 17.625 17.625
17.5 17.125 17.625 18.42857143 18.375 19.
17.25 13.25 12.875 13.25 13.125 13.
12.25 12.875 12.375 11.75 12.625 12.
12.5 11.25 11.5 12.75 15.375 17.125
16.375 15.5 15. 14.71428571 14. 14.375
15.75 15.83333333 17.375 17.125 15.5 15.85714286
15.625 15.75 18. 18.26666667 15.5625 13.
13.6 14. 13.26666667 12.35714286 12.06666667 12.1875
11.73333333 14.4375 11.1875 11.66666667 14.5625 17.58333333
16.85714286 19.5625 20.14285714 17.375 15.84615385 15.26666667
13.125 16.36363636 12.66666667 15.33333333 15.625 17.09090909
17.76923077 18.13333333 19.6875 19.2 17.06666667 17.64285714
17.6 18.21428571 18.71428571 19.46666667 23.15384615 23.625
21. 21. 21.42857143 21.6875 22.5625 22.8
22.8 23. 23.875 24.91666667 24.93333333 26.
27.3125 23.93333333 22.8125 23.71428571 23.42857143 24.
25.5625 25.06666667 24.5625 24.25 22.375 24.06666667
23.9375 26.3125 26.1875 26.78571429 27.13333333 26.625
25.0625 26.2 28.13333333 29.875 24.66666667 26.25
25.93333333 27.125 29.57142857 30. 30.57142857 32.3125
33.3125 32.8125 32.3125 31.375 29.93333333 29.26666667
30.73333333 32.25 29.8 30.2 31.75 33.125
33.625 35.6875 34.66666667 34.625 34. 34.0625
34. 33.25 31.91666667 31.3125 31.75 33.4375
33.125 34.15384615 34.07142857 33.0625 34.6875 38.
35.5 33.71428571 30.6 31.4375 33.3125 35.13333333
33.53333333 34. 32.5 35.375 37.29411765 36.5625
37.25 37.21428571 37.5 37.75 37.375 37.4
36.13333333 36.8 32.21428571 31.52631579 33.2173913 35.26923077
38.27272727 36.0625 31.5 26.8125 32.64285714 36.
37.5625 37.5625 38.2 36.16666667 35.42857143 34.625
36.07142857 35.73333333 36.13333333 33.4375 35.5 36.
32.625 34.73333333 33.5 34.1875 35.85714286 35.625
30.9375 32.875 33.125 33.84615385 36.4375 35.42857143
34.86666667 34.3125 30.78571429 35.375 35.46666667 32.125
28.4 29.5625 30.6875 33.25 33.26666667 33.5
30.8 33.25 32.5625 31.5 30.5 31.25
30.4375 31. 27.125 28.125 27.66666667 32.3125
34.1875 34.13333333 34.125 31.875 31.4375 31.9375
30.3125 28.3125 29.53333333 27.375 27.33333333 29.26666667
29.125 30.6875 32.5625 33.11111111 33.8 30.06666667
33.11764706 33.80952381 31.61538462 32. 28.10714286 29.03571429
30.32142857 28.93333333 31.67857143 31.33333333 29.92857143 29.88888889
32.07142857 33.18518519 31.59259259 32.18518519 31.48 30.17857143
31.52 31.22222222 31.78571429 33.4 29.57142857 30.04
27.25925926 27.96 30.73913043 30.89473684 31.69230769 31.07692308
30.375 31.1 31.91666667 30.55555556 31.23076923 31.
31.64285714 32.53333333 30.85714286 31.72727273 31.4 32.30769231
32.25 32.375 33.44444444 33.36 30.03703704 31.
31.24 31.13043478 31.48 32.18518519 32.44 32.22727273
32.21428571 32.54166667 32.81481481 33.26923077 30.55555556 28.83333333
30.7037037 30.96 30.6 30.92 29.77777778 29.66666667
29.57142857 29.96296296 29.75 27.74074074 28.42857143 28.6
28.5 28.92592593 29.07692308 28.40909091 29.33333333 27.5
28.5 28.04 27.57692308 26.55555556 25.51851852 25.81481481
24.82608696 24.53846154 24.38461538 23.72727273 25.64 24.81481481
23.11538462 22.92592593 24.54545455 23.73076923 23. 23.51851852
23.92 23.53846154 24.2962963 23.34615385 22.24 21.76923077
21.73076923 21.73076923 20.66666667 22.25 21.53846154 22.57894737
22.82608696 21.42105263 23.6 24.29411765 23.63636364 22.45454545
21.61111111 19.86956522 19.75 19.20833333 21.20833333 18.9
18.63636364 18.53846154 18.25 16.9 19.41666667 16.44444444
20.04166667 19.90909091 19.05 18.55555556 18.16666667 15.83333333
17.5 16.08333333 17.85714286 19.8 18.05 17.28571429
15.55 17.31818182 14. 17.14285714 16.85 17.2173913
15.23809524 14.0952381 15.05263158 15.91304348 18.5 17.11111111
18.7 18.38888889 19.31818182 14.70833333 15.68421053 14.57142857
12.11111111 11. 11.78947368 13.23529412 13.2 16.43478261
14.65 11.72222222 13.04166667 14.61904762 15.26315789 15.39130435
18.44 18.11764706 18.34782609 21. 16.17857143 16.5
14.86363636 15.66666667 16.44444444 16.125 15.25 17.09090909
15.63636364 18.7 18.63157895 16.88888889 15.125 15.7
15.375 14.66666667 15.625 16.25 16.33333333 16.875
17.57142857 20.25 21.3 21.125 22.36363636 23.375
21.83333333 19.125 18.625 19.125 19. 18.75
19.875 23.33333333 24.46153846 23.75 20.5 19.125
19.75 20. 22.625 21.54545455 20.78571429 19.9375
18.53333333 17.375 17.44444444 18. 19.875 24.
20.9 24.69230769 24.66666667 23.33333333 25. 27.25
28. 28.91666667 26.5 29.1 29.5 29.88888889
31. 29.28571429 30.625 31.375 29.75 30.5
30.93333333 29.23076923 31.22222222 27. 25.625 27.125
27.85714286 29.25 29.25 29.66666667 30.5 31.22222222
31. 32.55555556 34. 33.5 34.5 34.25
32.9 32.875 32. ])
temp_arr_1d.shape
(1575,)
Run the transform
https://numpy.org/doc/stable/reference/routines.fft.html
>>> np.fft.fft(temp_arr_1d).shape
(1575,)
How do I do this?
FFT gives you spectral + phase data.
The code to obtain the spectrum:
import numpy as np
import matplotlib.pyplot as plt
temp_arr_1d = np.array(...)
# transfrom and shift to get contiguous data
transform = np.fft.fftshift(np.fft.fft(temp_arr_1d))
# squared modulus is the spectral power
spectral_power = np.abs(transform)**2
# plotting routines
plt.figure(figsize=(10,10))
plt.plot(spectral_power)
plt.xlim(770,810)
plt.show()
Your signal contains one very strong frequency + two sidebands. See figure below.
What are you trying to achieve?
I am currently trying to convert all the row values to certain sign by using numpy built-in function np.sign
My code:
import numpy as np
pd_dataframe = pd.DataFrame({'id': [i for i in range(10)],
'values': [10,5,3,-1,0,-10,-4,10,0,10]})
sp_dataframe = spark.createDataFrame(pd_dataframe)
sign_acc_row = F.udf(lambda x: np.sign([x]), IntegerType())
sp_dataframe = sp_dataframe.withColumn('sign', sign_acc_row('values'))
sp_dataframe.show()
Errors:
y4JJavaError: An error occurred while calling o2586.showString.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 320.0 failed 1 times, most recent failure: Lost task 0.0 in stage 320.0 (TID 3199, localhost, executor driver): net.razorvine.pickle.PickleException: expected zero arguments for construction of ClassDict (for numpy.core.multiarray._reconstruct)
Expected output:
id values sign
0 0 10 1
1 1 5 1
2 2 3 1
3 3 -1 -1
4 4 0 0
5 5 -10 -1
6 6 -4 -1
7 7 10 1
8 8 0 0
9 9 10 1
Side question if allowed:
I wanted to create another column with which it returns an additional of 1 when the value is different from previous row.
Expected output:
id values sign numbering
0 0 10 1 1
1 1 5 1 1
2 2 3 1 1
3 3 -1 -1 2
4 4 0 0 3
5 5 -10 -1 4
6 6 -4 -1 4
7 7 10 1 5
8 8 0 0 6
9 9 10 1 7
You are almost there. np.sign return an numpy.int64 object which is not understood by pyspark. To make them compatiable, you can do:
sign_acc_row = F.udf(lambda x: int(np.sign(x)), IntegerType())
I am trying to join 2 SpatialPointsDataFrames by nearest neighbour analysis using sf::st_join(). Both files have been converted using st_as_sf() but when I try the join I get the error
Error in rep(seq_len(nrow(x)), lengths(i)) : invalid 'times' argument
At this point I have tried swapping the x and y arguments, and adjusting countless variations of the arguments but nothing seems to work. I have checked the help file for sf::st_join(), but don't see anything about a times argument? So I am unsure where from and why it keeps throwing this error...
below is a sample of my data set which produces the same error using the code further down
> head(sf.eSPDF[[1]])
Simple feature collection with 6 features and 8 fields
geometry type: POINT
dimension: XY
bbox: xmin: 35.9699 ymin: -3.74514 xmax: 35.97065 ymax: -3.74474
epsg (SRID): 4326
proj4string: +proj=longlat +datum=WGS84 +no_defs
# A tibble: 6 x 9
TIME ELEVATION LATITUDE LONGITUDE DATE V1 V2 Survey geometry
<dttm> <chr> <dbl> <dbl> <date> <dttm> <dttm> <dbl> <POINT [°]>
1 2012-01-20 07:26:05 1018 m -3.74 36.0 2012-01-20 2012-01-20 00:00:00 2012-01-31 00:00:00 1 (35.97047 -3.74474)
2 2012-01-20 07:27:35 1018 m -3.74 36.0 2012-01-20 2012-01-20 00:00:00 2012-01-31 00:00:00 1 (35.97057 -3.74486)
3 2012-01-20 07:27:39 1019 m -3.74 36.0 2012-01-20 2012-01-20 00:00:00 2012-01-31 00:00:00 1 (35.9706 -3.74489)
4 2012-01-20 07:27:47 1020 m -3.74 36.0 2012-01-20 2012-01-20 00:00:00 2012-01-31 00:00:00 1 (35.97065 -3.74489)
5 2012-01-20 07:28:05 1020 m -3.74 36.0 2012-01-20 2012-01-20 00:00:00 2012-01-31 00:00:00 1 (35.97035 -3.74498)
6 2012-01-20 07:28:26 1019 m -3.75 36.0 2012-01-20 2012-01-20 00:00:00 2012-01-31 00:00:00 1 (35.9699 -3.74514)
> head(sf.plt.centr)
Simple feature collection with 6 features and 1 field
geometry type: POINT
dimension: XY
bbox: xmin: 35.75955 ymin: -3.91594 xmax: 36.0933 ymax: -3.401
epsg (SRID): 4326
proj4string: +proj=longlat +datum=WGS84 +no_defs
PairID geometry
1 1 POINT (36.0933 -3.6731)
42 92 POINT (36.02593 -3.91594)
83 215 POINT (36.06496 -3.75837)
124 225 POINT (35.83156 -3.401)
165 251 POINT (35.75955 -3.54388)
206 2 POINT (36.08752 -3.69128)
Below is the code that I am using to check for working solutions
sf.eSPDF<-lapply(eSPDF, function(x){
st_as_sf(as(x, "SpatialPointsDataFrame"))
})
sf.plt.centr<-st_as_sf(as(plt.centr, "SpatialPointsDataFrame"))
x1<-head(sf.eSPDF[[1]])
x2<-head(sf.plt.centr)
check<-st_join(x1, x2, join=st_nn, maxdist = Inf, returnDist = T, progress = TRUE)
As you can see, the file I want to join to is an object within a list. All objects within that list have identical structure to the example given. Eventually I want to get a code that joins the sf.plt.centr file to each of the files within the list. Something like...
big.join<-lapply(sf.eSPDF, function(x){
st_join('['(x), sf.plt.centr, st_nn, maxdist = Inf, returnDist = T, progress = TRUE)
}
I used decision tree classifier in MATLAB. I used 7 attributes.But when i draw the decision tree it doesn't includes all attributes,it only includes 2 or 1 attributes.What is the problem of my code?
vars = {'Asymmetry' 'Border irregularity' 'colors' 'contrast' 'Co-relation'
'Homogeneity' 'Energy'};
x = [0.148 0.298 3 0.027 0.959 0.992 0.692
0.248 0.462 3 0.015 0.997 0.996 0.837
0.683 0.827 3 0.030 0.974 0.989 0.634
0.170 0.509 3 0.065 0.964 0.977 0.399
0.663 0.764 3 0.061 0.945 0.983 0.645
0.641 0.671 3 0.050 0.953 0.987 0.703
0.653 0.796 2 0.062 0.961 0.981 0.528
0.458 0.704 2 0.019 0.934 0.993 0.852
0.555 0.729 2 0.087 0.976 0.980 0.380
0.454 0.657 2 0.059 0.953 0.982 0.467
0.379 0.497 2 0.058 0.976 0.979 0.445
0.443 0.486 2 0.034 0.896 0.998 0.810
0.194 0.342 2 0.012 0.956 0.997 0.895
0.248 0.462 3 0.015 0.977 0.996 0.837
0.155 0.340 2 0.010 0.930 0.966 0.911
0.458 0.704 2 0.019 0.934 0.993 0.852];
y = {'Cancer';'Cancer';'Cancer';'Cancer';'Cancer';'Cancer';'Cancer';'Cancer';'Cancer';'Cancer';'Cancer';'Cancer';'non-Cancer';'non-Cancer';'non-Cancer';'non-Cancer'};
t = fitctree(x,y,'PredictorNames',vars, ...
'CategoricalPredictors',{},'Prune','off');
view(t);
X1=[0.148 0.186 2 0.139 0.984 0.992 0.558]
label = predict(t,X1);
view(t,'mode','graph');
The output image of the code:
There's nothing wrong with your code - the decision tree is not designed to use all of your variables, it's just designed to use the variables that give the best fit based on the given decision criteria. Using all of the variables would result in overfitting, especially considering that some of your variables are correlated with each other.