Export a collection of featureCollections as CSV? - export-to-csv

Sampling daily data but unable to export:
https://code.earthengine.google.com/ca236ee4e12cffb398f62465bc413e68
the bug should be somewhere between line 272 and 285.
Using .flatten() returns an empty collection!
thank you in advance,
Andrea

Related

How do I store YOLOv5 Outputs to one csv file rather than many .txt files per detection

Hi all I have been researching this challenge for some time now and all my efforts are futile. What I am trying to do?
I am running YOLOV5 and this is working fine in the training stage and the detection stages. The operations is outputting multiple .txt files per video for each detection per frame. This is the command I am using:
python3 detect.py --weights /Users/YOLO2ClassOnly/yolov5/runs/train/exp11/weights/best.pt --source /Users/YOLO2ClassOnly/yolov5/data/videos --conf 0.1 --line-thickness 1 --save-txt SAVE_TXT --save-conf
This command produces multiple text files, for example vid0_walking.txt, vid1_walking.txt, vid2_walking.txt...n/ etc.
This is depleting my storage resources and I am trying to avoid this.
What I would like to do?
Store the files in one .csv file in this format, please.
# xmin ymin xmax ymax confidence class name
# 0 749.50 43.50 1148.0 704.5 0.874023 0 person
# 2 114.75 195.75 1095.0 708.0 0.624512 0 person
# 3 986.00 304.00 1028.0 420.0 0.286865 27 tie
I have been following Glen Jorcher Links Here:
https://github.com/ultralytics/yolov5/issues/7499
But this is futile, this function print(results.pandas().xyxy[0])
is not working to generate the output for video as per above.
Please help, this is challenging me due to my lack of understanding.
Thanx in advance for acknowledging my digital presence and I am grateful for your guidance!

Powershell: Remove duplicate entries from columns, only keeping first unique value in its original cell of a csv file

I am pretty new to powershell. I have been trying to remove duplicate entries from columns, only keeping first unique value in its original cell of a csv file.
Any help is greatly appreciated. Thanks in advance.
1,TestSCript,Test1,Data1,passed
1,TestSCript,Test1,Data2,passed
1,TestSCript,Test2,Data3,passed
1,TestSCript,Test2,Data4,passed
1,TestSCript,Test2,Data5,passed
1,TestSCript,Test3,Data6,passed
1,TestSCript,Test3,Data7,passed
1,TestSCript,Test4,Data8,passed
1,TestSCript,Test5,Data9,passed
Expected Result:
1,TestSCript,Test1,Data1,passed
,,,Data2,passed
,,Test2,Data3,passed
,,,Data4,passed
,,,Data5,passed
,,Test3,Data6,passed
,,,Data7,passed
,,Test4,Data8,passed
,,Test5,Data9,passed

How to find highly similar observations in another dataset using Spark

I have two csv files. File 1:
D,FNAME,MNAME,LNAME,GENDER,DOB,snapshot
2,66M,J,Rock,F,1995,201211.0
3,David,HM,Lee,M,,201211.0
6,66M,,Rock,F,,201211.0
0,David,H M,Lee,,1990,201211.0
3,Marc,H,Robert,M,2000,201211.0
6,Marc,M,Robert,M,,201211.0
6,Marc,MS,Robert,M,2000,201211.0
3,David,M,Lee,,1990,201211.0
5,Paul,ABC,Row,F,2008,201211.0
3,Paul,ACB,Row,,,201211.0
4,David,,Lee,,1990,201211.0
4,66,J,Rock,,1995,201211.0
File 2:
PID,FNAME,MNAME,LNAME,GENDER,DOB,FNAMELNAMEMNAMEGENDERDOB
S2,66M,J,Rock,F,1995,66MRockJF1995
S3,David,HM,Lee,M,1990,DavidLeeHMM1990
S0,Marc,HM,Robert,M,2000,MarcRobertHMM2000
S1,Marc,MS,Robert,M,2000,MarcRobertMSM2000
S6,Paul,Row,M,2008,PaulRowM2008
S7,Sam,O,Baby,F,2018,SamBabyOF2018
For example, I want to extract those highly similar observations in File 2 with MarcHRobertM2000 in File 1.
My expected output will be:
S0,Marc,HM,Robert,M,2000,MarcRobertHMM2000
S1,Marc,MS,Robert,M,2000,MarcRobertMSM2000
I used the following code:
sqlContext.registerDataFrameAsTable(df2,'table')
query=""" SELECT PID, FNAMELNAMEMNAMEGENDERDOB, similarity(lower(FNAMELNAMEMNAMEGENDERDOB), 'MarcHRobertM2000') as sim
FROM table
WHERE sim>0.7 """
df=sqlContext.sql(query)
It looks like the similarity in SQL does not work in sqlcontext. I have no idea how to fix it. In addition, File 2 is big, around 5 GB so I did not use the fuzzywuzzy in python. And soundex is not satisfying. Could you help me? Thank you.
you can use Levenshtein distance function to check the similarity.
Please refer to the below code
query=""" SELECT PID, FNAMELNAMEMNAMEGENDERDOB, levenshtein(FNAMELNAMEMNAMEGENDERDOB, 'MarcHRobertM2000') as sim
FROM table
WHERE sim < 4 """
Also please check https://medium.com/#mrpowers/fuzzy-matching-in-spark-with-soundex-and-levenshtein-distance-6749f5af8f28 for good read.

Paraview create points array and set it as source array

Creating an array with p_points = vtk.vtkPoints
And then add data to it p_points.InsertNextPoint(value).
There is like 50000 points.
I want to put that point array in a source.
I tried with a programmable one with :
servermanager fetch getPoints that give me the Point Array in it (p_array)
Then with a loop p_array.InsertNextPoint(p_points.GetPoint(index_point))
But with no luck. The points array in the source remain empty.
Thanks
Use a Programmable Source with a script like:
import vtk
pts = vtk.vtkPoints()
for i in xrange(10):
pts.InsertNextPoint(i, i, i)
output.SetPoints(pts)
The output object is pre-defined in the Python environment executed by the Programmable Source and is the output object of the source.
That's working fine but it's quite slow when you have 50000 points (something like 400-500 seconds)
When I am using servermanager.Fetch(ProgrammableSource).GetPoints().InsertNextPoints(x,y,z),
it's updating the array.
I can see it doing a Render(). And a time.sleep().
But at the end of the paraview script execution, the content seems to be blanked.
Am I accessing a temporary set pointer and then it's unset ?
Resolved :
I found a way using a PolylineSource instead of ProgrammableSource.
Thanks for your help Cory

Change the file-name in tblwrite when created table

I want to create 15 txt tables with different names by using tblwrite in matlab. my attempt was in this code
for j=1:15
rownames=char(rownames1(:,j));
T(:,1)=[1:length(K_minus_t)];
table=char(T(j,1));
colnames{j}={'lambda_L';'C_L';'lambda_N';'tau_N';'lambda_c';'tau_c';};
values{j}=[coef_line lam_tau_curve lam_tau_cap];
tblwrite(values{j},colnames{j},rownames,'table.txt');
end
unfortunately, it was failed. Any help would be greatly appreciated. I will be grateful to you.