I'm using papaParse to parse an CSV file into JSON for further use. Upon parsin it returns
"errors": [ { "type": "FieldMismatch", "code": "TooFewFields", "message": "Too few fields: expected 21 fields but parsed 1", "row": 14 } ], "meta": { "delimiter": ";", "linebreak": "\r\n", "aborted": false, "truncated": false, "fields": [ "Output in top 10 percentiles (%)", "Overall", "1996", "1997", "1998", "1999", "2000", "2001", "2002", "2003", "2004", "2005", "2006", "2007", "2008", "2009", "2010", "2011", "2012", "2013", "2014" ] } }
can somebody please explain to me what this means? I read trough the documentation on their webpage but still don't understand what is wrong
the CSV file I'm working with is this (http://www.topdeckandwreck.com/excel%20graphs/Sheet10.csv)
In your config, add
skipEmptyLines: true
Reference: http://papaparse.com/docs#config-details
Solution was posted by Lasse V Karlsen in the comments, removing the last empty line in notepad so the CSV file contains only data removes the error
Related
I'm using pyspark to create a dataframe from a JSON file.
The structure of the JSON file is as follows:
[
{
"Volcano Name": "Abu",
"Country": "Japan",
"Region": "Honshu-Japan",
"Location": {
"type": "Point",
"coordinates": [
131.6,
34.5
]
},
"Elevation": 571,
"Type": "Shield volcano",
"Status": "Holocene",
"Last Known Eruption": "Unknown",
"id": "4cb67ab0-ba1a-0e8a-8dfc-d48472fd5766"
},
{
"Volcano Name": "Acamarachi",
"Country": "Chile",
"Region": "Chile-N",
"Location": {
"type": "Point",
"coordinates": [
-67.62,
-23.3
}]
I will read in the file using the following line of code:
myjson = spark.read.json("/FileStore/tables/sample.json")
However, I keep on getting the following error message:
Spark Jobs
myjson:pyspark.sql.dataframe.DataFrame
_corrupt_record:string
Can someone let me know what I might doing wrong?
Is the problem with the structure of the json file?
Seems like your JSON is multiple line Json that why issue is, to fix this below is code snippet,
myjson = spark.read.option("multiline", "true").option("mode", "PERMISSIVE")
.json("/FileStore/tables/sample.json")
Hope this will solve issue.
I was trying to update an entity with a semicolon inside an attribute:
{
"name": "Accelerometer",
"type": "string",
"value": "0;0;0",
"metadatas": [
{
"name": "TimeInstant",
"type": "ISO8601",
"value": "2019-12-17T14:13:01.398Z"
}
]
}
and got
{
"errorCode": {
"code": "400",
"reasonPhrase": "Bad Request",
"details": "Illegal value for JSON field"
}
}
Removing the ; solve the error. However, is this behaviour the one expected?
Thanks
Yes, that's normal behavior. Please have a look to the forbidden characters Orion documentation.
On my local overpass api server with only french data on which is applied hourly planet diff, some of the query responses are wrong.
It's not doing it for each query : but something like once every 200 requests , sometime more ...
for example :
[timeout:360][out:json];way(48.359900103518235,5.708088852670471,48.360439696481784,5.708900947329539)[highway];out ;
return 3 ways :
{
"version": 0.6,
"generator": "Overpass API 0.7.54.13 ff15392f",
"osm3s": {
"timestamp_osm_base": "2019-09-23T15:00:00Z",
},
"elements": [
{
"type": "way",
"id": 53290349,
"nodes": [...],
"tags": {
"highway": "secondary",
"maxspeed": "100",
"ref": "L 385"
}
},
{
"type": "way",
"id": 238493649,
"nodes": [...],
"tags": {
"highway": "residential",
"name": "Rue du Stand",
"ref": "C 3",
"source": "..."
}
},
{
"type": "way",
"id": 597978369,
"nodes": [...],
"tags": {
"highway": "service"
}
}
]
}
First one is in Germany, far East ...
My question :
On an overpass api server, is there a way to apply diff only for defined area ? it is not documented ( neither here : https://wiki.openstreetmap.org/wiki/Overpass_API/Installation
or here : https://wiki.openstreetmap.org/wiki/User:Breki/Overpass_API_Installation#Configuring_Diffs )
if not, how to get rid of those wrong results ?
Thanks,
Two questions, so two answer :
i found that there is French diff file existing : http://download.openstreetmap.fr/replication/europe/france/minute/ so i will restart my server with those diffs.
The best way to get rid of those wrong result is to have a consistant server : no world diff for just France Data.
{
"completionTime": 1477067415024,
"context": {
"environmentId": 78
},
"id": 51,
"jobId": 473,
"jobName": "Ravindra",
"reportIds": [
959
],
"startTime": 1477067357196,
"status": "PASSED",
"username": "svc.soaess"
}
from this json structure I have to get the value 959.
$response.ChildNodes.reportIds displays blank. Tried with array etc. no luck.
Presumably $response is a string, so you need to convert it to an object first, before you can access the object's properties.
($response | ConvertFrom-Json).reportIds
I'm trying to import a fake CSV file into OrientDB Server 2.1.2.
The ETL tool looks amazing, allowing people to input many options, however it seems to me that the csv transformer (when I tried to use the CSV extractor I got a Extractor 'csv' not found error) does not interpret correctly the "nullValue" option.
I used the following JSON to try to load a simple file and, when using "NULL" as null value both in the data and in the JSON I could import the file correctly, while when using "?" I couldn't.
`
{
"source": { "file": {"path": "Z:/test.tsv"}},
"extractor": { "row": {}},
"transformers": [
{"csv": {
"separator": " ",
"nullValue": "?",
"columnsOnFirstLine": true,
"columns": [
"a:STRING",
"b:STRING",
"c:String",
"n:Integer"
],
"dateFormat": "dd.mm.yyyy"
}
},
{"vertex": {"class": "Test", "skipDuplicates": true}}
],
"loader": {
"orientdb": {
"dbURL": "plocal:C:/Users/taatoal1/tmp/orientdb/databases/test",
"dbType": "graph",
"classes": [
{"name": "Test"}
]
}
}
}
`
Here is the data:
a b c 1
a0 b0 c0 2
a1 b1 c1 ?
Am I doing something wrong?
my suggestion is to try with (just released) latest version, 2.1.4:Orient Download
In 2.1.4 we add the support for the CSV extractor which internally uses commons-csv from Apache.