Remove duplicates from Json response flutter - flutter

i am unable to remove duplicates in Json. In below Json if title field has same values, i need to remove the duplicate node. IN below json title named "car" was repeated 3 times and i need to remove duplicates based on title field. I followed below stackoverflow links but not solved my problem.
Remove duplicate value in dart
Actual Json:
{
"totalSize": 6,
"done": true,
"records": [{
"attributes": {
"type": "ContentVersion",
"url": "https://sampleUrl"
},
"Id": "123456",
"Title": "car",
"Team_Category__c": "Vehicle"
},
{
"attributes": {
"type": "ContentVersion",
"url": "https://sampleUrl"
},
"Id": "123456",
"Title": "car",
"Team_Category__c": "Vehicle"
},
{
"attributes": {
"type": "ContentVersion",
"url": "https://sampleUrl"
},
"Id": "123456",
"Title": "cycle",
"Team_Category__c": "Vehicle"
},
{
"attributes": {
"type": "ContentVersion",
"url": "https://sampleUrl"
},
"Id": "123456",
"Title": "aeroplane",
"Team_Category__c": "Vehicle"
},
{
"attributes": {
"type": "ContentVersion",
"url": "https://sampleUrl"
},
"Id": "123456",
"Title": "car",
"Team_Category__c": "Vehicle"
}
]
}

Add kt_dart: to your pubspec.
Then import it in your dart file import 'package:kt_dart/kt.dart';
Then:
var json = jsonDecode(yourJsonAsString);
var records = mutableListFrom(json["records"]);
var distinct = records.distinctBy((it) => it["Title"]);

if nothing works out, you can create new similar structure and add only distinct items into it as below. I am not a dart programmer but it is a naive solution.
import 'dart:convert';
void main() {
var jsonData = '[{ "name" : "Dane", "alias" : "FilledStacks" },{ "name" : "Dane", "alias" : "FilledStacks" }]';
List uniqueList = new List();
List parsedJson = json.decode(jsonData);
for (var jsonElement in parsedJson) {
bool isPresent = false;
for (var uniqueEle in uniqueList) {
if (uniqueEle['name'] == jsonElement['name']) isPresent = true;
}
if (!isPresent) uniqueList.add(jsonElement);
}
print('$uniqueList');
print('');
print('${parsedJson.runtimeType} : $parsedJson');
}

Related

Transform nested JSON in data factory to sql

New to data factory. I have a json file that needs to manipulate but I can't figure out how to go about it. The file has a generic "name" property but it should have the value as the key name. How can I get it so that I can get the value as key?
So far been getting Complex JSON errors. This json is coming from file store.
[
{
"Version": "1.1",
"Documents": [
{
"DocumentState": "Correct",
"DocumentData": {
"Name": "Name1",
"$type": "Document",
"Fields": [
{
"Name": "Form",
"$type": "Text",
"Value": "Birthday Form"
},
{
"Name": "Date",
"$type": "Text",
"Value": "12/1/1999"
},
{
"Name": "FirstName",
"$type": "Text",
"Value": "John"
},
{
"Name": "FirstName",
"$type": "Text",
"Value": "Smith"
}
]
}
}
]
},
{
"Version": "1.1",
"Documents": [
{
"DocumentState": "Correct",
"DocumentData": {
"Name": "Name2",
"$type": "Document",
"Fields": [
{
"Name": "Form",
"$type": "Text",
"Value": "Entry Form"
},
{
"Name": "Date",
"$type": "Text",
"Value": "4/3/2010"
},
{
"Name": "FirstName",
"$type": "Text",
"Value": "Jane"
},
{
"Name": "LastName",
"$type": "Text",
"Value": "Doe"
}
]
}
}
]
}
]
Expected output
DocumentData: [
{
"Form":"Birthday Form",
"Date": "12/1/1999",
"FirstName": "John",
"LastName": "Smith"
},
{
"Form":"Entry Form",
"Date": "4/3/2010",
"FirstName": "Jane",
"LastName": "Doe"
}
]
#jaimers,
I was able to achieve it by making use of the Data Flow Activity
The below is the complete DataFlow
1) Source1
This step involves getting the data from source. You will have to configure the Source dataset.
The only change I had done in the source was to Convert Fields.Name,Field.Type,Field.Value as string[] (From string).
This was required to make/create key value pair of the fields in the Subsequent steps.
Flatten1
I had made use of Flatten at the Document level.
And got the values of DocumentData.DocumentName and DocumentData.Fields
Note : If you don't want DocumentData.DocumentName - You can safely ignore it.
4) DerivedColumn1
This is actual step where I convert name:key1 key:value1 to key1:value1.
To do that I had made use of the below expression :
keyValues(Fields.Name,Fields.Value)
Note: Keyvalues() function expects 2 array arguments. Hence, in the first step we had changed the type of Fields.Name and Fields.Value to array.
4) Select
Just to select the columns that need to be sent as an output
Output
You mentioned SQL in your title so if you have access to a SQL database, eg Azure SQL DB, then it is quite capable with manipulating JSON, eg using the OPENJSON and FOR JSON PATH methods. A simple example:
DECLARE #json VARCHAR(MAX) = '[
{
"Version": "1.1",
"Documents": [
{
"DocumentState": "Correct",
"DocumentData": {
"Name": "Name1",
"$type": "Document",
"Fields": [
{
"Name": "Form",
"$type": "Text",
"Value": "Birthday Form"
},
{
"Name": "Date",
"$type": "Text",
"Value": "12/1/1999"
},
{
"Name": "FirstName",
"$type": "Text",
"Value": "John"
},
{
"Name": "FirstName",
"$type": "Text",
"Value": "Smith"
}
]
}
}
]
},
{
"Version": "1.1",
"Documents": [
{
"DocumentState": "Correct",
"DocumentData": {
"Name": "Name2",
"$type": "Document",
"Fields": [
{
"Name": "Form",
"$type": "Text",
"Value": "Entry Form"
},
{
"Name": "Date",
"$type": "Text",
"Value": "4/3/2010"
},
{
"Name": "FirstName",
"$type": "Text",
"Value": "Jane"
},
{
"Name": "LastName",
"$type": "Text",
"Value": "Doe"
}
]
}
}
]
}
]'
-- Restructure the JSON and add a root
SELECT *
FROM OPENJSON ( #json )
WITH
(
Form VARCHAR(50) '$.Documents[0].DocumentData.Fields[0].Value',
[Date] DATE '$.Documents[0].DocumentData.Fields[1].Value',
FirstName VARCHAR(50) '$.Documents[0].DocumentData.Fields[2].Value',
LastName VARCHAR(50) '$.Documents[0].DocumentData.Fields[3].Value'
)
FOR JSON PATH, ROOT('DocumentData');
My results:
NB I've used the ROOT clause to add a root to the JSON document. You could make the #json a stored proc parameter and use a Stored Proc task from the pipeline.

Make a ListView from rest API for Category only without duplication in Flutter

[
{
"category": "Apple",
"name": "Macbook Air"
}, {
"category": "Apple",
"name": "Macbook Pro"
}, {
"category": "Microsoft",
"name": "Surface"
}, {
"category": "Apple",
"name": "iPad"
}, {
"category": "Microsoft",
"name": "Windows"
}, {
"category": "Apple",
"name": "Siri"
}, {
"category": "Microsoft",
"name": "Office"
}
]
I need to the common category into ListView from the Rest API data exampled above.
There is Apple & Microsoft are the category which common in the six data.
It's automatically done it.
Could you give a solution for this?
import 'dart:convert';
import 'dart:math';
const source = '''
{
"data": [
{ "category": "Apple", "name": "Macbook Air" },
{ "category": "Apple", "name": "Macbook Pro" },
{ "category": "Microsoft", "name": "Surface" },
{ "category": "Apple", "name": "iPad" },
{ "category": "Microsoft", "name": "Windows" },
{ "category": "Apple", "name": "Siri" },
{ "category": "Microsoft", "name": "Office" }
]
}
''';
main(List<String> args) {
final List data = jsonDecode(source)['data'];
var mode = Map<String, int>();
data.map<String>((e) => (e as Map)['category']).forEach((k) => mode[k] = (mode[k] ?? 0) + 1);
var maxVal = mode.values.toList().reduce(max);
var category = List<String>();
mode.forEach((k, v) => v==maxVal ? category.add(k) : null);
print(category.toString());
}
List getCategory(List data) {
List list = [];
for (var a in data)
if (!list.contains(a["category"]))
list.add(a["category"]);
return list;
}

In Netsuite REST, how do I assign an item location to an item line in a sales order?

I'm using Netsuite's Postman collection (which takes care of the Oauth1 stuff), and am POSTing to this endpoint:
{{proto}}://{{host}}/rest/platform/{{version}}/record/salesorder
... and the body is something like this:
{
"customForm": "999",
"entity": {
"id": "1111"
},
"department": {
"id": "2222"
},
"subsidiary": {
"id": "33"
},
"otherRefNum": "TEST-PO",
"location": {
"id": "444"
},
"item": {
"items": [
{
"item": { "id": "555555" },
"inventorylocation": { "id": "444" },
"price": { "id": "-1" },
"grossAmt": 999,
"quantity": 1
}
]
}
}
I'm trying to assign a location on the item level. The above request creates a sales order ok (without the line-level location assignment) if I remove the inventorylocation line, but with that in there, I get this error: Unknown reference or subrecord field inventorylocation in request body.
Netsuite's REST API documentation is here:
https://system.netsuite.com/help/helpcenter/en_US/APIs/REST_API_Browser/record/v1/2019.2/index.html#tag-salesorder
I have also tried substituting location and moving the fields around a bit, without success. (either the salesorder is created without a line-level location assignment, or I get an error similar to the above error.
Any ideas?
From the documentation you linked, it appears that the field id you need is inventorylocation rather than itemlocation.
salesorder-itemElement
...
giftCertRecipientName Recipient Name: string
id [Missing Label:id]: string
inventorydetail: salesorder-item-inventorydetail
inventorylocation: location
inventorysubsidiary: subsidiary
isClosed Closed: boolean
...
Based on the documentation for a salesOrder-itemElement, it looks like that key is correct.
Have you tried the "location": "{ID}" variation?
In LedgerSync it looks like the request for creating an invoice results in this body:
{
"entity": "309",
"location": "1",
"sublist": {
"items": [
{
"amount": 12345,
"description": "Test Line Item FLURYAOLJLFADYGR-1"
},
{
"amount": 12345,
"description": "Test Line Item FLURYAOLUFUTBYJD-2"
}
]
}
}
There also is a salesOrder-item-inventorydetail object that also contains a location. Perhaps you could use that one like so:
{
"customForm": "999",
"entity": {
"id": "1111"
},
"department": {
"id": "2222"
},
"subsidiary": {
"id": "33"
},
"otherRefNum": "TEST-PO",
"location": {
"id": "444"
},
"item": {
"items": [
{
"item": { "id": "555555" },
"inventorydetail": {
"location": "444"
},
"price": { "id": "-1" },
"grossAmt": 999,
"quantity": 1
}
]
}
}

Using Mongoose with a rich document?

I'm working on a prototype that will be used for reporting (read only) where the record is a very rich set of objects embedded into a single document. Essentially the document structure is this (edited for brevity):
{
"_id": ObjectId("56b3af6f84ef45c8903acc51"),
"id": "7815dd97-e895-46e5-b6c9-45184c6eae89",
"survey": {
"id": "1fb21c69-6a5c-4805-b1cf-fabef7a5d0e6",
"type": "Survey",
"data": {
"description": "Testing reporting and data ouput",
"id": "1fb21c69-6a5c-4805-b1cf-fabef7a5d0e6",
"start_date": "2016-02-04T11:12:46Z",
"questions": [
{
"sequence": 1,
"modified_at": "2016-02-04T16:11:04.505849+00:00",
"id": "2a77921b-6853-463b-80e7-5713c82c51ca",
"previous_question": null,
"created_at": "2016-02-04T16:10:56.647746+00:00",
"parent_question": "",
"next_question": "",
"validators": [
"required",
"email"
],
"question_data": {
"modified_at": "2016-02-04T16:10:37.542715+00:00",
"type": "open-ended",
"text": "Please provide your email address",
"id": "27aa00db-4a56-4a3e-bc30-226179062af0",
"reporting_name": "email address",
"created_at": "2016-02-04T16:10:37.542695+00:00"
}
},
{
"sequence": 2,
"modified_at": "2016-02-04T16:09:53.539073+00:00",
"id": "c034819d-9281-4943-801f-c53f4047d03e",
"previous_question": null,
"created_at": "2016-02-04T16:09:53.539051+00:00",
"parent_question": "",
"next_question": null,
"validators": [
"alpha-numeric"
],
"question_data": {
"modified_at": "2016-02-04T16:05:31.008363+00:00",
"type": "open-ended",
"text": "Is there anything else that we could have done to improve your experience?",
"id": "e33c7804-20cb-4473-abfa-77b3c2a3113c",
"reporting_name": "more info open-ended",
"created_at": "2016-02-01T20:19:55.036899+00:00"
}
},
{
"sequence": 1,
"modified_at": "2016-02-04T16:08:55.681461+00:00",
"id": "f91fd70e-f204-4c38-9a56-dd6ff25e4cd8",
"previous_question": "",
"created_at": "2016-02-04T16:08:55.681441+00:00",
"parent_question": "",
"next_question": null,
"validators": [
"required"
],
"question_data": {
"modified_at": "2016-02-04T16:04:56.848528+00:00",
"type": "nps",
"text": "On a scale of 0-10 how likely are you to recommend us to a friend?",
"id": "fdb6b74d-96a3-4680-af35-8b2f6aa2bbc9",
"reporting_name": "key nps",
"created_at": "2016-02-01T20:19:27.371920+00:00"
}
}
],
"name": "Reporting Survey",
"end_date": "2016-02-11T11:12:47Z",
"trigger_active": false,
"created_at": "2016-02-04T16:13:16.808108Z",
"url": "http://www.peoplemetrics.com",
"fatigue_limit": "monthly",
"modified_at": "2016-02-04T16:13:16.808132Z",
"template": {
"id": "0ea02379-c80b-4e17-b0a6-d621d49076b9",
"type": "Template"
},
"landing_page": null,
"trigger": null,
"slug": "test-reporting-survey"
}
},
"invite_code": "7801",
"end_date": null,
"created_at": "2016-02-04T19:38:31.931147Z",
"url": "http://127.0.0.1:8000/api/v0/responses/7815dd97-e895-46e5-b6c9-45184c6eae89",
"answers": {
"data": [
{
"id": "bcc3d0dd-5419-4661-9900-ccda3ac9a308",
"end_datetime": "2016-01-22T19:57:03Z",
"survey_question": {
"id": "662fcdf9-3c92-415e-b779-ac5b0fd330d3",
"type": "SurveyQuestion"
},
"response": {
"id": "7815dd97-e895-46e5-b6c9-45184c6eae89",
"type": "Response"
},
"modified_at": "2016-02-04T19:38:31.972717Z",
"value_type": "number",
"created_at": "2016-02-04T19:38:31.972687Z",
"value": "10",
"slug": "",
"start_datetime": "2016-01-21T10:10:21Z"
},
{
"id": "8696f11e-679a-43da-b6e2-aee72a70ca9b",
"end_datetime": "2016-01-28T13:45:37Z",
"survey_question": {
"id": "f118c9dd-1c03-47e0-80ef-2a36eb3b9a29",
"type": "SurveyQuestion"
},
"response": {
"id": "7815dd97-e895-46e5-b6c9-45184c6eae89",
"type": "Response"
},
"modified_at": "2016-02-04T19:38:32.001970Z",
"value_type": "boolean",
"created_at": "2016-02-04T19:38:32.001939Z",
"value": "True",
"slug": "",
"start_datetime": "2016-02-15T04:51:24Z"
}
]
},
"modified_at": "2016-02-04T19:38:31.931171Z",
"start_date": "2016-02-01T16:14:13Z",
"invite_date": "2016-02-01T13:14:08Z",
"contact": {
"id": "94833455-b9b8-4206-9bc9-a2f96c1706ca",
"type": "Contact",
"external_contactid": null,
"name": "Miss Marceline Herzog PhD"
},
"referring_source": "web"
}
given a structure in that format, I'm unsure the best path forward using Mongoose as the ORM. Again, this is read-only, so I was it would seem that creating a nested schema would work, but the mapping itself seems tedious to say the least. Is there a better/different option available for something with embedded?
Interesting. First, I would think if I need all the document and its embedded subdocuments fields. You said it will be read-only, so will each call needs the entire document?
If not, I recommend taking a look at the mongo drivers (node.js, .NET, Python, etc.) and using their aggregation pipelines to simplify the document if possible.
If you're using Mongoose, you will probably end up with two or three Schemas, and with schemas inside a list. Mongoose docs e.g.
var surveySchema = new Schema(
{ "type" : string,
"data" : [dataSchema],
"invite_code" : string,
"end_date" : DateTime,
"created_at" : DateTime,
"url" : string,
"answers" : { "data": [answersSchema]},
"modified_at" : DateTime,
"start_date" : DateTime,
"invite_date" : DateTime,
"contact" : [ContactSchema],
"referring_source" : string
});
Or, you can use mongoose references and build your own schema depending on what data you need to use for your report. A simple example:
var surveySchema = {
"id" : { type: Schema.Types.ObjectId }
"description" : { type: string , ref: dataSchema },
"contactSchema" : { type: string , ref: contactSchema }
}

How can I use CloudKit web services to query based on a reference field?

I've got two CloudKit data objects that look somewhat like this:
Parent Object:
{
"records": [
{
"recordName": "14102C0A-60F2-4457-AC1C-601BC628BF47-184-000000012D225C57",
"recordType": "ParentObject",
"fields": {
"fsYear": {
"value": "2015",
"type": "STRING"
},
"displayOrder": {
"value": 2015221153856287200,
"type": "INT64"
},
"fjpFSGuidForReference": {
"value": "14102C0A-60F2-4457-AC1C-601BC628BF47-184-000000012D225C57",
"type": "STRING"
},
"fsDateSearch": {
"value": "2015221153856287158",
"type": "STRING"
},
},
"recordChangeTag": "id4w7ivn",
"created": {
"timestamp": 1439149087571,
"userRecordName": "_0d26968032e31bbc72c213037b6cb35d",
"deviceID": "A19CD995FDA3093781096AF5D818033A241D65C1BFC3D32EC6C5D6B3B4A9AA6B"
},
"modified": {
"timestamp": 1439149087571,
"userRecordName": "_0d26968032e31bbc72c213037b6cb35d",
"deviceID": "A19CD995FDA3093781096AF5D818033A241D65C1BFC3D32EC6C5D6B3B4A9AA6B"
}
}
],
"total":
}
Child Object:
{
"records": [
{
"recordName": "2015221153856287168",
"recordType": "ChildObject",
"fields": {
"District": {
"value": "002",
"type": "STRING"
},
"ZipCode": {
"value": "12345",
"type": "STRING"
},
"InspecReference": {
"value": {
"recordName": "14102C0A-60F2-4457-AC1C-601BC628BF47-184-000000012D225C57",
"action": "NONE",
"zoneID": {
"zoneName": "_defaultZone"
}
},
"type": "REFERENCE"
},
},
"recordChangeTag": "id4w7lew",
"created": {
"timestamp": 1439149090856,
"userRecordName": "_0d26968032e31bbc72c213037b6cb35d",
"deviceID": "A19CD995FDA3093781096AF5D818033A241D65C1BFC3D32EC6C5D6B3B4A9AA6B"
},
"modified": {
"timestamp": 1439149090856,
"userRecordName": "_0d26968032e31bbc72c213037b6cb35d",
"deviceID": "A19CD995FDA3093781096AF5D818033A241D65C1BFC3D32EC6C5D6B3B4A9AA6B"
}
}
],
"total": 1
}
I'm trying to write a query to directly access the CloudKit web service and return the Child Object based on the reference of the parent object.
My test JSON looks something like this:
{"query":{"recordType":"ChildObject","filterBy":{"fieldName":"InspecReference","fieldValue":{ "value" : "14102C0A-60F2-4457-AC1C-601BC628BF47-184-000000012D225C57", "type" : "string" },"comparator":"EQUALS"}},"zoneID":{"zoneName":"_defaultZone"}}
However, I'm getting the following error from CloudKit:
{"uuid":"33db91f3-b768-4a68-9056-216ecc033e9e","serverErrorCode":"BAD_REQUEST","reason":"BadRequestException:
Unexpected input"}
I'm guessing I have the Record Field Dictionary in the query wrong. However, the documentation isn't clear on what this should look like on a reference object.
You have to re-create the actual object of the reference. In this particular case, the JSON looks like this:
{
"query": {
"recordType": "ChildObject",
"filterBy": {
"fieldName": "InspecReference",
"fieldValue": {
"value": {
"recordName": "14102C0A-60F2-4457-AC1C-601BC628BF47-184-000000012D225C57",
"action": "NONE"
},
"type": "REFERENCE"
},
"comparator": "EQUALS"
}
},
"zoneID": {
"zoneName": "_defaultZone"
}
}