JSON schema (2020-12) to scala case classes - scala

I am unable to find any library to convert nested JSON schema(2020-12) to scala case classes for scala project.
I am using api first approach, so its necessary.
Found few projects supporting scala case class to JSON Schema but none for JSON schema to scala case class.
Looking for something like this
Example JSON:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "http://example.com/product.schema.json",
"title": "Product",
"type": "object",
"properties": {
"productId": {
"type": "integer"
},
"tags": {
"description": "Tags for the product",
"type": "array",
"items": {
"type": "string"
},
"minItems": 1,
"uniqueItems": true
},
"dimensions": {
"type": "object",
"properties": {
"length": {
"type": "number"
},
"width": {
"type": "number"
},
"height": {
"type": "number"
}
},
"required": [
"length"
]
}
},
"required": [
"productId",
"price"
]
}
Converted to:
case class Product (
productId: Int,
tags: Option[List[String]],
dimensions: Option[Dimensions]
){
assert( tags.get.length >= 1, "`tags.get` violates 'minItems' constraint" )
assert( tags.get.length == tags.get.distinct.length, "`tags.get` violates 'uniqueItems' constraint" )
}
/**
* #param length
* #param width
* #param height
*/
case class Dimensions (
length: Double,
width: Option[Double],
height: Option[Double]
)

Related

PubSub Subscription error with REPEATED Column Type - Avro Schema

I am trying to use the PubSub Subscription "Write to BigQuery" but am running into an issue with the "REPEATED" column type. the message I get when update the subscription is
Incompatible schema mode for field 'Values': field is REQUIRED in the topic schema, but REPEATED in the BigQuery table schema
My Avro Schema is:
{
"type": "record",
"name": "Avro",
"fields": [
{
"name": "ItemID",
"type": "string"
},
{
"name": "UserType",
"type": "string"
},
{
"name": "Values",
"type": [
{
"type": "record",
"name": "Values",
"fields": [
{
"name": "AttributeID",
"type": "string"
},
{
"name": "AttributeValue",
"type": "string"
}
]
}
]
}
]
}
Input JSON That "Matches" Schema:
{
"ItemID": "Item_1234",
"UserType": "Item",
"Values": {
"AttributeID": "TEST_ID_1",
"AttributeValue": "Value_1"
}
}
my Table looks like:
ItemID | STRING | NULLABLE
UserType | STRING | NULLABLE
Values | RECORD | REPEATED
AttributeID | STRING | NULLABLE
AttributeValue | STRING | NULLABLE
I am able to "Test" and "Validate Schema" and it comes back with a success. Question is, what am I missing on the Avro for the Values node to make it "REPEATED" vs "Required" for subscription to be created.
The issue is that Values is not an array type in your Avro schema, meaning it expects only one in the message, while it is a repeated type in your BigQuery schema, meaning it expects a list of them.
Per Kamal's comment above, this schema works:
{
"type": "record",
"name": "Avro",
"fields": [
{
"name": "ItemID",
"type": "string"
},
{
"name": "UserType",
"type": "string"
},
{
"name": "Values",
"type": {
"type": "array",
"items": {
"name": "NameDetails",
"type": "record",
"fields": [
{
"name": "ID",
"type": "string"
},
{
"name": "Value",
"type": "string"
}
]
}
}
}
]
}
the payload:
{
"ItemID": "Item_1234",
"UserType": "Item",
"Values": [
{ "AttributeID": "TEST_ID_1" },
{ "AttributeValue": "Value_1" }
]
}

AWS-API gateway -- jsonschema child object should validate when parent object exists

I need to create Jsonschema for the following JSON input. Here properties under Vehicle like( Manufacturer, Model, etc) should be required only when Vehicle object exists.
{
"Manufacturer": "",
"Characteristics": {
"Starts": "new",
"vehicle": {
"Manufacturer": "hello",
"Model": "hh",
"Opening": "",
"Quantity": "",
"Principle": "",
"Type": ""
}
}
}
I tried the following JsonSchema but this works when Vehicle object is not there but if we rename Vehicle to some other ex: Vehicle1 it doesn't give an error. Please guide me on how to fix this.
{
"$schema": "http://json-schema.org/draft-07/schema",
"type": "object",
"properties": {
"Manufacturer": {
"type": [
"string",
"null"
]
},
"Characteristics": {
"type": "object",
"properties": {
"Starts": {
"type": [
"string",
"null"
]
},
"Vehicle": {
"$ref": "#/definitions/Vehicle"
}
},
"required": [
"Starts", "Vehcle"
]
}
},
"required": [
"Manufacturer"
],
"definitions": {
"Vehicle": {
"type": "object",
"properties": {
"Manufacturer": {
"type": [
"string",
"null"
]
},
"Model": {
"type": [
"string",
"null"
]
},
"Opening": {
"type": [
"string",
"null"
]
},
"PanelQuantity": {
"type": [
"string",
"null"
]
},
"Principle": {
"type": [
"string",
"null"
]
},
"Type": {
"type": [
"string",
"null"
]
}
},
"required": ["Manufacturer", "Model", "Opening", "Quantity", "Principle", "Type"]
}
}
}
Thanks,
Bhaskar
Sounds like you want to add "additionalProperties": false -- which will generate an error if any other properties are present that aren't defined under properties.

Open API 3.0 parameter dependencies: required parameters if type is "one of" (with shared parameters)

I'm creating an openapi.json (version 3.0.3) schema and I'm modelling a post request. The body can look like this:
{
type: "A",
aParam: "string",
sharedParam1: "string",
sharedParam2: "integer",
sharedParam3: "string"
}
where type is one of A or B. If the type is A, the parameter aParam is required if the type is B aParam must be left out. Basically, the other way the request can look is:
{
type: "B",
sharedParam1: "string",
sharedParam2: "integer",
sharedParam3: "string"
}
How can I model this?
Here is what I tried:
{
"requestBody": {
"content": {
"application/json": {
"schema": {
"oneOf": [
{
"type": "object",
"properties": {
"type": {
"type": "string",
"enum": ["A"]
},
"aParam": {
"type": "string"
},
"sharedParam1": {
"type": "string"
},
"sharedParam2": {
"type": "string"
},
"sharedParam3": {
"type": "string"
}
}
},
{
"type": "object",
"properties": {
"type": {
"type": "string",
"enum": ["B"]
},
"sharedParam1": {
"type": "string"
},
"sharedParam2": {
"type": "string"
},
"sharedParam3": {
"type": "string"
}
}
}
]
}
}
}
}
}
Basically, I "overloaded" the request body by using oneOf but that has a lot of duplication.
You may extract the shared properties to a base schema. It won't make the definition much less verbose but at least will remove duplicated properties definitions making them more maintainable:
"components": {
"schemas": {
"baseRequestBody": {
"type": "object",
"required": [
"type",
"sharedParam1",
"sharedParam2",
"sharedParam3"
],
"properties": {
"type": {
"type": "string",
"enum": [
"A",
"B"
]
},
"sharedParam1": {
"type": "string"
},
"sharedParam2": {
"type": "integer"
},
"sharedParam3": {
"type": "string"
}
}
},
"requestBodyA": {
"allOf": [
{
"$ref": "#/components/schemas/baseRequestBody"
},
{
"type": "object",
"required": [
"aParam"
],
"properties": {
"aParam": {
"type": "string"
}
}
}
]
},
"requestBodyB": {
"allOf": [
{
"$ref": "#/components/schemas/baseRequestBody"
}
]
}
}
}
Additionally, you might want to use Discriminator which can be used by some tools like code generators:
"requestBody": {
"content": {
"application/json": {
"schema": {
"oneOf": [
{
"$ref": "#/components/schemas/requestBodyA"
},
{
"$ref": "#/components/schemas/requestBodyB"
}
],
"discriminator": {
"propertyName": "type",
"mapping": {
"A": "#/components/schemas/requestBodyA",
"B": "#/components/schemas/requestBodyB"
}
}
}
}
}
}

Elastic4s ngram mapping

I have to create an ElasticSearch mapping like this using elastic4s:
"mappings": {
"properties": {
"id": {
"type": "keyword"
},
"name": {
"type": "text",
"analyzer": "ngram_analyzer",
"fielddata": true
},
"lang": {
"type": "keyword"
},
"order": {
"type": "long"
},
"active": {
"type": "boolean"
}
"description": {
"type": "text"
}
}
}
I can do
def mapping: Option[MappingDefinition] =
Some(
properties(
KeywordField("id"),
KeywordField("lang"),
BasicField("order", "long"),
BasicField("active", "boolean"),
TextField("description")
)
)
for id, lang, order, active and description.
But, how can I do such mapping for name. the problem is analyzer and fielddata inside it.
You should use this:
TextField("name").fielddata(true).analyzer("ngram_analyzer")
You also need to make sure to properly create the ngram_analyzer in your index settings.

Search and replace JSON multiline using regex in VSCode

I have a really long JSON schema. Using VSCode, I need to replace the partnerName type to be string, null (it appears more than 20 times, the snippet below is just 1 appearance).
How can I search and replace the multiline for the entire partnerName entry?
From other question, I've tried using regex [\n\s]+, (.*\n)+ to be
"partnerName": {(.*\n)+"type": "null"(.*\n)+}
But it's still not matching.
Search for:
"partnerName": {
"type": "null"
},
Replace with:
"partnerName": {
"type": "string, null"
},
Snippet example:
{
"type": "object",
"properties": {
"node": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"name": {
"type": "string"
},
"description": {
"type": "string"
},
"type": {
"type": "string"
},
"frequency": {
"type": "string"
},
"maxCount": {
"type": "integer"
},
"points": {
"type": "integer"
},
"startAt": {
"type": "string"
},
"endAt": {
"type": "string"
},
"partnerName": {
"type": "null"
},
"action": {
"type": "null"
}
},
"required": [
"id",
"name",
"description",
"type",
"frequency",
"maxCount",
"points",
"startAt",
"endAt",
"partnerName",
"action"
]
}
},
"required": [
"node"
]
},
Try this regex:
(partnerName".*\n\s*"type":\s*)"null"
and replace with:
$1"string, null"