Usage of $ref in json schema file - rest

I have a folder named schema having employee.schema file.
In the code snippet below address attribute is an array type.
I want to have address.schema file in the same folder where employee.schema is kept and refer it in employee.schema
Is this possible with draft-03 schema?
{
"type":"object",
"$schema": "http://json-schema.org/draft-03/schema",
"properties":{
"empId":{
"type":"integer",
"required":false
},
"empName":{
"type":"string",
"required":true,
"minLength":10,
"maxLength":20
},
"contactno":{
"type":"string",
"required":true,
"minLength":10,
"maxLength":10
},
"salary":{
"type":"integer",
"required":true
},
"address":{
"type":"array",
"items":{
"type": "object",
"properties": {
"city":{
"type":"string",
"required":true
},
"pincode":{
"type":"string",
"required":true
}
}
}
}
}
}

I think the draft-03 spec says it all:
This attribute defines a URI of a schema that contains the full
representation of this schema. When a validator encounters this
attribute, it SHOULD replace the current schema with the schema
referenced by the value's URI (if known and available) and re-
validate the instance. This URI MAY be relative or absolute, and
relative URIs SHOULD be resolved against the URI of the current
schema.
(https://datatracker.ietf.org/doc/html/draft-zyp-json-schema-03#section-5.28)
Simply define "address" as a "$ref" that points to that other schema.
{
"type": "object",
"$schema": "http://json-schema.org/draft-03/schema",
"properties": {
"empId": {
"type": "integer",
"required": false
},
"empName": {
"type": "string",
"required": true,
"minLength": 10,
"maxLength": 20
},
"contactno": {
"type": "string",
"required": true,
"minLength": 10,
"maxLength": 10
},
"salary": {
"type": "integer",
"required": true
},
"address": {
"$ref": "address.schema"
}
}
}
and then "address.schema" would look like this:
{
"type":"array",
"$schema": "http://json-schema.org/draft-03/schema",
"items": {
"type": "object",
"properties": {
"city": {
"type":"string",
"required":true
},
"pincode": {
"type":"string",
"required":true
}
}
}
}

Related

How is the value NULL?

I am getting query results that determine if user story hasn't been changed (changedate) in the last one day.
I'm following this article to build the logic app as the intention is similar
For some reason, despite the query returning a valid response (at least 1 user story result), the foreach expression is throwing this error:
ExpressionEvaluationFailed. The execution of template action 'For_each' failed: the result of the evaluation of 'foreach' expression '#body('Parse_JSON')?['body']?['value']' is of type 'Null'. The result must be a valid array.
How is it NULL when clearly there is a user story returned?
Get query results:
OUTPUTS:
[
{
"System.Id": 12345,
"System.WorkItemType": "User Story",
"System.State": "New",
"System.Title": "Experiment"
}
]
Parse JSON:
Inputs:
Content:
{
"value": [
{
"System.Id": 12345,
"System.WorkItemType": "User Story",
"System.State": "New",
"System.Title": "Experiment"
}
],
"#odata.nextLink": null
}
Schema
{
"type": "object",
"properties": {
"body": {
"type": "object",
"properties": {
"value": {
"type": "array",
"items": {
"type": "object",
"properties": {
"System.AssignedTo": {
"type": "string"
},
"System.Id": {
"type": "integer"
},
"System.State": {
"type": "string"
},
"System.Tags": {
"type": "string"
},
"System.Title": {
"type": "string"
},
"System.WorkItemType": {
"type": "string"
}
},
"required": [
"System.Id",
"System.WorkItemType",
"System.State",
"System.AssignedTo",
"System.Title"
]
}
},
"#odata.nextLink": {}
}
},
"headers": {
"type": "object",
"properties": {
"Cache-Control": {
"type": "string"
},
"Content-Length": {
"type": "string"
},
"Content-Type": {
"type": "string"
},
"Date": {
"type": "string"
},
"Expires": {
"type": "string"
},
"Pragma": {
"type": "string"
},
"Set-Cookie": {
"type": "string"
},
"Strict-Transport-Security": {
"type": "string"
},
"Timing-Allow-Origin": {
"type": "string"
},
"Transfer-Encoding": {
"type": "string"
},
"Vary": {
"type": "string"
},
"X-Content-Type-Options": {
"type": "string"
},
"X-Frame-Options": {
"type": "string"
},
"x-ms-apihub-cached-response": {
"type": "string"
},
"x-ms-apihub-obo": {
"type": "string"
},
"x-ms-request-id": {
"type": "string"
}
}
},
"statusCode": {
"type": "integer"
}
}
}
Outputs:
{
"value": [
{
"System.Id": 12345,
"System.WorkItemType": "User Story",
"System.State": "New",
"System.Title": "Experiment"
}
],
"#odata.nextLink": null
}
Using the Value from the Get Query Results directly works.

How to encrypt Azure Data Factory with CMK (customer managed key)

Parameters.json
"parameters": {
"dataFactoryName": { "type": "string", "metadata": { "description": "Name of the data factory. Name must be globally unique" } },
"resourceTags": { "type": "object" },
"diagnosticSettingsStorageAccount": { "type": "string", "metadata": { "description": "Resource ID of the storage account used to store diagnostic logs" } },
"cmkIdentity": {
"type": "string"
},
"vaultBaseUrl": {
"type": "string"
},
"keyName": {
"type": "string"
},
"keyVersion": {
"type": "string"
}
},
Template.json
{
"type": "Microsoft.DataFactory/factories",
"apiVersion": "2018-06-01",
"name": "[parameters('dataFactoryName')]",
"location": "[resourceGroup().location]",
"tags": "[parameters('resourceTags')]",
"identity": {
"type": "SystemAssigned,UserAssigned",
"userAssignedIdentities": {"[parameters('cmkIdentity')]": {}}},
"properties": {
"publicNetworkAccess": "Disabled",
"encryption": {
"identity": {
"userAssignedIdentity": "[parameters('cmkIdentity')]"
},
"vaultBaseUrl": "[parameters('vaultBaseUrl')]",
"keyName": "[parameters('keyName')]",
"keyVersion": "[parameters('keyVersion')]"
}
},
"dependsOn": ["[resourceId('Microsoft.OperationalInsights/workspaces',variables('workspaceName'))]"]
},
I am passing these values in:
cmkIdentity: "/subscriptions/xxxxx/resourcegroups/xxxxx/providers/Microsoft.ManagedIdentity/userAssignedIdentities/test-identity"
vaultBaseUrl: https://testkeyvault123.vault.azure.net/
keyName: test-key
keyVersion: t5dca2a5xxxxx399we5
The Validation passes and the Data Factory is deployed. I can see the test-identity in the Managed Identity section. But when I open the Data Factory's UI and navigate to Manage and in Customer managed key I don't see anything. All fields are empty, see picture below:
Provided access policy to test-identity to the test key vault. Can't figure out what the problem is.
UPDATED parameters and template
parameters
"parameters": {
"dataFactoryName": { "type": "string", "metadata": { "description": "Name of the data factory. Name must be globally unique" } },
"resourceTags": { "type": "object" },
"diagnosticSettingsStorageAccount": { "type": "string", "metadata": { "description": "Resource ID of the storage account used to store diagnoistic logs" } },
"cmkIdentity": {
"type": "object",
"defaultValue": {
"/subscriptions/xxxxx/resourcegroups/xxxxx/providers/Microsoft.ManagedIdentity/userAssignedIdentities/test-identity": {
}
}
},
"vaultBaseUrl": {
"type": "string"
},
"keyName": {
"type": "string"
},
"keyVersion": {
"type": "string"
}
},
template.json
{
"type": "Microsoft.DataFactory/factories",
"apiVersion": "2018-06-01",
"name": "[parameters('dataFactoryName')]",
"location": "[resourceGroup().location]",
"tags": "[parameters('resourceTags')]",
"identity": {
"type": "SystemAssigned,UserAssigned",
"principalId": "",
"tenantId": "",
"userAssignedIdentities": "[parameters('cmkIdentity')]"
},
"properties": {
"publicNetworkAccess": "Disabled",
"encryption": {
"identity": {
"userAssignedIdentity": "[parameters('cmkIdentity')]"
},
"vaultBaseUrl": "[parameters('vaultBaseUrl')]",
"keyName": "[parameters('keyName')]",
"keyVersion": "[parameters('keyVersion')]"
}
},
"dependsOn": ["[resourceId('Microsoft.OperationalInsights/workspaces',variables('workspaceName'))]"]
},
I have tried doing it both the ways via json template and even through portal works fine, just to have another parameter which is object type as below:
Updated Info : Please add a new parameter as an cmkidentity_obj which is type: object and have the other parameter cmkidentity as type: string
and pass the string in :
"encryption": {
"identity": {
"userAssignedIdentity": "[parameters('cmkidentity')]"
},
"VaultBaseUrl": "[parameters('dataFactory_properties_encryption_VaultBaseUrl')]",
"KeyName": "[parameters('dataFactory_properties_encryption_KeyName')]",
"KeyVersion": "[parameters('dataFactory_properties_encryption_KeyVersion')]"
}
Parameter- cmkidentity_obj abd pass it in below template.json :
"cmkidentity_obj": {
"type": "object",
"defaultValue": {
"/subscriptions/xxxxx/resourcegroups/xxxxx/providers/Microsoft.ManagedIdentity/userAssignedIdentities/test-identity": {
}
}
}
and then passed this object in my template.json :
"identity": {
"type": "[parameters('dataFactory_identity_type')]",
"principalId": "",
"tenantId": "",
"userAssignedIdentities": "[parameters('cmkidentity_obj')]"
}
This deployed successfully with about any error and was able to view my Customer Managed Key in Azure Data Factory(UI), kindly try the same and see.
My Template.json:
"resources": [
{
"name": "[parameters('factoryName')]",
"type": "Microsoft.DataFactory/factories",
"apiVersion": "2018-06-01",
"properties": {
"encryption": {
"identity": {
"userAssignedIdentity": "[parameters('cmkidentity')]"
},
"VaultBaseUrl": "[parameters('dataFactory_properties_encryption_VaultBaseUrl')]",
"KeyName": "[parameters('dataFactory_properties_encryption_KeyName')]",
"KeyVersion": "[parameters('dataFactory_properties_encryption_KeyVersion')]"
}
},
"dependsOn": [],
"location": "[parameters('dataFactory_location')]",
"identity": {
"type": "[parameters('dataFactory_identity_type')]",
"principalId": "",
"tenantId": "",
"userAssignedIdentities": "[parameters('cmkIdentity_obj')]"
}
}
]

Getting error on null and empty string while copying a csv file from blob container to Azure SQL DB

I tried all combination on the datatype of my data but each time my data factory pipeline is giving me this error:
{
"errorCode": "2200",
"message": "ErrorCode=UserErrorColumnNameNotAllowNull,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Empty or Null string found in Column Name 2. Please make sure column name not null and try again.,Source=Microsoft.DataTransfer.Common,'",
"failureType": "UserError",
"target": "xxx",
"details": []
}
My Copy data source code is something like this:{
"name": "xxx",
"description": "uuu",
"type": "Copy",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "DelimitedTextSource",
"storeSettings": {
"type": "AzureBlobStorageReadSettings",
"recursive": true,
"wildcardFileName": "*"
},
"formatSettings": {
"type": "DelimitedTextReadSettings"
}
},
"sink": {
"type": "AzureSqlSink"
},
"enableStaging": false,
"translator": {
"type": "TabularTranslator",
"mappings": [
{
"source": {
"name": "populationId",
"type": "Guid"
},
"sink": {
"name": "PopulationID",
"type": "String"
}
},
{
"source": {
"name": "inputTime",
"type": "DateTime"
},
"sink": {
"name": "inputTime",
"type": "DateTime"
}
},
{
"source": {
"name": "inputCount",
"type": "Decimal"
},
"sink": {
"name": "inputCount",
"type": "Decimal"
}
},
{
"source": {
"name": "inputBiomass",
"type": "Decimal"
},
"sink": {
"name": "inputBiomass",
"type": "Decimal"
}
},
{
"source": {
"name": "inputNumber",
"type": "Decimal"
},
"sink": {
"name": "inputNumber",
"type": "Decimal"
}
},
{
"source": {
"name": "utcOffset",
"type": "String"
},
"sink": {
"name": "utcOffset",
"type": "Int32"
}
},
{
"source": {
"name": "fishGroupName",
"type": "String"
},
"sink": {
"name": "fishgroupname",
"type": "String"
}
},
{
"source": {
"name": "yearClass",
"type": "String"
},
"sink": {
"name": "yearclass",
"type": "String"
}
}
]
}
},
"inputs": [
{
"referenceName": "DelimitedTextFTDimensions",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "AzureSqlTable1",
"type": "DatasetReference"
}
]
}
Can anyone please help me understand the issue. I see in some blogs they ask me use treatnullasempty but I am not allowed to modify the JSON. is there a way to do that??
I suggest to using Data Flow DerivedColumn, DerivedColumn can help you build expression to replace the null column.
For example:
Derived Column, if Column_2 is null =true, return 'dd' :
iifNull(Column_2,'dd')
Mapping the column
Reference: Data transformation expressions in mapping data flow
Hope this helps.
fixed it.it was a easy fix as one of my column in destination was marked as not null, i changed it as null and it worked.

Optional object with required fields in react-jsonschema-form

Given a json schema like the one below, the react-jsonschema-form validator essentially requires both shipping_address and billing_address even though the billing_address is not listed as required. This is because the address type requires all three of its properties. How can I make the billing_address optional? It seems that react-jsonschema-form should simply no submit billing_address if not all of its address properties are filled in. Here is a link to the react-jsonschema-form playground.
{
"definitions": {
"address": {
"type": "object",
"properties": {
"street_address": {
"type": "string"
},
"city": {
"type": "string"
},
"state": {
"type": "string"
}
},
"required": [
"street_address",
"city",
"state"
]
}
},
"type": "object",
"properties": {
"billing_address": {
"title": "Billing address",
"$ref": "#/definitions/address"
},
"shipping_address": {
"title": "Shipping address",
"$ref": "#/definitions/address"
}
},
"required": [
"shipping_address"
]
}
You can use dynamic schema dependencies to make the billing address conditionally displayed and required. This isn't the same as having an optional object but seem to suffice if you're willing to have a slightly different user experience. Here is a link to the react-jsonschema-form playground. It is best viewed, in my opinion, with live validation disabled (there's a checkbox in the upper-right of the page).
{
"definitions": {
"address": {
"type": "object",
"properties": {
"street_address": {
"type": "string"
},
"city": {
"type": "string"
},
"state": {
"type": "string"
}
},
"required": [
"street_address",
"city",
"state"
]
}
},
"type": "object",
"properties": {
"different_addresses": {
"title": "My billing address is different than my shipping address.",
"type": "boolean",
"default": false
},
"shipping_address": {
"title": "Shipping address",
"$ref": "#/definitions/address"
}
},
"required": [
"shipping_address"
],
"dependencies": {
"different_addresses": {
"oneOf": [
{
"properties": {
"different_addresses": {
"enum": [
false
]
}
}
},
{
"properties": {
"different_addresses": {
"enum": [
true
]
},
"billing_address": {
"title": "Billing address",
"$ref": "#/definitions/address"
}
},
"required": [
"billing_address"
]
}
]
}
}
}
my question is related to your JSON schema.
I need to target the "grandchild" of a parent object for a dependency.
Is this possible? In case of "different_addresses" being an object.
For example:
"dependencies": {
"different_addresses": {
"properties": {
"OTHER_FIELD": {
"oneOf": [
{
"properties": {
"different_addresses": {
"properties": {
"OTHER_FIELD": {
"enum": [
false
]
}
}
}
}
}
]
}
}
}
}

How do I change the schema of a postgres data source in strongloop?

I'm trying to tell strongloop that my gallery table has moved to the products schema. Adding it to the model definition in common/models/gallery.json seemingly has no effect. New to strongloop. What am I doing wrong?
My current schema. "schema": "products" is the only thing added.
{
"name": "gallery",
"plural": "galleries",
"base": "PersistedModel",
"idInjection": true,
"options": {
"validateUpsert": true,
"schema": "products"
},
"properties": {
"id": {
"type": "number"
},
"name": {
"type": "string",
"required": true
},
"description": {
"type": "string"
},
"uuid": {
"type": "uuid"
},
"test": {
"type": "string"
},
"order": {
"type": "number"
}
},
"validations": [],
"relations": {},
"acls": [],
"methods": {}
}
uuid is a placeholder
Answer is here: https://docs.strongloop.com/display/public/LB/PostgreSQL+connector
Correct options value:
"options": {
"validateUpsert": true,
"postgresql": {
"schema": "products"
}
}