Why OpenAPI does not define '$ref' as allowed property? - openapi

In compare to draft-07 it defines:
{
"type": ["object", "boolean"],
"properties": {
...
"$ref": {
"type": "string",
"format": "uri-reference"
},
}
...
}
Currently I am trying to write validator for openapi. But validation fails because openapi schema (yes, it is schema from google apis) does not define $ref as allowed property for schema.
Is this a typo? What is recommendation about how to check $ref property?

$ref is a JSON Reference. It's not part of the schema definition, instead it's part of the reference definition:
"reference": {
"type": "object",
"description": "A simple object to allow referencing other components in the specification, internally and externally. The Reference Object is defined by JSON Reference and follows the same structure, behavior and rules. For this specification, reference resolution is accomplished as defined by the JSON Reference specification and not by the JSON Schema specification.",
"required": [
"$ref"
],
"additionalProperties": false,
"properties": {
"$ref": {
"type": "string"
}
}
},
And then other definitions where $ref is allowed use oneOf something or reference (example):
"schemaOrReference": {
"oneOf": [
{
"$ref": "#/definitions/schema"
},
{
"$ref": "#/definitions/reference"
}
]
},
By the way, there are currently two different draft OAS3 JSON Schemas in the official OpenAPI Specification repository. Feel free to try them instead, and provide your feedback in the corresponding discussions.
[WIP] Alternative OAS3 JSON Schema (link to schema)
OpenAPI v3 JSON Schema (link to schema)

Related

JSONAPI: is it correct to have different types in data[]?

is it correct to have different types in data array ? didn't found answer in specification, but looks wrong.
{
"data": [
{
"type": "type1",
"id": "id1"
},
{
"type": "type2",
"id": "id2"
}
]
}
Polymorphic collections are supported by JSON:API specification. Your example is a valid JSON:API document.
This is not mentioned explicitly in the specification. It is only implicitly stated by not requiring all resource objects included as primary data to have the same type.
Please note that it could even include two resource objects with the same id as long as they have different type.

Kafka Connect JSON Schema does not appear to support "$ref" tags

I am using Kafka Connect with JSONSchema and am in a situation where I need to convert the JSON schema manually (to "Schema") within a Kafka Connect plugin. I can successfully retrieve the JSON Schema from the Schema Registry and am successful converting with simple JSON Schemas but I am having difficulties with ones that are complex and have valid "$ref" tags referencing components within a single JSON Schema definition.
I have several questions:
The JsonConverter.java does not appear to handle "$ref". Am I correct, or does it handle it in another way elsewhere?
Does the Schema Registry handle the referencing of sub-definitions? If yes, is there code that shows how the dereferencing is handled?
Should the JSON Schema be resolved to a string without references (ie. inline the references) before submitting to the Schema Registry and thereby remove the "$ref" issue?
I am looking at the Kafka Source code module JsonConverter.java below:
https://github.com/apache/kafka/blob/trunk/connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java#L428
An example of the complex schema (taken from the JSON Schema site) is shown below (notice the "$ref": "#/$defs/veggie" tag the references a later sub-definition)
{
"$id": "https://example.com/arrays.schema.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"description": "A representation of a person, company, organization, or place",
"title": "complex-schema",
"type": "object",
"properties": {
"fruits": {
"type": "array",
"items": {
"type": "string"
}
},
"vegetables": {
"type": "array",
"items": { "$ref": "#/$defs/veggie" }
}
},
"$defs": {
"veggie": {
"type": "object",
"required": [ "veggieName", "veggieLike" ],
"properties": {
"veggieName": {
"type": "string",
"description": "The name of the vegetable."
},
"veggieLike": {
"type": "boolean",
"description": "Do I like this vegetable?"
}
}
}
}
}
Below is the actual schema returned from the Schema Registry after it the schema was successfully registered:
[
{
"subject": "complex-schema",
"version": 1,
"id": 1,
"schemaType": "JSON",
"schema": "{\"$id\":\"https://example.com/arrays.schema.json\",\"$schema\":\"https://json-schema.org/draft/2020-12/schema\",\"description\":\"A representation of a person, company, organization, or place\",\"title\":\"complex-schema\",\"type\":\"object\",\"properties\":{\"fruits\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"vegetables\":{\"type\":\"array\",\"items\":{\"$ref\":\"#/$defs/veggie\"}}},\"$defs\":{\"veggie\":{\"type\":\"object\",\"required\":[\"veggieName\",\"veggieLike\"],\"properties\":{\"veggieName\":{\"type\":\"string\",\"description\":\"The name of the vegetable.\"},\"veggieLike\":{\"type\":\"boolean\",\"description\":\"Do I like this vegetable?\"}}}}}"
}
]
The actual schema is embedded in the above returned string (the contents of the "schema" field) and contains the $ref references:
{\"$id\":\"https://example.com/arrays.schema.json\",\"$schema\":\"https://json-schema.org/draft/2020-12/schema\",\"description\":\"A representation of a person, company, organization, or place\",\"title\":\"complex-schema\",\"type\":\"object\",\"properties\":{\"fruits\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"vegetables\":{\"type\":\"array\",\"items\":{\"$ref\":\"#/$defs/veggie\"}}},\"$defs\":{\"veggie\":{\"type\":\"object\",\"required\":[\"veggieName\",\"veggieLike\"],\"properties\":{\"veggieName\":{\"type\":\"string\",\"description\":\"The name of the vegetable.\"},\"veggieLike\":{\"type\":\"boolean\",\"description\":\"Do I like this vegetable?\"}}}}}
Again, the JsonConverter in the Apache Kafka source code has no notion of JSONSchema, therefore, no, $ref doesn't work and it also doesn't integrate with the Registry.
You seem to be looking for the io.confluent.connect.json.JsonSchemaConverter class + logic

How do I use an extended definition and not allow additional properties in a way that is compatible with multiple validators (JSON schema draft 7)?

I am creating a strict validator for a complex JSON file and want to re-use various definitions in order to keep the schema manageable and easier to update.
According to the documentation it is necessary to use allOf to extend a definition to add more properties. This is exactly what I've done, but I find that without use of additionalProperties set to false validation doesn't prevent arbitrary other properties being added.
The following massively cut-down schema demonstrates what I'm doing:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://example.com/schema/2021/02/example.json",
"description": "This schema demonstrates how VSCode's JSON schema mechanism fails with allOf used to extend a definition",
"definitions": {
"valueProvider": {
"type": "object",
"properties": {
"example": {
"type": "string"
},
"alternative": {
"type": "string"
}
},
"oneOf": [
{
"required": [
"example"
]
},
{
"required": [
"alternative"
]
}
]
},
"selector": {
"type": "object",
"allOf": [
{
"$ref": "#/definitions/valueProvider"
},
{
"required": [
"operator",
"value"
],
"properties": {
"operator": {
"type": "string",
"enum": [
"IsNull",
"Equals",
"NotEquals",
"Greater",
"GreaterOrEquals",
"Less",
"LessOrEquals"
]
},
"value": {
"type": "string"
}
}
}
],
"additionalProperties": false
}
},
"properties": {
"show": {
"properties": {
"name": {
"type": "string"
},
"selector": {
"description": "This property does not function correctly in VSCode",
"allOf": [
{
"$ref": "#/definitions/selector"
},
{
"additionalProperties": false
}
]
}
},
"additionalProperties": false
}
}
}
This works a treat in IntelliJ IDEA's JSON editor (2020.3.2 ultimate edition) when editing JSON against this schema (using a schema mapping). For example, the file ex-fail.json's content of:
{
"show": {
"name": "a",
"selector": {
"example": "a",
"operator": "IsNull",
"value": "false",
"d": "a"
}
}
}
Is correctly validated, simply highlighting "d" as not allowed, thus:
However, when I use the very same schema and JSON file with VSCode (1.53.2) with vanilla configuration (except for a schema mapping) VSCode erroneously marks "example", "operator", "value" and "d" as not allowed. It looks like this in the VSCode editor:
If I remove the additionalProperties definition from the show.selector property, both IDEA and VSCode indicate that all is well, including allowing the "d" property - in doing this I can simplify that property definition to:
"selector": {
"description": "This property does not function correctly in VSCode",
"$ref": "#/definitions/selector"
}
What can I do to the schema to support both IDEA and VSCode whilst disallowing additional properties where they should not appear?
PS: The schema mapping in VSCode is simply along the lines of:
{
"json.schemas": [
{
"fileMatch": [
"*/config/ex-*.json"
],
"url": "file:///C:/my/path/to/example-schema.json"
}
]
}
You cannot do what you ask with JSON Schema draft-07 or prior.
The reason is, when $ref is used in a schema object, all other properties MUST be ignored.
An object schema with a "$ref" property MUST be interpreted as a
"$ref" reference. The value of the "$ref" property MUST be a URI
Reference. Resolved against the current URI base, it identifies the
URI of a schema to use. All other properties in a "$ref" object MUST
be ignored.
https://datatracker.ietf.org/doc/html/draft-handrews-json-schema-01#section-8.3
We changed this to not be the case for draft 2019-09.
It sounds like VSCode is merging the properties in applicators upwards to the nearest schema object (which is wrong), and IntelliJ IDEA is doing something similar but in a different way (which is also wrong).
The correct validation result for your schema and instance is VALID. See the live demo here: https://jsonschema.dev/s/C6ent
additionalProperties relies on the values of properties and patternProperties within the SAME schema object. It cannot "see through" applicators such as $ref and allOf.
For draft 2019-09, we added unevaluatedProperties, which CAN "see through" applicator keywords (although it's a little more complex than that).
Update:
After reviewing your update, sadly the same is still true.
One approach makes it sort of possible but involves some duplication, and only works when you control the schemas you are referencing.
You would need to redefine your selector property like this...
"selector": {
"description": "This property did not function correctly in VSCode",
"allOf": [
{
"$ref": "#/definitions/selector"
},
{
"properties": {
"operator": true,
"value": true,
"example": true,
"alternative": true
},
"additionalProperties": false
}
]
}
The values of a property object are schema values, and booleans are valid schemas. You don't need (or want to) deal with their validation here, only say these are the allowed ones, followed by no additionalProperties.
You'll also need to remove the additionalProperties: false from your definition of selector, as that is preventing ALL properties (which I now guess is why you saw that issue in one of the editors).
It involves some duplication, but is the only way I'm aware of that you can do this for draft-07 or previous. As I said, not a problem for draft 2019-09 or above due to new kewords.
additionalProperties is problematic because it depends on the properties and patternProperties. The result is that "additionalProperties": false effectively blocks schema composition. #Relequestual showed one alternative approach, here is another approach that is a little less verbose, but still requires duplication of property names.
draft-06 and up
{
"allOf": [{ "$ref": "#/definitions/base" }],
"properties": {
"bar": { "type": "number" }
},
"propertyNames": { "enum": ["foo", "bar"] },
"definitions": {
"base": {
"properties": {
"foo": { "type": "string" }
}
}
}
}

How to validate http PATCH data with a JSON schema having required fields

I'm trying to validate a classic JSON schema (with Ajv and json-server) with required fields, but for a HTTP PATCH request.
It's okay for a POST because the data arrive in full and nothing should be ommited.
However, the required fields of the schema make problem when attempting to PATCH an existing resource.
Here's what I'm currently doing for a POST :
const schema = require('./movieSchema.json');
const validate = new Ajv().compile(schema);
// ...
movieValidation.post('/:id', function (req, res, next) {
const valid = validate(req.body);
if (!valid) {
const [err] = validate.errors;
let field = (err.keyword === 'required') ? err.params.missingProperty : err.dataPath;
return res.status(400).json({
errorMessage: `Erreur de type '${err.keyword}' sur le champs '${field}' : '${err.message}'`
});
}
next();
});
... but if i'm doing the same for a movieValidation.patch(...) and tries to send only this chunk of data :
{
"release_date": "2020-07-15",
"categories": [
"Action",
"Aventure",
"Science-Fiction",
"Thriller"
]
}
... it will fail the whole validation (whereas all the fields are okay and they validate the schema)
Here's my complete moviesSchema.json :
{
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
"title": {
"title": "Titre",
"type": "string",
"description": "Titre complet du film"
},
"release_date": {
"title": "Date de sortie",
"description": "Date de sortie du film au cinéma",
"type": "string",
"format": "date",
"example": "2019-06-28"
},
"categories": {
"title": "Catégories",
"description": "Catégories du film",
"type": "array",
"items": {
"type": "string"
}
},
"description": {
"title": "Résumé",
"description": "Résumé du film",
"type": "string"
},
"poster": {
"title": "Affiche",
"description": "Affiche officielle du film",
"type": "string",
"pattern": "^https?:\/\/"
},
"backdrop": {
"title": "Fond",
"description": "Image de fond",
"type": "string",
"pattern": "^https?:\/\/"
}
},
"required": [
"title",
"release_date",
"categories",
"description"
],
"additionalProperties": false
}
For now, I did the trick using a different schema which is the same as the original one, but without the required clause at the end. But I don't like this solution as it's duplicating code unnecessarily (and it's not elegant at all).
Is there any clever solution/tool to achieve this properly?
Thanks
If you're using HTTP PATCH correctly, there's another way to deal with this problem.
The body of a PATCH request is supposed be a diff media type of some kind, not plain JSON. The diff media type defines a set of operations (add, remove, replace) to perform to transform the JSON. The diff is then applied to the original resource resulting in the new state of the resource. A couple of JSON diff media types are JSON Patch (more powerful) and JSON Merge Patch (more natural).
If you validate the request body for a PATCH, you aren't really validating your resource, you are validating the diff format. However, if you apply the patch to your resource first, then you can validate the result with the full schema (then persist the changes or 400 depending on the result).
Remember, in REST it's resources and representations that matter, not requests and responses.
It's not uncommon to have multiple schemas, one per payload you want to validate.
In your situation, it looks like you've done exactly the right thing.
You can de-duplicate your schemas using references ($ref), splitting your property subschemas into a separate file.
You end up with a schema which contains your model, and a schema for each representation of said model, but without duplication (where possible).
If you need more guidance on how exactly you go about this, please comment and I'll update the answer with more details.
Here is an example of how you could do what you want.
You will need to create multiple schema files, and reference the right schema when you need to validate POST or PATCH requests.
I've simplified the examples to only include "title".
In one schema, you have something like...
{
"$id": "https://example.com/object/movie",
"$schema": "http://json-schema.org/draft-07/schema#",
"definitions": {
"movie": {
"properties": {
"title": {
"$ref": "#/definitions/title"
}
},
"additionalProperties": false
},
"title": {
"title": "Titre",
"type": "string",
"description": "Titre complet du film"
}
}
}
Then you would have one for POST and PATCH...
{
"$id": "https://example.com/movie/patch",
"$schema": "http://json-schema.org/draft-07/schema#",
"allOf": [{
"$ref": "/object/movie#/definitions/movie"
}],
}
{
"$id": "https://example.com/movie/post",
"$schema": "http://json-schema.org/draft-07/schema#",
"allOf": [{
"$ref": "/object/movie#/definitions/movie"
}],
"required": ["title"]
}
Change example.com to whatever domain you want to use for your ID.
You will then need to load in to the implementation all of your schemas.
Once loaded in, the references will work, as they are based on URI resolution, using $id for each schema resource.
Notice the $ref values in the POST and PATCH schemas do not start with a #, meaning the target is not THIS schema resource.

Build a RESTful POST API in ObjectScript

how to build a RESTful POST API in ObjectScript whereas schema is like this. It should accept myList conforming to the schema.
{
"$schema": "link",
"type": "object",
"properties": {
"myList": {
"type": "array",
"items":{
"type": "string"
}
}
},
"required": [
"myList"
]
}
Refer to Creating REST Services chapter in documentation. It describes the process.
See also REST in InterSystems Caché articles here