Can i add an index to a table in Tilda for performance? - postgresql

We created a simple table in Tilda and want to create an index. How to do that?
{ "name": "Test"
,"description": "The table to keep track tests amd owners"
,"columns":[
{ "name": "id" , "type": "STRING" , "nullable": false, "size": 128, "invariant": true, "description": "The name of the test" }
,{ "name": "name" , "type": "STRING" , "nullable": false, "size": 128, "invariant": true, "description": "The name of the test" }
,{ "name": "test_fk" , "sameAs":"Test.refnum" , "nullable": false, "invariant": true, "description": "The name of the test" }
]
,"primary": { "autogen": true }
,"foreign": [
{ "name":"Test", "srcColumns":["test_fk"], "destObject":"Test" }
]
}

You can see documentation details about Index creation in table definitions in the Wiki:
https://github.com/CapsicoHealth/Tilda/wiki/Tilda-JSON-Syntax%3A-Object-Indices
// Within an Object definition, you can define any number of indices (or none).
"indices":[
// A unique index that exists in the database over colA and colB
{ "name":"Blah1", "columns":["colA", "ColB"], "db":true }
// A regular (non-unique) index that overlaps with the previous definition
// but only exists for application needs (no database definition).
,{ "name":"Blah2", "columns":["colA"] , "orderBy":["colB"], "db":false}
// A regular index with only an orderBy definition.
,{ "name":"Blah3", "orderBy":["colB"], "db":true }
]
The markup above shows the JSON to create a variety of indices with comments explaining what each attribute does.

Related

POSTGRES jsonb document GIN Indexes are created on what all objects?

I am using Postgres DB 13.5. From pgdocs -
The technical difference between a jsonb_ops and a jsonb_path_ops GIN
index is that the former creates independent index items for each key
and value in the data, while the latter creates index items only for
each value in the data. Basically, each jsonb_path_ops index item
is a hash of the value and the key(s) leading to it; for example to
index {"foo": {"bar": "baz"}}
Understanding the above in detail is important for me coz my jdata (document) is big with many keys and nested objects. Consider my json data that is stored as jsonb in a column named jdata looks like below -
{
"supplier": {
"id": "3c67b6eb-3b0d-492d-8736-66df107b83b3",
"customer": {
"type": "pro",
"name": "John George",
"address": [
{
"add-id": "098ad4df-2a90-4fda-8f92-dbe8d7196732",
"addressActive": true,
"street": "abc street",
"zip": 94044,
"staying-since": "long long",
"accessibility": {
"traffic": "heavy/congested",
"bestwaytoreach": {
"weekdays": {
"bart/metro/calltrain": true,
"price": {
"off-peak-hours": "affordable",
"peak-hours": "high"
},
"journey-time": "super-fast"
}
},
"weekends": {
"byroad": {
"ok": true,
"distance": "long",
"has-tolls": {
"true": true,
"toll-price": "relatively-high"
},
"journey-speed": "fast"
}
}
}
},
{
"add-id": "ddd1d2a0-9050-4bcf-a3ad-2e608d65e468",
"addressActive": true,
"street": "xyz street",
"zip": 10001,
"staying-since": "moved recently",
"accessibility": {
"traffic": "heavy/congested",
"bestwaytoreach": {
"weekdays": {
"subway": true,
"price": {
"off-peak-hours": "affordable",
"peak-hours": "high"
},
"journey-speed": "super-fast"
}
},
"weekends": {
"byroad": {
"ok": true,
"distance": "moderate",
"tolls": {
"has-tolls": true,
"toll-price": "relatively-high"
},
"journey-time": "super-fast"
}
}
}
}
],
"firstName": "John",
"lastName": "CRAWFORD",
"emailAddresses": {
"personal": [
"johnreplies#jg.com",
"ursjohn#jg.com",
"1234#jg.com"
],
"official": [
{
"repies-in": "1 day",
"email": "jg#jg.com"
},
{
"check's regularly": true,
"repies-in": "1 Hour",
"email": "jg-watching#jg.com"
}
]
},
"cities": [
"NYC",
"LA",
"SF",
"DC"
],
"splCustFlag": null,
"isPerson": true,
"isEntity": false,
"allowEmailSolicit": "Y",
"allowPhoneSolicit": "Y",
"taxPayer": true,
"suffix": null,
"title": null,
"birthDate": "05/10/1993",
"loyaltyPrograms": null,
"phoneNumbers-summary": [
1234567890,
1234567899,
1234567898,
1234567897
],
"phoneNumbers": [
{
"description": null,
"extension": null,
"number": 1234567890,
"countryCode": null,
"type": "Business"
},
{
"description": null,
"extension": null,
"number": 1234567899,
"countryCode": null,
"type": "Home"
}
],
"data-privacy": {
"required": true,
"laws": [
"CCPA",
"GDPR"
]
}
}
}
}
Now if I create GIN jsonb_ops index for jdata column - I want to clarify what all keys and values will be part of index.
For example - "staying-since" is a key nested at below path and it's part of "address" array too. But it's still a key, thought nested deep in the document. So will it be part of the index.
{
"supplier": {
"customer": {
"address": [
{
"staying-since": "long long" ...
And similarly "long long" is a value of a deeply nested key. Will it also be indexed.
And if GIN jsonb_path_ops index is created for jdata column --
Will a hash of "long long" value along with the path that leads to it will also be indexed.
hash(
"supplier": {
"customer": {
"address":[{"staying-since": "long long"}]
}
}
)
will the above also gets index.
I am aware about the operators that are supported by the GIN index types and am aware about the usage of these operators -
jsonb_ops ? ?& ?| #> #? ##
jsonb_path_ops #> #? ##

How do I create an Azure Cognitive Search index for embedded document with $oid unique identifier type?

My current data structure is something like this,
"customer": {
"_id": {
"$oid": "623a4b1bdb6d0a1210fd0234"
},
"customerName": "Andrew Jr"
},
I will need to create ACS (Azure Cognitive Search for the above data structure. So I was using something like this:
{
"name": "customer",
"type": "Edm.ComplexType",
"fields":
[
{
"name": "customerName",
"type": "Edm.String",
"searchable": true,
"filterable": true,
"sortable": true,
"facetable": true
}
]
},
But it will throw errors to me when I add the _id as a string data type, for example, below:
{
"name": "customer",
"type": "Edm.ComplexType",
"fields":
[
{
"name": "_id",
"type": "Edm.String",
"searchable": true,
"filterable": true,
"sortable": true,
"facetable": true
},
{
"name": "customerName",
"type": "Edm.String",
"searchable": true,
"filterable": true,
"sortable": true,
"facetable": true
}
]
},
Any ideas how to add the following format to the search?
"_id": {
"$oid": "623a4b1bdb6d0a1210fd0234"
},
Thanks,
Most likely you are getting the error is because of invalid name for the field (_id).
Based on the naming rules defined here, a field name can only contain letters, numbers, underscores ("_") however the first character of the field name must be a letter.
Since you are naming your field as _id, you will get an error.

Loopback indexes - how to specify different index types in model definition?

In Loopback (v3), when defining indexes in my model.json files, how do I specify different types of indexes (such as a BRIN)? Also, how do I specify index conditions (such as if I want to create a partial index)? I'm using postgres for the database, if that's relevant.
You can configure the index type via type field.
{
"name": "MyModel",
"properties": {
// ...
},
"indexes": {
"myindex": {
"columns": "name, email",
"type": "BRIN",
// ...
}
}
}
I am afraid LoopBack does not support index conditions (partial indexes) yet. Feel free to open a new issue in https://github.com/strongloop/loopback-connector-postgresql/issues.
i was trying to add in Lb4. Its pretty straightforward there (should be same for lb3 as well i hope)
#model({
name: 'tablename',
settings: {
indexes: {
idx_tablename: {
columnA : '',
columnB : '',
columnC: ''
}
}
}
})
once the build is done, the index name idx_tablename with 3 columns will get created
In PostgreSQL and Loopback 3 you can specify an index for multi-column like this.
The following loopback JSON code creates index in Postgres with fields message and type are unique together.
{
"name": "notification",
"base": "PersistedModel",
"idInjection": true,
"options": {
"validateUpsert": true
},
"properties": {
"message": {
"type": "string",
"required": true
},
"type": {
"type": "string",
"required": true
},
"seen": {
"type": "boolean",
"required": true,
"default": false
}
},
"validations": [],
"relations": {},
"acls": [],
"methods": {},
"indexes": {
"message_type_index": {
"keys": "message, type",
"options": {"unique": true}
}
}
}

Remove Parse-specific fields from database results

Every time I query something from database I get objects like this:
{
"title": "faq",
"description": "",
"type": "application/pdf",
"size": 122974,
"filename": "faq.pdf",
"order": 0,
"createdAt": "2017-08-17T08:10:33.101Z",
"updatedAt": "2017-08-17T08:10:33.101Z",
"ACL": {
"role:cxcccccc_public": {
"read": true
},
"role:cxaaaaaaa_admin": {
"read": true,
"write": true
}
},
"objectId": "l6L5J1mRpH",
"__type": "Object",
"className": "Document"
}
A lot of these fields are Parse-specific: createdAt, updatedAt, ACL, className...
I didn't insert them when I added the row in the database but still they are returned when I do a query.
I want to expose this data through a clean REST API so I want to get rid of them.
Is there a way to get only the data I specified at insert time?
For now I am using lowdash, filtering with
data = _.omit(data, ['ACL', '__type', 'className', ...])

Get all vertices having a labelname

I am using ibm graph in bluemix and new to this.
I created a graph named 'test' using the GUI provided by bluemix and uploaded the sample data 'Music Festival' provided by ibm in that graph.
Now I am trying to query all the vertices having label 'attendee' using below query.
def gt = graph.traversal();
gt.V().hasLabel("attendee");
But I am getting error as
Error: Error encountered evaluating script def gt = graph.traversal();gt.V().hasLabel("attendee"); with reason com.thinkaurelius.titan.core.TitanException: Could not find a suitable index to answer graph query and graph scans are disabled: [(~label = attendee)]:VERTEX
Not sure what I am doing wrong.
Can somebody tell where am i going wrong?
How can i get rid of this error and get the expected output?
Thanks
#Radhika, Your Gremlin query is a valid Gremlin query. However, some vendors (such as IBM Graph and Titan) chose to only allow users to start their queries with a query that is indexed.This is to make sure you get the performance of your queries. Calling hasLabel() by itself will give you the Could not find a suitable index... error as you can't create indexes for labels. What you need to do is follow this step with a step that uses a indexed property as in this query :
graph.traversal();gt.V().hasLabel("band").has("genre","pop");
An index for genre has been created in the schema for the sample music festival data as you can see below
{
"propertyKeys": [
{ "name": "name", "dataType": "String", "cardinality": "SINGLE" },
{ "name": "gender", "dataType": "String", "cardinality": "SINGLE" },
{ "name": "age", "dataType": "Integer", "cardinality": "SINGLE" },
{ "name": "genre", "dataType": "String", "cardinality": "SINGLE" },
{ "name": "monthly_listeners", "dataType": "String", "cardinality": "SINGLE" },
{ "name":"date","dataType":"String","cardinality":"SINGLE" },
{ "name":"time","dataType":"String","cardinality":"SINGLE" }
],
"vertexLabels": [
{ "name": "attendee" },
{ "name": "band" },
{ "name": "venue" }
],
"edgeLabels": [
{ "name": "bought_ticket", "multiplicity": "MULTI" },
{ "name":"advertised_to","multiplicity":"MULTI" },
{ "name":"performing_at","multiplicity":"MULTI" }
],
"vertexIndexes": [
{ "name": "vByName", "propertyKeys": ["name"], "composite": true, "unique": false },
{ "name": "vByGender", "propertyKeys": ["gender"], "composite": true, "unique": false },
{ "name": "vByGenre", "propertyKeys": ["genre"], "composite": true, "unique": false}
],
"edgeIndexes" :[
{ "name": "eByBoughtTicket", "propertyKeys": ["time"], "composite": true, "unique": false }
]
That's why the above query works and you need to do the same.
If you don't have a schema, create one. You can model it after the
one above or follow the API
doc
Create an (Vertex/Label) index for the properties that you'll start
your traversals from. In this example, Name, Gender and Genre for
vertex properties and name for the edge properties.
Call the schema
endpoint
to add your schema to your graph
It's recommended to create your schema before adding any data to
your graph so that you don't have to reindex later. That'll save you
a lot of time.
Once you create your schema, you can't modify what you created
already, but you can add new properties/indexes later on.
Look at the following code samples for Java and Nodejs for the exact code to use.
I hope that helps