how use findAll() method in sequelize to get table data from postgresql? - postgresql

I have used postgres for database and sequelize for ORM in nodejs, but when I tried to get all data from table using findAll() method in sequelize it gets undefined, and select query executes but at the place of data it shows undefined, tell me what i do wrong here?
const Data=require('./Data')
var Sequelize=require('sequelize')
const connection = new Sequelize('mydb', 'postgres', '12345', {
host: 'localhost',
dialect: 'postgres'
});
var Emp=connection.define('emp',{
fullName: {
type: Sequelize.STRING
},
email: {
type: Sequelize.STRING
},
})
connection.sync().then(function(){
Emp.findAll().then(function(emps){
console.log(emps.dataValues)
})
})
output
Executing (default): CREATE TABLE IF NOT EXISTS "emps" ("id" SERIAL , "fullName" VARCHAR(255), "email" VARCHAR(255), "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL, "updatedAt" TIMESTAMP WITH TIME ZONE NOT NULL, PRIMARY KEY ("id"));
Executing (default): SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND t.relkind = 'r' and t.relname = 'emps' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;
Executing (default): SELECT "id", "fullName", "email", "createdAt", "updatedAt" FROM "emps" AS "emp";
undefined

I have tried to replicate the code with the information you have provided with.
But it seems everything is working properly on my side here.
I do not know if you have set sequelize configuration options database, or how you have structure and organized the project itself.
Maybe this would help: instantiate sequelize, plese look under for "Instantiate sequelize with name of database, username and password.", the params and attributes will help to get an idea of it.
Anyways, I also took the liberty to make an basic code on the information you have shared us with: sequelize how use findall().
Hope these can help you to go thru with it. Have a great one!

I have solved my problem , just i did simple mistake and i solved it now and data is fetched from database table by using findAll() function in sequelize...
**code before changes:**
connection.sync().then(function(){
Emp.findAll().then(function(emps){
console.log(emps.dataValues) //line with mistake
})
})
**code after changes:**
connection.sync().then(function(){
Emp.findAll().then(function(emps){
console.log(emps) //here i did changes
})
})

Related

Fetching distinct rows from multiple includes with sequelize

I have some nested joins below. I am trying to fetch distinct rows out of it.
Product.findAll({
attributes:[sequelize.fn('DISTINCT', sequelize.col('product.id')), 'product.id']],
include: {
model: Course,
attributes: [],
required: true,
include: {
model: Class,
where: {
classId: args.classId
},
attributes: []
}
}
})
syntax error at or near "DISTINCT"
Query being produced by sequelize,
SELECT "product"."id", DISTINCT("product"."id") AS "product.id" FROM "my_schema"."product_groups" AS "product" INNER JOIN "my_schema"."courses" AS "courses" ON "product"."id" = "courses"."product_group_id" INNER JOIN "my_schema"."classes" AS "courses->classes" ON "courses"."id" = "courses->classes"."course_id" AND "courses->classes"."organization_id" = '68700940-f509-4662-975f-a3ba3382aa9b';;
Working sql query,
SELECT DISTINCT("product"."id") FROM "my_schema"."product_groups" AS "product" INNER JOIN "my_schema"."courses" AS "courses" ON "product"."id" = "courses"."product_group_id" INNER JOIN "my_schema"."classes" AS "courses->classes" ON "courses"."id" = "courses->classes"."course_id" AND "courses->classes"."organization_id" = '68700940-f509-4662-975f-a3ba3382aa9b';
How can I product above query with sequelize to return distinct rows. I also tried distinct: true but it doesn't make any change. With this option query works and returns result but no distinct is there in the query generated.
The DISTINCT syntax in Postgres should look like this.
SELECT DISTINCT column AS alias FROM table;
DISTINCT is a clause and not function so you need to use literal.
Product.findAll({
attributes:[[sequelize.literal('DISTINCT "product"."id"'), 'product.id']],
...
})

Sequelize upsert throws unique violation for schema with nullable unique column

I'm facing an unique constraint violation issue when doing an upsert, because the UPDATE query built by sequelize ignores the partial index constraint defined by the model (unless it doesn't matter). I'm new to node+sequelize so I might be missing something obvious, but I went through all the potential places for finding the appropriate answers, inclusive of the sequelize code, but I'm not able to find the answer I'm looking for. Really appreciate your help!
My current versions:
"pg": "7.9.0",
"sequelize": "5.21.3"
I have a model that consists of a primary key: id and two other unique indexes of which one of them is a nullable field.
module.exports.Entities = sequelize.define('entities', {
id: {type: Sequelize.UUID, defaultValue: Sequelize.UUIDV4, allowNull: false, primaryKey: true},
cId: {type: Sequelize.STRING, allowNull: false},
pId: {type: Sequelize.UUID, allowNull: false},
eKey: {type: Sequelize.INTEGER, allowNull: true}
}, {
indexes: [
{
name: 'unique_c_id_p_id',
fields: ['c_id', 'p_id'],
unique: true
},
{
name: 'unique_e_key',
fields: ['e_key'],
unique: true,
where: {
eKey: {
[Op.not]: null
}
}
}
]
})
and the table itself looks like below:
CREATE TABLE public.entities (
id UUID DEFAULT uuid_generate_v4 (),
c_id UUID NOT NULL,
p_id UUID NOT NULL,
e_key INTEGER DEFAULT NULL,
CONSTRAINT ENTITY_SERVICE_PKEY PRIMARY KEY (id),
CONSTRAINT unique_c_id_p_id UNIQUE (c_id, p_id)
);
CREATE UNIQUE INDEX unique_e_key ON public.entities (e_key) WHERE e_key IS NOT NULL;
The upsert method call looks like:
module.exports.upsert = async (Model, values) => Model.upsert(values, {returning: true})
I pass the above Entities model, and the below value as arguments to this function.
{
"id"="3169d4e2-8e2d-451e-8be0-40c0b28e2aa9",
"c_id"="00000000-0000-0000-0000-000000000000",
"p_id"="78bce392-4a15-4a8a-986b-c9398787345f",
"e_key"= null
}
Issue: SequelizeUniqueConstraintError
Sequelize tries to do an insert followed by an update query when we attempt to update an existing record using the upsert method.
The insert query shows a conflict, since the record exists already, and sequelize upsert call proceeds on to invoke the update query.
However, the query that it builds to UPDATE looks something like below:
"SQL statement UPDATE entities SET id='3169d4e2-8e2d-451e-8be0-40c0b28e2aa9',c_id='00000000-0000-0000-0000-000000000000',p_id='78bce392-4a15-4a8a-986b-c9398787345f',e_key=NULL
WHERE (id = '3169d4e2-8e2d-451e-8be0-40c0b28e2aa9'
OR e_key IS NULL
OR (c_id = '00000000-0000-0000-0000-000000000000' AND p_id = '78bce392-4a15-4a8a-986b-c9398787345f'))
RETURNING id\nPL/pgSQL function pg_temp_5.sequelize_upsert() line 1 at SQL statement"
Now, I do understand the reason why it's throwing the unique constraint violation, since in the above query's WHERE clause sequelize calls OR e_key IS NULL since e_key = null and that could potentially return more than 1 record, and the SET is trying to update the same value for all those records that were returned thereby violating the primaryKey constraints, unique constraints etc.
What I would like to understand is that:
Why does sequelize not exclude the e_key unique constraint based on the partial index defined given that it picks the WHERE clause attributes based on the constraints defined in the Model & it's indexes?
Is there anything that I could do to get past this issue?
Or, am I missing something obvious that I could fix and try?
Really appreciate you taking your time to read and respond. Thanks!

Knex Raw Select Postgresql with variable

Can I use a variable in a knex query? And what is wrong with db.raw(select usr_vote.vote where usr_vote.user.id = ${loggedInUserId})? Everything else works fine.
In the db.raw that is nowt working, I'm trying to use a variable (loggedInUserId) to get the logged-in users' vote history for the Question (they can upvote / downvote so the value is either -1 or 1 or null). Thanks in advance for any help!
There are 4 tables that look like:
askify_users
id
user_name
askify_questions
id
title
body
tags
date_created
user_id (FK references askify_users.id)
askify_answers
id
answer
question_id (FK references askify_question.id)
user_id (FK references askify_users.id)
askify_question_votes
List item
question_id (FK references askify_questions.id)
user_id (FK references askify_users.id)
vote (-1 or 1)
PRIMARY KEY (question_id, user_id)
getAllQuestions(db, loggedInUserId) {
return db
.from('askify_questions AS q')
.select(
'q.id AS question_id',
'q.title AS question_title',
'q.body AS question_body',
'q.date_created AS date_created',
'q.tags',
db.raw(
`count(DISTINCT ans) AS number_of_answers`
),
db.raw(
`SUM(DISTINCT usr_vote.vote) AS sum_of_votes`
),
db.raw(
`select usr_vote.vote where usr_vote.user_id = ${loggedInUserId}`
),
db.raw(
`json_strip_nulls(
json_build_object(
'user_id', usr.id,
'user_name', usr.user_name,
'full_name', usr.full_name,
'date_created', usr.date_created
)
) AS "user"`
)
)
.leftJoin(
'askify_answers AS ans',
'q.id',
'ans.question_id'
)
.leftJoin(
'askify_users AS usr',
'q.user_id',
'usr.id'
)
.leftJoin(
'askify_question_vote AS usr_vote',
'q.id',
'usr_vote.question_id'
)
.groupBy('q.id', 'usr.id')
},
The query should look as follows. Everything except for 'user_vote_history' is working.
serializeQuestion(question) {
const { user } = question
return {
id: question.question_id,
question_title: xss(question.question_title),
question_body: xss(question.question_body),
date_created: new Date(question.date_created),
number_of_answers: Number(question.number_of_answers) || 0,
user_vote_history: question.user_vote_history,
sum_of_votes: Number(question.sum_of_votes),
tags: xss(question.tags),
user: {
user_id: user.user_id,
user_name: user.user_name,
full_name: user.full_name,
user_vote: user.user_vote,
date_created: new Date(user.date_created)
},
}
},
I note that #felixmosh is correct here regarding bound values, but just to elaborate: the key here is when the string substitution takes place. If you do this:
db.raw(`SELECT vote WHERE user_id = ${loggedInUserId}`)
the substitution takes place in JavaScript, as soon as the JS interpreter reaches this line. The database engine has nothing to do with whatever is in loggedInUserId and neither does Knex: you're essentially bypassing all the built-in protections.
Slightly better is:
db.raw("SELECT vote WHERE user_id = ?", loggedInUserId)
This allows Knex to escape the string in loggedInUserId. If you prefer, you can use a named binding:
db.raw("SELECT vote WHERE user_id = :loggedInUserId", { loggedInUserId })
However, all of this mucking about with bindings is easily avoided by using the facility Knex already provides for subqueries: just put the subquery in a function.
db
.from("askify_questions AS q")
.select(
"q.id AS question_id",
qb => qb.select("usr_vote.vote").where({ user_id: loggedInUserId })
)
.leftJoin(
"askify_question_vote AS usr_vote",
"q.id",
"usr_vote.question_id"
);
The qb parameter stands for "query builder", and is passed by Knex to your function. It behaves very much like your db object.
This generates SQL akin to:
SELECT
"q"."id" AS "question_id",
(
SELECT "usr_vote"."user_id" WHERE "user_id" = ?
)
FROM "askify_questions AS q"
LEFT JOIN "askify_question_vote" AS "usr_vote"
ON "q"."id" = "usr_vote"."question_id"

How does nested SELECT and NOT IN work in knex

I'm trying to write this query in knex and am stuck.
I currently have 3 tables (group_tasks, completed_group_tasks and group_users)
I want to filter out the results when grouptask_id from completed_group_tasks is the same as the innerjoin.
SELECT * FROM group_tasks INNER JOIN group_users ON group_users.group_id = group_tasks.group_id WHERE user_id = 2 AND grouptask_id NOT IN (SELECT grouptask_id FROM completed_group_tasks WHERE user_id = 2)
I tried writing this:
db.from('group_users')
.select('groups.group_id', 'group_name')
.innerJoin('groups', 'group_users.group_id', 'groups.group_id')
.where({user_id: user_id})
.whereNotIn('grouptask_id', function() {
this.from('completed_group_tasks')
.select('grouptask_id')
.where({user_id: user_id})
})
This fails stating column "grouptask_id" does not exist
Subquery in Knex needs to be a separate Knex builder.
db.from('group_users')
.select('groups.group_id', 'group_name')
.innerJoin('groups', 'group_users.group_id', 'groups.group_id')
.where({ user_id: user_id })
.whereNotIn(
'grouptask_id',
db
.from('completed_group_tasks')
.select('grouptask_id')
.where({ user_id: user_id }),
);

Add timestamp fields to ogr2ogr import into Postgis

I managed to create a model/schema and insert geo-Points into Postgis using sequelize. Since I have a lot of points (>100K) I am longing back to the old way I used to import, using ogr2ogr (gdal), which was much faster (almost instant instead of >20 minutes). As I would like to continue to work with sequelize after the initial import I still want sequelize to create the model/schema upfront, but then do import with ogr2ogr, and then continue CRUD with sequelize.
Here I found this fragment “[….] One way to get around this is to create your table structures before hand and use OGR2OGR to just load the data.” Which gave me the idea that this could work for Postgres/Postgis as well.
Sequelize creates timestamp columns for createdAt and updatedAt, which I like. But when I use ogr2ogr I get “null value in column "createdAt" violates not-null constraint” as loginfo.
Based on this slightly similar issue I tried to add a createdAt column by adding an -sql option:
ogr2ogr -f PostgreSQL PG:"host='0.0.0.0' user='root' port='5432' dbname='db' password='pw'" /home/user/geojsonImportfile.json -nln "DataPoints" -a_srs EPSG:4326 -sql "SELECT url, customfield, wkb_geometry, CURRENT_TIMESTAMP as createdAt FROM '/home/usr/geojsonImportfile.json'" -dialect 'PostgreSQL'
The error I get when running this is:
ERROR 1: SQL Expression Parsing Error: syntax error, unexpected end of string, expecting '.'. Occurred around :
home/user/geojsonImportfile0.json'
Besides my lack of SQL knowledge I am not sure if this can work at all.
How can I solve this, i.e. make the import with ogr2ogr but keep the timestamp columns?
When you create a table with sequelize.define, createdAt and updatedAt columns are created automatically as timestamp with time zone NOT NULL.
But you can rule not-null constraint in your sequelize definition script:
const Mytable = sequelize.define('mytable', {
id: {type: Sequelize.INTEGER, primaryKey: true},
createdAt: {type: Sequelize.DATE, validate: {notNull:false}}
});
Then table is created like :
CREATE TABLE mytables
(
id integer NOT NULL,
"createdAt" timestamp with time zone,
"updatedAt" timestamp with time zone NOT NULL,
CONSTRAINT mytables_pkey PRIMARY KEY (id)
)
http://docs.sequelizejs.com/manual/tutorial/models-definition.html#configuration
#JGH following your suggestion it makes sense to have a default timestamp. I can already set this up using sequelize, as discussed here:
var user = sequelize.define('user', {
createdAt: {
type: DataTypes.DATE,
field: 'beginTime',
defaultValue: sequelize.literal('NOW()')
}
}, {
timestamps: true,
});