Is there a way to ignore values with missing columns when using INSERT INTO in PostgreSQL?
For example:
INSERT INTO tblExample(col_Exist1, col_Exist2, col_NotExist) VALUES ('Val1', 'Val2', 'Val3)
I want to insert a new row containing values Val1 and Val2, but ignore Val3 since its column does not exist, so the result would be:
# | col_Exist1 | col_Exist2
-----------------------------
1 | Val1 | Val2
I see that there is a INSERT ... ON CONFLICT DO NOTHING construct, but this seems to apply to an entire row only - not a singular value.
For explanation, I realise this may be not best practice, but my application is using dynamically created queries based on properties from documents - the properties can vary, but there are lots of columns, so defining them explicitly is painful. Instead, I'm using a 'template' document to define them and, hopefully, I can just ignore properties from other documents that don't exist in the template document.
Thanks in advance.
EDIT: I've figured out a workaround for now - I'm just querying the table to get the list of columns - if the column name exists, add the property to the new INSERT INTO query. The original question still stands.
What about moving document's data to json?
Form one table where you will have following fields:
Table: Documents
id: uuid4
name: varchar or text
data: json type according https://www.postgresql.org/docs/devel/datatype-json.html
After this trick you can store any dynamical data you'd like
Related
How do I add calculated field for this?
I have table in tableau in following format (connected through redshift and every day 100000 rows are appended for the same ID1 and ID2, and sometimes new values for ID1 and ID2 are also added):
Value in Date Field 1 could either be Null or some date value (which is appended every day). What I want is whenever for specific combination of ID1 and ID2, Date Field 1 is not Null, I want that value to be copied to other rows also as follows:
How do I do it?
Note: I got an answer for this question under SQL tag but it is with UPDATE table method: SQL: Copy values for unique keys from one row to other
But I specifically need answer for tableau also to add calculated field.
You can create a calculated field using IF THEN statement as in the images below to look for a Match:
You can see the data reflects the new Match field as well:
I have two columns, I want the second column to have the same values as the first column always, in PostgreSQL.
The columns are landmark_id (integer) and name (varchar), I want the name column to always have the same values (id's) from landmark_id.
landmark_id (integer) | name (varchar)
1 1
2 2
3 3
I don't understand why you would want to do that, but I can think of two ways to accomplish your request. One is by using a generated column
CREATE TABLE g (
landmark_id int,
name varchar(100) GENERATED ALWAYS AS (landmark_id::varchar) STORED
)
and the other is by enforcing a constraint
CREATE TABLE c (
landmark_id int,
name varchar(100),
CONSTRAINT equality_cc CHECK (landmark_id = name::varchar)
)
Both approaches will cause the name column to occupy disk space. The first approach will not allow you to specify the name column in INSERT or UPDATE statements. In the latter case, you will be forced to specify both columns when inserting.
You could also have used a trigger to update the second column.
Late edit: Others suggested using a view. I agree that it's a better idea than what I wrote.
Create a view, as suggested by #jarlh in comments. This automatically generates column name for you on the fly. This is usually preferred to storing essentially the same data multiple times as in an actual table, where the data occupies more disk space and also can get out of sync. For example:
CREATE VIEW landmarks_names AS
SELECT landmark_id,
landmark_id::text AS name
FROM landmarks;
I have two working statements to insert a row into a table named document with Slick.
The first one inserts all columns, taken from the row object and returns the ID generated by Postgres:
(Tables.Document returning Tables.Document.map(_.id)) +=
DocumentRow(id=-1, name="Name", fulltext="The text")
The second one ignores the column named fulltext and only inserts the name but does not return the generated ID:
Tables.Document.map(r => (r.name)) += ("Name")
How can I combine both (limiting the insert to a subset of columns and returning the generated ID at the same time)?
Background:
The reason why I want to exclude the fulltext column from the insert is the fact that it is of Postgres type tsvector, but the generated Slick code treats it as a String. At insert time the value (even if null or None) is converted into some text type which is incompatible with tsvector and raises an exception. I found no solution to insert a tsvector without an additional library. Please comment if you think there is and I should be rather following this path.
Although I believe the right way is to fix tsvector issue, I don't have enough experience with Postres to help you with it. As for your workaround, you can do it and the code should look something like this:
(Tables.Document.map(r => (r.name)) returning Tables.Document.map(_.id)) += ("Name")
If you split it into parts you can see that first you create a Query as in your second example but then rather than apply += to it immediately you first chain it with returning and only then call +=.
P.S. What is the issue with using some additional Postres-aware library?
What is the best way to setup multicolumn index using the full_name column and the state column? The search will use the exact state with a partial search on the full_name column. The query will like this:
WHERE full_name ~* 'jones' AND state = 'CA';
Searching roughly 20 million records.
Thanks!
John
The state seems straight-forward enough -- a normal index should suffice. As far as the full name search, this is a lot of work, but with 20 million records, I think the dividends will speak for themselves.
Create a new fields in your table as a tsvector, and call it full_name_search for the sake of this example:
alter table <blah> add column full_name_search tsvector;
Do an initial population of the column:
update <blah>
set full_name_search = to_tsvector (full_name);
If possible, make the field non-nullable.
Create a trigger that will now automatically populate this field whenever it's updated:
create trigger <blah>_insert_update
before insert or update on <blah>
for each row execute procedure
tsvector_update_trigger(full_name_search,'pg_catalog.english',full_name);
Add an index on the new field:
create index <blah>_ix1 on <blah>
using gin(full_name_search);
From here, restructure the query to search on the tsvector field instead of the text field:
WHERE full_name_search ## to_tsquery('jones') AND state = 'CA';
You can take shortcuts on some of these steps (for example, don't create an extra field but use a function-based index instead), and it will get you improved performance, but not as good as what you can get.
One caveat -- I think the to_tsvector will split into vector components based on logical breaks in the contents, so this:
Catherine Jones Is a Nice Lady
will work fine, but this:
I've been Jonesing all day
Probably won't.
I have a sqlite3 database with this schema:
CREATE TABLE [dict] (
[Entry] [CHAR(209)],
[Definition] [CHAR(924975)]);
CREATE INDEX [i_dict_entry] ON [dict] ([Entry]);
it's a kind of dictionary with 260000 records and nearly 1GB of size; I have created an index for the Entry column to improve performance;
a sample of a row's entry column is like this:
|love|lovingly|loves|loved|loving|
All the words which are separated with | are referring to the same definition;(I put all of them in one string, separated with | to prevent duplication of data in Definition column)
and this is the command that I use to retrieve the results:
SELECT * FROM dict WHERE Entry like '%|loves|%'
execution time: ~1.7s
if I use = operator instead of LIKE operator, the execution is nearly instantaneous;
SELECT * FROM dict WHERE Entry='|love|lovingly|loves|loved|loving|'
but this way I can't search for words like: love,loves...(separately I mean)
My questions:
Although I have created an index for the Entry column, is indexing really effective while we are using LIKE operator with % in it?
what about the idea that I create different rows for each part of composite Entry columns(one for love another for loves...then all will have the same definition) and then use = operator? if yes; is there anyway of referencing of data? I mean rather than repeating the same Definition for each entry, create one and all others point to it; is it possible?
thanks in advance for any tip and suggestion;
Every entry should have a separate row in the database:
CREATE TABLE Definitions (
DefinitionID INTEGER PRIMARY KEY,
Definition TEXT
);
CREATE TABLE Entries (
EntryID INTEGER PRIMARY KEY,
DefinitionID INTEGER REFERENCES Definitions(DefinitionID),
Entry TEXT
);
CREATE INDEX i_entry ON Entries(Entry);
You can then query the definition by joiing the two tables:
SELECT Definition
FROM Entries
JOIN Definitions USING (DefinitionID)
WHERE Entry = 'loves'
Also see Database normalization.