How to generalize an Opaleye Query in Haskell (Using Vinyl)? - postgresql

My question is between the huge banners in the code block below.
Forgive the code dump, this is all pasted here for anyone wanting to replicate, and this code does work as expected, although it's a bit strange. Notice the last two lines, they print proper SQL.
Goal:
I have tables with primary keys of type Text, specifically, emails. Instead of writing a new query function for each table, I took upon the task of generalizing the function, so that I could type-safely query any table that has emails.
Problem:
In order to get this to work, I had to include:
instance Default Constant CEmail (Column PGText) where
def = undefined
Which makes me think I'm doing something wrong. Any advice for building a query that can find records from any table that has Emails?
{- stack
--resolver lts-8.2
--install-ghc
exec ghci
--package aeson
--package composite-base
--package composite-aeson
--package text
--package string-conversions
--package postgres-simple
--package vinyl
-}
{-# LANGUAGE
Arrows
, DataKinds
, OverloadedStrings
, PatternSynonyms
, TypeOperators
, TemplateHaskell
, FlexibleContexts
, RankNTypes
, ConstraintKinds
, TypeSynonymInstances
, FlexibleInstances
, MultiParamTypeClasses
#-}
import Data.Vinyl (RElem)
import Data.Functor.Identity (Identity)
import Data.Vinyl.TypeLevel (RIndex)
import Composite.Aeson (JsonFormat, defaultJsonFormatRec, recJsonFormat, toJsonWithFormat)
import Composite.Opaleye (defaultRecTable)
import Composite.Record (Record, Rec(RNil), (:->), pattern (:*:))
import Composite.TH (withOpticsAndProxies)
import Control.Arrow (returnA)
import Control.Lens (view)
import Data.Int (Int64)
import Data.Proxy (Proxy(Proxy))
import Data.Text (Text)
import Opaleye
import Opaleye.Internal.TableMaker (ColumnMaker)
import Data.String.Conversions (cs)
import qualified Data.Aeson as Aeson
import qualified Database.PostgreSQL.Simple as PGS -- used for printSql
import Data.Profunctor.Product.Default (Default(def))
--------------------------------------------------
-- | Types
-- | Newtype ClearPassword so it can't be passed around as ordinary Text
newtype ClearPassword a = ClearPassword a
withOpticsAndProxies [d|
type FEmail = "email" :-> Text
type CEmail = "email" :-> Column PGText
type FAge = "age" :-> Text
type CAge = "age" :-> Column PGText
type FClearPassword = "clearpass" :-> ClearPassword Text
type CHashPassword = "hashpass" :-> Column PGText
|]
--------------------------------------------------
-- | Db Setup
-- | Helper Fn
printSql :: Default Unpackspec a a => Query a -> IO ()
printSql = putStrLn . maybe "Empty query" id . showSqlForPostgres
-- | Db Records
type DbUser = '[CEmail, CAge]
type DbPassword = '[CEmail, CHashPassword]
--------------------------------------------------
--------------------------------------------------
--
-- LOOK HERE vvvvvvvvvvvvvvvvvvvvvvvv
--
--------------------------------------------------
--------------------------------------------------
type RecWith f rs = (Default ColumnMaker (Record rs) (Record rs),
Default Constant f (Column PGText),
RElem f rs (RIndex f rs))
-- | queryByEmail needs this, but totally works if `def` is declared
-- as `undefined` ???
instance Default Constant CEmail (Column PGText) where
def = undefined
queryByEmail :: (RecWith CEmail rs) =>
Table a (Record rs) -> Text -> QueryArr () (Record rs)
queryByEmail table email = proc () -> do
u <- queryTable table -< ()
let uEmail = view cEmail u
restrict -< uEmail .=== constant email
returnA -< u
--------------------------------------------------
--------------------------------------------------
--
-- LOOK UP ^^^^^^^^^^^^^^^^^^^^^^^^
--
--------------------------------------------------
--------------------------------------------------
userTable :: Table (Record DbUser) (Record DbUser)
userTable = Table "user" defaultRecTable
-- | Password
passwordTable :: Table (Record DbPassword) (Record DbPassword)
passwordTable = Table "password" defaultRecTable
-- SELECT ... FROM "user" ...
queryUserTest = printSql $ queryByEmail userTable "hi"
-- SELECT ... FROM "password" ...
queryPasswordTest = printSql $ queryByEmail passwordTable "hi"

Drop the extraneous Default Constant f (Column PGTest) constraint and you should be good to go:
#!/usr/bin/env stack
{- stack --resolver lts-8.11 --install-ghc exec ghci --package aeson --package composite-base --package composite-aeson --package text --package string-conversions --package vinyl --package composite-opaleye -}
{-# LANGUAGE Arrows, DataKinds, OverloadedStrings, PatternSynonyms, TypeOperators, TemplateHaskell, FlexibleContexts, RankNTypes, ConstraintKinds, TypeSynonymInstances, FlexibleInstances, MultiParamTypeClasses #-}
import Composite.Opaleye (defaultRecTable)
import Composite.Record (Record, (:->))
import Composite.TH (withOpticsAndProxies)
import Control.Arrow (returnA)
import Control.Lens (view)
import Data.Profunctor.Product.Default (Default)
import Data.Text (Text)
import Data.Vinyl (RElem)
import Data.Vinyl.TypeLevel (RIndex)
import Opaleye.Internal.TableMaker (ColumnMaker)
import Opaleye
newtype ClearPassword a = ClearPassword a
withOpticsAndProxies [d|
type FEmail = "email" :-> Text
type CEmail = "email" :-> Column PGText
type FAge = "age" :-> Text
type CAge = "age" :-> Column PGText
type FClearPassword = "clearpass" :-> ClearPassword Text
type CHashPassword = "hashpass" :-> Column PGText
|]
type DbUser = '[CEmail, CAge]
type DbPassword = '[CEmail, CHashPassword]
printSql :: Default Unpackspec a a => Query a -> IO ()
printSql = putStrLn . maybe "Empty query" id . showSqlForPostgres
queryByEmail :: (RElem CEmail rs (RIndex CEmail rs), Default ColumnMaker (Record rs) (Record rs)) => Table a (Record rs) -> Text -> QueryArr () (Record rs)
queryByEmail table email = proc () -> do
u <- queryTable table -< ()
let uEmail = view cEmail u
restrict -< uEmail .=== constant email
returnA -< u
userTable :: Table (Record DbUser) (Record DbUser)
userTable = Table "user" defaultRecTable
passwordTable :: Table (Record DbPassword) (Record DbPassword)
passwordTable = Table "password" defaultRecTable
queryUserTest = printSql $ queryByEmail userTable "hi"
queryPasswordTest = printSql $ queryByEmail passwordTable "hi"
The constant email call uses the (already extant) Default Constant Text (Column PGText) constraint; were email to have type CEmail instead you would need a non-trivial non-undefined-using instance.

Related

Passing list of values to SELECT PostgreSQL query in Haskell

I'm studying PostgreSQL with Haskell with this lib: https://hackage.haskell.org/package/postgresql-simple-0.4.10.0/docs/Database-PostgreSQL-Simple.html#v:query
While I could select an user like this:
(query_ conn "SELECT * FROM users WHERE NAME == john" :: IO [Only Int]) >>= mapM_ print
using query_:
query_ :: FromRow r => Connection -> Query -> IO [r]
I think I should use query:
query :: (ToRow q, FromRow r) => Connection -> Query -> q -> IO [r]
to pass a list of values. However, how do I pass this list?
For example, for INSERT, I was able to do this:
(execute conn "INSERT INTO users (NAME, PASSWORD) VALUES (?,?)") (["john", "123456"]::[String]) >>= print
but what is the equivalent for SELECT?
I'm not sure I understand your question, since you ask about lists and I don't see how they enter into the picture. But the parameterized version of your select query is this:
query conn "SELECT * FROM users where NAME == ?" (Only ("john" :: String))

How to insert bytea value using postgresql-simple in Haskell

I have a table defined as
CREATE TABLE users (id SERIAL PRIMARY KEY, val BYTEA);
Then I want to serialize my data structure with binary and store in the table, and then retrieve and deserialize back.
{-# LANGUAGE OverloadedStrings, DeriveAnyClass #-}
import Control.Monad (forM_)
import Data.Binary (encode, decode, Binary)
import Database.PostgreSQL.Simple
import GHC.Generics (Generic)
data User = { name :: Text, email :: Text } deriving (Show, Generic, Binary)
main = do
conn <- connect --...
let encoded = encode User {name = "me", email = "me#home.net" }
execute conn "INSERT INTO users(val) values(?)" $ Only encoded
rs <- query_ conn "SELECT id, val FROM users"
forM_ rs $ \(id,val) ->
putStrLn $ (show (id :: Int)) ++ ": " ++ show (decode val :: User)
But I get error Data.Binary.Get.runGet at position 0: not enough bytes.
Query
SELECT * FROM users;
gives
id | val
----+-----
1 | \x
I can't figure out how to map ByteStrings to 'BYTEA`s. According to the docs everything should be OK. What am I doing wrong?
Fixed by replacing the line
execute conn "INSERT INTO users(val) values(?)" $ Only encoded
with
execute conn "INSERT INTO users(val) values(?)" $ Only $ Binary encoded
It's because toField(ByteString) yields Escape whereas toField(Binary ByteString) yields EscapeByteA

PostgreSQL complains about inexistent comparison function for element in primary key

I have a table in a PostgreSQL database in which I want to store the following columns:
STATION LOCATION SERVICE NORTH EAST
text point text real real
Each tuple(STATION, LOCATION, SERVICE) is unique, so I decided to make it a composite type and make it the primary key.
However, when I try to insert a new entry in the database I get the following error:
psycopg2.ProgrammingError: could not identify a comparison function for type point
I guess it is complaining that you cannot order two points in a 2D plane, but I cannot see how that is relevant. I have managed to use composite types that made use of points as primary keys in a test example, so I cannot see how this is different.
I want to know:
Why this is happening.
How it can be fixed, preferrably without changing the table schema.
Debugging information:
testdb=> \d ERROR_KEY
Composite type "public.error_key"
Column | Type | Modifiers
----------+-------+-----------
station | text |
location | point |
service | text |
testdb=> \d testtable
Table "public.testtable"
Column | Type | Modifiers
--------+-----------+-----------
key | error_key | not null
north | real |
east | real |
Indexes:
"testtable_pkey" PRIMARY KEY, btree (key)
For reference, this is the code I am using for the insertion:
from collections import namedtuple
import psycopg2
DB_NAME = 'testdb'
DB_USER = 'testuser'
DB_HOST = 'localhost'
DB_PASSWORD = '123456'
PVT_TABLE_NAME = 'testtable'
Coordinate = namedtuple('Coordinate', ['lat', 'lon'])
PVT_Error_Key = namedtuple('PVT_Error_Key',
['station', 'location', 'service'])
PVT_Error_Entry = namedtuple(
'PVT_Error_Entry', ['key', 'north', 'east'])
def _adapt_coordinate(coord):
"""
Adapter from Python class to Postgre geometric point
"""
lat = psycopg2.extensions.adapt(coord.lat)
lon = psycopg2.extensions.adapt(coord.lon)
return psycopg2.extensions.AsIs("'(%s, %s)'" % (lat, lon))
def _connect_to_db(db_name, db_user, db_host, db_password):
"""
Connects to a database and returns a cursor object to handle the connection
"""
connection_str = ('dbname=\'%s\' user=\'%s\' host=\'%s\' password=\'%s\''
% (db_name, db_user, db_host, db_password))
return psycopg2.connect(connection_str).cursor()
def main():
# Register the adapter for the location
psycopg2.extensions.register_adapter(Coordinate, _adapt_coordinate)
cursor = _connect_to_db(DB_NAME, DB_USER, DB_HOST, DB_PASSWORD)
# Create a dummy entry
entry = PVT_Error_Entry(
key=PVT_Error_Key(station='GKIR',
location=Coordinate(lat=12, lon=10),
service='E1'),
north=1, east=2)
# Insert the dummy entry in the database
cursor.execute(
'INSERT INTO %s '
'(KEY, NORTH, EAST) '
'VALUES((%%s, %%s, %%s), %%s, %%s)'
% PVT_TABLE_NAME,
(entry.key.station, entry.key.location, entry.key.service,
entry.north, entry.east))
# Retrieve and print all entries of the database
cursor.execute('SELECT * FROM %s', (PVT_TABLE_NAME))
rows = cursor.fetchall()
print(rows)
if __name__ == '__main__':
main()
You cannot use a column of type point in a primary key, e.g.:
create table my_table(location point primary key);
ERROR: data type point has no default operator class for access method "btree"
HINT: You must specify an operator class for the index or define a default operator class for the data type.
The error message is clear enough, you need to create a complete btree operator class for the type.
The full procedure is described in this answer: Creating custom “equality operator” for PostgreSQL type (point) for DISTINCT calls.
Update. With the workaround you mentioned in your comment
create table my_table(
x numeric,
y numeric,
primary key (x, y));
insert into my_table values
(1.1, 1.2);
you can always create a view, which can be queried just like a table:
create view my_view as
select point(x, y) as location
from my_table;
select *
from my_view;
location
-----------
(1.1,1.2)
(1 row)

haskell postgresql-simple incompatible type _int8 and Int64 (and Integer)

The erroneous function below is part of a program called subdivide working with Postgis geospatial intersections on the server side and processing the returned array of Int64 on the client side.
It is built and run under Stack, resolving to Nightly 2016-08-02 and explicitly specifying architecture x86_64.
I get the following runtime error executing the Postgres query defined as "intersectionsSql" (see the comment RUNTIME ERROR HERE):
"Created table: server : [Only {fromOnly = \"PostgreSQL 9.6beta2 on x86_64-pc-linux-gnu, compiled by gcc (Debian 4.9.2-10) 4.9.2, 64-bit\"}] quadrant: BOX3D(-180.0 90.0, -90.0 45.0)"
subdivide: Incompatible {errSQLType = "_int8", errSQLTableOid = Nothing, errSQLField = "object_ids", errHaskellType = "Int64", errMessage = "types incompatible"}
I have tried Integer, Int64 and Int, all with the same result, which is counter-intuitive as those Haskell types should all be compatible with _int8 according to the PostgreSQL-simple instance documentation:
https://hackage.haskell.org/package/postgresql-simple-0.5.0.0/candidate/docs/Database-PostgreSQL-Simple-FromField.html
The SQL query should return a single row of postgres bigint[], which I have confirmed via PGAdmin.
Any ideas?
Also any comments around how I have written the code - its over a decade since last I worked with GHC and times have changed.
Thanks for your consideration.
Mike Thomas
accumulateIntersections :: Identifier -> Identifier -> ConnectInfo -> ((Double,Double),(Double,Double)) -> IO ()
accumulateIntersections sourceTable accumulationTable connectionInfo q =
let
theBox = makeBox3D (fst (fst q)) (snd (fst q)) (fst (snd q)) (snd (snd q))
theValue = (Only theBox)
dropTable = [sql| DROP TABLE IF EXISTS ? CASCADE |]
createTable = [sql| CREATE TABLE ? ( quadrant_id BIGSERIAL, area_metres_squared FLOAT8, shape GEOMETRY, object_ids BIGINT[] ) |]
aggregateSql = [sql| DROP AGGREGATE IF EXISTS _array_agg (anyarray);
CREATE AGGREGATE _array_agg(anyarray) (SFUNC = array_cat, STYPE = anyarray);
|]
intersectionsSql = [sql| SELECT _array_agg (object_ids) object_ids
FROM ?
WHERE ST_Intersects(ST_SetSRID ( ?::box3d, 4326 ), shape)
|]
insertIntersections = [sql| INSERT INTO ? (shape, object_ids)
VALUES ( ST_SetSRID ( ?::box3d, 4326 )
, ? ) |]
in
do
connection <- connect connectionInfo
execute_ connection aggregateSql
postgresVersion <- (query_ connection "SELECT version()" :: IO [Only String])
i0 <- execute connection dropTable (Only accumulationTable)
i1 <- execute connection createTable (Only accumulationTable)
print ("Created table: server : " ++ (show postgresVersion) ++ " quadrant: " ++ theBox)
is :: [Only Int64] <- query connection intersectionsSql (sourceTable, theBox) -- RUNTIME ERROR HERE
print ("Intersections done.")
ids::[Int64] <- forM is (\(Only id) -> return id)
print ("Ids done.")
close connection
return ()
See the above comment relayed from LP Smith, who I contacted when no answers were forthcoming here. It resolves my issue.
The key was to recognize that _int8 represents an array of 8 byte integers, rather than thinking, as I had done, that it was an internal representation for a single 8 byte integer. Leon's suggested change was to substitute "[Only (Vector Int64)]" for "[Only Int64]" in the line marked above as the point of the runtime error.
Thank you Leon.

postgresql-simple query error

EDIT: I now have a better idea of what is going wrong. When I perform that query in plain old psql, I get the following result:
lwm#verbos
=# SELECT * FROM gerund LIMIT1;
infinitive │ gerund │ gerund_english
────────────┼─────────────┼────────────────
abandonar │ abandonando │ abandoning
So, I am getting back 3 strings? However, I say that I am getting back IO [Only String]. I am sure it is my type signature here that is messing things up ...
I am trying to make a simple query using the postgresql-simple library with Haskell. My code is pasted below along with the error I am seeing. Anyone got any ideas?
My database is called verbos and within it, I have a table called gerund. I am able to run a query_ that contains: conn "SELECT 2 + 2" and that works fine. I can also connect to my database with the default data as specified with the default information (password = 'postgres' : psql -h localhost -p 5432 -U postgres (from the docs[1])
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Control.Monad
import Control.Applicative
import Database.PostgreSQL.Simple
main = do
conn <- connect defaultConnectInfo {
connectPassword = "postgres",
connectDatabase = "verbos"
}
mapM_ print =<< (query_ conn "SELECT * FROM gerund LIMIT 1" :: IO [Only String])
Gives me the following error:
ConversionFailed {errSQLType = "3 values: [(Basic {typoid = Oid 1043,
typcategory = 'S', typdelim = ',', typname = \"varchar\"},Just
\"abandonar\"),(Basic {typoid = Oid 1043, typcategory = 'S', typdelim
= ',', typname = \"varchar\"},Just \"abandonando\"),(Basic {typoid = Oid 1043, typcategory = 'S', typdelim = ',', typname =
\"varchar\"},Just \"abandoning\")]", errSQLTableOid = Nothing,
errSQLField = "", errHaskellType = "1 slots in target type",
errMessage = "mismatch between number of columns to convert and number
in target type"}
OK, Thanks to #AlpMestanogullari, #muistooshort, I got an answer here. My final code is:
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Control.Applicative
import Database.PostgreSQL.Simple
import Database.PostgreSQL.Simple.FromRow
data Gerund = Gerund {
f :: String,
s :: String,
t :: String
} deriving (Show)
instance FromRow Gerund where
fromRow = Gerund <$> field <*> field <*> field
main = do
conn <- connect defaultConnectInfo {
connectPassword = "postgres",
connectDatabase = "verbos"
}
mapM_ print =<< (query_ conn q :: IO [Gerund])
where q = "SELECT * FROM gerund LIMIT 1"
Notes:
Knowing that my result contained 3 result columns, I needed to define a type that had 'space' for the results (f, s and t in the Gerund type)
I followed the docs[1] for FromRow closely to get my type and instance defined.
You need to import import Database.PostgreSQL.Simple.FromRow to access things like field.