How to access internal representation of JSONb? - postgresql

Question

In big-data queries the intermediary "CAST to text" is a performance bottleneck... The good binary information is there, at the JSONb datatype: how to rescue it?
Typical "select where" example:
with t(x,j) as (select 'hello','{"flag1":true,"flag2":false}'::jsonb)
SELECT x FROM t
WHERE (j->>'flag1')::boolean AND NOT((j->>'flag2')::boolean)
The the "casting to text" is a big loss of performance. Ideal is a mechanism to do direct, from JSONb to Boolean, something as
WHERE (j->'flag1')::magic_boolean AND NOT((j->'flag2')::magic_boolean)
PS: it is possible using C++? Is possible a CREATE CAST C++ implementation to solve this problem?

Answer 1

The feature is implemented in Postgres 11:
E.4.3.4. Data Types
[...]
Add casts from JSONB scalars to numeric and boolean data types (Anastasia Lubennikova)
Db<>Fiddle.

Answer 2

TL;DR
Performance-wise it's best to use #> with an appropriate index covering all JSON attributes including type conversions (to avoid type conversions when accessing the index): https://dbfiddle.uk/?rdbms=postgres_11&fiddle=4da77576874651f4d2cf801142ae34d2
CREATE INDEX idx_flags_btree_jsonb ON t ((j#>'{flag1}'), (j#>'{flag2}'));
Times (all selecting the same 5,195 rows out of 1,000,000):
->>::boolean | ~75 ms
->::boolean | ~55 ms
#> | ~80 ms
#> | ~40 ms
Scalability:
Interestingly, a local test with 40M rows (all cached in memory, no I/O latency here) revealed the following (best) numbers out of 10 runs (excluding the first and last run) for each query:
->>::boolean | 222.333 ms
->::boolean | 268.002 ms
#> | 1644.605 ms
#> | 207.230 ms
So, in fact, the new cast seems to slow things down on larger data sets (which I suspect is due to the fact that it still converts to text before converting to boolean but within a wrapper, not directly).
We also can see that the #> operator using the GIN index doesn't scale very well here, which is expected, as it is much more generic than the other special-purpose indexes and hence, needs to do a lot more under-the-hood.
However, in case these special purpose btree indexes cannot be put in place or I/O becomes a bottleneck, then the GIN index will be superior as it consumes only a fraction of the space on disk (and also in memory), increasing the chance of an index buffer hit.
But that depends on a lot of factors and needs to be decided with all accessing applications understood.
Details:
Preferably use the #> containment operator with a single GIN index as it saves a lot of special-purpose indexes:
with t(x,j) as (select 'hello','{"flag1":true,"flag2":false}'::jsonb)
SELECT x FROM t
WHERE j #> '{"flag1":true, "flag2":false}'::jsonb;
...which gives a plan like:
QUERY PLAN
-----------------------------------------------------------
CTE Scan on t (cost=0.01..0.03 rows=1 width=32)
Filter: (j #> '{"flag1": true, "flag2": false}'::jsonb)
CTE t
-> Result (cost=0.00..0.01 rows=1 width=64)
(4 rows)
As an alternative (if you can afford creating special-purpose indexes and the resulting write penalty) use the #> operator instead of -> or ->> and by that skip any performance-costly type conversions, e.g.
with t(x,j) as (select 'hello','{"flag1":true,"flag2":false}'::jsonb)
SELECT x FROM t
WHERE j#>'{flag1}' = 'true'::jsonb AND j#>'{flag2}' = 'false'::jsonb;
...resulting in a plan like:
QUERY PLAN
--------------------------------------------------------------------------------------------------------
CTE Scan on t (cost=0.01..0.04 rows=1 width=32)
Filter: (((j #> '{flag1}'::text[]) = 'true'::jsonb) AND ((j #> '{flag2}'::text[]) = 'false'::jsonb))
CTE t
-> Result (cost=0.00..0.01 rows=1 width=64)
(4 rows)
So, no more implicit type conversion here (only for the given constants, but that's a one-time operation, not for every row).