How much of an issue is connection pinning with RDS Proxy? - postgresql

Question

I'm using Postgres with row-level security to lock down all queries across tables to a particular tenancy. I do this by calling a SET app.tenant_id = x; statement each time my service opens a connection, and my RLS policies use this session-level setting to limit the return data. Basically the approach described here under 'Alternative Approach'.
If this service is deployed in AWS, with RDS Proxy in between it and the database then I understand it'll be subject to 'connection pinning' since I'm using a SET statement. I'm trying to get a feel for how big an issue this actually is. A few questions:
Are SET LOCAL statements also going to cause pinning?
If my service connections to RDS Proxy are short-lived and a single transaction (which they will be 99% of the time) does this lessen the impact?
Does service connection pooling (service -> RDS Proxy) help or hinder?
Basically any advice on how much of an issue this is, how I can make this work, or any workarounds, would be appreciated.

Answer 1

I had to delete my previous answer because I was originally using pgAdmin, which is apparently very keen on pinning connections. This meant I couldn't trust my data. I have redone this experiment with another more well behaved tool, psql.
I understand where you're coming from. According to the documentation, SET will cause pinning, but it's unclear if that includes SET LOCAL. Since I need this information too, I will run a test today and post the results here.
I will do the following
Step 1: Open one connection through our proxy and use a regular SET so ensure that the DatabaseConnectionsCurrentlySessionPinned metric increases to 1. I will use the following query:
SET search_path TO blah;
SET app.tenant_id TO 123;
Step 2: I will close that connection and see that the metric decreases back down to 0.
Step 3: I will open a new connection, but this time I will use the following query:
SET LOCAL search_path TO blah;
SET LOCAL app.tenant_id TO 123;
Step 4: I will close the connection, and if the connection is pinned, I will monitor the metric to see if and when it decreases back to 0.
Let's do this
Caveat: don't look at the metrics in RDS Management Console. See: https://repost.aws/questions/QUfPWoiFxmR7-lios5NrFwBA/fix-database-connections-currently-session-pinned-metric-on-rds-proxy-dashboard
Step 1
The connection between proxy and server was pinned immediately when I ran SET commands, as expected.
Step 2
The pinned connection between proxy and server was closed immediately when I closed the connection between client and proxy.
Step 3
The connection between proxy and server was not pinned when I ran SET LOCAL commands.
Step 4
The connection was not pinned, so this step is superfluous.
Conclusion
SET LOCAL does circumvent pinning in RDS Proxy, with the caveat that it must be done within a transaction.
In my previous attempt to answer this question, pgAdmin's behavior made me conclude that pinning does occur in both cases. This is wrong.
To answer your other questions, if pinning does occur, it doesn't matter that your transactions are short. The server connection will remain pinned until the client connection is gone. Your only resort is to make sure you close client connections once they're pinned.
The documentation states that "when a connection is pinned, each later transaction uses the same underlying database connection until the session ends. Other client connections also can't reuse that database connection until the session ends. The session ends when the client connection is dropped."