Can't connect to Postgresql with specific external IP - postgresql

Question

I can connect to my DigitalOcean Ubuntu 20LTS VM instant that has PostgreSQL 14 installed without issue, but I'm trying to make it more secure with only specific IPs that can connect to the database.
I heard the way to do this is to modify the /etc/postgresql/14/main/postgresql.conf file.
When I have this line, I can connect to my database without issue.
listen_addresses='0.0.0.0'
However, if I modify this line with:
listen_addresses='123.123.123.123'
I get this DataGrip error message: [08001] Connection to 111.111.111.111:12345 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
111.111.111.111:12345 is my (fake) VM's IP and port that I already set up.
123.123.123.123 is my (fake) computer's external IP that I get from here or here
Any suggestions? Is there a log I can search from that will give me a better understanding of what is going on?
Also to note, with listen_addresses='0.0.0.0', running ss -ptl gives an output of
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 4096 127.0.0.53%lo:domain 0.0.0.0:*
LISTEN 0 128 0.0.0.0:ssh 0.0.0.0:*
LISTEN 0 244 0.0.0.0:12345 0.0.0.0:*
LISTEN 0 128 [::]:ssh [::]:*
with listen_addresses='123.123.123.123', running ss -ptl gives an output of
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 4096 127.0.0.53%lo:domain 0.0.0.0:*
LISTEN 0 128 0.0.0.0:ssh 0.0.0.0:*
LISTEN 0 128 [::]:ssh [::]:*
Documentation that I used so far:
https://www.postgresql.org/docs/current/runtime-config-connection.html
https://www.postgresql.org/docs/current/auth-pg-hba-conf.html