How to understand/debug bad index result on slave instance - postgresql

Question

I have the following schema and 2 postgresql instances with a slave instance replicating a master instance.
CREATE TABLE t (id serial PRIMARY KEY, c text);
CREATE INDEX ON t (upper(c));
I get this incorrect result on the slave instance.
# SELECT id, c, upper(c), upper(c) = upper('FOO') FROM t WHERE id IN (123, 456);
id | c | upper | ?column?
-----+-----+-------+----------
123 | Foo | FOO | t
456 | foo | FOO | t
(2 rows)
# SELECT id, c, upper(c), upper(c) = upper('FOO') FROM t WHERE upper(c) = upper('FOO');
id | c | upper | ?column?
----+---+-------+----------
(0 rows)
The second query should return the same rows as the first query.
However, the result is correct on the master instance.
# SELECT id, c, upper(c), upper(c) = upper('FOO') FROM t WHERE id IN (123, 456);
id | c | upper | ?column?
-----+-----+-------+----------
123 | Foo | FOO | t
456 | foo | FOO | t
(2 rows)
# SELECT id, c, upper(c), upper(c) = upper('FOO') FROM t WHERE upper(c) = upper('FOO');
id | c | upper | ?column?
-----+-----+-------+----------
123 | Foo | FOO | t
456 | foo | FOO | t
(2 rows)
Using EXPLAIN on the second query, I can see that it's using the index as expected, so I suspect the index data is somehow incorrect on the slave instance. Doing a REINDEX on the master instance does not resolve the issue and doing it on the slave instance is not possible because of the replication.
Is it possible that the index data is correct on the master instance and incorrect on the slave instance? How to further debug the issue?
UPDATE: This is the query plan of the second query on both the master and the slave instance
Index Scan using t_upper_idx on t (cost=0.43..8.46 rows=1 width=60)
Index Cond: (upper((c)::text) = 'FOO'::text)
There are ~3M rows in the t table.
UPDATE: Server version is server 11.4 (Debian 11.4-1.pgdg90+1)) on the master, and server 11.7 (Debian 11.7-0+deb10u1)) on the slave.