Databricks - read table from Snowflake to Databricks - pyspark

Question

I've seen a few questions on Databricks to Snowflake but my question is how to get a table from Snowflake into Databricks.
What I've done so far:
Created a cluster and attached the cluster to my notebook (I'm using Python)
# Use secrets DBUtil to get Snowflake credentials.
user = dbutils.secrets.get("snowflake-user", "secret-user")
password = dbutils.secrets.get("snowflake-pw", "secret-pw")
sf_url = dbutils.secrets.get("snowflake-url", "secret-sf-url")
# snowflake connection options
options = {
"sfUrl": sf_url,
"sfUser": user,
"sfPassword": password,
"sfDatabase": "DEV",
"sfSchema": "PUBLIC",
"sfWarehouse": "DEV_WH"
}
then I tried to use spark.read to read the FBK_VIDEOS table in Snowflake:
# Read table from Snowflake.
df = spark.read.format("snowflake").options(**options).option("dbtable", "FBK_VIDEOS").load()
I've also tried: option("dbtable", "SELECT * FROM FBK_VIDEOS").load()
but I see the following error for df:
net.snowflake.client.jdbc.SnowflakeSQLException: SQL compilation
error:
The Traceback shows this when expanded:
Py4JJavaError Traceback (most recent call last)
<command-3339556253176158> in <module>
1 # Read table from Snowflake.
----> 2 df = spark.read.format("snowflake").options(**options).option("dbtable", "FBK_VIDEOS").load()
3
4 display(df)
/databricks/spark/python/pyspark/sql/readwriter.py in load(self, path, format, schema, **options)
208 return self._df(self._jreader.load(self._spark._sc._jvm.PythonUtils.toSeq(path)))
209 else:
--> 210 return self._df(self._jreader.load())
211
212 def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,
/databricks/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py in __call__(self, *args)
1302
1303 answer = self.gateway_client.send_command(command)
-> 1304 return_value = get_return_value(
1305 answer, self.gateway_client, self.target_id, self.name)

Answer 1

Answering for completeness and for future users who might have a similar problem.
As answered in the comments: Snowflake uses a role-based access control system, so it is vitally important that the role being used has the necessary privileges. In this case, there is no USE ROLE shown in the code so whatever role was active when the query was run did not have sufficient privileges.