Utilizing macro to drop multiple schemas from a snowflake database based on conditions - macros

Question

{% macro drop_schema_str(schema) %}
{% set relation = api.Relation.create(database=target.database, schema=schema) %}
{% do drop_schema(relation) %}
{% endmacro %}
This macro works perfectly. But I'd like to drop a bunch of schemas at a time. I couldn't figure out how to add this query into this macro and run it?
SELECT
SCHEMA_NAME
FROM
`{{ target.database }}.INFORMATION_SCHEMA.SCHEMATA`
WHERE
SCHEMA_NAME != 'INFORMATION_SCHEMA'
AND SCHEMA_NAME ILIKE '%DEV_EL_%'
I tried for-loop methods in the macro but while running it continued to give errors.Thank you

Answer 1

I believe the reason this isn't working is that you're trying to execute some SQL to populate your SQL query. In principle that isn't impossible, but DBT constructs models in two stages:
compile the SQL queries from your templates
execute those queries in the appropriate sequence
So I imagine that when your macro tries to compile, it's trying to create a set of SQL queries to drop relations, but the information needed for those queries is not yet available.
Fortunately, I think they have you covered. There is a special construction, {% if execute %}, which appears to be designed for exactly your use case!
Adapting the example from the docs, it may be possible to take the following and append a for loop with calls to your macro:
{% set to_drop_query %}
SELECT
SCHEMA_NAME
FROM
`{{ target.database }}.INFORMATION_SCHEMA.SCHEMATA`
WHERE
SCHEMA_NAME != 'INFORMATION_SCHEMA'
AND SCHEMA_NAME ILIKE '%DEV_EL_%'
{% endset %}
{% set results = run_query(to_drop_query) %}
{% if execute %}
{# Return the first column #}
{% set relations_to_drop = results.columns[0].values() %}
{% endif %}