Stream Joins based on distinct values from a previous stream - pyspark

Question

I have a main table A which stores all the events which occur with some details.
Then for each "event" in Table A, there is a separate table by that name (called the event table)
I am using the main table A as a stream and ideally need to perform a join to the event table so that I get a single table for Table A with its respective details.
In the case below there are two distinct event types tables each with its own table and schema.
Example:
Table A:
id
time
detail_1
detail_2
event
Table event 1:
id
detail_6
detail_8
detail_9
Table event 2:
id
detail 11
detail 12
How do I union it so I have a single table in the end with the corresponding details from Table A, event 1 and event 2?
Here is what I was trying to do:
df = (
spark.readStream.format("delta")
.option("ignoreChanges", "true")
.load(f"{table_name}")
)
event_types = df.select("event").distinct().collect()
for row in event_types:
event = row[0].replace(" ", "_").replace(":","").lower()
if event in ["task", "module", "_created", "test_created", "test_to_be_deleted"]:
df_event = spark.readStream.format("delta").option("ignoreChanges", "true").load(f"{event}")
joined_df = df.join(df_event, Seq("message_id"),"inner")
df.writeStream.format("delta").outputMode("append").option(
"checkpointLocation",
f"{table}",
).trigger(once=True).foreachBatch(apple_a_bunch_of_changes).start()
is there a better way to do this?