AWS Glue CDK - create job type Spark (Glue 2.0) - pyspark

Question

I could not find any documentation on how to create a glue job with the type spark. The way examples and documentation suggests creates type python shell. Example:
glueETLJob = _glue.CfnJob(
self,
"glue_CDK_job",
command =_glue.CfnJob.JobCommandProperty(
name = "glue_CDK_job",
python_version= '3',
script_location = bucket + "/code/glue_CDK_job.py"
),
role= glueRole.role_arn,
max_retries= 0,
name= "glue_CDK_job",
timeout=30,
glue_version="1.0"
)
Creates a python shell with version 1, but I cannot set glue_version="2.0" because that only seems to exist in type Spark.
Does anyone know how to do create a glue job with type Spark and glue version 2.0 with CDK
Thanks

Answer 1

Turns out the name in theJobCommandProperty is not a id like others, but the type that I was looking for. So if anybody has the same issue it should look like:
glueETLJob = _glue.CfnJob(
self,
"glue_CDK_job",
command =_glue.CfnJob.JobCommandProperty(
name = "glueetl",
python_version= '3',
script_location = bucket + "/code/glue_CDK_job.py"
),
role= glueRole.role_arn,
max_retries= 0,
name= "glue_CDK_job",
timeout=30,
glue_version="2.0"
)
Got the answer from: https://github.com/aws/aws-cdk/issues/4480

Answer 2

I got the same issue, but I realized that the issue was caused by name in JobCommandProperty. Change glue_CDK_job to glueetl will be work.
You can check the documentation of AWS Cloud Formation.
https://github.com/awsdocs/aws-cloudformation-user-guide/blob/main/doc_source/aws-resource-glue-job.md

Answer 3

const processFifaDataJobName = 'process-data-fifa';
const PYTHON_VERSION = "3";
const GLUE_VERSION = "1.0";
const COMMAND_NAME = "glueetl";
const glueJobProcessFifaData = new glue.CfnJob(this, processFifaDataJobName, {
name: processFifaDataJobName,
role: role.roleArn,
command: {
name: COMMAND_NAME,
pythonVersion: PYTHON_VERSION,
scriptLocation: 's3://' + bucketName + '/Scripts/process-data.py'
},
glueVersion: GLUE_VERSION
});
For me above code worked. Although it is very similar what you are trying to do. But this created job with type spark.