Is it possible to retain newlines when using CDATA in BIML Variables? - biml

Question

I have some SQL in a variable that I want to use in a BIML template. (Originally I was getting the SQL from a C# variable, but that doesn't seem to be relevant) I'm using BIML like this:
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Connections>
<OleDbConnection Name="Source" ConnectionString="Provider=SQLNCLI11;Data Source=localhost\dev2017;Integrated Security=SSPI;Initial Catalog=tempdb;" />
</Connections>
<Packages>
<Package Name="SO_51407483" ConstraintMode="Linear">
<Variables>
<Variable Name="querySql" DataType="String" IncludeInDebugDump="Exclude" EvaluateAsExpression="true">
<![CDATA["SELECT --This is a comment
1 AS Col1"]]>
</Variable>
</Variables>
<Tasks>
<Dataflow Name="DFT Demo">
<Transformations>
<OleDbSource Name="OLESRC query" ConnectionName="Source">
<VariableInput VariableName="User.querySql" />
</OleDbSource>
</Transformations>
</Dataflow>
</Tasks>
</Package>
</Packages>
</Biml>
The issue I have is that when the file is compiled the line breaks in querySql appear to be stripped out, but only during the validation step of the compilation. This means that anything after the first comment in querySql becomes part of the comment and BIML's validation fails.
The error message is:
Node OLESRC query: Could not execute Query on Connection Source:
SELECT --This is a comment1 AS Col1
Incorrect syntax near 'SELECT'
If I change querySql to use the /* */ style of comment the validation passes. If I then open the generated dtsx and look at the variable in SSIS the line breaks are there again.
Is there a way around this without having to use /* */ everywhere?
Experiments
This doesn't compile:
<Variable Name="querySql" DataType="String" IncludeInDebugDump="Exclude" EvaluateAsExpression="true">
"SELECT --This is a comment
1 AS Col1"
</Variable>
This compiles, and the resulting dtsx variable ends up with one line break
<Variable Name="querySql" DataType="String" IncludeInDebugDump="Exclude" EvaluateAsExpression="true">
"SELECT /*This is a comment*/
1 AS Col1"
</Variable>
This compiles, and the resulting dtsx variable ends up with two line breaks
<Variable Name="querySql" DataType="String" IncludeInDebugDump="Exclude" EvaluateAsExpression="true">
"SELECT --This is a comment
1 AS Col1"
</Variable>
I've raised this with Variagence: https://github.com/varigence/BimlPit/issues/82

Answer 1

As a minimal repro, I'm getting the carriage return/line feeds preserved.
I declare my query variable and then use it in both an Execute SQL Task and a Dataflow's OLEDB Source.
The Execute SQL Task doesn't look like it preserves the CR/LF until you click the ellipses but then the popup correctly displays the code.
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<#
string query = #"--This is a comment
SELECT 1 AS Col1";
#>
<Connections>
<OleDbConnection Name="Source" ConnectionString="Provider=SQLNCLI11;Data Source=localhost\dev2017;Integrated Security=SSPI;Initial Catalog=tempdb;" />
</Connections>
<Packages>
<Package Name="SO_51407483" ConstraintMode="Linear">
<Tasks>
<ExecuteSQL Name="SQL Test Task" ConnectionName="Source">
<DirectInput><![CDATA[<#=query#>]]></DirectInput>
</ExecuteSQL>
<Dataflow Name="DFT Demo">
<Transformations>
<OleDbSource Name="OLESRC query" ConnectionName="Source">
<DirectInput><![CDATA[<#=query#>]]></DirectInput>
</OleDbSource>
</Transformations>
</Dataflow>
</Tasks>
</Package>
</Packages>
</Biml>