Preserve structure of YAML with ruamel yaml - ruamel.yaml

Question

I've been using ruamel yaml to edit my YAML files and dump them back. I need help understanding how to keep the same structure as the original file has, because all I do is duplicate it, edit, and write it again.
For example, this is the original file:
ElasticLoadBalancingV2Listener:
Type: "AWS::ElasticLoadBalancingV2::Listener"
Properties:
LoadBalancerArn: !Ref ElasticLoadBalancingV2LoadBalancer
Port: !FindInMap [NLBPorts, Port1, Port]
Protocol: "TCP"
DefaultActions:
-
Order: 1
TargetGroupArn: !Ref ElasticLoadBalancingV2TargetGroup1
Type: "forward"
The new file doesn't look the same:
ElasticLoadBalancingV2Listener:
Type: "AWS::ElasticLoadBalancingV2::Listener"
Properties:
LoadBalancerArn: !Ref ElasticLoadBalancingV2LoadBalancer
Port: !FindInMap [NLBPorts, Port1, Port]
Protocol: "TCP"
DefaultActions:
- Order: 1
TargetGroupArn: !Ref ElasticLoadBalancingV2TargetGroup1
Type: "forward"
The biggest issue is that I used all sorts of tricks that ruamel has to fix this, but each time some different part of the yaml breaks.
This is my function:
def editEndpointServiceTemplate(endpoint_service_template_path):
yaml = YAML()
yaml.preserve_quotes = True
# yaml.compact(seq_seq=False, seq_map=False)
# yaml.indent(mapping=4, sequence=3, offset=0)
#Load yaml file
with open(endpoint_service_template_path) as fp:
data = yaml.load(fp)
#Edit the yaml
data['Description'] = "CloudFormation"
#Write new yaml file
with open(endpoint_service_template_path, 'w') as fp:
yaml.dump(data, fp)
As you can see with the commented commands, I tinkered around with the settings but couldn't find the sweet spot.

Answer 1

In this case it is obvious that your function did not produce the output you present
(different indent, missing word "CloudFormation"), but in general you should take care
in questions this is the same, and that your program is complete so that results can
more easily be reproduced.
ruamel.yaml does not have built in functions for all kinds of seldom seen format, but yours
is relatively close to the output using the method .indent(mapping=4, sequence=4, offset=2) and
transform by checking line by line.
As it is less likely that withing string scalars you have a sequence indicators followed by three spaces ("- ")
(which additionally would have to occur wrapped to be the first non-space on a line), you better do
.indent(mapping=4, sequence=4, offset=0) transform that:
import sys
import ruamel.yaml
yaml_str = """\
ElasticLoadBalancingV2Listener:
Type: "AWS::ElasticLoadBalancingV2::Listener"
Properties:
LoadBalancerArn: !Ref ElasticLoadBalancingV2LoadBalancer
Port: !FindInMap [NLBPorts, Port1, Port]
Protocol: "TCP"
DefaultActions:
-
Order: 1
TargetGroupArn: !Ref ElasticLoadBalancingV2TargetGroup1
Type: "forward"
"""
yaml = ruamel.yaml.YAML()
yaml.indent(mapping=4, sequence=4, offset=0)
yaml.preserve_quotes = True
data = yaml.load(yaml_str)
data['Description'] = "CloudFormation"
def break_seq(s):
result = []
PAT = '- '
for line in s.splitlines():
ls_line = line.lstrip()
if ls_line.startswith(PAT):
line = line.replace(PAT, ' - \n' + ' ' * (line.index(PAT) + 4))
result.append(line)
return '\n'.join(result)
yaml.dump(data, sys.stdout, transform=break_seq)
which gives:
ElasticLoadBalancingV2Listener:
Type: "AWS::ElasticLoadBalancingV2::Listener"
Properties:
LoadBalancerArn: !Ref ElasticLoadBalancingV2LoadBalancer
Port: !FindInMap [NLBPorts, Port1, Port]
Protocol: "TCP"
DefaultActions:
-
Order: 1
TargetGroupArn: !Ref ElasticLoadBalancingV2TargetGroup1
Type: "forward"
Description: CloudFormation
The above can be done by "hacking" the routines that serialize to sequences, but
it is often easier to just transform the output, although not as efficient in
time/space usage.
Unless the program consuming this output uses an incomplete YAML parser, the
actual data structure loaded will not change, it is just less readable for people
unused to such uncommon formatting.