How can I dump changes corresponding to list using ruamel.yaml? - ruamel.yaml

I am using solution in the related answer for How to auto-dump modified values in nested dictionaries using ruamel.yaml
with RoundTripRepresenter.
If I make chnages on a list, ruamel.yaml is able to make changes on the local variable, but it does not dump/write the changes into the file. Would it be possible to achive it?
Example config file:
live:
- name: Ethereum
networks:
- chainid: 1
explorer: https://api.etherscan.io/api
For example I changed name into alper and tried to append new item into the list.
my code:
class YamlUpdate:
def __init__(self):
self.config_file = Path.home() / "alper.yaml"
self.network_config = Yaml(self.config_file)
self.change_item()
def change_item(self):
for network in self.network_config["live"]:
if network["name"] == "Ethereum":
network["name"] = "alper"
print(network)
network.append("new_item")
yy = YamlUpdate()
print(yy.config_file.read_text())
output is where name remains unchanged on the original file:
{'name': 'alper', 'networks': [{'chainid': 1, 'explorer': 'https://api.etherscan.io/api'}]}
live:
- name: Ethereum
networks:
- chainid: 1
explorer: https://api.etherscan.io/api

I think you should look at making a class SubConfigList that behaves like a list but notifies its parent (in the datastructure), like in the other answer where SubConfig notifies its parent.
You'll also need to make sure to represent SubConfigList as a sequence into the YAML document.
If you ever going to have list at the root of your data structure, you'll need to have list-like alternative to the dict-like Config. (Or document for the consumers/users of your code that the root always needs to be a mapping).

Related

Typo3 Define storagePid for CommandController command

I would like to import different data using a CommandController (scheduler).
I allready figured out that it is possible to set a global storagePid like:
module.tx_myextension.persistence.storagePid = 123
source: https://worksonmymachine.org/blog/commandcontroller-and-storagepid
That works fine, but my extension contains multiple models which should be saved on different Pid's
I also found an old post where someone said it is possible to define a pid for each model which would be exactly what I need:
module.tx_myextension.persistence.classes.tx_myextension_domain_model_player.storagePid = 124
module.tx_myextension.persistence.classes.tx_myextension_domain_model_customer.storagePid = 125
source: https://typo3-german.typo3.narkive.com/WxjjtxXa/scheduler-storage-pid
But it seems like this lines get ignored. Is this the correct way or do I do something wrong?
I am on TYPO3 6.2.44
I suggest to create params for the controller action. For each model a storage pid.
so you have myCommand($domain1Pid, $domain2Pid,$domain3Pid, ...)
Now as first call in your function you get the querySettings for the repositories and apply the storage pids:
$querySettings = $this->domain1Repository->createQuery()->getQuerySettings();
$querySettings->setStoragePageIds([$domain1Pid]);
$this->domain1Repository->setDefaultQuerySettings($querySettings);
repeat this for each repository. In the scheduler job settings or cli you can now define the pids for each storage.
btw: you can also use $domain->setPid(123) to set the pid of each model where to save.

NoSQL Injections with Rasa

Security Concern
I have recently been playing around with Rasa and having MongoDB as the database behind.
I was wondering whether one should preprocess the inputs to Rasa somehow in order to prevent NoSQL injections?
I tried putting a Custom Component as a part of Rasa NLU Pipeline, but as soon as something hits the first element of the NLU pipeline, it seems that the original text is also saved in Mongo.
domain_file
language: "de"
pipeline:
- name: "nlu_components.length_limiter.LengthLimiter"
- name: "tokenizer_whitespace"
- name: "intent_entity_featurizer_regex"
- name: "ner_crf"
- name: "ner_synonyms"
- name: "intent_featurizer_count_vectors"
- name: "intent_classifier_tensorflow_embedding"
length_limiter.py - notice the "process" method
from rasa_nlu.components import Component
MAX_LENGTH = 300
class LengthLimiter(Component):
"""
This component shortens the input message to MAX_LENGTH chars
in order to prevent overloading the bot
"""
# Name of the component to be used when integrating it in a
# pipeline. E.g. ``[ComponentA, ComponentB]``
# will be a proper pipeline definition where ``ComponentA``
# is the name of the first component of the pipeline.
name = "LengthLimiter"
# Defines what attributes the pipeline component will
# provide when called. The listed attributes
# should be set by the component on the message object
# during test and train, e.g.
# ```message.set("entities", [...])```
provides = []
# Which attributes on a message are required by this
# component. e.g. if requires contains "tokens", than a
# previous component in the pipeline needs to have "tokens"
# within the above described `provides` property.
requires = []
# Defines the default configuration parameters of a component
# these values can be overwritten in the pipeline configuration
# of the model. The component should choose sensible defaults
# and should be able to create reasonable results with the defaults.
defaults = {
"MAX_LENGTH": 300
}
# Defines what language(s) this component can handle.
# This attribute is designed for instance method: `can_handle_language`.
# Default value is None which means it can handle all languages.
# This is an important feature for backwards compatibility of components.
language_list = None
def __init__(self, component_config=None):
super(LengthLimiter, self).__init__(component_config)
def train(self, training_data, cfg, **kwargs):
"""Train this component.
This is the components chance to train itself provided
with the training data. The component can rely on
any context attribute to be present, that gets created
by a call to :meth:`components.Component.pipeline_init`
of ANY component and
on any context attributes created by a call to
:meth:`components.Component.train`
of components previous to this one."""
pass
def process(self, message, **kwargs):
"""Process an incoming message.
This is the components chance to process an incoming
message. The component can rely on
any context attribute to be present, that gets created
by a call to :meth:`components.Component.pipeline_init`
of ANY component and
on any context attributes created by a call to
:meth:`components.Component.process`
of components previous to this one."""
message.text = message.text[:self.defaults["MAX_LENGTH"]]
def persist(self, model_dir):
"""Persist this component to disk for future loading."""
pass
#classmethod
def load(cls, model_dir=None, model_metadata=None, cached_component=None,
**kwargs):
"""Load this component from file."""
if cached_component:
return cached_component
else:
component_config = model_metadata.for_component(cls.name)
return cls(component_config)
I played around the mongo tracker store and could not inject anything.
However, if you want to add your own component to go absolutely sure you'd have to implement your own input channel. There you can change the messages, before it is processed by Rasa Core.

Moving Wagtail pages in a migration

I'm restructuring my Wagtail app to remove an IndexPage that only has a single item in it, and moving that item to be a child of the current IndexPage's parent.
basically moving from this:
Page--|
|--IndexPage--|
|--ChildPages (there's only ever 1 of these)
to this:
Page--|
|--ChildPage
I've made the changes to the models so that this structure is used for creating new content and fixed the relevant views to point to the ChildPage directly. But now I want to migrate the current data to the new structure and I'm not sure how to go about it... Ideally this would be done in a migration so that we would not have to do any of this manipulation by hand.
Is there a way to move these ChildPage's up the tree programmatically during a migration?
Unfortunately there's a hard limitation that (probably) rules out the possibility of doing page tree adjustments within migrations: tree operations such as inserting, moving and deleting pages are implemented as methods on the Page model, and within a migration you only have access to a 'dummy' version of that model, which only gives you access to the database fields and basic ORM methods, not those custom methods.
(You might be able to work around this by putting from wagtail.wagtailcore.models import Page in your migration and using that instead of the standard Page = apps.get_model("wagtailcore", "Page") approach, but I wouldn't recommend that - it's liable to break if the migration is run at a point in the migration sequence where the Page model is still being built up and doesn't match the 'real' state of the model.)
Instead, I'd suggest writing a Django management command to do the tree manipulation - within a management command it is safe to import the Page model from wagtailcore, as well as your specific page models. Page provides a method move(target, pos) which works as per the Treebeard API - the code for moving your child pages might look something like:
from myapp.models import IndexPage
# ...
for index_page in IndexPage.objects.all():
for child_page in index_page.get_children():
child_page.move(index_page, 'right')
index_page.delete()
Theoretically it should be possible to build a move() using the same sort of manipulations that Daniele Miele demonstrates in Django-treebeard and Wagtail page creation. It'd look something like this Python pseudocode:
def move(page, target):
# assuming pos='last_child' but other cases follow similarly,
# just with more bookkeeping
# first, cut it out of its old tree
page.parent.numchild -= 1
for sib in page.right_siblings: # i.e. those with a greater path
old = sib.path
new = sib.path[:-4] + (int(sib.path[-4:])-1):04
sib.path = new
for nib in sib.descendants:
nib.path = nib.path.replace_prefix(old, new)
# now, update itself
old_path = page.path
new_path = target.path + (target.numchild+1):04
page.path = new_path
old_url_path = page.url_path
new_url_path = target.url_path + page.url_path.last
page.url_path = new_url_path
old_depth = page.depth
new_depth = target.depth + 1
page.depth = new_depth
# and its descendants
depth_change = new_depth - old_depth
for descendant in page.descendants:
descendant.path = descendant.path.replace_prefix(old_path, new_path)
descendant.url_path = descendant.url_path.replace_prefix(old_path, new_path)
descendant.depth += depth_change
# finally, update its new parent
target.numchild += 1
The core concept that makes this manipulation simpler than it looks is: when a node gets reordered or moved, all its descendants need to be updated, but the only update they need is the exact same update their ancestor got. It's applied as a prefix replacement (if str) or a difference (if int), neither of which requires knowing anything about the descendant's exact value.
That said, I haven't tested it; it's complex enough to be easy to mess up; and there's no way of knowing if I updated every invariant that Wagtail cares about. So there's something to be said for the management command way as well.

Python w/QT Creator form - Possible to grab multiple values?

I'm surprised to not find a previous question about this, but I did give an honest try before posting.
I've created a ui with Qt Creator which contains quite a few QtWidgets of type QLineEdit, QTextEdit, and QCheckbox. I've used pyuic5 to convert to a .py file for use in a small python app. I've successfully got the form connected and working, but this is my first time using python with forms.
I'm searching to see if there is a built-in function or object that would allow me to pull the ObjectNames and Values of all widgets contained within the GUI form and store them in a dictionary with associated keys:values, because I need to send off the information for post-processing.
I guess something like this would work manually:
...
dict = []
dict['checkboxName1'] = self.checkboxName1.isChecked()
dict['checkboxName2'] = self.checkboxName2.isChecked()
dict['checkboxName3'] = self.checkboxName3.isChecked()
dict['checkboxName4'] = self.checkboxName4.isChecked()
dict['lineEditName1'] = self.lineEditName1.text()
... and on and on
But is there a way to grab all the objects and loop through them, even if each different type (i.e. checkboxes, lineedits, etc) needs to be done separately?
I hope I've explained that clearly.
Thank you.
Finally got it working. Couldn't find a python specific example anywhere, so through trial and error this worked perfectly. I'm including the entire working code of a .py file that can generate a list of all QCheckBox objectNames on a properly referenced form.
I named my form main_form.ui from within Qt Creator. I then converted it into a .py file with pyuic5
pyuic5 main_form.ui -o main_form.py
This is the contents of a sandbox.py file:
from PyQt5 import QtCore, QtGui, QtWidgets
import sys
import main_form
# the name of my Qt Creator .ui form converted to main_form.py with pyuic5
# pyuic5 original_form_name_in_creator.ui -o main_form.py
class MainApp(QtWidgets.QMainWindow, main_form.Ui_MainWindow):
def __init__(self):
super(self.__class__, self).__init__()
self.setupUi(self)
# Push button object on main_form named btn_test
self.btn_test.clicked.connect(self.runTest)
def runTest(self):
# I believe this creates a List of all QCheckBox objects on entire UI page
c = self.findChildren(QtWidgets.QCheckBox)
# This is just to show how to access objectName property as an example
for box in c:
print(box.objectName())
def main():
app = QtWidgets.QApplication(sys.argv) # A new instance of QApplication
form = MainApp() # We set the form to be our ExampleApp (design)
form.show() # Show the form
app.exec_() # and execute the app
if __name__ == '__main__': # if we're running file directly and not importing it
main() # run the main function
See QObject::findChildren()
In C++ the template argument would allow one to specify which type of widget to retrieve, e.g. to just retrieve the QLineEdit objects, but I don't know if or how that is mapped into Python.
Might need to retrieve all types and then switch handling while iterating over the resulting list.

How do I get Swashbuckle to have Swagger UI to group by Version?

I am playing with the new Azure API Apps (template in Visual Studio 2013 w/ the new SDK bits from 3/24/15) and I'd like have my Swagger UI group my calls by Version #. In my case, I'm currently versioning by URI (I realize REST purists will tell me not to do this - please don't try to "correct my error" here). For instance, I may have these calls:
http://example.com/api/Contacts <-- "latest"
http://example.com/api/1/Contacts
http://example.com/api/2/Contacts
http://example.com/api/Contacts{id} <-- "latest"
http://example.com/api/1/Contacts/{id}
http://example.com/api/2/Contacts/{id}
Functionally, this works great! (Yes, I know some of you will cringe. Sorry this hurts your feelings.) However, my problem is w/ Swagger UI organization. By default, Swagger UI groups these by the Controller Name (Contacts in this case). I see in the SwaggerConfig.cs file that I can change this:
// Each operation be assigned one or more tags which are then used by consumers for various reasons.
// For example, the swagger-ui groups operations according to the first tag of each operation.
// By default, this will be controller name but you can use the "GroupActionsBy" option to
// override with any value.
//
//c.GroupActionsBy(apiDesc => apiDesc.HttpMethod.ToString());
What I don't understand is how I can tweak this to group all of the "latest" together and then all of v1 together and then all of v2 together, etc.
How can I do this? If it absolutely must require that I add the word "latest" (or equiv) into the path in place of the version number, then I can do that but I'd prefer not have to do that.
I believe what you're looking to do is to uncomment a line a few lines below that one in SwaggerConfig.cs
c.OrderActionGroupsBy(new DescendingAlphabeticComparer());
except you'd change the name of the class to something like ApiVersionComparer() and then implement it as a new class:
public class ApiVersionComparer : IComparer<string>
{
public int Compare(string x, string y)
{
// Write whatever comparer you'd like to here.
// Yours would likely involve parsing the strings and having
// more complex logic than this....
return -(string.Compare(x, y));
}
}
If you've gotten far enough to ask this question, I'm sure I can leave the sort implementation for you. :-)