I have an issue when I'm working with NestedStack in AWS CDK, the problem is when I'm using the cdk diff command its returns the diff output return to me.
But when I have a diff inside my NestedStack it's just a reference and i really don't know what it will change inside my NestedStack.
[~] AWS::CloudFormation::Stack IAMPolicyStack.NestedStack/IAMPolicyStack.NestedStackResource IAMPolicyStackNestedStackIAMPolicyStackNestedStackResource4B98A1D2
├─ [~] NestedTemplate
│ └─ [~] .Resources:
│ └─ [~] .CDKMetadata:
│ └─ [~] .Properties:
│ └─ [~] .Analytics:
│ ├─ [-] v2:deflate64:H4sIAAAAAAAA/zPSMzLWM1BMLC/WTU7J1s3JTNKr9kstLklNCS5JTM7WcU7LC0otzi8tSk4FsZ3z81IySzLz82p1AipLMvLz9I31DA30TBSzijMzdYtK80oyc1P1giA0AJZoScZcAAAA
│ └─ [+] v2:deflate64:H4sIAAAAAAAA/zPSMzLWM1BMLC/WTU7J1s3JTNKr9kstLklNCS5JTM7WcU7LC0otzi8tSk4FsZ3z81IySzLz82p1AipLMvLz9I31LPUMjRSzijMzdYtK80oyc1P1giA0ALbtmvJcAAAA
Hope someone out there has hit the same issue as me and has a solution about how to get the diff output from NestedStack.
Updated 30/05/2022
The featuer look like its not ready yet, but its coming in the feature based my ticket here - https://github.com/aws/aws-cdk/issues/20392
This commit adds a workaround where you can run
cdk deploy --no-execute
and then view the nested stack changes in the CloudFormation change set that gets created.
Edit: Looks like this has not been merged yet as of 1.163.1, here is the open issue.
Related
I am reading D-Bus API Design Guidelines:
Interface files for public API should be installed to $(datadir)/dbus-1/interfaces so that other services can load them.
OK... in order to understand how these interface files work, I open the file /usr/share/dbus-1/interfaces/org.gnome.Shell.Screenshot.xml (I'm using Ubuntu 20.04)
<node>
<interface name="org.gnome.Shell.Screenshot">
...
<method name="PickColor">
<arg type="a{sv}" direction="out" name="result"/>
</method>
Well, I see the interface name, I see the method name, but that is not enough. If I want to call this method using dbus-send
$ dbus-send --print-reply --dest=<service.name> </path/to/object> org.gnome.Shell.Screenshot.PickColor
I need <service.name> and <path/to/object>, which this xml-file does not provide.
The queston is: what is the purpose of this file if it does not provide full information about the service interface? Is it used by dbus-daemon or by any other programs? Are such files really needed for something? Should I make a conclusion that an xml-file, which does not provide a path to object in , is incomplete? Should such file also mention a service known name (org.gnome.Shell.Screenshot in this case)?
The org.freedesktop.DBus.Introspectable interface has one method:
org.freedesktop.DBus.Introspectable.Introspect (out STRING xml_data)
Object instances may implement Introspect which returns an XML description of the object. The introspection format is documented at:
https://dbus.freedesktop.org/doc/dbus-specification.html#introspection-format
Using busctl on the command line I can use list to get all the services, tree to get all the objects, and then intropspect to find the API. Using these three commands I can discover the D-Bus services on a system
$ busctl list
$ busctl tree <service>
$ busctl introspect <service> <object>
The service you mentioned in your question is on the Session/User bus so it would be:
$ busctl --user list | grep Screenshot
NAME PID PROCESS USER CONNECTION UNIT SESSION DESCRIPTION
---------------------------------------------------------------------------------------------------------------------------------
org.gnome.Screenshot - - - (activatable) - - -
org.gnome.Shell.Screenshot 1259 gnome-shell LinuxMachine1 :1.37 user#1000.service - -
$ busctl --user tree org.gnome.Shell.Screenshot
├─/StatusNotifierWatcher
└─/org
├─/org/freedesktop
│ ├─/org/freedesktop/Notifications
│ └─/org/freedesktop/portal
│ └─/org/freedesktop/portal/desktop
├─/org/gnome
│ ├─/org/gnome/Mutter
│ │ ├─/org/gnome/Mutter/DisplayConfig
│ │ └─/org/gnome/Mutter/IdleMonitor
│ │ └─/org/gnome/Mutter/IdleMonitor/Core
│ ├─/org/gnome/ScreenSaver
│ ├─/org/gnome/SessionManager
│ │ └─/org/gnome/SessionManager/EndSessionDialog
│ ├─/org/gnome/Shell
│ │ ├─/org/gnome/Shell/AudioDeviceSelection
│ │ ├─/org/gnome/Shell/Introspect
│ │ ├─/org/gnome/Shell/Screencast
│ │ ├─/org/gnome/Shell/Screenshot
│ │ └─/org/gnome/Shell/Wacom
│ └─/org/gnome/keyring
│ └─/org/gnome/keyring/Prompter
└─/org/gtk
├─/org/gtk/MountOperationHandler
└─/org/gtk/Notifications
$ busctl --user introspect org.gnome.Shell.Screenshot /org/gnome/Shell/Screenshot
NAME TYPE SIGNATURE RESULT/VALUE FLAGS
org.freedesktop.DBus.Introspectable interface - - -
.Introspect method - s -
org.freedesktop.DBus.Peer interface - - -
.GetMachineId method - s -
.Ping method - - -
org.freedesktop.DBus.Properties interface - - -
.Get method ss v -
.GetAll method s a{sv} -
.Set method ssv - -
.PropertiesChanged signal sa{sv}as - -
org.gnome.Shell.Screenshot interface - - -
.FlashArea method iiii - -
.PickColor method - a{sv} -
.Screenshot method bbs bs -
.ScreenshotArea method iiiibs bs -
.ScreenshotWindow method bbbs bs -
.SelectArea method - iiii -
And for completeness, running the same command as you did:
$ busctl --user call org.gnome.Shell.Screenshot /org/gnome/Shell/Screenshot org.gnome.Shell.Screenshot PickColor
a{sv} 1 "color" (ddd) 1 1 1
The answer is in the name: it’s a D-Bus interface file. It defines an interface, not an instantiation of that interface. There may be zero or more instantiations of an interface in D-Bus, and they may be dynamic. So their paths can’t be specified in an XML file.
If D-Bus was designed now, perhaps this XML file would have been structured or implemented differently. But this is what there is.
I have stumbled the unfortunate situation, having to be in a directory in which another directory is located:
C:\Test\[Folder with unknown name]\theFileINeed.txt
The structure mentioned above originates from a Zip-file from an external source. So i can not change the structure.
My goal is to navigate to the Directory with the unknown name, so it is my working directroy and I can execute further commands there. (Like Get-Childitem e.g.)
Is there a simple way to e.g. use the cd command to move into that directory?
I have fiddled around a bit with Resolve-Path but couldn't find a helpful solution.
Thanks in advance.
Consider this structure:
C:\TMP\TEST
├───unknowndir1
│ │ nonuniquefile.txt
│ │ uniquefile.txt
│ │
│ ├───nonuniquesubdir
│ └───uniquesubdir
└───unknowndir2
│ nonuniquefile.txt
│
└───nonuniquesubdir
You could do cd .\test\*\uniquesubdir but you can't cd .\test\*\nonuniquesubdir as you'll gen an error (...) path (...) resolved to multiple containers. The same error is even with cd .\test\*\uniquesubdir\.. as if it didn't even check for existence of uniquesubdir.
So if you want to enter unknown directory based of a file it contains, you'd have to do something like this: cd (Get-Item .\test\*\uniquefile.txt).DirectoryName.
It will fail if you use nonuniquefile.txt as it again resolves to two different directories. You could enter the first of these directories with cd (Get-Item .\test\*\nonuniquefile.txt).DirectoryName[0] if you don't care which of them you use.
Can someone explain how DVC stores differences on the directory level into DVC cache.
I understand that the DVC-files (.dvc) are metafiles to track data, models and reproduce pipeline stages. However, it is not clear for me how the process of creating branches, commiting them and switching back to a master file is exactly saved in differences.
Short version:
.dvc file contains info (md5) about JSON file inside cache that describes current state of directory.
When directory gets updated, there is new md5 in .dvc file and new JSON file is created with updated state of directory.
In git, you store the .dvc file, so that DVC know (basing on md5) where to look for information about directory.
Longer version:
Let me try to break particular steps of directory handling with DVC.
Lets assume we have some data directory you want to add under DVC control:
data
├── 1
└── 2
You are using dvc add data to make DVC track you directory. In result, DVC produces data.dvc file. As you noted this file contains metadata required to connect your git repository with your data storage. Inside this file (besides other things) you can see:
outs:
- md5: f437247ec66d73ba66b0ade0246fcb49.dir
path: data
The md5 part is used to store information about directory in DVC cache (.dvc/cache):
(dvc3.7) ➜ repo$ tree .dvc/cache
.dvc/cache
├── 26
│ └── ab0db90d72e28ad0ba1e22ee510510
├── b0
│ └── 26324c6904b2a9cb4b88d6d61c81d1
└── f4
└── 37247ec66d73ba66b0ade0246fcb49.dir
If you will open the file with .dir suffix, you will see that it contains description of current data state:
(dvc3.7) ➜ repo$ cat .dvc/cache/f4/37247ec66d73ba66b0ade0246fcb49.dir
[{"md5": "b026324c6904b2a9cb4b88d6d61c81d1", "relpath": "1"},
{"md5": "26ab0db90d72e28ad0ba1e22ee510510", "relpath": "2"}]
As you can see, particular files(1 and 2) are described by entries in this file
When you change your directory:
(dvc3.7) ➜ repo$ echo 3 >> data/3
(dvc3.7) ➜ repo$ dvc commit data.dvc
The content of data.dvc will be updated:
outs:
- md5: 12f4b7d54a32e58818e27fba28376fba.dir
path: data
And there is new file inside the cache:
├── 12
│ └── f4b7d54a32e58818e27fba28376fba.dir
...
(dvc3.7) ➜ repo$ cat .dvc/cache/12/f4b7d54a32e58818e27fba28376fba.dir
[{"md5": "b026324c6904b2a9cb4b88d6d61c81d1", "relpath": "1"},
{"md5": "26ab0db90d72e28ad0ba1e22ee510510", "relpath": "2"},
{"md5": "6d7fce9fee471194aa8b5b6e47267f03", "relpath": "3"}]
From perspecitve of git the only change is inside data.dvc.
(Assuming you did git commit after adding data with 1 and 2 inside):
diff --git a/data.dvc b/data.dvc
index 098aec5..88d1a90 100644
--- a/data.dvc
+++ b/data.dvc
## -1,6 +1,6 ##
-md5: a427c5bf8680fbf8d1951806b28b82fe
+md5: 1b674d61c195eea7a6b14f176c020b9c
outs:
-- md5: f437247ec66d73ba66b0ade0246fcb49.dir
+- md5: 12f4b7d54a32e58818e27fba28376fba.dir
path: data
cache: true
metric: false
NOTE: First md5 corresponds to md5 of this file, so it had to change with dir md5 change
I have the following directory structure:
root
├─ files
│ ├─ folder1
│ │ ├─ file1.js
│ | └─ file2.js
│ ├─ folder2
│ │ └─ file3.js
│ ├─ file4.js
| └─ file5.js
└─ config.js
How can I match every file inside of file (and subdirectories) except the files that are in folder1, in this case file3.js, file4.js and file5.js?
I know I could exclude folder1 with the following: files/!(folder1)/*.js, but this only matches file3.js.
Try **/files/{*.js,!(folder1)*/*.js}. You can test using globster.xyz
There is probably a more elegant way to do this as I am not too familiar with glob, but I think this will get what you are asking for.
import glob
exclude_pattern = ['folder1']
file_list = glob.glob('./files/**/*', recursive=True)
for pattern in exclude_pattern:
exclude_patternmatch = list(filter(lambda x: pattern in x, file_list))
for item in exclude_patternmatch:
file_list.remove(item)
print(file_list)
output:
['./files/file6.js', './files/file5.js', './files/folder2/file3.js', './files/folder2/file4.js']
Performing a select on Clickhouse, on a MergeTree table that is loaded from a KafkaEngine table via a Materialized View, a simple select shows output split in groups in the clickhouse-client:
:) select * from customersVisitors;
SELECT * FROM customersVisitors
┌────────day─┬─────────createdAt───┬──────────────────_id─┬───────────mSId─┬───────xId──┬─yId─┐
│ 2018-08-17 │ 2018-08-17 11:42:04 │ 8761310857292948227 │ DV-1811114459 │ 846817 │ 0 │
│ 2018-08-17 │ 2018-08-17 11:42:04 │ 11444873433837702032 │ DV-2164132903 │ 780066 │ 0 │
└────────────┴─────────────────────┴──────────────────────┴────────────────┴────────────┴─────┘
┌────────day─┬─────────createdAt───┬──────────────────_id─┬───────────────────mSId──┬────────xId─┬─yId─┐
│ 2018-08-17 │ 2018-08-17 10:25:11 │ 14403835623731794748 │ DV-07680633204819271839 │ 307597 │ 0 │
└────────────┴─────────────────────┴──────────────────────┴─────────────────────────┴────────────┴─────┘
3 rows in set. Elapsed: 0.013 sec.
Engine is ENGINE = MergeTree(day, (mSId, xId, day), 8192)
Why does the output appear splitted in two groups?
If I'm not mistaken, the output is split when the data came from different blocks, also often it leads to being processed in different threads. If you want to get rid of it, wrap your query in outer select
select * from (...)
MergeTree Engine is designed for faster WRITE and READ operations.
Fater writes are achieved by inserting data in parts and then the data is merged offline into a single part for faster reads.
you can see the data partition the following directory :
ls /var/lib/clickhouse/data/database_name/table_name
If you run the following query, you will find this that the data is now available in a single group and also a new partition is available at the above location :
optimize table MY_TABLE_NAME
Optimize table forces merging of partition, but in usual cases, you can just leave it on Click house .