I'm just starting out on learning how to use Q and was wondering how do I create two directories inside the directory that I start q from?
If you are trying to make two directories in a q script, you can use the system command in q. Docs on this command are here
You could do something like:
q)system"mkdir test"
which will make the directory test in your current directory from which you started q. You can adjust the system command as needed.
From your comment it appears you don't want to create empty directories, but instead eventually serialize tables inside.
The answer is therefore simple: don't bother create the directories! Just write your tables, KDB will take care of the rest.
$ tree .
.
0 directories, 0 files
q)$[`;":dir1/foo"] set ([]i: 1 2 3)
`:dir1/foo
q)$[`;":dir2/bar"] set ([]j: "abc")
`:dir2/bar
$ tree .
.
├── dir1
│ └── foo
└── dir2
└── bar
2 directories, 2 files
Related
I have processed over 10 000 mp4 into gifs, they have their "category" in the titles from a batch I used earlier because there's around 1000 category ( so 1000 folders) and my software could only process one folder at a time, so i batch renamed them with their parent folder (so I could sort them later) and put them in a "mother" folder. They all have randomly generated letters + titles + category in their names. Neither the categories or actual file name have the same amount of letters (in case that's relevant)
the folder and file names always start with "reddit_sub_CATEGORY".
Categories always have all lowercase, no number or special chars.
files example (there are over 10 000 of them)
d:\motherfolder\reddit_sub_funnycatswithdog_983j331_cutecattries-to_eatdog.gif
d:\motherfolder\reddit_sub_funnydogs_fdsljdsd9s_kiotdawg_123.gif
d:\motherfolder\reddit_sub_randommemes_023093x23_uploadedtitle.gif
d:\motherfolder\reddit_sub_imoutofrandomnames_92382j38du8yu_socute.gif
d:\motherfolder\reddit_sub_ and over 9001 more
Folders (over 1000 of them)
d:\sortedfolders\reddit_sub_funnycatswithdog\
d:\sortedfolders\reddit_sub_funnydogs\
d:\sortedfolders\reddit_sub_randommemes\
d:\sortedfolders\reddit_sub_imoutofrandomnames\
d:\sortedfolders\reddit_sub_ and over 999 more
I tried all the scripts I could find, spend many hours trying to make them work with not avail.
I actually lost the work due to a bad batch I made accidentally but i do have the batch that Made this happen.. maybe it can be worked around to do the opposite.
Update - can't find it!!
It was something along the lines of
ECho off
rem Enter into Root Folder
cd /d %~dp0
cd d:\rip\mp4\mothermp4
rem Process each SUB folder
for /D %%u in (*) do (
rem Enter into the SUB folder
cd "%%u"
rem Process each Project
for %%p in (*) do (
rem Move all files one level up
move *.* ..
rem Go back one level up to root folder
)
cd ..
)
this one i found of this website and modified it to what I needed. but I can only find this not-working version. this one doesnt even rename yet
I expect folders to have ALL their appropriate files sorted into them.
(no copy if possible, its around 150gb!)
eg;
d:\sortedfolders\reddit_sub_funnycatswithdog\reddit_sub_funnycatswithdog_983j331_cutecattries-to_eatdog.gif
d:\sortedfolders\reddit_sub_funnydogs_fdsljdsd9s\reddit_sub_funnydogs_fdsljdsd9s_kiotdawg_123.gif
d:\sortedfolders\reddit_sub_randommemes\reddit_sub_randommemes_023093x23_uploadedtitle.gif
d:\sortedfolders\reddit_sub_imoutofrandomnames\reddit_sub_imoutofrandomnames_92382j38du8yu_socute.gif
updated request ---
Removing "reddit_sub_category_" at the same time in the file name
d:\sortedfolders\reddit_sub_funnycatswithdog\83j331_cutecattries-to_eatdog.gif
d:\sortedfolders\reddit_sub_funnydogs_fdsljdsd9s\fdsljdsd9s_kiotdawg_123.gif
d:\sortedfolders\reddit_sub_randommemes\023093x23_uploadedtitle.gif
d:\sortedfolders\reddit_sub_imoutofrandomnames\92382j38du8yu_socute.gif
Tree running before
> tree D:\ /F
D:\
├───motherfolder
│ reddit_sub_funnycatswithdog_983j331_cutecattries-to_eatdog.gif
│ reddit_sub_funnydogs_fdsljdsd9s_kiotdawg_123.gif
│ reddit_sub_imoutofrandomnames_92382j38du8yu_socute.gif
│ reddit_sub_randommemes_023093x23_uploadedtitle.gif
│
└───sortedfolders
├───reddit_sub_funnycatswithdog
├───reddit_sub_funnydogs
├───reddit_sub_imoutofrandomnames
└───reddit_sub_randommemes
This batch
:: Q:\Test\2019\01\25\SO_54372309.cmd
#Echo off
Set "FileBase=D:\motherfolder"
Set "FolderBase=D:\sortedfolders"
for /d %%A in (%FolderBase%\*) Do Move "%FileBase%\%%~nA*" "%%A\"
and after:
> tree /F D:\
D:\
├───motherfolder
└───sortedfolders
├───reddit_sub_funnycatswithdog
│ reddit_sub_funnycatswithdog_983j331_cutecattries-to_eatdog.gif
│
├───reddit_sub_funnydogs
│ reddit_sub_funnydogs_fdsljdsd9s_kiotdawg_123.gif
│
├───reddit_sub_imoutofrandomnames
│ reddit_sub_imoutofrandomnames_92382j38du8yu_socute.gif
│
└───reddit_sub_randommemes
reddit_sub_randommemes_023093x23_uploadedtitle.gif
On extfs, if there are only file-creations and no -deletions in a directory, I expect that find . -type f would list the files either in their chronological order of creation (or mtime), or if not, at least in their reverse chronological order... depending on how a directory's contents are traversed.
But that isn't the behavior I'm seeing.
The following code, eg, creates a fresh set of directories and files:
#!/bin/bash -u
for i in a/ a/{1,2,3,4,5} b/ b/{1,2,3,4,5}; do
if echo "$i" | egrep -q "/$"; then
echo "Creating dir $i"
mkdir -p "$i"
else
echo "Creating file $i"
touch "$i"
fi
sleep 0.500
done
Output of the above snippet:
Creating dir a/
Creating file a/1
Creating file a/2
Creating file a/3
Creating file a/4
Creating file a/5
Creating dir b/
Creating file b/1
Creating file b/2
Creating file b/3
Creating file b/4
Creating file b/5
However, find lists files in somewhat random order. For example, a/2 doesn't follows a/1, and b/2 doesn't follow b/1:
$ find . -type f
./a/1
./a/3
./a/4
./a/2 <----
./a/5
./b/1
./b/3
./b/4
./b/2 <----
./b/5
Any idea why this should happen?
My main problem is: I have a very large volume storing 100s of 1000s of files. I need to traverse these files and directories in the order of their creation/modification (mtime) and pipe each file to another process for further processing. But I don't necessarily want to first create a temporary list of this large set of files and then sort it based on mtime before piping it to my process.
find lists objects in the order that they are reported by the underlying filesystem implementation. You can tell ls to show you this "raw" order by passing it the -f option.
The order could be anything at all -- alphabetical, by mtime, by atime, by length of name, by permissions, or something completely different. The ordering can even vary from one listing to the next.
It's common for filesystems to report in an order that reflects the filesystem's strategy for allocating directory slots to files. If this is some sort of hash-based strategy based on filename then the order can appear nonsensical. This is what happens with widely-used Linux and BSD filesystem implementations. Since you mention extfs this is probably what causes the ordering you're seeing.
So, if you need the output from find to be ordered in a particular way then you'll have to create that order yourself. Maybe based on something like:
find . -type f -exec ls -ltr --time-style=+%s {} \; | sort -n -k6
I have a Makefile where I currently have two files that should be copied to different directories. Currently, I've tested
echo ${dirs} | xargs -n 1 cp ${sources}
So I understand that this will not work since it will try to copy both source files to one of the directory every time. But is there a way that I can execute the copy command for every source file and directory each?
Best regards,
Simon
I think it is possible to deduce what you want from what you wrote, but as others pointed out, you should be more clear, so we don't have to spend time deducing it.
Anyway, since you want to not copy all files to all directories, you must somehow tell Make where you want to copy which files. The easiest way is to list the full paths of the copies you want in a variable such as $(COPIES), and not just ${dirs}. In this answer I am going to assume the destination directories already exist.
.PHONY: all
all: $(COPIES)
PERCENT := %
.SECONDEXPANSION:
$(COPIES): %: $$(filter $$(PERCENT)/$$(notdir $$*), $(sources)) Makefile
cp $< $#
I have a bunch of directories named project1, project2, etc.
In those folders are a bunch of perl files (extension ".pl").
Basically, I want to just copy the contents of those .pl files into a new file, let's call it "everything.txt".
Can someone help me out with this? I really don't care which programming language it's done in, although I'd prefer something commandline. But perl, python, and Java would work too.
Edit: Also, there are some duplicate names, which shouldn't be a problem given I just want to write their contents out to a file, but just thought I'd let you know.
bash: cat project*/*.pl > everything.txt
In Unix-y systems:
find project1 project2 ... -name \*.pl -exec cat {} \; > everything.txt
To make, say, a proper .tar archive file that will let you recover the original file names and permissions:
tar cf everything.txt.tar $(find project1 project2 ... -name \*.pl)
(The $(...) syntax requires the bash shell).
I have two or more directories which contains an ample of files.
I want to take a diff between the two files(which will be of same name for sure) but exists in different directories.Please help on how can in do this in perl scripting.Thanks.
You don't need Perl to accomplish it.
Try:
diff -r -N folder1/ folder2/