I'm looking for pointers to solutions for a problem that must have been solved multiple times (or pointers to the proper terms to search for):
How to get a reasonably quick/cheap directory snapshot/hash that can be used to determine whether something has changed compared to a previous state (which can be during a previous run of the code)?
Edit: I'm thinking here in the 1st place of added or deleted files/directories, but also of file changes that can be determined without reading the entire file.
Edit2: I'm looking for something cross-platform, and not a notification mechanism (sadly).
Thanks!
Related
I tried to find in the documentation and I did not find indication.
I read the win32 watch code (because I am familiar with the API and it seems it does not).
Tracking for file deletion is extremely convoluted on windows, because the API provides the file names "randomly" mangled and if the file is not there anymore to resolve the real name "long path" it is hard to say what was the original.
I have implemented such thing for my previous company, with preserving the current state of files and reconciling the list of files for every directory that changes were happening inside. It worked well.
My question is: did I miss it and watchman notifies for deleted files? And if it does not - is there plans for these notifications to be added in the near future.
The short answer: yes.
The longer answer: the more convoluted portions are not entirely windows specific.
The bit that handles the mangled names can be found here:
https://github.com/facebook/watchman/blob/master/root.c#L748
The bit that handles deletes is in two parts; the first handles things when we're checking exactly the file that we were notified about:
https://github.com/facebook/watchman/blob/master/root.c#L853
The second is a kind of "garbage collection" approach where we're looking at the containing directory and discover that the file has gone away:
https://github.com/facebook/watchman/blob/master/root.c#L1353
Not sure how to even search for this topic so I'm just going to ask. I have a couple places in my code where changes to one function have to be also made to a similar function. Basically the operations in the functions are very similar but the data being worked on is different.
I don't like it but for now that's how it is. My question is anyone knows of a way or a tool that will notify me if a certain function has changed. If I get the notification I know to confirm the changes were made to other function.
Please don't tell me to change my code. I know this isn't ideal and thought has been put into alternatives, but nothing elegant has popped up.
Thanks,
Gunner
Here is an idea I have. Maybe it will set the train of thought...
I can add a special comment in my code for a block that I want to be tracked. The comment will have opening and closing syntax:
my_code.php...
//track-start-1234
[CODE]
//track-end-1234
//track-start-5678
[CODE]
//track-end-5678
I will then have a script that I tell to check a list of files, my_code.php, for now. It will look for the special comment syntax. "1234" will represent a file that holds the last modified code for that function, so 1234.php. If the contents between the comments do not match the code stored in the file then an alert is fired. Also I might add a relationship to the other code so the alert says something like
Code 1234 has changed. Please make sure code 5678 has been updated as well.
I think I can add this script to GIT somehow so it either is part of the core git commands like "push" or maybe just another command I have to run as part of our deployment process. A second option might be adding it to phpstorm's code inspector or something like that.
When the code has been modified and everything is good I will just manually copy the new code into the comparison file. There will not be a ton of updates so this will not be too tedious.
For now this is probably the route I'll go, but any critiques or suggestions are definately welcome.
What I need
a fast/performant way to open any file under a large (git) repo (~9.8k files).
Context
I have tried various solutions, like Textmate.el and find-file-in-repository. I found these solutions via previous SO questions like this and this and through the LocateFilesAnywhere EmacsWiki.
While both solutions work wonderfully for small-to-mdeium repos, in this case they are practically unusable. When I start typing a filename, there's a delay of several seconds before I see any result. And changing any part of the search is very laggy too.
I think the main problem is that on typing any character, emacs/find-file-in-repository starts a shell command (git ls-files...). I really only need to do that when I have stopped typing.
Questions
is there a better library out there for this use-case?
if not, how can I introduce a delay into the command when I'm typing? i.e. while I'm in find-file-in-repository, I want the find-command to be invoked only when I stop typing (let's say a gap of 300ms).
Summary
After I received the three answers I tried them out (also answering my own question as none of the above solutions worked for me). I finally settled for helm-ls-git. Here's a comparison from my point-of-view:
Projectile
took around 30 minutes to index the repo. Since projectile is not aware of .gitignore, the actual number of files is more like 52k.
can be customized but something that just works (i.e. understands git) is preferable
may need to invalidate cache re-index time to time. That would be costly and frequent since new files are added everyday to the repo.
helm-cmd-t
looked good from the description and the source.
hard to install since it's not published in melpa/marmalade etc. More details in this issue I opened up.
GNU Global
Didn't try as it's likely to have the same problems as Projectile (git-unaware, needs it's own "index" that may need to be maintained time to time)
event-jr's answer however opened up some more options: I was unaware of helm till now. Looking at melpa for helm related plugins I found the following:
helm-git
This looked really promising
Was easy to install with package.el since it's in melpa
I also use and love magit - so this looked a good fit.
However, it kept failing with a magit-git-dir: symbol is void kind of error. Did not dive in too much but looks like it needs to be updated. Opened up an issue
helm-ls-git
As the readme says, this is magit independent.
Has been working wonderfully so far. Easy to install (melpa) and is fast.
I use GNU global for this. I have around 20K files in my project. You can run
M-x gtags-find-file and type first few characters. TAB will complete and show all the matching. You can type any characters which is part of the file name and press enter. Will show all the files that contains these characters.
I tried to use projectile for this. But it was way too slow for the 'project indexing'. It didn't complete the indexing even after 1.5 hours and I have to kill it!. Not sure some thing is wrong here. GNU global is much faster and finishes the entire tag creation within 15 min.
You can check out Projectile. It was basically created to provide something similar to C-p, but has a lot of extra project level features as well. First time project indexing will be fairly slow on such a big project, but afterwards Projectile will cache the project files (both on memory and on the hard drive) and subsequent projectile invocations should be nearly instantaneous.
Projectile also has a Helm plugin to display project files and buffers with Helm.
I use helm-cmd-t happily. It will cache the file list in memory. The cache controls are flexible enough for my needs.
I just answered your question about new repo address here:
https://stackoverflow.com/a/8025310/903943
It's https://github.com/lewang/helm-cmd-t
My question is very similar to this one: What is the optimal way to monitor changes in a directory with a kqueue()? but I'm not satisfied with the answer there.
I have a kqueue setup to be notified when a file is copied to my app's Documents directory. Of course the notification fires as soon as the copying begins, but I want to know when it's completed. Surely there's a better way than polling modification times?
Do you initiate the copying and have control over the filenames ?
Then the easiest solution could be to just copy to a temp file and then after that rename it to it's final name.that way you would know: if this file is there-it ist no longer written to.
For now, the answer seems to be 'no', so I've resorted to polling the modification time of the file(s) in question.
If someone else is able to come up with a better answer, I'd be glad to hear it (and will un-accept this one).
Does anyone know of a software or system for automatically tracking and logging exact line code changes into a log file?
For example, lets say I edit 7 files on Jan 16th, and I add and modify several lines of code in each file.
Is there a software or tool of any kind that would automagically know what edits I made, what was changed, and log the details in date order line-item file?
I am guessing something exists like this, but I cannot find out what I should be searching for. I looked at SVN and some other similar, but didn't get these capabilities from the descriptions I read.
There's nothing that would automagically know this. This sort of information is easy to come by as a side-effect of using virtually any version control system though. It sounds like you're not using any VCS, you should start doing so now. SVN is a good option to start with.