I have a huge text codebase on Windows and I am facing the problem of searching arbitrary words across the whole project.
Seemingly, no IDE scores good performances when doing so. What I am looking for is a free editor that:
searches full text through the folder and subfolder in an acceptable time (ideally less than 10 seconds for 10M lines of code).
Almost surely, the previous feature entails in using an index. So I'd like such index to be automatically in sync with file changes. Or at least to have the option to rebuild it on demand incrementally (not a full rebuild)
shows the search results in context (with snippets of the surroundng code around, pretty much like Sublime does).
ideally, be able to search for regex patterns
...and even, to search for regex through multiline strings.
I do not need it to be code-aware, but if it were able to understand PL/SQL it would be my dream editor.
So far my first choice was Sublime, which is an incredibly responsive editor.
Sublime 3 shipped with an internal index, but unfortunately for me, indexes just the symbols, not the full text.
The built-in brute force search cannot use that index and takes around 40 mins to return the results.
I tried several Sublime extensions which plug into external indexing tools (CodeSearch, Platinum Searcher, Whoosh). But all failed to provide automated or incremental index rebuild, or when they did offer those features, the indexing itself took so much memory that the entire app became unresponsive forever, or the plugin host broke down.
I also tried similar approaches with Atom and Visual Studio Code, but I believe that the editors and their plugins weren't written for such an insanely large codebase as I have at hands.
The only editor that performed well was Eclipse with the InstaSearch plugin.
However the speed that I gain in the search is lost in ordinary usage: opening a file, scrolling through its content, etc (not to mention that the plugin uses Lucene's syntax, not regex).
(It feels like if one is not using Java related technologies, there is no real benefit in using Eclipse which really offsets the heaviness of operating with it)
Related
I know this is too strange question, but we have multiple authors of one document and some contributors use OpenOffice to edit document, originating and edited by majority in MS word. Document is quite complex with differently structured paragraphs and fonts, bullets, numbering, embedded pictures, references to comments under the line, copied/pasted sections pasted with source formatting instead of pure text etc., so generally "fragile" and maybe little bit exceeding expectations of OpenOffice authors for MS compatibility. Bottom line is about various formatting issues, glue-ing of some words (occasionally space is missing), page footer/header modified or completely disappeared etc. We are unable to control behaviour of contributors and editors to the extent I would like to have, so I am trying to findout whether is there a way how to force users to use exclusively MS word for particular docx and to prevent using anything else? (I am not on MS payroll, I personally moved couple of people around me with "standard" document writing needs to OpenOffice, but incompatibility in this case creates useless redaction work for us.)
Thanks for any hint.
whether is there a way how to force users to use exclusively MS word for particular docx and to prevent using anything else
To me, it sounds like a terrible idea to try to enforce this with a macro or similar (and it probably wouldn't work even if you tried). Instead, come up with a better workflow and communicate with anyone who may be involved so they know what to do.
First question, is the document under configuration control? For example, if a bad change is made, do you have a way of going back to a previous version? There are many different configuration management tools available, both free and commercial.
Next, I would strongly recommend making final changes with only one Office suite. Pick either LibreOffice (or Apache OpenOffice - is that what you mean by OpenOffice? The OpenOffice.org suite was forked several years ago) or MS Word to be the official editing tool, but not both.
If you pick MS Word, then people can still make preliminary changes to the document using LibreOffice. However, someone with MS Word will then need to use a Diff tool to see the changes and then use MS Word to incorporate those changes into the document. Or ideally, Track Changes would be turned on to make it easier to see what changes were made and who made them. Comments can also be added to explain why changes were made.
What is even better is to get people to send marked-up PDF files that contain their proposed changes. PDF files cannot be edited, which is good because it avoids the kinds of problems that led you to write this question, and also the formatting changes they made will not appear differently on another computer. However, this requires a certain amount of education so that everyone agrees to do it this way, and in my experience, that's not easy with a diverse group.
If you ever see that someone has made changes to the main document using LibreOffice, you or someone else needs to go back to the latest version not edited by LibreOffice and then use MS Word to incorporate all of the new changes.
At this point, if both suites have been used to edit the document, then I would probably start off with a new blank document and copy all of the text unformatted into it. This would require redoing all tables and other formatting. Otherwise, it's likely to be nearly impossible to get a clean document, and the underlying formatting may have no end to the number of problems that keep popping up.
I am using Eclipse 2019. It is very slow on Save Action. It is necessary for me to enable the Save Action to do code format and indent check. But it is really a pain to watch Eclipse busy and slow at that point.
Is there a way to improve this?
I am trying Intellij. It is much much better than Eclipse on the Save Action.
I really hope Eclipse has the same or similar performance.
EDIT:
I am using Eclipse IDE for Enterprise Java Developers, Version 2019-12.
I only installed two editor plugins: One for resource bundle file and one for bash file.
EDIT 2:
Sorry, I should say that the Save Action on big Java file is slow. We have many files are more than 2 thousand lines. Some of them are double, triple that size, or even more. We often see the dialog telling us that we can turn off the Save Action to shorten waiting time. I tested with Intellij. The same file, the same formatter and Save Action, there is no noticeable delay in Intellij but Eclipse needs more time (noticeable even that dialog does not show up) to finish the action. There is nothing to do with the change itself, change many places, or change one line is the same. My guess is that Intelliji may be tracing the changes during the process and do format and Save Action only on where is changed. But Eclipse may not tracing the changes during the process and do format and Save Action on the entire file at the end.
I have no source control and other things attached.
Were you saving with Ctrl-S?
I was having a similar (or the same problem) whenever I was Ctrl-C/V to copy/paste. It seems the underlying problem may have been that my Hyperlink key was "Ctrl", so that whenever I was about to Copy/Paste, it was trying to work out Hyperlink options - stalling the editor.
Changing the Hyperlink modifier key (General > Editors > Text Editors > Hyperlinking) to Alt and the slow copy/paste is no more.
I recently started using Sublime Text 3, mostly for HTML and CSS and still learning the tricks.
In Dreamweaver, if I change a linked file/folder name or move it to different folder (within the project folder), it asks if all the html file containing those links should be updated and if confirmed it does updated the all the files in the project folder automatically, even the unopened files are updated! It saves a lot of time and errors.
This is the only feature I missed in Notepad++ when I used it for few projects.
Now, I am wondering if the above feature is there in Sublime Text (by default or with the help of some plug ins)?
I will greatly appreciate any helpful suggestion.
This feature is not present in Sublime by default, nor in any plugins of which I am aware, and although it is theoretically possible to write such a plugin it would be quite computationally intensive to have the feature "live". I find it much easier to keep CSS, JS, and markup in defined directories, and not move them around after creating them :)
You have to keep in mind that Notepad++ and Sublime Text are text editors, not IDEs, and so don't have all of the features of giant programs like Dreamweaver that are hundreds of times their size. Their primary purpose is editing and otherwise manipulating text, and ST at least (I'm not terribly familiar with N++) has a nice plugin API for writing functions to assist with that (such as inserting the path to a file), and people have even been able to write much more complex plugins to do things like linting and code intelligence, but things like keeping an eye on large groups of files and changing them all in response to certain events, or completely refactoring significant amounts of code, just isn't what it's designed for.
I am developing a couple of packages for sublime text, and to avoid copy and pasting massive amounts of code I began to move my classes into separate files. I have been avoiding this so far, since, in my current workflow, changes to files that are not in the main plugin file won't get updated when saved and only go into effect when I restart sublime.
Is there a way to reload a package, including all it's files, without restarting Sublime Text?
You don't actually have to restart the editor. You will have to restructure your plugins though to take advantage of this. Essentially, you can load the plugin files from some top level file. As an example, take a look at Package Control. I also do it in PersistentRegexHighlight (though the package control solution is likely more robust (I did base it on that). Still not as good as simply saving a particular file, but better than restarting! In fact, you could probably tie into the on_post_save event to automatically save the top level file when you modify a child file.
I personally found the easiest solution was to install Package Reloader, and just put a new file in the top directory of my plugin named .build. Save your top-level plugin file and enjoy not having to restart.
Virtually no restructuring of code required.
From the unofficial docs:
Sublime Text will reload top-level Python modules from packages as they change (perhaps because you are editing a .py file). By contrast, Python subpackages won’t be reloaded automatically, and this can lead to confusion while you’re developing plugins. Generally speaking, it’s best to restart Sublime Text after you’ve made changes to plugin files, so all changes can take effect.
Unfortunately, plugins are not loaded into a scope visible from the console (Ctrl`), so you can't just reload() it. EDIT But, you can call reload() from within your top-level plugin file, as detailed in #skuroda's answer.
You'll have to make the decisions on when to break classes out into separate files vs. keeping them together in one monolithic collection. Having 50 files, each with only two or three function definitions is overkill in one direction, while having 20 classes each with 10 or 15 methods all in one file is going overboard in the other, so just do what feels best for the particular project. In my experience killing/restarting ST2 doesn't take too long in any of the supported operating systems (except on XP, for some reason...), so hopefully it's not too much of a delay on your workflow. One suggestion I'd give is to create a portable installation (if you're on Windows) with just the bare essentials in extra plugins if your startup time is too long.
Good luck!
What I need
a fast/performant way to open any file under a large (git) repo (~9.8k files).
Context
I have tried various solutions, like Textmate.el and find-file-in-repository. I found these solutions via previous SO questions like this and this and through the LocateFilesAnywhere EmacsWiki.
While both solutions work wonderfully for small-to-mdeium repos, in this case they are practically unusable. When I start typing a filename, there's a delay of several seconds before I see any result. And changing any part of the search is very laggy too.
I think the main problem is that on typing any character, emacs/find-file-in-repository starts a shell command (git ls-files...). I really only need to do that when I have stopped typing.
Questions
is there a better library out there for this use-case?
if not, how can I introduce a delay into the command when I'm typing? i.e. while I'm in find-file-in-repository, I want the find-command to be invoked only when I stop typing (let's say a gap of 300ms).
Summary
After I received the three answers I tried them out (also answering my own question as none of the above solutions worked for me). I finally settled for helm-ls-git. Here's a comparison from my point-of-view:
Projectile
took around 30 minutes to index the repo. Since projectile is not aware of .gitignore, the actual number of files is more like 52k.
can be customized but something that just works (i.e. understands git) is preferable
may need to invalidate cache re-index time to time. That would be costly and frequent since new files are added everyday to the repo.
helm-cmd-t
looked good from the description and the source.
hard to install since it's not published in melpa/marmalade etc. More details in this issue I opened up.
GNU Global
Didn't try as it's likely to have the same problems as Projectile (git-unaware, needs it's own "index" that may need to be maintained time to time)
event-jr's answer however opened up some more options: I was unaware of helm till now. Looking at melpa for helm related plugins I found the following:
helm-git
This looked really promising
Was easy to install with package.el since it's in melpa
I also use and love magit - so this looked a good fit.
However, it kept failing with a magit-git-dir: symbol is void kind of error. Did not dive in too much but looks like it needs to be updated. Opened up an issue
helm-ls-git
As the readme says, this is magit independent.
Has been working wonderfully so far. Easy to install (melpa) and is fast.
I use GNU global for this. I have around 20K files in my project. You can run
M-x gtags-find-file and type first few characters. TAB will complete and show all the matching. You can type any characters which is part of the file name and press enter. Will show all the files that contains these characters.
I tried to use projectile for this. But it was way too slow for the 'project indexing'. It didn't complete the indexing even after 1.5 hours and I have to kill it!. Not sure some thing is wrong here. GNU global is much faster and finishes the entire tag creation within 15 min.
You can check out Projectile. It was basically created to provide something similar to C-p, but has a lot of extra project level features as well. First time project indexing will be fairly slow on such a big project, but afterwards Projectile will cache the project files (both on memory and on the hard drive) and subsequent projectile invocations should be nearly instantaneous.
Projectile also has a Helm plugin to display project files and buffers with Helm.
I use helm-cmd-t happily. It will cache the file list in memory. The cache controls are flexible enough for my needs.
I just answered your question about new repo address here:
https://stackoverflow.com/a/8025310/903943
It's https://github.com/lewang/helm-cmd-t