Usually I pipe my log through a lot of greps to remove the "noise" before i open it in an editor.
I think it should be possible to do this filtering inside an editor (Especially Emacs)
Is this what chainsaw is doing? For log4j format only or more general?
(It is the only logfile viewer tool I can find)
How do you guys do it?
(I think UNIX grep syntax would be easiest for me)
Chainsaw does support both positive and negative filter matching. You can define positive and negative matches based on the logger tree (right click on nodes for the options), and you can define positive-match expressions in the 'refine focus' field, and negative-match expressions using the 'ignore' option below the logger tree. There is a tutorial available from the help menu which describes the expression syntax.
Chainsaw has had a lot of new features added since the last official release. The developer snapshot (including a reworked configuration screen) is available here:
http://people.apache.org/~sdeboy
Chainsaw doesn't just work with log4j. There are 'receivers' available that make it work with log4net, java.util.logging, log4php and others.
You can also process any regularly formatted text file using a VFSLogFilePatternReceiver (use the 'process a log file' option to configure Chainsaw to define one). There are some pre-defined log formats in the configuration dialog that act as example formats - tweak one to match your format. The JavaDoc provides more information: http://logging.apache.org/chainsaw/apidocs/org/apache/log4j/chainsaw/vfs/VFSLogFilePatternReceiver.html
If the greps are the same, you can simply code a script to do all that work for you (e.g. vimscript for vi). That way, you don't have to repeat all the tasks while leaving the logs intact for further investigation.
You're right on chainsaw and log4j - it's a logging viewer with different capabilities, e.g. a filter mechanism. However, I am unsure, if you are able to have multiple filters activated simultaneously.
Yes, you should try Chainsaw first.. it does support various ways of getting your logs.
Necroposting: I created a mode for Emacs that is specifically targeted at Log4j-like logs, but supports many more formats, especially if you customize it for yourself.
Features:
Colorization (just customize faces if that's too much)
Interactive filtering in the same buffer:
by level
by logger name
by thread
by message
easy narrowing
edit all currently set filters at once
A few log-specific movement commands
Copy (with M-w) only visible text, goes well with filtering (this is customizable)
Related
When I like to create a new variable of type ImageObserver and the line starts like this:
IO
and I press Ctrl+Space , I get a couple of wrong suggestions! Suggestions for abbrevs that do not fully match!
Eclipse's content assist suggestions are:
IOException (no! full abbrev is IOE, not IO!)
IOError (no! full abbrev is IOE too, but not IO!)
IndexOutOfBoundsException (no! full abbrev is IOOBE, not IO!)
ImageObserver (yes! full abbrev is IO)
ImageOutputStream (no! full abbrev is IOS, not IO!)
How to get rid of all the other non-full-matching-abbrevs?
Keep in mind that the completion proposals fall into a bunch of categories (ie, lots of different sources of potential matches). Your example text, IO, can logically be interpreted as either a camel-case abbreviation (such as ImageObserver) or as the start of a class name (such as IOException), or even as a camel-case-starts-with match (such as ImageOutputStream, whose abbreviation starts with IO).
I don't know of a tool that's clairvoyant enough to know which of those you want given such a small input. Eclipse, by default, uses a "relevance" ordering for completion proposals and it weighs "starts with" matches ahead of camel-case matches (rightly so, in my opinion). There's no publicly exposed way to change that weighting algorithm, that I know of.
If these are classes or packages that you simply never want to see in Content-Assist, you can define Type Filters to exclude them. In Preferences, go to Java > Appearance > Type Filters to do so. But be aware that Type Filters apply to the entire workspace and in multiple places in the UI, not just Content Assist completion proposals.
You really want IOException to be excluded on auto-complete when IO is typed even though IOException starts with IO?
If so, this sounds like a feature request, but unless you are ready to contribute I can't see it getting added anytime soon.
Contributing
If you are interested in contributing, you need to write a Java Completion Proposal Computer. Once you do that, you can enable only your proposer in the preferences (Java / Editor / Content Assist / Advanced ).
You can even bind your special completer to its own Keyboard Short-cut so you can get "full abbrev" completions only.
Here is a screenshot of my binding of Template Proposals to Ctrl+4 so with a single key combination I can get quickly get them all.
Code Recommenders
I also recommend reading about Code Recommenders, the project adds significantly more sophisticated code completion to Eclipse.
I have raised a bug with Eclipse to see if sorting perfect Camel Case matches could be on the cards.
I like rgrep a lot. The interface for finding and jumping between matches is great.
But I also want the ability to do interactive and non-interactive replacements
from the *grep* results window.
I mean the information for replacing is already laid out perfectly:
the thing to be replaced is nicely highlighted
some context is given (although it wouldn't hurt to bind
+ to increase the number of context lines)
the files and positions are here as well
But there's no interface for replacement (I checked with f1 m).
Did I miss it or it's not available? Or is it available is some package?
I think you are looking for wgrep. It uses wdired approach, and it is available in the usual package repositories.
Is there any diff tool for Lotus Notes which allows to compare scripts, design elements and documents?
I see this is an old question, and most of the other answers are a little outdated now, so I thought I would add some hopefully valuable information for those who should stumble upon this now.
In Domino Designer, open either the Navigator or Package Explorer (Window menu -> Show Eclipse Views). Here you can expand databases/templates to see the design elements they contain. Select two or three elements (CTRL-click). They can be in different databases or the same database. Right click on one of the elements and select Compare with -> Each other.
You can also compare two databases element by element by selecting two databases/templates, right-clicking and selecting Compare with -> Each other. You will then get the differences between the two databases listed. You will be able to see which elements differ between the two databases, and which elements exist in one database but not the other. By double-clicking on a differing element, you will open a diff tool which lets you see differences line by line, and you can easily copy changes from left to right or right to left.
There is a tool from TeamStudio called Delta: http://www.teamstudio.com/products/delta.html
If all else fails (and by "all else" I mean the often ridiculous corporate procurement system) you can always do a an export to DXL (or a Design Synopsis for code alone) and use any decent text editor with a diff function. It's not TeamStudio Delta, but it will get you where you want to go.
There is a free tool from OpenNTF which does document comparisons:
http://www.openntf.org/Projects/pmt.nsf/ProjectLookup/Compare%20Notes%20Documents
Ytria also has a product which, among other things, will compare data documents (I don't believe it compares design elements).
http://www.ytria.com/website.nsf/WebPageRequest/Solutions_scanEZ_specen
And, I believe Martin Scott (http://www.martinscott.com) has a similar product which compares documents.
DDE (Domino Designer on Eclipse) let's you compare design elements natively. Same way as the search. It's pretty efficient (faster than a DXL exportation) and it's free.
I had a discussion on my blog a little while back about this:
http://rosshawkins.net/archive/2009/12/24/notesdomino-refactoringanalysis-tools.aspx
However what I've ended up doing in the past is exporting the design to the filesystem and using standard text tools (WinMerge and SublimeText for me personally) to do what I need.
Being able to do the raw dump is something that was added with the Eclipse based designer, and isn't overly obvious, but you can read more about it here:
rosshawkins.net/archive/2010/01/20/searching-the-contents-of-notesdomino-design-elements.aspx
(link mangled as my rep is too low to post 2 links in one post yet!)
Teamstudio Delta is really nice. However it might kill you with too many details. As Ross pointed out the Domino Designer 8.5 can use the Diff tool inherited from Eclipse. You also could head over to http://www.openntf.org and look for the DXLMagic project. It can generate a report that shows differences (including code) between 2 databases (typically a template and a variation of it). It is not as complete as Delta, but shows the essentials. It's free and source is included (Disclaimer: I wrote it).
This is what I do. I run a design synopsis of the database using the Notes Designer. Dump the file to a text file. You can actually split the synopsis out to different objects like Agents, Forms, Views, etc. Then you can run UNIX/Linux/Mac Unix commands to compare the elements. By doing this operation you find out what code is active, and have a complete documented source code. You do a lot of csplit and a few sed commands.
Version 12.0.1 has such a tool as part of the server. Look for comparedbs.ntf and designsynopsis.ntf on the Domino server.
I have a lot of HTML files (10,000's and GBs worth) scraped from a server and I want to check to make sure the server produces the same results after some modifications but ignore kinds of differences that don't matter, e.g. whitespace, missing newlines, timestamps, small changes in some kinds of number, etc.
Does anyone know of a tool for doing this? I'd really rather not do more filtering than I have to.
(Oh and it needs to run under linux)
You might consider using a clone detector such as our CloneDR. This tool parses large sets of computer program (HTML is special case) files, builds abstract syntax trees representing the essential structure of each files, and compares programs for similarity.
Because it is comparing essential program structure, it ignores inessential differences such as comments and whitespace, and deterimines that two code segments are either identical or one can be obtained from the other by substituting other blocks of code. The latter allows the recognition of code that has been modified in various ways. You can see samples of clone detection runs on a variety of computer languages at the web site.
In your case, what you would be looking for are files in system A which are essentially clones (exact or near misses) of files in system B. As a general rule, if a file a is a variant of file b (e.g., with a few changes) the CloneDr will report it as a clone and show the exact differences.
At the scale of 20,000 files, I can see why you want a tool, and I can see why you want near-miss matches rather than exact matches.
Doesn't run under Linux, but I assume your problem is hard to enough to solve so that isn't what you are optimizing.
I use winmerge alot in windows and from what i can see some people enjoy meld in linux, so perhaps that could do the trick for you
http://meld.sourceforge.net/
Other examples i saw from a quick googling was Kompare,xxdiff.sourceforge.net, and kdiff3.sourceforge.net
(could only post 1 link so wrote the adresses to xxdiff and kdiff3 as text)
Beyond Compare is purchased software that is actually worth the money (I never thought I'd hear myself typing that!). It is GUI based but handles thousands of files very well. It will allow you to specify unimportant changes with regular expressions as well as whitespace (beginning, middle and end of line). The feature set is very extensive, check out a trial download.
I do not work for this company, I just use Beyond Compare every day at work and enjoy it every time!
What patterns contribute or detract from the usability of a CLI interface?
As an example consider the CLI for ClearCase. The CLI is very comprehensive (+1) but it is has several glaring opportunities. Recently, I wanted to force the files to lower case into ClearCase using clearfsimport. Unfortunately I wound up on the documentation for its cousin clearimport. It may seem slight but it cost me more hours than I care to admit. The variation in the middle got me.
Why provide such nearly identical functionality with such nearly identical names? There are many better options in my opinion
clearimport -fs
fsclearimport
clear_fs_import
clearimport_fs
Anything would be better than what they went with. The code I am working on IS a CLI and this experience made me look at my own choices. I think I have all the basics covered (standard help, long-form vs short-form, short meaningful names, providing examples, eliminate ambiguity, accurately handling spaces within quotes, etc).
There is some literature on this subject.
Perhaps a bad CLI is no different than a bad API. CLI are type of an API in some sense. The goals are naturally common:: flexibility, readability, and completeness. Several factors differentiate CLI from a typical API. One is that CLI needs to support scriptability (participate many times perhaps in a series of pipes). Another is that autocompletion and namespaces don't exist in the same way. You don't always have a nice colorful GUI doing stuff for you. CLIs must document themselves externally to customer directly. And finally the audience of a CLI is vastly different than the standard API. I appreciate any insight you may have.
I like the subcommand pattern, which I'm most familiar with as its implemented in the command-line Subversion client.
svn [subcommand] [options] [files]
Without the subcommands, subversion would have waaaaay too many different options for me to remember them effectively, and the help system would be a pain to slog through.
But, if I don't remember how any particular subcommand works, I can just type:
svn help [subcommand]
...and it shows me only the relevant portions of the help documentation.
As noted above, this format:
[master verb] [subverb] [optionally, noun] [options]
is good in terms of remembering what commands are available. cvs, svn, Perforce, git, all adhere to this. It improves discoverability of commands, a major CLI problem. One wrinkle that occurs here is options for the master-verb vs. options for the subverb. I.e.,
cvs -d dir command bar
is different than
cvs command -d dir bar
This was a confusing situation in cvs, which svn "fixed" by allowing options specified in any order. Your own solution may vary; if you have a very good reason to pass options to the master verb, okay, just be aware of the overhead.
Looking to API usability is a good idea too, but beware that there is no real typing in CLI commands, and there is a lot of richness in what CLI commands 'return', since you've got both a return code and an output to work with. In the unixy/streams world, the output is usually much more important than the return code. Getting the format of your output right is crucial. Also, while tempting, I've found that sending different things to stdout vs. stderr is not always useful; it confuses novice and even intermediate users (because they both get dumped to console in most cases), and rarely is useful advanced users. So unless there's a real need for it I avoid it; it's too easy for (e.g.) someone to get very confused about why the output of a command was '' in an error condition just because the programmer nicely dumped the errors to stderr.
Another issue in design is the "what next" problem. In a GUI, the next steps for the user are spelled out by the available buttons, menus, etc. In a CLI, the user can literally type any command next, and pipe any command to any other. (Or try, at least.) I design my commands to give hints (either in the help or the output) as to what potential next steps might be in a typical workflow.
Another good pattern is allowing user customization of the output. While it is possible for users to use cut, sort, etc. to tailor the output, being able to specify a format string magnifies the utility of a command. The example I cite here is top, which lets you tell it which columns you want.