I have 2 source files, they are different versions of the same thing. However, one has been through a different editor that made indent changes, so all the lines are showing up different in diff.
Is there a diff command or a filter I can use to diff with so that the output will only be lines that are different after ignoring the leading spaces/tabs?
diff has some options that can be useful to you:
-E, --ignore-tab-expansion
ignore changes due to tab expansion
-Z, --ignore-trailing-space
ignore white space at line end
-b, --ignore-space-change
ignore changes in the amount of white space
-w, --ignore-all-space
ignore all white space
-B, --ignore-blank-lines
ignore changes whose lines are all blank
So diff -w old new should ignore all spaces and thus report only substantially different lines.
diff -bB file[12]
-b, --ignore-space-change
ignore changes in the amount of white space
-B, --ignore-blank-lines
ignore changes whose lines are all blank
Please note that -w option will ignoring all whitespaces before diffing, so a line like this i s a line and this is a line in each file will compare as thisisaline and will not report differences.
Beside of -w option problem, even -b option has minor issues and that doesn't ignore whitespaces if come at the begging of a line
So you should use sed to remove those whitespaces occurred at start first then do `diff -bB.
diff -bB <(sed 's/^[ \t]*//' file1) <(sed 's/^[ \t]*//' file2)
If one is using tabs incorrectly, you
can fix that
expand bad_file
My open-source Linux tool 'dif' compares files while ignoring various differences including whitespace.
It has many other options for ignoring comments or timestamps, sorting the input files, doing search/replace, ignoring certain lines, etc.
After preprocessing the input files, it runs the Linux tools meld, gvimdiff, tkdiff, or kompare on these intermediate files.
Installation is not required, just download and run the 'dif' executable from https://github.com/koknat/dif
To condense any whitespace to a single space, use the -white option:
dif file1 file2 -white
To remove all whitespace (except for newlines), use the -nowhite option:
dif file1 file2 -nowhite
Related
I have a list of files .tex file that contain fragments in the tex that build ps pictures which can be slow to process.
There are multiple fragments across multiple files and the end delimiter is \end{pspicture}
% this is the beginning of the fragment
\begin{pspicture}(0,0)(23,5)
\rput{0}(0,3){\crdKs}
\rput(1,3){\crdtres}
\rput(5,3){\crdAh}
\rput(6,3){\crdKh}
\rput(7,3){\crdsixh}
\rput(8,3){\crdtreh}
\rput(12,3){\crdQd}
\rput(13,3){\crdeigd}
\rput(14,3){\crdsixd}
\rput(15,3){\crdfived}
\rput(16,3){\crdtwod}
\rput(20,3){\crdKc}
\rput(21,3){\crdfourc}
\end{pspicture}
I would like to extract the fragments.
I am not sure how to go about this? can awk do this or sed?
They seem to work line by line, rather than work on the whole fragment.
I am not really looking for a solution just a good candidate tool.
sed -En '/^\\begin\{pspicture\}.*$/,/^\\end\{pspicture\}.*$/p' file
Utilising sed with -E for regular expressions.
Use //,// to determine start and ending regular expressions and print all lines from the start to the end.
I recently started experimenting with the clang-tidy tool of llvm. Now I am trying to suppress false warnings from third party library code. For this I want to use the command line options
-header-filter=<string> or -line-filter=<string>
but so far without success. So for people with limited time I will put the question here at the beginning and explain later what I already tried.
Question
What option do I need to give to the clang-tidy tool to suppress a warning from a certain line and file?
if this is not possible
What option works to suppress warnings from external header files?
What I did so far
My original call to clang-tidy looks like this
clang-tidy-3.8 -checks=-*,clang-analyzer-*,-clang-analyzer-alpha* -p Generated/LinuxMakeClangNoPCH Sources/CodeAssistant/ModuleListsFileManipulator_fixtures.cpp
and the first line of the yielded warning that I want to suppress looks like this
.../gmock/gmock-spec-builders.h:1272:5: warning: Use of memory after it is freed [clang-analyzer-cplusplus.NewDelete]
return function_mocker_->AddNewExpectation(
The gmock people told me that this is a false positive so I want to suppress it. First I tried to use the -line-filter=<string> option. The documentation says:
-line-filter=<string> - List of files with line ranges to filter the
warnings. Can be used together with
-header-filter. The format of the list is a JSON
array of objects:
[
{"name":"file1.cpp","lines":[[1,3],[5,7]]},
{"name":"file2.h"}
]
I assumed that warnings in the given lines are filtered out. But the doc doesent say if they are filterd out or in.
After some fiddeling arround I created a .json file with the content
[
{"name":"gmock-spec-builders.h","lines":[[1272,1272]]}
]
and modified the command line to
clang-tidy-3.8 -checks=-*,clang-analyzer-*,-clang-analyzer-alpha* -p Generated/LinuxMakeClangNoPCH -line-filter="$(< Sources/CodeAssistant/CodeAssistant_ClangTidySuppressions.json)" Sources/CodeAssistant/ModuleListsFileManipulator_fixtures.cpp
which writes the content of the file into the argument. This suppresses the warning, but not only this warning, but all warnings from the ModuleListsFileManipulator_fixtures.cpp file. I tried more stuff but I could not make it work.
So I tried the -header-filter=<string> option. Here the documentation states that one has to give a regular expression that matches all the header files from which diagnostics shall be displayed. Ok, I thought, lets use a regualar expression that matches everything that is in the same folder as the analyzed .cpp file. I can live with that although it may remove warnings that result from me using external headers wrong.
Here I was not sure if the regular expression must match the full (absolute) filename or only a part of the filename. I tried
-header-filter=.*\/CodeAssistant\/.*.h
which matches all absolute header filenames in the CodeAssistant folder but it did not suppress the warnings from the gmock-spec-builders.h file.
So preferably I would like to suppress each warning individually so I can determine for each if it is a real problem or not, but if this is not possible I could also live with suppressing warnings from entire external headers.
Thank you for your time.
I solved the problem by adding // NOLINT to line 1790 of gmock-spec-builders.h
Here is the diff:
--- gmock-spec-builders.orig.h 2016-09-17 09:46:48.527313088 +0200
+++ gmock-spec-builders.h 2016-09-17 09:46:58.958353697 +0200
## -1787,7 +1787,7 ##
#define ON_CALL(obj, call) GMOCK_ON_CALL_IMPL_(obj, call)
#define GMOCK_EXPECT_CALL_IMPL_(obj, call) \
- ((obj).gmock_##call).InternalExpectedAt(__FILE__, __LINE__, #obj, #call)
+ ((obj).gmock_##call).InternalExpectedAt(__FILE__, __LINE__, #obj, #call) // NOLINT
#define EXPECT_CALL(obj, call) GMOCK_EXPECT_CALL_IMPL_(obj, call)
#endif // GMOCK_INCLUDE_GMOCK_GMOCK_SPEC_BUILDERS_H_
It would be nice to either upstream this patch (I see other NOLINT in the code) or post a bug report with the clang-tidy folks.
I have found another non-invasive (without adding // NOLINT to a third-party library) way to suppress warnings. For example, the current version of Google Test fails some cppcoreguidelines-* checks. The following code allows you to validate the current diff excluding lines that contain gtest's macros:
git diff -U3 | sed '
s/^+\( *TEST(\)/ \1/;
s/^+\( *EXPECT_[A-Z]*(\)/ \1/;
s/^+\( *ASSERT_[A-Z]*(\)/ \1/;
' | recountdiff | interdiff -U0 /dev/null /dev/stdin | clang-tidy-diff.py -p1 -path build
It assumes that file build/compile_commands.json is generated before and clang-tidy-diff.py is available from your environment. recountdiff and interdiff from patchutils are the standard tools for manipulating patches.
The script works as follows:
git diff -U3 generates a patch with 3 context lines.
sed ... removes prefix + from the undesired lines, i.e. transform them to the context.
recountdiff correct offsets (in first ranges) in the chunk headers.
interdiff -U0 /dev/null /dev/stdin just removes all context lines from a patch. As a result, it splits the initial hunks.
clang-tidy-diff.py reads only second ranges from chunk headers and passes them to clang-tidy via -line-filter option.
UPD: It's important to provide interdiff with a sufficient number of context lines, otherwise it may produce some artifacts in the result. See the citation from man interdiff:
For best results, the diffs must have at least three lines of context.
Particularly, I have found that git diff -U0 | ... | interdiff generates some spurious literals $!otj after splitting chunks.
Use -isystem instead of -I to set your system and 3rd party include paths. -I should only be used to include code that is part of the project being built.
This is the only thing required to make clang-tidy ignore all errors in external code. All the other answers (at the point of writing) are just poor workarounds for something that is perfectly solved with -isystem.
If you use a build system like CMake or Meson it will automatically set -I and -isystem correctly for you.
-isystem is also the mechanism that is used for telling compilers, at least GCC and Clang, what's not your code. If you start to use -isystem you can also enable more compiler warnings without getting "false positives" from external code.
I couldn't achive what I wanted with the commmand line options, so I will use the // NOLINT comments in the cpp files which was proposed by the accepted answer.
I will also try to push the fix to googletest.
I found out that lines in the -line-filter options are filtered in.
But giving concrete lines is no real solution for my problem anyways.
I rather need a suppression mechanism like it is implemented in Valgrind.
I want to make diff between two files which contains lines beginning with "line_$NR". I want to make diff between the files making abstraction of the presence of "lines_$NR" but when the differences are printed I want lines_$NR to be displayed.
It is possible to do that?
Thanks.
I believe in this case, you have to preprocess your iput files to remove /^line_[0-9]*/, diff the resulting files, then recombine the diff output with the removed words according to line numbers in diff output.
Python's difflib should be very handy here, or same from perl. If you want to stick to shell, I suppose you could get by with awk.
If you don't need exact output, perhaps you can use diff's --line-format=... directive to inject actual line number in a diff, rather than the word you removed in preprocessing step.
This StackOverflow answer has an image of KDiff3 highlighting intra-line differences. Does someone know of a tool which can show the same (ex, via color) on the command line?
Another way to think of this is wanting to diff each difference in a patch file.
I don't know if this is sufficiently command line for your purpose, but vimdiff can do this (even does colour). See for example the image in this related question.
I tried all the tools I found: wdiff, dwdiff, kdiff3, vimdiff to show the difference between two long and slightly different lines. My favourite is diff-highlight (part of git contrib)
it supports diff format - great advantage over tools requiring two files like (dwdiff), e.g. if you need to visualize the output of unit tests
it highlights with black+white or with color if you connect it to colordiff
highlights characterwise - helpful for comparing long lines without spaces (better than wdiff)
Installation
On Ubuntu, you probably already have it as part of git contrib (installed within the git deb package).
Copy or link it into your ~/bin folder from /usr/share/doc/git/contrib/diff-highlight/diff-highlight
Usage example
cat tmp.diff | diff-highlight | colordiff
Result:
Another intuitive way to see all word-sized differences (though not side-by-side) is to use wdiff together with colordiff (you might need to install both). An example of this would be:
wdiff -n {file-A} {file-A} | colordiff
You can optionally pipe this into less -R to scroll through the output (-R is used to show the colors in less).
I had a similar problem and wanted to avoid using vimdiff. I found dwdiff (which is available in Debian) to have several advantages over wdiff.
The most useful feature of dwdiff is that you can customise the delimiters with -d [CHARS], so it's useful for comparing all kinds of output. It also has color built in with the -c flag.
You might be able to use colordiff for this.
In their man page:
Any options passed to colordiff are
passed through to diff except for the
colordiff-specific option 'difftype',
e.g.
colordiff --difftype=debdiff file1
file2
Valid values for 'difftype' are: diff,
diffc, diffu, diffy, wdiff, debdiff;
these correspond to plain diffs,
context diffs, unified diffs,
side-by-side diffs, wdiff output and
debdiff output respectively. Use these
overrides when colordiff is not able
to determine the diff-type
automatically.
I haven't tested it, but the side-by-side output (as produced by diff -y file1 file2) might give you the equivalent of in-line differences.
ccdiff is a convenient dedicated tool for the task. Here is what an example may look like with it:
By default, it highlights the differences in color, but it can be used on a console without color support too.
The package is included in the main repository of Debian:
ccdiff is a colored diff that also colors inside changed lines.
All command-line tools that show the difference between two files fall short in showing minor changes visuably useful. ccdiff tries to give the look and feel of diff --color or colordiff, but extending the display of colored output from colored deleted and added lines to colors for deleted and addedd characters within the changed lines.
I tried running 'diff' against two source directories get a patch file with a 'diff' between the two directories.
diff -rupN flyingsaucer-R8pre2_b/ flyingsaucer-R8pre2/ > a.patch
The command above does not seem to work, it generates a diff of everything and I get a 13 MB file, when in reality, it should be a couple of changes.
Should work with any recent version of gnu diff (tested here with gnu diff 2.8.1.)
You might want to add -b (and perhaps -B) to ignore difference in white space which perhaps generate large patch files unnecessarily.
I don't see any reason why it wouldn't work. Try adding "wb" to the argument list to ignore whitespace changes. Are you sure you got the trailing slashes the same on both sides?