diff for lazy developers or how to compare incomparable

In the old days, on Windows, for comparing oracle schemes (read databases), I was satisfied with the compiler integrated in Quest Software TOAD. He was not bad, and he coped with his task. But reseeding in linux, I was disappointed. None of the [and so few] tools provide even half of their former comfort. Namely, an elementary comparison and merging of two files (DDL, SQL, any source is not the essence) of several thousand lines each, where each is formatted in its own way and the real changes are nothing, it becomes a natural torture.



Realizing the futility of trying to find the necessary functionality in one tool, I decided to look for crutches in the form of third-party tools. Namely, independent comparisons and merging tools ([visual] compare and merging tools). After going through almost all the popular tools , it turned out that very few (read almost none) were able to skip the end of line characters in comparing two text files.

For clarity, I will explain by example. Let's say we have 2 files, typically old and new.

Old (ora1.sql):

declare
 nfunctionresult NUMBER(9);
begin

 nFunctionResult :=
  orauser.pack_util.FGetNeededValue (
   in_nSomeParam => somevalue
  );

 dbms_output.put_line('Function result: '||
            nFunctionResult);
exception
 when SOME_EXCEPTION THEN
  orauser.pack_util.PReportError('SOME_EXCEPTION');
 when OTHERS THEN
  orauser.pack_util.PReportError('OTHER exception');
end;

New (ora2.sql):

declare
 nFunctionResult NUMBER(9);
begin
 nFunctionResult := orauser.pack_util.FGetNeededValue (in_nSomeParam => somevalue);
 dbms_output.put_line('Function result: '||nFunctionResult);
exception
 when SOME_EXCEPTION THEN
  orauser.pack_util.PReportError('SOME_EXCEPTION');
 when OTHERS THEN
  orauser.pack_util.PReportError('OTHER');
end;



As you can see with an inquiring human eye, the essence of the differences between the ora1.sql and ora2.sql files is only one word and different formatting, but their diff is completely unreadable. Here is the classic result of comparing them:

[bugman@localhost 1]$ diff ora1.sql ora2.sql
2c2
< nfunctionresult NUMBER(9);
---
> nFunctionResult NUMBER(9);
4,11c4,5
<
< nFunctionResult :=
< orauser.pack_util.FGetNeededValue (
< in_nSomeParam => somevalue
< );
<
< dbms_output.put_line('Function result: '||
< nFunctionResult);
---
> nFunctionResult := orauser.pack_util.FGetNeededValue (in_nSomeParam => somevalue);
> dbms_output.put_line('Function result: '||nFunctionResult);
16c10
< orauser.pack_util.PReportError('OTHER exception');
---
> orauser.pack_util.PReportError('OTHER');


The "put on all spaces" key -w does not improve the situation:

[bugman@localhost 1]$ diff -wi ora1.sql ora2.sql
4,11c4,5
<
< nFunctionResult :=
< orauser.pack_util.FGetNeededValue (
< in_nSomeParam => somevalue
< );
<
< dbms_output.put_line('Function result: '||
< nFunctionResult);
---
> nFunctionResult := orauser.pack_util.FGetNeededValue (in_nSomeParam => somevalue);
> dbms_output.put_line('Function result: '||nFunctionResult);
16c10
< orauser.pack_util.PReportError('OTHER exception');
---
> orauser.pack_util.PReportError('OTHER');


Despite the fact that the original documentation says that the characters are the characters of the end of the line , the bug I started was closed in the redhat bugzilla marked “NOT BAG”, and GNUshniki themselves unsubscribed saying that yes, this is BAG, but the BAG of the documentation . And in the last letter they promised to correct everything in the world.

Of all the tried and tested utilities, xxdiff turned out to be the most convenient (more specifically, its ignore per-hunk whitespaces option ), but unfortunately the utility itself did not know how to work with Unicode files and did not have a built-in editor. in order to "edit" files on the fly, it did not provide the CLI for possible integration with other tools and also had a not-so-nice interface.

But the closest in meaning came the utility command line dwdiff. This little wrapper around diff produces exactly what I need - differences at the level of WORDS. But there was a fly in the ointment - the output of this utility is original and has no standard options for bringing the difference to diff format:

[bugman@localhost 1]$ dwdiff -P -i ora1.sql ora2.sql
declare
nFunctionResult NUMBER(9);
begin
nFunctionResult := orauser.pack_util.FGetNeededValue (in_nSomeParam => somevalue);
dbms_output.put_line('Function result: '||nFunctionResult);
exception
when SOME_EXCEPTION THEN
orauser.pack_util.PReportError('SOME_EXCEPTION');
when OTHERS THEN
orauser.pack_util.PReportError('OTHER [-exception-]');
end;


On the other hand, this is not surprising, because the standard patch has a line level, and in what form to display the difference of two files if they differ in different ways in each line? I was puzzled by this question and wrote to the author of this utility, the GP Halkes commander, the result of the correspondence was a small crutch:

[bugman@localhost 1]$ ./diffwrap.sh ora1.sql ora2.sql diff -i
10c10
< orauser.pack_util.PReportError('OTHER exception');
---
> orauser.pack_util.PReportError('OTHER');


This wrapper allows you to use including visual differs, such as kdiff3. The script itself, a refined version of which will soon become part of dwdiff and will be distributed with it to the delight of lazy coders, but who can’t wait to try it out right now, has no magic:

[bugman@localhost 1]$ cat ~/diffwrap.sh
#!/bin/bash

if [ $# -lt 3 ] ; then
echo "Usage: script.sh "
exit 1
fi

OLD="$1"
NEW="$2"
shift 2

# First create a version of the old file that is formated like
# the new file...
TMP="`mktemp oldconvertXXXXXX`"
dwdiff -P -2 -w '' -x '' "$OLD" "$NEW" > "$TMP"

# ... and then call a diff program to show the changes (per line).
"$@" "$TMP" "$NEW"

# Finally, clean up the temporary file
rm "$TMP"


P. S. Безусловно у описываемой мною проблемы существуют и другие решения. Одно из них - натравить на сравниваемые исходники какой-нить автоформатер. Но мне пришлось отказатся от этого по следующим соображениям: 1) в коллективе выработался некий coding style guide, привести в соответствие с которым любой автоформатер дело не одного даже наверное дня, тратить на это время, чтобы потом разочароваться в выборе пока нет возможности; 2) если откинуть предыдущий пункт, и просто привести копии упоминаемых файлов в какой-нибудь единый формат только для сравнивания, появляется другая задача - как спроецировать найденные различия на оригиналы?