How to compute a 32 bits cyclic redundancy check (CRC-32) as a function in PostgreSQL, the same way as MySQL?
You can create the function yourself, this is a working example for PostgreSQL 9.6
CREATE OR REPLACE FUNCTION crc32(text_string text) RETURNS bigint AS $$
DECLARE
tmp bigint;
i int;
j int;
byte_length int;
binary_string bytea;
BEGIN
IF text_string = '' THEN
RETURN 0;
END IF;
i = 0;
tmp = 4294967295;
byte_length = bit_length(text_string) / 8;
binary_string = decode(replace(text_string, E'\\\\', E'\\\\\\\\'), 'escape');
LOOP
tmp = (tmp # get_byte(binary_string, i))::bigint;
i = i + 1;
j = 0;
LOOP
tmp = ((tmp >> 1) # (3988292384 * (tmp & 1)))::bigint;
j = j + 1;
IF j >= 8 THEN
EXIT;
END IF;
END LOOP;
IF i >= byte_length THEN
EXIT;
END IF;
END LOOP;
RETURN (tmp # 4294967295);
END
$$ IMMUTABLE LANGUAGE plpgsql;
Inspired from an old post with an non crc32 accepted answer. I couldn't find the original code from thinking sphinx.
create it in PG11.
text to crc32(for Little-endian)。
result is equal to java.util.zip.CRC32.
if you want crc32(t text), you can select crc32_update(0, t)
create or replace function crc32_update(crc bigint, t text) returns bigint as $$
declare
bytes bytea;
byte bigint;
len bigint;
long_mask bigint = 4294967295; -- 0xFFFFFFFF
byte_mask integer = 255; -- 0xFF
rt8_mask bigint = 16777215; -- 0xFF000000
crc_table bigint[] = array[
0, 1996959894, -301047508, -1727442502, 124634137, 1886057615, -379345611, -1637575261,
249268274, 2044508324, -522852066, -1747789432, 162941995, 2125561021, -407360249, -1866523247,
498536548, 1789927666, -205950648, -2067906082, 450548861, 1843258603, -187386543, -2083289657,
325883990, 1684777152, -43845254, -1973040660, 335633487, 1661365465, -99664541, -1928851979,
997073096, 1281953886, -715111964, -1570279054, 1006888145, 1258607687, -770865667, -1526024853,
901097722, 1119000684, -608450090, -1396901568, 853044451, 1172266101, -589951537, -1412350631,
651767980, 1373503546, -925412992, -1076862698, 565507253, 1454621731, -809855591, -1195530993,
671266974, 1594198024, -972236366, -1324619484, 795835527, 1483230225, -1050600021, -1234817731,
1994146192, 31158534, -1731059524, -271249366, 1907459465, 112637215, -1614814043, -390540237,
2013776290, 251722036, -1777751922, -519137256, 2137656763, 141376813, -1855689577, -429695999,
1802195444, 476864866, -2056965928, -228458418, 1812370925, 453092731, -2113342271, -183516073,
1706088902, 314042704, -1950435094, -54949764, 1658658271, 366619977, -1932296973, -69972891,
1303535960, 984961486, -1547960204, -725929758, 1256170817, 1037604311, -1529756563, -740887301,
1131014506, 879679996, -1385723834, -631195440, 1141124467, 855842277, -1442165665, -586318647,
1342533948, 654459306, -1106571248, -921952122, 1466479909, 544179635, -1184443383, -832445281,
1591671054, 702138776, -1328506846, -942167884, 1504918807, 783551873, -1212326853, -1061524307,
-306674912, -1698712650, 62317068, 1957810842, -355121351, -1647151185, 81470997, 1943803523,
-480048366, -1805370492, 225274430, 2053790376, -468791541, -1828061283, 167816743, 2097651377,
-267414716, -2029476910, 503444072, 1762050814, -144550051, -2140837941, 426522225, 1852507879,
-19653770, -1982649376, 282753626, 1742555852, -105259153, -1900089351, 397917763, 1622183637,
-690576408, -1580100738, 953729732, 1340076626, -776247311, -1497606297, 1068828381, 1219638859,
-670225446, -1358292148, 906185462, 1090812512, -547295293, -1469587627, 829329135, 1181335161,
-882789492, -1134132454, 628085408, 1382605366, -871598187, -1156888829, 570562233, 1426400815,
-977650754, -1296233688, 733239954, 1555261956, -1026031705, -1244606671, 752459403, 1541320221,
-1687895376, -328994266, 1969922972, 40735498, -1677130071, -351390145, 1913087877, 83908371,
-1782625662, -491226604, 2075208622, 213261112, -1831694693, -438977011, 2094854071, 198958881,
-2032938284, -237706686, 1759359992, 534414190, -2118248755, -155638181, 1873836001, 414664567,
-2012718362, -15766928, 1711684554, 285281116, -1889165569, -127750551, 1634467795, 376229701,
-1609899400, -686959890, 1308918612, 956543938, -1486412191, -799009033, 1231636301, 1047427035,
-1362007478, -640263460, 1088359270, 936918000, -1447252397, -558129467, 1202900863, 817233897,
-1111625188, -893730166, 1404277552, 615818150, -1160759803, -841546093, 1423857449, 601450431,
-1285129682, -1000256840, 1567103746, 711928724, -1274298825, -1022587231, 1510334235, 755167117
];
begin
crc = ~crc;
len = bit_length(t) / 8;
bytes = decode(t, 'escape');
for i in 0..len-1 loop
byte = (get_byte(bytes, i))::bigint;
crc = (crc >> 8 & rt8_mask) # crc_table[((crc # byte) & byte_mask) + 1];
end loop;
crc = ~crc;
return crc & long_mask;
end
$$ immutable language plpgsql
If you want a checksum of the results which does not need to be crc32, you can use md5:
select sum(('x' || lpad(substring(md5(mycolumn), 27, 6), 8, '0'))::bit(32)::int) from mytable
md5 returns hex (128 bits). I use the final 6 characters of the md5 so it does not overflow an int. If the sum overflows an int, it may error. This should give a rudimentary checksum of the results.
SELECT 'xffffffff'::bit(32)::int;
> -1
SELECT 'x7fffffff'::bit(32)::int + 1;
ERROR: integer out of range
SELECT 'x7fffffffffffffff'::bit(64)::bigint + 1;
ERROR: bigint out of range
I have a problem which occurs only in a very small customer range and I would like to ask if you might give me a hint where the problem might be. The program works for 98% of the customers. Alas, it is not possible that I work with the customers to debug the issue, because their knowledge of Windows and computers is very basic. It is also not possible that I send multiple versions of the product to them, since they don't even know how to install software (the admins do all the stuff).
First of all, I translate all RT_STRING resources on-the-fly, so that the language-switching in the program also affects hardcoded stuff like "Yes", "No", "Cancel" etc., which would only be possible by compiling 2 EXE files.
The code (I have tried to left away as much unnecessary stuff as possible, but since I don't know where the problem is, I provided as much details for the bug as possible):
The ony-the-fly resource translation
procedure TranslateResources;
var
i: integer;
s: string;
{$IF NOT Declared(FILE_ATTRIBUTE_NOT_CONTENT_INDEXED)}
const
FILE_ATTRIBUTE_NOT_CONTENT_INDEXED = $2000;
{$IFEND}
begin
// I copy all resources in a dummy DLL (without code), because
// 1) The resources are the only thing we need when changing the resource module
// 2) If the EXE code/debug sections are too long, BeginUpdateResource() will ruin the performance heavily
FTempFile := IncludeTrailingPathDelimiter(GetTempDirectory) + GetRandomString(8)+'.dll';
// Transfers all resources from ParamStr(0) into the dummy DLL at FTempFile
ReGenerateResourceFile(FTempFile);
// if necessary, remove readonly flag
SetFileAttributes(PChar(FTempFile), FILE_ATTRIBUTE_OFFLINE or
FILE_ATTRIBUTE_NOT_CONTENT_INDEXED or
FILE_ATTRIBUTE_TEMPORARY );
for i := 0 to Length(RTLResStringTranslationArray)-1 do
begin
s := Translate(RTLResStringTranslationArray[i].TranslationID);
if s <> '' then
begin
// Translate the string
UpdateResString(RTLResStringTranslationArray[i].ResStrDescriptor.Identifier, s);
end;
end;
LoadNewResourceModule(FTempFile):
end;
procedure ReGenerateResourceFile(OutputFile: string);
var
hUpd: Cardinal;
rs: TResourceStream;
fs: TFileStream;
begin
// As template we use a dummy DLL which contains no code.
// We will implement all resources from ParamStr(0) into it, before we translate the strings.
rs := TResourceStream.Create(HInstance, 'DUMMYDLL', 'DLL');
fs := TFileStream.Create(OutputFile, fmCreate or fmOpenWrite);
try
fs.CopyFrom(rs, rs.Size)
finally
rs.Free;
fs.Free;
end;
// Transfer resources from our EXE into the dummy DLL file
hUpd := BeginUpdateResource(PChar(OutputFile), true);
try
EnumResourceTypes(hInstance, #_enumResTypesProc, hUpd);
finally
EndUpdateResource(hUpd, false)
end;
end;
// This is based on reinit.pas from Borland's RichEdit example; slightly modified
function LoadNewResourceModule(PatchedFile: string): LongInt;
var
NewInst: Longint;
CurModule: PLibModule;
begin
Result := 0;
// Win95: "Initialization routine failed"
// NewInst := LoadLibrary(PChar(PatchedFile));
NewInst := LoadLibraryEx(PChar(PatchedFile), 0, LOAD_LIBRARY_AS_DATAFILE);
CurModule := LibModuleList;
Result := 0;
while CurModule <> nil do
begin
if CurModule.Instance = HInstance then
begin
if CurModule.ResInstance <> CurModule.Instance then
FreeLibrary(CurModule.ResInstance);
// Win95: ERangeError
CurModule^.ResInstance := NewInst;
Result := NewInst;
Exit;
end;
CurModule := CurModule.Next;
end;
end;
// Based on http://stackoverflow.com/questions/1498658/modifying-a-string-in-resource-of-an-exe
// Modified
procedure UpdateResString(const AStringIdent: Integer; const ANewString: WideString);
var
ResData, TempData: TWordArray;
iSection, iIndexInSection: Integer;
i, iLen, iSkip, iPos: Integer;
begin
// Calculate the resource string area and the string index in that area
iSection := AStringIdent div 16 + 1;
iIndexInSection := AStringIdent mod 16;
ResData := ReadSectionCached(iSection);
// Calculate the position of the string
iLen := Length(ANewString);
iPos := 0;
for i := 0 to iIndexInSection do
begin
if iPos > High(ResData) then
begin
SetLength(ResData, iPos + 1);
ResData[iPos] := 0;
end;
if i <> iIndexInSection then
begin
iSkip := ResData[iPos] + 1;
Inc(iPos, iSkip);
end;
end;
// Put data behind strings into TempData
iSkip := 1{size} + ResData[iPos];
SetLength(TempData, Length(ResData) - (iPos + iSkip));
if Length(TempData) > 0 then
begin
CopyMemory(#TempData[0], #ResData[iPos + iSkip], Length(TempData)*SizeOf(TempData[0]));
end;
SetLength(ResData, iPos + (iLen + 1{size}) + Length(TempData));
// Overwrite string
ResData[iPos] := iLen;
Inc(iPos);
if iLen > 0 then
begin
CopyMemory(#ResData[iPos], #ANewString[1], iLen*SizeOf(ANewString[1]));
Inc(iPos, iLen);
end;
// Append TempData after our new string
if Length(TempData) > 0 then
begin
CopyMemory(#ResData[iPos], #TempData[0], Length(TempData)*SizeOf(TempData[0]));
end;
CacheSet(iSection, ResData);
end;
type
TGlobalData = record
GlobalPtr: Pointer;
Length: integer;
end;
function LoadResourcePtr(hModule: HMODULE; restype, resname: PChar; wIDLanguage: WORD): TGlobalData;
var
hFind, hRes: THandle;
begin
result.GlobalPtr := nil;
result.Length := -1;
hFind := Windows.FindResourceEx(hModule, restype, resname, wIDLanguage);
if hFind = 0 then RaiseLastOSError;
hres := Windows.LoadResource(hModule, hFind);
if hres = 0 then RaiseLastOSError;
result.GlobalPtr := Windows.LockResource(hres);
result.Length := Windows.SizeofResource(hModule, hFind);
end;
function _enumResLangsProc(hmodule: HMODULE; restype, resname: PChar; wIDLanguage: WORD;
lParam: LongInt): BOOL; stdcall;
var
rs: TGlobalData;
begin
rs := LoadResourcePtr(hmodule, restype, resname, wIDLanguage);
UpdateResource(lParam, restype, resname, wIDLanguage, rs.GlobalPtr, rs.Length);
result := true;
end;
function _enumResNamesProc(hmodule: HMODULE; restype, resname: PChar;
lParam: LongInt): BOOL; stdcall;
begin
EnumResourceLanguages(hmodule, restype, resname, #_enumResLangsProc, lParam);
result := true;
end;
function _enumResTypesProc(hmodule: HMODULE; restype: PChar;
lParam: LongInt): BOOL; stdcall;
begin
EnumResourceNames(hmodule, restype, #_enumResNamesProc, lParam);
result := true;
end;
{$R '..\dummydll\dummydll.RES'}
Then I use a wait form:
unit Wait;
interface
uses
...
type
TWaitForm = class(TForm)
...
end;
var
WaitForm: TWaitForm;
implementation
{$R *.dfm}
...
end;
The wait form will be called by dynamically showing the form:
procedure ShowWaitForm;
begin
...
{ I use my own _CreateForm function because it solves many workarounds for
juicy stuff like half-modal windows (which can be hidden without user action),
miscellaneous deadlocks etc. and to allow the form to be shown in a shared PAS file
without the requirement to add it to every DPR file where the WaitForm API is used. }
WaitForm := _CreateForm(TWaitForm, {Application.MainForm}AParent) as TWaitForm;
WaitForm.Show;
...
end;
function _CreateForm(InstanceClass: TCustomFormClass; AParent: TCustomForm): TCustomForm;
var
LOwner: TComponent;
begin
if Assigned(AParent) then
begin
LOwner := AParent;
end
else if Assigned(Application) then
begin
LOwner := Application;
end
else
begin
LOwner := nil;
end;
result := InstanceClass.Create(LOwner);
end;
The error message at 2% of the customers:
Resource TWaitForm was not found
However, other forms are working.
There are 2 theories I can think of:
1) Did the resource translation corrupt the DLL file / part of the RCData section? (Maybe a bug in the WinAPI's UpdateResource ?)
2) Is there a problem with the dynamic showing of the wait form (since other "static" forms are shown?)
I am just beginning to learn VHDL in modelsim, so i apologize in advance if what I'm doing seems really noob.
Basically what i am trying to create is a synthesizable VHDL code for a one-digit up/down BCD counter. The counter will count when "Enable" is '1', or else it stays the same. When input "Init" is initialized, the counter is set to 0 or 9 depending on the value of "Direction" input. (When "Direction" is '1', it is an up counter).
I'm just wondering if there are better tools to use for this to work other than using 100 if and else in a row.
Here is my code, I am writing a test bench for it right now, so I am not yet sure if this will even work. So if you happen to also spot some mistake please point it out for me.
Thanks a lot in advance, and here is my code
entity BCD_counter is
port(clk, direction, init, enable: in bit;
q_out: out integer);
end entity BCD_counter;
architecture behaviour of BCD_counter is
signal q: integer;
begin
process(clk)
begin
if(Clk'event and Clk = '1') then
if(direction = '1') then -- counting up
if(init = '1')then --initialize
q<=0; -- reset to 0
else
if(enable = '1')then -- counting
if (q<9) then
q<=q+1;
else
q<=0;
end if;
else
q<=q;
end if;
end if;
elsif(direction = '0') then --counting down
if(init = '1') then --initialize
q<=9; --reset to 9
else
if(enable = '1') then --counting
if (q>0) then
q<=q-1;
else
q<=9;
end if;
else
q<=q;
end if;
end if;
end if;
end if;
end process;
q_out <= q;
end architecture behaviour;
Slightly different style, but this is how I would write it
architecture behaviour of BCD_counter is
signal next_q : integer;
signal q : integer;
begin
pReg : process
begin -- process pReg
wait until clk'event and clk = '1';
if init = '1' then
q <= 0;
else
q <= next_q;
end if;
end process pReg;
pCount : process (direction, enable, q)
begin -- process pCount
next_q <= q;
if enable = '1' then
if direction = '1' then
next_q <= q + 1;
if q = 9 then
next_q <= 0;
end if;
else
next_q <= q - 1;
if q = 0 then
next_q <= 9;
end if;
end if;
end if;
end process pCount;
q_out <= q;
end architecture behaviour;
Points to note:
The register behaviour is in a separate process from the logic. I find this style is clean. Others have different opinions which normally come down to not liking having next_blah signals.
init is your reset signal, so it overrides everything else, and I put reset signals in the register process.
The first line of the counter process is what I want to happen by default. In this case it says have the next state of q be the current state.
I have the outer if be the check of enable. If we're not enabled we don't do anything, so why check direction first?
Inside each half of the direction condition the code structure is the same. Set-up what I want the normal case to be, and then check for the exception to the rule. For example: if we're going up, I set the next state to be q+1, but if q is 9 I override it with q <= 0.
I'm not using >, < in any comparisons. These are more expensive than =, and = is just fine.
Alternatively you could take the enable into the register process. It's slightly shorter, and possibly clearer. It also makes it obvious that you're expecting to use the enable pin of a register for this function.
pReg : process
begin -- process pReg
wait until clk'event and clk = '1';
if init = '1' then
q <= 0;
else
if enable = '1' then
q <= next_q;
end if;
end if;
end process pReg;
pCount : process (direction, q)
begin -- process pCount
if direction = '1' then
next_q <= q + 1;
if q = 9 then
next_q <= 0;
end if;
else
next_q <= q - 1;
if q = 0 then
next_q <= 9;
end if;
end if;
end process pCount;
The only thing that comes to my mind is that you could ommit the two
else
q<=q;
statements because this is implicitly done if none of the other conditions check out.
I do a lot of this - I created a function called modulo_increment which takes an input integer, and "modulus" integer and returns the wrapped around value.
so you can then do
q <= modulo_increment(q, 10);